diff mbox

Cilk Keywords (_Cilk_spawn and _Cilk_sync) for C

Message ID BF230D13CA30DD48930C31D4099330003A458F86@FMSMSX101.amr.corp.intel.com
State New
Headers show

Commit Message

Iyer, Balaji V Aug. 13, 2013, 8:31 p.m. UTC
Hi Aldy et al.,
	Attached, please find a fixed patch and the change-log entries below. Please see my answers to your questions below. The patch is tested on x86_64. So, is this Ok for trunk?

gcc/ChangeLog:
2013-08-13  Balaji V. Iyer  <balaji.v.iyer@intel.com>

	* builtins.c (is_builtin_name): Added a check for __cilkrts_detach and
	__cilkrts_pop_frame.  If matched, then return true for built-in function
	name.
	(expand_builtin): Added BUILT_IN_CILK_DETACH and BUILT_IN_CILK_POP_FRAME
	case.
	* langhooks-def.h (lhd_install_body_with_frame_cleanup): New prototype.
	(lhs_cilk_valid_spawn): Likewise.
	(LANG_HOOKS_DECLS): Added LANG_HOOKS_CILKPLUS.
	(LANG_HOOKS_CILKPLUS_SPAWNABLE_CTOR): New #define.
	(LANG_HOOKS_CILKPLUS_RECOGNIZE_SPAWN): Likewise.
	(LANG_HOOKS_CILKPLUS_VALID_SPAWN): Likewise.
	(LANG_HOOKS_CILKPLUS_FRAME_CLEANUP): Likewise.
	(LANG_HOOKS_CILKPLUS_GIMPLIFY_SPAWN): Likewise.
	(LANG_HOOKS_CILKPLUS_GIMPLIFY_SYNC): Likewise.
	(LANG_HOOKS_CILKPLUS): Likewise.
	* builtin.def (DEF_CILK_BUILTIN_STUB): Likewise.
	* Makefile.in (C_COMMON_OBJS): Added c-family/cilk.o.
	(OBJS): Added cilk-common.o.
	* langhooks.c (lhd_install_body_with_frame_cleanup): New function.
	(lhd_cilk_valid_spawn): Likewise.
	* langhooks.h (lang_hooks_for_cilkplus): New struct.
	(struct lang_hooks): Added new field called "cilkplus."
	* cilk-common.c: New file.
	* cilk.h: Likewise.
	* cilk-builtins.def: Likewise.
	* cppbuiltin.c (define_builtin_macros_for_compilation_flags): Added
	"__cilk" macro and set it to 200.
	* function.h (struct function::cilk_frame_decl): New field.
	(struct function::is_cilk_function): Likewise.
	(struct function::calls_cilk_spawn): Likewise.
	* gimplify.c (gimplify_call_expr): Added a check if the function call
	being gimplified is a spawn detach point.  If so, then add pop_frame
	and detach function calls.
	(gimplify_expr): If a function call is a valid spawned function, then
	gimplify it using gimplify_cilk_spawn function call.  Also, added
	a CILK_SYNC_STMT case for gimplifying _Cilk_sync statement.
	* ipa-inline-analysis (initialize_inline_failed): Prevent inlining of
	spawner function.
	(can_inline_edge_p): Prevent inling of spawnee function.
	* ira.c (ira_setup_eliminable_regset): Force usage of frame pointer for
	functions that use Cilk keywords.
	* tree-inline.h (struct copy_body_data::remap_var_for_cilk): New field.
	* tree-pretty-print.c (dump_generic_node): Added CILK_SPAWN_STMT and
	CILK_SYNC_STMT cases.
	* tree.def (DEFTREECODE): Added CILK_SPAWN_STMT and CILK_SYNC_STMT
	trees.
	* tree.h (CILK_SPAWN_FN): New #define.
	* generic.texi (CILK_SPAWN_STMT): Added documentation for _Cilk_spawn.
	(CILK_SYNC_STMT): Added documentation for _Cilk_sync.
	* passes.texi (Cilk Keywords): New section that describes the compiler
	code changes for handling Cilk Keywords.

gcc/c/ChangeLog
2013-08-13  Balaji V. Iyer  <balaji.v.iyer@intel.com>

	* c-decl.c (finish_function): Added a call for insert_cilk_frame when
	a spawning function is found.
	* c-objc-common.h (LANG_HOOKS_CILKPLUS_GIMPLIFY_SPAWN): New #define.
	(LANG_HOOKS_CILKPLUS_GIMPLIFY_SYNC): Likewise.
	(LANG_HOOKS_CILKPLUS_FRAME_CLEANUP): Likewise.
	(LANG_HOOKS_CILKPLUS_VALID_SPAWN): Likewise.
	* c-parser.c (c_parser_statement_after_labels): Added RID_CILK_SYNC
	case.
	(c_parser_postfix_expression): Added RID_CILK_SPAWN case.
	* c-tree.h (c_build_cilk_sync): New prototype.
	(c_build_cilk_spawn): Likewise.
	* c-typeck.c (build_compound_expr): Reject _Cilk_spawn in a comma expr.
	(c_finish_return): Added a check to reject _Cilk_spawn in return expression.
	(c_build_cilk_spawn): New function.
	(c_build_cilk_sync): Likewise.

gcc/c-family/ChangeLog
2013-08-13  Balaji V. Iyer  <balaji.v.iyer@intel.com>

	* c-common.c (c_common_reswords[]): Added _Cilk_spawn and _Cilk_sync
	fields.
	(c_define_builtins): Called cilk_init_builtins if Cilk Plus is enabled.
	(c_common_init_ts): Marked CILK_SPAWN_STMT and CILK_SYNC_STMT as typed.
	* c-common.h (enum rid): Added RID_CILK_SPAWN and RID_CILK_SYNC.
	(insert_cilk_frame): New prototype.
	(cilk_init_builtins): Likewise.
	(gimplify_cilk_spawn): Likewise.
	(gimplify_cilk_sync): Likewise.
	(c_cilk_install_body_w_frame_cleanup): Likewise.
	(cilk_valid_spawn): Likewise.
	(cilk_set_spawn_marker): Likewise.
	* cilk.c: New file.

gcc/lto/ChangeLog.cilk
2013-08-13  Balaji V. Iyer  <balaji.v.iyer@intel.com>

	* Make-lang.in (lto/lto-lang.o): Added cilk.h in dependency list.
	* lto-lang.c (lto_init): Added a call to cilk_init_builtins if Cilk Plus
	is enabled.

gcc/testsuite/ChangeLog.cilk
2013-08-13  Balaji V. Iyer  <balaji.v.iyer@intel.com>

	* c-c++-common/cilk-plus/CK/compound_cilk_spawn.c: New test.
	* c-c++-common/cilk-plus/CK/concec_cilk_spawn.c: Likewise.
	* c-c++-common/cilk-plus/CK/fib.c: Likewise.
	* c-c++-common/cilk-plus/CK/no_args_error.c: Likewise.
	* c-c++-common/cilk-plus/CK/spawnee_inline.c: Likewise.
	* c-c++-common/cilk-plus/CK/spawner_inline.c: Likewise.
	* c-c++-common/cilk-plus/CK/spawning_arg.c: Likewise.
	* c-c++-common/cilk-plus/CK/steal_check.c: Likewise.
	* c-c++-common/cilk-plus/CK/test__cilk.c: Likewise.
	* c-c++-common/cilk-plus/CK/varargs_test.c: Likewise.
	* c-c++-common/cilk-plus/CK/sync_wo_spawn.c: Likewise.
	* c-c++-common/cilk-plus/CK/invalid_spawn.c: Likewise.
	* c-c++-common/cilk-plus/CK/spawn_in_return.c: Likewise.
	* gcc.dg/cilk-plus/cilk-plus.exp: Added support to run Cilk Keywords
	test stored in c-c++-common.  Also, added the Cilk runtime's library to
	the ld_library_path.


> -----Original Message-----
> From: gcc-patches-owner@gcc.gnu.org [mailto:gcc-patches-
> owner@gcc.gnu.org] On Behalf Of Aldy Hernandez
> Sent: Friday, August 09, 2013 6:40 AM
> To: Iyer, Balaji V
> Cc: rth@redhat.com; Jeff Law; gcc-patches@gcc.gnu.org
> Subject: Re: [PATCH] Cilk Keywords (_Cilk_spawn and _Cilk_sync) for C
> 
> 
> 
> >>> --- gcc/expr.c
> >>> +++ gcc/expr.c
> >>> @@ -9569,6 +9569,21 @@ expand_expr_real_1 (tree exp, rtx target,
> >>> enum
> >> machine_mode tmode,
> >>>   	}
> >>>
> >>>         return expand_constructor (exp, target, modifier, false);
> >>> +    case INDIRECT_REF:
> >>> +      {
> >>> +	tree exp1 = TREE_OPERAND (exp, 0);
> >>> +	if (modifier != EXPAND_WRITE)
> >>> +	  {
> >>> +	    tree t = fold_read_from_constant_string (exp);
> >>> +	    if (t)
> >>> +	      return expand_expr (t, target, tmode, modifier);
> >>> +	  }
> >>> +	op0 = expand_expr (exp1, NULL_RTX, VOIDmode, EXPAND_SUM);
> >>> +	op0 = memory_address (mode, op0);
> >>> +	temp = gen_rtx_MEM (mode, op0);
> >>> +	set_mem_attributes (temp, exp, 0);
> >>> +	return temp;
> >>> +      }
> >>
> >> Ughhh, what's the rationale for this?  Are generic changes to
> >> expand_expr really needed?
> >
> > Yes, I am expanding some variables of type "ptr->value" and those are emitted
> as INDIRECT_REF.
> 
> The fact that you are getting an INDIRECT_REF this late in the game is suspect.
> 

I have removed handling of indirect_ref case in expr.c

> Are you building with ENABLE_CHECKING, because it seems this should have
> been caught.  See the places in tree-cfg.c with this:
> 
>      case INDIRECT_REF:
>        error ("INDIRECT_REF in gimple IL");
>        return t;
> >
> >>
> >>> +  /* During LTO, the is_cilk_function flag gets cleared.
> >>> +     If __cilkrts_pop_frame is called, then this definitely must be a
> >>> +     cilk function.  */
> >>> +  if (cfun)
> >>> +    cfun->is_cilk_function = 1;
> >>
> >> I don't know much about our LTO implementation, but perhaps you need
> >> to teach LTO to stream this bit in/out?  And of course, an
> >> accompanying LTO test to check for this problem you're encountering would
> be appropriate.
> >>
> >
> > I also have a limited knowledge of LTO. This seem to be the most
> straightforward way of doing it (atleast for me).
> 
> See how other bits in `struct function' are streamed in/out in LTO, for example in
> output_struct_function_base()
> 
>    bp_pack_value (&bp, fn->calls_alloca, 1);
>    bp_pack_value (&bp, fn->calls_setjmp, 1);
>    bp_pack_value (&bp, fn->va_list_fpr_size, 8);
>    bp_pack_value (&bp, fn->va_list_gpr_size, 8);
> 
> and the corresponding in input_struct_function_base():
> 
>    fn->calls_alloca = bp_unpack_value (&bp, 1);
>    fn->calls_setjmp = bp_unpack_value (&bp, 1);
>    fn->va_list_fpr_size = bp_unpack_value (&bp, 8);
>    fn->va_list_gpr_size = bp_unpack_value (&bp, 8);
> 
> Also, you will need a testcase to make sure later changes to the compiler do not
> break LTO wrt Cilk features you have added.
> 

I have added lto tests

> >
> >>> +       /* We need frame pointer for all Cilk Plus functions that uses
> >>> +	  Cilk Keywords.  */
> >>> +       || (flag_enable_cilkplus && cfun->is_cilk_function)
> >>
> >> "need a frame pointer"
> >>
> >> "that use"
> >>
> >> s/Keywords/keywords/
> >>
> >
> > It should be keywords, because you need frame-pointer for "_Cilk_spawn and
> _Cilk_sync" and "_Cilk_for"
> 
> I meant that you should lowercase the "K".
> 

This is fixed also

> >
> >>> +  /* This variable will tell whether we are on a spawn helper or
> >>> + not */  unsigned int is_cilk_helper_function : 1;
> >>
> >> Where is this used?
> >>
> >
> > Well, it is not used now but later on when I add Tools support it will be. I will
> remove it for now.
> 
> Yes, please.
> 

Yep, this is removed also.

> >>> --- gcc/ipa-inline-analysis.c
> >>> +++ gcc/ipa-inline-analysis.c
> >>> @@ -1433,6 +1433,9 @@ initialize_inline_failed (struct cgraph_edge *e)
> >>>       e->inline_failed = CIF_REDEFINED_EXTERN_INLINE;
> >>>     else if (e->call_stmt_cannot_inline_p)
> >>>       e->inline_failed = CIF_MISMATCHED_ARGUMENTS;
> >>> +  else if (flag_enable_cilkplus && cfun && cfun->calls_spawn)
> >>> +    /* We can't inline if the function is spawing a function.  */
> >>> +    e->inline_failed = CIF_BODY_NOT_AVAILABLE;
> >>
> >> Hmmm, if we don't have a cfun, perhaps we should be sticking this
> >> calls_spawn bit in the cgraph node.
> >>
> >> Richard?  Anyone?
> >>
> >
> > When I am first setting this, I don't think cgraph is available.
> 
> See Richard's comment with regards to struct function and its availability via the
> callee edge.  Also see his comment regarding the inappropriate error message.
> 

Yes, I changed it to FUNCTION_NOT_INLINABLE.

> >>> @@ -3496,7 +3510,7 @@ struct GTY(()) tree_function_decl {
> >>>        ???  The bitfield needs to be able to hold all target function
> >>>   	  codes as well.  */
> >>>     ENUM_BITFIELD(built_in_function) function_code : 11;
> >>> -  ENUM_BITFIELD(built_in_class) built_in_class : 2;
> >>> +  ENUM_BITFIELD(built_in_class) built_in_class  ;
> >>
> >> What's this for?
> >>
> >
> > Added a new enum field called BUILT_IN_CILK so we need 3 bits instead of 2,
> since there are 5 fields instead of 4.
> 
> Hmm, yeah.  I see you added another field here:
> 

Well, as requested by Joseph, I have removed this change.

> > diff --git a/gcc/tree.h b/gcc/tree.h
> > index 0058a4b..952362f 100644
> > --- a/gcc/tree.h
> > +++ b/gcc/tree.h
> > @@ -262,6 +262,7 @@ enum built_in_class
> >    NOT_BUILT_IN = 0,
> >    BUILT_IN_FRONTEND,
> >    BUILT_IN_MD,
> > +  BUILT_IN_CILK,
> >    BUILT_IN_NORMAL
> >  };
> 
> If you look at the comment above enum built_in_class, you will see that these
> classes specify which part of the compiler created the built-in (the frontend, the
> backend (MD), or a normal builtin).  I don't see how Cilk should be treated
> specially.  And even so, I don't see how you use this BUILT_IN_CILK class
> anywhere.
> 

This field is also removed as requested by Joseph.

> >>> +  if (DECL_BUILT_IN_CLASS (exp) == BUILT_IN_NORMAL)
> >>> +    return DECL_FUNCTION_CODE (exp) == BUILT_IN_MEMCPY;  return
> >>> + lang_hooks.cilkplus.spawnable_constructor (exp);  return false;
> >>
> >
> > That would be necessary for C++, but it returns false for C. So,
> > should I take out this hook for now? Would prefer to keep it in
> 
> You can keep the hook, but do put a comment specifying that it's a place holder
> for C++.  And also, please remove the second return.
> 
> >> Are these two hooks ever set to anything but hook_bool_tree_false?
> >> If so, why the need for them?
> >>
> >
> > Used in C++ but not in C.
> 
> Leave them.  Similar comment please.
> 

Added the comment in langhooks.h under its description.

> >>> +     struct __cilkrts_worker {
> >>> +       __cilkrts_stack_frame *volatile *volatile tail;
> >>> +       __cilkrts_stack_frame *volatile *volatile head;
> >>> +       __cilkrts_stack_frame *volatile *volatile exc;
> >>> +       __cilkrts_stack_frame *volatile *volatile protected_tail;
> >>> +       __cilkrts_stack_frame *volatile *ltq_limit;
> >>
> >> What's this "*volatile *volatile" stuff?
> >>
> >
> > That is how the field is described in Cilk Runtime source (please see:
> > http://gcc.gnu.org/svn/gcc/branches/cilkplus/libcilkrts/include/intern
> > al/abi.h)
> 
> Ok, I'm not a language expert, I presume this is specifying volatile on both
> indirections and is proper form.
> 
> >>> +/* Gimplifies the cilk_sync expression passed in *EXPR_P.  Returns
> >> GS_ALL_DONE
> >>> +   when finished.  */
> >>> +
> >>> +int
> >>> +gimplify_cilk_sync (tree *expr_p, gimple_seq *pre_p, gimple_seq *post_p
> >>> +		    ATTRIBUTE_UNUSED)
> >>> +{
> >>> +  tree sync_expr = build_cilk_sync ();
> >>> +  *expr_p = NULL_TREE;
> >>> +  gimplify_and_add (sync_expr, pre_p);
> >>> +  return GS_ALL_DONE;
> >>
> >> I'm not a big fan of functions that always return the same thing.
> >> The caller should set GS_ALL_DONE accordingly.
> >>
> >
> > The reason why I did this is that, there is a generic empty hook for this in
> langhooks.c that returns int. So, to reduce the number of code addition, I just
> made it return int. Also in future, if i want to add more things, having a return
> value I thought would be helpful.
> 
> Use void.  Set GS_ALL_DONE in the caller.  You can add the return value when
> you use it in the future.
> 

Thanks,

Balaji V. Iyer.


> Aldy

Comments

Aldy Hernandez Aug. 19, 2013, 10:18 p.m. UTC | #1
> @@ -960,6 +960,7 @@ SCEV_H = tree-scalar-evolution.h $(GGC_H) tree-chrec.h $(PARAMS_H)
>  OMEGA_H = omega.h $(PARAMS_H)
>  TREE_DATA_REF_H = tree-data-ref.h $(OMEGA_H) graphds.h $(SCEV_H)
>  TREE_INLINE_H = tree-inline.h
> +CILK_H = cilk.h
>  REAL_H = real.h $(MACHMODE_H)
>  IRA_INT_H = ira.h ira-int.h $(CFGLOOP_H) alloc-pool.h

Doesn't cilk.h depend on tree.h?  So shouldn't that be:

CILK_H = cilk.h $(TREE_H)

Which would mean that whoever includes cilk.h doesn't need to include 
tree.h.  Although... what I would prefer is not to include tree.h from 
cilk.h at all, and have the caller/includer of cilk.h include tree.h first.

> +c-family/cilk.o : c-family/cilk.c $(TREE_H) $(SYSTEM_H) $(CONFIG_H) toplev.h \
> +	$(TREE_H) coretypes.h tree-iterator.h $(TREE_INLINE_H) $(CGRAPH_H) \
> +	$(DIAGNOSTIC_CORE_H) $(GIMPLE_H) $(CILK_H) $(C_COMMON_H) langhooks.h

TREE_H is duplicated.

Also, you are using DIAGNOSTIC_CORE_H whereas c-family/cilk.c is using

> +#include "diagnostic.h"

> +/* Cilk Keywords builtins.  */
> +#include "cilk-builtins.def"

keywords should be lowercase

> diff --git a/gcc/c-family/c-common.h b/gcc/c-family/c-common.h
> index dc430c3..ab77fa4 100644
> --- a/gcc/c-family/c-common.h
> +++ b/gcc/c-family/c-common.h
> @@ -148,6 +148,9 @@ enum rid
>    /* C++11 */
>    RID_CONSTEXPR, RID_DECLTYPE, RID_NOEXCEPT, RID_NULLPTR, RID_STATIC_ASSERT,
>
> +  /* Cilk Plus Keywords.  */
> +  RID_CILK_SPAWN, RID_CILK_SYNC,
> +

Same here.

> +/* In cilk.c.  */
> +extern tree insert_cilk_frame (tree);
> +extern void cilk_init_builtins (void);
> +extern int gimplify_cilk_spawn (tree *, gimple_seq *, gimple_seq *);
> +extern int gimplify_cilk_sync (tree *, gimple_seq *, gimple_seq *);
> +extern void c_cilk_install_body_w_frame_cleanup (tree, tree);
> +extern bool cilk_valid_spawn (tree *);
> +extern bool cilkplus_set_spawn_marker (location_t, tree);

You should be consistent with the prefix throughout the entire patch. 
For instance, now you have cilk_init_builtins() but 
cilkplus_set_spawn_marker().  Do whatever you like, but be consistent.

Also, insert_cilk_frame-- the prefix should be well...as a prefix should 
be...at the beginning.  However, gimplify_cilk_spawn/sync is ok since 
that is in keeping with the rest of the gimplify.c style.

> +  /* IF the function has _Cilk_spawn in front of a function call inside it
> +     i.e. it is a spawning function, then add the appropriate Cilk plus
> +     functions inside.  */

Lowercase the F in IF.

> +	  c_parser_skip_until_found (parser, CPP_SEMICOLON, "expected %<;%>");
> +	  if (!flag_enable_cilkplus)
> +	    error_at (loc, "-fcilkplus must be enabled to use _Cilk_sync");

The error message should use %<_Cilk_sync>

> +	case RID_CILK_SPAWN:
> +	  c_parser_consume_token (parser);
> +	  if (!flag_enable_cilkplus)
> +	    {
> +	      error_at (loc, "-fcilkplus must be enabled to use _Cilk_spawn");
> +	      expr = c_parser_postfix_expression (parser);
> +	      expr.value = error_mark_node;	

Similarly with _Cilk_spawn.

> +	  if (c_parser_peek_token (parser)->keyword == RID_CILK_SPAWN)
> +	    {
> +	      error_at (input_location, "consecutive _Cilk_spawn keywords "
> +			"are not permitted");

Similarly here-- for that matter, check the rest of the patch for any 
keywords in error messages, they should have %<blah>.

Also, avoid input_location when possible.  Surely, you can infer the 
actual location from the parser.

> +  if (flag_enable_cilkplus && retval && TREE_CODE (retval) == CILK_SPAWN_STMT)
> +    {
> +      error_at (loc, "use of _Cilk_spawn in return statement is not allowed");
> +      return error_mark_node;

s/in return/in a return/.  Also, use %<_Cilk_spawn>.

> +/* Returns a tree of type CILK_SYNC_STMT if Cilk Plus is enabled.  Otherwise
> +   return error_mark_node.  */
> +
> +tree
> +c_build_cilk_sync (void)
> +{
> +  tree sync = build0 (CILK_SYNC_STMT, void_type_node);
> +  TREE_SIDE_EFFECTS (sync) = 1;
> +  return sync;
> +}

Code seems correct and comment wrong.  Adjust accordingly.

> diff --git a/gcc/cilk.h b/gcc/cilk.h
> new file mode 100644
> index 0000000..038316a
> --- /dev/null
> +++ b/gcc/cilk.h
> @@ -0,0 +1,94 @@
> +/* This file is part of the Intel(R) Cilk(TM) Plus support
> +   This file contains Cilk Support files.
> +   Copyright (C) 2013  Free Software Foundation, Inc.

Only one space after 2013.  Similarly in other files.

> +
> +#ifndef GCC_CILK_H
> +#define GCC_CILK_H
> +
> +#include "tree.h"

As mentioned earlier, cilk.h should not include tree.h, the caller should.

> +/* Frame status bits known to compiler.  */
> +#define CILK_FRAME_STOLEN    0x01

Unused?

> +this keyword with a set of functions that are stored in the Cilk Runtime.

Lowercase "Runtime".

> +@file{c-family/cilk.c}.  In the spawn-helper, the gimplification function
> +@code{gimplify_call_expr}, inserts a function call @code{__cilkrts_detach}.
> +This function is expanded by @code{builtin_expand_cilk_detach} located in
> +@file{c-family/cilk.c}.
> +
> +@item @code{_Cilk_sync}:
> +@code{_Cilk_sync} is parsed like a keyword.  During gimplification,
> +the function @code{gimplify_cilk_sync} in @file{c-family/cilk.c}, will replace
> +this keyword with a set of functions that are stored in the Cilk Runtime.
> +One of the internal functions inserted during gimplification,
> +@code{__cilkrts_pop_frame} must be expanded by the compiler and is
> +done by @code{builtin_expand_cilk_pop_frame} in @file{cilk-common.c}.

Here and elsewhere in the documentation... It's up to you, but I would 
prefer to remove reference to the actual file the functions are defined 
in, because functions could be rearranged later.

> +	case CILK_SYNC_STMT:
> +	  {
> +	    if (flag_enable_cilkplus)
> +	      {
> +		if (!cfun->cilk_frame_decl)
> +		  {
> +		    error_at (input_location, "expected _Cilk_spawn before "
> +			      "_Cilk_sync");
> +		    ret = GS_ERROR;
> +		  }
> +		else
> +		  ret = (enum gimplify_status)
> +		    lang_hooks.cilkplus.gimplify_cilk_sync (expr_p, pre_p,
> +							    post_p);
> +		break;
> +	      }
> +	    else
> +	      /* _Cilk_sync without Cilk Plus enabling should be caught by
> +		 the parser.  */
> +	      gcc_unreachable ();

The check for !flag_enable_cilkplus is not necessary, as well as the 
gcc_unreachable.  You shouldn't be checking for flag_enable_cilkplus in 
the gimplifier; an earlier pass (parser?) should've caught this.  See 
how TRANSACTION_EXPR is handled.

Also, I would use the location of the CILK_SYNC_STMT instead of 
input_location.

> diff --git a/gcc/ipa-inline-analysis.c b/gcc/ipa-inline-analysis.c
> index 9a36292..7c63cc8 100644
> --- a/gcc/ipa-inline-analysis.c
> +++ b/gcc/ipa-inline-analysis.c
> @@ -1433,6 +1433,9 @@ initialize_inline_failed (struct cgraph_edge *e)
>      e->inline_failed = CIF_REDEFINED_EXTERN_INLINE;
>    else if (e->call_stmt_cannot_inline_p)
>      e->inline_failed = CIF_MISMATCHED_ARGUMENTS;
> +  else if (flag_enable_cilkplus && cfun && cfun->calls_cilk_spawn)
> +    /* We can't inline if the function is spawing a function.  */
> +    e->inline_failed = CIF_FUNCTION_NOT_INLINABLE;

Didn't rth mention that instead of looking through cfun (which may or 
may not be there), you could check the callee/caller edge?

> +void
> +lhd_install_body_with_frame_cleanup (tree, tree)
> +{
> +  return;
> +}

As mentioned before, avoid empty returns.

> +  /* Returns true if the call expr passed is a spawned function call.  */

Here and elsewhere, you probably want s/call expr/CALL_EXPR/g

> diff --git a/gcc/tree-inline.h b/gcc/tree-inline.h
> index b65dee9..32bedac 100644
> --- a/gcc/tree-inline.h
> +++ b/gcc/tree-inline.h
> @@ -123,6 +123,10 @@ typedef struct copy_body_data
>       the originals have been mapped to a value rather than to a
>       variable.  */
>    struct pointer_map_t *debug_map;
> +
> +  /* Cilk keywords currently need to replace some variables that
> +     ordinary nested functions do not.  */
> +  bool remap_var_for_cilk;
>  } copy_body_data;
>

Where is this field used?

> diff --git a/gcc/tree-pretty-print.c b/gcc/tree-pretty-print.c
> index 7745f73..f449c68 100644
> --- a/gcc/tree-pretty-print.c
> +++ b/gcc/tree-pretty-print.c
> @@ -2403,6 +2403,14 @@ dump_generic_node (pretty_printer *buffer, tree node, int spc, int flags,
>        dump_block_node (buffer, node, spc, flags);
>        break;
>
> +    case CILK_SPAWN_STMT:
> +      pp_string (buffer, "_Cilk_spawn ");
> +      dump_generic_node (buffer, TREE_OPERAND (node, 0), spc, flags, false);
> +      break;
> +
> +    case CILK_SYNC_STMT:
> +      pp_string (buffer, "_Cilk_sync;");
> +      break;
>      default:
>        NIY;


In keeping with the present code in this switch statement, empty line 
before the default.

> +/* Cilk spawn expression
> +   Operand 0 is the CALL_EXPR.  */
> +DEFTREECODE (CILK_SPAWN_STMT, "cilk_spawn_stmt", tcc_statement, 1)
> +
> +/* Cilk Sync Statement: Does not have any operands.  */
> +DEFTREECODE (CILK_SYNC_STMT, "cilk_sync_stmt", tcc_expression, 0)

Be consistent.  "Cilk spawn" but then "Cilk Sync"?  Also "expression" in 
one, but "statement" in the next?  Also, Statement should be lowercase, 
as well as Sync.

> +/* Cilk Keywords accessors.  */
> +#define CILK_SPAWN_FN(NODE) TREE_OPERAND (CILK_SPAWN_STMT_CHECK (NODE), 0)
> +

Lowercase "Keywords".

> +  if (!current_function_decl)
> +    {

extra whitespace after {

> +  else if (TREE_CODE (fcall) != CALL_EXPR)
> +    {

extra whitespace after {

> diff --git a/gcc/testsuite/c-c++-common/cilk-plus/CK/spawnee_inline.c b/gcc/testsuite/c-c++-common/cilk-plus/CK/spawnee_inline.c
> new file mode 100644
> index 0000000..daf932e
> --- /dev/null
> +++ b/gcc/testsuite/c-c++-common/cilk-plus/CK/spawnee_inline.c
> @@ -0,0 +1,81 @@
> +/* { dg-do compile } */
> +/* { dg-do run  { target { i?86-*-* x86_64-*-* arm*-*-* } } } */
> +/* { dg-options "-fcilkplus -w" } */
> +/* { dg-options "-lcilkrts" { target { i?86-*-* x86_64-*-* arm*-*-* } } } */
> +
> +#include <stdio.h>
> +#include <stdlib.h>
> +#define DEFAULT_VALUE "30"
> +
> +int fib (char *n_char)
> +{
> +  int n;
> +  char n_char_minus_one[20], n_char_minus_two[20];
> +  if (n_char)
> +    n = atoi (n_char);
> +  else
> +    n = atoi(DEFAULT_VALUE);
> +
> +  if (n < 2)
> +    return n;
> +  else
> +    {	

Extra whitespace after last {

> diff --git a/gcc/testsuite/c-c++-common/cilk-plus/CK/spawning_arg.c b/gcc/testsuite/c-c++-common/cilk-plus/CK/spawning_arg.c
> new file mode 100644
> index 0000000..465d1da
> --- /dev/null
> +++ b/gcc/testsuite/c-c++-common/cilk-plus/CK/spawning_arg.c
> @@ -0,0 +1,38 @@
> +/* { dg-do compile } */
> +/* { dg-do run  { target { i?86-*-* x86_64-*-* arm*-*-* } } } */
> +/* { dg-options "-fcilkplus" } */
> +/* { dg-options "-lcilkrts" { target { i?86-*-* x86_64-*-* arm*-*-* } } } */
> +
> +void f0(volatile int *steal_flag)
> +{

same

> +/* This function does whatever is necessary to make the compiler emit a newly

Extra space at end

> +/* Return true if this is a tree which is allowed to contain a spawn as
> +   operand 0.
> +   A spawn call may be wrapped in a series of unary operations such
> +   as conversions.  These conversions need not be "useless"
> +   to be disregarded because they are retained in the spawned
> +   statement.  They are bypassed only to look for a spawn
> +   within.
> +   A comparison to constant is simple enough to allow, and
> +   is used to convert to bool.  */

Please add empty lines between paragraphs.

> +static bool
> +cilk_spawnable_constructor (tree exp)
> +{
> +  if (TREE_CODE (exp) != ADDR_EXPR)
> +    return false;

Is this function ever called with a non ADDR_EXPR?

> +/* Helper function for walk_tree.  If *TP is a CILK_SPAWN_STMT, then unwrap
> +   this "wrapper" and  *WALK_SUBTREES is set to 0.  The function returns

two many spaces after "and"

Also, do not document what you are doing in the code by repeating it. 
For instance, instead of "*WALK_SUBTREES is set to 0", either omit this, 
or say what is being accomplished by setting WALK_SUBTREES to 0.  I 
would just omit it.

> +      /* Remove the CALL_EXPR from CILK_SPAWN_STMT wrapper and return true.  */

Again, don't document the obvious.  No need to say "and return true". 
That much is obvious from the code.
> +  /* Happens with C++ TARGET_EXPR.  */
> +  if (exp == NULL_TREE)
> +    return false;

Extra space after false;

> +  warn = !cilk_spawnable_constructor (CALL_EXPR_FN (exp));
> +
> +  /* The function address of a call may not be computed via a spawn.
> +     Look at the arglist only, and only the second argument which
> +     is the RHS of any plausible assignment or copy.  The first
> +     argument is the LHS.  A third argument could be a size for
> +     memcpy.  This path supports op= in addition to =, only because
> +     it is easy to do so. */
> +  if (call_expr_nargs (exp) < 2)
> +    return false;
> +
> +  exp = CALL_EXPR_ARG (exp, 0);
> +
> +  STRIP_USELESS_TYPE_CONVERSION (exp);
> +
> +  if (TREE_CODE (exp) == ADDR_EXPR)
> +    exp = TREE_OPERAND (exp, 0);
> +
> +  if (TREE_CODE (exp) == TARGET_EXPR)
> +    exp = TARGET_EXPR_INITIAL (exp);
> +
> +  if (!exp || !recognize_spawn (exp, exp0))
> +    return false;
> +
> +  if (warn)
> +    warning (0, "suspicious use of _Cilk_spawn");

This warning seems very ambiguous.  Do you think perhaps you could issue 
a more meaningful error in cilk_spawnable_constructor()?

> +/* This function will return a FNDECL using information from *WD.  */
> +
> +static tree
> +build_cilk_helper_decl (struct wrapper_data *wd)

I think a more meaningful comment is "This function will build and 
return a FUNCTION_DECL using..."

> +    case LABEL_DECL:
> +      error_at (EXPR_LOCATION (decl), "invalid use of label %q+D in spawn",
> +		decl);
> +      return error_mark_node;

Be consistent.  If you're using _Cilk_spawn then use that.  And you 
probably want %<_Cilk_spawn> or whatever.

> +  /* Copy FROM the function containing the spawn...  */
> +  id.src_fn = outer_fn;
> +
> +  /* ...TO the wrapper.  */

Lower case FROM and TO.
Aldy Hernandez Aug. 20, 2013, 10:04 p.m. UTC | #2
[rth, law, jakub: Your input required throughout...please.]

More review stuff...

Overall, I must say, I'm not a big fan of the super early expansion 
you're doing right after parsing.  I mean, you leave CILK_SPAWN and 
CILK_SYNC keywords as is (in tree form until gimplification) but there's 
this awful lot of expansion that goes on parallel to that.  For 
instance, for this code:

extern void testing();
extern void ending();

foo(){
     _Cilk_spawn bar();
     testing();
     _Cilk_sync;
     ending();
}

...right after parsing you already generate:

{
   __cilkrts_enter_frame_1 (&D.1776);
   try
     {
       _Cilk_spawn bar ();	// keywords as trees, fine
       testing ();
       _Cilk_sync;;		// keywords as trees, fine
       ending ();
     }
   finally			// but all this try/finally
				// support stuff too early??
     {
       D.1776.pedigree = D.1776.worker->pedigree;
       if ((D.1776.flags & 2) != 0)
         {
           __cilkrts_save_fp_ctrl_state (&D.1776);
           if (__builtin_setjmp (&D.1776.ctx) == 0)
             {
               __cilkrts_sync (&D.1776);
             }
           else
             {
               if ((D.1776.flags & 16) != 0)
                 {
                   __cilkrts_rethrow (&D.1776);
                 }
             }
         }
       D.1776.worker->pedigree.rank = D.1776.worker->pedigree.rank + 1;
       D.1776.worker->current_stack_frame = D.1776.call_parent;
       __cilkrts_pop_frame (&D.1776);
       if (D.1776.flags != 16777216)
         {
           __cilkrts_leave_frame (&D.1776);
         }
     }
}

You seem to be hijacking finish_function(), to generate all this 
try/finally and builtin stuff here:

> diff --git a/gcc/c/c-decl.c b/gcc/c/c-decl.c
> index f7ae648..ffd62c6 100644
> --- a/gcc/c/c-decl.c
> +++ b/gcc/c/c-decl.c
> @@ -8380,6 +8380,12 @@ finish_function (void)
>    /* Tie off the statement tree for this function.  */
>    DECL_SAVED_TREE (fndecl) = pop_stmt_list (DECL_SAVED_TREE (fndecl));
>
> +  /* IF the function has _Cilk_spawn in front of a function call inside it
> +     i.e. it is a spawning function, then add the appropriate Cilk plus
> +     functions inside.  */
> +  if (flag_enable_cilkplus && cfun->calls_cilk_spawn == 1)
> +    cfun->cilk_frame_decl = insert_cilk_frame (fndecl);

I would've preferred to leave all expansion until *at least* 
gimplification.  Although, looking at how OMP works, expansion happens 
even further down after the CFG has been built and EH has been handled. 
  Seeing that _Cilk_spawn and _Cilk_sync heavily alter the flow (and 
possibly exceptions) of the program, I would guess that you need to 
follow a similar path with these two Cilk keywords.

I don't think this is a blocker, especially since most everything seems 
to be contained in Cilk specific files, but I would like to get some 
feedback from Jeff/Richard/Jakub, and/or some other more globally savvy 
people :).  Perhaps this could be done as a follow-up patch, if 
necessary.  Having said that, at least expansion in finish_function() 
feels weird.

> +tree
> +c_build_cilk_spawn (location_t loc, tree call)
> +{
> +  if (!cilkplus_set_spawn_marker (loc, call))
> +    return error_mark_node;
> +  tree spawn_stmt = build1 (CILK_SPAWN_STMT, TREE_TYPE (call), call);

Do you need a TREE_TYPE here at all?  Is it used anywhere?

> +/* Marks CALL, a CALL_EXPR, as a spawned function call.  */
> +
> +tree
> +c_build_cilk_spawn (location_t loc, tree call)
> +{
> +  if (!cilkplus_set_spawn_marker (loc, call))
> +    return error_mark_node;

Can you inline cilkplus_set_spawn_marker?  It's not used anywhere else 
and it's relatively small.  If you need it for C++, perhaps you can make 
c_build_cilk_spawn() generic enough to be used for both FE's?

> +/* Helper function for walk_tree.  If *TP is a CILK_SPAWN_STMT, then unwrap
> +   this "wrapper" and  *WALK_SUBTREES is set to 0.  The function returns
> +   NULL_TREE regardless.  */
> +
> +static tree
> +unwrap_cilk_sync_stmt (tree *tp, int *walk_subtrees, void *)
> +{
> +  if (TREE_CODE (*tp) == CILK_SPAWN_STMT)
> +    {
> +      *tp = CILK_SPAWN_FN (*tp);
> +      *walk_subtrees = 0;
> +    }
> +  return NULL_TREE;
> +}

Why is this called unwrap_cilk_sync_stmt when it is unwrapping a spawn?

> +    case CILK_SYNC_STMT:
> +      pp_string (buffer, "_Cilk_sync;");
> +      break;

Drop the semi-colon.  The end of dump_generic_node() should add the 
semi-colon if it's a statement.  I bet you're probably getting two 
semi-colons at the end of _Cilk_sync in the dump.

> +/* Returns a wrapper function for a _Cilk_spawn.  */
> +
> +static tree
> +build_cilk_wrapper (tree exp, tree *args_out)

This name is confusing.  The world "build" is usually used to denote 
front-end building of trees, not gimplification.  Seeing that 
build_cilk_wrapper() is only called from gimplify_cilk_spawn(), it's a 
bit confusing.

> +/* This function will expand a cilk_sync call.  */
> +
> +static tree
> +build_cilk_sync (void)
> +{
> +  tree frame = cfun->cilk_frame_decl;

Similarly with this as well, which is mostly a helper for 
gimplify_cilk_sync().  And for that matter, it can be confused with 
c_build_cilk_sync() which does something completely different.

I do see that you also call build_cilk_sync() from 
build_cilk_function_exit() here:

> +/* This function will output the exit conditions for a spawn call.  */
> +
> +tree
> +build_cilk_function_exit (tree frame, bool detaches, bool needs_sync)
> +{
> +  tree sync_expr = NULL_TREE;
> +
> +  tree epi = alloc_stmt_list ();
> +
> +  if (needs_sync)
> +    {
> +      sync_expr = build_cilk_sync ();
> +      append_to_statement_list (sync_expr, &epi);
> +    }

It seems to me that you should have generated this implicit _Cilk_sync 
at function exit by an earlier pass.  If jakub/rth/others think it is 
incorrect to expand so early (as I have pointed out), perhaps adding the 
_Cilk_sync could be done similarly to how OMP generates implicit OMP 
returns in the omplower stage.

For example, for something like:

foo(){
     int i;
#pragma omp for
     for (i=0; i < 10; ++i)
       bar();
}

omplower expands to:

     #pragma omp for private(i)
     for (i = 0; i <= 9; i = i + 1)
     bar ();
     #pragma omp continue (i, i)
     #pragma omp return

Anyways... let's wait to hear from the heavy hitters.
Iyer, Balaji V Aug. 21, 2013, 12:41 a.m. UTC | #3
HI Aldy et al.,
	I would like to address the early expansion question. The reason why I did it is this way is because it was straight-forward for me to implement. I did some preliminary analysis, and it wasn't blocking any major optimization. Also, after parsing all I am really doing is to insert a couple functions. The expansion is actually done during gimplification (in gimplify_expr).

	If it is Ok with everyone, I would like to keep it as-is for now. After I finish getting all the Cilk Plus parts into trunk, I will go back and look into optimizing this. This way, I can have some time (november to feb, during stage3) to do some more analysis and see if I can come up with a better solution.

Thanks,

Balaji V. Iyer.

> -----Original Message-----
> From: Aldy Hernandez [mailto:aldyh@redhat.com]
> Sent: Tuesday, August 20, 2013 6:04 PM
> To: Iyer, Balaji V
> Cc: rth@redhat.com; Jeff Law; gcc-patches@gcc.gnu.org; Jakub Jelinek
> Subject: Re: [PATCH] Cilk Keywords (_Cilk_spawn and _Cilk_sync) for C
> 
> [rth, law, jakub: Your input required throughout...please.]
> 
> More review stuff...
> 
> Overall, I must say, I'm not a big fan of the super early expansion you're doing
> right after parsing.  I mean, you leave CILK_SPAWN and CILK_SYNC keywords as
> is (in tree form until gimplification) but there's this awful lot of expansion that
> goes on parallel to that.  For instance, for this code:
> 
> extern void testing();
> extern void ending();
> 
> foo(){
>      _Cilk_spawn bar();
>      testing();
>      _Cilk_sync;
>      ending();
> }
> 
> ...right after parsing you already generate:
> 
> {
>    __cilkrts_enter_frame_1 (&D.1776);
>    try
>      {
>        _Cilk_spawn bar ();	// keywords as trees, fine
>        testing ();
>        _Cilk_sync;;		// keywords as trees, fine
>        ending ();
>      }
>    finally			// but all this try/finally
> 				// support stuff too early??
>      {
>        D.1776.pedigree = D.1776.worker->pedigree;
>        if ((D.1776.flags & 2) != 0)
>          {
>            __cilkrts_save_fp_ctrl_state (&D.1776);
>            if (__builtin_setjmp (&D.1776.ctx) == 0)
>              {
>                __cilkrts_sync (&D.1776);
>              }
>            else
>              {
>                if ((D.1776.flags & 16) != 0)
>                  {
>                    __cilkrts_rethrow (&D.1776);
>                  }
>              }
>          }
>        D.1776.worker->pedigree.rank = D.1776.worker->pedigree.rank + 1;
>        D.1776.worker->current_stack_frame = D.1776.call_parent;
>        __cilkrts_pop_frame (&D.1776);
>        if (D.1776.flags != 16777216)
>          {
>            __cilkrts_leave_frame (&D.1776);
>          }
>      }
> }
> 
> You seem to be hijacking finish_function(), to generate all this try/finally and
> builtin stuff here:
> 
> > diff --git a/gcc/c/c-decl.c b/gcc/c/c-decl.c index f7ae648..ffd62c6
> > 100644
> > --- a/gcc/c/c-decl.c
> > +++ b/gcc/c/c-decl.c
> > @@ -8380,6 +8380,12 @@ finish_function (void)
> >    /* Tie off the statement tree for this function.  */
> >    DECL_SAVED_TREE (fndecl) = pop_stmt_list (DECL_SAVED_TREE
> > (fndecl));
> >
> > +  /* IF the function has _Cilk_spawn in front of a function call inside it
> > +     i.e. it is a spawning function, then add the appropriate Cilk plus
> > +     functions inside.  */
> > +  if (flag_enable_cilkplus && cfun->calls_cilk_spawn == 1)
> > +    cfun->cilk_frame_decl = insert_cilk_frame (fndecl);
> 
> I would've preferred to leave all expansion until *at least* gimplification.
> Although, looking at how OMP works, expansion happens even further down
> after the CFG has been built and EH has been handled.
>   Seeing that _Cilk_spawn and _Cilk_sync heavily alter the flow (and possibly
> exceptions) of the program, I would guess that you need to follow a similar path
> with these two Cilk keywords.
> 
> I don't think this is a blocker, especially since most everything seems to be
> contained in Cilk specific files, but I would like to get some feedback from
> Jeff/Richard/Jakub, and/or some other more globally savvy people :).  Perhaps
> this could be done as a follow-up patch, if necessary.  Having said that, at least
> expansion in finish_function() feels weird.
> 
> > +tree
> > +c_build_cilk_spawn (location_t loc, tree call) {
> > +  if (!cilkplus_set_spawn_marker (loc, call))
> > +    return error_mark_node;
> > +  tree spawn_stmt = build1 (CILK_SPAWN_STMT, TREE_TYPE (call), call);
> 
> Do you need a TREE_TYPE here at all?  Is it used anywhere?
> 
> > +/* Marks CALL, a CALL_EXPR, as a spawned function call.  */
> > +
> > +tree
> > +c_build_cilk_spawn (location_t loc, tree call) {
> > +  if (!cilkplus_set_spawn_marker (loc, call))
> > +    return error_mark_node;
> 
> Can you inline cilkplus_set_spawn_marker?  It's not used anywhere else and it's
> relatively small.  If you need it for C++, perhaps you can make
> c_build_cilk_spawn() generic enough to be used for both FE's?
> 
> > +/* Helper function for walk_tree.  If *TP is a CILK_SPAWN_STMT, then
> unwrap
> > +   this "wrapper" and  *WALK_SUBTREES is set to 0.  The function returns
> > +   NULL_TREE regardless.  */
> > +
> > +static tree
> > +unwrap_cilk_sync_stmt (tree *tp, int *walk_subtrees, void *) {
> > +  if (TREE_CODE (*tp) == CILK_SPAWN_STMT)
> > +    {
> > +      *tp = CILK_SPAWN_FN (*tp);
> > +      *walk_subtrees = 0;
> > +    }
> > +  return NULL_TREE;
> > +}
> 
> Why is this called unwrap_cilk_sync_stmt when it is unwrapping a spawn?
> 
> > +    case CILK_SYNC_STMT:
> > +      pp_string (buffer, "_Cilk_sync;");
> > +      break;
> 
> Drop the semi-colon.  The end of dump_generic_node() should add the semi-
> colon if it's a statement.  I bet you're probably getting two semi-colons at the
> end of _Cilk_sync in the dump.
> 
> > +/* Returns a wrapper function for a _Cilk_spawn.  */
> > +
> > +static tree
> > +build_cilk_wrapper (tree exp, tree *args_out)
> 
> This name is confusing.  The world "build" is usually used to denote front-end
> building of trees, not gimplification.  Seeing that
> build_cilk_wrapper() is only called from gimplify_cilk_spawn(), it's a bit
> confusing.
> 
> > +/* This function will expand a cilk_sync call.  */
> > +
> > +static tree
> > +build_cilk_sync (void)
> > +{
> > +  tree frame = cfun->cilk_frame_decl;
> 
> Similarly with this as well, which is mostly a helper for gimplify_cilk_sync().  And
> for that matter, it can be confused with
> c_build_cilk_sync() which does something completely different.
> 
> I do see that you also call build_cilk_sync() from
> build_cilk_function_exit() here:
> 
> > +/* This function will output the exit conditions for a spawn call.
> > +*/
> > +
> > +tree
> > +build_cilk_function_exit (tree frame, bool detaches, bool needs_sync)
> > +{
> > +  tree sync_expr = NULL_TREE;
> > +
> > +  tree epi = alloc_stmt_list ();
> > +
> > +  if (needs_sync)
> > +    {
> > +      sync_expr = build_cilk_sync ();
> > +      append_to_statement_list (sync_expr, &epi);
> > +    }
> 
> It seems to me that you should have generated this implicit _Cilk_sync at
> function exit by an earlier pass.  If jakub/rth/others think it is incorrect to expand
> so early (as I have pointed out), perhaps adding the _Cilk_sync could be done
> similarly to how OMP generates implicit OMP returns in the omplower stage.
> 
> For example, for something like:
> 
> foo(){
>      int i;
> #pragma omp for
>      for (i=0; i < 10; ++i)
>        bar();
> }
> 
> omplower expands to:
> 
>      #pragma omp for private(i)
>      for (i = 0; i <= 9; i = i + 1)
>      bar ();
>      #pragma omp continue (i, i)
>      #pragma omp return
> 
> Anyways... let's wait to hear from the heavy hitters.
Aldy Hernandez Aug. 21, 2013, 3:31 p.m. UTC | #4
Even more review stuff.  Are you keeping track of all this Balaji? :)

> +  if (warn)
> +    warning (0, "suspicious use of _Cilk_spawn");

First, as I've mentioned, this error message is very ambiguous.  You 
should strive to provide better error messages.  See my previous comment 
on this same line of code.

However... for all the checking you do in cilk_valid_spawn, I don't see 
a single corresponding test.

May I stress again the importance of tests-- which are especially
critical for new language features.  You don't want cilk silently
breaking thus rendering all your hard work moot, do you? :))

You particularly need tests for all quirks described in the Cilk Plus
language specification around here:

"A program is considered ill formed if the _Cilk_spawn form of this
expression appears other than in one of the following contexts: [blah
blah blah]".


> +  /* Strip off any conversion to void.  It does not affect whether spawn
> +     is supported here.  */
> +  if (TREE_CODE (exp) == CONVERT_EXPR && VOID_TYPE_P (TREE_TYPE (exp)))
> +    exp = TREE_OPERAND (exp, 0);

Don't you need to strip off various levels here with a loop?  Also, 
could any of the following do the job? STRIP_NOPS, STRIP_TYPE_NOPS, 
STRIP_USELESS_TYPE_CONVERSION.

> @@ -7086,6 +7087,19 @@ gimplify_expr (tree *expr_p, gimple_seq *pre_p, gimple_seq *post_p,
>        else if (ret != GS_UNHANDLED)
>  	break;
>
> +      if (flag_enable_cilkplus && lang_hooks.cilkplus.cilk_valid_spawn (expr_p))
> +	{
> +	  /* If there are errors, there is no point in expanding the
> +	     _Cilk_spawn.  Just gimplify like a normal call expr.  */
> +	  if (!seen_error ())
> +	    {
> +	      ret = (enum gimplify_status)
> +		lang_hooks.cilkplus.gimplify_cilk_spawn (expr_p, pre_p, post_p);
> +	      if (ret != GS_UNHANDLED)
> +		continue;
> +	    }
> +	}
> +

Oh, hell no!  You do realize you are drilling down and walking every 
single expression being passed to the gimplifier to find your spawn? 
That's not cool.  You need to find some way to annotate expressions or 
do this more efficiently.  It may help to bootstrap with -fcilkplus and 
do performance analysis, to make sure you're not making the compiler 
slower on the non cilkplus code path.

Could you not let the gimplifier do its thing and add a case for 
CILK_SPAWN_STMT where you do the unwrapping and everything else?  I do 
realize that cilk_valid_spawn() is doing all sorts of type checking, and 
validation, but the gimplifier is really not the place to do this.  When 
possible, you should do type checking as close to the source as 
possible, thus-- at the parser.  See how c_finish_omp_for() is called 
from the FE to do type checking, build the OMP_FOR tree node, *and* do 
the add_stmt().  Perhaps you need corresponding a 
c_finish_cilk_{spawn,sync}.  Definitely worth looking into.  But I can 
tell you now, drilling down into every expression being gimplified is a 
no-go.

Also, do you realy need two hooks to recognize spawns: recognize_spawn 
and cilk_valid_spawn?  And are C/C++ so different that you need a hook
with different versions of each?

> +/* Returns a setjmp CALL_EXPR with FRAME->context as its parameter.  */
> +
> +tree
> +cilk_call_setjmp (tree frame)

Is this used anywhere else but in this file?  If not, please declare static.

> +/* Expands the __cilkrts_pop_frame function call stored in EXP.
> +   Returns const0_rtx.  */
> +
> +void
> +expand_builtin_cilk_pop_frame (tree exp)
[snip]
> +/* Expands the cilk_detach function call stored in EXP.  Returns const0_rtx.  */
> +
> +void
> +expand_builtin_cilk_detach (tree exp)

Do these builtins really have to be expanded into rtl?  Can this not be 
modeled with trees or gimple?  Expansion into rtl should be used for 
truly architecture dependent stuff that cannot be modeled with anything 
higher level.

For the memory barrier stuff, we already have Andrew's atomic and memory 
model infrastructure which I think should be enough to model whatever 
you are expanding into RTL here.  But I may be wrong...

Aldy
Jeff Law Aug. 21, 2013, 6:44 p.m. UTC | #5
On 08/21/2013 09:31 AM, Aldy Hernandez wrote:
>
> May I stress again the importance of tests-- which are especially
> critical for new language features.  You don't want cilk silently
> breaking thus rendering all your hard work moot, do you? :))
Agreed.  While we don't have a strict policy for testing new features, 
adding tests for this kind of stuff is highly encouraged.  Not everyone 
doing GCC development is going to be familiar enough with Cilk+ and how 
their patch might interact with the Cilk+ support.

Having tests in the testsuite makes it much less likely someone will 
break the Cilk+ support accidentally.

>> +      if (flag_enable_cilkplus &&
>> lang_hooks.cilkplus.cilk_valid_spawn (expr_p))
>> +    {
>> +      /* If there are errors, there is no point in expanding the
>> +         _Cilk_spawn.  Just gimplify like a normal call expr.  */
>> +      if (!seen_error ())
>> +        {
>> +          ret = (enum gimplify_status)
>> +        lang_hooks.cilkplus.gimplify_cilk_spawn (expr_p, pre_p, post_p);
>> +          if (ret != GS_UNHANDLED)
>> +        continue;
>> +        }
>> +    }
>> +
>
> Oh, hell no!  You do realize you are drilling down and walking every
> single expression being passed to the gimplifier to find your spawn?
> That's not cool.  You need to find some way to annotate expressions or
> do this more efficiently.  It may help to bootstrap with -fcilkplus and
> do performance analysis, to make sure you're not making the compiler
> slower on the non cilkplus code path.
Yea, that would definitely be a problem (walking every expression 
looking for cilk spawns inside).  Having gimple nodes for these things 
would seem to make sense to me as well.

>
> Could you not let the gimplifier do its thing and add a case for
> CILK_SPAWN_STMT where you do the unwrapping and everything else?  I do
> realize that cilk_valid_spawn() is doing all sorts of type checking, and
> validation, but the gimplifier is really not the place to do this.  When
> possible, you should do type checking as close to the source as
> possible, thus-- at the parser.  See how c_finish_omp_for() is called
> from the FE to do type checking, build the OMP_FOR tree node, *and* do
> the add_stmt().  Perhaps you need corresponding a
> c_finish_cilk_{spawn,sync}.  Definitely worth looking into.  But I can
> tell you now, drilling down into every expression being gimplified is a
> no-go.
Yea, we *really* want the gimplification & checking separated.   If we 
think about where Andrew's proposals take us, then the checking needs to 
move into the front-end.  gimplification should be relegated, to the 
fullest extent possible, to converting the IL down to gimple.




Jeff
Iyer, Balaji V Aug. 21, 2013, 7:59 p.m. UTC | #6
> -----Original Message-----
> From: Aldy Hernandez [mailto:aldyh@redhat.com]
> Sent: Wednesday, August 21, 2013 11:31 AM
> To: Iyer, Balaji V
> Cc: rth@redhat.com; Jeff Law; gcc-patches@gcc.gnu.org
> Subject: Re: [PATCH] Cilk Keywords (_Cilk_spawn and _Cilk_sync) for C
> 
> Even more review stuff.  Are you keeping track of all this Balaji? :)
> 

Yes I am. Please keep an eye out for a fixed patch soon.

> > +  if (warn)
> > +    warning (0, "suspicious use of _Cilk_spawn");
> 
> First, as I've mentioned, this error message is very ambiguous.  You should strive
> to provide better error messages.  See my previous comment on this same line
> of code.
> 
> However... for all the checking you do in cilk_valid_spawn, I don't see a single
> corresponding test.
> 

Well, the name of the function is misleading.  I will fix that. I think it should call it "detect_cilk_spawn" instead

What the function does it NOT to find whether there are syntax or other issues in the spawned statement, but to check if spawn is used in appropriate location.
Here are some cases were you can use spawn (I am sure I am missing something):

X = _Cilk_spawn foo ();

_Cilk_spawn foo ()

operator=(x, _Cilk_spawn foo ())

and these things can be kept in different kind of trees and so adding this in individual tree's case statement can be a lot of code-addition and is error prone.

The warning you see is more like an "heads up." I can take out of if you like. If you notice, when I see an error, I don't bother gimplifying the spawned function (but just let the compiler go ahead as a regular function call) thereby not creating a new nested function etc.

> May I stress again the importance of tests-- which are especially critical for new
> language features.  You don't want cilk silently breaking thus rendering all your
> hard work moot, do you? :))
> 
> You particularly need tests for all quirks described in the Cilk Plus language
> specification around here:
> 
> "A program is considered ill formed if the _Cilk_spawn form of this expression
> appears other than in one of the following contexts: [blah blah blah]".
>

I have several of those already (e.g. using spawn outside a function, spawning something that is not a function, etc)
 
> 
> > +  /* Strip off any conversion to void.  It does not affect whether spawn
> > +     is supported here.  */
> > +  if (TREE_CODE (exp) == CONVERT_EXPR && VOID_TYPE_P (TREE_TYPE
> (exp)))
> > +    exp = TREE_OPERAND (exp, 0);
> 
> Don't you need to strip off various levels here with a loop?  Also, could any of
> the following do the job? STRIP_NOPS, STRIP_TYPE_NOPS,
> STRIP_USELESS_TYPE_CONVERSION.
> 
> > @@ -7086,6 +7087,19 @@ gimplify_expr (tree *expr_p, gimple_seq *pre_p,
> gimple_seq *post_p,
> >        else if (ret != GS_UNHANDLED)
> >  	break;
> >
> > +      if (flag_enable_cilkplus && lang_hooks.cilkplus.cilk_valid_spawn (expr_p))
> > +	{
> > +	  /* If there are errors, there is no point in expanding the
> > +	     _Cilk_spawn.  Just gimplify like a normal call expr.  */
> > +	  if (!seen_error ())
> > +	    {
> > +	      ret = (enum gimplify_status)
> > +		lang_hooks.cilkplus.gimplify_cilk_spawn (expr_p, pre_p,
> post_p);
> > +	      if (ret != GS_UNHANDLED)
> > +		continue;
> > +	    }
> > +	}
> > +
> 
> Oh, hell no!  You do realize you are drilling down and walking every single
> expression being passed to the gimplifier to find your spawn?
> That's not cool.  You need to find some way to annotate expressions or do this
> more efficiently.  It may help to bootstrap with -fcilkplus and do performance
> analysis, to make sure you're not making the compiler slower on the non cilkplus
> code path.
> 
> Could you not let the gimplifier do its thing and add a case for
> CILK_SPAWN_STMT where you do the unwrapping and everything else?  I do
> realize that cilk_valid_spawn() is doing all sorts of type checking, and validation,
> but the gimplifier is really not the place to do this.  When possible, you should do
> type checking as close to the source as possible, thus-- at the parser.  See how
> c_finish_omp_for() is called from the FE to do type checking, build the OMP_FOR
> tree node, *and* do the add_stmt().  Perhaps you need corresponding a
> c_finish_cilk_{spawn,sync}.  Definitely worth looking into.  But I can tell you
> now, drilling down into every expression being gimplified is a no-go.
> 

Well, I think the name of the function is what that is misleading here.

I am not recursing through the entire tree to find the spawn keyword here. What I am trying to see is if "*expr_p" is an INIT_EXPR, or TARGET_EXPR or a CALL_EXPR etc.

I do agree with you about one thing. I should first check to see if the function has a _Cilk_spawn before I go through and check for individual trees. That can be done easily by looking at cfun->cilk_frame_decl != NULL_TREE. That change I will make in the next patch. 

> Also, do you realy need two hooks to recognize spawns: recognize_spawn and
> cilk_valid_spawn?  And are C/C++ so different that you need a hook with
> different versions of each?
> 
> > +/* Returns a setjmp CALL_EXPR with FRAME->context as its parameter.
> > +*/
> > +
> > +tree
> > +cilk_call_setjmp (tree frame)
> 
> Is this used anywhere else but in this file?  If not, please declare static.
> 
> > +/* Expands the __cilkrts_pop_frame function call stored in EXP.
> > +   Returns const0_rtx.  */
> > +
> > +void
> > +expand_builtin_cilk_pop_frame (tree exp)
> [snip]
> > +/* Expands the cilk_detach function call stored in EXP.  Returns
> > +const0_rtx.  */
> > +
> > +void
> > +expand_builtin_cilk_detach (tree exp)
> 
> Do these builtins really have to be expanded into rtl?  Can this not be modeled
> with trees or gimple?  Expansion into rtl should be used for truly architecture
> dependent stuff that cannot be modeled with anything higher level.
> 

Again, the reason why I do it there because it is easier for me. I don't think it is causing a huge performance hit both in the compiler or the executable.

> For the memory barrier stuff, we already have Andrew's atomic and memory
> model infrastructure which I think should be enough to model whatever you are
> expanding into RTL here.  But I may be wrong...
> 
> Aldy
Aldy Hernandez Aug. 22, 2013, 4:52 p.m. UTC | #7
On 08/21/13 14:59, Iyer, Balaji V wrote:
>
>
>> -----Original Message----- From: Aldy Hernandez
>> [mailto:aldyh@redhat.com] Sent: Wednesday, August 21, 2013 11:31 AM
>> To: Iyer, Balaji V Cc: rth@redhat.com; Jeff Law;
>> gcc-patches@gcc.gnu.org Subject: Re: [PATCH] Cilk Keywords
>> (_Cilk_spawn and _Cilk_sync) for C
>>
>> Even more review stuff.  Are you keeping track of all this Balaji?
>> :)
>>
>
> Yes I am. Please keep an eye out for a fixed patch soon.
>
>>> +  if (warn) +    warning (0, "suspicious use of _Cilk_spawn");
>>
>> First, as I've mentioned, this error message is very ambiguous. You
>> should strive to provide better error messages.  See my previous
>> comment on this same line of code.
>>
>> However... for all the checking you do in cilk_valid_spawn, I
>> don't see a single corresponding test.
>>
>
> Well, the name of the function is misleading.  I will fix that. I
> think it should call it "detect_cilk_spawn" instead
>
> What the function does it NOT to find whether there are syntax or
> other issues in the spawned statement, but to check if spawn is used
> in appropriate location. Here are some cases were you can use spawn
> (I am sure I am missing something):
>
> X = _Cilk_spawn foo ();
>
> _Cilk_spawn foo ()
>
> operator=(x, _Cilk_spawn foo ())
>
> and these things can be kept in different kind of trees and so
> adding this in individual tree's case statement can be a lot of
> code-addition and is error prone.

It still sounds like some sort of type checking best done at the source
level.  You can analyze all this in the parser as suggested.  Checking 
if spawn is used in the appropriate location, as you mention, should not 
be done in the gimplifier.

And btw, the reason you have to check all these variants (whether in a a 
MODIFY_EXPR, INIT_EXPR, CALL_EXPR, operator, etc) is because you are 
dealing with trees.  If you handled this as a gimple tuple, things would 
have already been simplified enough for you to only have to worry about 
GIMPLE_CALL.  That being said, I understand it would be more work to 
redesign things to work at the gimple level, but it is something we want 
to do eventually.

>
> The warning you see is more like an "heads up." I can take out of if
> you like. If you notice, when I see an error, I don't bother
> gimplifying the spawned function (but just let the compiler go ahead
> as a regular function call) thereby not creating a new nested
> function etc.

No, I don't want you to take it out.  For that matter (as I've suggested 
earlier up-thread), I would like you to expand on this and provide more 
verbose errors/warnings for each of the different things you can catch, 
instead of the the generic "suspicious" warning.

>
>> May I stress again the importance of tests-- which are especially
>> critical for new language features.  You don't want cilk silently
>> breaking thus rendering all your hard work moot, do you? :))
>>
>> You particularly need tests for all quirks described in the Cilk
>> Plus language specification around here:
>>
>> "A program is considered ill formed if the _Cilk_spawn form of
>> this expression appears other than in one of the following
>> contexts: [blah blah blah]".
>>
>
> I have several of those already (e.g. using spawn outside a
> function, spawning something that is not a function, etc)

I just didn't see any test for the "suspicious" warning.

>
>>
>>> +  /* Strip off any conversion to void.  It does not affect
>>> whether spawn +     is supported here.  */ +  if (TREE_CODE
>>> (exp) == CONVERT_EXPR && VOID_TYPE_P (TREE_TYPE
>> (exp)))
>>> +    exp = TREE_OPERAND (exp, 0);
>>
>> Don't you need to strip off various levels here with a loop?
>> Also, could any of the following do the job? STRIP_NOPS,
>> STRIP_TYPE_NOPS, STRIP_USELESS_TYPE_CONVERSION.
>>
>>> @@ -7086,6 +7087,19 @@ gimplify_expr (tree *expr_p, gimple_seq
>>> *pre_p,
>> gimple_seq *post_p,
>>> else if (ret != GS_UNHANDLED) break;
>>>
>>> +      if (flag_enable_cilkplus &&
>>> lang_hooks.cilkplus.cilk_valid_spawn (expr_p)) +	{ +	  /* If
>>> there are errors, there is no point in expanding the +
>>> _Cilk_spawn.  Just gimplify like a normal call expr.  */ +	  if
>>> (!seen_error ()) +	    { +	      ret = (enum gimplify_status) +
>>> lang_hooks.cilkplus.gimplify_cilk_spawn (expr_p, pre_p,
>> post_p);
>>> +	      if (ret != GS_UNHANDLED) +		continue; +	    } +	} +
>>
>> Oh, hell no!  You do realize you are drilling down and walking
>> every single expression being passed to the gimplifier to find
>> your spawn? That's not cool.  You need to find some way to
>> annotate expressions or do this more efficiently.  It may help to
>> bootstrap with -fcilkplus and do performance analysis, to make sure
>> you're not making the compiler slower on the non cilkplus code
>> path.
>>
>> Could you not let the gimplifier do its thing and add a case for
>> CILK_SPAWN_STMT where you do the unwrapping and everything else?
>> I do realize that cilk_valid_spawn() is doing all sorts of type
>> checking, and validation, but the gimplifier is really not the
>> place to do this.  When possible, you should do type checking as
>> close to the source as possible, thus-- at the parser.  See how
>> c_finish_omp_for() is called from the FE to do type checking,
>> build the OMP_FOR tree node, *and* do the add_stmt().  Perhaps you
>> need corresponding a c_finish_cilk_{spawn,sync}.  Definitely worth
>> looking into.  But I can tell you now, drilling down into every
>> expression being gimplified is a no-go.
>>
>
> Well, I think the name of the function is what that is misleading
> here.
>
> I am not recursing through the entire tree to find the spawn keyword
> here. What I am trying to see is if "*expr_p" is an INIT_EXPR, or
> TARGET_EXPR or a CALL_EXPR etc.

You are still iterating through every expression passed to 
gimplify_expr.  I mean, this is some of the stuff you do in 
cilk_valid_spawn:

+  /* Strip off any conversion to void.  It does not affect whether spawn
+     is supported here.  */
+  if (TREE_CODE (exp) == CONVERT_EXPR && VOID_TYPE_P (TREE_TYPE (exp)))
+    exp = TREE_OPERAND (exp, 0);
+
+  if (TREE_CODE (exp) == MODIFY_EXPR || TREE_CODE (exp) == INIT_EXPR)
+    exp = TREE_OPERAND (exp, 1);
+
+  while (cilk_ignorable_spawn_rhs_op (exp))
+    exp = TREE_OPERAND (exp, 0);

Whether we agree to go the gimple route or not, you need to rearrange 
this code in gimplify_expr to have a case for CILK_SPAWN_STMT instead of 
doing this ad-hoc iterating you're doing.

>
> I do agree with you about one thing. I should first check to see if
> the function has a _Cilk_spawn before I go through and check for
> individual trees. That can be done easily by looking at
> cfun->cilk_frame_decl != NULL_TREE. That change I will make in the
> next patch.

I thought you were already doing that.  See the top of cilk_valid_spawn():

+  /* If the function contains no Cilk code, this isn't a spawn.  */
+  if (!cfun->cilk_frame_decl)
+    return false;


>
>> Also, do you realy need two hooks to recognize spawns:
>> recognize_spawn and cilk_valid_spawn?  And are C/C++ so different
>> that you need a hook with different versions of each?
>>
>>> +/* Returns a setjmp CALL_EXPR with FRAME->context as its
>>> parameter. +*/ + +tree +cilk_call_setjmp (tree frame)
>>
>> Is this used anywhere else but in this file?  If not, please
>> declare static.
>>
>>> +/* Expands the __cilkrts_pop_frame function call stored in EXP.
>>>  +   Returns const0_rtx.  */ + +void
>>> +expand_builtin_cilk_pop_frame (tree exp)
>> [snip]
>>> +/* Expands the cilk_detach function call stored in EXP. Returns
>>> +const0_rtx.  */ + +void +expand_builtin_cilk_detach (tree exp)
>>
>> Do these builtins really have to be expanded into rtl?  Can this
>> not be modeled with trees or gimple?  Expansion into rtl should be
>> used for truly architecture dependent stuff that cannot be modeled
>> with anything higher level.
>>
>
> Again, the reason why I do it there because it is easier for me. I
> don't think it is causing a huge performance hit both in the
> compiler or the executable.

I realize it is easier for you, but so is hard-coding things throughout 
the compiler.  We need to look for the long term maintainability of the 
compiler.  I will let the appropriate maintainers chime in, but for the 
record I would much prefer to expand into trees instead of rtl.

Aldy
Aldy Hernandez Aug. 28, 2013, 6:35 p.m. UTC | #8
On 08/27/13 16:27, Iyer, Balaji V wrote:
> Hello Aldy, I went through all the emails and here are the major
> issues that I could gather (other than lowering the keywords after
> gimplification, which I am skipping since it is more of an
> optimization for now).

Ok, for now I am fine with delaying handling all this as a gimple tuple 
since most of your code lives in it's only little world :).  But I will 
go on record saying that part of the reason that you have to handle 
CALL_EXPR, MODIFY_EXPR, INIT_EXPR and such is because you don't have 
easy gimplified code to examine.  Anyways, agreed, you can do this later.

>
> 1. Calling the gimplify_cilk_spawn on top of the gimplify_expr before
> the switch-statement could slow the compiler down 2. I need a
> CILK_SPAWN_STMT case in the switch statement in gimplify_expr (). 3.
> No test for catching the suspicious spawned function warning 4.
> Reasoning for expanding the 2 builtin functions in builtins.c instead
> of just inserting the appropriate expanded-code when I am inserting
> the function call.
>
> Did I miss anything else (or misunderstand anything you pointed
> out)?
>
> Here are my answers to those questions above and am attaching a fixed
> patch with the changelog entries:
>
> 1 & 2(partial): There are 3 places where we could have _Cilk_spawn:
> INIT_EXPR, CALL_EXPR and MODIFY_EXPR. INIT_EXPR and MODIFY_EXPRS are
> both gimplified using gimplify_modify_expr. I have moved the
> cilk_detect_spawn into this function. We will go into the
> cilk_detect_spawn if cilk plus is enabled, and if there is a
> cilk_frame (meaning the function has a Cilk_spawn in it) thereby
> reducing the number of hits into this function significantly. Inside
> this function, it will go into the function that has a spawned
> function call and then unwrap the CILK_SPAWN_STMT wrapper and returns
> true. This shouldn't cause a huge compilation time hit. 2. To handle
> CALL_EXPR (e.g. _Cilk_spawn foo (x), where foo returns a void or the
> return value of it is ignored), I have added a CILK_SPAWN_STMT case.
> Again, I am calling the detect_cilk_spawn and we will only step into
> this function if Cilk Plus is enabled and if there is a cilk-frame
> (i.e saying the function has a cilk spawn in it). If there is an
> error (seen_error () == true), then it just falls through into
> CALL_EXPR and is handled like a normal call expr not spawned
> expression. 3. This warning rarely get hit, and I have seen it hit

See my comments below on this.

> only when you are spawning a constructor in C++. To my knowledge, we
> have not had anyone report that they have hit this warning. I just

Ok, leave the warning, but do include a test case.

> kept it in there just in case as a heads-up. 4. The reason why I am
> handling pop_frame and detach functions in builtins.c is because one
> of the things that LTO does is to remove the frame pointer. All Cilk
> functions must use the frame pointer. When LTO is invoked, it is hard
> to see what function is a cilk function and what is not. The CFUN
> flag that says it is a cilk function gets cleared out. But, the
> builtin functions are expanded during LTO phase and I set the
> is_cilk_function flag when it is expanding pop_frame and detach. The
> other option that I was thinking was to have frame pointer on when
> Cilk Plus is enabled, but this is a bit over-kill and can cause
> performance issues.

I think the reason you have to do all these gymnastics is bececause you 
are waiting so late to expand.  You could expand into trees and not have 
to worry about frame pointers and such.  See fold_builtin_* in 
builtins.c.  *However*, if you think you can generate better code by 
delaying expansion all the way until RTL, then I'm fine with your 
current approach.

>
> Also, I had added a couple more tests to catch a couple cases.

> +  /* Implicit _Cilk_sync must be inserted right before any return statement
> +     if there is a _Cilk_spawn in the function (checked by seeing if
> +     cilk_frame_decl is not NULL_TREE).  If the user has provided a
> +     _Cilk_sync, the optimizer should remove this duplicate one.  */
> +  if (flag_enable_cilkplus && cfun->cilk_frame_decl != NULL_TREE)

Again, never document the obvious things your code is doing.  For 
example, you can remove "(checked by seeing if
 > +     cilk_frame_decl is not NULL_TREE)".  It's obvious by looking at 
the code.

> +      /* If there are errors, there is no point in expanding the
> +         _Cilk_spawn.  Just gimplify like a normal CALL_EXPR.  */
> +      && !seen_error ())

Similarly here.  No need to document the obvious.

> +      /* If there are errors, there is no point in expanding the
> +         _Cilk_spawn.  Just gimplify like a normal MODIFY or INIT_EXPR.  */
> +      && !seen_error ())

And here.

Alos, if the canonical way of checking if a function has a cilk_spawn in 
it is always "cfun->cilk_frame_decl != NULL_TREE", then you should 
abstract this into an inline function.  The implementation will be 
trivial to change if we ever decide to keep that information elsewhere. 
  Perhaps something like:

static bool inline
cilk_function_has_spawn (struct function *func)
{
   return func->cilk_frame_decl != NULL_TREE;
}

Do all these hooks you have in lang_hooks_for_cilkplus really have to be 
hooks?  That is, do you need different variants of them for C and for 
C++, or can you make do with one?  Because if you only need one, there's 
no need for a hook.

For example:

> +int
> +gimplify_cilk_sync (tree *expr_p, gimple_seq *pre_p, gimple_seq *post_p
> +		    ATTRIBUTE_UNUSED)
> +{
> +  tree sync_expr = expand_cilk_sync ();
> +  *expr_p = NULL_TREE;
> +  gimplify_and_add (sync_expr, pre_p);
> +  return GS_ALL_DONE;
> +}

The above is a hook.  Will there be a different version of this for C++? 
  If not, just call it directly.  Similarly for gimplify_cilk_spawn(), 
and the rest of the hooks.

> +/* Returns true if *EXP0 is a recognized form of spawn.  Recognized forms are,
> +   after conversion to void, a call expression at outer level or an assignment
> +   at outer level with the right hand side being a spawned call.
> +   Note that `=' in C++ may turn into a CALL_EXPR rather than a MODIFY_EXPR.  */
> +
> +bool
> +cilk_detect_spawn_in_expr (tree *exp0)
> +{

I don't like the name of this function or the corresponding hook 
(cilk_detect_spawn).  You are actually detecting whether the expression 
contains a spawn *and* unwrapping it.  So its use hides the fact that 
you are also modifying the expression in place.

> +  if (flag_enable_cilkplus && cfun->cilk_frame_decl != NULL_TREE
> +      && lang_hooks.cilkplus.cilk_detect_spawn (expr_p)
> +      /* If there are errors, there is no point in expanding the
> +         _Cilk_spawn.  Just gimplify like a normal CALL_EXPR.  */
> +      && !seen_error ())
> +    return (enum gimplify_status)
> +      lang_hooks.cilkplus.gimplify_cilk_spawn (expr_p, pre_p, NULL);

I would much rather have cilk_detect_spawn_and_unwrap() or something. 
Not pretty, but at least it doesn't hide what it's doing.  That is, it's 
not just a check... it can transform the expression.

Also, please document the fact that you are unwrapping and changing the 
expression in the comment to cilk_detect_spawn_in_expr.

> +	case CILK_SPAWN_STMT:
> +	  gcc_assert (flag_enable_cilkplus
> +		      && cfun->cilk_frame_decl != NULL_TREE
> +		      && lang_hooks.cilkplus.cilk_detect_spawn (expr_p));
> +	  if (!seen_error ())
> +	    {
> +	      ret = (enum gimplify_status)
> +		lang_hooks.cilkplus.gimplify_cilk_spawn (expr_p, pre_p,
> +							 post_p);
> +	      break;
> +	    }

Remove the check for flag_enable_cilkplus.  No one really builds that 
but cilk plus, and if for some reason it gets generated, it's pretty 
easy to debug/see why.  See how we handled a slew of other front-end 
specific TREEs (like TRANSACTION_EXPR).  No one ever checks for flag_*.

But BTW, your changes to gimplify.c are much better.  It's a much 
cleaner approach.  Thanks.

> +	case CILK_SYNC_STMT:
> +	  {
> +	    if (!cfun->cilk_frame_decl)
> +	      {
> +		error_at (input_location, "expected %<_Cilk_spawn%> before "
> +			  "%<_Cilk_sync%>");
> +		ret = GS_ERROR;
> +	      }

First, surely you have a location you can use, instead of the generic 
input_location (perhaps the location for the CILK_SYNC_STMT??).  Also, 
Can you not check for this in c_finish_cilk_sync_stmt(), or the 
corresponding code-- that is, in the FE somewhere?  And hopefully, in a 
place you can share with the C++ FE?  If it is a real pain, I am willing 
to let this go, since it happens only in the Cilk code path, though the 
general trend (especially with Andrew's proposed changes) is to do all 
type checking as close to the source as possible.

Aldy
Iyer, Balaji V Aug. 30, 2013, 5:02 p.m. UTC | #9
The email seem to be bouncing gcc-patches. I have gzipped my patch. 

Thanks,

Balaji V. Iyer.


> > -----Original Message-----
> > From: Iyer, Balaji V
> > Sent: Friday, August 30, 2013 11:42 AM
> > To: 'Aldy Hernandez'
> > Cc: rth@redhat.com; Jeff Law; gcc-patches@gcc.gnu.org
> > Subject: RE: [PATCH] Cilk Keywords (_Cilk_spawn and _Cilk_sync) for C
> >
> > Hi Aldy,
> > 	Attached, please find a fixed patch and the changelog entries.
> >
> > > -----Original Message-----
> > > From: Aldy Hernandez [mailto:aldyh@redhat.com]
> > > Sent: Wednesday, August 28, 2013 2:36 PM
> > > To: Iyer, Balaji V
> > > Cc: rth@redhat.com; Jeff Law; gcc-patches@gcc.gnu.org
> > > Subject: Re: [PATCH] Cilk Keywords (_Cilk_spawn and _Cilk_sync) for
> > > C
> > >
> > > On 08/27/13 16:27, Iyer, Balaji V wrote:
> > > > Hello Aldy, I went through all the emails and here are the major
> > > > issues that I could gather (other than lowering the keywords after
> > > > gimplification, which I am skipping since it is more of an
> > > > optimization for now).
> > >
> > > Ok, for now I am fine with delaying handling all this as a gimple
> > > tuple since most of your code lives in it's only little world :).
> > > But I will go on record saying that part of the reason that you have
> > > to handle CALL_EXPR, MODIFY_EXPR, INIT_EXPR and such is because you
> > > don't
> > have easy gimplified code to examine.
> > > Anyways, agreed, you can do this later.
> > >
> > > >
> > > > 1. Calling the gimplify_cilk_spawn on top of the gimplify_expr
> > > > before the switch-statement could slow the compiler down 2. I need
> > > > a CILK_SPAWN_STMT case in the switch statement in gimplify_expr (). 3.
> > > > No test for catching the suspicious spawned function warning 4.
> > > > Reasoning for expanding the 2 builtin functions in builtins.c
> > > > instead of just inserting the appropriate expanded-code when I am
> > > > inserting the function call.
> > > >
> > > > Did I miss anything else (or misunderstand anything you pointed out)?
> > > >
> > > > Here are my answers to those questions above and am attaching a
> > > > fixed patch with the changelog entries:
> > > >
> > > > 1 & 2(partial): There are 3 places where we could have _Cilk_spawn:
> > > > INIT_EXPR, CALL_EXPR and MODIFY_EXPR. INIT_EXPR and MODIFY_EXPRS
> > are
> > > > both gimplified using gimplify_modify_expr. I have moved the
> > > > cilk_detect_spawn into this function. We will go into the
> > > > cilk_detect_spawn if cilk plus is enabled, and if there is a
> > > > cilk_frame (meaning the function has a Cilk_spawn in it) thereby
> > > > reducing the number of hits into this function significantly.
> > > > Inside this function, it will go into the function that has a
> > > > spawned function call and then unwrap the CILK_SPAWN_STMT wrapper
> > > > and returns true. This shouldn't cause a huge compilation time hit. 2.
> > > > To handle CALL_EXPR (e.g. _Cilk_spawn foo (x), where foo returns a
> > > > void or the return value of it is ignored), I have added a
> > > > CILK_SPAWN_STMT
> > case.
> > > > Again, I am calling the detect_cilk_spawn and we will only step
> > > > into this function if Cilk Plus is enabled and if there is a
> > > > cilk-frame (i.e saying the function has a cilk spawn in it). If
> > > > there is an error (seen_error () == true), then it just falls
> > > > through into CALL_EXPR and is handled like a normal call expr not
> > > > spawned expression. 3. This warning rarely get hit, and I have
> > > > seen it hit
> > >
> > > See my comments below on this.
> > >
> > > > only when you are spawning a constructor in C++. To my knowledge,
> > > > we have not had anyone report that they have hit this warning. I
> > > > just
> > >
> > > Ok, leave the warning, but do include a test case.
> > >
> >
> > Ok. Will do when I submit the C++ patch (since constructors are really
> > in C++)
> >
> > > > kept it in there just in case as a heads-up. 4. The reason why I
> > > > am handling pop_frame and detach functions in builtins.c is
> > > > because one of the things that LTO does is to remove the frame
> > > > pointer. All Cilk functions must use the frame pointer. When LTO
> > > > is invoked, it is hard to see what function is a cilk function and
> > > > what is not. The CFUN flag that says it is a cilk function gets
> > > > cleared out. But, the builtin functions are expanded during LTO
> > > > phase and I set the is_cilk_function flag when it is expanding
> > > > pop_frame and detach. The other option that I was thinking was to
> > > > have frame pointer on when Cilk Plus is enabled, but this is a bit
> > > > over-kill and can cause performance
> > issues.
> > >
> > > I think the reason you have to do all these gymnastics is bececause
> > > you are waiting so late to expand.  You could expand into trees and
> > > not have to worry about frame pointers and such.  See fold_builtin_*
> > > in builtins.c.  *However*, if you think you can generate better code
> > > by delaying expansion all the way until RTL, then I'm fine with your
> > > current
> > approach.
> > >
> >
> > Well, one of the things the Cilk ABI requires is the usage of the frame pointer.
> > LTO will remove frame pointer if the backend has
> > FRAME_POINTER_REQUIRED set to zero (which is true in x86 in general)
> regardless where I decompose things.
> >
> > > >
> > > > Also, I had added a couple more tests to catch a couple cases.
> > >
> > > > +  /* Implicit _Cilk_sync must be inserted right before any return statement
> > > > +     if there is a _Cilk_spawn in the function (checked by seeing if
> > > > +     cilk_frame_decl is not NULL_TREE).  If the user has provided a
> > > > +     _Cilk_sync, the optimizer should remove this duplicate one.
> > > > + */ if (flag_enable_cilkplus && cfun->cilk_frame_decl !=
> > > > + NULL_TREE)
> > >
> > > Again, never document the obvious things your code is doing.  For
> > > example, you can remove "(checked by seeing if
> >
> > Fixed.
> >
> > >  > +     cilk_frame_decl is not NULL_TREE)".  It's obvious by looking at
> > > the code.
> > >
> > > > +      /* If there are errors, there is no point in expanding the
> > > > +         _Cilk_spawn.  Just gimplify like a normal CALL_EXPR.  */
> > > > +      && !seen_error ())
> > >
> > > Similarly here.  No need to document the obvious.
> > >
> >
> > Fixed.
> > .
> > > > +      /* If there are errors, there is no point in expanding the
> > > > +         _Cilk_spawn.  Just gimplify like a normal MODIFY or INIT_EXPR.  */
> > > > +      && !seen_error ())
> > >
> > > And here.
> > >
> >
> > Fixed.
> >
> > > Alos, if the canonical way of checking if a function has a
> > > cilk_spawn in it is always "cfun->cilk_frame_decl != NULL_TREE",
> > > then you should abstract this into an inline function.  The
> > > implementation will be trivial to change if we ever decide to keep that
> information elsewhere.
> > >   Perhaps something like:
> > >
> > > static bool inline
> > > cilk_function_has_spawn (struct function *func) {
> > >    return func->cilk_frame_decl != NULL_TREE; }
> > >
> >
> > OK. Fixed.
> >
> > > Do all these hooks you have in lang_hooks_for_cilkplus really have
> > > to be
> > hooks?
> > > That is, do you need different variants of them for C and for
> > > C++, or can you make do with one?  Because if you only need one,
> > > C++there's
> > > no need for a hook.
> > >
> > Well, many of the the things are in language hook struct is because
> > C++ has exceptions and C doesn't. Thus, I need to add exception
> > related stuff into the
> > C++.
> >
> > The code to handle cilk_sync is same for C and C++. The reason why I
> > put it in the struct is for conformity (since spawn and sync go
> > together). I have taken it out of langhooks in the above patch.
> >
> >
> > > For example:
> > >
> > > > +int
> > > > +gimplify_cilk_sync (tree *expr_p, gimple_seq *pre_p, gimple_seq *post_p
> > > > +		    ATTRIBUTE_UNUSED)
> > > > +{
> > > > +  tree sync_expr = expand_cilk_sync ();
> > > > +  *expr_p = NULL_TREE;
> > > > +  gimplify_and_add (sync_expr, pre_p);
> > > > +  return GS_ALL_DONE;
> > > > +}
> > >
> > > The above is a hook.  Will there be a different version of this for C++?
> > >   If not, just call it directly.  Similarly for
> > > gimplify_cilk_spawn(), and the rest of the hooks.
> > >
> > > > +/* Returns true if *EXP0 is a recognized form of spawn.
> > > > +Recognized forms
> > > are,
> > > > +   after conversion to void, a call expression at outer level or an
> assignment
> > > > +   at outer level with the right hand side being a spawned call.
> > > > +   Note that `=' in C++ may turn into a CALL_EXPR rather than a
> > > MODIFY_EXPR.  */
> > > > +
> > > > +bool
> > > > +cilk_detect_spawn_in_expr (tree *exp0) {
> > >
> > > I don't like the name of this function or the corresponding hook
> > > (cilk_detect_spawn).  You are actually detecting whether the
> > > expression contains a spawn *and* unwrapping it.  So its use hides
> > > the fact that you are also modifying the expression in place.
> > >
> > Fixed as you suggested below.
> >
> > > > +  if (flag_enable_cilkplus && cfun->cilk_frame_decl != NULL_TREE
> > > > +      && lang_hooks.cilkplus.cilk_detect_spawn (expr_p)
> > > > +      /* If there are errors, there is no point in expanding the
> > > > +         _Cilk_spawn.  Just gimplify like a normal CALL_EXPR.  */
> > > > +      && !seen_error ())
> > > > +    return (enum gimplify_status)
> > > > +      lang_hooks.cilkplus.gimplify_cilk_spawn (expr_p, pre_p,
> > > > + NULL);
> > >
> > > I would much rather have cilk_detect_spawn_and_unwrap() or something.
> > > Not pretty, but at least it doesn't hide what it's doing.  That is,
> > > it's not just a check... it can transform the expression.
> > >
> > > Also, please document the fact that you are unwrapping and changing
> > > the expression in the comment to cilk_detect_spawn_in_expr.
> > >
> >
> >
> > Fixed.
> >
> > > > +	case CILK_SPAWN_STMT:
> > > > +	  gcc_assert (flag_enable_cilkplus
> > > > +		      && cfun->cilk_frame_decl != NULL_TREE
> > > > +		      && lang_hooks.cilkplus.cilk_detect_spawn (expr_p));
> > > > +	  if (!seen_error ())
> > > > +	    {
> > > > +	      ret = (enum gimplify_status)
> > > > +		lang_hooks.cilkplus.gimplify_cilk_spawn (expr_p, pre_p,
> > > > +							 post_p);
> > > > +	      break;
> > > > +	    }
> > >
> > > Remove the check for flag_enable_cilkplus.  No one really builds
> > > that but cilk plus, and if for some reason it gets generated, it's
> > > pretty easy to debug/see why.  See how we handled a slew of other
> > > front-end specific TREEs (like TRANSACTION_EXPR).  No one ever checks for
> flag_*.
> > >
> >
> > OK. Fixed.
> >
> > > But BTW, your changes to gimplify.c are much better.  It's a much
> > > cleaner approach.  Thanks.
> > >
> >
> > Thanks!
> >
> > > > +	case CILK_SYNC_STMT:
> > > > +	  {
> > > > +	    if (!cfun->cilk_frame_decl)
> > > > +	      {
> > > > +		error_at (input_location, "expected %<_Cilk_spawn%> before "
> > > > +			  "%<_Cilk_sync%>");
> > > > +		ret = GS_ERROR;
> > > > +	      }
> > >
> > > First, surely you have a location you can use, instead of the
> > > generic input_location (perhaps the location for the
> > > CILK_SYNC_STMT??).  Also, Can you not check for this in
> > > c_finish_cilk_sync_stmt(), or the corresponding code-- that is, in
> > > the FE somewhere?  And hopefully, in a place you can share with the
> > > C++ FE?  If it is a real pain, I am willing to let this go, since it
> > > happens only in the Cilk code path, though the general trend
> > > (especially with Andrew's proposed changes) is to do all type checking as
> close to the source as possible.
> >
> > If you look at the codes above (e.g. TRUTH_XOR_EXPR), they all use
> > input_location. Also, in the beginning of the function there is a line  like this:
> >
> > if (save_expr != error_mark_node
> >       && EXPR_HAS_LOCATION (*expr_p))
> >     input_location = EXPR_LOCATION (*expr_p);
> >
> > So, isn't input_location the same value as the location of the *expr_p?
> >
> > For the 2nd point, there can be a case where (with the help of Gotos)
> > _Cilk_sync can come before _Cilk_spawn. So, the only way to do this
> > check is to do it after the entire function is parsed.
> >
> >
> >
> > >
> > > Aldy
Aldy Hernandez Sept. 2, 2013, 1:10 p.m. UTC | #10
>>> +	case CILK_SYNC_STMT: +	  { +	    if (!cfun->cilk_frame_decl) +
>>> { +		error_at (input_location, "expected %<_Cilk_spawn%> before
>>> " +			  "%<_Cilk_sync%>"); +		ret = GS_ERROR; +	      }
>>
>> First, surely you have a location you can use, instead of the
>> generic input_location (perhaps the location for the
>> CILK_SYNC_STMT??).  Also, Can you not check for this in
>> c_finish_cilk_sync_stmt(), or the corresponding code-- that is, in
>> the FE somewhere?  And hopefully, in a place you can share with the
>> C++ FE?  If it is a real pain, I am willing to let this go, since
>> it happens only in the Cilk code path, though the general trend
>> (especially with Andrew's proposed changes) is to do all type
>> checking as close to the source as possible.
>
> If you look at the codes above (e.g. TRUTH_XOR_EXPR), they all use
> input_location. Also, in the beginning of the function there is a
> line  like this:
>
> if (save_expr != error_mark_node && EXPR_HAS_LOCATION (*expr_p))
> input_location = EXPR_LOCATION (*expr_p);

Yes, that is old legacy code.  The present move is to steer away from 
input_location altogether.

> For the 2nd point, there can be a case where (with the help of Gotos)
> _Cilk_sync can come before _Cilk_spawn. So, the only way to do this
> check is to do it after the entire function is parsed.

Fair enough.

Alright, I'm fine with this current incantation.  Thanks for all your 
hard work taking care of the things I've pointed out.

It's now up to one of the core maintainers to take it from here.

Aldy
diff mbox

Patch

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index fb0cb4b..e3da111 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -869,7 +869,7 @@  RTL_ERROR_H = rtl-error.h $(RTL_H) $(DIAGNOSTIC_CORE_H)
 READ_MD_H = $(OBSTACK_H) $(HASHTAB_H) read-md.h
 PARAMS_H = params.h params.def
 BUILTINS_DEF = builtins.def sync-builtins.def omp-builtins.def \
-	gtm-builtins.def sanitizer.def cilkplus.def
+	gtm-builtins.def sanitizer.def cilkplus.def cilk-builtins.def
 INTERNAL_FN_DEF = internal-fn.def
 INTERNAL_FN_H = internal-fn.h $(INTERNAL_FN_DEF)
 TREE_H = coretypes.h tree.h all-tree.def tree.def c-family/c-common.def \
@@ -960,6 +960,7 @@  SCEV_H = tree-scalar-evolution.h $(GGC_H) tree-chrec.h $(PARAMS_H)
 OMEGA_H = omega.h $(PARAMS_H)
 TREE_DATA_REF_H = tree-data-ref.h $(OMEGA_H) graphds.h $(SCEV_H)
 TREE_INLINE_H = tree-inline.h
+CILK_H = cilk.h
 REAL_H = real.h $(MACHMODE_H)
 IRA_INT_H = ira.h ira-int.h $(CFGLOOP_H) alloc-pool.h
 LRA_INT_H = lra.h $(BITMAP_H) $(RECOG_H) $(INSN_ATTR_H) insn-codes.h \
@@ -1153,7 +1154,7 @@  C_COMMON_OBJS = c-family/c-common.o c-family/c-cppbuiltin.o c-family/c-dump.o \
   c-family/c-omp.o c-family/c-opts.o c-family/c-pch.o \
   c-family/c-ppoutput.o c-family/c-pragma.o c-family/c-pretty-print.o \
   c-family/c-semantics.o c-family/c-ada-spec.o tree-mudflap.o \
-  c-family/array-notation-common.o
+  c-family/array-notation-common.o c-family/cilk.o
 
 # Language-independent object files.
 # We put the insn-*.o files first so that a parallel make will build
@@ -1198,6 +1199,7 @@  OBJS = \
 	cgraphbuild.o \
 	cgraphunit.o \
 	cgraphclones.o \
+	cilk-common.o \
 	combine.o \
 	combine-stack-adj.o \
 	compare-elim.o \
@@ -2022,6 +2024,10 @@  c-family/c-ada-spec.o : c-family/c-ada-spec.c c-family/c-ada-spec.h \
 c-family/array-notation-common.o : c-family/array-notation-common.c $(TREE_H) \
 	$(SYSTEM_H) $(TREE_H) coretypes.h tree-iterator.h $(DIAGNOSTIC_CORE_H)
 
+c-family/cilk.o : c-family/cilk.c $(TREE_H) $(SYSTEM_H) $(CONFIG_H) toplev.h \
+	$(TREE_H) coretypes.h tree-iterator.h $(TREE_INLINE_H) $(CGRAPH_H) \
+	$(DIAGNOSTIC_CORE_H) $(GIMPLE_H) $(CILK_H) $(C_COMMON_H) langhooks.h
+
 c-family/stub-objc.o : c-family/stub-objc.c $(CONFIG_H) $(SYSTEM_H) \
 	coretypes.h $(TREE_H) $(C_COMMON_H) c-family/c-objc.h
 
@@ -2541,7 +2547,7 @@  tree-optimize.o : tree-optimize.c $(TREE_FLOW_H) $(CONFIG_H) $(SYSTEM_H) \
    $(TREE_PASS_H) $(CFGLOOP_H) $(EXCEPT_H)
 
 gimplify.o : gimplify.c $(CONFIG_H) $(SYSTEM_H) $(TREE_H) $(GIMPLE_H) \
-   $(DIAGNOSTIC_H) $(TREE_INLINE_H) langhooks.h \
+   $(DIAGNOSTIC_H) $(TREE_INLINE_H) langhooks.h $(CILK_H) \
    $(LANGHOOKS_DEF_H) $(TREE_FLOW_H) $(CGRAPH_H) $(TIMEVAR_H) $(TM_H) \
    coretypes.h $(EXCEPT_H) $(FLAGS_H) $(RTL_H) $(FUNCTION_H) $(EXPR_H) \
    $(GGC_H) gt-gimplify.h $(HASHTAB_H) $(TARGET_H) $(DIAGNOSTIC_CORE_H) $(OPTABS_H) \
@@ -2820,7 +2826,7 @@  builtins.o : builtins.c builtins.h $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
    hard-reg-set.h $(DIAGNOSTIC_CORE_H) hard-reg-set.h $(EXCEPT_H) \
    $(TM_P_H) $(PREDICT_H) $(LIBFUNCS_H) langhooks.h $(BASIC_BLOCK_H) \
    tree-mudflap.h realmpfr.h $(BUILTINS_DEF) $(MACHMODE_H) \
-   $(DIAGNOSTIC_CORE_H) $(TREE_FLOW_H) value-prof.h
+   $(DIAGNOSTIC_CORE_H) $(TREE_FLOW_H) value-prof.h $(CILK_H)
 calls.o : calls.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(RTL_H) \
    $(TREE_H) $(FLAGS_H) $(EXPR_H) $(OPTABS_H) langhooks.h $(TARGET_H) \
    $(LIBFUNCS_H) $(REGS_H) $(DIAGNOSTIC_CORE_H) output.h \
@@ -2917,6 +2923,8 @@  cgraphclones.o : cgraphclones.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
    tree-iterator.h $(COVERAGE_H) \
    $(GIMPLE_PRETTY_PRINT_H) $(IPA_INLINE_H) $(IPA_UTILS_H) \
    $(LTO_STREAMER_H) $(EXCEPT_H) $(GCC_PLUGIN_H) gt-cgraphclones.h
+cilk-common.o : cilk-common.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TREE_H) \
+   langhooks.h $(EXPR_H) $(OPTABS_H) $(RECOG_H) $(CILK_H)
 cgraphbuild.o : cgraphbuild.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
    $(TREE_H) langhooks.h $(CGRAPH_H) intl.h pointer-set.h $(GIMPLE_H) \
    $(TREE_FLOW_H) $(TREE_PASS_H) $(IPA_UTILS_H) $(EXCEPT_H) \
diff --git a/gcc/builtins.c b/gcc/builtins.c
index 78b0d84..b8ab3c7 100644
--- a/gcc/builtins.c
+++ b/gcc/builtins.c
@@ -48,6 +48,7 @@  along with GCC; see the file COPYING3.  If not see
 #include "value-prof.h"
 #include "diagnostic-core.h"
 #include "builtins.h"
+#include "cilk.h"
 
 
 #ifndef PAD_VARARGS_DOWN
@@ -237,6 +238,9 @@  is_builtin_name (const char *name)
     return true;
   if (strncmp (name, "__atomic_", 9) == 0)
     return true;
+  if (flag_enable_cilkplus && (!strcmp (name, "__cilkrts_detach")   
+			       || !strcmp (name, "__cilkrts_pop_frame")))
+    return true;
   return false;
 }
 
@@ -6869,6 +6873,14 @@  expand_builtin (tree exp, rtx target, rtx subtarget, enum machine_mode mode,
       expand_builtin_set_thread_pointer (exp);
       return const0_rtx;
 
+    case BUILT_IN_CILK_DETACH:
+      expand_builtin_cilk_detach (exp);
+      return const0_rtx;
+      
+    case BUILT_IN_CILK_POP_FRAME:
+      expand_builtin_cilk_pop_frame (exp);
+      return const0_rtx;
+
     default:	/* just do library call, if unknown builtin */
       break;
     }
diff --git a/gcc/builtins.def b/gcc/builtins.def
index 9b55b1f..5955d6f 100644
--- a/gcc/builtins.def
+++ b/gcc/builtins.def
@@ -141,6 +141,13 @@  along with GCC; see the file COPYING3.  If not see
                false, true, true, ATTRS, false, \
 	       (flag_openmp || flag_tree_parallelize_loops))
 
+/* Builtin used by implementation of Cilk Plus. Most of these are decomposed
+   by the compiler but a few are implemented in libcilkrts.  */ 
+#undef DEF_CILK_BUILTIN_STUB
+#define DEF_CILK_BUILTIN_STUB(ENUM, NAME) \
+  DEF_BUILTIN (ENUM, NAME, BUILT_IN_NORMAL, BT_LAST, BT_LAST, false, false, \
+	       false, ATTR_LAST, false, false)
+
 /* Builtin used by the implementation of GNU TM.  These
    functions are mapped to the actual implementation of the STM library. */
 #undef DEF_TM_BUILTIN
@@ -836,6 +843,9 @@  DEF_GCC_BUILTIN (BUILT_IN_LINE, "LINE", BT_FN_INT, ATTR_NOTHROW_LEAF_LIST)
 /* OpenMP builtins.  */
 #include "omp-builtins.def"
 
+/* Cilk Keywords builtins.  */
+#include "cilk-builtins.def"
+
 /* GTM builtins. */
 #include "gtm-builtins.def"
 
diff --git a/gcc/c-family/c-common.c b/gcc/c-family/c-common.c
index 7bba376..5e61399 100644
--- a/gcc/c-family/c-common.c
+++ b/gcc/c-family/c-common.c
@@ -403,6 +403,8 @@  const struct c_common_resword c_common_reswords[] =
   { "_Alignof",		RID_ALIGNOF,   D_CONLY },
   { "_Bool",		RID_BOOL,      D_CONLY },
   { "_Complex",		RID_COMPLEX,	0 },
+  { "_Cilk_spawn",      RID_CILK_SPAWN, 0 },
+  { "_Cilk_sync",       RID_CILK_SYNC,  0 },
   { "_Imaginary",	RID_IMAGINARY, D_CONLY },
   { "_Decimal32",       RID_DFLOAT32,  D_CONLY | D_EXT },
   { "_Decimal64",       RID_DFLOAT64,  D_CONLY | D_EXT },
@@ -5176,6 +5178,9 @@  c_define_builtins (tree va_list_ref_type_node, tree va_list_arg_type_node)
 
   if (flag_mudflap)
     mudflap_init ();
+
+  if (flag_enable_cilkplus)
+    cilk_init_builtins ();
 }
 
 /* Like get_identifier, but avoid warnings about null arguments when
@@ -11464,6 +11469,8 @@  c_common_init_ts (void)
   MARK_TS_TYPED (C_MAYBE_CONST_EXPR);
   MARK_TS_TYPED (EXCESS_PRECISION_EXPR);
   MARK_TS_TYPED (ARRAY_NOTATION_REF);
+  MARK_TS_TYPED (CILK_SYNC_STMT);
+  MARK_TS_TYPED (CILK_SPAWN_STMT);
 }
 
 /* Build a user-defined numeric literal out of an integer constant type VALUE
diff --git a/gcc/c-family/c-common.h b/gcc/c-family/c-common.h
index dc430c3..ab77fa4 100644
--- a/gcc/c-family/c-common.h
+++ b/gcc/c-family/c-common.h
@@ -148,6 +148,9 @@  enum rid
   /* C++11 */
   RID_CONSTEXPR, RID_DECLTYPE, RID_NOEXCEPT, RID_NULLPTR, RID_STATIC_ASSERT,
 
+  /* Cilk Plus Keywords.  */
+  RID_CILK_SPAWN, RID_CILK_SYNC,
+  
   /* Objective-C ("AT" reserved words - they are only keywords when
      they follow '@')  */
   RID_AT_ENCODE,   RID_AT_END,
@@ -1206,4 +1209,13 @@  extern void cilkplus_extract_an_triplets (vec<tree, va_gc> *, size_t, size_t,
 					  vec<vec<an_parts> > *);
 extern vec <tree, va_gc> *fix_sec_implicit_args
   (location_t, vec <tree, va_gc> *, vec<an_loop_parts>, size_t, tree);
+
+/* In cilk.c.  */
+extern tree insert_cilk_frame (tree);
+extern void cilk_init_builtins (void);
+extern int gimplify_cilk_spawn (tree *, gimple_seq *, gimple_seq *);
+extern int gimplify_cilk_sync (tree *, gimple_seq *, gimple_seq *);
+extern void c_cilk_install_body_w_frame_cleanup (tree, tree);
+extern bool cilk_valid_spawn (tree *);
+extern bool cilkplus_set_spawn_marker (location_t, tree);
 #endif /* ! GCC_C_COMMON_H */
diff --git a/gcc/c-family/cilk.c b/gcc/c-family/cilk.c
new file mode 100644
index 0000000..cc8e9fa
--- /dev/null
+++ b/gcc/c-family/cilk.c
@@ -0,0 +1,1467 @@ 
+/* This file is part of the Intel(R) Cilk(TM) Plus support
+   This file contains the CilkPlus Intrinsics
+   Copyright (C) 2013  Free Software Foundation, Inc.
+   Contributed by Balaji V. Iyer <balaji.v.iyer@intel.com>,
+   Intel Corporation
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 3, or (at your option)
+any later version.
+
+GCC is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3.  If not see
+<http://www.gnu.org/licenses/>.  */
+
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+#include "tree.h"
+#include "langhooks.h"
+#include "gimple.h"
+#include "tree-iterator.h"
+#include "tree-inline.h"
+#include "c-family/c-common.h"
+#include "toplev.h" 
+#include "cgraph.h"
+#include "diagnostic.h"
+#include "cilk.h"
+
+enum add_variable_type {
+    /* Reference to previously-defined variable.  */
+    ADD_READ,
+    /* Definition of a new variable in inner-scope.  */
+    ADD_BIND,
+    /* Write to possibly previously-defined variable.  */
+    ADD_WRITE
+};
+
+enum cilk_block_type {
+    /* Indicates a _Cilk_spawn block.  30 was an arbitary number picked for 
+       ease of debugging.  */
+    CILK_BLOCK_SPAWN = 30,
+    /* Indicates _Cilk_for statement block.  */
+    CILK_BLOCK_FOR
+};
+
+struct wrapper_data
+{
+  /* Kind of function to be created. */
+  enum cilk_block_type type;
+  /* Signature of helper function.  */
+  tree fntype;
+  /* Containing function.  */
+  tree context;
+  /* Disposition of all variables in the inner statement.  */
+  struct pointer_map_t *decl_map;
+  /* True if this function needs a static chain.  */
+  bool nested;
+  /* Arguments to be passed to wrapper function, currently a list. */
+  tree arglist;
+  /* Argument types, a list.  */
+  tree argtypes;
+  /* Incoming parameters.  */
+  tree parms;
+  /* Outer BLOCK object.  */
+  tree block;
+};
+
+static void extract_free_variables (tree, struct wrapper_data *,
+				    enum add_variable_type);
+static HOST_WIDE_INT cilk_wrapper_count;
+
+/* Marks the CALL_EXPR, FCALL, as a spawned function call and the current
+   function as a spawner.  Emit error if the function call is outside a
+   function or if a non function-call is spawned.  */
+
+bool
+cilkplus_set_spawn_marker (location_t loc, tree fcall)
+{
+  if (!current_function_decl)
+    { 
+      error_at (loc, "_Cilk_spawn may only be used inside a function");
+      return false;
+    }
+  else if (fcall == error_mark_node)
+    /* Error reporting here is not necessary here since if FCALL is an
+       error_mark_node, the function marking it as error would have reported
+       it.  */
+    return false; 
+  else if (TREE_CODE (fcall) != CALL_EXPR)
+    { 
+      error_at (loc, "only function calls can be spawned");
+      return false;
+    }
+  else
+    {
+      cfun->calls_cilk_spawn = true;
+      return true;
+    }
+}
+
+/* Returns a setjmp CALL_EXPR with FRAME->context as its parameter.  */
+
+tree
+cilk_call_setjmp (tree frame)
+{
+  tree c;
+
+  c = cilk_dot (frame, CILK_TI_FRAME_CONTEXT, false);
+  c = build1 (ADDR_EXPR, build_pointer_type (ptr_type_node), c);
+  return build_call_expr (builtin_decl_implicit (BUILT_IN_SETJMP), 1, c);
+}
+
+/* This function will expand a cilk_sync call.  */
+
+static tree
+build_cilk_sync (void)
+{
+  tree frame = cfun->cilk_frame_decl;
+
+  /* Cilk_sync is converted to the following code:
+
+     sf.pedigree = sf.worker->pedigree;
+     if (frame.flags & CILK_FRAME_UNSYNCHED)
+     {
+        __cilkrts_save_fp_csw (&sf);
+        if (!builtin_setjmp (sf.ctx) 
+	    __cilkrts_sync(&sf); 
+	else 
+	   if (sf.flags & CILK_FRAME_EXCEPTING) 
+	     __cilkrts_rethrow (&sf); 
+      }
+      sf.worker->pedigree.rank = sf.worker->pedigree.rank + 1;  */
+
+  tree flags = cilk_dot (frame, CILK_TI_FRAME_FLAGS, false);
+  
+  tree unsynched = fold_build2 (BIT_AND_EXPR, TREE_TYPE (flags), flags,
+				build_int_cst (TREE_TYPE (flags),
+					       CILK_FRAME_UNSYNCHED));
+
+  unsynched = fold_build2 (NE_EXPR, TREE_TYPE (unsynched), unsynched,
+			   build_int_cst (TREE_TYPE (unsynched), 0));
+
+  tree frame_addr = build1 (ADDR_EXPR, cilk_frame_ptr_type_decl, frame);
+
+  /* Check if exception (0x10) bit is set in the sf->flags.  */
+  tree except_flag = fold_build2 (BIT_AND_EXPR, TREE_TYPE (flags), flags,
+				  build_int_cst (TREE_TYPE (flags),
+						 CILK_FRAME_EXCEPTING));
+  except_flag = fold_build2 (NE_EXPR, TREE_TYPE (except_flag), except_flag,
+			     build_int_cst (TREE_TYPE (except_flag), 0));
+
+  /* If the exception flag is set then call the __cilkrts_rethrow (&sf).  */
+  tree except_cond = fold_build3 (COND_EXPR, void_type_node, except_flag,
+				  build_call_expr (cilk_rethrow_fndecl, 1,
+						   frame_addr),
+				  build_empty_stmt (EXPR_LOCATION (unsynched)));
+  
+  tree sync_expr = build_call_expr (cilk_sync_fndecl, 1, frame_addr);
+  tree setjmp_expr = cilk_call_setjmp (frame);
+  setjmp_expr = fold_build2 (EQ_EXPR, TREE_TYPE (setjmp_expr), setjmp_expr,
+			     build_int_cst (TREE_TYPE (setjmp_expr), 0));
+  
+  setjmp_expr = fold_build3 (COND_EXPR, void_type_node, setjmp_expr,
+			     sync_expr, except_cond);
+  tree sync_list = alloc_stmt_list ();
+  append_to_statement_list (build_call_expr (cilk_save_fp_fndecl, 1,
+					     frame_addr), &sync_list);
+  append_to_statement_list (setjmp_expr, &sync_list);
+  tree sync = fold_build3 (COND_EXPR, void_type_node, unsynched, sync_list,
+			   build_empty_stmt (EXPR_LOCATION (unsynched)));
+  tree parent_pedigree = cilk_dot (frame, CILK_TI_FRAME_PEDIGREE, false);
+  tree worker = cilk_dot (frame, CILK_TI_FRAME_WORKER, false);
+  tree worker_pedigree = cilk_arrow (worker, CILK_TI_WORKER_PEDIGREE, false);
+  tree assign_pedigree = fold_build2 (MODIFY_EXPR, void_type_node,
+				      parent_pedigree, worker_pedigree);
+  tree w_ped_rank = cilk_dot (unshare_expr (worker_pedigree), 
+			      CILK_TI_PEDIGREE_RANK, false);
+  tree incr_ped_rank = fold_build2 (PLUS_EXPR, TREE_TYPE (w_ped_rank),
+				    w_ped_rank,
+				    build_one_cst (TREE_TYPE (w_ped_rank)));
+  incr_ped_rank = fold_build2 (MODIFY_EXPR, void_type_node, w_ped_rank,
+			       incr_ped_rank);
+  tree ret_sync_exp = alloc_stmt_list ();
+  append_to_statement_list (assign_pedigree, &ret_sync_exp);
+  append_to_statement_list (sync, &ret_sync_exp);
+  append_to_statement_list (incr_ped_rank, &ret_sync_exp);
+  return ret_sync_exp;
+}
+
+/* This function will output the exit conditions for a spawn call.  */
+
+tree
+build_cilk_function_exit (tree frame, bool detaches, bool needs_sync)
+{
+  tree sync_expr = NULL_TREE;
+
+  tree epi = alloc_stmt_list ();
+
+  if (needs_sync)
+    {
+      sync_expr = build_cilk_sync ();
+      append_to_statement_list (sync_expr, &epi);
+    }
+  
+  tree func_ptr = build1 (ADDR_EXPR, cilk_frame_ptr_type_decl, frame);
+  tree pop_frame = build_call_expr (cilk_pop_fndecl, 1, func_ptr);
+  tree worker = cilk_dot (frame, CILK_TI_FRAME_WORKER, 0);
+  tree current = cilk_arrow (worker, CILK_TI_WORKER_CUR, 0);
+  tree parent = cilk_dot (frame, CILK_TI_FRAME_PARENT, 0);
+  tree set_current = build2 (MODIFY_EXPR, void_type_node, current, parent);
+  append_to_statement_list (set_current, &epi);
+  append_to_statement_list (pop_frame, &epi);
+  tree call = build_call_expr (cilk_leave_fndecl, 1, func_ptr);
+  if (!detaches)
+    {
+      tree flags_cmp_expr = NULL_TREE;
+      tree flags = cilk_dot (frame, CILK_TI_FRAME_FLAGS, false);
+      flags_cmp_expr = fold_build2 (NE_EXPR, TREE_TYPE (flags), flags,
+				    build_int_cst (TREE_TYPE (flags),
+						   CILK_FRAME_VERSION));
+      call = fold_build3 (COND_EXPR, void_type_node, flags_cmp_expr,
+			  call, build_empty_stmt (EXPR_LOCATION (flags)));
+    }
+  append_to_statement_list (call, &epi);  
+  return epi;
+}
+
+/* Gimplifies the cilk_sync expression passed in *EXPR_P.  Returns GS_ALL_DONE 
+   when finished.  */
+
+int
+gimplify_cilk_sync (tree *expr_p, gimple_seq *pre_p, gimple_seq *post_p
+		    ATTRIBUTE_UNUSED)
+{
+  tree sync_expr = build_cilk_sync ();
+  *expr_p = NULL_TREE;
+  gimplify_and_add (sync_expr, pre_p);
+  return GS_ALL_DONE;
+}
+
+/* Trying to get the correct cfun for the FUNCTION_DECL indicated by OUTER.  */
+
+static void
+pop_cfun_to (tree outer)
+{
+  pop_cfun ();
+  current_function_decl = outer;
+  gcc_assert (cfun == DECL_STRUCT_FUNCTION (current_function_decl));
+  gcc_assert (cfun->decl == current_function_decl);
+}
+
+/* This function does whatever is necessary to make the compiler emit a newly 
+   generated function, FNDECL.  */
+
+static void
+call_graph_add_fn (tree fndecl)
+{
+  const tree outer = current_function_decl;
+  struct function *f = DECL_STRUCT_FUNCTION (fndecl);
+
+  gcc_assert (TREE_CODE (fndecl) == FUNCTION_DECL);
+
+  f->is_cilk_function = 1;
+  f->curr_properties = cfun->curr_properties;
+  gcc_assert (cfun == DECL_STRUCT_FUNCTION (outer)); 
+  gcc_assert (cfun->decl == outer);
+
+  push_cfun (f); 
+
+  cgraph_add_new_function (fndecl, false);
+  cgraph_finalize_function (fndecl, true); 
+
+  pop_cfun_to (outer);
+}
+
+/* Return true if this is a tree which is allowed to contain a spawn as 
+   operand 0.
+   A spawn call may be wrapped in a series of unary operations such
+   as conversions.  These conversions need not be "useless"
+   to be disregarded because they are retained in the spawned
+   statement.  They are bypassed only to look for a spawn
+   within.
+   A comparison to constant is simple enough to allow, and
+   is used to convert to bool.  */
+
+static bool
+cilk_ignorable_spawn_rhs_op (tree exp)
+{
+  enum tree_code code = TREE_CODE (exp);
+  switch (TREE_CODE_CLASS (code))
+    {
+    case tcc_expression:
+      return code == ADDR_EXPR;
+    case tcc_comparison:
+      /* We need the spawn as operand 0 for now.   That's where it
+	 appears in the only case we really care about, conversion
+	 to bool.  */
+      return (TREE_CODE (TREE_OPERAND (exp, 1)) == INTEGER_CST);
+    case tcc_unary:
+    case tcc_reference:
+      return true;
+    default:
+      return false;
+    }
+}
+
+/* Helper function for walk_tree.  If *TP is a CILK_SPAWN_STMT, then unwrap
+   this "wrapper" and  *WALK_SUBTREES is set to 0.  The function returns 
+   NULL_TREE regardless.  */
+
+static tree
+unwrap_cilk_sync_stmt (tree *tp, int *walk_subtrees, void *)
+{
+  if (TREE_CODE (*tp) == CILK_SPAWN_STMT)
+    {
+      *tp = CILK_SPAWN_FN (*tp);
+      *walk_subtrees = 0;
+    }
+  return NULL_TREE;
+}
+
+/* This function checks to see if the constructor in EXP can be spawnable.  */
+
+static bool
+cilk_spawnable_constructor (tree exp)
+{
+  if (TREE_CODE (exp) != ADDR_EXPR)
+    return false;
+  exp = TREE_OPERAND (exp, 0);
+  if (TREE_CODE (exp) != FUNCTION_DECL)
+    return false;
+  if (DECL_BUILT_IN_CLASS (exp) == BUILT_IN_NORMAL)
+    return DECL_FUNCTION_CODE (exp) == BUILT_IN_MEMCPY;
+  return lang_hooks.cilkplus.spawnable_constructor (exp);
+}
+
+/* Returns true when EXP is a CALL_EXPR with _Cilk_spawn in front.  Unwraps
+   CILK_SPAWN_STMT wrapper from the CALL_EXPR in *EXP0 statement.  */
+
+static bool
+recognize_spawn (tree exp, tree *exp0)
+{
+  bool spawn_found = false;
+  if (TREE_CODE (exp) == CILK_SPAWN_STMT)
+    {
+      /* Remove the CALL_EXPR from CILK_SPAWN_STMT wrapper and return true.  */
+      exp = CILK_SPAWN_FN (exp);
+      walk_tree (exp0, unwrap_cilk_sync_stmt, NULL, NULL);
+      spawn_found = true;
+    }
+  else
+    {
+      if (TREE_CODE (exp) != CALL_EXPR && TREE_CODE (exp) != TARGET_EXPR) 
+	spawn_found = lang_hooks.cilkplus.recognize_spawn (exp);
+      else
+	return false;
+    }
+  return spawn_found;
+}
+
+/* Returns true if *EXP0 is a recognized form of spawn.  Recognized forms are,
+   after conversion to void, a call expression at outer level or an assignment
+   at outer level with the right hand side being a spawned call.
+   Note that `=' in C++ may turn into a CALL_EXPR rather than a MODIFY_EXPR.  */
+
+bool
+cilk_valid_spawn (tree *exp0)
+{
+  tree exp = *exp0;
+  bool warn;
+
+  if (!TREE_SIDE_EFFECTS (exp))
+    return false;
+
+  /* If the function contains no Cilk code, this isn't a spawn.  */
+  if (!cfun->cilk_frame_decl)
+    return false;
+
+  /* Strip off any conversion to void.  It does not affect whether spawn 
+     is supported here.  */
+  if (TREE_CODE (exp) == CONVERT_EXPR && VOID_TYPE_P (TREE_TYPE (exp)))
+    exp = TREE_OPERAND (exp, 0);
+
+  if (TREE_CODE (exp) == MODIFY_EXPR || TREE_CODE (exp) == INIT_EXPR)
+    exp = TREE_OPERAND (exp, 1);
+
+  while (cilk_ignorable_spawn_rhs_op (exp))
+    exp = TREE_OPERAND (exp, 0);
+
+  if (TREE_CODE (exp) == TARGET_EXPR)
+    if (TARGET_EXPR_INITIAL (exp)
+	&& TREE_CODE (TARGET_EXPR_INITIAL (exp)) != AGGR_INIT_EXPR)
+      exp = TARGET_EXPR_INITIAL (exp);
+
+  /* Happens with C++ TARGET_EXPR.  */
+  if (exp == NULL_TREE)
+    return false; 
+
+  while (TREE_CODE (exp) == CLEANUP_POINT_EXPR || TREE_CODE (exp) == EXPR_STMT)
+    exp = TREE_OPERAND (exp, 0);
+  
+  /* Now we have a call, or this isn't a valid spawn.  This will reject any 
+     outer non-spawn AGGR_INIT_EXPR that is valid because of a spawn inside.  */
+  if (recognize_spawn (exp, exp0))
+    return true;
+
+  if (TREE_CODE (exp) != CALL_EXPR)
+    return false;
+
+  /* This may be a call that is not a spawn itself but contains a spawn.
+     In that case, the call should be a constructor.
+
+     x = spawn f();
+
+     may expand to
+
+     (call operator= (&var1, (convert &(target var2 (aggr_init/spawn ...))))
+
+     operator= may be a function or a call to __builtin_memcpy (which
+     will have one more argument, the size).
+
+     What we specifically support is the address of the value
+     initialized by a spawning AGGR_INIT_EXPR being passed as
+     the second argument to a function.  */
+
+  warn = !cilk_spawnable_constructor (CALL_EXPR_FN (exp));
+
+  /* The function address of a call may not be computed via a spawn.
+     Look at the arglist only, and only the second argument which
+     is the RHS of any plausible assignment or copy.  The first
+     argument is the LHS.  A third argument could be a size for
+     memcpy.  This path supports op= in addition to =, only because
+     it is easy to do so. */
+  if (call_expr_nargs (exp) < 2)
+    return false;
+
+  exp = CALL_EXPR_ARG (exp, 0);
+
+  STRIP_USELESS_TYPE_CONVERSION (exp);
+
+  if (TREE_CODE (exp) == ADDR_EXPR)
+    exp = TREE_OPERAND (exp, 0);
+
+  if (TREE_CODE (exp) == TARGET_EXPR)
+    exp = TARGET_EXPR_INITIAL (exp);
+
+  if (!exp || !recognize_spawn (exp, exp0))
+    return false;
+
+  if (warn) 
+    warning (0, "suspicious use of _Cilk_spawn");
+  return true;
+}
+
+/* This function will return a FNDECL using information from *WD.  */
+
+static tree
+build_cilk_helper_decl (struct wrapper_data *wd)
+{
+  char name[20];
+  if (wd->type == CILK_BLOCK_FOR)
+    sprintf (name, "_cilk_for_%ld", cilk_wrapper_count++);
+  else if (wd->type == CILK_BLOCK_SPAWN)
+    sprintf (name, "_cilk_spn_%ld", cilk_wrapper_count++);
+  else
+    gcc_unreachable (); 
+  
+  clean_symbol_name (name);
+  tree fndecl = build_decl (UNKNOWN_LOCATION, FUNCTION_DECL, 
+			    get_identifier (name), wd->fntype);
+
+  TREE_PUBLIC (fndecl) = 0;
+  TREE_STATIC (fndecl) = 1;
+  TREE_USED (fndecl) = 1;
+  DECL_ARTIFICIAL (fndecl) = 0;
+  DECL_IGNORED_P (fndecl) = 0;
+  DECL_EXTERNAL (fndecl) = 0;
+
+  if (wd->nested) 
+    DECL_CONTEXT (fndecl) = wd->context;
+  else 
+    /* In C++, copying the outer function's context makes the loop 
+       function appear like a static member function.  */ 
+    DECL_CONTEXT (fndecl) = DECL_CONTEXT (wd->context);
+
+  tree block = make_node (BLOCK);
+  DECL_INITIAL (fndecl) = block;
+  TREE_USED (block) = 1;
+  gcc_assert (!DECL_SAVED_TREE (fndecl));
+
+  /* Inlining would defeat the purpose of this wrapper.
+     Either it secretly switches stack frames or it allocates
+     a stable stack frame to hold function arguments even if
+     the parent stack frame is stolen.  */
+  DECL_UNINLINABLE (fndecl) = 1;
+
+  tree result_decl = build_decl (UNKNOWN_LOCATION, RESULT_DECL, NULL_TREE, 
+				 void_type_node);
+  DECL_ARTIFICIAL (result_decl) = 0;
+  DECL_IGNORED_P (result_decl) = 1;
+  DECL_CONTEXT (result_decl) = fndecl;
+  DECL_RESULT (fndecl) = result_decl;
+  
+  return fndecl;
+}
+
+/* A function used by walk tree to find wrapper parms.  */
+
+static bool
+wrapper_parm_cb (const void *key0, void **val0, void *data)
+{
+  struct wrapper_data *wd = (struct wrapper_data *) data;
+  tree arg = * (tree *)&key0;
+  tree val = (tree)*val0;
+  tree parm;
+
+  if (val == error_mark_node || val == arg)
+    return true;
+
+  if (TREE_CODE (val) == PAREN_EXPR)
+    {
+      /* We should not reach here with a register receiver.
+	 We may see a register variable modified in the
+	 argument list.  Because register variables are
+	 worker-local we don't need to work hard to support
+	 them in code that spawns. */
+      if ((TREE_CODE (arg) == VAR_DECL) && DECL_HARD_REGISTER (arg))
+	{
+	  error_at (EXPR_LOCATION (arg),
+		    "explicit register variable %qD may not be modified in "
+		    "spawn", arg);
+	  arg = null_pointer_node;
+	}
+      else
+	arg = build1 (ADDR_EXPR, build_pointer_type (TREE_TYPE (arg)), arg);
+	
+      val = TREE_OPERAND (val, 0);
+      *val0 = val;
+      gcc_assert (TREE_CODE (val) == INDIRECT_REF);
+      parm = TREE_OPERAND (val, 0);
+      STRIP_NOPS (parm);
+    }
+  else
+    parm = val;
+  TREE_CHAIN (parm) = wd->parms;
+  wd->parms = parm;
+  wd->argtypes = tree_cons (NULL_TREE, TREE_TYPE (parm), wd->argtypes); 
+  wd->arglist = tree_cons (NULL_TREE, arg, wd->arglist); 
+  return true;
+}
+
+/* This function is used to build a wrapper of a certain type.  */
+
+static void
+build_wrapper_type (struct wrapper_data *wd)
+{
+  wd->arglist = NULL_TREE;
+  wd->parms = NULL_TREE;
+  wd->argtypes = void_list_node;
+
+  pointer_map_traverse (wd->decl_map, wrapper_parm_cb, wd);
+  gcc_assert (wd->type != CILK_BLOCK_FOR);
+
+  /* Now build a function.
+     Its return type is void (all side effects are via explicit parameters).
+     Its parameters are WRAPPER_PARMS with type WRAPPER_TYPES.
+     Actual arguments in the caller are WRAPPER_ARGS.  */
+  wd->fntype = build_function_type (void_type_node, wd->argtypes);
+}
+
+/* This function checks all the CALL_EXPRs in *TP found by cilk_outline.  */
+
+static tree
+check_outlined_calls (tree *tp, int *walk_subtrees ATTRIBUTE_UNUSED, 
+		      void *data)
+{
+  bool *throws = (bool *) data;
+  tree t = *tp;
+  int flags;
+
+  if (TREE_CODE (t) != CALL_EXPR)
+    return 0;
+  flags = call_expr_flags (t);
+
+  if (!(flags & ECF_NOTHROW) && flag_exceptions)
+    *throws = true;
+  if (flags & ECF_RETURNS_TWICE)
+    error_at (EXPR_LOCATION (t), 
+	      "cannot spawn call to function that returns twice");
+  return 0;
+}
+
+/* Each DECL in the source code (spawned statement) is passed to this function
+   once.  Each instance of the DECL is replaced with the result of this 
+   function.
+
+   The parameters of the wrapper should have been entered into the map already.
+   This function only deals with variables with scope limited to the 
+   spawned expression.  */
+
+static tree
+copy_decl_for_cilk (tree decl, copy_body_data *id)
+{
+  switch (TREE_CODE (decl))
+    {
+    case VAR_DECL:
+      return copy_decl_no_change (decl, id);
+
+    case LABEL_DECL:
+      error_at (EXPR_LOCATION (decl), "invalid use of label %q+D in spawn", 
+		decl);
+      return error_mark_node;
+
+    case RESULT_DECL:
+    case PARM_DECL:
+      /* RESULT_DECL and PARM_DECL has already been entered into the map.  */
+    default:
+      gcc_unreachable ();
+      return error_mark_node;
+    }
+}
+
+/* Copy all local variables.  */
+
+static bool
+for_local_cb (const void *k_v, void **vp, void *p)
+{
+  tree k = *(tree *) &k_v;
+  tree v = (tree) *vp;
+
+  if (v == error_mark_node)
+    *vp = copy_decl_no_change (k, (copy_body_data *) p);
+  return true;
+}
+
+/* Copy all local declarations from a _Cilk_spawned function's body.  */
+
+static bool
+wrapper_local_cb (const void *k_v, void **vp, void *data)
+{
+  copy_body_data *id = (copy_body_data *) data;
+  tree key = *(tree *) &k_v;
+  tree val = (tree) *vp;
+
+  if (val == error_mark_node)
+    *vp = copy_decl_for_cilk (key, id);
+
+  return true;
+}
+
+/* Alter a tree STMT from OUTER_FN to form the body of INNER_FN.  */
+
+static void
+cilk_outline (tree inner_fn, tree *stmt_p, struct wrapper_data *wd)
+{
+  const tree outer_fn = wd->context;	      
+  const bool nested = (wd->type == CILK_BLOCK_FOR);
+  copy_body_data id;
+  bool throws;
+
+  DECL_STATIC_CHAIN (outer_fn) = 1;
+
+  memset (&id, 0, sizeof (id));
+  /* Copy FROM the function containing the spawn...  */
+  id.src_fn = outer_fn;
+
+  /* ...TO the wrapper.  */
+  id.dst_fn = inner_fn; 
+  id.src_cfun = DECL_STRUCT_FUNCTION (outer_fn);
+
+  /* There shall be no RETURN in spawn.  */
+  id.retvar = 0; 
+  id.decl_map = wd->decl_map;
+  id.copy_decl = nested ? copy_decl_no_change : copy_decl_for_cilk;
+  id.block = DECL_INITIAL (inner_fn);
+  id.transform_lang_insert_block = NULL;
+
+  id.transform_new_cfg = true;
+  id.transform_call_graph_edges = CB_CGE_MOVE;
+  id.remap_var_for_cilk = true;
+  id.regimplify = true; /* unused? */
+
+  insert_decl_map (&id, wd->block, DECL_INITIAL (inner_fn));
+
+  /* We don't want the private variables any more.  */
+  pointer_map_traverse (wd->decl_map, nested ? for_local_cb : wrapper_local_cb,
+			&id);
+
+  walk_tree (stmt_p, copy_tree_body_r, &id, NULL);
+
+  /* See if this function can throw or calls something that should
+     not be spawned.  The exception part is only necessary if
+     flag_exceptions && !flag_non_call_exceptions.  */
+  throws = false ;
+  (void) walk_tree_without_duplicates (stmt_p, check_outlined_calls, &throws);
+}
+
+/* Generate the body of a wrapper function that assigns the
+   result of the expression RHS into RECEIVER.  RECEIVER must
+   be NULL if this is not a spawn -- the wrapper will return
+   a value.  If this is a spawn, the wrapper will return void.  */
+
+static tree
+build_cilk_wrapper_body (tree stmt, struct wrapper_data *wd)
+{
+  const tree outer = current_function_decl;
+  tree fndecl;
+  tree p;
+
+   /* Build the type of the wrapper and its argument list from the
+     variables that it requires.  */
+  build_wrapper_type (wd);
+
+  /* Emit a function that takes WRAPPER_PARMS incoming and applies ARGS 
+     (modified) to the wrapped function.  Return the wrapper and modified ARGS 
+     to the caller to generate a function call.  */
+  fndecl = build_cilk_helper_decl (wd);
+  push_struct_function (fndecl);
+  if (wd->nested && (wd->type == CILK_BLOCK_FOR))
+    {
+      gcc_assert (TREE_VALUE (wd->arglist) == NULL_TREE);
+      TREE_VALUE (wd->arglist) = build2 (FDESC_EXPR, ptr_type_node ,
+					 fndecl, integer_one_node);
+    }
+  DECL_ARGUMENTS (fndecl) = wd->parms;
+
+  for (p = wd->parms; p; p = TREE_CHAIN (p))
+    DECL_CONTEXT (p) = fndecl;
+
+  cilk_outline (fndecl, &stmt, wd);
+  stmt = fold_build_cleanup_point_expr (void_type_node, stmt);
+  gcc_assert (!DECL_SAVED_TREE (fndecl));
+  lang_hooks.cilkplus.install_body_with_frame_cleanup (fndecl, stmt);
+  gcc_assert (DECL_SAVED_TREE (fndecl));
+
+  pop_cfun_to (outer);
+
+  /* Recognize the new function.  */
+  call_graph_add_fn (fndecl);
+  return fndecl;
+}
+
+/* Initializes the wrapper data structure.  */
+
+static void
+init_wd (struct wrapper_data *wd, enum cilk_block_type type)
+{
+  wd->type = type;
+  wd->fntype = NULL_TREE;
+  wd->context = current_function_decl;
+  wd->decl_map = pointer_map_create ();
+  /* _Cilk_for bodies are always nested.  Others start off as 
+     normal functions.  */
+  wd->nested = (type == CILK_BLOCK_FOR);
+  wd->arglist = NULL_TREE;
+  wd->argtypes = NULL_TREE;
+  wd->block = NULL_TREE;
+}
+
+/* Clears the wrapper data structure.  */
+
+static void
+free_wd (struct wrapper_data *wd)
+{
+  pointer_map_destroy (wd->decl_map);
+  wd->nested = false;
+  wd->arglist = NULL_TREE;
+  wd->argtypes = NULL_TREE;
+  wd->parms = NULL_TREE;
+}
+
+
+ /* Given a variable in an expression to be extracted into
+   a helper function, declare the helper function parameter
+   to receive it.
+
+   On entry the value of the (key, value) pair may be
+
+   (*, error_mark_node) -- Variable is private to helper function,
+   do nothing.
+
+   (var, var) -- Reference to outer scope (function or global scope).
+
+   (var, integer 0) -- Capture by value, save newly-declared PARM_DECL
+   for value in value slot.
+
+   (var, integer 1) -- Capture by reference, declare pointer to type
+   as new PARM_DECL and store (spawn_stmt (indirect_ref (parm)).
+   
+   (var, ???) -- Pure output argument, handled similarly to above.
+*/
+
+static bool
+declare_one_free_variable (const void *var0, void **map0,
+			   void *data ATTRIBUTE_UNUSED)
+{
+  const_tree var = (const_tree) var0;
+  tree map = (tree)*map0;
+  tree var_type = TREE_TYPE (var), arg_type;
+  bool by_reference;
+  tree parm;
+
+  gcc_assert (DECL_P (var));
+
+  /* Ignore truly local variables.  */
+  if (map == error_mark_node)
+    return true;
+  /* Ignore references to the parent function.  */
+  if (map == var)
+    return true;
+
+  gcc_assert (TREE_CODE (map) == INTEGER_CST);
+
+  /* A value is passed by reference if:
+
+     1. It is addressable, so that a copy may not be made.
+     2. It is modified in the spawned statement.
+     In the future this function may want to arrange
+     a warning if the spawned statement is a loop body
+     because an output argument would indicate a race.
+     Note: Earlier passes must have marked the variable addressable.
+     3. It is expensive to copy.  */
+  by_reference =
+    (TREE_ADDRESSABLE (var_type)
+     /* Arrays must be passed by reference.  This is required for C
+	semantics -- arrays are not first class objects.  Other
+	aggregate types can and should be passed by reference if
+	they are not passed to the spawned function.  We aren't yet
+	distinguishing safe uses in argument calculation from unsafe
+	uses as outgoing function arguments, so we make a copy to
+	stabilize the value.  */
+     || TREE_CODE (var_type) == ARRAY_TYPE
+     || (tree) map == integer_one_node);
+
+  if (by_reference)
+    var_type = build_qualified_type (build_pointer_type (var_type),
+				     TYPE_QUAL_RESTRICT);
+  gcc_assert (!TREE_ADDRESSABLE (var_type));
+
+  /* Maybe promote to int.  */
+  if (INTEGRAL_TYPE_P (var_type) && COMPLETE_TYPE_P (var_type)
+      && INT_CST_LT_UNSIGNED (TYPE_SIZE (var_type),
+			      TYPE_SIZE (integer_type_node)))
+    arg_type = integer_type_node;
+  else
+    arg_type = var_type;
+
+  parm = build_decl (UNKNOWN_LOCATION, PARM_DECL, NULL_TREE, var_type);
+  DECL_ARG_TYPE (parm) = arg_type;
+  DECL_ARTIFICIAL (parm) = 0;
+  TREE_READONLY (parm) = 1;
+  
+  if (by_reference)
+    {
+      parm = build1 (INDIRECT_REF, TREE_TYPE (var_type), parm);
+      parm = build1 (PAREN_EXPR, void_type_node, parm);
+    }
+  *map0 = parm;
+  return true;
+}
+ 
+/* Returns a wrapper function for a _Cilk_spawn.  */
+
+static tree
+build_cilk_wrapper (tree exp, tree *args_out)
+{
+  struct wrapper_data wd;
+  tree fndecl;
+
+  init_wd (&wd, CILK_BLOCK_SPAWN);
+
+  if (TREE_CODE (exp) == CONVERT_EXPR)
+    exp = TREE_OPERAND (exp, 0);
+  
+  /* Special handling for top level INIT_EXPR.  Usually INIT_EXPR means the 
+     variable is defined in the spawned expression and can be private to the 
+     spawn helper.  A top level INIT_EXPR defines a variable to be initialized 
+     by spawn and the variable must remain in the outer function. */
+  if (TREE_CODE (exp) == INIT_EXPR)
+    {
+      extract_free_variables (TREE_OPERAND (exp, 0), &wd, ADD_WRITE);
+      extract_free_variables (TREE_OPERAND (exp, 1), &wd, ADD_READ);
+      /* TREE_TYPE should be void.  Be defensive.  */
+      if (TREE_TYPE (exp) != void_type_node)
+	extract_free_variables (TREE_TYPE (exp), &wd, ADD_READ);
+    }
+  else
+    extract_free_variables (exp, &wd, ADD_READ);
+  pointer_map_traverse (wd.decl_map, declare_one_free_variable, &wd);
+  wd.block = TREE_BLOCK (exp);
+  if (!wd.block)
+    wd.block = DECL_INITIAL (current_function_decl);
+
+  /* Now fvars maps the old variable to incoming variable.  Update
+     the expression and arguments to refer to the new names.  */
+  fndecl = build_cilk_wrapper_body (exp, &wd);
+  *args_out = wd.arglist;
+  
+  free_wd (&wd);
+
+  return fndecl;
+}
+
+/* Transform *SPAWN_P, a spawned CALL_EXPR, to gimple.  *SPAWN_P can be a
+   CALL_EXPR, INIT_EXPR or MODIFY_EXPR.  Returns GS_OK if everything is fine,
+   and GS_UNHANDLED, otherwise.  */
+
+int
+gimplify_cilk_spawn (tree *spawn_p, gimple_seq *before ATTRIBUTE_UNUSED,
+		     gimple_seq *after ATTRIBUTE_UNUSED)
+{
+  tree expr = *spawn_p;
+  tree function, call1, call2, new_args;
+  tree ii_args = NULL_TREE;
+  int total_args = 0, ii = 0;
+  tree *arg_array;
+  tree setjmp_cond_expr = NULL_TREE;
+  tree setjmp_expr, spawn_expr, setjmp_value = NULL_TREE;
+
+  cfun->calls_cilk_spawn = 1;
+  cfun->is_cilk_function = 1;
+
+  gcc_assert (flag_enable_cilkplus);
+
+  /* Remove CLEANUP_POINT_EXPR and EXPR_STMT from *spawn_p.  */
+  while (TREE_CODE (expr) == CLEANUP_POINT_EXPR
+	 || TREE_CODE (expr) == EXPR_STMT)
+    expr = TREE_OPERAND (expr, 0);
+  
+  new_args = NULL;
+  function = build_cilk_wrapper (expr, &new_args);
+
+  /* This should give the number of parameters.  */
+  total_args = list_length (new_args);
+  arg_array = XNEWVEC (tree, total_args);
+
+  ii_args = new_args;
+  for (ii = 0; ii < total_args; ii++)
+    {
+      arg_array[ii] = TREE_VALUE (ii_args);
+      ii_args = TREE_CHAIN (ii_args);
+    }
+  
+  TREE_USED (function) = 1;
+  rest_of_decl_compilation (function, 0, 0);
+
+  call1 = cilk_call_setjmp (cfun->cilk_frame_decl);
+
+  if (*arg_array == NULL_TREE)
+    call2 = build_call_expr (function, 0);
+  else 
+    call2 = build_call_expr_loc_array (EXPR_LOCATION (*spawn_p), function, 
+					 total_args, arg_array);
+  *spawn_p = alloc_stmt_list ();
+  gcc_assert (cfun->cilk_frame_decl != NULL_TREE);
+
+  tree frame_ptr =
+    build1 (ADDR_EXPR, build_pointer_type (TREE_TYPE (cfun->cilk_frame_decl)),
+	    cfun->cilk_frame_decl);
+  tree save_fp = build_call_expr (cilk_save_fp_fndecl, 1, frame_ptr);
+  append_to_statement_list (save_fp, spawn_p);		  
+  setjmp_value = create_tmp_var (TREE_TYPE (call1), NULL);
+  setjmp_expr = fold_build2 (MODIFY_EXPR, void_type_node, setjmp_value, call1);
+
+  append_to_statement_list_force (setjmp_expr, spawn_p);
+  
+  setjmp_cond_expr = fold_build2 (EQ_EXPR, TREE_TYPE (call1), setjmp_value,
+				  build_int_cst (TREE_TYPE (call1), 0));
+  spawn_expr = fold_build3 (COND_EXPR, void_type_node, setjmp_cond_expr,
+			    call2, build_empty_stmt (EXPR_LOCATION (call1)));
+  append_to_statement_list (spawn_expr, spawn_p);
+
+  return GS_OK;
+}
+
+/* Make the frames necessary for a spawn call.  */
+
+static tree
+make_cilk_frame (tree fn)
+{
+  struct function *f = DECL_STRUCT_FUNCTION (fn);
+  tree decl;
+
+  if (f->cilk_frame_decl)
+    return f->cilk_frame_decl;
+
+  decl = build_decl (EXPR_LOCATION (fn), VAR_DECL, NULL_TREE, 
+		     cilk_frame_type_decl);
+  DECL_CONTEXT (decl) = fn;
+  DECL_SEEN_IN_BIND_EXPR_P (decl) = 1;
+  f->cilk_frame_decl = decl;
+  return decl;
+}
+
+/* Creates the internal functions for spawn helper and parent.  */
+
+/* Inserts "cleanup" functions after the function-body of FNDECL.  FNDECL is a 
+   spawn-helper and BODY is the newly created body for FNDECL.  */
+
+void
+c_cilk_install_body_w_frame_cleanup (tree fndecl, tree body)
+{
+  tree list = alloc_stmt_list ();
+  tree frame = make_cilk_frame (fndecl);
+  tree dtor = build_cilk_function_exit (frame, false, false);
+  add_local_decl (cfun, frame);
+  
+  DECL_SAVED_TREE (fndecl) = list;
+  tree body_list = alloc_stmt_list ();
+  tree frame_ptr = build1 (ADDR_EXPR, build_pointer_type (TREE_TYPE (frame)), 
+			   frame); 
+  tree enter_frame = build_call_expr (cilk_enter_fast_fndecl, 1, frame_ptr); 
+  append_to_statement_list (enter_frame, &body_list);
+  
+  tree parent = cilk_arrow (frame_ptr, CILK_TI_FRAME_PARENT, 0);
+  tree worker = cilk_arrow (frame_ptr, CILK_TI_FRAME_WORKER, 0);
+
+  tree pedigree = cilk_arrow (frame_ptr, CILK_TI_FRAME_PEDIGREE, 0);
+  tree pedigree_rank = cilk_dot (pedigree, CILK_TI_PEDIGREE_RANK, 0);
+  tree parent_pedigree = cilk_dot (pedigree, CILK_TI_PEDIGREE_PARENT, 0);
+  tree pedigree_parent = cilk_arrow (parent, CILK_TI_FRAME_PEDIGREE, 0);
+  tree pedigree_parent_rank = cilk_dot (pedigree_parent, 
+					CILK_TI_PEDIGREE_RANK, 0);
+  tree pedigree_parent_parent = cilk_dot (pedigree_parent, 
+				     CILK_TI_PEDIGREE_PARENT, 0);
+  tree worker_pedigree = cilk_arrow (worker, CILK_TI_WORKER_PEDIGREE, 1);
+  tree w_pedigree_rank = cilk_dot (worker_pedigree, CILK_TI_PEDIGREE_RANK, 0);
+  tree w_pedigree_parent = cilk_dot (worker_pedigree, 
+				     CILK_TI_PEDIGREE_PARENT, 0);
+
+  /* sf.pedigree.rank = worker->pedigree.rank.  */
+  tree exp1 = build2 (MODIFY_EXPR, void_type_node, pedigree_rank,
+		     w_pedigree_rank);
+  append_to_statement_list (exp1, &body_list);
+
+  /* sf.pedigree.parent = worker->pedigree.parent.  */
+  exp1 = build2 (MODIFY_EXPR, void_type_node, parent_pedigree,
+		 w_pedigree_parent);
+  append_to_statement_list (exp1, &body_list);
+
+  /* sf.call_parent->pedigree.rank = worker->pedigree.rank.  */
+  exp1 = build2 (MODIFY_EXPR, void_type_node, pedigree_parent_rank,
+		 w_pedigree_rank);
+  append_to_statement_list (exp1, &body_list);
+
+  /* sf.call_parent->pedigree.parent = worker->pedigree.parent.  */
+  exp1 = build2 (MODIFY_EXPR, void_type_node, pedigree_parent_parent,
+		 w_pedigree_parent);
+  append_to_statement_list (exp1, &body_list);
+
+  /* sf->worker.pedigree.rank = 0.  */
+  exp1 = build2 (MODIFY_EXPR, void_type_node, w_pedigree_rank, 
+		 build_zero_cst (uint64_type_node));
+  append_to_statement_list (exp1, &body_list);
+
+  /* sf->pedigree.parent = &sf->pedigree.  */
+  exp1 = build2 (MODIFY_EXPR, void_type_node, w_pedigree_parent,
+		 build1 (ADDR_EXPR,
+			 build_pointer_type (cilk_pedigree_type_decl),
+			 pedigree));
+  append_to_statement_list (exp1, &body_list);
+
+  tree detach_expr = build_call_expr (cilk_detach_fndecl, 1, frame_ptr); 
+  append_to_statement_list (detach_expr, &body_list);
+  append_to_statement_list (body, &body_list);
+  append_to_statement_list (build_stmt (EXPR_LOCATION (body), TRY_FINALLY_EXPR,
+				       	body_list, dtor), &list);
+}
+
+/* Add a new variable, VAR to a variable list in WD->DECL_MAP.  HOW indicates
+   whether the variable is previously defined, currently defined, or a variable 
+   that is being written to.  */
+
+static void
+add_variable (struct wrapper_data *wd, tree var, enum add_variable_type how)
+{
+  void **valp;
+  
+  valp = pointer_map_contains (wd->decl_map, (void *) var);
+  if (valp)
+    {
+      tree val = (tree) *valp;
+      /* If the variable is local, do nothing.  */
+      if (val == error_mark_node)
+	return;
+      /* If the variable was entered with itself as value,
+	 meaning it belongs to an outer scope, do not alter
+	 the value.  */
+      if (val == var) 
+	return;
+      /* A statement expression may cause a variable to be
+	 bound twice, once in BIND_EXPR and again in a
+	 DECL_EXPR.  That case caused a return in the 
+	 test above.  Any other duplicate definition is
+	 an error.  */
+      gcc_assert (how != ADD_BIND);
+      if (how != ADD_WRITE)
+	return;
+      /* This variable might have been entered as read but is now written.  */
+      *valp = (void *) var;
+      wd->nested = true;
+      return;
+    }
+  else
+    {
+      tree val = NULL_TREE;
+
+      /* Nested function rewriting silently discards hard register
+	 assignments for function scope variables, and they wouldn't
+	 work anyway.  Warn here.  This misses one case: if the
+	 register variable is used as the loop bound or increment it
+	 has already been added to the map.  */
+      if ((how != ADD_BIND) && (TREE_CODE (var) == VAR_DECL)
+	  && !DECL_EXTERNAL (var) && DECL_HARD_REGISTER (var))
+	warning (0, "register assignment ignored for %qD used in Cilk block",
+		 var);
+
+      switch (how)
+	{
+	  /* ADD_BIND means always make a fresh new variable.  */
+	case ADD_BIND:
+	  val = error_mark_node;
+	  break;
+	  /* ADD_READ means
+	     1. For cilk_for, refer to the outer scope definition as-is
+	     2. For a spawned block, take a scalar in an rgument
+	     and otherwise refer to the outer scope definition as-is.
+	     3. For a spawned call, take a scalar in an argument.  */
+	case ADD_READ:
+	  switch (wd->type)
+	    {
+	    case CILK_BLOCK_FOR:
+	      val = var;
+	      break;
+	    case CILK_BLOCK_SPAWN:
+	      if (TREE_ADDRESSABLE (var))
+		{
+		  val = var;
+		  wd->nested = true;
+		  break;
+		}
+	      val = integer_zero_node;
+	      break;
+	    }
+	  break;
+	case ADD_WRITE:
+	  switch (wd->type)
+	    {
+	    case CILK_BLOCK_FOR:
+	      val = var;
+	      wd->nested = true;
+	      break;
+	    case CILK_BLOCK_SPAWN:
+	      if (TREE_ADDRESSABLE (var))
+		val = integer_one_node;
+	      else
+		{
+		  val = var;
+		  wd->nested = true;
+		}
+	      break;
+	    }
+	}
+      *pointer_map_insert (wd->decl_map, (void *) var) = val;
+    }
+}
+
+/* Find the variables referenced in an expression T.  This does not avoid 
+   duplicates because a variable may be read in one context and written in 
+   another.  HOW describes the context in which the reference is seen.  If 
+   NESTED is true a nested function is being generated and variables in the 
+   original context should not be remapped.  */
+
+static void
+extract_free_variables (tree t, struct wrapper_data *wd,
+			enum add_variable_type how)
+{
+#define SUBTREE(EXP)  extract_free_variables (EXP, wd, ADD_READ)
+#define MODIFIED(EXP) extract_free_variables (EXP, wd, ADD_WRITE)
+#define INITIALIZED(EXP) extract_free_variables (EXP, wd, ADD_BIND)
+  
+  if (t == NULL_TREE)
+    return;
+
+  enum tree_code code = TREE_CODE (t);
+  bool is_expr = IS_EXPR_CODE_CLASS (TREE_CODE_CLASS (code));
+
+  if (is_expr)
+    SUBTREE (TREE_TYPE (t));
+
+  switch (code)
+    {
+    case ERROR_MARK:
+    case IDENTIFIER_NODE:
+    case INTEGER_CST:
+    case REAL_CST:
+    case FIXED_CST:
+    case STRING_CST:
+    case BLOCK:
+    case PLACEHOLDER_EXPR:
+    case FIELD_DECL:
+    case VOID_TYPE:
+    case REAL_TYPE:
+      /* These do not contain variable references.  */
+      return;
+
+    case SSA_NAME:
+      /* Currently we don't see SSA_NAME.  */
+      extract_free_variables (SSA_NAME_VAR (t), wd, how);
+      return;
+
+    case LABEL_DECL:
+      /* This might be a reference to a label outside the Cilk block,
+	 which is an error, or a reference to a label in the Cilk block
+	 that we haven't seen yet.  We can't tell.  Ignore it.  An
+	 invalid use will cause an error later in copy_decl_for_cilk.  */
+      return;
+
+    case RESULT_DECL:
+      if (wd->type != CILK_BLOCK_SPAWN)
+	TREE_ADDRESSABLE (t) = 1;
+    case VAR_DECL:
+    case PARM_DECL:
+      if (!TREE_STATIC (t) && !DECL_EXTERNAL (t))
+	add_variable (wd, t, how);
+      return;
+
+    case NON_LVALUE_EXPR:
+    case CONVERT_EXPR:
+    case NOP_EXPR:
+      SUBTREE (TREE_OPERAND (t, 0));
+      return;
+
+    case INIT_EXPR:
+      INITIALIZED (TREE_OPERAND (t, 0));
+      SUBTREE (TREE_OPERAND (t, 1));
+      return;
+
+    case MODIFY_EXPR:
+    case PREDECREMENT_EXPR:
+    case PREINCREMENT_EXPR:
+    case POSTDECREMENT_EXPR:
+    case POSTINCREMENT_EXPR:
+      /* These write their result.  */
+      MODIFIED (TREE_OPERAND (t, 0));
+      SUBTREE (TREE_OPERAND (t, 1));
+      return;
+
+    case ADDR_EXPR:
+      /* This might modify its argument, and the value needs to be
+	 passed by reference in any case to preserve identity and
+	 type if is a promoting type.  In the case of a nested loop
+	 just notice that we touch the variable.  It will already
+	 be addressable, and marking it modified will cause a spurious
+	 warning about writing the control variable.  */
+      if (wd->type != CILK_BLOCK_SPAWN)
+	SUBTREE (TREE_OPERAND (t, 0));
+      else
+	MODIFIED (TREE_OPERAND (t, 0));
+      return;
+
+    case ARRAY_REF:
+      /* Treating ARRAY_REF and BIT_FIELD_REF identically may
+	 mark the array as written but the end result is correct
+	 because the array is passed by pointer anyway.  */
+    case BIT_FIELD_REF:
+      /* Propagate the access type to the object part of which
+	 is being accessed here.  As for ADDR_EXPR, don't do this
+	 in a nested loop, unless the access is to a fixed index.  */
+      if (wd->type != CILK_BLOCK_FOR || TREE_CONSTANT (TREE_OPERAND (t, 1)))
+	extract_free_variables (TREE_OPERAND (t, 0), wd, how);
+      else
+	SUBTREE (TREE_OPERAND (t, 0));
+      SUBTREE (TREE_OPERAND (t, 1));
+      SUBTREE (TREE_OPERAND (t, 2));
+      return;
+
+    case TREE_LIST:
+      SUBTREE (TREE_PURPOSE (t));
+      SUBTREE (TREE_VALUE (t));
+      SUBTREE (TREE_CHAIN (t));
+      return;
+
+    case TREE_VEC:
+      {
+	int len = TREE_VEC_LENGTH (t);
+	int i;
+	for (i = 0; i < len; i++)
+	  SUBTREE (TREE_VEC_ELT (t, i));
+	return;
+      }
+
+    case VECTOR_CST:
+      {
+	unsigned ii = 0;
+	for (ii = 0; ii < VECTOR_CST_NELTS (t); ii++)
+	  SUBTREE (VECTOR_CST_ELT (t, ii)); 
+	break;
+      }
+
+    case COMPLEX_CST:
+      SUBTREE (TREE_REALPART (t));
+      SUBTREE (TREE_IMAGPART (t));
+      return;
+
+    case BIND_EXPR:
+      {
+	tree decl;
+	for (decl = BIND_EXPR_VARS (t); decl; decl = TREE_CHAIN (decl))
+	  {
+	    add_variable (wd, decl, ADD_BIND);
+	    /* A self-referential initialization is no problem because
+	       we already entered the variable into the map as local.  */
+	    SUBTREE (DECL_INITIAL (decl));
+	    SUBTREE (DECL_SIZE (decl));
+	    SUBTREE (DECL_SIZE_UNIT (decl));
+	  }
+	SUBTREE (BIND_EXPR_BODY (t));
+	return;
+      }
+
+    case STATEMENT_LIST:
+      {
+	tree_stmt_iterator i;
+	for (i = tsi_start (t); !tsi_end_p (i); tsi_next (&i))
+	  SUBTREE (*tsi_stmt_ptr (i));
+	return;
+      }
+
+    case TARGET_EXPR:
+      {
+	INITIALIZED (TREE_OPERAND (t, 0));
+	SUBTREE (TREE_OPERAND (t, 1));
+	SUBTREE (TREE_OPERAND (t, 2));
+	if (TREE_OPERAND (t, 3) != TREE_OPERAND (t, 1))
+	  SUBTREE (TREE_OPERAND (t, 3));
+	return;
+      }
+
+    case RETURN_EXPR:
+      if (TREE_NO_WARNING (t))
+	{
+	  gcc_assert (errorcount);
+	  return;
+	}
+      return;
+
+    case DECL_EXPR:
+      if (TREE_CODE (DECL_EXPR_DECL (t)) != TYPE_DECL)
+	INITIALIZED (DECL_EXPR_DECL (t));
+      return;
+
+    case INTEGER_TYPE:
+    case ENUMERAL_TYPE:
+    case BOOLEAN_TYPE:
+      SUBTREE (TYPE_MIN_VALUE (t));
+      SUBTREE (TYPE_MAX_VALUE (t));
+      return;
+
+    case POINTER_TYPE:
+      SUBTREE (TREE_TYPE (t));
+      break;
+
+    case ARRAY_TYPE:
+      SUBTREE (TREE_TYPE (t));
+      SUBTREE (TYPE_DOMAIN (t));
+      return;
+
+    case RECORD_TYPE:
+      SUBTREE (TYPE_FIELDS (t));
+      return;
+    
+    case METHOD_TYPE:
+      SUBTREE (TYPE_ARG_TYPES (t));
+      SUBTREE (TYPE_METHOD_BASETYPE (t));
+      return;
+
+    case AGGR_INIT_EXPR:
+    case CALL_EXPR:
+      {
+	int len = 0;
+	int ii = 0;
+	if (TREE_CODE (TREE_OPERAND (t, 0)) == INTEGER_CST)
+	  {
+	    len = TREE_INT_CST_LOW (TREE_OPERAND (t, 0));
+
+	    for (ii = 0; ii < len; ii++)
+	      SUBTREE (TREE_OPERAND (t, ii));
+	    SUBTREE (TREE_TYPE (t));
+	  }
+	break;
+      }
+
+    default:
+      if (is_expr)
+	{
+	  int i, len;
+
+	  /* Walk over all the sub-trees of this operand.  */
+	  len = TREE_CODE_LENGTH (code);
+
+	  /* Go through the subtrees.  We need to do this in forward order so
+	     that the scope of a FOR_EXPR is handled properly.  */
+	  for (i = 0; i < len; ++i)
+	    SUBTREE (TREE_OPERAND (t, i));
+	}
+    }
+}
+
+
+/* Add appropriate frames needed for a Cilk spawned function call, FNDECL. 
+   Returns the __cilkrts_stack_frame * variable.  */
+
+tree
+insert_cilk_frame (tree fndecl)
+{
+  tree addr, body, enter, out, orig_body;
+  location_t loc = EXPR_LOCATION (fndecl);
+
+  if (!cfun || cfun->decl != fndecl)
+    push_cfun (DECL_STRUCT_FUNCTION (fndecl)); 
+
+  tree decl = cfun->cilk_frame_decl;
+  if (!decl)
+    {
+      tree *saved_tree = &DECL_SAVED_TREE (fndecl);
+      decl = make_cilk_frame (fndecl);
+      add_local_decl (cfun, decl);
+
+      addr = build1 (ADDR_EXPR, cilk_frame_ptr_type_decl, decl);
+      enter = build_call_expr (cilk_enter_fndecl, 1, addr);
+      out = build_cilk_function_exit (cfun->cilk_frame_decl, false, true);
+
+      /* The new body will be:
+	 __cilkrts_enter_frame_1 (&sf);
+	 try {
+	    orig_body;
+	 } 
+	 finally {
+	     __cilkrts_pop_frame (&sf);
+	     __cilkrts_leave_frame (&sf);
+         }  */
+
+      body = alloc_stmt_list ();
+      orig_body = *saved_tree;
+
+      if (TREE_CODE (orig_body) == BIND_EXPR)
+	orig_body = BIND_EXPR_BODY (orig_body);
+ 
+      append_to_statement_list (enter, &body);
+      append_to_statement_list (build_stmt (loc, TRY_FINALLY_EXPR, orig_body, 
+					    out), &body);
+      if (TREE_CODE (*saved_tree) == BIND_EXPR)
+	BIND_EXPR_BODY (*saved_tree) = body;
+      else
+	*saved_tree = body;
+    }
+  return decl;
+}
diff --git a/gcc/c/c-decl.c b/gcc/c/c-decl.c
index f7ae648..ffd62c6 100644
--- a/gcc/c/c-decl.c
+++ b/gcc/c/c-decl.c
@@ -8380,6 +8380,12 @@  finish_function (void)
   /* Tie off the statement tree for this function.  */
   DECL_SAVED_TREE (fndecl) = pop_stmt_list (DECL_SAVED_TREE (fndecl));
 
+  /* IF the function has _Cilk_spawn in front of a function call inside it
+     i.e. it is a spawning function, then add the appropriate Cilk plus
+     functions inside.  */
+  if (flag_enable_cilkplus && cfun->calls_cilk_spawn == 1)
+    cfun->cilk_frame_decl = insert_cilk_frame (fndecl);
+
   finish_fname_decls ();
 
   /* Complain if there's just no return statement.  */
diff --git a/gcc/c/c-objc-common.h b/gcc/c/c-objc-common.h
index e144824..2f0b59e 100644
--- a/gcc/c/c-objc-common.h
+++ b/gcc/c/c-objc-common.h
@@ -105,4 +105,15 @@  along with GCC; see the file COPYING3.  If not see
 #undef LANG_HOOKS_TREE_INLINING_VAR_MOD_TYPE_P
 #define LANG_HOOKS_TREE_INLINING_VAR_MOD_TYPE_P c_vla_unspec_p
 
+#undef  LANG_HOOKS_CILKPLUS_GIMPLIFY_SPAWN
+#define LANG_HOOKS_CILKPLUS_GIMPLIFY_SPAWN gimplify_cilk_spawn
+
+#undef  LANG_HOOKS_CILKPLUS_GIMPLIFY_SYNC
+#define LANG_HOOKS_CILKPLUS_GIMPLIFY_SYNC gimplify_cilk_sync
+
+#undef  LANG_HOOKS_CILKPLUS_FRAME_CLEANUP
+#define LANG_HOOKS_CILKPLUS_FRAME_CLEANUP c_cilk_install_body_w_frame_cleanup
+
+#undef  LANG_HOOKS_CILKPLUS_VALID_SPAWN
+#define LANG_HOOKS_CILKPLUS_VALID_SPAWN cilk_valid_spawn
 #endif /* GCC_C_OBJC_COMMON */
diff --git a/gcc/c/c-parser.c b/gcc/c/c-parser.c
index b612e29..7941002 100644
--- a/gcc/c/c-parser.c
+++ b/gcc/c/c-parser.c
@@ -4497,6 +4497,14 @@  c_parser_statement_after_labels (c_parser *parser)
 	case RID_FOR:
 	  c_parser_for_statement (parser);
 	  break;
+	case RID_CILK_SYNC:
+	  c_parser_consume_token (parser);
+	  c_parser_skip_until_found (parser, CPP_SEMICOLON, "expected %<;%>");
+	  if (!flag_enable_cilkplus) 
+	    error_at (loc, "-fcilkplus must be enabled to use _Cilk_sync");
+	  else 
+	    add_stmt (c_build_cilk_sync ());
+	  break;
 	case RID_GOTO:
 	  c_parser_consume_token (parser);
 	  if (c_parser_next_token_is (parser, CPP_NAME))
@@ -7046,6 +7054,29 @@  c_parser_postfix_expression (c_parser *parser)
 	case RID_GENERIC:
 	  expr = c_parser_generic_selection (parser);
 	  break;
+	case RID_CILK_SPAWN:
+	  c_parser_consume_token (parser);
+	  if (!flag_enable_cilkplus)
+	    {
+	      error_at (loc, "-fcilkplus must be enabled to use _Cilk_spawn");
+	      expr = c_parser_postfix_expression (parser);
+	      expr.value = error_mark_node;	      
+	    }
+	  if (c_parser_peek_token (parser)->keyword == RID_CILK_SPAWN)
+	    {
+	      error_at (input_location, "consecutive _Cilk_spawn keywords "
+			"are not permitted");
+	      /* Now flush out all the _Cilk_spawns.  */
+	      while (c_parser_peek_token (parser)->keyword == RID_CILK_SPAWN)
+		c_parser_consume_token (parser);
+	      expr = c_parser_postfix_expression (parser);
+	    }
+	  else
+	    {
+	      expr = c_parser_postfix_expression (parser);
+	      expr.value = c_build_cilk_spawn (input_location, expr.value);
+	    }
+	  break; 
 	default:
 	  c_parser_error (parser, "expected expression");
 	  expr.value = error_mark_node;
diff --git a/gcc/c/c-tree.h b/gcc/c/c-tree.h
index d1a871d..bb6f052 100644
--- a/gcc/c/c-tree.h
+++ b/gcc/c/c-tree.h
@@ -640,6 +640,8 @@  extern tree c_finish_omp_task (location_t, tree, tree);
 extern tree c_finish_omp_clauses (tree);
 extern tree c_build_va_arg (location_t, tree, tree);
 extern tree c_finish_transaction (location_t, tree, int);
+extern tree c_build_cilk_sync (void);
+extern tree c_build_cilk_spawn (location_t, tree);
 
 /* Set to 0 at beginning of a function definition, set to 1 if
    a return statement that specifies a return value is seen.  */
diff --git a/gcc/c/c-typeck.c b/gcc/c/c-typeck.c
index 30871db..ab53a5c 100644
--- a/gcc/c/c-typeck.c
+++ b/gcc/c/c-typeck.c
@@ -4375,6 +4375,14 @@  build_compound_expr (location_t loc, tree expr1, tree expr2)
   tree eptype = NULL_TREE;
   tree ret;
 
+  if (flag_enable_cilkplus
+      && (TREE_CODE (expr1) == CILK_SPAWN_STMT
+	  || TREE_CODE (expr2) == CILK_SPAWN_STMT))
+    {
+      error_at (loc,
+		"spawned function call cannot be part of a comma expression");
+      return error_mark_node;
+    }
   expr1_int_operands = EXPR_INT_CONST_OPERANDS (expr1);
   if (expr1_int_operands)
     expr1 = remove_c_maybe_const_expr (expr1);
@@ -8682,6 +8690,11 @@  c_finish_return (location_t loc, tree retval, tree origtype)
 	  return error_mark_node;
 	}
     }
+  if (flag_enable_cilkplus && retval && TREE_CODE (retval) == CILK_SPAWN_STMT)
+    {
+      error_at (loc, "use of _Cilk_spawn in return statement is not allowed");
+      return error_mark_node;
+    }
   if (retval)
     {
       tree semantic_type = NULL_TREE;
@@ -10972,3 +10985,26 @@  c_build_va_arg (location_t loc, tree expr, tree type)
 		"C++ requires promoted type, not enum type, in %<va_arg%>");
   return build_va_arg (loc, expr, type);
 }
+
+/* Marks CALL, a CALL_EXPR, as a spawned function call.  */
+
+tree
+c_build_cilk_spawn (location_t loc, tree call)
+{
+  if (!cilkplus_set_spawn_marker (loc, call))
+    return error_mark_node;
+  tree spawn_stmt = build1 (CILK_SPAWN_STMT, TREE_TYPE (call), call);
+  TREE_SIDE_EFFECTS (spawn_stmt) = 1;
+  return spawn_stmt;
+}
+
+/* Returns a tree of type CILK_SYNC_STMT if Cilk Plus is enabled.  Otherwise
+   return error_mark_node.  */
+
+tree
+c_build_cilk_sync (void)
+{
+  tree sync = build0 (CILK_SYNC_STMT, void_type_node);
+  TREE_SIDE_EFFECTS (sync) = 1;
+  return sync;
+}
diff --git a/gcc/cilk-builtins.def b/gcc/cilk-builtins.def
new file mode 100644
index 0000000..8634194
--- /dev/null
+++ b/gcc/cilk-builtins.def
@@ -0,0 +1,33 @@ 
+/* This file contains the definitions and documentation for the
+   Cilk Plus builtins used in the GNU compiler.
+   Copyright (C) 2013 Free Software Foundation, Inc.
+
+   Contributed by Balaji V. Iyer <balaji.v.iyer@intel.com>
+   	          Intel Corporation.
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3.  If not see
+<http://www.gnu.org/licenses/>.  */
+
+DEF_CILK_BUILTIN_STUB (BUILT_IN_CILK_ENTER_FRAME, "__cilkrts_enter_frame_1")
+DEF_CILK_BUILTIN_STUB (BUILT_IN_CILK_ENTER_FRAME_FAST, 
+		       "__cilkrts_enter_frame_fast_1")
+DEF_CILK_BUILTIN_STUB (BUILT_IN_CILK_DETACH, "__cilkrts_detach")
+DEF_CILK_BUILTIN_STUB (BUILT_IN_CILK_RETHROW, "__cilkrts_rethrow")
+DEF_CILK_BUILTIN_STUB (BUILT_IN_CILK_SYNCHED, "__cilkrts_synched")
+DEF_CILK_BUILTIN_STUB (BUILT_IN_CILK_SYNC, "__cilkrts_sync")
+DEF_CILK_BUILTIN_STUB (BUILT_IN_CILK_LEAVE_FRAME, "__cilkrts_leave_frame")
+DEF_CILK_BUILTIN_STUB (BUILT_IN_CILK_POP_FRAME, "__cilkrts_pop_frame")
+DEF_CILK_BUILTIN_STUB (BUILT_IN_CILK_SAVE_FP, "__cilkrts_save_fp_ctrl_state")
diff --git a/gcc/cilk-common.c b/gcc/cilk-common.c
new file mode 100644
index 0000000..ae39525
--- /dev/null
+++ b/gcc/cilk-common.c
@@ -0,0 +1,390 @@ 
+/* This file is part of the Intel(R) Cilk(TM) Plus support
+   This file contains the CilkPlus Intrinsics
+   Copyright (C) 2013  Free Software Foundation, Inc.
+   Contributed by Balaji V. Iyer <balaji.v.iyer@intel.com>,
+   Intel Corporation
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 3, or (at your option)
+any later version.
+
+GCC is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3.  If not see
+<http://www.gnu.org/licenses/>.  */
+
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+#include "tree.h"
+#include "langhooks.h"
+#include "expr.h"
+#include "optabs.h"
+#include "recog.h"
+#include "cilk.h"
+
+/* This structure holds all the important fields of the internal structures,
+   internal built-in functions, and Cilk-specific data types.  Explanation of 
+   all the these fielsd are given in cilk.h.  */
+tree cilk_trees[(int) CILK_TI_MAX];
+
+
+/* Returns the value in structure FRAME pointed by the FIELD_NUMBER
+   (e.g. X.y).  
+   FIELD_NUMBER is an index to the structure FRAME_PTR.  For details
+   about these fields, refer to cilk_trees structure in cilk.h and
+   cilk_init_builtins function  in this file.  Returns a TREE that is the type 
+   of the field represented by FIELD_NUMBER.  */
+
+tree
+cilk_dot (tree frame, int field_number, bool volatil)
+{
+  tree field = cilk_trees[field_number];
+  field = fold_build3 (COMPONENT_REF, TREE_TYPE (field), frame, field, 
+		       NULL_TREE);
+  TREE_THIS_VOLATILE (field) = volatil;
+  return field;
+}
+
+/* Returns the address of a field in FRAME_PTR, pointed by FIELD_NUMBER.  
+   (e.g. (&X)->y).   Please see cilk_dot function for explanation of the 
+   FIELD_NUMBER.  Returns a tree that is the type of the field represented 
+   by FIELD_NUMBER.  */
+
+tree
+cilk_arrow (tree frame_ptr, int field_number, bool volatil)
+{
+  return cilk_dot (fold_build1 (INDIRECT_REF, 
+				TREE_TYPE (TREE_TYPE (frame_ptr)), frame_ptr), 
+		   field_number, volatil);
+}
+
+
+/* This function will add FIELD of type TYPE to a defined built-in 
+   structure.  */
+
+static tree
+add_field (const char *name, tree type, tree fields)
+{
+  tree  t = get_identifier (name);
+  tree field = build_decl (BUILTINS_LOCATION, FIELD_DECL, t, type);
+  TREE_CHAIN (field) = fields;
+  return field;
+}
+
+/* This function will define a built-in function of NAME, of type FNTYPE and
+   register it under the built-in function code CODE.  */
+
+static tree
+install_builtin (const char *name, tree fntype, enum built_in_function code,
+                 bool publish)
+{
+  tree fndecl = build_fn_decl (name, fntype);
+  DECL_BUILT_IN_CLASS (fndecl) = BUILT_IN_NORMAL;
+  DECL_FUNCTION_CODE (fndecl) = code;
+  if (publish)
+    {
+      tree t = lang_hooks.decls.pushdecl (fndecl);
+      if (t)
+        fndecl = t;
+    }
+  set_builtin_decl (code, fndecl, true);
+  return fndecl;
+}
+
+/* Creates and initializes all the built-in Cilk keywords functions and three
+   internal structures: __cilkrts_stack_frame, __cilkrts_pedigree and
+   __cilkrts_worker.  Detailed information about __cilkrts_stack_frame and
+   __cilkrts_worker structures are given in libcilkrts/include/internal/abi.h.
+   __cilkrts_pedigree is described in libcilkrts/include/cilk/common.h.  */
+
+void
+cilk_init_builtins (void)
+{
+  /* Now build the following __cilkrts_pedigree struct:
+     struct __cilkrts_pedigree {
+        uint64_t rank;
+        struct __cilkrts_pedigree *parent;
+      }  */
+       
+  tree pedigree_type = lang_hooks.types.make_type (RECORD_TYPE);
+  tree pedigree_ptr  = build_pointer_type (pedigree_type);
+  tree field = add_field ("rank", uint64_type_node, NULL_TREE);
+  cilk_trees[CILK_TI_PEDIGREE_RANK] = field;
+  field = add_field ("parent", pedigree_ptr, field);
+  cilk_trees[CILK_TI_PEDIGREE_PARENT] = field;
+  finish_builtin_struct (pedigree_type, "__cilkrts_pedigree_GCC", field,
+			 NULL_TREE);
+  lang_hooks.types.register_builtin_type (pedigree_type,
+					  "__cilkrts_pedigree_t");
+  DECL_ALIGN (field) = BIGGEST_ALIGNMENT;
+  cilk_pedigree_type_decl = pedigree_type; 
+  
+  /* Build the Cilk Stack Frame:
+     struct __cilkrts_stack_frame {
+       uint32_t flags;
+       uint32_t size;
+       struct __cilkrts_stack_frame *call_parent;
+       __cilkrts_worker *worker;
+       void *except_data;
+       void *ctx[4];
+       uint32_t mxcsr;
+       uint16_t fpcsr;
+       uint16_t reserved;
+       __cilkrts_pedigree pedigree;
+     };  */
+
+  tree frame = lang_hooks.types.make_type (RECORD_TYPE);
+  tree frame_ptr = build_pointer_type (frame);
+  tree worker_type = lang_hooks.types.make_type (RECORD_TYPE);
+  tree worker_ptr = build_pointer_type (worker_type);
+  tree s_type_node = build_int_cst (size_type_node, 4);
+
+  tree flags = add_field ("flags", uint32_type_node, NULL_TREE);
+  tree size = add_field ("size", uint32_type_node, flags);
+  tree parent = add_field ("call_parent", frame_ptr, size);
+  tree worker = add_field ("worker", worker_ptr, parent);
+  tree except = add_field ("except_data", frame_ptr, worker);
+  tree context = add_field ("ctx",
+			    build_array_type (ptr_type_node,
+					      build_index_type (s_type_node)),
+			    except);
+  tree mxcsr = add_field ("mxcsr", uint32_type_node, context);
+  tree fpcsr = add_field ("fpcsr", uint16_type_node, mxcsr);
+  tree reserved = add_field ("reserved", uint16_type_node, fpcsr);
+  tree pedigree = add_field ("pedigree", pedigree_type, reserved);
+  
+  /* Now add them to a common structure whose fields are #defined to something
+     that is used at a later stage.  */
+  cilk_trees[CILK_TI_FRAME_FLAGS] = flags;
+  cilk_trees[CILK_TI_FRAME_PARENT] = parent;
+  cilk_trees[CILK_TI_FRAME_WORKER] = worker;
+  cilk_trees[CILK_TI_FRAME_EXCEPTION] = except;
+  cilk_trees[CILK_TI_FRAME_CONTEXT] = context;
+  /* We don't care about reserved, so no need to store it in cilk_trees.  */
+  cilk_trees[CILK_TI_FRAME_PEDIGREE] = pedigree;
+
+  TYPE_ALIGN (frame) = PREFERRED_STACK_BOUNDARY;
+  TREE_ADDRESSABLE (frame) = 1;
+
+  finish_builtin_struct (frame, "__cilkrts_st_frame_GCC", pedigree, NULL_TREE);
+  cilk_frame_type_decl = frame;
+  lang_hooks.types.register_builtin_type (frame, "__cilkrts_frame_t");
+
+  cilk_frame_ptr_type_decl = build_qualified_type (frame_ptr,
+						   TYPE_QUAL_VOLATILE);
+  /* Now let's do the following worker struct:
+
+     struct __cilkrts_worker {
+       __cilkrts_stack_frame *volatile *volatile tail;
+       __cilkrts_stack_frame *volatile *volatile head;
+       __cilkrts_stack_frame *volatile *volatile exc;
+       __cilkrts_stack_frame *volatile *volatile protected_tail;
+       __cilkrts_stack_frame *volatile *ltq_limit;
+       int32_t self;
+       global_state_t *g;
+       local_state *l;
+       cilkred_map *reducer_map;
+       __cilkrts_stack_frame *current_stack_frame;
+       void *reserved;
+       __cilkrts_worker_sysdep_state *sysdep;
+       __cilkrts_pedigree pedigree;
+    }   */
+
+  tree fptr_volatil_type = build_qualified_type (frame_ptr, TYPE_QUAL_VOLATILE);
+  tree fptr_volatile_ptr = build_pointer_type (fptr_volatil_type);
+  tree fptr_vol_ptr_vol = build_qualified_type (fptr_volatile_ptr,
+						TYPE_QUAL_VOLATILE);
+  tree g = lang_hooks.types.make_type (RECORD_TYPE);
+  finish_builtin_struct (g, "__cilkrts_global_state", NULL_TREE, NULL_TREE);
+  tree l = lang_hooks.types.make_type (RECORD_TYPE);
+  finish_builtin_struct (l, "__cilkrts_local_state", NULL_TREE, NULL_TREE);
+  tree sysdep_t = lang_hooks.types.make_type (RECORD_TYPE);
+  finish_builtin_struct (sysdep_t, "__cilkrts_worker_sysdep_state", NULL_TREE,
+			 NULL_TREE);
+  
+  field = add_field ("tail", fptr_vol_ptr_vol, NULL_TREE);
+  cilk_trees[CILK_TI_WORKER_TAIL] = field;
+  field = add_field ("head", fptr_vol_ptr_vol, field);
+  field  = add_field ("exc", fptr_vol_ptr_vol, field);
+  field = add_field ("protected_tail", fptr_vol_ptr_vol, field);
+  field = add_field ("ltq_limit", fptr_volatile_ptr, field);
+  field = add_field ("self", integer_type_node, field);
+  field = add_field ("g", build_pointer_type (g), field);
+  field = add_field ("l", build_pointer_type (g), field);
+  field = add_field ("reducer_map", ptr_type_node, field);
+  field = add_field ("current_stack_frame", frame_ptr, field);
+  cilk_trees[CILK_TI_WORKER_CUR] = field;
+  field = add_field ("saved_protected_tail", fptr_volatile_ptr, field);
+  field = add_field ("sysdep", build_pointer_type (sysdep_t), field);
+  field = add_field ("pedigree", pedigree_type, field);
+  cilk_trees[CILK_TI_WORKER_PEDIGREE] = field;
+  DECL_ALIGN (field) = BIGGEST_ALIGNMENT;
+  finish_builtin_struct (worker_type, "__cilkrts_worker_GCC", field,
+			 NULL_TREE);
+
+  tree fptr_arglist = tree_cons (NULL_TREE, frame_ptr, void_list_node);
+  tree fptr_fun = build_function_type (void_type_node, fptr_arglist);
+  
+  /* void __cilkrts_enter_frame_1 (__cilkrts_stack_frame *);  */
+  cilk_enter_fndecl = install_builtin ("__cilkrts_enter_frame_1", fptr_fun,
+				       BUILT_IN_CILK_ENTER_FRAME, false);
+
+  /* void __cilkrts_enter_frame_fast_1 (__cilkrts_stack_frame *);  */
+  cilk_enter_fast_fndecl = 
+    install_builtin ("__cilkrts_enter_frame_fast_1", fptr_fun, 
+		     BUILT_IN_CILK_ENTER_FRAME_FAST, false);
+  
+  /* void __cilkrts_pop_frame (__cilkrts_stack_frame *);  */
+  cilk_pop_fndecl = install_builtin ("__cilkrts_pop_frame", fptr_fun,
+				     BUILT_IN_CILK_POP_FRAME, false);
+
+  /* void __cilkrts_leave_frame (__cilkrts_stack_frame *);  */
+  cilk_leave_fndecl = install_builtin ("__cilkrts_leave_frame", fptr_fun,
+				       BUILT_IN_CILK_LEAVE_FRAME, false);
+
+  /* void __cilkrts_sync (__cilkrts_stack_frame *);  */
+  cilk_sync_fndecl = install_builtin ("__cilkrts_sync", fptr_fun,
+				      BUILT_IN_CILK_SYNC, false);
+
+  /* void __cilkrts_detach (__cilkrts_stack_frame *);  */
+  cilk_detach_fndecl = install_builtin ("__cilkrts_detach", fptr_fun,
+					BUILT_IN_CILK_DETACH, false);
+
+  /* __cilkrts_rethrow (struct stack_frame *);  */
+  cilk_rethrow_fndecl = install_builtin ("__cilkrts_rethrow", fptr_fun, 
+					 BUILT_IN_CILK_RETHROW, false);
+
+  /* __cilkrts_save_fp_ctrl_state (__cilkrts_stack_frame *);  */
+  cilk_save_fp_fndecl = install_builtin ("__cilkrts_save_fp_ctrl_state", 
+					 fptr_fun, BUILT_IN_CILK_SAVE_FP,
+					 false);
+}
+
+/* Get the appropriate frame arguments for CALL that is of type CALL_EXPR.  */
+
+static tree
+get_frame_arg (tree call)
+{
+  tree arg, argtype;
+
+  if (call_expr_nargs (call) < 1)
+    return NULL_TREE;
+
+  arg = CALL_EXPR_ARG (call, 0);
+  argtype = TREE_TYPE (arg);
+  if (TREE_CODE (argtype) != POINTER_TYPE)
+    return NULL_TREE;
+
+  argtype = TREE_TYPE (argtype);
+  
+  if (lang_hooks.types_compatible_p &&
+      !lang_hooks.types_compatible_p (argtype, cilk_frame_type_decl))
+    return NULL_TREE;
+
+  /* If it is passed in as an address, then just use the value directly 
+     since the function is inlined.  */
+  if (TREE_CODE (arg) == INDIRECT_REF || TREE_CODE (arg) == ADDR_EXPR)
+    return TREE_OPERAND (arg, 0);
+  return arg;
+}
+
+/* Expands the __cilkrts_pop_frame function call stored in EXP.
+   Returns const0_rtx.  */
+
+void
+expand_builtin_cilk_pop_frame (tree exp)
+{
+  tree frame = get_frame_arg (exp);
+  tree parent = cilk_dot (frame, CILK_TI_FRAME_PARENT, 0);
+
+  tree clear_parent = build2 (MODIFY_EXPR, void_type_node, parent,
+			      build_int_cst (TREE_TYPE (parent), 0));
+  expand_expr (clear_parent, const0_rtx, VOIDmode, EXPAND_NORMAL);
+
+  /* During LTO, the is_cilk_function flag gets cleared.
+     If __cilkrts_pop_frame is called, then this definitely must be a
+     cilk function.  */
+  if (cfun)
+    cfun->is_cilk_function = 1;
+}
+
+
+/* Expands the cilk_detach function call stored in EXP.  Returns const0_rtx.  */
+
+void
+expand_builtin_cilk_detach (tree exp)
+{
+  rtx insn;
+  tree fptr = get_frame_arg (exp);
+
+  if (fptr == NULL_TREE)
+    return;
+
+  tree parent = cilk_dot (fptr, CILK_TI_FRAME_PARENT, 0);
+  tree worker = cilk_dot (fptr, CILK_TI_FRAME_WORKER, 0);
+  tree tail = cilk_dot (worker, CILK_TI_WORKER_TAIL, 1);
+
+  rtx wreg = expand_expr (worker, NULL_RTX, Pmode, EXPAND_NORMAL);
+  if (GET_CODE (wreg) != REG)
+    wreg = copy_to_reg (wreg);
+  rtx preg = expand_expr (parent, NULL_RTX, Pmode, EXPAND_NORMAL);
+
+  /* TMP <- WORKER.TAIL
+    *TMP <- PARENT
+     TMP <- TMP + 1
+     WORKER.TAIL <- TMP   */
+
+  HOST_WIDE_INT worker_tail_offset =
+    tree_low_cst (DECL_FIELD_OFFSET (cilk_trees[CILK_TI_WORKER_TAIL]), 0) +
+    tree_low_cst (DECL_FIELD_BIT_OFFSET (cilk_trees[CILK_TI_WORKER_TAIL]), 0) /
+    BITS_PER_UNIT;
+  rtx tmem0 = gen_rtx_MEM (Pmode,
+			   plus_constant (Pmode, wreg, worker_tail_offset));
+  set_mem_attributes (tmem0, tail, 0);
+  MEM_NOTRAP_P (tmem0) = 1;
+  gcc_assert (MEM_VOLATILE_P (tmem0));
+  rtx treg = copy_to_mode_reg (Pmode, tmem0);
+  rtx tmem1 = gen_rtx_MEM (Pmode, treg);
+  set_mem_attributes (tmem1, TREE_TYPE (TREE_TYPE (tail)), 0);
+  MEM_NOTRAP_P (tmem1) = 1;
+  emit_move_insn (tmem1, preg);
+  emit_move_insn (treg, plus_constant (Pmode, treg, GET_MODE_SIZE (Pmode)));
+
+  /* There is a release barrier (st8.rel, membar #StoreStore,
+     sfence, lwsync, etc.) between the two stores.  On x86
+     normal volatile stores have proper semantics; the sfence
+     would only be needed for nontemporal stores (which we
+     could generate using the storent optab, for no benefit
+     in this case).
+
+     The predicate may return false even for a REG if this is
+     the limited release operation that only stores 0.  */
+  enum insn_code icode = direct_optab_handler (sync_lock_release_optab, Pmode); 
+  if (icode != CODE_FOR_nothing
+      && insn_data[icode].operand[1].predicate (treg, Pmode)
+      && (insn = GEN_FCN (icode) (tmem0, treg)) != NULL_RTX)
+    emit_insn (insn);
+  else
+    emit_move_insn (tmem0, treg);
+
+  /* The memory barrier inserted above should not prevent
+     the load of flags from being moved before the stores,
+     but in practice it does because it is implemented with
+     unspec_volatile.  In-order RISC machines should
+     explicitly load flags earlier.  */
+
+  tree flags = cilk_dot (fptr, CILK_TI_FRAME_FLAGS, 0);
+  expand_expr (build2 (MODIFY_EXPR, void_type_node, flags,
+		       build2 (BIT_IOR_EXPR, TREE_TYPE (flags), flags,
+			       build_int_cst (TREE_TYPE (flags),
+					      CILK_FRAME_DETACHED))),
+	       const0_rtx, VOIDmode, EXPAND_NORMAL);
+}
diff --git a/gcc/cilk.h b/gcc/cilk.h
new file mode 100644
index 0000000..038316a
--- /dev/null
+++ b/gcc/cilk.h
@@ -0,0 +1,94 @@ 
+/* This file is part of the Intel(R) Cilk(TM) Plus support
+   This file contains Cilk Support files.
+   Copyright (C) 2013  Free Software Foundation, Inc.
+   Contributed by Balaji V. Iyer <balaji.v.iyer@intel.com>,
+                  Intel Corporation
+
+   This file is part of GCC.
+
+   GCC is free software; you can redistribute it and/or modify it
+   under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3, or (at your option)
+   any later version.
+
+   GCC is distributed in the hope that it will be useful, but
+   WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with GCC; see the file COPYING3.  If not see
+   <http://www.gnu.org/licenses/>.  */
+
+#ifndef GCC_CILK_H
+#define GCC_CILK_H
+
+#include "tree.h"
+
+/* Frame status bits known to compiler.  */
+#define CILK_FRAME_STOLEN    0x01
+#define CILK_FRAME_UNSYNCHED 0x02
+#define CILK_FRAME_DETACHED  0x04
+#define CILK_FRAME_EXCEPTING 0x10
+#define CILK_FRAME_VERSION   (1 << 24)
+
+enum cilk_tree_index  {
+/* All the built-in functions for Cilk keywords.  */
+  CILK_TI_F_WORKER = 0,               /* __cilkrts_get_worker ().  */
+  CILK_TI_F_SYNC,                     /* __cilkrts_sync ().  */
+  CILK_TI_F_DETACH,                   /* __cilkrts_detach (...).   */
+  CILK_TI_F_ENTER,                    /* __cilkrts_enter_frame (...).  */
+  CILK_TI_F_ENTER_FAST,               /* __cilkrts_enter_frame_fast (.).  */
+  CILK_TI_F_LEAVE,                    /* __cilkrts_leave_frame (...).  */
+  CILK_TI_F_POP,                      /* __cilkrts_pop_frame (...).  */
+  CILK_TI_F_RETHROW,                  /* __cilkrts_rethrow (...).  */
+  CILK_TI_F_SAVE_FP,                  /* __cilkrts_save_fp_ctrl_state (...).  */
+  /* __cilkrts_stack_frame struct fields.  */
+  CILK_TI_FRAME_FLAGS,                /* stack_frame->flags.  */
+  CILK_TI_FRAME_PARENT,               /* stack_frame->parent.  */
+  CILK_TI_FRAME_WORKER,               /* stack_frame->worker.  */
+  CILK_TI_FRAME_EXCEPTION,            /* stack_frame->except_data.  */
+  CILK_TI_FRAME_CONTEXT,              /* stack_frame->context[4].  */
+  CILK_TI_FRAME_PEDIGREE,             /* stack_frame->pedigree.  */
+
+  /* __cilkrts_worker struct fields.  */
+  CILK_TI_WORKER_CUR,                 /* worker->current_stack_frame.  */
+  CILK_TI_WORKER_TAIL,                /* worker->tail.  */
+  CILK_TI_WORKER_PEDIGREE,            /* worker->pedigree.  */
+
+  /* __cilkrts_pedigree struct fields.  */
+  CILK_TI_PEDIGREE_RANK,              /* pedigree->rank.  */
+  CILK_TI_PEDIGREE_PARENT,            /* pedigree->parent.  */
+  
+  /* Types.  */
+  CILK_TI_FRAME_TYPE,                 /* struct __cilkrts_stack_frame.  */
+  CILK_TI_FRAME_PTR,                  /* __cilkrts_stack_frame *.  */
+  CILK_TI_WORKER_TYPE,                /* struct __cilkrts_worker.  */
+  CILK_TI_PEDIGREE_TYPE,              /* struct __cilkrts_pedigree.  */
+  CILK_TI_MAX
+};
+
+extern GTY (()) tree cilk_trees[CILK_TI_MAX];
+
+#define cilk_worker_fndecl            cilk_trees[CILK_TI_F_WORKER]
+#define cilk_sync_fndecl              cilk_trees[CILK_TI_F_SYNC]
+#define cilk_synched_fndecl           cilk_trees[CILK_TI_F_SYNCED]
+#define cilk_detach_fndecl            cilk_trees[CILK_TI_F_DETACH]
+#define cilk_enter_fndecl             cilk_trees[CILK_TI_F_ENTER]
+#define cilk_enter_fast_fndecl        cilk_trees[CILK_TI_F_ENTER_FAST]
+#define cilk_leave_fndecl             cilk_trees[CILK_TI_F_LEAVE]
+#define cilk_rethrow_fndecl           cilk_trees[CILK_TI_F_RETHROW]
+#define cilk_pop_fndecl               cilk_trees[CILK_TI_F_POP]
+#define cilk_save_fp_fndecl           cilk_trees[CILK_TI_F_SAVE_FP]
+
+#define cilk_worker_type_fndecl       cilk_trees[CILK_TI_WORKER_TYPE]
+#define cilk_frame_type_decl          cilk_trees[CILK_TI_FRAME_TYPE]
+#define cilk_frame_ptr_type_decl      cilk_trees[CILK_TI_FRAME_PTR]
+#define cilk_pedigree_type_decl       cilk_trees[CILK_TI_PEDIGREE_TYPE]
+
+extern void expand_builtin_cilk_detach (tree);
+extern void expand_builtin_cilk_pop_frame (tree);
+extern tree cilk_arrow (tree, int, bool);
+extern tree cilk_dot (tree, int, bool);
+extern void cilk_init_builtins (void);
+#endif
diff --git a/gcc/cppbuiltin.c b/gcc/cppbuiltin.c
index 7ce01cb..9e4751f 100644
--- a/gcc/cppbuiltin.c
+++ b/gcc/cppbuiltin.c
@@ -105,6 +105,8 @@  define_builtin_macros_for_compilation_flags (cpp_reader *pfile)
 
   cpp_define_formatted (pfile, "__FINITE_MATH_ONLY__=%d",
 			flag_finite_math_only);
+  if (flag_enable_cilkplus)
+    cpp_define (pfile, "__cilk=200");
 }
 
 
diff --git a/gcc/doc/generic.texi b/gcc/doc/generic.texi
index cacab01..d13296b 100644
--- a/gcc/doc/generic.texi
+++ b/gcc/doc/generic.texi
@@ -3155,6 +3155,30 @@  several statements chained together.
 Used to represent a @code{break} statement.  There are no additional
 fields.
 
+@item CILK_SPAWN_STMT
+
+Used to represent a spawning function in the Cilk Plus language extension.  
+This tree has one field that holds the name of the spawning function.
+@code{_Cilk_spawn} can be written in C in the following way:
+
+@smallexample
+@code{_Cilk_spawn} <function_name> (<parameters>);
+@end smallexample
+
+Detailed description for usage and functionality of @code{_Cilk_spawn} can be 
+found at http://www.cilkplus.org
+
+@item CILK_SYNC_STMT
+
+This statement is part of the Cilk Plus language extension.  It indicates that
+the current function cannot continue in parallel with its spawned children.  
+There are no additional fields.  @code{_Cilk_sync} can be written in C in the 
+following way:
+
+@smallexample
+@code{_Cilk_sync};
+@end smallexample
+
 @item CLEANUP_STMT
 
 Used to represent an action that should take place upon exit from the
diff --git a/gcc/doc/passes.texi b/gcc/doc/passes.texi
index 045f964..e7e28f4 100644
--- a/gcc/doc/passes.texi
+++ b/gcc/doc/passes.texi
@@ -124,13 +124,45 @@  true, then we expand them using either @code{expand_array_notation_exprs} or
 inside conditions, they are transformed using the function 
 @code{fix_conditional_array_notations}.  The C language-specific routines are 
 located in @file{c/c-array-notation.c} and the equivalent C++ routines are in 
-file @file{cp/cp-array-notation.c}.  Common routines such as functions to 
-initialize builtin functions are stored in @file{array-notation-common.c}.
+the file @file{cp/cp-array-notation.c}.  Common routines such as functions to 
+initialize built-in functions are stored in @file{array-notation-common.c}.
+
+@item Cilk Keywords:
+@itemize @bullet 
+@item @code{_Cilk_spawn}:
+The @code{_Cilk_spawn} keyword is parsed and the function it contains is marked 
+as a spawning function.  The spawning function is called the spawner.  At 
+the end of the parsing phase, appropriate built-in functions are 
+added to the spawner that are defined in the Cilk runtime.  The appropriate 
+locations of these functions, and the internal structures are detailed in 
+@code{cilk_init_builtins} in the file @file{cilk-common.c}.  The pointers to 
+Cilk functions and fields of internal structures are described 
+in @file{cilk.h}.  The built-in functions are described in 
+@file{cilk-builtins.def}.
+
+During gimplification, a new "spawn-helper" function is created.  
+The spawned function is replaced with a spawn helper function in the spawner.  
+The spawned function-call is moved into the spawn helper.  The main function
+that does these transformations is @code{gimplify_cilk_spawn} in
+@file{c-family/cilk.c}.  In the spawn-helper, the gimplification function 
+@code{gimplify_call_expr}, inserts a function call @code{__cilkrts_detach}.
+This function is expanded by @code{builtin_expand_cilk_detach} located in
+@file{c-family/cilk.c}.
+
+@item @code{_Cilk_sync}:
+@code{_Cilk_sync} is parsed like a keyword.  During gimplification, 
+the function @code{gimplify_cilk_sync} in @file{c-family/cilk.c}, will replace
+this keyword with a set of functions that are stored in the Cilk Runtime.  
+One of the internal functions inserted during gimplification, 
+@code{__cilkrts_pop_frame} must be expanded by the compiler and is 
+done by @code{builtin_expand_cilk_pop_frame} in @file{cilk-common.c}.
+
+@end itemize
 @end itemize
 
-Detailed information about Cilk Plus and language specification is provided in 
-@w{@uref{http://www.cilkplus.org/}}.  It is worth mentioning that the current 
-implementation follows ABI 0.9.
+Documentation about Cilk Plus and language specification is provided under the
+"Learn" section in @w{@uref{http://www.cilkplus.org/}}.  It is worth mentioning
+that the current implementation follows ABI 1.1.
 
 @node Gimplification pass
 @section Gimplification pass
diff --git a/gcc/function.h b/gcc/function.h
index c651f50..bd238ef 100644
--- a/gcc/function.h
+++ b/gcc/function.h
@@ -552,6 +552,9 @@  struct GTY(()) function {
   /* Vector of function local variables, functions, types and constants.  */
   vec<tree, va_gc> *local_decls;
 
+  /* In a Cilk function, the VAR_DECL for the frame descriptor. */
+  tree cilk_frame_decl;
+
   /* For md files.  */
 
   /* tm.h can use this to store whatever it likes.  */
@@ -607,6 +610,12 @@  struct GTY(()) function {
      either as a subroutine or builtin.  */
   unsigned int calls_alloca : 1;
 
+  /* This will indicate whether a function is a cilk function */
+  unsigned int is_cilk_function : 1;
+
+  /* Nonzero if this is a Cilk function that spawns. */
+  unsigned int calls_cilk_spawn : 1;
+  
   /* Nonzero if function being compiled receives nonlocal gotos
      from nested functions.  */
   unsigned int has_nonlocal_label : 1;
diff --git a/gcc/gimplify.c b/gcc/gimplify.c
index 4d39d53..d3748e5 100644
--- a/gcc/gimplify.c
+++ b/gcc/gimplify.c
@@ -42,6 +42,7 @@  along with GCC; see the file COPYING3.  If not see
 #include "pointer-set.h"
 #include "splay-tree.h"
 #include "vec.h"
+#include "cilk.h"
 
 #include "langhooks-def.h"	/* FIXME: for lhd_set_decl_assembler_name */
 #include "tree-pass.h"		/* FIXME: only for PROP_gimple_any */
@@ -7086,6 +7087,19 @@  gimplify_expr (tree *expr_p, gimple_seq *pre_p, gimple_seq *post_p,
       else if (ret != GS_UNHANDLED)
 	break;
 
+      if (flag_enable_cilkplus && lang_hooks.cilkplus.cilk_valid_spawn (expr_p))
+	{
+	  /* If there are errors, there is no point in expanding the
+	     _Cilk_spawn.  Just gimplify like a normal call expr.  */
+	  if (!seen_error ())
+	    {
+	      ret = (enum gimplify_status)
+		lang_hooks.cilkplus.gimplify_cilk_spawn (expr_p, pre_p, post_p);
+	      if (ret != GS_UNHANDLED)
+		continue;
+	    }
+	}
+
       /* Make sure that all the cases set 'ret' appropriately.  */
       ret = GS_UNHANDLED;
       switch (TREE_CODE (*expr_p))
@@ -7722,6 +7736,28 @@  gimplify_expr (tree *expr_p, gimple_seq *pre_p, gimple_seq *post_p,
 	    break;
 	  }
 
+	case CILK_SYNC_STMT:
+	  {
+	    if (flag_enable_cilkplus)
+	      {
+		if (!cfun->cilk_frame_decl)
+		  {
+		    error_at (input_location, "expected _Cilk_spawn before "
+			      "_Cilk_sync");
+		    ret = GS_ERROR;
+		  }
+		else
+		  ret = (enum gimplify_status)
+		    lang_hooks.cilkplus.gimplify_cilk_sync (expr_p, pre_p,
+							    post_p);
+		break;
+	      }
+	    else
+	      /* _Cilk_sync without Cilk Plus enabling should be caught by
+		 the parser.  */
+	      gcc_unreachable ();
+	  }
+
 	default:
 	  switch (TREE_CODE_CLASS (TREE_CODE (*expr_p)))
 	    {
diff --git a/gcc/ipa-inline-analysis.c b/gcc/ipa-inline-analysis.c
index 9a36292..7c63cc8 100644
--- a/gcc/ipa-inline-analysis.c
+++ b/gcc/ipa-inline-analysis.c
@@ -1433,6 +1433,9 @@  initialize_inline_failed (struct cgraph_edge *e)
     e->inline_failed = CIF_REDEFINED_EXTERN_INLINE;
   else if (e->call_stmt_cannot_inline_p)
     e->inline_failed = CIF_MISMATCHED_ARGUMENTS;
+  else if (flag_enable_cilkplus && cfun && cfun->calls_cilk_spawn)
+    /* We can't inline if the function is spawing a function.  */
+    e->inline_failed = CIF_FUNCTION_NOT_INLINABLE;
   else
     e->inline_failed = CIF_FUNCTION_NOT_CONSIDERED;
 }
diff --git a/gcc/ipa-inline.c b/gcc/ipa-inline.c
index 6eede0d..97c4632 100644
--- a/gcc/ipa-inline.c
+++ b/gcc/ipa-inline.c
@@ -258,7 +258,9 @@  can_inline_edge_p (struct cgraph_edge *e, bool report)
       e->inline_failed = CIF_BODY_NOT_AVAILABLE;
       inlinable = false;
     }
-  else if (!inline_summary (callee)->inlinable)
+  else if (!inline_summary (callee)->inlinable
+	   || (flag_enable_cilkplus && caller_cfun 
+	       && caller_cfun->calls_cilk_spawn))
     {
       e->inline_failed = CIF_FUNCTION_NOT_INLINABLE;
       inlinable = false;
diff --git a/gcc/ira.c b/gcc/ira.c
index ee0c5e8..5826344 100644
--- a/gcc/ira.c
+++ b/gcc/ira.c
@@ -1873,6 +1873,9 @@  ira_setup_eliminable_regset (bool from_ira_p)
        || (flag_stack_check && STACK_CHECK_MOVING_SP)
        || crtl->accesses_prior_frames
        || crtl->stack_realign_needed
+       /* We need a frame pointer for all Cilk Plus functions that use
+	  Cilk keywords.  */
+       || (flag_enable_cilkplus && cfun->is_cilk_function)
        || targetm.frame_pointer_required ());
 
   if (from_ira_p && ira_use_lra_p)
diff --git a/gcc/langhooks-def.h b/gcc/langhooks-def.h
index 7bd2e99..dc32af7 100644
--- a/gcc/langhooks-def.h
+++ b/gcc/langhooks-def.h
@@ -211,6 +211,24 @@  extern tree lhd_make_node (enum tree_code);
 #define LANG_HOOKS_OMP_CLAUSE_DTOR hook_tree_tree_tree_null
 #define LANG_HOOKS_OMP_FINISH_CLAUSE hook_void_tree
 
+extern void lhd_install_body_with_frame_cleanup (tree, tree);
+extern bool lhd_cilk_valid_spawn (tree *);
+#define LANG_HOOKS_CILKPLUS_SPAWNABLE_CTOR hook_bool_tree_false
+#define LANG_HOOKS_CILKPLUS_RECOGNIZE_SPAWN hook_bool_tree_false
+#define LANG_HOOKS_CILKPLUS_VALID_SPAWN lhd_cilk_valid_spawn
+#define LANG_HOOKS_CILKPLUS_FRAME_CLEANUP lhd_install_body_with_frame_cleanup
+#define LANG_HOOKS_CILKPLUS_GIMPLIFY_SPAWN lhd_gimplify_expr
+#define LANG_HOOKS_CILKPLUS_GIMPLIFY_SYNC lhd_gimplify_expr
+
+#define LANG_HOOKS_CILKPLUS {	        \
+  LANG_HOOKS_CILKPLUS_SPAWNABLE_CTOR,	\
+  LANG_HOOKS_CILKPLUS_RECOGNIZE_SPAWN,	\
+  LANG_HOOKS_CILKPLUS_VALID_SPAWN,	\
+  LANG_HOOKS_CILKPLUS_FRAME_CLEANUP,	\
+  LANG_HOOKS_CILKPLUS_GIMPLIFY_SPAWN,	\
+  LANG_HOOKS_CILKPLUS_GIMPLIFY_SYNC	\
+}
+
 #define LANG_HOOKS_DECLS { \
   LANG_HOOKS_GLOBAL_BINDINGS_P, \
   LANG_HOOKS_PUSHDECL, \
@@ -288,6 +306,7 @@  extern void lhd_end_section (void);
   LANG_HOOKS_TREE_DUMP_INITIALIZER, \
   LANG_HOOKS_DECLS, \
   LANG_HOOKS_FOR_TYPES_INITIALIZER, \
+  LANG_HOOKS_CILKPLUS, \
   LANG_HOOKS_LTO, \
   LANG_HOOKS_GET_INNERMOST_GENERIC_PARMS, \
   LANG_HOOKS_GET_INNERMOST_GENERIC_ARGS, \
diff --git a/gcc/langhooks.c b/gcc/langhooks.c
index 901f9b4..1564a12 100644
--- a/gcc/langhooks.c
+++ b/gcc/langhooks.c
@@ -666,3 +666,19 @@  lhd_end_section (void)
       saved_section = NULL;
     }
 }
+
+/* Empty function that is replaced with appropriate language dependent
+   frame cleanup function for _Cilk_spawn.  */
+
+void
+lhd_install_body_with_frame_cleanup (tree, tree)
+{
+  return;
+}
+
+/* Empty function to handle cilk_valid_spawn.  */
+bool
+lhd_cilk_valid_spawn (tree *)
+{
+  return false;
+}
diff --git a/gcc/langhooks.h b/gcc/langhooks.h
index 80d4ef3..9b8129a 100644
--- a/gcc/langhooks.h
+++ b/gcc/langhooks.h
@@ -136,6 +136,35 @@  struct lang_hooks_for_types
   tree (*reconstruct_complex_type) (tree, tree);
 };
 
+/* Language hooks related to Cilk Plus.  */
+
+struct lang_hooks_for_cilkplus
+{
+  /* Returns true if the constructor in C++ is spawnable.  Default is false.
+     This function is only used by C++.  */
+  bool (*spawnable_constructor) (tree);
+
+  /* Returns true if it is able to recognize a spawned function call 
+     inside the language-dependent trees (used by C++ front-end only).  */
+  bool (*recognize_spawn) (tree);
+  
+  /* Returns true if the call expr passed is a spawned function call.  */
+  bool (*cilk_valid_spawn) (tree *);
+
+  /* Function to add the clean up functions after spawn.  The reason why it is
+     language dependent is because in C++, it must handle exceptions.  */
+  void (*install_body_with_frame_cleanup) (tree, tree);
+
+  /* Function to gimplify a spawned function call.  Returns enum gimplify
+     status, but as mentioned in a previous comment, we can't see that type 
+     here, so just return an int.  */
+  int (*gimplify_cilk_spawn) (tree *, gimple_seq *, gimple_seq *);
+
+  /* Function to gimplify _Cilk_sync.  Same rationale as above for returning
+     int.  */
+  int (*gimplify_cilk_sync) (tree *, gimple_seq *, gimple_seq *);
+};
+
 /* Language hooks related to decls and the symbol table.  */
 
 struct lang_hooks_for_decls
@@ -405,6 +434,8 @@  struct lang_hooks
 
   struct lang_hooks_for_types types;
 
+  struct lang_hooks_for_cilkplus cilkplus;
+  
   struct lang_hooks_for_lto lto;
 
   /* Returns a TREE_VEC of the generic parameters of an instantiation of
diff --git a/gcc/lto/Make-lang.in b/gcc/lto/Make-lang.in
index 5f2f475..fd98ec5 100644
--- a/gcc/lto/Make-lang.in
+++ b/gcc/lto/Make-lang.in
@@ -78,7 +78,7 @@  $(LTO_EXE): $(LTO_OBJS) $(BACKEND) $(LIBDEPS)
 lto/lto-lang.o: lto/lto-lang.c $(CONFIG_H) coretypes.h debug.h \
 	flags.h $(GGC_H) langhooks.h $(LANGHOOKS_DEF_H) $(SYSTEM_H) \
 	$(TARGET_H) $(LTO_H) $(GIMPLE_H) gtype-lto.h gt-lto-lto-lang.h \
-	$(EXPR_H) $(LTO_STREAMER_H)
+	$(EXPR_H) $(LTO_STREAMER_H) cilk.h
 lto/lto.o: lto/lto.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(OPTS_H) \
 	toplev.h $(TREE_H) $(TREE_FLOW_H) $(DIAGNOSTIC_CORE_H) $(TM_H) \
 	$(CGRAPH_H) $(GGC_H) tree-ssa-operands.h $(TREE_PASS_H) \
diff --git a/gcc/lto/lto-lang.c b/gcc/lto/lto-lang.c
index 87a756d..cef0e28 100644
--- a/gcc/lto/lto-lang.c
+++ b/gcc/lto/lto-lang.c
@@ -35,6 +35,7 @@  along with GCC; see the file COPYING3.  If not see
 #include "diagnostic-core.h"
 #include "toplev.h"
 #include "lto-streamer.h"
+#include "cilk.h"
 
 static tree lto_type_for_size (unsigned, int);
 
@@ -1188,6 +1189,9 @@  lto_init (void)
       lto_define_builtins (va_list_type_node,
 			   build_reference_type (va_list_type_node));
     }
+  
+  if (flag_enable_cilkplus)
+    cilk_init_builtins ();
 
   targetm.init_builtins ();
   build_common_builtin_nodes ();
diff --git a/gcc/testsuite/c-c++-common/cilk-plus/CK/compound_cilk_spawn.c b/gcc/testsuite/c-c++-common/cilk-plus/CK/compound_cilk_spawn.c
new file mode 100644
index 0000000..6ed55e2
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/cilk-plus/CK/compound_cilk_spawn.c
@@ -0,0 +1,26 @@ 
+/* { dg-do compile } */
+/* { dg-options "-fcilkplus" } */
+
+/* <feature>
+   A program is considered ill formed if the _Cilk_spawn form of this
+    expression appears other than in one of the following contexts:
+    as the entire body of an expression statement,
+    as the entire right hand side of an assignment expression that is the entire
+    body of an expression statement, or as the entire initializer-clause in a 
+    simple declaration.
+   </feature>
+*/
+
+int spawn_func (int arg)
+{
+  return arg + 1;
+}
+
+int check()
+{
+  int z;
+  z = 23, _Cilk_spawn spawn_func (3), 3424; /* { dg-error "spawned function call cannot be part of a comma expression" } */
+  23, spawn_func (5), _Cilk_spawn spawn_func (3); /* { dg-error "spawned function call cannot be part of a comma expression" } */
+  _Cilk_spawn spawn_func (0), _Cilk_spawn spawn_func (3), 3, spawn_func (0); /* { dg-error "spawned function call cannot be part of a comma expression" } */
+  return _Cilk_spawn spawn_func (3), 23; /* { dg-error "spawned function call cannot be part of a comma expression" } */
+}
diff --git a/gcc/testsuite/c-c++-common/cilk-plus/CK/concec_cilk_spawn.c b/gcc/testsuite/c-c++-common/cilk-plus/CK/concec_cilk_spawn.c
new file mode 100644
index 0000000..04d5478
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/cilk-plus/CK/concec_cilk_spawn.c
@@ -0,0 +1,20 @@ 
+/* { dg-do compile } */
+/* { dg-options "-fcilkplus" } */
+
+/* <feature> Consecutive _Cilk_spawn tokens are not permitted
+   </feature>
+*/
+
+int spawn_func (int arg)
+{
+  return arg + 1;
+}
+
+void func ()
+{
+  int a;
+  a = _Cilk_spawn _Cilk_spawn spawn_func (4); /* { dg-error "consecutive _Cilk_spawn keywords are not permitted" } */
+  a = _Cilk_spawn _Cilk_spawn _Cilk_spawn spawn_func (4); /* { dg-error "consecutive _Cilk_spawn keywords are not permitted" } */
+  a = _Cilk_spawn _Cilk_spawn _Cilk_spawn _Cilk_spawn spawn_func (4); /* { dg-error "consecutive _Cilk_spawn keywords are not permitted" } */
+  return;
+}
diff --git a/gcc/testsuite/c-c++-common/cilk-plus/CK/fib.c b/gcc/testsuite/c-c++-common/cilk-plus/CK/fib.c
new file mode 100644
index 0000000..19614fc
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/cilk-plus/CK/fib.c
@@ -0,0 +1,62 @@ 
+/* { dg-do compile } */
+/* { dg-do run  { target { i?86-*-* x86_64-*-* arm*-*-* } } } */
+/* { dg-options "-fcilkplus" } */
+/* { dg-options "-lcilkrts" { target { i?86-*-* x86_64-*-* arm*-*-* } } } */
+
+#if HAVE_IO
+#include <stdio.h>
+#endif
+
+int fib        (int);
+int fib_serial (int);
+
+int main(void)
+{
+  int ii = 0, error = 0;
+  int fib_result[41], fib_serial_result[41];
+#if HAVE_IO
+
+  for (ii = 0; ii <= 40; ii++)
+    printf("fib (%2d) = %10d\n", ii, fib (ii));
+#else
+  for (ii = 0; ii <= 40; ii++)
+    {
+      fib_result[ii]        = fib (ii);
+      fib_serial_result[ii] = fib_serial (ii);
+    }
+
+  for (ii = 0; ii <= 40; ii++)
+    {
+      if (fib_result[ii] != fib_serial_result[ii])
+	error = 1;
+    }
+#endif
+  return error;
+}
+
+int fib_serial (int n)
+{
+  int x = 0, y = 0;
+  if (n < 2)
+    return n;
+  else
+    {
+      x = fib (n-1);
+      y = fib (n-2);
+      return (x+y);
+    }
+}
+
+int fib(int n)
+{
+  int x = 0, y = 0;
+  if (n < 2) 
+    return n;
+  else
+  {
+    x = _Cilk_spawn fib(n-1);
+    y = fib(n-2);
+    _Cilk_sync;
+    return (x+y);
+  }
+}
diff --git a/gcc/testsuite/c-c++-common/cilk-plus/CK/invalid_spawns.c b/gcc/testsuite/c-c++-common/cilk-plus/CK/invalid_spawns.c
new file mode 100644
index 0000000..2422a94
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/cilk-plus/CK/invalid_spawns.c
@@ -0,0 +1,11 @@ 
+extern int foo ();
+int bar = _Cilk_spawn foo (); /* { dg-error "_Cilk_spawn may only be used inside a function" } */
+
+
+int main (void)
+{
+  int x; 
+
+  _Cilk_spawn x; /* { dg-error "only function calls can be spawned" } */
+  return x;
+}
diff --git a/gcc/testsuite/c-c++-common/cilk-plus/CK/no_args_error.c b/gcc/testsuite/c-c++-common/cilk-plus/CK/no_args_error.c
new file mode 100644
index 0000000..9a08476
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/cilk-plus/CK/no_args_error.c
@@ -0,0 +1,11 @@ 
+/* { dg-do compile } */
+/* { dg-options "-fcilkplus" } */
+
+int spawn_1 ();
+typedef int(*func) (int);
+
+void check () {
+      func var = spawn_1;
+        _Cilk_spawn var (); /* { dg-error "too few arguments to function" } */
+}
+
diff --git a/gcc/testsuite/c-c++-common/cilk-plus/CK/spawn_in_return.c b/gcc/testsuite/c-c++-common/cilk-plus/CK/spawn_in_return.c
new file mode 100644
index 0000000..1029a0d
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/cilk-plus/CK/spawn_in_return.c
@@ -0,0 +1,8 @@ 
+/* { dg-do compile } */
+/* { dg-options "-fcilkplus" } */
+
+int main (void)
+{
+  extern int foo ();
+  return _Cilk_spawn foo (); /* { dg-error "in return statement is not allowed" } */
+}
diff --git a/gcc/testsuite/c-c++-common/cilk-plus/CK/spawnee_inline.c b/gcc/testsuite/c-c++-common/cilk-plus/CK/spawnee_inline.c
new file mode 100644
index 0000000..daf932e
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/cilk-plus/CK/spawnee_inline.c
@@ -0,0 +1,81 @@ 
+/* { dg-do compile } */
+/* { dg-do run  { target { i?86-*-* x86_64-*-* arm*-*-* } } } */
+/* { dg-options "-fcilkplus -w" } */
+/* { dg-options "-lcilkrts" { target { i?86-*-* x86_64-*-* arm*-*-* } } } */
+
+#include <stdio.h>
+#include <stdlib.h>
+#define DEFAULT_VALUE "30"
+
+int fib (char *n_char)
+{
+  int n;
+  char n_char_minus_one[20], n_char_minus_two[20];
+  if (n_char)
+    n = atoi (n_char);
+  else
+    n = atoi(DEFAULT_VALUE);
+  
+  if (n < 2)
+    return n;
+  else
+    {	   
+      int x, y;
+      sprintf (n_char_minus_one,"%d", n-1); 
+      sprintf (n_char_minus_two,"%d", n-2); 
+      x = _Cilk_spawn fib (n_char_minus_one);
+      y = _Cilk_spawn fib (n_char_minus_two);
+      _Cilk_sync;
+      return (x+y);
+    }
+}
+
+int fib_serial (int n)
+{
+  int x, y;
+  if (n < 2)
+    return n;
+  else
+    {
+      x = fib_serial (n-1);
+      y = fib_serial (n-2);
+      return (x+y);
+    }
+  return 0;
+}
+
+int main2_parallel (int argc, char *argv[])
+{
+  int n, result_parallel = 0;
+
+  if (argc == 2)
+    {
+      result_parallel = _Cilk_spawn fib (argv[1]);
+      _Cilk_sync; 
+    }
+  else
+    {
+      result_parallel = _Cilk_spawn fib("30");
+      _Cilk_sync; 
+    }
+  return result_parallel;
+}
+
+int main2_serial (int argc, char *argv[])
+{
+  int n, result_serial = 0;
+  if (argc == 2) 
+    result_serial = fib_serial (atoi (argv[1]));
+  else
+    result_serial = fib_serial (atoi (DEFAULT_VALUE));
+
+  return result_serial;
+}
+
+int main (void)
+{
+  if (main2_serial (1, 0) != main2_parallel (1,0))
+    return 1;
+  return 0;
+}
+
diff --git a/gcc/testsuite/c-c++-common/cilk-plus/CK/spawner_inline.c b/gcc/testsuite/c-c++-common/cilk-plus/CK/spawner_inline.c
new file mode 100644
index 0000000..12a44c8
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/cilk-plus/CK/spawner_inline.c
@@ -0,0 +1,68 @@ 
+/* { dg-do compile } */
+/* { dg-do run  { target { i?86-*-* x86_64-*-* arm*-*-* } } } */
+/* { dg-options "-fcilkplus" } */
+/* { dg-options "-lcilkrts" { target { i?86-*-* x86_64-*-* arm*-*-* } } } */
+
+#include <stdlib.h>
+#define DEFAULT_VALUE 30
+int fib (int n)
+{
+  if (n<2)
+    return n;
+  else
+    {
+      int x, y;
+      x = _Cilk_spawn fib (n-1);
+      y = _Cilk_spawn fib (n-2);
+      _Cilk_sync;
+      return (x+y);
+      return 5;
+    }
+}
+
+int main_parallel (int argc, char *argv[])
+{
+  int n, result;
+  if (argc == 2)
+    n = atoi(argv[1]);
+  else
+    n = DEFAULT_VALUE;
+  result = _Cilk_spawn fib(n);
+  _Cilk_sync; 
+  return result;
+}
+
+int fib_serial (int n)
+{
+  int x, y;
+  if (n < 2)
+    return n;
+  else
+    {
+      x = fib (n-1);
+      y = fib (n-2);
+      return (x+y);
+    }
+}
+  
+int main_serial (int argc, char *argv[])
+{
+  int n, result;
+
+  if (argc == 2)
+    n = atoi (argv[1]);
+  else
+    n = DEFAULT_VALUE;
+  result = fib_serial (n);
+
+  return result;
+}
+
+int main (void)
+{
+  if (main_serial (1, 0) != main_parallel (1,0))
+    return 1;
+  else 
+    return 0;
+}
+
diff --git a/gcc/testsuite/c-c++-common/cilk-plus/CK/spawning_arg.c b/gcc/testsuite/c-c++-common/cilk-plus/CK/spawning_arg.c
new file mode 100644
index 0000000..465d1da
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/cilk-plus/CK/spawning_arg.c
@@ -0,0 +1,38 @@ 
+/* { dg-do compile } */
+/* { dg-do run  { target { i?86-*-* x86_64-*-* arm*-*-* } } } */
+/* { dg-options "-fcilkplus" } */
+/* { dg-options "-lcilkrts" { target { i?86-*-* x86_64-*-* arm*-*-* } } } */
+
+void f0(volatile int *steal_flag)
+{ 
+  int i = 0;
+  /* Wait for steal_flag to be set */
+  while (!*steal_flag) 
+    ;
+}
+
+int f1()
+{
+
+  volatile int steal_flag = 0;
+  _Cilk_spawn f0(&steal_flag);
+  steal_flag = 1;  // Indicate stolen
+  _Cilk_sync; 
+  return 0;
+}
+
+void f2(int q)
+{
+  q = 5;
+}
+
+void f3()
+{
+   _Cilk_spawn f2(f1());
+}
+
+int main()
+{
+  f3();
+  return 0;
+}
diff --git a/gcc/testsuite/c-c++-common/cilk-plus/CK/steal_check.c b/gcc/testsuite/c-c++-common/cilk-plus/CK/steal_check.c
new file mode 100644
index 0000000..8430ce3
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/cilk-plus/CK/steal_check.c
@@ -0,0 +1,43 @@ 
+/* { dg-do compile } */
+/* { dg-do run  { target { i?86-*-* x86_64-*-* arm*-*-* } } } */
+/* { dg-options "-fcilkplus" } */
+/* { dg-options "-lcilkrts" { target { i?86-*-* x86_64-*-* arm*-*-* } } } */
+
+extern void __cilkrts_set_param (char *x, char *y);
+
+void foo(volatile int *);
+
+void main2(void);
+
+int main(void)
+{
+  __cilkrts_set_param ("nworkers", "2");
+  main2();
+  return 0;
+}
+
+
+void main2(void)
+{
+  int some_var = 0;
+
+  _Cilk_spawn foo(&some_var);
+
+  some_var=1;
+  some_var=5;
+  some_var=3;
+  some_var=4;
+
+  _Cilk_sync; 
+  return;
+}
+
+void foo(volatile int *some_other_var)
+{
+  while (*some_other_var == 0)
+  {
+   ;
+  }
+}
+
+
diff --git a/gcc/testsuite/c-c++-common/cilk-plus/CK/sync_wo_spawn.c b/gcc/testsuite/c-c++-common/cilk-plus/CK/sync_wo_spawn.c
new file mode 100644
index 0000000..8985b00
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/cilk-plus/CK/sync_wo_spawn.c
@@ -0,0 +1,9 @@ 
+/* { dg-do compile } */
+/* { dg-options "-fcilkplus" } */
+
+int main (void)
+{
+  _Cilk_sync; /* { dg-error "expected _Cilk_spawn before _Cilk_sync" } */
+  return 0;
+}
+
diff --git a/gcc/testsuite/c-c++-common/cilk-plus/CK/test__cilk.c b/gcc/testsuite/c-c++-common/cilk-plus/CK/test__cilk.c
new file mode 100644
index 0000000..69197fc
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/cilk-plus/CK/test__cilk.c
@@ -0,0 +1,10 @@ 
+/* { dg-do compile } */
+/* { dg-do run  { target { i?86-*-* x86_64-*-* arm*-*-* } } } */
+/* { dg-options "-fcilkplus" } */
+
+int main (void)
+{
+  if (__cilk == 200)
+   return 0; 
+  return 1;
+}
diff --git a/gcc/testsuite/c-c++-common/cilk-plus/CK/varargs_test.c b/gcc/testsuite/c-c++-common/cilk-plus/CK/varargs_test.c
new file mode 100644
index 0000000..e617376
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/cilk-plus/CK/varargs_test.c
@@ -0,0 +1,48 @@ 
+/* { dg-do compile } */
+/* { dg-do run  { target { i?86-*-* x86_64-*-* arm*-*-* } } } */
+/* { dg-options "-fcilkplus" } */
+/* { dg-options "-lcilkrts" { target { i?86-*-* x86_64-*-* arm*-*-* } } } */
+
+#include <stdarg.h>
+#include <stdlib.h>
+
+
+double compute_total (int no_elements, ...);
+
+int main(int argc, char **argv)
+{
+  double array[5] = {5.0, 4.0, 9.0, 3.0, 4.0};
+  double array2[5] = {5.0, 6.0, 8.0, 6.0};
+  double yy=0, xx=0, xx_serial, yy_serial;
+
+  yy = _Cilk_spawn compute_total(5,array[0],array[1],array[2],
+                                 array[3], array[4]);
+  xx= compute_total(4,array2[0],array2[1],array2[2], array2[3]);
+  
+  _Cilk_sync;
+
+  yy_serial = compute_total(5,array[0],array[1],array[2], array[3], array[4]);
+  xx_serial = compute_total(4,array2[0],array2[1],array2[2], array2[3]);
+
+  if ((xx + yy) != (xx_serial + yy_serial)) 
+    return 1;
+  return 0;
+  
+}
+
+
+double compute_total (int no_elements, ...)
+{
+  double total = 0;
+  va_list args;
+  va_start(args, no_elements);
+  int ii = 0;
+  for (ii = 0; ii < no_elements; ii++)
+  {
+    total += va_arg(args,double);
+  }
+  va_end(args);
+
+  return total;
+}
+
diff --git a/gcc/testsuite/gcc.dg/cilk-plus/cilk-plus.exp b/gcc/testsuite/gcc.dg/cilk-plus/cilk-plus.exp
index 2533feb..7aa1839 100644
--- a/gcc/testsuite/gcc.dg/cilk-plus/cilk-plus.exp
+++ b/gcc/testsuite/gcc.dg/cilk-plus/cilk-plus.exp
@@ -23,6 +23,11 @@  if { ![check_effective_target_cilkplus] } {
     return;
 }
 
+verbose "$tool $libdir" 1
+set library_var "[get_multilibs]"
+# Pointing the ld_library_path to the Cilk Runtime library binaries. 
+set ld_library_path "$[get_multilibs]/libcilkrts/.libs"
+
 dg-init
 dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/AN/*.c]] " -fcilkplus" " "
 dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/AN/*.c]] " -O0 -fcilkplus" " "
@@ -46,4 +51,31 @@  dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/AN/*.c]] " -f
 dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/AN/*.c]] " -fcilkplus -g -O2 -ftree-vectorize -std=c99" " "
 dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/AN/*.c]] " -fcilkplus -g -O3 -std=c99" " "
 dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/AN/*.c]] " -O3 -ftree-vectorize -std=c99 -g -fcilkplus" " "
+
+
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -fcilkplus" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -O0 -fcilkplus" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -O1 -fcilkplus" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -O2 -ftree-vectorize -fcilkplus" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -O3 -fcilkplus" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -g -fcilkplus" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -g -O0 -fcilkplus" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -g -O1 -fcilkplus" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -g -O2 -ftree-vectorize -fcilkplus" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -g -O3 -fcilkplus" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -O3 -ftree-vectorize -fcilkplus -g" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -fcilkplus -std=c99" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -fcilkplus -O0 -std=c99" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -fcilkplus -O1 -std=c99" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -fcilkplus -O2 -ftree-vectorize -std=c99" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -fcilkplus -O3 -std=c99" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -fcilkplus -g -std=c99" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -fcilkplus -g -O0 -std=c99" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -fcilkplus -g -O1 -std=c99" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -fcilkplus -g -O2 -ftree-vectorize -std=c99" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -fcilkplus -g -O3 -std=c99" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -O3 -ftree-vectorize -std=c99 -g -fcilkplus" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -O0 -flto -g -fcilkplus" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -O2 -flto -g -fcilkplus" " "
+dg-runtest [lsort [glob -nocomplain $srcdir/c-c++-common/cilk-plus/CK/*.c]] " -O3  -flto -g -fcilkplus" " "
 dg-finish
diff --git a/gcc/tree-inline.h b/gcc/tree-inline.h
index b65dee9..32bedac 100644
--- a/gcc/tree-inline.h
+++ b/gcc/tree-inline.h
@@ -123,6 +123,10 @@  typedef struct copy_body_data
      the originals have been mapped to a value rather than to a
      variable.  */
   struct pointer_map_t *debug_map;
+ 
+  /* Cilk keywords currently need to replace some variables that
+     ordinary nested functions do not.  */ 
+  bool remap_var_for_cilk;
 } copy_body_data;
 
 /* Weights of constructions for estimate_num_insns.  */
diff --git a/gcc/tree-pretty-print.c b/gcc/tree-pretty-print.c
index 7745f73..f449c68 100644
--- a/gcc/tree-pretty-print.c
+++ b/gcc/tree-pretty-print.c
@@ -2403,6 +2403,14 @@  dump_generic_node (pretty_printer *buffer, tree node, int spc, int flags,
       dump_block_node (buffer, node, spc, flags);
       break;
 
+    case CILK_SPAWN_STMT:
+      pp_string (buffer, "_Cilk_spawn ");
+      dump_generic_node (buffer, TREE_OPERAND (node, 0), spc, flags, false);
+      break;
+
+    case CILK_SYNC_STMT:
+      pp_string (buffer, "_Cilk_sync;");
+      break;
     default:
       NIY;
     }
diff --git a/gcc/tree.def b/gcc/tree.def
index da30074..1c98861 100644
--- a/gcc/tree.def
+++ b/gcc/tree.def
@@ -1227,6 +1227,14 @@  DEFTREECODE (OPTIMIZATION_NODE, "optimization_node", tcc_exceptional, 0)
 /* TARGET_OPTION_NODE.  Node to store the target specific options.  */
 DEFTREECODE (TARGET_OPTION_NODE, "target_option_node", tcc_exceptional, 0)
 
+/* Cilk spawn expression
+   Operand 0 is the CALL_EXPR.  */
+DEFTREECODE (CILK_SPAWN_STMT, "cilk_spawn_stmt", tcc_statement, 1)
+
+/* Cilk Sync Statement: Does not have any operands.  */
+DEFTREECODE (CILK_SYNC_STMT, "cilk_sync_stmt", tcc_expression, 0)
+
+
 /*
 Local variables:
 mode:c
diff --git a/gcc/tree.h b/gcc/tree.h
index 0058a4b..44bb375 100644
--- a/gcc/tree.h
+++ b/gcc/tree.h
@@ -1749,6 +1749,9 @@  extern void protected_set_expr_location (tree, location_t);
 #define CALL_EXPR_ARGP(NODE) \
   (&(TREE_OPERAND (CALL_EXPR_CHECK (NODE), 0)) + 3)
 
+/* Cilk Keywords accessors.  */
+#define CILK_SPAWN_FN(NODE) TREE_OPERAND (CILK_SPAWN_STMT_CHECK (NODE), 0)
+
 /* TM directives and accessors.  */
 #define TRANSACTION_EXPR_BODY(NODE) \
   TREE_OPERAND (TRANSACTION_EXPR_CHECK (NODE), 0)