Message ID | 20141115010412.GA32699@virgil.suse |
---|---|
State | New |
Headers | show |
On 11/15/2014 02:04 AM, Martin Jambor wrote: > Hi, > > this patch adds very simple propagation of alignment of pointers to > IPA-CP. Because I have not attempted to estimate profitability of > such propagation in any way, it does not do any cloning, just > propagation when the alignment is known and the same in all contexts. Thanks for this improvement ! From the Fortran side, arrays can be "created" in the following ways: 1. Statically in the main program. 2. As a subroutine-temporary "automatic array". 3. By allocating an allocatable array. Arrays under 1. are aligned properly by the compiler. Arrays under 2. are aligned properly because of the proper alignment of the stack nowadays. Arrays under 3. are aligned properly because Fortran "ALLOCATE" ultimately calls malloc. So Fortran arrays are always suitably aligned (the exception being an array actual argument passed as "CALL SUB(.., A(2:), ..)", which is extremely rare). So this propagation of alignment information will result in basically removing all alignment peeling for Fortran code. Thanks !
On Sat, Nov 15, 2014 at 2:04 AM, Martin Jambor <mjambor@suse.cz> wrote: > Hi, > > this patch adds very simple propagation of alignment of pointers to > IPA-CP. Because I have not attempted to estimate profitability of > such propagation in any way, it does not do any cloning, just > propagation when the alignment is known and the same in all contexts. > > I have seen this shrinking code in a few vectorization testcases in > our testsuite, especially if run with LTO. > > The patch is slightly bigger because the streaming of transformation > summaries had to be modified (and many parts renamed) a little bit to > also hold the resultant alignment information. > > In the future we may replace this by a more fancy VRP-based IPA-CP but > I think it is worth having this simple addition in 5.0. Bootstrapped > and tested on x86_64-linux. OK for trunk? > > Thanks, > > Martin > > > 2014-11-15 Martin Jambor <mjambor@suse.cz> > > * ipa-prop.h (ipa_alignment): New type. > (ipa_jump_func): New field alignment. > (ipcp_transformation_summary) New type. > (ipcp_grow_transformations_if_necessary): Declare. > (ipa_node_agg_replacements): Removed. > (ipcp_transformations): Declare. > (ipcp_get_transformation_summary): New function. > (ipa_get_agg_replacements_for_node): Use it. > * ipa-cp.c (ipcp_param_lattices): New field alignment. > (print_all_lattices): Also print alignment. > (alignment_bottom_p): New function. > (set_alignment_to_bottom): Likewise. > (set_all_contains_variable): Also set alignment to bottom. > (initialize_node_lattices): Likewise. > (propagate_alignment_accross_jump_function): New function. > (propagate_constants_accross_call): Call it. > (ipcp_store_alignment_results): New function. > (ipcp_driver): Call it. > * ipa-prop.c (ipa_node_agg_replacements): Removed. > (ipcp_transformations): New. > (ipa_print_node_jump_functions_for_edge): Also print alignment. > (ipa_set_jf_unknown): New function. > (detect_type_change_from_memory_writes): Use ipa_set_jf_unknown. > (ipa_compute_jump_functions_for_edge): Also calculate alignment. > (update_jump_functions_after_inlining): Use ipa_set_jf_unknown. > (ipcp_grow_transformations_if_necessary): New function. > (ipa_set_node_agg_value_chain): Use ipcp_transformations. > (ipa_node_removal_hook): Likewise. > (ipa_node_duplication_hook): Also duplicate alignment results. > (ipa_write_jump_function): Also stream alignments. > (ipa_read_jump_function): Use ipa_set_jf_unknown, also stream > alignments. > (write_agg_replacement_chain): Renamed to > write_ipcp_transformation_info, also stream alignments. > (read_agg_replacement_chain): Renamed to > read_ipcp_transformation_info, also stream alignments. > (ipa_prop_write_all_agg_replacement): Renamed to > ipcp_write_transformation_summaries. Stream always. > (ipa_prop_read_all_agg_replacement): Renamed to > ipcp_read_transformation_summaries. > (ipcp_update_alignments): New function. > (ipcp_transform_function): Call it, free also alignments. > > testsuite/ > * gcc.dg/ipa/propalign-1.c: New test. > * gcc.dg/ipa/propalign-2.c: Likewise. > > Index: src/gcc/ipa-cp.c > =================================================================== > --- src.orig/gcc/ipa-cp.c 2014-11-14 22:06:06.240031030 +0100 > +++ src/gcc/ipa-cp.c 2014-11-15 01:40:38.260521577 +0100 > @@ -262,6 +262,9 @@ public: > ipcp_lattice<ipa_polymorphic_call_context> ctxlat; > /* Lattices describing aggregate parts. */ > ipcp_agg_lattice *aggs; > + /* Alignment information. Very basic one value lattice where !known means > + TOP and zero alignment bottom. */ > + ipa_alignment alignment; > /* Number of aggregate lattices */ > int aggs_count; > /* True if aggregate data were passed by reference (as opposed to by > @@ -444,6 +447,13 @@ print_all_lattices (FILE * f, bool dump_ > plats->itself.print (f, dump_sources, dump_benefits); > fprintf (f, " ctxs: "); > plats->ctxlat.print (f, dump_sources, dump_benefits); > + if (plats->alignment.known && plats->alignment.align > 0) > + fprintf (f, " Alignment %u, misaglignment %u\n", > + plats->alignment.align, plats->alignment.bitpos); > + else if (plats->alignment.known) > + fprintf (f, " Alignment unusable\n"); > + else > + fprintf (f, " Alignment unknown\n"); > if (plats->virt_call) > fprintf (f, " virt_call flag set\n"); > > @@ -761,6 +771,27 @@ set_agg_lats_contain_variable (struct ip > return ret; > } > > +/* Return true if alignemnt informatin in PLATS is known to be unusable. */ > + > +static inline bool > +alignment_bottom_p (ipcp_param_lattices *plats) > +{ > + return plats->alignment.known && (plats->alignment.align == 0); > +} > + > +/* Set alignment information in PLATS to unusable. Return true if it > + previously was usable or unknown. */ > + > +static inline bool > +set_alignment_to_bottom (ipcp_param_lattices *plats) > +{ > + if (alignment_bottom_p (plats)) > + return false; > + plats->alignment.known = true; > + plats->alignment.align = 0; > + return true; > +} > + > /* Mark bot aggregate and scalar lattices as containing an unknown variable, > return true is any of them has not been marked as such so far. */ > > @@ -771,6 +802,7 @@ set_all_contains_variable (struct ipcp_p > ret = plats->itself.set_contains_variable (); > ret |= plats->ctxlat.set_contains_variable (); > ret |= set_agg_lats_contain_variable (plats); > + ret |= set_alignment_to_bottom (plats); > return ret; > } > > @@ -807,6 +839,7 @@ initialize_node_lattices (struct cgraph_ > plats->itself.set_to_bottom (); > plats->ctxlat.set_to_bottom (); > set_agg_lats_to_bottom (plats); > + set_alignment_to_bottom (plats); > } > else > set_all_contains_variable (plats); > @@ -1372,6 +1405,80 @@ propagate_context_accross_jump_function > return ret; > } > > +/* Propagate alignments accross jump function JFUNC that is associated with > + edge CS and update DEST_LAT accordingly. */ > + > +static bool > +propagate_alignment_accross_jump_function (struct cgraph_edge *cs, > + struct ipa_jump_func *jfunc, > + struct ipcp_param_lattices *dest_lat) > +{ > + if (alignment_bottom_p (dest_lat)) > + return false; > + > + ipa_alignment cur; > + cur.known = false; > + if (jfunc->alignment.known) > + cur = jfunc->alignment; > + else if (jfunc->type == IPA_JF_PASS_THROUGH > + || jfunc->type == IPA_JF_ANCESTOR) > + { > + struct ipa_node_params *caller_info = IPA_NODE_REF (cs->caller); > + struct ipcp_param_lattices *src_lats; > + HOST_WIDE_INT offset = 0; > + int src_idx; > + > + if (jfunc->type == IPA_JF_PASS_THROUGH) > + { > + enum tree_code op = ipa_get_jf_pass_through_operation (jfunc); > + if (op != NOP_EXPR) > + { > + if (op != POINTER_PLUS_EXPR > + && op != PLUS_EXPR > + && op != MINUS_EXPR) > + goto prop_fail; > + tree operand = ipa_get_jf_pass_through_operand (jfunc); > + if (!tree_fits_shwi_p (operand)) > + goto prop_fail; > + offset = tree_to_shwi (operand); > + } > + src_idx = ipa_get_jf_pass_through_formal_id (jfunc); > + } > + else > + { > + src_idx = ipa_get_jf_ancestor_formal_id (jfunc); > + offset = ipa_get_jf_ancestor_offset (jfunc); > + } > + > + src_lats = ipa_get_parm_lattices (caller_info, src_idx); > + if (!src_lats->alignment.known > + || alignment_bottom_p (src_lats)) > + goto prop_fail; > + > + cur = src_lats->alignment; > + > + if (offset != 0 > + && (offset % cur.align != 0)) > + goto prop_fail; why fail? shouldn't you simply add offset to cur.bitpos? > + } > + > + if (cur.known) > + { > + if (!dest_lat->alignment.known) > + { > + dest_lat->alignment = cur; > + return true; > + } > + else if (dest_lat->alignment.align == cur.align > + && dest_lat->alignment.bitpos == cur.bitpos) > + return false; > + } > + > + prop_fail: > + set_alignment_to_bottom (dest_lat); > + return true; > +} > + > /* If DEST_PLATS already has aggregate items, check that aggs_by_ref matches > NEW_AGGS_BY_REF and if not, mark all aggs as bottoms and return true (in all > other cases, return false). If there are no aggregate items, set > @@ -1708,6 +1815,8 @@ propagate_constants_accross_call (struct > &dest_plats->itself); > ret |= propagate_context_accross_jump_function (cs, jump_func, i, > &dest_plats->ctxlat); > + ret |= propagate_alignment_accross_jump_function (cs, jump_func, > + dest_plats); > ret |= propagate_aggs_accross_jump_function (cs, jump_func, > dest_plats); > } > @@ -4152,6 +4261,63 @@ ipcp_decision_stage (struct ipa_topo_inf > } > } > > +/* Look up all alignment information that we have discovered and copy it over > + to the transformation summary. */ > + > +static void > +ipcp_store_alignment_results (void) > +{ > + cgraph_node *node; > + > + FOR_EACH_FUNCTION_WITH_GIMPLE_BODY (node) > + { > + ipa_node_params *info = IPA_NODE_REF (node); > + bool dumped_sth = false; > + bool found_useful_result = false; > + > + if (info->ipcp_orig_node) > + info = IPA_NODE_REF (info->ipcp_orig_node); > + > + unsigned count = ipa_get_param_count (info); > + for (unsigned i = 0; i < count ; i++) > + { > + ipcp_param_lattices *plats = ipa_get_parm_lattices (info, i); > + if (plats->alignment.known > + && plats->alignment.align > 0) > + { > + found_useful_result = true; > + break; > + } > + } > + if (!found_useful_result) > + continue; > + > + ipcp_grow_transformations_if_necessary (); > + ipcp_transformation_summary *ts = ipcp_get_transformation_summary (node); > + vec_safe_reserve_exact (ts->alignments, count); > + > + for (unsigned i = 0; i < count ; i++) > + { > + ipcp_param_lattices *plats = ipa_get_parm_lattices (info, i); > + > + if (plats->alignment.align == 0) > + plats->alignment.known = false; > + > + ts->alignments->quick_push (plats->alignment); > + if (!dump_file || !plats->alignment.known) > + continue; > + if (!dumped_sth) > + { > + fprintf (dump_file, "Propagated alignment info for function %s/%i:\n", > + node->name (), node->order); > + dumped_sth = true; > + } > + fprintf (dump_file, " param %i: align: %u, bitpos: %u\n", > + i, plats->alignment.align, plats->alignment.bitpos); > + } > + } > +} > + > /* The IPCP driver. */ > > static unsigned int > @@ -4193,6 +4359,8 @@ ipcp_driver (void) > ipcp_propagate_stage (&topo); > /* Decide what constant propagation and cloning should be performed. */ > ipcp_decision_stage (&topo); > + /* Store results of alignment propagation. */ > + ipcp_store_alignment_results (); > > /* Free all IPCP structures. */ > free_toporder_info (&topo); > @@ -4265,9 +4433,9 @@ public: > ipcp_generate_summary, /* generate_summary */ > ipcp_write_summary, /* write_summary */ > ipcp_read_summary, /* read_summary */ > - ipa_prop_write_all_agg_replacement, /* > + ipcp_write_transformation_summaries, /* > write_optimization_summary */ > - ipa_prop_read_all_agg_replacement, /* > + ipcp_read_transformation_summaries, /* > read_optimization_summary */ > NULL, /* stmt_fixup */ > 0, /* function_transform_todo_flags_start */ > Index: src/gcc/ipa-prop.c > =================================================================== > --- src.orig/gcc/ipa-prop.c 2014-11-14 22:06:06.240031030 +0100 > +++ src/gcc/ipa-prop.c 2014-11-15 01:40:05.564520331 +0100 > @@ -133,8 +133,8 @@ struct func_body_info > > /* Vector where the parameter infos are actually stored. */ > vec<ipa_node_params> ipa_node_params_vector; > -/* Vector of known aggregate values in cloned nodes. */ > -vec<ipa_agg_replacement_value_p, va_gc> *ipa_node_agg_replacements; > +/* Vector of IPA-CP transformation data for each clone. */ > +vec<ipcp_transformation_summary, va_gc> *ipcp_transformations; > /* Vector where the parameter infos are actually stored. */ > vec<ipa_edge_args, va_gc> *ipa_edge_args_vector; > > @@ -374,6 +374,15 @@ ipa_print_node_jump_functions_for_edge ( > fprintf (f, " Context: "); > ctx->dump (dump_file); > } > + > + if (jump_func->alignment.known) > + { > + fprintf (f, " Alignment: %u, bitpos: %u\n", > + jump_func->alignment.align, > + jump_func->alignment.bitpos); > + } > + else > + fprintf (f, " Unknown alignment\n"); > } > } > > @@ -446,6 +455,15 @@ ipa_print_all_jump_functions (FILE *f) > } > } > > +/* Set jfunc to be a know-really nothing jump function. */ > + > +static void > +ipa_set_jf_unknown (struct ipa_jump_func *jfunc) > +{ > + jfunc->type = IPA_JF_UNKNOWN; > + jfunc->alignment.known = false; > +} > + > /* Set JFUNC to be a copy of another jmp (to be used by jump function > combination code). The two functions will share their rdesc. */ > > @@ -750,7 +768,7 @@ detect_type_change_from_memory_writes (t > if (!tci.type_maybe_changed) > return false; > > - jfunc->type = IPA_JF_UNKNOWN; > + ipa_set_jf_unknown (jfunc); > return true; > } > > @@ -1716,6 +1734,24 @@ ipa_compute_jump_functions_for_edge (str > useful_context = true; > } > > + if (POINTER_TYPE_P (TREE_TYPE(arg))) > + { > + unsigned HOST_WIDE_INT hwi_bitpos; > + unsigned align; > + > + if (get_pointer_alignment_1 (arg, &align, &hwi_bitpos) > + && (hwi_bitpos / BITS_PER_UNIT) < UINT_MAX) hwi_bitpos is never >= align. A better thing to check here is align > BITS_PER_UNIT. It's not useful to propagate "unaligned", no? > + { > + jfunc->alignment.known = true; > + jfunc->alignment.align = align; > + jfunc->alignment.bitpos = hwi_bitpos / BITS_PER_UNIT; > + } > + else > + gcc_assert (!jfunc->alignment.known); > + } > + else > + gcc_assert (!jfunc->alignment.known); > + > if (is_gimple_ip_invariant (arg)) > ipa_set_jf_constant (jfunc, arg, cs); > else if (!is_gimple_reg_type (TREE_TYPE (arg)) > @@ -2411,7 +2447,7 @@ update_jump_functions_after_inlining (st > don't. */ > if (dst_fid >= ipa_get_cs_argument_count (top)) > { > - dst->type = IPA_JF_UNKNOWN; > + ipa_set_jf_unknown (dst); > continue; > } > > @@ -2465,7 +2501,7 @@ update_jump_functions_after_inlining (st > src->value.ancestor.agg_preserved; > } > else > - dst->type = IPA_JF_UNKNOWN; > + ipa_set_jf_unknown (dst); > } > else if (dst->type == IPA_JF_PASS_THROUGH) > { > @@ -2503,7 +2539,7 @@ update_jump_functions_after_inlining (st > switch (src->type) > { > case IPA_JF_UNKNOWN: > - dst->type = IPA_JF_UNKNOWN; > + ipa_set_jf_unknown (dst); > break; > case IPA_JF_CONST: > ipa_set_jf_cst_copy (dst, src); > @@ -2557,7 +2593,7 @@ update_jump_functions_after_inlining (st > } > } > else > - dst->type = IPA_JF_UNKNOWN; > + ipa_set_jf_unknown (dst); > } > } > } > @@ -3327,18 +3363,24 @@ ipa_free_all_node_params (void) > ipa_node_params_vector.release (); > } > > +/* Grow ipcp_transformations if necessary. */ > + > +void > +ipcp_grow_transformations_if_necessary (void) > +{ > + if (vec_safe_length (ipcp_transformations) > + <= (unsigned) symtab->cgraph_max_uid) > + vec_safe_grow_cleared (ipcp_transformations, symtab->cgraph_max_uid + 1); > +} > + > /* Set the aggregate replacements of NODE to be AGGVALS. */ > > void > ipa_set_node_agg_value_chain (struct cgraph_node *node, > struct ipa_agg_replacement_value *aggvals) > { > - if (vec_safe_length (ipa_node_agg_replacements) > - <= (unsigned) symtab->cgraph_max_uid) > - vec_safe_grow_cleared (ipa_node_agg_replacements, > - symtab->cgraph_max_uid + 1); > - > - (*ipa_node_agg_replacements)[node->uid] = aggvals; > + ipcp_grow_transformations_if_necessary (); > + (*ipcp_transformations)[node->uid].agg_values = aggvals; > } > > /* Hook that is called by cgraph.c when an edge is removed. */ > @@ -3379,8 +3421,11 @@ ipa_node_removal_hook (struct cgraph_nod > /* During IPA-CP updating we can be called on not-yet analyze clones. */ > if (ipa_node_params_vector.length () > (unsigned)node->uid) > ipa_free_node_params_substructures (IPA_NODE_REF (node)); > - if (vec_safe_length (ipa_node_agg_replacements) > (unsigned)node->uid) > - (*ipa_node_agg_replacements)[(unsigned)node->uid] = NULL; > + if (vec_safe_length (ipcp_transformations) > (unsigned)node->uid) > + { > + (*ipcp_transformations)[(unsigned)node->uid].agg_values = NULL; > + (*ipcp_transformations)[(unsigned)node->uid].alignments = NULL; > + } > } > > /* Hook that is called by cgraph.c when an edge is duplicated. */ > @@ -3508,21 +3553,35 @@ ipa_node_duplication_hook (struct cgraph > new_info->node_enqueued = old_info->node_enqueued; > > old_av = ipa_get_agg_replacements_for_node (src); > - if (!old_av) > - return; > - > - new_av = NULL; > - while (old_av) > + if (old_av) > { > - struct ipa_agg_replacement_value *v; > + new_av = NULL; > + while (old_av) > + { > + struct ipa_agg_replacement_value *v; > + > + v = ggc_alloc<ipa_agg_replacement_value> (); > + memcpy (v, old_av, sizeof (*v)); > + v->next = new_av; > + new_av = v; > + old_av = old_av->next; > + } > + ipa_set_node_agg_value_chain (dst, new_av); > + } > + > + ipcp_transformation_summary *src_trans = ipcp_get_transformation_summary (src); > > - v = ggc_alloc<ipa_agg_replacement_value> (); > - memcpy (v, old_av, sizeof (*v)); > - v->next = new_av; > - new_av = v; > - old_av = old_av->next; > + if (src_trans && vec_safe_length (src_trans->alignments) > 0) > + { > + ipcp_grow_transformations_if_necessary (); > + src_trans = ipcp_get_transformation_summary (src); > + const vec<ipa_alignment, va_gc> *src_alignments = src_trans->alignments; > + vec<ipa_alignment, va_gc> *&dst_alignments > + = ipcp_get_transformation_summary (dst)->alignments; > + vec_safe_reserve_exact (dst_alignments, src_alignments->length ()); > + for (unsigned i = 0; i < src_alignments->length (); ++i) > + dst_alignments->quick_push ((*src_alignments)[i]); > } > - ipa_set_node_agg_value_chain (dst, new_av); > } > > > @@ -4450,6 +4509,15 @@ ipa_write_jump_function (struct output_b > streamer_write_uhwi (ob, item->offset); > stream_write_tree (ob, item->value, true); > } > + > + bp = bitpack_create (ob->main_stream); > + bp_pack_value (&bp, jump_func->alignment.known, 1); > + streamer_write_bitpack (&bp); > + if (jump_func->alignment.known) > + { > + streamer_write_uhwi (ob, jump_func->alignment.align); > + streamer_write_uhwi (ob, jump_func->alignment.bitpos); > + } > } > > /* Read in jump function JUMP_FUNC from IB. */ > @@ -4468,7 +4536,7 @@ ipa_read_jump_function (struct lto_input > switch (jftype) > { > case IPA_JF_UNKNOWN: > - jump_func->type = IPA_JF_UNKNOWN; > + ipa_set_jf_unknown (jump_func); > break; > case IPA_JF_CONST: > ipa_set_jf_constant (jump_func, stream_read_tree (ib, data_in), cs); > @@ -4515,6 +4583,17 @@ ipa_read_jump_function (struct lto_input > item.value = stream_read_tree (ib, data_in); > jump_func->agg.items->quick_push (item); > } > + > + struct bitpack_d bp = streamer_read_bitpack (ib); > + bool alignment_known = bp_unpack_value (&bp, 1); > + if (alignment_known) > + { > + jump_func->alignment.known = true; > + jump_func->alignment.align = streamer_read_uhwi (ib); > + jump_func->alignment.bitpos = streamer_read_uhwi (ib); > + } > + else > + jump_func->alignment.known = false; > } > > /* Stream out parts of cgraph_indirect_call_info corresponding to CS that are > @@ -4826,7 +4905,7 @@ ipa_update_after_lto_read (void) > } > > void > -write_agg_replacement_chain (struct output_block *ob, struct cgraph_node *node) > +write_ipcp_transformation_info (output_block *ob, cgraph_node *node) > { > int node_ref; > unsigned int count = 0; > @@ -4854,14 +4933,37 @@ write_agg_replacement_chain (struct outp > bp_pack_value (&bp, av->by_ref, 1); > streamer_write_bitpack (&bp); > } > + > + ipcp_transformation_summary *ts = ipcp_get_transformation_summary (node); > + if (ts && vec_safe_length (ts->alignments) > 0) > + { > + count = ts->alignments->length (); > + > + streamer_write_uhwi (ob, count); > + for (unsigned i = 0; i < count; ++i) > + { > + ipa_alignment *parm_al = &(*ts->alignments)[i]; > + > + struct bitpack_d bp; > + bp = bitpack_create (ob->main_stream); > + bp_pack_value (&bp, parm_al->known, 1); > + streamer_write_bitpack (&bp); > + if (parm_al->known) > + { > + streamer_write_uhwi (ob, parm_al->align); > + streamer_write_uhwi (ob, parm_al->bitpos); > + } > + } > + } > + else > + streamer_write_uhwi (ob, 0); > } > > /* Stream in the aggregate value replacement chain for NODE from IB. */ > > static void > -read_agg_replacement_chain (struct lto_input_block *ib, > - struct cgraph_node *node, > - struct data_in *data_in) > +read_ipcp_transformation_info (lto_input_block *ib, cgraph_node *node, > + data_in *data_in) > { > struct ipa_agg_replacement_value *aggvals = NULL; > unsigned int count, i; > @@ -4882,12 +4984,35 @@ read_agg_replacement_chain (struct lto_i > aggvals = av; > } > ipa_set_node_agg_value_chain (node, aggvals); > + > + count = streamer_read_uhwi (ib); > + if (count > 0) > + { > + ipcp_grow_transformations_if_necessary (); > + > + ipcp_transformation_summary *ts = ipcp_get_transformation_summary (node); > + vec_safe_grow_cleared (ts->alignments, count); > + > + for (i = 0; i < count; i++) > + { > + ipa_alignment *parm_al; > + parm_al = &(*ts->alignments)[i]; > + struct bitpack_d bp; > + bp = streamer_read_bitpack (ib); > + parm_al->known = bp_unpack_value (&bp, 1); > + if (parm_al->known) > + { > + parm_al->align = streamer_read_uhwi (ib); > + parm_al->bitpos = streamer_read_uhwi (ib); > + } > + } > + } > } > > /* Write all aggregate replacement for nodes in set. */ > > void > -ipa_prop_write_all_agg_replacement (void) > +ipcp_write_transformation_summaries (void) > { > struct cgraph_node *node; > struct output_block *ob; > @@ -4895,9 +5020,6 @@ ipa_prop_write_all_agg_replacement (void > lto_symtab_encoder_iterator lsei; > lto_symtab_encoder_t encoder; > > - if (!ipa_node_agg_replacements) > - return; > - > ob = create_output_block (LTO_section_ipcp_transform); > encoder = ob->decl_state->symtab_node_encoder; > ob->symbol = NULL; > @@ -4905,8 +5027,7 @@ ipa_prop_write_all_agg_replacement (void > lsei_next_function_in_partition (&lsei)) > { > node = lsei_cgraph_node (lsei); > - if (node->has_gimple_body_p () > - && ipa_get_agg_replacements_for_node (node) != NULL) > + if (node->has_gimple_body_p ()) > count++; > } > > @@ -4916,9 +5037,8 @@ ipa_prop_write_all_agg_replacement (void > lsei_next_function_in_partition (&lsei)) > { > node = lsei_cgraph_node (lsei); > - if (node->has_gimple_body_p () > - && ipa_get_agg_replacements_for_node (node) != NULL) > - write_agg_replacement_chain (ob, node); > + if (node->has_gimple_body_p ()) > + write_ipcp_transformation_info (ob, node); > } > streamer_write_char_stream (ob->main_stream, 0); > produce_asm (ob, NULL); > @@ -4960,7 +5080,7 @@ read_replacements_section (struct lto_fi > node = dyn_cast<cgraph_node *> (lto_symtab_encoder_deref (encoder, > index)); > gcc_assert (node->definition); > - read_agg_replacement_chain (&ib_main, node, data_in); > + read_ipcp_transformation_info (&ib_main, node, data_in); > } > lto_free_section_data (file_data, LTO_section_jump_functions, NULL, data, > len); > @@ -4970,7 +5090,7 @@ read_replacements_section (struct lto_fi > /* Read IPA-CP aggregate replacements. */ > > void > -ipa_prop_read_all_agg_replacement (void) > +ipcp_read_transformation_summaries (void) > { > struct lto_file_decl_data **file_data_vec = lto_get_file_decl_data (); > struct lto_file_decl_data *file_data; > @@ -5137,6 +5257,67 @@ ipcp_modif_dom_walker::before_dom_childr > > } > > +/* Update alignment of formal parameters as described in > + ipcp_transformation_summary. */ > + > +static void > +ipcp_update_alignments (struct cgraph_node *node) > +{ > + tree fndecl = node->decl; > + tree parm = DECL_ARGUMENTS (fndecl); > + tree next_parm = parm; > + ipcp_transformation_summary *ts = ipcp_get_transformation_summary (node); > + if (!ts || vec_safe_length (ts->alignments) == 0) > + return; > + const vec<ipa_alignment, va_gc> &alignments = *ts->alignments; > + unsigned count = alignments.length (); > + > + for (unsigned i = 0; i < count; ++i, parm = next_parm) > + { > + if (node->clone.combined_args_to_skip > + && bitmap_bit_p (node->clone.combined_args_to_skip, i)) > + continue; > + gcc_checking_assert (parm); > + next_parm = DECL_CHAIN (parm); > + > + if (!alignments[i].known || !is_gimple_reg (parm)) > + continue; > + tree ddef = ssa_default_def (DECL_STRUCT_FUNCTION (node->decl), parm); > + if (!ddef) > + continue; > + > + if (dump_file) > + fprintf (dump_file, " Adjusting alignment of param %u to %u, " > + "bitpos to %u\n", i, alignments[i].align, > + alignments[i].bitpos); > + > + struct ptr_info_def *pi = get_ptr_info (ddef); > + gcc_checking_assert (pi); > + unsigned old_align; > + unsigned old_misalign; > + bool old_known = get_ptr_info_alignment (pi, &old_align, &old_misalign); > + > + if (old_known) > + { I _think_ nothing sets alignment info for param default defs - did you hit this case? > + if (old_align >= alignments[i].align) > + { > + if (dump_file) > + fprintf (dump_file, " But the alignment has already " > + "been %u.\n", old_align); > + continue; > + } > + if (old_misalign != 0) > + { > + if (dump_file) > + fprintf (dump_file, " But there has been a noted " > + "misalignment.\n"); > + continue; > + } I think you want to trust both - thus simply take what is computed from IPA-CP even if old_misalign is != 0. I didn't look at the "IPA" part, but otherwise it looks good to me apart from the stuff above. Thanks, Richard. > + } > + set_ptr_info_alignment (pi, alignments[i].align, alignments[i].bitpos); > + } > +} > + > /* IPCP transformation phase doing propagation of aggregate values. */ > > unsigned int > @@ -5155,6 +5336,7 @@ ipcp_transform_function (struct cgraph_n > fprintf (dump_file, "Modification phase of node %s/%i\n", > node->name (), node->order); > > + ipcp_update_alignments (node); > aggval = ipa_get_agg_replacements_for_node (node); > if (!aggval) > return 0; > @@ -5184,7 +5366,8 @@ ipcp_transform_function (struct cgraph_n > free_ipa_bb_info (bi); > fbi.bb_infos.release (); > free_dominance_info (CDI_DOMINATORS); > - (*ipa_node_agg_replacements)[node->uid] = NULL; > + (*ipcp_transformations)[node->uid].agg_values = NULL; > + (*ipcp_transformations)[node->uid].alignments = NULL; > descriptors.release (); > > if (!something_changed) > Index: src/gcc/ipa-prop.h > =================================================================== > --- src.orig/gcc/ipa-prop.h 2014-11-14 22:06:06.240031030 +0100 > +++ src/gcc/ipa-prop.h 2014-11-15 01:40:06.924520383 +0100 > @@ -155,6 +155,16 @@ struct GTY(()) ipa_agg_jump_function > > typedef struct ipa_agg_jump_function *ipa_agg_jump_function_p; > > +/* Info about poiner alignments. */ > +struct GTY(()) ipa_alignment > +{ > + /* The data fields below are valid only if known is true. */ > + bool known; > + /* See get_pointer_alignment_1 for description of these two. */ > + unsigned bitpos; > + unsigned align; > +}; > + > /* A jump function for a callsite represents the values passed as actual > arguments of the callsite. See enum jump_func_type for the various > types of jump functions supported. */ > @@ -164,6 +174,9 @@ struct GTY (()) ipa_jump_func > description. */ > struct ipa_agg_jump_function agg; > > + /* Information about alignment of pointers. */ > + struct ipa_alignment alignment; > + > enum jump_func_type type; > /* Represents a value of a jump function. pass_through is used only in jump > function context. constant represents the actual constant in constant jump > @@ -413,10 +426,19 @@ struct GTY(()) ipa_agg_replacement_value > bool by_ref; > }; > > -typedef struct ipa_agg_replacement_value *ipa_agg_replacement_value_p; > +/* Structure holding information for the transformation phase of IPA-CP. */ > + > +struct GTY(()) ipcp_transformation_summary > +{ > + /* Linked list of known aggregate values. */ > + ipa_agg_replacement_value *agg_values; > + /* Alignemnt information for pointers. */ > + vec<ipa_alignment, va_gc> *alignments; > +}; > > void ipa_set_node_agg_value_chain (struct cgraph_node *node, > struct ipa_agg_replacement_value *aggvals); > +void ipcp_grow_transformations_if_necessary (void); > > /* ipa_edge_args stores information related to a callsite and particularly its > arguments. It can be accessed by the IPA_EDGE_REF macro. */ > @@ -462,8 +484,8 @@ ipa_get_ith_polymorhic_call_context (str > > /* Vector where the parameter infos are actually stored. */ > extern vec<ipa_node_params> ipa_node_params_vector; > -/* Vector of known aggregate values in cloned nodes. */ > -extern GTY(()) vec<ipa_agg_replacement_value_p, va_gc> *ipa_node_agg_replacements; > +/* Vector of IPA-CP transformation data for each clone. */ > +extern GTY(()) vec<ipcp_transformation_summary, va_gc> *ipcp_transformations; > /* Vector where the parameter infos are actually stored. */ > extern GTY(()) vec<ipa_edge_args, va_gc> *ipa_edge_args_vector; > > @@ -521,14 +543,21 @@ ipa_edge_args_info_available_for_edge_p > return ((unsigned) edge->uid < vec_safe_length (ipa_edge_args_vector)); > } > > +static inline ipcp_transformation_summary * > +ipcp_get_transformation_summary (cgraph_node *node) > +{ > + if ((unsigned) node->uid >= vec_safe_length (ipcp_transformations)) > + return NULL; > + return &(*ipcp_transformations)[node->uid]; > +} > + > /* Return the aggregate replacements for NODE, if there are any. */ > > static inline struct ipa_agg_replacement_value * > -ipa_get_agg_replacements_for_node (struct cgraph_node *node) > +ipa_get_agg_replacements_for_node (cgraph_node *node) > { > - if ((unsigned) node->uid >= vec_safe_length (ipa_node_agg_replacements)) > - return NULL; > - return (*ipa_node_agg_replacements)[node->uid]; > + ipcp_transformation_summary *ts = ipcp_get_transformation_summary (node); > + return ts ? ts->agg_values : NULL; > } > > /* Function formal parameters related computations. */ > @@ -657,8 +686,8 @@ void ipa_dump_agg_replacement_values (FI > struct ipa_agg_replacement_value *av); > void ipa_prop_write_jump_functions (void); > void ipa_prop_read_jump_functions (void); > -void ipa_prop_write_all_agg_replacement (void); > -void ipa_prop_read_all_agg_replacement (void); > +void ipcp_write_transformation_summaries (void); > +void ipcp_read_transformation_summaries (void); > void ipa_update_after_lto_read (void); > int ipa_get_param_decl_index (struct ipa_node_params *, tree); > tree ipa_value_from_jfunc (struct ipa_node_params *info, > Index: src/gcc/testsuite/gcc.dg/ipa/propalign-1.c > =================================================================== > --- /dev/null 1970-01-01 00:00:00.000000000 +0000 > +++ src/gcc/testsuite/gcc.dg/ipa/propalign-1.c 2014-11-15 00:38:53.420380387 +0100 > @@ -0,0 +1,32 @@ > +/* { dg-do compile } */ > +/* { dg-options "-O2 -fno-early-inlining -fdump-ipa-cp -fdump-tree-optimized" } */ > + > +#include <stdint.h> > + > +extern int fail_the_test(void *); > +extern int pass_the_test(void *); > +extern int diversion (void *); > + > +static int __attribute__((noinline)) > +foo (void *p) > +{ > + uintptr_t a = (uintptr_t) p; > + > + if (a % 4) > + return fail_the_test (p); > + else > + return pass_the_test (p); > +} > + > +int > +bar (void) > +{ > + double buf[8] __attribute__ ((__aligned__(__BIGGEST_ALIGNMENT__))); > + return foo (&buf); > +} > + > + > +/* { dg-final { scan-ipa-dump "Adjusting alignment of param" "cp" } } */ > +/* { dg-final { scan-tree-dump-not "fail_the_test" "optimized" } } */ > +/* { dg-final { cleanup-ipa-dump "cp" } } */ > +/* { dg-final { cleanup-tree-dump "optimized" } } */ > Index: src/gcc/testsuite/gcc.dg/ipa/propalign-2.c > =================================================================== > --- /dev/null 1970-01-01 00:00:00.000000000 +0000 > +++ src/gcc/testsuite/gcc.dg/ipa/propalign-2.c 2014-11-15 00:38:53.420380387 +0100 > @@ -0,0 +1,58 @@ > +/* { dg-do compile } */ > +/* { dg-options "-O2 -fno-early-inlining -fdump-ipa-cp -fdump-tree-optimized" } */ > + > +#include <stdint.h> > + > +extern int fail_the_test(void *); > +extern int pass_the_test(void *); > +extern int diversion (void *); > + > +struct somestruct > +{ > + void *whee; > + void *oops; > +}; > + > +struct container > +{ > + struct somestruct first; > + struct somestruct buf[32]; > +}; > + > +static int __attribute__((noinline)) > +foo (void *p) > +{ > + uintptr_t a = (uintptr_t) p; > + > + if (a % 4) > + return fail_the_test (p); > + else > + return pass_the_test (p); > +} > + > +int > +bar (void) > +{ > + struct container c; > + return foo (c.buf); > +} > + > + > +static int > +through (struct somestruct *p) > +{ > + diversion (p); > + return foo (&p[16]); > +} > + > +int > +bar2 (void) > +{ > + struct container c; > + through (c.buf); > +} > + > +/* { dg-final { scan-ipa-dump "Adjusting alignment of param" "cp" } } */ > +/* { dg-final { scan-tree-dump-not "fail_the_test" "optimized" } } */ > +/* { dg-final { cleanup-ipa-dump "cp" } } */ > +/* { dg-final { cleanup-tree-dump "optimized" } } */
Index: src/gcc/ipa-cp.c =================================================================== --- src.orig/gcc/ipa-cp.c 2014-11-14 22:06:06.240031030 +0100 +++ src/gcc/ipa-cp.c 2014-11-15 01:40:38.260521577 +0100 @@ -262,6 +262,9 @@ public: ipcp_lattice<ipa_polymorphic_call_context> ctxlat; /* Lattices describing aggregate parts. */ ipcp_agg_lattice *aggs; + /* Alignment information. Very basic one value lattice where !known means + TOP and zero alignment bottom. */ + ipa_alignment alignment; /* Number of aggregate lattices */ int aggs_count; /* True if aggregate data were passed by reference (as opposed to by @@ -444,6 +447,13 @@ print_all_lattices (FILE * f, bool dump_ plats->itself.print (f, dump_sources, dump_benefits); fprintf (f, " ctxs: "); plats->ctxlat.print (f, dump_sources, dump_benefits); + if (plats->alignment.known && plats->alignment.align > 0) + fprintf (f, " Alignment %u, misaglignment %u\n", + plats->alignment.align, plats->alignment.bitpos); + else if (plats->alignment.known) + fprintf (f, " Alignment unusable\n"); + else + fprintf (f, " Alignment unknown\n"); if (plats->virt_call) fprintf (f, " virt_call flag set\n"); @@ -761,6 +771,27 @@ set_agg_lats_contain_variable (struct ip return ret; } +/* Return true if alignemnt informatin in PLATS is known to be unusable. */ + +static inline bool +alignment_bottom_p (ipcp_param_lattices *plats) +{ + return plats->alignment.known && (plats->alignment.align == 0); +} + +/* Set alignment information in PLATS to unusable. Return true if it + previously was usable or unknown. */ + +static inline bool +set_alignment_to_bottom (ipcp_param_lattices *plats) +{ + if (alignment_bottom_p (plats)) + return false; + plats->alignment.known = true; + plats->alignment.align = 0; + return true; +} + /* Mark bot aggregate and scalar lattices as containing an unknown variable, return true is any of them has not been marked as such so far. */ @@ -771,6 +802,7 @@ set_all_contains_variable (struct ipcp_p ret = plats->itself.set_contains_variable (); ret |= plats->ctxlat.set_contains_variable (); ret |= set_agg_lats_contain_variable (plats); + ret |= set_alignment_to_bottom (plats); return ret; } @@ -807,6 +839,7 @@ initialize_node_lattices (struct cgraph_ plats->itself.set_to_bottom (); plats->ctxlat.set_to_bottom (); set_agg_lats_to_bottom (plats); + set_alignment_to_bottom (plats); } else set_all_contains_variable (plats); @@ -1372,6 +1405,80 @@ propagate_context_accross_jump_function return ret; } +/* Propagate alignments accross jump function JFUNC that is associated with + edge CS and update DEST_LAT accordingly. */ + +static bool +propagate_alignment_accross_jump_function (struct cgraph_edge *cs, + struct ipa_jump_func *jfunc, + struct ipcp_param_lattices *dest_lat) +{ + if (alignment_bottom_p (dest_lat)) + return false; + + ipa_alignment cur; + cur.known = false; + if (jfunc->alignment.known) + cur = jfunc->alignment; + else if (jfunc->type == IPA_JF_PASS_THROUGH + || jfunc->type == IPA_JF_ANCESTOR) + { + struct ipa_node_params *caller_info = IPA_NODE_REF (cs->caller); + struct ipcp_param_lattices *src_lats; + HOST_WIDE_INT offset = 0; + int src_idx; + + if (jfunc->type == IPA_JF_PASS_THROUGH) + { + enum tree_code op = ipa_get_jf_pass_through_operation (jfunc); + if (op != NOP_EXPR) + { + if (op != POINTER_PLUS_EXPR + && op != PLUS_EXPR + && op != MINUS_EXPR) + goto prop_fail; + tree operand = ipa_get_jf_pass_through_operand (jfunc); + if (!tree_fits_shwi_p (operand)) + goto prop_fail; + offset = tree_to_shwi (operand); + } + src_idx = ipa_get_jf_pass_through_formal_id (jfunc); + } + else + { + src_idx = ipa_get_jf_ancestor_formal_id (jfunc); + offset = ipa_get_jf_ancestor_offset (jfunc); + } + + src_lats = ipa_get_parm_lattices (caller_info, src_idx); + if (!src_lats->alignment.known + || alignment_bottom_p (src_lats)) + goto prop_fail; + + cur = src_lats->alignment; + + if (offset != 0 + && (offset % cur.align != 0)) + goto prop_fail; + } + + if (cur.known) + { + if (!dest_lat->alignment.known) + { + dest_lat->alignment = cur; + return true; + } + else if (dest_lat->alignment.align == cur.align + && dest_lat->alignment.bitpos == cur.bitpos) + return false; + } + + prop_fail: + set_alignment_to_bottom (dest_lat); + return true; +} + /* If DEST_PLATS already has aggregate items, check that aggs_by_ref matches NEW_AGGS_BY_REF and if not, mark all aggs as bottoms and return true (in all other cases, return false). If there are no aggregate items, set @@ -1708,6 +1815,8 @@ propagate_constants_accross_call (struct &dest_plats->itself); ret |= propagate_context_accross_jump_function (cs, jump_func, i, &dest_plats->ctxlat); + ret |= propagate_alignment_accross_jump_function (cs, jump_func, + dest_plats); ret |= propagate_aggs_accross_jump_function (cs, jump_func, dest_plats); } @@ -4152,6 +4261,63 @@ ipcp_decision_stage (struct ipa_topo_inf } } +/* Look up all alignment information that we have discovered and copy it over + to the transformation summary. */ + +static void +ipcp_store_alignment_results (void) +{ + cgraph_node *node; + + FOR_EACH_FUNCTION_WITH_GIMPLE_BODY (node) + { + ipa_node_params *info = IPA_NODE_REF (node); + bool dumped_sth = false; + bool found_useful_result = false; + + if (info->ipcp_orig_node) + info = IPA_NODE_REF (info->ipcp_orig_node); + + unsigned count = ipa_get_param_count (info); + for (unsigned i = 0; i < count ; i++) + { + ipcp_param_lattices *plats = ipa_get_parm_lattices (info, i); + if (plats->alignment.known + && plats->alignment.align > 0) + { + found_useful_result = true; + break; + } + } + if (!found_useful_result) + continue; + + ipcp_grow_transformations_if_necessary (); + ipcp_transformation_summary *ts = ipcp_get_transformation_summary (node); + vec_safe_reserve_exact (ts->alignments, count); + + for (unsigned i = 0; i < count ; i++) + { + ipcp_param_lattices *plats = ipa_get_parm_lattices (info, i); + + if (plats->alignment.align == 0) + plats->alignment.known = false; + + ts->alignments->quick_push (plats->alignment); + if (!dump_file || !plats->alignment.known) + continue; + if (!dumped_sth) + { + fprintf (dump_file, "Propagated alignment info for function %s/%i:\n", + node->name (), node->order); + dumped_sth = true; + } + fprintf (dump_file, " param %i: align: %u, bitpos: %u\n", + i, plats->alignment.align, plats->alignment.bitpos); + } + } +} + /* The IPCP driver. */ static unsigned int @@ -4193,6 +4359,8 @@ ipcp_driver (void) ipcp_propagate_stage (&topo); /* Decide what constant propagation and cloning should be performed. */ ipcp_decision_stage (&topo); + /* Store results of alignment propagation. */ + ipcp_store_alignment_results (); /* Free all IPCP structures. */ free_toporder_info (&topo); @@ -4265,9 +4433,9 @@ public: ipcp_generate_summary, /* generate_summary */ ipcp_write_summary, /* write_summary */ ipcp_read_summary, /* read_summary */ - ipa_prop_write_all_agg_replacement, /* + ipcp_write_transformation_summaries, /* write_optimization_summary */ - ipa_prop_read_all_agg_replacement, /* + ipcp_read_transformation_summaries, /* read_optimization_summary */ NULL, /* stmt_fixup */ 0, /* function_transform_todo_flags_start */ Index: src/gcc/ipa-prop.c =================================================================== --- src.orig/gcc/ipa-prop.c 2014-11-14 22:06:06.240031030 +0100 +++ src/gcc/ipa-prop.c 2014-11-15 01:40:05.564520331 +0100 @@ -133,8 +133,8 @@ struct func_body_info /* Vector where the parameter infos are actually stored. */ vec<ipa_node_params> ipa_node_params_vector; -/* Vector of known aggregate values in cloned nodes. */ -vec<ipa_agg_replacement_value_p, va_gc> *ipa_node_agg_replacements; +/* Vector of IPA-CP transformation data for each clone. */ +vec<ipcp_transformation_summary, va_gc> *ipcp_transformations; /* Vector where the parameter infos are actually stored. */ vec<ipa_edge_args, va_gc> *ipa_edge_args_vector; @@ -374,6 +374,15 @@ ipa_print_node_jump_functions_for_edge ( fprintf (f, " Context: "); ctx->dump (dump_file); } + + if (jump_func->alignment.known) + { + fprintf (f, " Alignment: %u, bitpos: %u\n", + jump_func->alignment.align, + jump_func->alignment.bitpos); + } + else + fprintf (f, " Unknown alignment\n"); } } @@ -446,6 +455,15 @@ ipa_print_all_jump_functions (FILE *f) } } +/* Set jfunc to be a know-really nothing jump function. */ + +static void +ipa_set_jf_unknown (struct ipa_jump_func *jfunc) +{ + jfunc->type = IPA_JF_UNKNOWN; + jfunc->alignment.known = false; +} + /* Set JFUNC to be a copy of another jmp (to be used by jump function combination code). The two functions will share their rdesc. */ @@ -750,7 +768,7 @@ detect_type_change_from_memory_writes (t if (!tci.type_maybe_changed) return false; - jfunc->type = IPA_JF_UNKNOWN; + ipa_set_jf_unknown (jfunc); return true; } @@ -1716,6 +1734,24 @@ ipa_compute_jump_functions_for_edge (str useful_context = true; } + if (POINTER_TYPE_P (TREE_TYPE(arg))) + { + unsigned HOST_WIDE_INT hwi_bitpos; + unsigned align; + + if (get_pointer_alignment_1 (arg, &align, &hwi_bitpos) + && (hwi_bitpos / BITS_PER_UNIT) < UINT_MAX) + { + jfunc->alignment.known = true; + jfunc->alignment.align = align; + jfunc->alignment.bitpos = hwi_bitpos / BITS_PER_UNIT; + } + else + gcc_assert (!jfunc->alignment.known); + } + else + gcc_assert (!jfunc->alignment.known); + if (is_gimple_ip_invariant (arg)) ipa_set_jf_constant (jfunc, arg, cs); else if (!is_gimple_reg_type (TREE_TYPE (arg)) @@ -2411,7 +2447,7 @@ update_jump_functions_after_inlining (st don't. */ if (dst_fid >= ipa_get_cs_argument_count (top)) { - dst->type = IPA_JF_UNKNOWN; + ipa_set_jf_unknown (dst); continue; } @@ -2465,7 +2501,7 @@ update_jump_functions_after_inlining (st src->value.ancestor.agg_preserved; } else - dst->type = IPA_JF_UNKNOWN; + ipa_set_jf_unknown (dst); } else if (dst->type == IPA_JF_PASS_THROUGH) { @@ -2503,7 +2539,7 @@ update_jump_functions_after_inlining (st switch (src->type) { case IPA_JF_UNKNOWN: - dst->type = IPA_JF_UNKNOWN; + ipa_set_jf_unknown (dst); break; case IPA_JF_CONST: ipa_set_jf_cst_copy (dst, src); @@ -2557,7 +2593,7 @@ update_jump_functions_after_inlining (st } } else - dst->type = IPA_JF_UNKNOWN; + ipa_set_jf_unknown (dst); } } } @@ -3327,18 +3363,24 @@ ipa_free_all_node_params (void) ipa_node_params_vector.release (); } +/* Grow ipcp_transformations if necessary. */ + +void +ipcp_grow_transformations_if_necessary (void) +{ + if (vec_safe_length (ipcp_transformations) + <= (unsigned) symtab->cgraph_max_uid) + vec_safe_grow_cleared (ipcp_transformations, symtab->cgraph_max_uid + 1); +} + /* Set the aggregate replacements of NODE to be AGGVALS. */ void ipa_set_node_agg_value_chain (struct cgraph_node *node, struct ipa_agg_replacement_value *aggvals) { - if (vec_safe_length (ipa_node_agg_replacements) - <= (unsigned) symtab->cgraph_max_uid) - vec_safe_grow_cleared (ipa_node_agg_replacements, - symtab->cgraph_max_uid + 1); - - (*ipa_node_agg_replacements)[node->uid] = aggvals; + ipcp_grow_transformations_if_necessary (); + (*ipcp_transformations)[node->uid].agg_values = aggvals; } /* Hook that is called by cgraph.c when an edge is removed. */ @@ -3379,8 +3421,11 @@ ipa_node_removal_hook (struct cgraph_nod /* During IPA-CP updating we can be called on not-yet analyze clones. */ if (ipa_node_params_vector.length () > (unsigned)node->uid) ipa_free_node_params_substructures (IPA_NODE_REF (node)); - if (vec_safe_length (ipa_node_agg_replacements) > (unsigned)node->uid) - (*ipa_node_agg_replacements)[(unsigned)node->uid] = NULL; + if (vec_safe_length (ipcp_transformations) > (unsigned)node->uid) + { + (*ipcp_transformations)[(unsigned)node->uid].agg_values = NULL; + (*ipcp_transformations)[(unsigned)node->uid].alignments = NULL; + } } /* Hook that is called by cgraph.c when an edge is duplicated. */ @@ -3508,21 +3553,35 @@ ipa_node_duplication_hook (struct cgraph new_info->node_enqueued = old_info->node_enqueued; old_av = ipa_get_agg_replacements_for_node (src); - if (!old_av) - return; - - new_av = NULL; - while (old_av) + if (old_av) { - struct ipa_agg_replacement_value *v; + new_av = NULL; + while (old_av) + { + struct ipa_agg_replacement_value *v; + + v = ggc_alloc<ipa_agg_replacement_value> (); + memcpy (v, old_av, sizeof (*v)); + v->next = new_av; + new_av = v; + old_av = old_av->next; + } + ipa_set_node_agg_value_chain (dst, new_av); + } + + ipcp_transformation_summary *src_trans = ipcp_get_transformation_summary (src); - v = ggc_alloc<ipa_agg_replacement_value> (); - memcpy (v, old_av, sizeof (*v)); - v->next = new_av; - new_av = v; - old_av = old_av->next; + if (src_trans && vec_safe_length (src_trans->alignments) > 0) + { + ipcp_grow_transformations_if_necessary (); + src_trans = ipcp_get_transformation_summary (src); + const vec<ipa_alignment, va_gc> *src_alignments = src_trans->alignments; + vec<ipa_alignment, va_gc> *&dst_alignments + = ipcp_get_transformation_summary (dst)->alignments; + vec_safe_reserve_exact (dst_alignments, src_alignments->length ()); + for (unsigned i = 0; i < src_alignments->length (); ++i) + dst_alignments->quick_push ((*src_alignments)[i]); } - ipa_set_node_agg_value_chain (dst, new_av); } @@ -4450,6 +4509,15 @@ ipa_write_jump_function (struct output_b streamer_write_uhwi (ob, item->offset); stream_write_tree (ob, item->value, true); } + + bp = bitpack_create (ob->main_stream); + bp_pack_value (&bp, jump_func->alignment.known, 1); + streamer_write_bitpack (&bp); + if (jump_func->alignment.known) + { + streamer_write_uhwi (ob, jump_func->alignment.align); + streamer_write_uhwi (ob, jump_func->alignment.bitpos); + } } /* Read in jump function JUMP_FUNC from IB. */ @@ -4468,7 +4536,7 @@ ipa_read_jump_function (struct lto_input switch (jftype) { case IPA_JF_UNKNOWN: - jump_func->type = IPA_JF_UNKNOWN; + ipa_set_jf_unknown (jump_func); break; case IPA_JF_CONST: ipa_set_jf_constant (jump_func, stream_read_tree (ib, data_in), cs); @@ -4515,6 +4583,17 @@ ipa_read_jump_function (struct lto_input item.value = stream_read_tree (ib, data_in); jump_func->agg.items->quick_push (item); } + + struct bitpack_d bp = streamer_read_bitpack (ib); + bool alignment_known = bp_unpack_value (&bp, 1); + if (alignment_known) + { + jump_func->alignment.known = true; + jump_func->alignment.align = streamer_read_uhwi (ib); + jump_func->alignment.bitpos = streamer_read_uhwi (ib); + } + else + jump_func->alignment.known = false; } /* Stream out parts of cgraph_indirect_call_info corresponding to CS that are @@ -4826,7 +4905,7 @@ ipa_update_after_lto_read (void) } void -write_agg_replacement_chain (struct output_block *ob, struct cgraph_node *node) +write_ipcp_transformation_info (output_block *ob, cgraph_node *node) { int node_ref; unsigned int count = 0; @@ -4854,14 +4933,37 @@ write_agg_replacement_chain (struct outp bp_pack_value (&bp, av->by_ref, 1); streamer_write_bitpack (&bp); } + + ipcp_transformation_summary *ts = ipcp_get_transformation_summary (node); + if (ts && vec_safe_length (ts->alignments) > 0) + { + count = ts->alignments->length (); + + streamer_write_uhwi (ob, count); + for (unsigned i = 0; i < count; ++i) + { + ipa_alignment *parm_al = &(*ts->alignments)[i]; + + struct bitpack_d bp; + bp = bitpack_create (ob->main_stream); + bp_pack_value (&bp, parm_al->known, 1); + streamer_write_bitpack (&bp); + if (parm_al->known) + { + streamer_write_uhwi (ob, parm_al->align); + streamer_write_uhwi (ob, parm_al->bitpos); + } + } + } + else + streamer_write_uhwi (ob, 0); } /* Stream in the aggregate value replacement chain for NODE from IB. */ static void -read_agg_replacement_chain (struct lto_input_block *ib, - struct cgraph_node *node, - struct data_in *data_in) +read_ipcp_transformation_info (lto_input_block *ib, cgraph_node *node, + data_in *data_in) { struct ipa_agg_replacement_value *aggvals = NULL; unsigned int count, i; @@ -4882,12 +4984,35 @@ read_agg_replacement_chain (struct lto_i aggvals = av; } ipa_set_node_agg_value_chain (node, aggvals); + + count = streamer_read_uhwi (ib); + if (count > 0) + { + ipcp_grow_transformations_if_necessary (); + + ipcp_transformation_summary *ts = ipcp_get_transformation_summary (node); + vec_safe_grow_cleared (ts->alignments, count); + + for (i = 0; i < count; i++) + { + ipa_alignment *parm_al; + parm_al = &(*ts->alignments)[i]; + struct bitpack_d bp; + bp = streamer_read_bitpack (ib); + parm_al->known = bp_unpack_value (&bp, 1); + if (parm_al->known) + { + parm_al->align = streamer_read_uhwi (ib); + parm_al->bitpos = streamer_read_uhwi (ib); + } + } + } } /* Write all aggregate replacement for nodes in set. */ void -ipa_prop_write_all_agg_replacement (void) +ipcp_write_transformation_summaries (void) { struct cgraph_node *node; struct output_block *ob; @@ -4895,9 +5020,6 @@ ipa_prop_write_all_agg_replacement (void lto_symtab_encoder_iterator lsei; lto_symtab_encoder_t encoder; - if (!ipa_node_agg_replacements) - return; - ob = create_output_block (LTO_section_ipcp_transform); encoder = ob->decl_state->symtab_node_encoder; ob->symbol = NULL; @@ -4905,8 +5027,7 @@ ipa_prop_write_all_agg_replacement (void lsei_next_function_in_partition (&lsei)) { node = lsei_cgraph_node (lsei); - if (node->has_gimple_body_p () - && ipa_get_agg_replacements_for_node (node) != NULL) + if (node->has_gimple_body_p ()) count++; } @@ -4916,9 +5037,8 @@ ipa_prop_write_all_agg_replacement (void lsei_next_function_in_partition (&lsei)) { node = lsei_cgraph_node (lsei); - if (node->has_gimple_body_p () - && ipa_get_agg_replacements_for_node (node) != NULL) - write_agg_replacement_chain (ob, node); + if (node->has_gimple_body_p ()) + write_ipcp_transformation_info (ob, node); } streamer_write_char_stream (ob->main_stream, 0); produce_asm (ob, NULL); @@ -4960,7 +5080,7 @@ read_replacements_section (struct lto_fi node = dyn_cast<cgraph_node *> (lto_symtab_encoder_deref (encoder, index)); gcc_assert (node->definition); - read_agg_replacement_chain (&ib_main, node, data_in); + read_ipcp_transformation_info (&ib_main, node, data_in); } lto_free_section_data (file_data, LTO_section_jump_functions, NULL, data, len); @@ -4970,7 +5090,7 @@ read_replacements_section (struct lto_fi /* Read IPA-CP aggregate replacements. */ void -ipa_prop_read_all_agg_replacement (void) +ipcp_read_transformation_summaries (void) { struct lto_file_decl_data **file_data_vec = lto_get_file_decl_data (); struct lto_file_decl_data *file_data; @@ -5137,6 +5257,67 @@ ipcp_modif_dom_walker::before_dom_childr } +/* Update alignment of formal parameters as described in + ipcp_transformation_summary. */ + +static void +ipcp_update_alignments (struct cgraph_node *node) +{ + tree fndecl = node->decl; + tree parm = DECL_ARGUMENTS (fndecl); + tree next_parm = parm; + ipcp_transformation_summary *ts = ipcp_get_transformation_summary (node); + if (!ts || vec_safe_length (ts->alignments) == 0) + return; + const vec<ipa_alignment, va_gc> &alignments = *ts->alignments; + unsigned count = alignments.length (); + + for (unsigned i = 0; i < count; ++i, parm = next_parm) + { + if (node->clone.combined_args_to_skip + && bitmap_bit_p (node->clone.combined_args_to_skip, i)) + continue; + gcc_checking_assert (parm); + next_parm = DECL_CHAIN (parm); + + if (!alignments[i].known || !is_gimple_reg (parm)) + continue; + tree ddef = ssa_default_def (DECL_STRUCT_FUNCTION (node->decl), parm); + if (!ddef) + continue; + + if (dump_file) + fprintf (dump_file, " Adjusting alignment of param %u to %u, " + "bitpos to %u\n", i, alignments[i].align, + alignments[i].bitpos); + + struct ptr_info_def *pi = get_ptr_info (ddef); + gcc_checking_assert (pi); + unsigned old_align; + unsigned old_misalign; + bool old_known = get_ptr_info_alignment (pi, &old_align, &old_misalign); + + if (old_known) + { + if (old_align >= alignments[i].align) + { + if (dump_file) + fprintf (dump_file, " But the alignment has already " + "been %u.\n", old_align); + continue; + } + if (old_misalign != 0) + { + if (dump_file) + fprintf (dump_file, " But there has been a noted " + "misalignment.\n"); + continue; + } + } + set_ptr_info_alignment (pi, alignments[i].align, alignments[i].bitpos); + } +} + /* IPCP transformation phase doing propagation of aggregate values. */ unsigned int @@ -5155,6 +5336,7 @@ ipcp_transform_function (struct cgraph_n fprintf (dump_file, "Modification phase of node %s/%i\n", node->name (), node->order); + ipcp_update_alignments (node); aggval = ipa_get_agg_replacements_for_node (node); if (!aggval) return 0; @@ -5184,7 +5366,8 @@ ipcp_transform_function (struct cgraph_n free_ipa_bb_info (bi); fbi.bb_infos.release (); free_dominance_info (CDI_DOMINATORS); - (*ipa_node_agg_replacements)[node->uid] = NULL; + (*ipcp_transformations)[node->uid].agg_values = NULL; + (*ipcp_transformations)[node->uid].alignments = NULL; descriptors.release (); if (!something_changed) Index: src/gcc/ipa-prop.h =================================================================== --- src.orig/gcc/ipa-prop.h 2014-11-14 22:06:06.240031030 +0100 +++ src/gcc/ipa-prop.h 2014-11-15 01:40:06.924520383 +0100 @@ -155,6 +155,16 @@ struct GTY(()) ipa_agg_jump_function typedef struct ipa_agg_jump_function *ipa_agg_jump_function_p; +/* Info about poiner alignments. */ +struct GTY(()) ipa_alignment +{ + /* The data fields below are valid only if known is true. */ + bool known; + /* See get_pointer_alignment_1 for description of these two. */ + unsigned bitpos; + unsigned align; +}; + /* A jump function for a callsite represents the values passed as actual arguments of the callsite. See enum jump_func_type for the various types of jump functions supported. */ @@ -164,6 +174,9 @@ struct GTY (()) ipa_jump_func description. */ struct ipa_agg_jump_function agg; + /* Information about alignment of pointers. */ + struct ipa_alignment alignment; + enum jump_func_type type; /* Represents a value of a jump function. pass_through is used only in jump function context. constant represents the actual constant in constant jump @@ -413,10 +426,19 @@ struct GTY(()) ipa_agg_replacement_value bool by_ref; }; -typedef struct ipa_agg_replacement_value *ipa_agg_replacement_value_p; +/* Structure holding information for the transformation phase of IPA-CP. */ + +struct GTY(()) ipcp_transformation_summary +{ + /* Linked list of known aggregate values. */ + ipa_agg_replacement_value *agg_values; + /* Alignemnt information for pointers. */ + vec<ipa_alignment, va_gc> *alignments; +}; void ipa_set_node_agg_value_chain (struct cgraph_node *node, struct ipa_agg_replacement_value *aggvals); +void ipcp_grow_transformations_if_necessary (void); /* ipa_edge_args stores information related to a callsite and particularly its arguments. It can be accessed by the IPA_EDGE_REF macro. */ @@ -462,8 +484,8 @@ ipa_get_ith_polymorhic_call_context (str /* Vector where the parameter infos are actually stored. */ extern vec<ipa_node_params> ipa_node_params_vector; -/* Vector of known aggregate values in cloned nodes. */ -extern GTY(()) vec<ipa_agg_replacement_value_p, va_gc> *ipa_node_agg_replacements; +/* Vector of IPA-CP transformation data for each clone. */ +extern GTY(()) vec<ipcp_transformation_summary, va_gc> *ipcp_transformations; /* Vector where the parameter infos are actually stored. */ extern GTY(()) vec<ipa_edge_args, va_gc> *ipa_edge_args_vector; @@ -521,14 +543,21 @@ ipa_edge_args_info_available_for_edge_p return ((unsigned) edge->uid < vec_safe_length (ipa_edge_args_vector)); } +static inline ipcp_transformation_summary * +ipcp_get_transformation_summary (cgraph_node *node) +{ + if ((unsigned) node->uid >= vec_safe_length (ipcp_transformations)) + return NULL; + return &(*ipcp_transformations)[node->uid]; +} + /* Return the aggregate replacements for NODE, if there are any. */ static inline struct ipa_agg_replacement_value * -ipa_get_agg_replacements_for_node (struct cgraph_node *node) +ipa_get_agg_replacements_for_node (cgraph_node *node) { - if ((unsigned) node->uid >= vec_safe_length (ipa_node_agg_replacements)) - return NULL; - return (*ipa_node_agg_replacements)[node->uid]; + ipcp_transformation_summary *ts = ipcp_get_transformation_summary (node); + return ts ? ts->agg_values : NULL; } /* Function formal parameters related computations. */ @@ -657,8 +686,8 @@ void ipa_dump_agg_replacement_values (FI struct ipa_agg_replacement_value *av); void ipa_prop_write_jump_functions (void); void ipa_prop_read_jump_functions (void); -void ipa_prop_write_all_agg_replacement (void); -void ipa_prop_read_all_agg_replacement (void); +void ipcp_write_transformation_summaries (void); +void ipcp_read_transformation_summaries (void); void ipa_update_after_lto_read (void); int ipa_get_param_decl_index (struct ipa_node_params *, tree); tree ipa_value_from_jfunc (struct ipa_node_params *info, Index: src/gcc/testsuite/gcc.dg/ipa/propalign-1.c =================================================================== --- /dev/null 1970-01-01 00:00:00.000000000 +0000 +++ src/gcc/testsuite/gcc.dg/ipa/propalign-1.c 2014-11-15 00:38:53.420380387 +0100 @@ -0,0 +1,32 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -fno-early-inlining -fdump-ipa-cp -fdump-tree-optimized" } */ + +#include <stdint.h> + +extern int fail_the_test(void *); +extern int pass_the_test(void *); +extern int diversion (void *); + +static int __attribute__((noinline)) +foo (void *p) +{ + uintptr_t a = (uintptr_t) p; + + if (a % 4) + return fail_the_test (p); + else + return pass_the_test (p); +} + +int +bar (void) +{ + double buf[8] __attribute__ ((__aligned__(__BIGGEST_ALIGNMENT__))); + return foo (&buf); +} + + +/* { dg-final { scan-ipa-dump "Adjusting alignment of param" "cp" } } */ +/* { dg-final { scan-tree-dump-not "fail_the_test" "optimized" } } */ +/* { dg-final { cleanup-ipa-dump "cp" } } */ +/* { dg-final { cleanup-tree-dump "optimized" } } */ Index: src/gcc/testsuite/gcc.dg/ipa/propalign-2.c =================================================================== --- /dev/null 1970-01-01 00:00:00.000000000 +0000 +++ src/gcc/testsuite/gcc.dg/ipa/propalign-2.c 2014-11-15 00:38:53.420380387 +0100 @@ -0,0 +1,58 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -fno-early-inlining -fdump-ipa-cp -fdump-tree-optimized" } */ + +#include <stdint.h> + +extern int fail_the_test(void *); +extern int pass_the_test(void *); +extern int diversion (void *); + +struct somestruct +{ + void *whee; + void *oops; +}; + +struct container +{ + struct somestruct first; + struct somestruct buf[32]; +}; + +static int __attribute__((noinline)) +foo (void *p) +{ + uintptr_t a = (uintptr_t) p; + + if (a % 4) + return fail_the_test (p); + else + return pass_the_test (p); +} + +int +bar (void) +{ + struct container c; + return foo (c.buf); +} + + +static int +through (struct somestruct *p) +{ + diversion (p); + return foo (&p[16]); +} + +int +bar2 (void) +{ + struct container c; + through (c.buf); +} + +/* { dg-final { scan-ipa-dump "Adjusting alignment of param" "cp" } } */ +/* { dg-final { scan-tree-dump-not "fail_the_test" "optimized" } } */ +/* { dg-final { cleanup-ipa-dump "cp" } } */ +/* { dg-final { cleanup-tree-dump "optimized" } } */