| Message ID | ri6r1rr4w04.fsf@suse.cz |
|---|---|
| State | New |
| Headers | show |
| Series | ipa: Merge vectors describing argument values into one type | expand |
> Hi, > > this large patch is a semi-mechanical change which aims to replace > uses of separate vectors about known scalar values (usually called > known_vals or known_csts), known aggregate values (known_aggs), known > virtual call contexts (known_contexts) and known value > ranges (known_value_ranges) with uses of one type called > ipa_call_arg_values. The main benefit is bringing down the number of > arguments that various call context evaluating functions have. > Because the new type uses auto_vecs deallocation is simpler and it > also takes advantage of storage within the auto_vecs themselves, > eliminating the need for allocating memory when analyzing most > functions. > > The one place where this patch is not mechanical is the > node_context_cache_entry RCU cache. Before the elements contained > directly ipa_call_context instances which however owned their vectors, > they did not share them with their users like non-cache contexts. Now > the vectors are behind the pointer which means ipa_call_arg_values > would have to be allocated, for many functions at once, and so it > could not make use of auto_vecs with generous internal storage. > > I avoided both by reworking the cache to contain copies of all fields > o interest of ipa_call_context, including the interesting vectors. > This makes the type a bit more verbose but the main functionality, > storing to the cache and comparing a context with a cache entry, > remained very similar. Can you pleae benchmark this on building Firfox or something similar? I was running into two problems here. First was overhead of malloc allocations and that is reason why I am trying to not allocate the vector when not necessary and also place them on stack rather then heap for those having short lifetime (not being in cache). Second problem is memory use - we may end up with quite many contextes since calls are very numerous. Honza > > This patch does not unify information about scalar values, aggregate > values, virtual contexts and value ranges that IPA-CP stores for > clones it creates. The main reason is not to make this patch even > bigger. Another one is that such change would not be as mechanical, > the aggregate values for clones are stored in a different format. I > am considering doing something similar for them too however, even > though using ipa_call_arg_values directly would also mean not using > big auto_vecs in it. > > I have bootstrapped and tested what I believe to be an identical patch > (before I fixed some comments) on x86_64-linux, I am running LTO > bootstrap on exactly this patch now. OK for master if it passes? > > Thanks, > > Martin > > > > gcc/ChangeLog: > > 2020-08-27 Martin Jambor <mjambor@suse.cz> > > * ipa-prop.h (ipa_call_arg_values): New type. > (ipa_get_indirect_edge_target): Replaced vector arguments > with ipa_call_arg_values in declaration. > * ipa-fnsummary.h (ipa_node_context_cache_entry): New forward > declaration. > (class ipa_call_context): Removed members m_known_vals, > m_known_contexts, m_known_aggs, duplicate_from, release and > equal_to, new members m_avals, store_to_cache and equivalent_to_p. > Adjusted construcotr arguments. > (estimate_ipcp_clone_size_and_time): Replaced vector arguments > with ipa_call_arg_values in declaration. > (evaluate_properties_for_edge): Likewise. > * ipa-cp.c (ipa_get_indirect_edge_target): Adjust to work on > ipa_call_arg_values rather than on separate vectors. > (devirtualization_time_bonus): Likewise. > (gather_context_independent_values): Likewise. > (perform_estimation_of_a_value): Likewise. > (estimate_local_effects): Likewise. > (modify_known_vectors_with_val): Adjust both variants to work on > ipa_call_arg_values and rename them to copy_known_vectors_add_val. > (decide_about_value): Adjust to work on ipa_call_arg_values rather > than on separate vectors. > (decide_whether_version_node): Likewise. > * ipa-fnsummary.c (evaluate_conditions_for_known_args): Adjust to > work on ipa_call_arg_values rather than on separate vectors. > (evaluate_properties_for_edge): Likewise. > (ipa_fn_summary_t::duplicate): Likewise. > (estimate_edge_devirt_benefit): Likewise. > (estimate_edge_size_and_time): Likewise. > (estimate_calls_size_and_time_1): Likewise. > (summarize_calls_size_and_time): Adjust calls to > estimate_edge_size_and_time. > (estimate_calls_size_and_time): Adjust to work on > ipa_call_arg_values rather than on separate vectors. > (ipa_call_context::ipa_call_context): Likewise. > (ipa_call_context::duplicate_from): Removed. > (ipa_call_context::release): Likewise. > (ipa_call_context::equal_to): Likewise. > (ipa_call_context::estimate_size_and_time): Adjust to work on > ipa_call_arg_values rather than on separate vectors. > (estimate_ipcp_clone_size_and_time): Likewise. > (ipa_merge_fn_summary_after_inlining): Likewise. > * ipa-inline-analysis.c (node_context_cache_entry): Replaced with > ipa_node_context_cache_entry which does not directly hold context > but irectly its necessary fields. > (ipa_node_context_cache_entry::release): New. > (ipa_call_context::store_to_cache): Likewise. > (ipa_call_context::equivalent_to_p): Likewise. > (do_estimate_edge_time): Adjust to work on ipa_call_arg_values > rather than on separate vectors. Use the new cache. Changed the > mechanism of cache miss dtection. Added comment about caching hints. > (do_estimate_edge_size): Adjust to work on ipa_call_arg_values > rather than on separate vectors. > (do_estimate_edge_hints): Likewise. > * ipa-prop.c (ipa_call_arg_values::~ipa_call_arg_values): New > destructor. > --- > gcc/ipa-cp.c | 231 +++++++--------- > gcc/ipa-fnsummary.c | 555 ++++++++++---------------------------- > gcc/ipa-fnsummary.h | 34 +-- > gcc/ipa-inline-analysis.c | 326 ++++++++++++++++++---- > gcc/ipa-prop.c | 10 + > gcc/ipa-prop.h | 50 +++- > 6 files changed, 585 insertions(+), 621 deletions(-) > > diff --git a/gcc/ipa-cp.c b/gcc/ipa-cp.c > index e4910a04ffa..970675efe49 100644 > --- a/gcc/ipa-cp.c > +++ b/gcc/ipa-cp.c > @@ -3117,30 +3117,26 @@ ipa_get_indirect_edge_target_1 (struct cgraph_edge *ie, > return target; > } > > - > -/* If an indirect edge IE can be turned into a direct one based on KNOWN_CSTS, > - KNOWN_CONTEXTS (which can be vNULL) or KNOWN_AGGS (which also can be vNULL) > - return the destination. */ > +/* If an indirect edge IE can be turned into a direct one based on data in > + AVALS, return the destination. Store into *SPECULATIVE a boolean determinig > + whether the discovered target is only speculative guess. */ > > tree > ipa_get_indirect_edge_target (struct cgraph_edge *ie, > - vec<tree> known_csts, > - vec<ipa_polymorphic_call_context> known_contexts, > - vec<ipa_agg_value_set> known_aggs, > + ipa_call_arg_values *avals, > bool *speculative) > { > - return ipa_get_indirect_edge_target_1 (ie, known_csts, known_contexts, > - known_aggs, NULL, speculative); > + return ipa_get_indirect_edge_target_1 (ie, avals->m_known_vals, > + avals->m_known_contexts, > + avals->m_known_aggs, NULL, speculative); > } > > -/* Calculate devirtualization time bonus for NODE, assuming we know KNOWN_CSTS > - and KNOWN_CONTEXTS. */ > +/* Calculate devirtualization time bonus for NODE, assuming we know information > + about arguments stored in AVALS. */ > > static int > devirtualization_time_bonus (struct cgraph_node *node, > - vec<tree> known_csts, > - vec<ipa_polymorphic_call_context> known_contexts, > - vec<ipa_agg_value_set> known_aggs) > + ipa_call_arg_values *avals) > { > struct cgraph_edge *ie; > int res = 0; > @@ -3153,8 +3149,7 @@ devirtualization_time_bonus (struct cgraph_node *node, > tree target; > bool speculative; > > - target = ipa_get_indirect_edge_target (ie, known_csts, known_contexts, > - known_aggs, &speculative); > + target = ipa_get_indirect_edge_target (ie, avals, &speculative); > if (!target) > continue; > > @@ -3306,31 +3301,31 @@ context_independent_aggregate_values (class ipcp_param_lattices *plats) > return res; > } > > -/* Allocate KNOWN_CSTS, KNOWN_CONTEXTS and, if non-NULL, KNOWN_AGGS and > - populate them with values of parameters that are known independent of the > - context. INFO describes the function. If REMOVABLE_PARAMS_COST is > - non-NULL, the movement cost of all removable parameters will be stored in > - it. */ > +/* Grow vectors in AVALS and fill them with information about values of > + parameters that are known to be independent of the context. Only calculate > + m_known_aggs if CALCULATE_AGGS is true. INFO describes the function. If > + REMOVABLE_PARAMS_COST is non-NULL, the movement cost of all removable > + parameters will be stored in it. > + > + TODO: Also grow context independent value range vectors. */ > > static bool > gather_context_independent_values (class ipa_node_params *info, > - vec<tree> *known_csts, > - vec<ipa_polymorphic_call_context> > - *known_contexts, > - vec<ipa_agg_value_set> *known_aggs, > + ipa_call_arg_values *avals, > + bool calculate_aggs, > int *removable_params_cost) > { > int i, count = ipa_get_param_count (info); > bool ret = false; > > - known_csts->create (0); > - known_contexts->create (0); > - known_csts->safe_grow_cleared (count); > - known_contexts->safe_grow_cleared (count); > - if (known_aggs) > + avals->m_known_vals.reserve_exact (count); > + avals->m_known_vals.quick_grow_cleared (count); > + avals->m_known_contexts.reserve_exact (count); > + avals->m_known_contexts.quick_grow_cleared (count); > + if (calculate_aggs) > { > - known_aggs->create (0); > - known_aggs->safe_grow_cleared (count); > + avals->m_known_aggs.reserve_exact (count); > + avals->m_known_aggs.safe_grow_cleared (count); > } > > if (removable_params_cost) > @@ -3345,7 +3340,7 @@ gather_context_independent_values (class ipa_node_params *info, > { > ipcp_value<tree> *val = lat->values; > gcc_checking_assert (TREE_CODE (val->value) != TREE_BINFO); > - (*known_csts)[i] = val->value; > + avals->m_known_vals[i] = val->value; > if (removable_params_cost) > *removable_params_cost > += estimate_move_cost (TREE_TYPE (val->value), false); > @@ -3363,15 +3358,15 @@ gather_context_independent_values (class ipa_node_params *info, > /* Do not account known context as reason for cloning. We can see > if it permits devirtualization. */ > if (ctxlat->is_single_const ()) > - (*known_contexts)[i] = ctxlat->values->value; > + avals->m_known_contexts[i] = ctxlat->values->value; > > - if (known_aggs) > + if (calculate_aggs) > { > vec<ipa_agg_value> agg_items; > struct ipa_agg_value_set *agg; > > agg_items = context_independent_aggregate_values (plats); > - agg = &(*known_aggs)[i]; > + agg = &avals->m_known_aggs[i]; > agg->items = agg_items; > agg->by_ref = plats->aggs_by_ref; > ret |= !agg_items.is_empty (); > @@ -3381,25 +3376,22 @@ gather_context_independent_values (class ipa_node_params *info, > return ret; > } > > -/* Perform time and size measurement of NODE with the context given in > - KNOWN_CSTS, KNOWN_CONTEXTS and KNOWN_AGGS, calculate the benefit and cost > - given BASE_TIME of the node without specialization, REMOVABLE_PARAMS_COST of > - all context-independent removable parameters and EST_MOVE_COST of estimated > - movement of the considered parameter and store it into VAL. */ > +/* Perform time and size measurement of NODE with the context given in AVALS, > + calculate the benefit compared to the node without specialization and store > + it into VAL. Also, if non-NULL, calculate REMOVABLE_PARAMS_COST of all > + context-independent or unused removable parameters and EST_MOVE_COST, the > + estimated movement of the considered parameter. */ > > static void > -perform_estimation_of_a_value (cgraph_node *node, vec<tree> known_csts, > - vec<ipa_polymorphic_call_context> known_contexts, > - vec<ipa_agg_value_set> known_aggs, > - int removable_params_cost, > - int est_move_cost, ipcp_value_base *val) > +perform_estimation_of_a_value (cgraph_node *node, ipa_call_arg_values *avals, > + int removable_params_cost, int est_move_cost, > + ipcp_value_base *val) > { > int size, time_benefit; > sreal time, base_time; > ipa_hints hints; > > - estimate_ipcp_clone_size_and_time (node, known_csts, known_contexts, > - known_aggs, &size, &time, > + estimate_ipcp_clone_size_and_time (node, avals, &size, &time, > &base_time, &hints); > base_time -= time; > if (base_time > 65535) > @@ -3412,8 +3404,7 @@ perform_estimation_of_a_value (cgraph_node *node, vec<tree> known_csts, > time_benefit = 0; > else > time_benefit = base_time.to_int () > - + devirtualization_time_bonus (node, known_csts, known_contexts, > - known_aggs) > + + devirtualization_time_bonus (node, avals) > + hint_time_bonus (node, hints) > + removable_params_cost + est_move_cost; > > @@ -3454,9 +3445,7 @@ estimate_local_effects (struct cgraph_node *node) > { > class ipa_node_params *info = IPA_NODE_REF (node); > int i, count = ipa_get_param_count (info); > - vec<tree> known_csts; > - vec<ipa_polymorphic_call_context> known_contexts; > - vec<ipa_agg_value_set> known_aggs; > + ipa_call_arg_values avals; > bool always_const; > int removable_params_cost; > > @@ -3466,11 +3455,9 @@ estimate_local_effects (struct cgraph_node *node) > if (dump_file && (dump_flags & TDF_DETAILS)) > fprintf (dump_file, "\nEstimating effects for %s.\n", node->dump_name ()); > > - always_const = gather_context_independent_values (info, &known_csts, > - &known_contexts, &known_aggs, > + always_const = gather_context_independent_values (info, &avals, true, > &removable_params_cost); > - int devirt_bonus = devirtualization_time_bonus (node, known_csts, > - known_contexts, known_aggs); > + int devirt_bonus = devirtualization_time_bonus (node, &avals); > if (always_const || devirt_bonus > || (removable_params_cost && node->can_change_signature)) > { > @@ -3482,8 +3469,7 @@ estimate_local_effects (struct cgraph_node *node) > init_caller_stats (&stats); > node->call_for_symbol_thunks_and_aliases (gather_caller_stats, &stats, > false); > - estimate_ipcp_clone_size_and_time (node, known_csts, known_contexts, > - known_aggs, &size, &time, > + estimate_ipcp_clone_size_and_time (node, &avals, &size, &time, > &base_time, &hints); > time -= devirt_bonus; > time -= hint_time_bonus (node, hints); > @@ -3536,18 +3522,17 @@ estimate_local_effects (struct cgraph_node *node) > > if (lat->bottom > || !lat->values > - || known_csts[i]) > + || avals.m_known_vals[i]) > continue; > > for (val = lat->values; val; val = val->next) > { > gcc_checking_assert (TREE_CODE (val->value) != TREE_BINFO); > - known_csts[i] = val->value; > + avals.m_known_vals[i] = val->value; > > int emc = estimate_move_cost (TREE_TYPE (val->value), true); > - perform_estimation_of_a_value (node, known_csts, known_contexts, > - known_aggs, > - removable_params_cost, emc, val); > + perform_estimation_of_a_value (node, &avals, removable_params_cost, > + emc, val); > > if (dump_file && (dump_flags & TDF_DETAILS)) > { > @@ -3559,7 +3544,7 @@ estimate_local_effects (struct cgraph_node *node) > val->local_time_benefit, val->local_size_cost); > } > } > - known_csts[i] = NULL_TREE; > + avals.m_known_vals[i] = NULL_TREE; > } > > for (i = 0; i < count; i++) > @@ -3574,15 +3559,14 @@ estimate_local_effects (struct cgraph_node *node) > > if (ctxlat->bottom > || !ctxlat->values > - || !known_contexts[i].useless_p ()) > + || !avals.m_known_contexts[i].useless_p ()) > continue; > > for (val = ctxlat->values; val; val = val->next) > { > - known_contexts[i] = val->value; > - perform_estimation_of_a_value (node, known_csts, known_contexts, > - known_aggs, > - removable_params_cost, 0, val); > + avals.m_known_contexts[i] = val->value; > + perform_estimation_of_a_value (node, &avals, removable_params_cost, > + 0, val); > > if (dump_file && (dump_flags & TDF_DETAILS)) > { > @@ -3594,20 +3578,18 @@ estimate_local_effects (struct cgraph_node *node) > val->local_time_benefit, val->local_size_cost); > } > } > - known_contexts[i] = ipa_polymorphic_call_context (); > + avals.m_known_contexts[i] = ipa_polymorphic_call_context (); > } > > for (i = 0; i < count; i++) > { > class ipcp_param_lattices *plats = ipa_get_parm_lattices (info, i); > - struct ipa_agg_value_set *agg; > - struct ipcp_agg_lattice *aglat; > > if (plats->aggs_bottom || !plats->aggs) > continue; > > - agg = &known_aggs[i]; > - for (aglat = plats->aggs; aglat; aglat = aglat->next) > + ipa_agg_value_set *agg = &avals.m_known_aggs[i]; > + for (ipcp_agg_lattice *aglat = plats->aggs; aglat; aglat = aglat->next) > { > ipcp_value<tree> *val; > if (aglat->bottom || !aglat->values > @@ -3624,9 +3606,8 @@ estimate_local_effects (struct cgraph_node *node) > item.value = val->value; > agg->items.safe_push (item); > > - perform_estimation_of_a_value (node, known_csts, known_contexts, > - known_aggs, > - removable_params_cost, 0, val); > + perform_estimation_of_a_value (node, &avals, removable_params_cost, > + 0, val); > > if (dump_file && (dump_flags & TDF_DETAILS)) > { > @@ -3645,10 +3626,6 @@ estimate_local_effects (struct cgraph_node *node) > } > } > } > - > - known_csts.release (); > - known_contexts.release (); > - ipa_release_agg_values (known_aggs); > } > > > @@ -5371,31 +5348,32 @@ copy_useful_known_contexts (vec<ipa_polymorphic_call_context> known_contexts) > return vNULL; > } > > -/* Copy KNOWN_CSTS and modify the copy according to VAL and INDEX. If > - non-empty, replace KNOWN_CONTEXTS with its copy too. */ > +/* Copy known scalar values from AVALS into KNOWN_CSTS and modify the copy > + according to VAL and INDEX. If non-empty, replace KNOWN_CONTEXTS with its > + copy too. */ > > static void > -modify_known_vectors_with_val (vec<tree> *known_csts, > - vec<ipa_polymorphic_call_context> *known_contexts, > - ipcp_value<tree> *val, > - int index) > +copy_known_vectors_add_val (ipa_call_arg_values *avals, vec<tree> *known_csts, > + vec<ipa_polymorphic_call_context> *known_contexts, > + ipcp_value<tree> *val, int index) > { > - *known_csts = known_csts->copy (); > - *known_contexts = copy_useful_known_contexts (*known_contexts); > + *known_csts = avals->m_known_vals.copy (); > + *known_contexts = copy_useful_known_contexts (avals->m_known_contexts); > (*known_csts)[index] = val->value; > } > > -/* Replace KNOWN_CSTS with its copy. Also copy KNOWN_CONTEXTS and modify the > - copy according to VAL and INDEX. */ > +/* Copy known scalar values from AVALS into KNOWN_CSTS. Similarly, copy > + contexts to KNOWN_CONTEXTS and modify the copy according to VAL and > + INDEX. */ > > static void > -modify_known_vectors_with_val (vec<tree> *known_csts, > - vec<ipa_polymorphic_call_context> *known_contexts, > - ipcp_value<ipa_polymorphic_call_context> *val, > - int index) > +copy_known_vectors_add_val (ipa_call_arg_values *avals, vec<tree> *known_csts, > + vec<ipa_polymorphic_call_context> *known_contexts, > + ipcp_value<ipa_polymorphic_call_context> *val, > + int index) > { > - *known_csts = known_csts->copy (); > - *known_contexts = known_contexts->copy (); > + *known_csts = avals->m_known_vals.copy (); > + *known_contexts = avals->m_known_contexts.copy (); > (*known_contexts)[index] = val->value; > } > > @@ -5432,16 +5410,15 @@ ipcp_val_agg_replacement_ok_p (ipa_agg_replacement_value *, > return offset == -1; > } > > -/* Decide whether to create a special version of NODE for value VAL of parameter > - at the given INDEX. If OFFSET is -1, the value is for the parameter itself, > - otherwise it is stored at the given OFFSET of the parameter. KNOWN_CSTS, > - KNOWN_CONTEXTS and KNOWN_AGGS describe the other already known values. */ > +/* Decide whether to create a special version of NODE for value VAL of > + parameter at the given INDEX. If OFFSET is -1, the value is for the > + parameter itself, otherwise it is stored at the given OFFSET of the > + parameter. AVALS describes the other already known values. */ > > template <typename valtype> > static bool > decide_about_value (struct cgraph_node *node, int index, HOST_WIDE_INT offset, > - ipcp_value<valtype> *val, vec<tree> known_csts, > - vec<ipa_polymorphic_call_context> known_contexts) > + ipcp_value<valtype> *val, ipa_call_arg_values *avals) > { > struct ipa_agg_replacement_value *aggvals; > int freq_sum, caller_count; > @@ -5491,13 +5468,16 @@ decide_about_value (struct cgraph_node *node, int index, HOST_WIDE_INT offset, > fprintf (dump_file, " Creating a specialized node of %s.\n", > node->dump_name ()); > > + vec<tree> known_csts; > + vec<ipa_polymorphic_call_context> known_contexts; > + > callers = gather_edges_for_value (val, node, caller_count); > if (offset == -1) > - modify_known_vectors_with_val (&known_csts, &known_contexts, val, index); > + copy_known_vectors_add_val (avals, &known_csts, &known_contexts, val, index); > else > { > - known_csts = known_csts.copy (); > - known_contexts = copy_useful_known_contexts (known_contexts); > + known_csts = avals->m_known_vals.copy (); > + known_contexts = copy_useful_known_contexts (avals->m_known_contexts); > } > find_more_scalar_values_for_callers_subset (node, known_csts, callers); > find_more_contexts_for_caller_subset (node, &known_contexts, callers); > @@ -5521,8 +5501,7 @@ decide_whether_version_node (struct cgraph_node *node) > { > class ipa_node_params *info = IPA_NODE_REF (node); > int i, count = ipa_get_param_count (info); > - vec<tree> known_csts; > - vec<ipa_polymorphic_call_context> known_contexts; > + ipa_call_arg_values avals; > bool ret = false; > > if (count == 0) > @@ -5532,8 +5511,7 @@ decide_whether_version_node (struct cgraph_node *node) > fprintf (dump_file, "\nEvaluating opportunities for %s.\n", > node->dump_name ()); > > - gather_context_independent_values (info, &known_csts, &known_contexts, > - NULL, NULL); > + gather_context_independent_values (info, &avals, false, NULL); > > for (i = 0; i < count;i++) > { > @@ -5542,12 +5520,11 @@ decide_whether_version_node (struct cgraph_node *node) > ipcp_lattice<ipa_polymorphic_call_context> *ctxlat = &plats->ctxlat; > > if (!lat->bottom > - && !known_csts[i]) > + && !avals.m_known_vals[i]) > { > ipcp_value<tree> *val; > for (val = lat->values; val; val = val->next) > - ret |= decide_about_value (node, i, -1, val, known_csts, > - known_contexts); > + ret |= decide_about_value (node, i, -1, val, &avals); > } > > if (!plats->aggs_bottom) > @@ -5556,22 +5533,20 @@ decide_whether_version_node (struct cgraph_node *node) > ipcp_value<tree> *val; > for (aglat = plats->aggs; aglat; aglat = aglat->next) > if (!aglat->bottom && aglat->values > - /* If the following is false, the one value is in > - known_aggs. */ > + /* If the following is false, the one value has been considered > + for cloning for all contexts. */ > && (plats->aggs_contain_variable > || !aglat->is_single_const ())) > for (val = aglat->values; val; val = val->next) > - ret |= decide_about_value (node, i, aglat->offset, val, > - known_csts, known_contexts); > + ret |= decide_about_value (node, i, aglat->offset, val, &avals); > } > > if (!ctxlat->bottom > - && known_contexts[i].useless_p ()) > + && avals.m_known_contexts[i].useless_p ()) > { > ipcp_value<ipa_polymorphic_call_context> *val; > for (val = ctxlat->values; val; val = val->next) > - ret |= decide_about_value (node, i, -1, val, known_csts, > - known_contexts); > + ret |= decide_about_value (node, i, -1, val, &avals); > } > > info = IPA_NODE_REF (node); > @@ -5594,11 +5569,9 @@ decide_whether_version_node (struct cgraph_node *node) > if (!adjust_callers_for_value_intersection (callers, node)) > { > /* If node is not called by anyone, or all its caller edges are > - self-recursive, the node is not really be in use, no need to > - do cloning. */ > + self-recursive, the node is not really in use, no need to do > + cloning. */ > callers.release (); > - known_csts.release (); > - known_contexts.release (); > info->do_clone_for_all_contexts = false; > return ret; > } > @@ -5607,6 +5580,9 @@ decide_whether_version_node (struct cgraph_node *node) > fprintf (dump_file, " - Creating a specialized node of %s " > "for all known contexts.\n", node->dump_name ()); > > + vec<tree> known_csts = avals.m_known_vals.copy (); > + vec<ipa_polymorphic_call_context> known_contexts > + = copy_useful_known_contexts (avals.m_known_contexts); > find_more_scalar_values_for_callers_subset (node, known_csts, callers); > find_more_contexts_for_caller_subset (node, &known_contexts, callers); > ipa_agg_replacement_value *aggvals > @@ -5624,11 +5600,6 @@ decide_whether_version_node (struct cgraph_node *node) > IPA_NODE_REF (clone)->is_all_contexts_clone = true; > ret = true; > } > - else > - { > - known_csts.release (); > - known_contexts.release (); > - } > > return ret; > } > diff --git a/gcc/ipa-fnsummary.c b/gcc/ipa-fnsummary.c > index 2cfab40156e..f154d5ee9fc 100644 > --- a/gcc/ipa-fnsummary.c > +++ b/gcc/ipa-fnsummary.c > @@ -320,19 +320,15 @@ set_hint_predicate (predicate **p, predicate new_predicate) > is always false in the second and also builtin_constant_p tests cannot use > the fact that parameter is indeed a constant. > > - KNOWN_VALS is partial mapping of parameters of NODE to constant values. > - KNOWN_AGGS is a vector of aggregate known offset/value set for each > - parameter. Return clause of possible truths. When INLINE_P is true, assume > - that we are inlining. > + ARG_VALUES contains known information about argument values. Return clause > + of possible truths. When INLINE_P is true, assume that we are inlining. > > - ERROR_MARK means compile time invariant. */ > + ERROR_MARK value of an argument means compile time invariant. */ > > static void > evaluate_conditions_for_known_args (struct cgraph_node *node, > bool inline_p, > - vec<tree> known_vals, > - vec<value_range> known_value_ranges, > - vec<ipa_agg_value_set> known_aggs, > + ipa_call_arg_values *avals, > clause_t *ret_clause, > clause_t *ret_nonspec_clause) > { > @@ -351,10 +347,12 @@ evaluate_conditions_for_known_args (struct cgraph_node *node, > > /* We allow call stmt to have fewer arguments than the callee function > (especially for K&R style programs). So bound check here (we assume > - known_aggs vector, if non-NULL, has the same length as > - known_vals). */ > - gcc_checking_assert (!known_aggs.length () || !known_vals.length () > - || (known_vals.length () == known_aggs.length ())); > + m_known_aggs vector is either empty or has the same length as > + m_known_vals). */ > + gcc_checking_assert (!avals->m_known_aggs.length () > + || !avals->m_known_vals.length () > + || (avals->m_known_vals.length () > + == avals->m_known_aggs.length ())); > > if (c->agg_contents) > { > @@ -362,26 +360,23 @@ evaluate_conditions_for_known_args (struct cgraph_node *node, > > if (c->code == predicate::changed > && !c->by_ref > - && c->operand_num < (int)known_vals.length () > - && (known_vals[c->operand_num] == error_mark_node)) > + && c->operand_num < (int) avals->m_known_vals.length () > + && (avals->m_known_vals[c->operand_num] == error_mark_node)) > continue; > > - if (c->operand_num < (int)known_aggs.length ()) > + if (c->operand_num < (int) avals->m_known_aggs.length ()) > { > - agg = &known_aggs[c->operand_num]; > - val = ipa_find_agg_cst_for_param (agg, > - c->operand_num > - < (int) known_vals.length () > - ? known_vals[c->operand_num] > - : NULL, > - c->offset, c->by_ref); > + agg = &avals->m_known_aggs[c->operand_num]; > + tree sval = avals->safe_sval_at (c->operand_num); > + val = ipa_find_agg_cst_for_param (agg, sval, c->offset, > + c->by_ref); > } > else > val = NULL_TREE; > } > - else if (c->operand_num < (int) known_vals.length ()) > + else if (c->operand_num < (int) avals->m_known_vals.length ()) > { > - val = known_vals[c->operand_num]; > + val = avals->m_known_vals[c->operand_num]; > if (val == error_mark_node && c->code != predicate::changed) > val = NULL_TREE; > } > @@ -446,53 +441,54 @@ evaluate_conditions_for_known_args (struct cgraph_node *node, > continue; > } > } > - if (c->operand_num < (int) known_value_ranges.length () > + if (c->operand_num < (int) avals->m_known_value_ranges.length () > && !c->agg_contents > - && !known_value_ranges[c->operand_num].undefined_p () > - && !known_value_ranges[c->operand_num].varying_p () > - && TYPE_SIZE (c->type) > - == TYPE_SIZE (known_value_ranges[c->operand_num].type ()) > && (!val || TREE_CODE (val) != INTEGER_CST)) > { > - value_range vr = known_value_ranges[c->operand_num]; > - if (!useless_type_conversion_p (c->type, vr.type ())) > + value_range vr = avals->m_known_value_ranges[c->operand_num]; > + if (!vr.undefined_p () > + && !vr.varying_p () > + && (TYPE_SIZE (c->type) == TYPE_SIZE (vr.type ()))) > { > - value_range res; > - range_fold_unary_expr (&res, NOP_EXPR, > - c->type, &vr, vr.type ()); > - vr = res; > - } > - tree type = c->type; > - > - for (j = 0; vec_safe_iterate (c->param_ops, j, &op); j++) > - { > - if (vr.varying_p () || vr.undefined_p ()) > - break; > - > - value_range res; > - if (!op->val[0]) > - range_fold_unary_expr (&res, op->code, op->type, &vr, type); > - else if (!op->val[1]) > + if (!useless_type_conversion_p (c->type, vr.type ())) > { > - value_range op0 (op->val[0], op->val[0]); > - range_fold_binary_expr (&res, op->code, op->type, > - op->index ? &op0 : &vr, > - op->index ? &vr : &op0); > + value_range res; > + range_fold_unary_expr (&res, NOP_EXPR, > + c->type, &vr, vr.type ()); > + vr = res; > + } > + tree type = c->type; > + > + for (j = 0; vec_safe_iterate (c->param_ops, j, &op); j++) > + { > + if (vr.varying_p () || vr.undefined_p ()) > + break; > + > + value_range res; > + if (!op->val[0]) > + range_fold_unary_expr (&res, op->code, op->type, &vr, type); > + else if (!op->val[1]) > + { > + value_range op0 (op->val[0], op->val[0]); > + range_fold_binary_expr (&res, op->code, op->type, > + op->index ? &op0 : &vr, > + op->index ? &vr : &op0); > + } > + else > + gcc_unreachable (); > + type = op->type; > + vr = res; > + } > + if (!vr.varying_p () && !vr.undefined_p ()) > + { > + value_range res; > + value_range val_vr (c->val, c->val); > + range_fold_binary_expr (&res, c->code, boolean_type_node, > + &vr, > + &val_vr); > + if (res.zero_p ()) > + continue; > } > - else > - gcc_unreachable (); > - type = op->type; > - vr = res; > - } > - if (!vr.varying_p () && !vr.undefined_p ()) > - { > - value_range res; > - value_range val_vr (c->val, c->val); > - range_fold_binary_expr (&res, c->code, boolean_type_node, > - &vr, > - &val_vr); > - if (res.zero_p ()) > - continue; > } > } > > @@ -538,24 +534,20 @@ fre_will_run_p (struct cgraph_node *node) > (if non-NULL) conditions evaluated for nonspecialized clone called > in a given context. > > - KNOWN_VALS_PTR and KNOWN_AGGS_PTR must be non-NULL and will be filled by > - known constant and aggregate values of parameters. > - > - KNOWN_CONTEXT_PTR, if non-NULL, will be filled by polymorphic call contexts > - of parameter used by a polymorphic call. */ > + Vectors in AVALS will be populated with useful known information about > + argument values - information not known to have any uses will be omitted - > + except for m_known_contexts which will only be calculated if > + COMPUTE_CONTEXTS is true. */ > > void > evaluate_properties_for_edge (struct cgraph_edge *e, bool inline_p, > clause_t *clause_ptr, > clause_t *nonspec_clause_ptr, > - vec<tree> *known_vals_ptr, > - vec<ipa_polymorphic_call_context> > - *known_contexts_ptr, > - vec<ipa_agg_value_set> *known_aggs_ptr) > + ipa_call_arg_values *avals, > + bool compute_contexts) > { > struct cgraph_node *callee = e->callee->ultimate_alias_target (); > class ipa_fn_summary *info = ipa_fn_summaries->get (callee); > - auto_vec<value_range, 32> known_value_ranges; > class ipa_edge_args *args; > > if (clause_ptr) > @@ -563,7 +555,7 @@ evaluate_properties_for_edge (struct cgraph_edge *e, bool inline_p, > > if (ipa_node_params_sum > && !e->call_stmt_cannot_inline_p > - && (info->conds || known_contexts_ptr) > + && (info->conds || compute_contexts) > && (args = IPA_EDGE_REF (e)) != NULL) > { > struct cgraph_node *caller; > @@ -608,15 +600,15 @@ evaluate_properties_for_edge (struct cgraph_edge *e, bool inline_p, > if (cst) > { > gcc_checking_assert (TREE_CODE (cst) != TREE_BINFO); > - if (!known_vals_ptr->length ()) > - vec_safe_grow_cleared (known_vals_ptr, count); > - (*known_vals_ptr)[i] = cst; > + if (!avals->m_known_vals.length ()) > + avals->m_known_vals.safe_grow_cleared (count); > + avals->m_known_vals[i] = cst; > } > else if (inline_p && !es->param[i].change_prob) > { > - if (!known_vals_ptr->length ()) > - vec_safe_grow_cleared (known_vals_ptr, count); > - (*known_vals_ptr)[i] = error_mark_node; > + if (!avals->m_known_vals.length ()) > + avals->m_known_vals.safe_grow_cleared (count); > + avals->m_known_vals[i] = error_mark_node; > } > > /* If we failed to get simple constant, try value range. */ > @@ -624,19 +616,20 @@ evaluate_properties_for_edge (struct cgraph_edge *e, bool inline_p, > && vrp_will_run_p (caller) > && ipa_is_param_used_by_ipa_predicates (callee_pi, i)) > { > - value_range vr > + value_range vr > = ipa_value_range_from_jfunc (caller_parms_info, e, jf, > ipa_get_type (callee_pi, > i)); > if (!vr.undefined_p () && !vr.varying_p ()) > { > - if (!known_value_ranges.length ()) > + if (!avals->m_known_value_ranges.length ()) > { > - known_value_ranges.safe_grow (count); > + avals->m_known_value_ranges.safe_grow (count); > for (int i = 0; i < count; ++i) > - new (&known_value_ranges[i]) value_range (); > + new (&avals->m_known_value_ranges[i]) > + value_range (); > } > - known_value_ranges[i] = vr; > + avals->m_known_value_ranges[i] = vr; > } > } > > @@ -648,25 +641,25 @@ evaluate_properties_for_edge (struct cgraph_edge *e, bool inline_p, > caller, &jf->agg); > if (agg.items.length ()) > { > - if (!known_aggs_ptr->length ()) > - vec_safe_grow_cleared (known_aggs_ptr, count); > - (*known_aggs_ptr)[i] = agg; > + if (!avals->m_known_aggs.length ()) > + avals->m_known_aggs.safe_grow_cleared (count); > + avals->m_known_aggs[i] = agg; > } > } > } > > /* For calls used in polymorphic calls we further determine > polymorphic call context. */ > - if (known_contexts_ptr > + if (compute_contexts > && ipa_is_param_used_by_polymorphic_call (callee_pi, i)) > { > ipa_polymorphic_call_context > ctx = ipa_context_from_jfunc (caller_parms_info, e, i, jf); > if (!ctx.useless_p ()) > { > - if (!known_contexts_ptr->length ()) > - known_contexts_ptr->safe_grow_cleared (count); > - (*known_contexts_ptr)[i] > + if (!avals->m_known_contexts.length ()) > + avals->m_known_contexts.safe_grow_cleared (count); > + avals->m_known_contexts[i] > = ipa_context_from_jfunc (caller_parms_info, e, i, jf); > } > } > @@ -685,18 +678,14 @@ evaluate_properties_for_edge (struct cgraph_edge *e, bool inline_p, > cst = NULL; > if (cst) > { > - if (!known_vals_ptr->length ()) > - vec_safe_grow_cleared (known_vals_ptr, count); > - (*known_vals_ptr)[i] = cst; > + if (!avals->m_known_vals.length ()) > + avals->m_known_vals.safe_grow_cleared (count); > + avals->m_known_vals[i] = cst; > } > } > } > > - evaluate_conditions_for_known_args (callee, inline_p, > - *known_vals_ptr, > - known_value_ranges, > - *known_aggs_ptr, > - clause_ptr, > + evaluate_conditions_for_known_args (callee, inline_p, avals, clause_ptr, > nonspec_clause_ptr); > } > > @@ -781,7 +770,7 @@ ipa_fn_summary_t::duplicate (cgraph_node *src, > vec<size_time_entry, va_gc> *entry = info->size_time_table; > /* Use SRC parm info since it may not be copied yet. */ > class ipa_node_params *parms_info = IPA_NODE_REF (src); > - vec<tree> known_vals = vNULL; > + ipa_call_arg_values avals; > int count = ipa_get_param_count (parms_info); > int i, j; > clause_t possible_truths; > @@ -792,7 +781,7 @@ ipa_fn_summary_t::duplicate (cgraph_node *src, > struct cgraph_edge *edge, *next; > > info->size_time_table = 0; > - known_vals.safe_grow_cleared (count); > + avals.m_known_vals.safe_grow_cleared (count); > for (i = 0; i < count; i++) > { > struct ipa_replace_map *r; > @@ -801,20 +790,17 @@ ipa_fn_summary_t::duplicate (cgraph_node *src, > { > if (r->parm_num == i) > { > - known_vals[i] = r->new_tree; > + avals.m_known_vals[i] = r->new_tree; > break; > } > } > } > evaluate_conditions_for_known_args (dst, false, > - known_vals, > - vNULL, > - vNULL, > + &avals, > &possible_truths, > /* We are going to specialize, > so ignore nonspec truths. */ > NULL); > - known_vals.release (); > > info->account_size_time (0, 0, true_pred, true_pred); > > @@ -3009,15 +2995,14 @@ compute_fn_summary_for_current (void) > return 0; > } > > -/* Estimate benefit devirtualizing indirect edge IE, provided KNOWN_VALS, > - KNOWN_CONTEXTS and KNOWN_AGGS. */ > +/* Estimate benefit devirtualizing indirect edge IE and return true if it can > + be devirtualized and inlined, provided m_known_vals, m_known_contexts and > + m_known_aggs in AVALS. Return false straight away if AVALS is NULL. */ > > static bool > estimate_edge_devirt_benefit (struct cgraph_edge *ie, > int *size, int *time, > - vec<tree> known_vals, > - vec<ipa_polymorphic_call_context> known_contexts, > - vec<ipa_agg_value_set> known_aggs) > + ipa_call_arg_values *avals) > { > tree target; > struct cgraph_node *callee; > @@ -3025,13 +3010,13 @@ estimate_edge_devirt_benefit (struct cgraph_edge *ie, > enum availability avail; > bool speculative; > > - if (!known_vals.length () && !known_contexts.length ()) > + if (!avals > + || (!avals->m_known_vals.length() && !avals->m_known_contexts.length ())) > return false; > if (!opt_for_fn (ie->caller->decl, flag_indirect_inlining)) > return false; > > - target = ipa_get_indirect_edge_target (ie, known_vals, known_contexts, > - known_aggs, &speculative); > + target = ipa_get_indirect_edge_target (ie, avals, &speculative); > if (!target || speculative) > return false; > > @@ -3055,17 +3040,13 @@ estimate_edge_devirt_benefit (struct cgraph_edge *ie, > } > > /* Increase SIZE, MIN_SIZE (if non-NULL) and TIME for size and time needed to > - handle edge E with probability PROB. > - Set HINTS if edge may be devirtualized. > - KNOWN_VALS, KNOWN_AGGS and KNOWN_CONTEXTS describe context of the call > - site. */ > + handle edge E with probability PROB. Set HINTS accordingly if edge may be > + devirtualized. AVALS, if non-NULL, describe the context of the call site as > + far as values of parameters are concerened. */ > > static inline void > estimate_edge_size_and_time (struct cgraph_edge *e, int *size, int *min_size, > - sreal *time, > - vec<tree> known_vals, > - vec<ipa_polymorphic_call_context> known_contexts, > - vec<ipa_agg_value_set> known_aggs, > + sreal *time, ipa_call_arg_values *avals, > ipa_hints *hints) > { > class ipa_call_summary *es = ipa_call_summaries->get (e); > @@ -3074,8 +3055,7 @@ estimate_edge_size_and_time (struct cgraph_edge *e, int *size, int *min_size, > int cur_size; > > if (!e->callee && hints && e->maybe_hot_p () > - && estimate_edge_devirt_benefit (e, &call_size, &call_time, > - known_vals, known_contexts, known_aggs)) > + && estimate_edge_devirt_benefit (e, &call_size, &call_time, avals)) > *hints |= INLINE_HINT_indirect_call; > cur_size = call_size * ipa_fn_summary::size_scale; > *size += cur_size; > @@ -3087,9 +3067,9 @@ estimate_edge_size_and_time (struct cgraph_edge *e, int *size, int *min_size, > > > /* Increase SIZE, MIN_SIZE and TIME for size and time needed to handle all > - calls in NODE. POSSIBLE_TRUTHS, KNOWN_VALS, KNOWN_AGGS and KNOWN_CONTEXTS > - describe context of the call site. > - > + calls in NODE. POSSIBLE_TRUTHS and AVALS describe the context of the call > + site. > + > Helper for estimate_calls_size_and_time which does the same but > (in most cases) faster. */ > > @@ -3098,9 +3078,7 @@ estimate_calls_size_and_time_1 (struct cgraph_node *node, int *size, > int *min_size, sreal *time, > ipa_hints *hints, > clause_t possible_truths, > - vec<tree> known_vals, > - vec<ipa_polymorphic_call_context> known_contexts, > - vec<ipa_agg_value_set> known_aggs) > + ipa_call_arg_values *avals) > { > struct cgraph_edge *e; > for (e = node->callees; e; e = e->next_callee) > @@ -3109,10 +3087,8 @@ estimate_calls_size_and_time_1 (struct cgraph_node *node, int *size, > { > gcc_checking_assert (!ipa_call_summaries->get (e)); > estimate_calls_size_and_time_1 (e->callee, size, min_size, time, > - hints, > - possible_truths, > - known_vals, known_contexts, > - known_aggs); > + hints, possible_truths, avals); > + > continue; > } > class ipa_call_summary *es = ipa_call_summaries->get (e); > @@ -3130,9 +3106,7 @@ estimate_calls_size_and_time_1 (struct cgraph_node *node, int *size, > so we do not need to compute probabilities. */ > estimate_edge_size_and_time (e, size, > es->predicate ? NULL : min_size, > - time, > - known_vals, known_contexts, > - known_aggs, hints); > + time, avals, hints); > } > } > for (e = node->indirect_calls; e; e = e->next_callee) > @@ -3142,9 +3116,7 @@ estimate_calls_size_and_time_1 (struct cgraph_node *node, int *size, > || es->predicate->evaluate (possible_truths)) > estimate_edge_size_and_time (e, size, > es->predicate ? NULL : min_size, > - time, > - known_vals, known_contexts, known_aggs, > - hints); > + time, avals, hints); > } > } > > @@ -3166,8 +3138,7 @@ summarize_calls_size_and_time (struct cgraph_node *node, > int size = 0; > sreal time = 0; > > - estimate_edge_size_and_time (e, &size, NULL, &time, > - vNULL, vNULL, vNULL, NULL); > + estimate_edge_size_and_time (e, &size, NULL, &time, NULL, NULL); > > struct predicate pred = true; > class ipa_call_summary *es = ipa_call_summaries->get (e); > @@ -3181,8 +3152,7 @@ summarize_calls_size_and_time (struct cgraph_node *node, > int size = 0; > sreal time = 0; > > - estimate_edge_size_and_time (e, &size, NULL, &time, > - vNULL, vNULL, vNULL, NULL); > + estimate_edge_size_and_time (e, &size, NULL, &time, NULL, NULL); > struct predicate pred = true; > class ipa_call_summary *es = ipa_call_summaries->get (e); > > @@ -3193,17 +3163,15 @@ summarize_calls_size_and_time (struct cgraph_node *node, > } > > /* Increase SIZE, MIN_SIZE and TIME for size and time needed to handle all > - calls in NODE. POSSIBLE_TRUTHS, KNOWN_VALS, KNOWN_AGGS and KNOWN_CONTEXTS > - describe context of the call site. */ > + calls in NODE. POSSIBLE_TRUTHS and AVALS (the latter if non-NULL) describe > + context of the call site. */ > > static void > estimate_calls_size_and_time (struct cgraph_node *node, int *size, > int *min_size, sreal *time, > ipa_hints *hints, > clause_t possible_truths, > - vec<tree> known_vals, > - vec<ipa_polymorphic_call_context> known_contexts, > - vec<ipa_agg_value_set> known_aggs) > + ipa_call_arg_values *avals) > { > class ipa_fn_summary *sum = ipa_fn_summaries->get (node); > bool use_table = true; > @@ -3222,9 +3190,10 @@ estimate_calls_size_and_time (struct cgraph_node *node, int *size, > use_table = false; > /* If there is an indirect edge that may be optimized, we need > to go the slow way. */ > - else if ((known_vals.length () > - || known_contexts.length () > - || known_aggs.length ()) && hints) > + else if (avals && hints > + && (avals->m_known_vals.length () > + || avals->m_known_contexts.length () > + || avals->m_known_aggs.length ())) > { > class ipa_node_params *params_summary = IPA_NODE_REF (node); > unsigned int nargs = params_summary > @@ -3233,13 +3202,13 @@ estimate_calls_size_and_time (struct cgraph_node *node, int *size, > for (unsigned int i = 0; i < nargs && use_table; i++) > { > if (ipa_is_param_used_by_indirect_call (params_summary, i) > - && ((known_vals.length () > i && known_vals[i]) > - || (known_aggs.length () > i > - && known_aggs[i].items.length ()))) > + && ((avals->m_known_vals.length () > i && avals->m_known_vals[i]) > + || (avals->m_known_aggs.length () > i > + && avals->m_known_aggs[i].items.length ()))) > use_table = false; > else if (ipa_is_param_used_by_polymorphic_call (params_summary, i) > - && (known_contexts.length () > i > - && !known_contexts[i].useless_p ())) > + && (avals->m_known_contexts.length () > i > + && !avals->m_known_contexts[i].useless_p ())) > use_table = false; > } > } > @@ -3282,8 +3251,7 @@ estimate_calls_size_and_time (struct cgraph_node *node, int *size, > < ipa_fn_summary::max_size_time_table_size) > { > estimate_calls_size_and_time_1 (node, &old_size, NULL, &old_time, NULL, > - possible_truths, known_vals, > - known_contexts, known_aggs); > + possible_truths, avals); > gcc_assert (*size == old_size); > if (time && (*time - old_time > 1 || *time - old_time < -1) > && dump_file) > @@ -3295,242 +3263,24 @@ estimate_calls_size_and_time (struct cgraph_node *node, int *size, > /* Slow path by walking all edges. */ > else > estimate_calls_size_and_time_1 (node, size, min_size, time, hints, > - possible_truths, known_vals, known_contexts, > - known_aggs); > + possible_truths, avals); > } > > -/* Default constructor for ipa call context. > - Memory allocation of known_vals, known_contexts > - and known_aggs vectors is owned by the caller, but can > - be release by ipa_call_context::release. > - > - inline_param_summary is owned by the caller. */ > -ipa_call_context::ipa_call_context (cgraph_node *node, > - clause_t possible_truths, > - clause_t nonspec_possible_truths, > - vec<tree> known_vals, > - vec<ipa_polymorphic_call_context> > - known_contexts, > - vec<ipa_agg_value_set> known_aggs, > - vec<inline_param_summary> > - inline_param_summary) > +/* Default constructor for ipa call context. Memory allocation of ARG_VALUES > + is owned by the caller. INLINE_PARAM_SUMMARY is also owned by the > + caller. */ > +ipa_call_context > +::ipa_call_context (cgraph_node *node, clause_t possible_truths, > + clause_t nonspec_possible_truths, > + vec<inline_param_summary> inline_param_summary, > + ipa_call_arg_values *arg_values) > : m_node (node), m_possible_truths (possible_truths), > m_nonspec_possible_truths (nonspec_possible_truths), > m_inline_param_summary (inline_param_summary), > - m_known_vals (known_vals), > - m_known_contexts (known_contexts), > - m_known_aggs (known_aggs) > + m_avals (arg_values) > { > } > > -/* Set THIS to be a duplicate of CTX. Copy all relevant info. */ > - > -void > -ipa_call_context::duplicate_from (const ipa_call_context &ctx) > -{ > - m_node = ctx.m_node; > - m_possible_truths = ctx.m_possible_truths; > - m_nonspec_possible_truths = ctx.m_nonspec_possible_truths; > - class ipa_node_params *params_summary = IPA_NODE_REF (m_node); > - unsigned int nargs = params_summary > - ? ipa_get_param_count (params_summary) : 0; > - > - m_inline_param_summary = vNULL; > - /* Copy the info only if there is at least one useful entry. */ > - if (ctx.m_inline_param_summary.exists ()) > - { > - unsigned int n = MIN (ctx.m_inline_param_summary.length (), nargs); > - > - for (unsigned int i = 0; i < n; i++) > - if (ipa_is_param_used_by_ipa_predicates (params_summary, i) > - && !ctx.m_inline_param_summary[i].useless_p ()) > - { > - m_inline_param_summary > - = ctx.m_inline_param_summary.copy (); > - break; > - } > - } > - m_known_vals = vNULL; > - if (ctx.m_known_vals.exists ()) > - { > - unsigned int n = MIN (ctx.m_known_vals.length (), nargs); > - > - for (unsigned int i = 0; i < n; i++) > - if (ipa_is_param_used_by_indirect_call (params_summary, i) > - && ctx.m_known_vals[i]) > - { > - m_known_vals = ctx.m_known_vals.copy (); > - break; > - } > - } > - > - m_known_contexts = vNULL; > - if (ctx.m_known_contexts.exists ()) > - { > - unsigned int n = MIN (ctx.m_known_contexts.length (), nargs); > - > - for (unsigned int i = 0; i < n; i++) > - if (ipa_is_param_used_by_polymorphic_call (params_summary, i) > - && !ctx.m_known_contexts[i].useless_p ()) > - { > - m_known_contexts = ctx.m_known_contexts.copy (); > - break; > - } > - } > - > - m_known_aggs = vNULL; > - if (ctx.m_known_aggs.exists ()) > - { > - unsigned int n = MIN (ctx.m_known_aggs.length (), nargs); > - > - for (unsigned int i = 0; i < n; i++) > - if (ipa_is_param_used_by_indirect_call (params_summary, i) > - && !ctx.m_known_aggs[i].is_empty ()) > - { > - m_known_aggs = ipa_copy_agg_values (ctx.m_known_aggs); > - break; > - } > - } > -} > - > -/* Release memory used by known_vals/contexts/aggs vectors. > - If ALL is true release also inline_param_summary. > - This happens when context was previously duplicated to be stored > - into cache. */ > - > -void > -ipa_call_context::release (bool all) > -{ > - /* See if context is initialized at first place. */ > - if (!m_node) > - return; > - ipa_release_agg_values (m_known_aggs, all); > - if (all) > - { > - m_known_vals.release (); > - m_known_contexts.release (); > - m_inline_param_summary.release (); > - } > -} > - > -/* Return true if CTX describes the same call context as THIS. */ > - > -bool > -ipa_call_context::equal_to (const ipa_call_context &ctx) > -{ > - if (m_node != ctx.m_node > - || m_possible_truths != ctx.m_possible_truths > - || m_nonspec_possible_truths != ctx.m_nonspec_possible_truths) > - return false; > - > - class ipa_node_params *params_summary = IPA_NODE_REF (m_node); > - unsigned int nargs = params_summary > - ? ipa_get_param_count (params_summary) : 0; > - > - if (m_inline_param_summary.exists () || ctx.m_inline_param_summary.exists ()) > - { > - for (unsigned int i = 0; i < nargs; i++) > - { > - if (!ipa_is_param_used_by_ipa_predicates (params_summary, i)) > - continue; > - if (i >= m_inline_param_summary.length () > - || m_inline_param_summary[i].useless_p ()) > - { > - if (i < ctx.m_inline_param_summary.length () > - && !ctx.m_inline_param_summary[i].useless_p ()) > - return false; > - continue; > - } > - if (i >= ctx.m_inline_param_summary.length () > - || ctx.m_inline_param_summary[i].useless_p ()) > - { > - if (i < m_inline_param_summary.length () > - && !m_inline_param_summary[i].useless_p ()) > - return false; > - continue; > - } > - if (!m_inline_param_summary[i].equal_to > - (ctx.m_inline_param_summary[i])) > - return false; > - } > - } > - if (m_known_vals.exists () || ctx.m_known_vals.exists ()) > - { > - for (unsigned int i = 0; i < nargs; i++) > - { > - if (!ipa_is_param_used_by_indirect_call (params_summary, i)) > - continue; > - if (i >= m_known_vals.length () || !m_known_vals[i]) > - { > - if (i < ctx.m_known_vals.length () && ctx.m_known_vals[i]) > - return false; > - continue; > - } > - if (i >= ctx.m_known_vals.length () || !ctx.m_known_vals[i]) > - { > - if (i < m_known_vals.length () && m_known_vals[i]) > - return false; > - continue; > - } > - if (m_known_vals[i] != ctx.m_known_vals[i]) > - return false; > - } > - } > - if (m_known_contexts.exists () || ctx.m_known_contexts.exists ()) > - { > - for (unsigned int i = 0; i < nargs; i++) > - { > - if (!ipa_is_param_used_by_polymorphic_call (params_summary, i)) > - continue; > - if (i >= m_known_contexts.length () > - || m_known_contexts[i].useless_p ()) > - { > - if (i < ctx.m_known_contexts.length () > - && !ctx.m_known_contexts[i].useless_p ()) > - return false; > - continue; > - } > - if (i >= ctx.m_known_contexts.length () > - || ctx.m_known_contexts[i].useless_p ()) > - { > - if (i < m_known_contexts.length () > - && !m_known_contexts[i].useless_p ()) > - return false; > - continue; > - } > - if (!m_known_contexts[i].equal_to > - (ctx.m_known_contexts[i])) > - return false; > - } > - } > - if (m_known_aggs.exists () || ctx.m_known_aggs.exists ()) > - { > - for (unsigned int i = 0; i < nargs; i++) > - { > - if (!ipa_is_param_used_by_indirect_call (params_summary, i)) > - continue; > - if (i >= m_known_aggs.length () || m_known_aggs[i].is_empty ()) > - { > - if (i < ctx.m_known_aggs.length () > - && !ctx.m_known_aggs[i].is_empty ()) > - return false; > - continue; > - } > - if (i >= ctx.m_known_aggs.length () > - || ctx.m_known_aggs[i].is_empty ()) > - { > - if (i < m_known_aggs.length () > - && !m_known_aggs[i].is_empty ()) > - return false; > - continue; > - } > - if (!m_known_aggs[i].equal_to (ctx.m_known_aggs[i])) > - return false; > - } > - } > - return true; > -} > - > /* Estimate size and time needed to execute call in the given context. > Additionally determine hints determined by the context. Finally compute > minimal size needed for the call that is independent on the call context and > @@ -3574,7 +3324,7 @@ ipa_call_context::estimate_size_and_time (int *ret_size, > estimate_calls_size_and_time (m_node, &size, &min_size, > ret_time ? &time : NULL, > ret_hints ? &hints : NULL, m_possible_truths, > - m_known_vals, m_known_contexts, m_known_aggs); > + m_avals); > > sreal nonspecialized_time = time; > > @@ -3681,22 +3431,17 @@ ipa_call_context::estimate_size_and_time (int *ret_size, > > void > estimate_ipcp_clone_size_and_time (struct cgraph_node *node, > - vec<tree> known_vals, > - vec<ipa_polymorphic_call_context> > - known_contexts, > - vec<ipa_agg_value_set> known_aggs, > + ipa_call_arg_values *avals, > int *ret_size, sreal *ret_time, > sreal *ret_nonspec_time, > ipa_hints *hints) > { > clause_t clause, nonspec_clause; > > - /* TODO: Also pass known value ranges. */ > - evaluate_conditions_for_known_args (node, false, known_vals, vNULL, > - known_aggs, &clause, &nonspec_clause); > + evaluate_conditions_for_known_args (node, false, avals, &clause, > + &nonspec_clause); > ipa_call_context ctx (node, clause, nonspec_clause, > - known_vals, known_contexts, > - known_aggs, vNULL); > + vNULL, avals); > ctx.estimate_size_and_time (ret_size, NULL, ret_time, > ret_nonspec_time, hints); > } > @@ -3914,10 +3659,9 @@ ipa_merge_fn_summary_after_inlining (struct cgraph_edge *edge) > > if (callee_info->conds) > { > - auto_vec<tree, 32> known_vals; > - auto_vec<ipa_agg_value_set, 32> known_aggs; > + ipa_call_arg_values avals; > evaluate_properties_for_edge (edge, true, &clause, NULL, > - &known_vals, NULL, &known_aggs); > + &avals, false); > } > if (ipa_node_params_sum && callee_info->conds) > { > @@ -4011,8 +3755,7 @@ ipa_merge_fn_summary_after_inlining (struct cgraph_edge *edge) > int edge_size = 0; > sreal edge_time = 0; > > - estimate_edge_size_and_time (edge, &edge_size, NULL, &edge_time, vNULL, > - vNULL, vNULL, 0); > + estimate_edge_size_and_time (edge, &edge_size, NULL, &edge_time, NULL, 0); > /* Unaccount size and time of the optimized out call. */ > info->account_size_time (-edge_size, -edge_time, > es->predicate ? *es->predicate : true, > @@ -4054,7 +3797,7 @@ ipa_update_overall_fn_summary (struct cgraph_node *node, bool reset) > estimate_calls_size_and_time (node, &size_info->size, &info->min_size, > &info->time, NULL, > ~(clause_t) (1 << predicate::false_condition), > - vNULL, vNULL, vNULL); > + NULL); > size_info->size = RDIV (size_info->size, ipa_fn_summary::size_scale); > info->min_size = RDIV (info->min_size, ipa_fn_summary::size_scale); > } > diff --git a/gcc/ipa-fnsummary.h b/gcc/ipa-fnsummary.h > index c6ddc9f3199..34050516d91 100644 > --- a/gcc/ipa-fnsummary.h > +++ b/gcc/ipa-fnsummary.h > @@ -287,6 +287,8 @@ public: > ipa_call_summary *dst_data); > }; > > +class ipa_node_context_cache_entry; > + > /* This object describe a context of call. That is a summary of known > information about its parameters. Main purpose of this context is > to give more realistic estimations of function runtime, size and > @@ -297,10 +299,8 @@ public: > ipa_call_context (cgraph_node *node, > clause_t possible_truths, > clause_t nonspec_possible_truths, > - vec<tree> known_vals, > - vec<ipa_polymorphic_call_context> known_contexts, > - vec<ipa_agg_value_set> known_aggs, > - vec<inline_param_summary> m_inline_param_summary); > + vec<inline_param_summary> inline_param_summary, > + ipa_call_arg_values *arg_values); > ipa_call_context () > : m_node(NULL) > { > @@ -309,9 +309,8 @@ public: > sreal *ret_time, > sreal *ret_nonspecialized_time, > ipa_hints *ret_hints); > - void duplicate_from (const ipa_call_context &ctx); > - void release (bool all = false); > - bool equal_to (const ipa_call_context &); > + void store_to_cache (ipa_node_context_cache_entry *cache) const; > + bool equivalent_to_p (const ipa_node_context_cache_entry &cache) const; > bool exists_p () > { > return m_node != NULL; > @@ -328,14 +327,9 @@ private: > /* Inline summary maintains info about change probabilities. */ > vec<inline_param_summary> m_inline_param_summary; > > - /* The following is used only to resolve indirect calls. */ > - > - /* Vector describing known values of parameters. */ > - vec<tree> m_known_vals; > - /* Vector describing known polymorphic call contexts. */ > - vec<ipa_polymorphic_call_context> m_known_contexts; > - /* Vector describing known aggregate values. */ > - vec<ipa_agg_value_set> m_known_aggs; > + /* Even after having calculated clauses, the information about argument > + values is used to resolve indirect calls. */ > + ipa_call_arg_values *m_avals; > }; > > extern fast_call_summary <ipa_call_summary *, va_heap> *ipa_call_summaries; > @@ -349,9 +343,7 @@ void ipa_free_fn_summary (void); > void ipa_free_size_summary (void); > void inline_analyze_function (struct cgraph_node *node); > void estimate_ipcp_clone_size_and_time (struct cgraph_node *, > - vec<tree>, > - vec<ipa_polymorphic_call_context>, > - vec<ipa_agg_value_set>, > + ipa_call_arg_values *, > int *, sreal *, sreal *, > ipa_hints *); > void ipa_merge_fn_summary_after_inlining (struct cgraph_edge *edge); > @@ -363,10 +355,8 @@ void evaluate_properties_for_edge (struct cgraph_edge *e, > bool inline_p, > clause_t *clause_ptr, > clause_t *nonspec_clause_ptr, > - vec<tree> *known_vals_ptr, > - vec<ipa_polymorphic_call_context> > - *known_contexts_ptr, > - vec<ipa_agg_value_set> *); > + ipa_call_arg_values *avals, > + bool compute_contexts); > > void ipa_fnsummary_c_finalize (void); > HOST_WIDE_INT ipa_get_stack_frame_offset (struct cgraph_node *node); > diff --git a/gcc/ipa-inline-analysis.c b/gcc/ipa-inline-analysis.c > index 148efbc09ef..d8cb9294c36 100644 > --- a/gcc/ipa-inline-analysis.c > +++ b/gcc/ipa-inline-analysis.c > @@ -52,31 +52,253 @@ along with GCC; see the file COPYING3. If not see > /* Cached node/edge growths. */ > fast_call_summary<edge_growth_cache_entry *, va_heap> *edge_growth_cache = NULL; > > -/* The context cache remembers estimated time/size and hints for given > - ipa_call_context of a call. */ > -class node_context_cache_entry > +/* This class caches estimated times, size and hints for an ipa_call_context > + together with all information necessary to figure out whether the original > + context is th same as another one for the purposes of these estimations. */ > + > +class ipa_node_context_cache_entry > { > public: > - ipa_call_context ctx; > - sreal time, nonspec_time; > - int size; > - ipa_hints hints; > + friend class ipa_call_context; > > - node_context_cache_entry () > - : ctx () > + ~ipa_node_context_cache_entry () > { > + release (); > } > - ~node_context_cache_entry () > - { > - ctx.release (); > - } > + /* Free all memory held by member vectors of this class. */ > + void release (); > + > + /* Estimated time and nonspecialized time. */ > + sreal m_time = 0; > + sreal m_nonspec_time = 0; > + /* Estimated size. */ > + int m_size = 0; > + /* Estimated hints. */ > + ipa_hints m_hints = 0; > + > +private: > + /* m_possible_truths copied from the initial ipa_call_context. */ > + clause_t m_possible_truths = 0; > + /* m_nonspec_possible_truths copied from the initial ipa_call_context. */ > + clause_t m_nonspec_possible_truths = 0; > + /* m_inline_param_summary copied from the initial ipa_call_context. */ > + vec<inline_param_summary> m_inline_param_summary = vNULL; > + /* Copy of m_known_vals of m_avals of the initial ipa_call_context. */ > + vec<tree> m_known_vals = vNULL; > + /* Copy of m_known_contexts of m_avals of the initial ipa_call_context. */ > + vec<ipa_polymorphic_call_context> m_known_contexts = vNULL; > + /* Copy of m_known_vals of m_avals of the initial ipa_call_context. */ > + vec<ipa_agg_value_set> m_known_aggs = vNULL; > + /* m_known_value_ranges from m_avals of the initial ipa_call_context is not > + cached (yet), it turns out not to be necessary any more as it is not used > + for discovering targets of indirect call graph edges. */ > + > }; > > +void > +ipa_node_context_cache_entry::release () > +{ > + ipa_release_agg_values (m_known_aggs, true); > + m_known_vals.release (); > + m_known_contexts.release (); > + m_inline_param_summary.release (); > +} > + > +/* Copy all information necessary to subsequently identify equivalent contexts > + to CACHE. */ > + > +void > +ipa_call_context::store_to_cache (ipa_node_context_cache_entry *cache) const > +{ > + cache->m_possible_truths = m_possible_truths; > + cache->m_nonspec_possible_truths = m_nonspec_possible_truths; > + class ipa_node_params *params_summary = IPA_NODE_REF (m_node); > + unsigned int nargs = params_summary > + ? ipa_get_param_count (params_summary) : 0; > + > + cache->m_inline_param_summary = vNULL; > + /* Copy the info only if there is at least one useful entry. */ > + if (m_inline_param_summary.exists ()) > + { > + unsigned int n = MIN (m_inline_param_summary.length (), nargs); > + > + for (unsigned int i = 0; i < n; i++) > + if (ipa_is_param_used_by_ipa_predicates (params_summary, i) > + && !m_inline_param_summary[i].useless_p ()) > + { > + cache->m_inline_param_summary = m_inline_param_summary.copy (); > + break; > + } > + } > + cache->m_known_vals = vNULL; > + if (m_avals->m_known_vals.exists ()) > + { > + unsigned int n = MIN (m_avals->m_known_vals.length (), nargs); > + > + for (unsigned int i = 0; i < n; i++) > + if (ipa_is_param_used_by_indirect_call (params_summary, i) > + && m_avals->m_known_vals[i]) > + { > + cache->m_known_vals = m_avals->m_known_vals.copy (); > + break; > + } > + } > + > + cache->m_known_contexts = vNULL; > + if (m_avals->m_known_contexts.exists ()) > + { > + unsigned int n = MIN (m_avals->m_known_contexts.length (), nargs); > + > + for (unsigned int i = 0; i < n; i++) > + if (ipa_is_param_used_by_polymorphic_call (params_summary, i) > + && !m_avals->m_known_contexts[i].useless_p ()) > + { > + cache->m_known_contexts = m_avals->m_known_contexts.copy (); > + break; > + } > + } > + > + cache->m_known_aggs = vNULL; > + if (m_avals->m_known_aggs.exists ()) > + { > + unsigned int n = MIN (m_avals->m_known_aggs.length (), nargs); > + > + for (unsigned int i = 0; i < n; i++) > + if (ipa_is_param_used_by_indirect_call (params_summary, i) > + && !m_avals->m_known_aggs[i].is_empty ()) > + { > + cache->m_known_aggs = ipa_copy_agg_values (m_avals->m_known_aggs); > + break; > + } > + } > +} > + > +/* Return true if CACHE describes the same call context as THIS for the > + purposes of context caching. */ > + > +bool > +ipa_call_context::equivalent_to_p (const ipa_node_context_cache_entry &cache) > + const > +{ > + if (m_possible_truths != cache.m_possible_truths > + || m_nonspec_possible_truths != cache.m_nonspec_possible_truths) > + return false; > + > + class ipa_node_params *params_summary = IPA_NODE_REF (m_node); > + unsigned int nargs = params_summary > + ? ipa_get_param_count (params_summary) : 0; > + > + if (m_inline_param_summary.exists () || cache.m_inline_param_summary.exists ()) > + { > + for (unsigned int i = 0; i < nargs; i++) > + { > + if (!ipa_is_param_used_by_ipa_predicates (params_summary, i)) > + continue; > + if (i >= m_inline_param_summary.length () > + || m_inline_param_summary[i].useless_p ()) > + { > + if (i < cache.m_inline_param_summary.length () > + && !cache.m_inline_param_summary[i].useless_p ()) > + return false; > + continue; > + } > + if (i >= cache.m_inline_param_summary.length () > + || cache.m_inline_param_summary[i].useless_p ()) > + { > + if (i < m_inline_param_summary.length () > + && !m_inline_param_summary[i].useless_p ()) > + return false; > + continue; > + } > + if (!m_inline_param_summary[i].equal_to > + (cache.m_inline_param_summary[i])) > + return false; > + } > + } > + if (m_avals->m_known_vals.exists () || cache.m_known_vals.exists ()) > + { > + for (unsigned int i = 0; i < nargs; i++) > + { > + if (!ipa_is_param_used_by_indirect_call (params_summary, i)) > + continue; > + if (i >= m_avals->m_known_vals.length () || !m_avals->m_known_vals[i]) > + { > + if (i < cache.m_known_vals.length () && cache.m_known_vals[i]) > + return false; > + continue; > + } > + if (i >= cache.m_known_vals.length () || !cache.m_known_vals[i]) > + { > + if (i < m_avals->m_known_vals.length () > + && m_avals->m_known_vals[i]) > + return false; > + continue; > + } > + if (m_avals->m_known_vals[i] != cache.m_known_vals[i]) > + return false; > + } > + } > + if (m_avals->m_known_contexts.exists () || cache.m_known_contexts.exists ()) > + { > + for (unsigned int i = 0; i < nargs; i++) > + { > + if (!ipa_is_param_used_by_polymorphic_call (params_summary, i)) > + continue; > + if (i >= m_avals->m_known_contexts.length () > + || m_avals->m_known_contexts[i].useless_p ()) > + { > + if (i < cache.m_known_contexts.length () > + && !cache.m_known_contexts[i].useless_p ()) > + return false; > + continue; > + } > + if (i >= cache.m_known_contexts.length () > + || cache.m_known_contexts[i].useless_p ()) > + { > + if (i < m_avals->m_known_contexts.length () > + && !m_avals->m_known_contexts[i].useless_p ()) > + return false; > + continue; > + } > + if (!m_avals->m_known_contexts[i].equal_to > + (cache.m_known_contexts[i])) > + return false; > + } > + } > + if (m_avals->m_known_aggs.exists () || cache.m_known_aggs.exists ()) > + { > + for (unsigned int i = 0; i < nargs; i++) > + { > + if (!ipa_is_param_used_by_indirect_call (params_summary, i)) > + continue; > + if (i >= m_avals->m_known_aggs.length () > + || m_avals->m_known_aggs[i].is_empty ()) > + { > + if (i < cache.m_known_aggs.length () > + && !cache.m_known_aggs[i].is_empty ()) > + return false; > + continue; > + } > + if (i >= cache.m_known_aggs.length () > + || cache.m_known_aggs[i].is_empty ()) > + { > + if (i < m_avals->m_known_aggs.length () > + && !m_avals->m_known_aggs[i].is_empty ()) > + return false; > + continue; > + } > + if (!m_avals->m_known_aggs[i].equal_to (cache.m_known_aggs[i])) > + return false; > + } > + } > + return true; > +} > + > /* At the moment we implement primitive single entry LRU cache. */ > class node_context_summary > { > public: > - node_context_cache_entry entry; > + ipa_node_context_cache_entry entry; > > node_context_summary () > : entry () > @@ -184,30 +406,26 @@ do_estimate_edge_time (struct cgraph_edge *edge, sreal *ret_nonspec_time) > ipa_hints hints; > struct cgraph_node *callee; > clause_t clause, nonspec_clause; > - auto_vec<tree, 32> known_vals; > - auto_vec<ipa_polymorphic_call_context, 32> known_contexts; > - auto_vec<ipa_agg_value_set, 32> known_aggs; > + ipa_call_arg_values avals; > class ipa_call_summary *es = ipa_call_summaries->get (edge); > int min_size = -1; > > callee = edge->callee->ultimate_alias_target (); > > gcc_checking_assert (edge->inline_failed); > - evaluate_properties_for_edge (edge, true, > - &clause, &nonspec_clause, &known_vals, > - &known_contexts, &known_aggs); > - ipa_call_context ctx (callee, clause, nonspec_clause, known_vals, > - known_contexts, known_aggs, es->param); > + evaluate_properties_for_edge (edge, true, &clause, &nonspec_clause, > + &avals, true); > + ipa_call_context ctx (callee, clause, nonspec_clause, es->param, &avals); > if (node_context_cache != NULL) > { > - node_context_summary *e = node_context_cache->get_create (callee); > - if (e->entry.ctx.equal_to (ctx)) > + node_context_summary *e = node_context_cache->get (callee); > + if (e && ctx.equivalent_to_p (e->entry)) > { > node_context_cache_hit++; > - size = e->entry.size; > - time = e->entry.time; > - nonspec_time = e->entry.nonspec_time; > - hints = e->entry.hints; > + size = e->entry.m_size; > + time = e->entry.m_time; > + nonspec_time = e->entry.m_nonspec_time; > + hints = e->entry.m_hints; > if (flag_checking > && !opt_for_fn (callee->decl, flag_profile_partial_training) > && !callee->count.ipa_p ()) > @@ -226,18 +444,21 @@ do_estimate_edge_time (struct cgraph_edge *edge, sreal *ret_nonspec_time) > } > else > { > - if (e->entry.ctx.exists_p ()) > - node_context_cache_miss++; > + if (e) > + { > + node_context_cache_miss++; > + e->entry.release (); > + } > else > - node_context_cache_clear++; > - e->entry.ctx.release (true); > + e = node_context_cache->get_create (callee); > + > ctx.estimate_size_and_time (&size, &min_size, > &time, &nonspec_time, &hints); > - e->entry.size = size; > - e->entry.time = time; > - e->entry.nonspec_time = nonspec_time; > - e->entry.hints = hints; > - e->entry.ctx.duplicate_from (ctx); > + ctx.store_to_cache (&e->entry); > + e->entry.m_size = size; > + e->entry.m_time = time; > + e->entry.m_nonspec_time = nonspec_time; > + e->entry.m_hints = hints; > } > } > else > @@ -255,7 +476,6 @@ do_estimate_edge_time (struct cgraph_edge *edge, sreal *ret_nonspec_time) > : edge->caller->count.ipa ()))) > hints |= INLINE_HINT_known_hot; > > - ctx.release (); > gcc_checking_assert (size >= 0); > gcc_checking_assert (time >= 0); > > @@ -272,6 +492,8 @@ do_estimate_edge_time (struct cgraph_edge *edge, sreal *ret_nonspec_time) > > entry->size = size + (size >= 0); > hints |= simple_edge_hints (edge); > + /* This is how we distinguish valid entries, consumer must decrement hint > + to get the real value. */ > entry->hints = hints + 1; > } > if (ret_nonspec_time) > @@ -307,9 +529,7 @@ do_estimate_edge_size (struct cgraph_edge *edge) > int size; > struct cgraph_node *callee; > clause_t clause, nonspec_clause; > - auto_vec<tree, 32> known_vals; > - auto_vec<ipa_polymorphic_call_context, 32> known_contexts; > - auto_vec<ipa_agg_value_set, 32> known_aggs; > + ipa_call_arg_values avals; > > /* When we do caching, use do_estimate_edge_time to populate the entry. */ > > @@ -325,14 +545,10 @@ do_estimate_edge_size (struct cgraph_edge *edge) > > /* Early inliner runs without caching, go ahead and do the dirty work. */ > gcc_checking_assert (edge->inline_failed); > - evaluate_properties_for_edge (edge, true, > - &clause, &nonspec_clause, > - &known_vals, &known_contexts, > - &known_aggs); > - ipa_call_context ctx (callee, clause, nonspec_clause, known_vals, > - known_contexts, known_aggs, vNULL); > + evaluate_properties_for_edge (edge, true, &clause, &nonspec_clause, &avals, > + true); > + ipa_call_context ctx (callee, clause, nonspec_clause, vNULL, &avals); > ctx.estimate_size_and_time (&size, NULL, NULL, NULL, NULL); > - ctx.release (); > return size; > } > > @@ -346,9 +562,7 @@ do_estimate_edge_hints (struct cgraph_edge *edge) > ipa_hints hints; > struct cgraph_node *callee; > clause_t clause, nonspec_clause; > - auto_vec<tree, 32> known_vals; > - auto_vec<ipa_polymorphic_call_context, 32> known_contexts; > - auto_vec<ipa_agg_value_set, 32> known_aggs; > + ipa_call_arg_values avals; > > /* When we do caching, use do_estimate_edge_time to populate the entry. */ > > @@ -364,14 +578,10 @@ do_estimate_edge_hints (struct cgraph_edge *edge) > > /* Early inliner runs without caching, go ahead and do the dirty work. */ > gcc_checking_assert (edge->inline_failed); > - evaluate_properties_for_edge (edge, true, > - &clause, &nonspec_clause, > - &known_vals, &known_contexts, > - &known_aggs); > - ipa_call_context ctx (callee, clause, nonspec_clause, known_vals, > - known_contexts, known_aggs, vNULL); > + evaluate_properties_for_edge (edge, true, &clause, &nonspec_clause, &avals, > + true); > + ipa_call_context ctx (callee, clause, nonspec_clause, vNULL, &avals); > ctx.estimate_size_and_time (NULL, NULL, NULL, NULL, &hints); > - ctx.release (); > hints |= simple_edge_hints (edge); > return hints; > } > diff --git a/gcc/ipa-prop.c b/gcc/ipa-prop.c > index da50f0837fd..1220ba9351a 100644 > --- a/gcc/ipa-prop.c > +++ b/gcc/ipa-prop.c > @@ -5795,4 +5795,14 @@ ipa_agg_value::equal_to (const ipa_agg_value &other) > return offset == other.offset > && operand_equal_p (value, other.value, 0); > } > + > +/* Clean-up also */ > + > +ipa_call_arg_values::~ipa_call_arg_values () > +{ > + ipa_release_agg_values (m_known_aggs, false); > +} > + > + > + > #include "gt-ipa-prop.h" > diff --git a/gcc/ipa-prop.h b/gcc/ipa-prop.h > index 23fcf905ef3..cc35848f53e 100644 > --- a/gcc/ipa-prop.h > +++ b/gcc/ipa-prop.h > @@ -433,6 +433,48 @@ ipa_get_jf_ancestor_type_preserved (struct ipa_jump_func *jfunc) > return jfunc->value.ancestor.agg_preserved; > } > > +/* Class bundling the various potentially known properties about actual > + arguments of a particular call. */ > + > +class ipa_call_arg_values > +{ > +public: > + ~ipa_call_arg_values (); > + > + /* If m_known_vals (vector of known "scalar" values) is sufficiantly long, > + return its element at INDEX, otherwise return NULL. */ > + tree safe_sval_at (int index) > + { > + /* TODO: Assert non-negative index here and test. */ > + if ((unsigned) index < m_known_vals.length ()) > + return m_known_vals[index]; > + return NULL; > + } > + > + /* If m_known_aggs is sufficiantly long, return the pointer rto its element > + at INDEX, otherwise return NULL. */ > + ipa_agg_value_set *safe_aggval_at (int index) > + { > + /* TODO: Assert non-negative index here and test. */ > + if ((unsigned) index < m_known_aggs.length ()) > + return &m_known_aggs[index]; > + return NULL; > + } > + > + /* Vector describing known values of parameters. */ > + auto_vec<tree, 32> m_known_vals; > + > + /* Vector describing known polymorphic call contexts. */ > + auto_vec<ipa_polymorphic_call_context, 32> m_known_contexts; > + > + /* Vector describing known aggregate values. */ > + auto_vec<ipa_agg_value_set, 32> m_known_aggs; > + > + /* Vector describing known value ranges of arguments. */ > + auto_vec<value_range, 32> m_known_value_ranges; > +}; > + > + > /* Summary describing a single formal parameter. */ > > struct GTY(()) ipa_param_descriptor > @@ -970,12 +1012,10 @@ void ipa_initialize_node_params (struct cgraph_node *node); > bool ipa_propagate_indirect_call_infos (struct cgraph_edge *cs, > vec<cgraph_edge *> *new_edges); > > -/* Indirect edge and binfo processing. */ > +/* Indirect edge processing and target discovery. */ > tree ipa_get_indirect_edge_target (struct cgraph_edge *ie, > - vec<tree>, > - vec<ipa_polymorphic_call_context>, > - vec<ipa_agg_value_set>, > - bool *); > + ipa_call_arg_values *avals, > + bool *speculative); > struct cgraph_edge *ipa_make_edge_direct_to_target (struct cgraph_edge *, tree, > bool speculative = false); > tree ipa_impossible_devirt_target (struct cgraph_edge *, tree); > -- > 2.28.0 >
Hi, On Sat, Aug 29 2020, Jan Hubicka wrote: >> Hi, >> >> this large patch is a semi-mechanical change which aims to replace >> uses of separate vectors about known scalar values (usually called >> known_vals or known_csts), known aggregate values (known_aggs), known >> virtual call contexts (known_contexts) and known value >> ranges (known_value_ranges) with uses of one type called >> ipa_call_arg_values. The main benefit is bringing down the number of >> arguments that various call context evaluating functions have. >> Because the new type uses auto_vecs deallocation is simpler and it >> also takes advantage of storage within the auto_vecs themselves, >> eliminating the need for allocating memory when analyzing most >> functions. >> >> The one place where this patch is not mechanical is the >> node_context_cache_entry RCU cache. Before the elements contained >> directly ipa_call_context instances which however owned their vectors, >> they did not share them with their users like non-cache contexts. Now >> the vectors are behind the pointer which means ipa_call_arg_values >> would have to be allocated, for many functions at once, and so it >> could not make use of auto_vecs with generous internal storage. >> >> I avoided both by reworking the cache to contain copies of all fields >> o interest of ipa_call_context, including the interesting vectors. >> This makes the type a bit more verbose but the main functionality, >> storing to the cache and comparing a context with a cache entry, >> remained very similar. > > Can you pleae benchmark this on building Firfox or something similar? > I was running into two problems here. First was overhead of malloc > allocations and that is reason why I am trying to not allocate the > vector when not necessary and also place them on stack rather then heap > for those having short lifetime (not being in cache). > > Second problem is memory use - we may end up with quite many contextes > since calls are very numerous. > I can but the patch does not change where the vectors are allocated and does not store more stuff to cache than the current code. In fact, because the new cache does not store m_node, it saves one pointer. When I wrote the patch "copies of all fields of interest of ipa_call_context" I meant in the source code of GCC, from one class into another, not when the cache operates. The storing mechanism was created by modifying the current duplicate_from so that it operates on two types instead of one, it still meticulously makes sure it copies only useful data and all that. But sure, I will double check the series does not increase memory use. Martin
diff --git a/gcc/ipa-cp.c b/gcc/ipa-cp.c index e4910a04ffa..970675efe49 100644 --- a/gcc/ipa-cp.c +++ b/gcc/ipa-cp.c @@ -3117,30 +3117,26 @@ ipa_get_indirect_edge_target_1 (struct cgraph_edge *ie, return target; } - -/* If an indirect edge IE can be turned into a direct one based on KNOWN_CSTS, - KNOWN_CONTEXTS (which can be vNULL) or KNOWN_AGGS (which also can be vNULL) - return the destination. */ +/* If an indirect edge IE can be turned into a direct one based on data in + AVALS, return the destination. Store into *SPECULATIVE a boolean determinig + whether the discovered target is only speculative guess. */ tree ipa_get_indirect_edge_target (struct cgraph_edge *ie, - vec<tree> known_csts, - vec<ipa_polymorphic_call_context> known_contexts, - vec<ipa_agg_value_set> known_aggs, + ipa_call_arg_values *avals, bool *speculative) { - return ipa_get_indirect_edge_target_1 (ie, known_csts, known_contexts, - known_aggs, NULL, speculative); + return ipa_get_indirect_edge_target_1 (ie, avals->m_known_vals, + avals->m_known_contexts, + avals->m_known_aggs, NULL, speculative); } -/* Calculate devirtualization time bonus for NODE, assuming we know KNOWN_CSTS - and KNOWN_CONTEXTS. */ +/* Calculate devirtualization time bonus for NODE, assuming we know information + about arguments stored in AVALS. */ static int devirtualization_time_bonus (struct cgraph_node *node, - vec<tree> known_csts, - vec<ipa_polymorphic_call_context> known_contexts, - vec<ipa_agg_value_set> known_aggs) + ipa_call_arg_values *avals) { struct cgraph_edge *ie; int res = 0; @@ -3153,8 +3149,7 @@ devirtualization_time_bonus (struct cgraph_node *node, tree target; bool speculative; - target = ipa_get_indirect_edge_target (ie, known_csts, known_contexts, - known_aggs, &speculative); + target = ipa_get_indirect_edge_target (ie, avals, &speculative); if (!target) continue; @@ -3306,31 +3301,31 @@ context_independent_aggregate_values (class ipcp_param_lattices *plats) return res; } -/* Allocate KNOWN_CSTS, KNOWN_CONTEXTS and, if non-NULL, KNOWN_AGGS and - populate them with values of parameters that are known independent of the - context. INFO describes the function. If REMOVABLE_PARAMS_COST is - non-NULL, the movement cost of all removable parameters will be stored in - it. */ +/* Grow vectors in AVALS and fill them with information about values of + parameters that are known to be independent of the context. Only calculate + m_known_aggs if CALCULATE_AGGS is true. INFO describes the function. If + REMOVABLE_PARAMS_COST is non-NULL, the movement cost of all removable + parameters will be stored in it. + + TODO: Also grow context independent value range vectors. */ static bool gather_context_independent_values (class ipa_node_params *info, - vec<tree> *known_csts, - vec<ipa_polymorphic_call_context> - *known_contexts, - vec<ipa_agg_value_set> *known_aggs, + ipa_call_arg_values *avals, + bool calculate_aggs, int *removable_params_cost) { int i, count = ipa_get_param_count (info); bool ret = false; - known_csts->create (0); - known_contexts->create (0); - known_csts->safe_grow_cleared (count); - known_contexts->safe_grow_cleared (count); - if (known_aggs) + avals->m_known_vals.reserve_exact (count); + avals->m_known_vals.quick_grow_cleared (count); + avals->m_known_contexts.reserve_exact (count); + avals->m_known_contexts.quick_grow_cleared (count); + if (calculate_aggs) { - known_aggs->create (0); - known_aggs->safe_grow_cleared (count); + avals->m_known_aggs.reserve_exact (count); + avals->m_known_aggs.safe_grow_cleared (count); } if (removable_params_cost) @@ -3345,7 +3340,7 @@ gather_context_independent_values (class ipa_node_params *info, { ipcp_value<tree> *val = lat->values; gcc_checking_assert (TREE_CODE (val->value) != TREE_BINFO); - (*known_csts)[i] = val->value; + avals->m_known_vals[i] = val->value; if (removable_params_cost) *removable_params_cost += estimate_move_cost (TREE_TYPE (val->value), false); @@ -3363,15 +3358,15 @@ gather_context_independent_values (class ipa_node_params *info, /* Do not account known context as reason for cloning. We can see if it permits devirtualization. */ if (ctxlat->is_single_const ()) - (*known_contexts)[i] = ctxlat->values->value; + avals->m_known_contexts[i] = ctxlat->values->value; - if (known_aggs) + if (calculate_aggs) { vec<ipa_agg_value> agg_items; struct ipa_agg_value_set *agg; agg_items = context_independent_aggregate_values (plats); - agg = &(*known_aggs)[i]; + agg = &avals->m_known_aggs[i]; agg->items = agg_items; agg->by_ref = plats->aggs_by_ref; ret |= !agg_items.is_empty (); @@ -3381,25 +3376,22 @@ gather_context_independent_values (class ipa_node_params *info, return ret; } -/* Perform time and size measurement of NODE with the context given in - KNOWN_CSTS, KNOWN_CONTEXTS and KNOWN_AGGS, calculate the benefit and cost - given BASE_TIME of the node without specialization, REMOVABLE_PARAMS_COST of - all context-independent removable parameters and EST_MOVE_COST of estimated - movement of the considered parameter and store it into VAL. */ +/* Perform time and size measurement of NODE with the context given in AVALS, + calculate the benefit compared to the node without specialization and store + it into VAL. Also, if non-NULL, calculate REMOVABLE_PARAMS_COST of all + context-independent or unused removable parameters and EST_MOVE_COST, the + estimated movement of the considered parameter. */ static void -perform_estimation_of_a_value (cgraph_node *node, vec<tree> known_csts, - vec<ipa_polymorphic_call_context> known_contexts, - vec<ipa_agg_value_set> known_aggs, - int removable_params_cost, - int est_move_cost, ipcp_value_base *val) +perform_estimation_of_a_value (cgraph_node *node, ipa_call_arg_values *avals, + int removable_params_cost, int est_move_cost, + ipcp_value_base *val) { int size, time_benefit; sreal time, base_time; ipa_hints hints; - estimate_ipcp_clone_size_and_time (node, known_csts, known_contexts, - known_aggs, &size, &time, + estimate_ipcp_clone_size_and_time (node, avals, &size, &time, &base_time, &hints); base_time -= time; if (base_time > 65535) @@ -3412,8 +3404,7 @@ perform_estimation_of_a_value (cgraph_node *node, vec<tree> known_csts, time_benefit = 0; else time_benefit = base_time.to_int () - + devirtualization_time_bonus (node, known_csts, known_contexts, - known_aggs) + + devirtualization_time_bonus (node, avals) + hint_time_bonus (node, hints) + removable_params_cost + est_move_cost; @@ -3454,9 +3445,7 @@ estimate_local_effects (struct cgraph_node *node) { class ipa_node_params *info = IPA_NODE_REF (node); int i, count = ipa_get_param_count (info); - vec<tree> known_csts; - vec<ipa_polymorphic_call_context> known_contexts; - vec<ipa_agg_value_set> known_aggs; + ipa_call_arg_values avals; bool always_const; int removable_params_cost; @@ -3466,11 +3455,9 @@ estimate_local_effects (struct cgraph_node *node) if (dump_file && (dump_flags & TDF_DETAILS)) fprintf (dump_file, "\nEstimating effects for %s.\n", node->dump_name ()); - always_const = gather_context_independent_values (info, &known_csts, - &known_contexts, &known_aggs, + always_const = gather_context_independent_values (info, &avals, true, &removable_params_cost); - int devirt_bonus = devirtualization_time_bonus (node, known_csts, - known_contexts, known_aggs); + int devirt_bonus = devirtualization_time_bonus (node, &avals); if (always_const || devirt_bonus || (removable_params_cost && node->can_change_signature)) { @@ -3482,8 +3469,7 @@ estimate_local_effects (struct cgraph_node *node) init_caller_stats (&stats); node->call_for_symbol_thunks_and_aliases (gather_caller_stats, &stats, false); - estimate_ipcp_clone_size_and_time (node, known_csts, known_contexts, - known_aggs, &size, &time, + estimate_ipcp_clone_size_and_time (node, &avals, &size, &time, &base_time, &hints); time -= devirt_bonus; time -= hint_time_bonus (node, hints); @@ -3536,18 +3522,17 @@ estimate_local_effects (struct cgraph_node *node) if (lat->bottom || !lat->values - || known_csts[i]) + || avals.m_known_vals[i]) continue; for (val = lat->values; val; val = val->next) { gcc_checking_assert (TREE_CODE (val->value) != TREE_BINFO); - known_csts[i] = val->value; + avals.m_known_vals[i] = val->value; int emc = estimate_move_cost (TREE_TYPE (val->value), true); - perform_estimation_of_a_value (node, known_csts, known_contexts, - known_aggs, - removable_params_cost, emc, val); + perform_estimation_of_a_value (node, &avals, removable_params_cost, + emc, val); if (dump_file && (dump_flags & TDF_DETAILS)) { @@ -3559,7 +3544,7 @@ estimate_local_effects (struct cgraph_node *node) val->local_time_benefit, val->local_size_cost); } } - known_csts[i] = NULL_TREE; + avals.m_known_vals[i] = NULL_TREE; } for (i = 0; i < count; i++) @@ -3574,15 +3559,14 @@ estimate_local_effects (struct cgraph_node *node) if (ctxlat->bottom || !ctxlat->values - || !known_contexts[i].useless_p ()) + || !avals.m_known_contexts[i].useless_p ()) continue; for (val = ctxlat->values; val; val = val->next) { - known_contexts[i] = val->value; - perform_estimation_of_a_value (node, known_csts, known_contexts, - known_aggs, - removable_params_cost, 0, val); + avals.m_known_contexts[i] = val->value; + perform_estimation_of_a_value (node, &avals, removable_params_cost, + 0, val); if (dump_file && (dump_flags & TDF_DETAILS)) { @@ -3594,20 +3578,18 @@ estimate_local_effects (struct cgraph_node *node) val->local_time_benefit, val->local_size_cost); } } - known_contexts[i] = ipa_polymorphic_call_context (); + avals.m_known_contexts[i] = ipa_polymorphic_call_context (); } for (i = 0; i < count; i++) { class ipcp_param_lattices *plats = ipa_get_parm_lattices (info, i); - struct ipa_agg_value_set *agg; - struct ipcp_agg_lattice *aglat; if (plats->aggs_bottom || !plats->aggs) continue; - agg = &known_aggs[i]; - for (aglat = plats->aggs; aglat; aglat = aglat->next) + ipa_agg_value_set *agg = &avals.m_known_aggs[i]; + for (ipcp_agg_lattice *aglat = plats->aggs; aglat; aglat = aglat->next) { ipcp_value<tree> *val; if (aglat->bottom || !aglat->values @@ -3624,9 +3606,8 @@ estimate_local_effects (struct cgraph_node *node) item.value = val->value; agg->items.safe_push (item); - perform_estimation_of_a_value (node, known_csts, known_contexts, - known_aggs, - removable_params_cost, 0, val); + perform_estimation_of_a_value (node, &avals, removable_params_cost, + 0, val); if (dump_file && (dump_flags & TDF_DETAILS)) { @@ -3645,10 +3626,6 @@ estimate_local_effects (struct cgraph_node *node) } } } - - known_csts.release (); - known_contexts.release (); - ipa_release_agg_values (known_aggs); } @@ -5371,31 +5348,32 @@ copy_useful_known_contexts (vec<ipa_polymorphic_call_context> known_contexts) return vNULL; } -/* Copy KNOWN_CSTS and modify the copy according to VAL and INDEX. If - non-empty, replace KNOWN_CONTEXTS with its copy too. */ +/* Copy known scalar values from AVALS into KNOWN_CSTS and modify the copy + according to VAL and INDEX. If non-empty, replace KNOWN_CONTEXTS with its + copy too. */ static void -modify_known_vectors_with_val (vec<tree> *known_csts, - vec<ipa_polymorphic_call_context> *known_contexts, - ipcp_value<tree> *val, - int index) +copy_known_vectors_add_val (ipa_call_arg_values *avals, vec<tree> *known_csts, + vec<ipa_polymorphic_call_context> *known_contexts, + ipcp_value<tree> *val, int index) { - *known_csts = known_csts->copy (); - *known_contexts = copy_useful_known_contexts (*known_contexts); + *known_csts = avals->m_known_vals.copy (); + *known_contexts = copy_useful_known_contexts (avals->m_known_contexts); (*known_csts)[index] = val->value; } -/* Replace KNOWN_CSTS with its copy. Also copy KNOWN_CONTEXTS and modify the - copy according to VAL and INDEX. */ +/* Copy known scalar values from AVALS into KNOWN_CSTS. Similarly, copy + contexts to KNOWN_CONTEXTS and modify the copy according to VAL and + INDEX. */ static void -modify_known_vectors_with_val (vec<tree> *known_csts, - vec<ipa_polymorphic_call_context> *known_contexts, - ipcp_value<ipa_polymorphic_call_context> *val, - int index) +copy_known_vectors_add_val (ipa_call_arg_values *avals, vec<tree> *known_csts, + vec<ipa_polymorphic_call_context> *known_contexts, + ipcp_value<ipa_polymorphic_call_context> *val, + int index) { - *known_csts = known_csts->copy (); - *known_contexts = known_contexts->copy (); + *known_csts = avals->m_known_vals.copy (); + *known_contexts = avals->m_known_contexts.copy (); (*known_contexts)[index] = val->value; } @@ -5432,16 +5410,15 @@ ipcp_val_agg_replacement_ok_p (ipa_agg_replacement_value *, return offset == -1; } -/* Decide whether to create a special version of NODE for value VAL of parameter - at the given INDEX. If OFFSET is -1, the value is for the parameter itself, - otherwise it is stored at the given OFFSET of the parameter. KNOWN_CSTS, - KNOWN_CONTEXTS and KNOWN_AGGS describe the other already known values. */ +/* Decide whether to create a special version of NODE for value VAL of + parameter at the given INDEX. If OFFSET is -1, the value is for the + parameter itself, otherwise it is stored at the given OFFSET of the + parameter. AVALS describes the other already known values. */ template <typename valtype> static bool decide_about_value (struct cgraph_node *node, int index, HOST_WIDE_INT offset, - ipcp_value<valtype> *val, vec<tree> known_csts, - vec<ipa_polymorphic_call_context> known_contexts) + ipcp_value<valtype> *val, ipa_call_arg_values *avals) { struct ipa_agg_replacement_value *aggvals; int freq_sum, caller_count; @@ -5491,13 +5468,16 @@ decide_about_value (struct cgraph_node *node, int index, HOST_WIDE_INT offset, fprintf (dump_file, " Creating a specialized node of %s.\n", node->dump_name ()); + vec<tree> known_csts; + vec<ipa_polymorphic_call_context> known_contexts; + callers = gather_edges_for_value (val, node, caller_count); if (offset == -1) - modify_known_vectors_with_val (&known_csts, &known_contexts, val, index); + copy_known_vectors_add_val (avals, &known_csts, &known_contexts, val, index); else { - known_csts = known_csts.copy (); - known_contexts = copy_useful_known_contexts (known_contexts); + known_csts = avals->m_known_vals.copy (); + known_contexts = copy_useful_known_contexts (avals->m_known_contexts); } find_more_scalar_values_for_callers_subset (node, known_csts, callers); find_more_contexts_for_caller_subset (node, &known_contexts, callers); @@ -5521,8 +5501,7 @@ decide_whether_version_node (struct cgraph_node *node) { class ipa_node_params *info = IPA_NODE_REF (node); int i, count = ipa_get_param_count (info); - vec<tree> known_csts; - vec<ipa_polymorphic_call_context> known_contexts; + ipa_call_arg_values avals; bool ret = false; if (count == 0) @@ -5532,8 +5511,7 @@ decide_whether_version_node (struct cgraph_node *node) fprintf (dump_file, "\nEvaluating opportunities for %s.\n", node->dump_name ()); - gather_context_independent_values (info, &known_csts, &known_contexts, - NULL, NULL); + gather_context_independent_values (info, &avals, false, NULL); for (i = 0; i < count;i++) { @@ -5542,12 +5520,11 @@ decide_whether_version_node (struct cgraph_node *node) ipcp_lattice<ipa_polymorphic_call_context> *ctxlat = &plats->ctxlat; if (!lat->bottom - && !known_csts[i]) + && !avals.m_known_vals[i]) { ipcp_value<tree> *val; for (val = lat->values; val; val = val->next) - ret |= decide_about_value (node, i, -1, val, known_csts, - known_contexts); + ret |= decide_about_value (node, i, -1, val, &avals); } if (!plats->aggs_bottom) @@ -5556,22 +5533,20 @@ decide_whether_version_node (struct cgraph_node *node) ipcp_value<tree> *val; for (aglat = plats->aggs; aglat; aglat = aglat->next) if (!aglat->bottom && aglat->values - /* If the following is false, the one value is in - known_aggs. */ + /* If the following is false, the one value has been considered + for cloning for all contexts. */ && (plats->aggs_contain_variable || !aglat->is_single_const ())) for (val = aglat->values; val; val = val->next) - ret |= decide_about_value (node, i, aglat->offset, val, - known_csts, known_contexts); + ret |= decide_about_value (node, i, aglat->offset, val, &avals); } if (!ctxlat->bottom - && known_contexts[i].useless_p ()) + && avals.m_known_contexts[i].useless_p ()) { ipcp_value<ipa_polymorphic_call_context> *val; for (val = ctxlat->values; val; val = val->next) - ret |= decide_about_value (node, i, -1, val, known_csts, - known_contexts); + ret |= decide_about_value (node, i, -1, val, &avals); } info = IPA_NODE_REF (node); @@ -5594,11 +5569,9 @@ decide_whether_version_node (struct cgraph_node *node) if (!adjust_callers_for_value_intersection (callers, node)) { /* If node is not called by anyone, or all its caller edges are - self-recursive, the node is not really be in use, no need to - do cloning. */ + self-recursive, the node is not really in use, no need to do + cloning. */ callers.release (); - known_csts.release (); - known_contexts.release (); info->do_clone_for_all_contexts = false; return ret; } @@ -5607,6 +5580,9 @@ decide_whether_version_node (struct cgraph_node *node) fprintf (dump_file, " - Creating a specialized node of %s " "for all known contexts.\n", node->dump_name ()); + vec<tree> known_csts = avals.m_known_vals.copy (); + vec<ipa_polymorphic_call_context> known_contexts + = copy_useful_known_contexts (avals.m_known_contexts); find_more_scalar_values_for_callers_subset (node, known_csts, callers); find_more_contexts_for_caller_subset (node, &known_contexts, callers); ipa_agg_replacement_value *aggvals @@ -5624,11 +5600,6 @@ decide_whether_version_node (struct cgraph_node *node) IPA_NODE_REF (clone)->is_all_contexts_clone = true; ret = true; } - else - { - known_csts.release (); - known_contexts.release (); - } return ret; } diff --git a/gcc/ipa-fnsummary.c b/gcc/ipa-fnsummary.c index 2cfab40156e..f154d5ee9fc 100644 --- a/gcc/ipa-fnsummary.c +++ b/gcc/ipa-fnsummary.c @@ -320,19 +320,15 @@ set_hint_predicate (predicate **p, predicate new_predicate) is always false in the second and also builtin_constant_p tests cannot use the fact that parameter is indeed a constant. - KNOWN_VALS is partial mapping of parameters of NODE to constant values. - KNOWN_AGGS is a vector of aggregate known offset/value set for each - parameter. Return clause of possible truths. When INLINE_P is true, assume - that we are inlining. + ARG_VALUES contains known information about argument values. Return clause + of possible truths. When INLINE_P is true, assume that we are inlining. - ERROR_MARK means compile time invariant. */ + ERROR_MARK value of an argument means compile time invariant. */ static void evaluate_conditions_for_known_args (struct cgraph_node *node, bool inline_p, - vec<tree> known_vals, - vec<value_range> known_value_ranges, - vec<ipa_agg_value_set> known_aggs, + ipa_call_arg_values *avals, clause_t *ret_clause, clause_t *ret_nonspec_clause) { @@ -351,10 +347,12 @@ evaluate_conditions_for_known_args (struct cgraph_node *node, /* We allow call stmt to have fewer arguments than the callee function (especially for K&R style programs). So bound check here (we assume - known_aggs vector, if non-NULL, has the same length as - known_vals). */ - gcc_checking_assert (!known_aggs.length () || !known_vals.length () - || (known_vals.length () == known_aggs.length ())); + m_known_aggs vector is either empty or has the same length as + m_known_vals). */ + gcc_checking_assert (!avals->m_known_aggs.length () + || !avals->m_known_vals.length () + || (avals->m_known_vals.length () + == avals->m_known_aggs.length ())); if (c->agg_contents) { @@ -362,26 +360,23 @@ evaluate_conditions_for_known_args (struct cgraph_node *node, if (c->code == predicate::changed && !c->by_ref - && c->operand_num < (int)known_vals.length () - && (known_vals[c->operand_num] == error_mark_node)) + && c->operand_num < (int) avals->m_known_vals.length () + && (avals->m_known_vals[c->operand_num] == error_mark_node)) continue; - if (c->operand_num < (int)known_aggs.length ()) + if (c->operand_num < (int) avals->m_known_aggs.length ()) { - agg = &known_aggs[c->operand_num]; - val = ipa_find_agg_cst_for_param (agg, - c->operand_num - < (int) known_vals.length () - ? known_vals[c->operand_num] - : NULL, - c->offset, c->by_ref); + agg = &avals->m_known_aggs[c->operand_num]; + tree sval = avals->safe_sval_at (c->operand_num); + val = ipa_find_agg_cst_for_param (agg, sval, c->offset, + c->by_ref); } else val = NULL_TREE; } - else if (c->operand_num < (int) known_vals.length ()) + else if (c->operand_num < (int) avals->m_known_vals.length ()) { - val = known_vals[c->operand_num]; + val = avals->m_known_vals[c->operand_num]; if (val == error_mark_node && c->code != predicate::changed) val = NULL_TREE; } @@ -446,53 +441,54 @@ evaluate_conditions_for_known_args (struct cgraph_node *node, continue; } } - if (c->operand_num < (int) known_value_ranges.length () + if (c->operand_num < (int) avals->m_known_value_ranges.length () && !c->agg_contents - && !known_value_ranges[c->operand_num].undefined_p () - && !known_value_ranges[c->operand_num].varying_p () - && TYPE_SIZE (c->type) - == TYPE_SIZE (known_value_ranges[c->operand_num].type ()) && (!val || TREE_CODE (val) != INTEGER_CST)) { - value_range vr = known_value_ranges[c->operand_num]; - if (!useless_type_conversion_p (c->type, vr.type ())) + value_range vr = avals->m_known_value_ranges[c->operand_num]; + if (!vr.undefined_p () + && !vr.varying_p () + && (TYPE_SIZE (c->type) == TYPE_SIZE (vr.type ()))) { - value_range res; - range_fold_unary_expr (&res, NOP_EXPR, - c->type, &vr, vr.type ()); - vr = res; - } - tree type = c->type; - - for (j = 0; vec_safe_iterate (c->param_ops, j, &op); j++) - { - if (vr.varying_p () || vr.undefined_p ()) - break; - - value_range res; - if (!op->val[0]) - range_fold_unary_expr (&res, op->code, op->type, &vr, type); - else if (!op->val[1]) + if (!useless_type_conversion_p (c->type, vr.type ())) { - value_range op0 (op->val[0], op->val[0]); - range_fold_binary_expr (&res, op->code, op->type, - op->index ? &op0 : &vr, - op->index ? &vr : &op0); + value_range res; + range_fold_unary_expr (&res, NOP_EXPR, + c->type, &vr, vr.type ()); + vr = res; + } + tree type = c->type; + + for (j = 0; vec_safe_iterate (c->param_ops, j, &op); j++) + { + if (vr.varying_p () || vr.undefined_p ()) + break; + + value_range res; + if (!op->val[0]) + range_fold_unary_expr (&res, op->code, op->type, &vr, type); + else if (!op->val[1]) + { + value_range op0 (op->val[0], op->val[0]); + range_fold_binary_expr (&res, op->code, op->type, + op->index ? &op0 : &vr, + op->index ? &vr : &op0); + } + else + gcc_unreachable (); + type = op->type; + vr = res; + } + if (!vr.varying_p () && !vr.undefined_p ()) + { + value_range res; + value_range val_vr (c->val, c->val); + range_fold_binary_expr (&res, c->code, boolean_type_node, + &vr, + &val_vr); + if (res.zero_p ()) + continue; } - else - gcc_unreachable (); - type = op->type; - vr = res; - } - if (!vr.varying_p () && !vr.undefined_p ()) - { - value_range res; - value_range val_vr (c->val, c->val); - range_fold_binary_expr (&res, c->code, boolean_type_node, - &vr, - &val_vr); - if (res.zero_p ()) - continue; } } @@ -538,24 +534,20 @@ fre_will_run_p (struct cgraph_node *node) (if non-NULL) conditions evaluated for nonspecialized clone called in a given context. - KNOWN_VALS_PTR and KNOWN_AGGS_PTR must be non-NULL and will be filled by - known constant and aggregate values of parameters. - - KNOWN_CONTEXT_PTR, if non-NULL, will be filled by polymorphic call contexts - of parameter used by a polymorphic call. */ + Vectors in AVALS will be populated with useful known information about + argument values - information not known to have any uses will be omitted - + except for m_known_contexts which will only be calculated if + COMPUTE_CONTEXTS is true. */ void evaluate_properties_for_edge (struct cgraph_edge *e, bool inline_p, clause_t *clause_ptr, clause_t *nonspec_clause_ptr, - vec<tree> *known_vals_ptr, - vec<ipa_polymorphic_call_context> - *known_contexts_ptr, - vec<ipa_agg_value_set> *known_aggs_ptr) + ipa_call_arg_values *avals, + bool compute_contexts) { struct cgraph_node *callee = e->callee->ultimate_alias_target (); class ipa_fn_summary *info = ipa_fn_summaries->get (callee); - auto_vec<value_range, 32> known_value_ranges; class ipa_edge_args *args; if (clause_ptr) @@ -563,7 +555,7 @@ evaluate_properties_for_edge (struct cgraph_edge *e, bool inline_p, if (ipa_node_params_sum && !e->call_stmt_cannot_inline_p - && (info->conds || known_contexts_ptr) + && (info->conds || compute_contexts) && (args = IPA_EDGE_REF (e)) != NULL) { struct cgraph_node *caller; @@ -608,15 +600,15 @@ evaluate_properties_for_edge (struct cgraph_edge *e, bool inline_p, if (cst) { gcc_checking_assert (TREE_CODE (cst) != TREE_BINFO); - if (!known_vals_ptr->length ()) - vec_safe_grow_cleared (known_vals_ptr, count); - (*known_vals_ptr)[i] = cst; + if (!avals->m_known_vals.length ()) + avals->m_known_vals.safe_grow_cleared (count); + avals->m_known_vals[i] = cst; } else if (inline_p && !es->param[i].change_prob) { - if (!known_vals_ptr->length ()) - vec_safe_grow_cleared (known_vals_ptr, count); - (*known_vals_ptr)[i] = error_mark_node; + if (!avals->m_known_vals.length ()) + avals->m_known_vals.safe_grow_cleared (count); + avals->m_known_vals[i] = error_mark_node; } /* If we failed to get simple constant, try value range. */ @@ -624,19 +616,20 @@ evaluate_properties_for_edge (struct cgraph_edge *e, bool inline_p, && vrp_will_run_p (caller) && ipa_is_param_used_by_ipa_predicates (callee_pi, i)) { - value_range vr + value_range vr = ipa_value_range_from_jfunc (caller_parms_info, e, jf, ipa_get_type (callee_pi, i)); if (!vr.undefined_p () && !vr.varying_p ()) { - if (!known_value_ranges.length ()) + if (!avals->m_known_value_ranges.length ()) { - known_value_ranges.safe_grow (count); + avals->m_known_value_ranges.safe_grow (count); for (int i = 0; i < count; ++i) - new (&known_value_ranges[i]) value_range (); + new (&avals->m_known_value_ranges[i]) + value_range (); } - known_value_ranges[i] = vr; + avals->m_known_value_ranges[i] = vr; } } @@ -648,25 +641,25 @@ evaluate_properties_for_edge (struct cgraph_edge *e, bool inline_p, caller, &jf->agg); if (agg.items.length ()) { - if (!known_aggs_ptr->length ()) - vec_safe_grow_cleared (known_aggs_ptr, count); - (*known_aggs_ptr)[i] = agg; + if (!avals->m_known_aggs.length ()) + avals->m_known_aggs.safe_grow_cleared (count); + avals->m_known_aggs[i] = agg; } } } /* For calls used in polymorphic calls we further determine polymorphic call context. */ - if (known_contexts_ptr + if (compute_contexts && ipa_is_param_used_by_polymorphic_call (callee_pi, i)) { ipa_polymorphic_call_context ctx = ipa_context_from_jfunc (caller_parms_info, e, i, jf); if (!ctx.useless_p ()) { - if (!known_contexts_ptr->length ()) - known_contexts_ptr->safe_grow_cleared (count); - (*known_contexts_ptr)[i] + if (!avals->m_known_contexts.length ()) + avals->m_known_contexts.safe_grow_cleared (count); + avals->m_known_contexts[i] = ipa_context_from_jfunc (caller_parms_info, e, i, jf); } } @@ -685,18 +678,14 @@ evaluate_properties_for_edge (struct cgraph_edge *e, bool inline_p, cst = NULL; if (cst) { - if (!known_vals_ptr->length ()) - vec_safe_grow_cleared (known_vals_ptr, count); - (*known_vals_ptr)[i] = cst; + if (!avals->m_known_vals.length ()) + avals->m_known_vals.safe_grow_cleared (count); + avals->m_known_vals[i] = cst; } } } - evaluate_conditions_for_known_args (callee, inline_p, - *known_vals_ptr, - known_value_ranges, - *known_aggs_ptr, - clause_ptr, + evaluate_conditions_for_known_args (callee, inline_p, avals, clause_ptr, nonspec_clause_ptr); } @@ -781,7 +770,7 @@ ipa_fn_summary_t::duplicate (cgraph_node *src, vec<size_time_entry, va_gc> *entry = info->size_time_table; /* Use SRC parm info since it may not be copied yet. */ class ipa_node_params *parms_info = IPA_NODE_REF (src); - vec<tree> known_vals = vNULL; + ipa_call_arg_values avals; int count = ipa_get_param_count (parms_info); int i, j; clause_t possible_truths; @@ -792,7 +781,7 @@ ipa_fn_summary_t::duplicate (cgraph_node *src, struct cgraph_edge *edge, *next; info->size_time_table = 0; - known_vals.safe_grow_cleared (count); + avals.m_known_vals.safe_grow_cleared (count); for (i = 0; i < count; i++) { struct ipa_replace_map *r; @@ -801,20 +790,17 @@ ipa_fn_summary_t::duplicate (cgraph_node *src, { if (r->parm_num == i) { - known_vals[i] = r->new_tree; + avals.m_known_vals[i] = r->new_tree; break; } } } evaluate_conditions_for_known_args (dst, false, - known_vals, - vNULL, - vNULL, + &avals, &possible_truths, /* We are going to specialize, so ignore nonspec truths. */ NULL); - known_vals.release (); info->account_size_time (0, 0, true_pred, true_pred); @@ -3009,15 +2995,14 @@ compute_fn_summary_for_current (void) return 0; } -/* Estimate benefit devirtualizing indirect edge IE, provided KNOWN_VALS, - KNOWN_CONTEXTS and KNOWN_AGGS. */ +/* Estimate benefit devirtualizing indirect edge IE and return true if it can + be devirtualized and inlined, provided m_known_vals, m_known_contexts and + m_known_aggs in AVALS. Return false straight away if AVALS is NULL. */ static bool estimate_edge_devirt_benefit (struct cgraph_edge *ie, int *size, int *time, - vec<tree> known_vals, - vec<ipa_polymorphic_call_context> known_contexts, - vec<ipa_agg_value_set> known_aggs) + ipa_call_arg_values *avals) { tree target; struct cgraph_node *callee; @@ -3025,13 +3010,13 @@ estimate_edge_devirt_benefit (struct cgraph_edge *ie, enum availability avail; bool speculative; - if (!known_vals.length () && !known_contexts.length ()) + if (!avals + || (!avals->m_known_vals.length() && !avals->m_known_contexts.length ())) return false; if (!opt_for_fn (ie->caller->decl, flag_indirect_inlining)) return false; - target = ipa_get_indirect_edge_target (ie, known_vals, known_contexts, - known_aggs, &speculative); + target = ipa_get_indirect_edge_target (ie, avals, &speculative); if (!target || speculative) return false; @@ -3055,17 +3040,13 @@ estimate_edge_devirt_benefit (struct cgraph_edge *ie, } /* Increase SIZE, MIN_SIZE (if non-NULL) and TIME for size and time needed to - handle edge E with probability PROB. - Set HINTS if edge may be devirtualized. - KNOWN_VALS, KNOWN_AGGS and KNOWN_CONTEXTS describe context of the call - site. */ + handle edge E with probability PROB. Set HINTS accordingly if edge may be + devirtualized. AVALS, if non-NULL, describe the context of the call site as + far as values of parameters are concerened. */ static inline void estimate_edge_size_and_time (struct cgraph_edge *e, int *size, int *min_size, - sreal *time, - vec<tree> known_vals, - vec<ipa_polymorphic_call_context> known_contexts, - vec<ipa_agg_value_set> known_aggs, + sreal *time, ipa_call_arg_values *avals, ipa_hints *hints) { class ipa_call_summary *es = ipa_call_summaries->get (e); @@ -3074,8 +3055,7 @@ estimate_edge_size_and_time (struct cgraph_edge *e, int *size, int *min_size, int cur_size; if (!e->callee && hints && e->maybe_hot_p () - && estimate_edge_devirt_benefit (e, &call_size, &call_time, - known_vals, known_contexts, known_aggs)) + && estimate_edge_devirt_benefit (e, &call_size, &call_time, avals)) *hints |= INLINE_HINT_indirect_call; cur_size = call_size * ipa_fn_summary::size_scale; *size += cur_size; @@ -3087,9 +3067,9 @@ estimate_edge_size_and_time (struct cgraph_edge *e, int *size, int *min_size, /* Increase SIZE, MIN_SIZE and TIME for size and time needed to handle all - calls in NODE. POSSIBLE_TRUTHS, KNOWN_VALS, KNOWN_AGGS and KNOWN_CONTEXTS - describe context of the call site. - + calls in NODE. POSSIBLE_TRUTHS and AVALS describe the context of the call + site. + Helper for estimate_calls_size_and_time which does the same but (in most cases) faster. */ @@ -3098,9 +3078,7 @@ estimate_calls_size_and_time_1 (struct cgraph_node *node, int *size, int *min_size, sreal *time, ipa_hints *hints, clause_t possible_truths, - vec<tree> known_vals, - vec<ipa_polymorphic_call_context> known_contexts, - vec<ipa_agg_value_set> known_aggs) + ipa_call_arg_values *avals) { struct cgraph_edge *e; for (e = node->callees; e; e = e->next_callee) @@ -3109,10 +3087,8 @@ estimate_calls_size_and_time_1 (struct cgraph_node *node, int *size, { gcc_checking_assert (!ipa_call_summaries->get (e)); estimate_calls_size_and_time_1 (e->callee, size, min_size, time, - hints, - possible_truths, - known_vals, known_contexts, - known_aggs); + hints, possible_truths, avals); + continue; } class ipa_call_summary *es = ipa_call_summaries->get (e); @@ -3130,9 +3106,7 @@ estimate_calls_size_and_time_1 (struct cgraph_node *node, int *size, so we do not need to compute probabilities. */ estimate_edge_size_and_time (e, size, es->predicate ? NULL : min_size, - time, - known_vals, known_contexts, - known_aggs, hints); + time, avals, hints); } } for (e = node->indirect_calls; e; e = e->next_callee) @@ -3142,9 +3116,7 @@ estimate_calls_size_and_time_1 (struct cgraph_node *node, int *size, || es->predicate->evaluate (possible_truths)) estimate_edge_size_and_time (e, size, es->predicate ? NULL : min_size, - time, - known_vals, known_contexts, known_aggs, - hints); + time, avals, hints); } } @@ -3166,8 +3138,7 @@ summarize_calls_size_and_time (struct cgraph_node *node, int size = 0; sreal time = 0; - estimate_edge_size_and_time (e, &size, NULL, &time, - vNULL, vNULL, vNULL, NULL); + estimate_edge_size_and_time (e, &size, NULL, &time, NULL, NULL); struct predicate pred = true; class ipa_call_summary *es = ipa_call_summaries->get (e); @@ -3181,8 +3152,7 @@ summarize_calls_size_and_time (struct cgraph_node *node, int size = 0; sreal time = 0; - estimate_edge_size_and_time (e, &size, NULL, &time, - vNULL, vNULL, vNULL, NULL); + estimate_edge_size_and_time (e, &size, NULL, &time, NULL, NULL); struct predicate pred = true; class ipa_call_summary *es = ipa_call_summaries->get (e); @@ -3193,17 +3163,15 @@ summarize_calls_size_and_time (struct cgraph_node *node, } /* Increase SIZE, MIN_SIZE and TIME for size and time needed to handle all - calls in NODE. POSSIBLE_TRUTHS, KNOWN_VALS, KNOWN_AGGS and KNOWN_CONTEXTS - describe context of the call site. */ + calls in NODE. POSSIBLE_TRUTHS and AVALS (the latter if non-NULL) describe + context of the call site. */ static void estimate_calls_size_and_time (struct cgraph_node *node, int *size, int *min_size, sreal *time, ipa_hints *hints, clause_t possible_truths, - vec<tree> known_vals, - vec<ipa_polymorphic_call_context> known_contexts, - vec<ipa_agg_value_set> known_aggs) + ipa_call_arg_values *avals) { class ipa_fn_summary *sum = ipa_fn_summaries->get (node); bool use_table = true; @@ -3222,9 +3190,10 @@ estimate_calls_size_and_time (struct cgraph_node *node, int *size, use_table = false; /* If there is an indirect edge that may be optimized, we need to go the slow way. */ - else if ((known_vals.length () - || known_contexts.length () - || known_aggs.length ()) && hints) + else if (avals && hints + && (avals->m_known_vals.length () + || avals->m_known_contexts.length () + || avals->m_known_aggs.length ())) { class ipa_node_params *params_summary = IPA_NODE_REF (node); unsigned int nargs = params_summary @@ -3233,13 +3202,13 @@ estimate_calls_size_and_time (struct cgraph_node *node, int *size, for (unsigned int i = 0; i < nargs && use_table; i++) { if (ipa_is_param_used_by_indirect_call (params_summary, i) - && ((known_vals.length () > i && known_vals[i]) - || (known_aggs.length () > i - && known_aggs[i].items.length ()))) + && ((avals->m_known_vals.length () > i && avals->m_known_vals[i]) + || (avals->m_known_aggs.length () > i + && avals->m_known_aggs[i].items.length ()))) use_table = false; else if (ipa_is_param_used_by_polymorphic_call (params_summary, i) - && (known_contexts.length () > i - && !known_contexts[i].useless_p ())) + && (avals->m_known_contexts.length () > i + && !avals->m_known_contexts[i].useless_p ())) use_table = false; } } @@ -3282,8 +3251,7 @@ estimate_calls_size_and_time (struct cgraph_node *node, int *size, < ipa_fn_summary::max_size_time_table_size) { estimate_calls_size_and_time_1 (node, &old_size, NULL, &old_time, NULL, - possible_truths, known_vals, - known_contexts, known_aggs); + possible_truths, avals); gcc_assert (*size == old_size); if (time && (*time - old_time > 1 || *time - old_time < -1) && dump_file) @@ -3295,242 +3263,24 @@ estimate_calls_size_and_time (struct cgraph_node *node, int *size, /* Slow path by walking all edges. */ else estimate_calls_size_and_time_1 (node, size, min_size, time, hints, - possible_truths, known_vals, known_contexts, - known_aggs); + possible_truths, avals); } -/* Default constructor for ipa call context. - Memory allocation of known_vals, known_contexts - and known_aggs vectors is owned by the caller, but can - be release by ipa_call_context::release. - - inline_param_summary is owned by the caller. */ -ipa_call_context::ipa_call_context (cgraph_node *node, - clause_t possible_truths, - clause_t nonspec_possible_truths, - vec<tree> known_vals, - vec<ipa_polymorphic_call_context> - known_contexts, - vec<ipa_agg_value_set> known_aggs, - vec<inline_param_summary> - inline_param_summary) +/* Default constructor for ipa call context. Memory allocation of ARG_VALUES + is owned by the caller. INLINE_PARAM_SUMMARY is also owned by the + caller. */ +ipa_call_context +::ipa_call_context (cgraph_node *node, clause_t possible_truths, + clause_t nonspec_possible_truths, + vec<inline_param_summary> inline_param_summary, + ipa_call_arg_values *arg_values) : m_node (node), m_possible_truths (possible_truths), m_nonspec_possible_truths (nonspec_possible_truths), m_inline_param_summary (inline_param_summary), - m_known_vals (known_vals), - m_known_contexts (known_contexts), - m_known_aggs (known_aggs) + m_avals (arg_values) { } -/* Set THIS to be a duplicate of CTX. Copy all relevant info. */ - -void -ipa_call_context::duplicate_from (const ipa_call_context &ctx) -{ - m_node = ctx.m_node; - m_possible_truths = ctx.m_possible_truths; - m_nonspec_possible_truths = ctx.m_nonspec_possible_truths; - class ipa_node_params *params_summary = IPA_NODE_REF (m_node); - unsigned int nargs = params_summary - ? ipa_get_param_count (params_summary) : 0; - - m_inline_param_summary = vNULL; - /* Copy the info only if there is at least one useful entry. */ - if (ctx.m_inline_param_summary.exists ()) - { - unsigned int n = MIN (ctx.m_inline_param_summary.length (), nargs); - - for (unsigned int i = 0; i < n; i++) - if (ipa_is_param_used_by_ipa_predicates (params_summary, i) - && !ctx.m_inline_param_summary[i].useless_p ()) - { - m_inline_param_summary - = ctx.m_inline_param_summary.copy (); - break; - } - } - m_known_vals = vNULL; - if (ctx.m_known_vals.exists ()) - { - unsigned int n = MIN (ctx.m_known_vals.length (), nargs); - - for (unsigned int i = 0; i < n; i++) - if (ipa_is_param_used_by_indirect_call (params_summary, i) - && ctx.m_known_vals[i]) - { - m_known_vals = ctx.m_known_vals.copy (); - break; - } - } - - m_known_contexts = vNULL; - if (ctx.m_known_contexts.exists ()) - { - unsigned int n = MIN (ctx.m_known_contexts.length (), nargs); - - for (unsigned int i = 0; i < n; i++) - if (ipa_is_param_used_by_polymorphic_call (params_summary, i) - && !ctx.m_known_contexts[i].useless_p ()) - { - m_known_contexts = ctx.m_known_contexts.copy (); - break; - } - } - - m_known_aggs = vNULL; - if (ctx.m_known_aggs.exists ()) - { - unsigned int n = MIN (ctx.m_known_aggs.length (), nargs); - - for (unsigned int i = 0; i < n; i++) - if (ipa_is_param_used_by_indirect_call (params_summary, i) - && !ctx.m_known_aggs[i].is_empty ()) - { - m_known_aggs = ipa_copy_agg_values (ctx.m_known_aggs); - break; - } - } -} - -/* Release memory used by known_vals/contexts/aggs vectors. - If ALL is true release also inline_param_summary. - This happens when context was previously duplicated to be stored - into cache. */ - -void -ipa_call_context::release (bool all) -{ - /* See if context is initialized at first place. */ - if (!m_node) - return; - ipa_release_agg_values (m_known_aggs, all); - if (all) - { - m_known_vals.release (); - m_known_contexts.release (); - m_inline_param_summary.release (); - } -} - -/* Return true if CTX describes the same call context as THIS. */ - -bool -ipa_call_context::equal_to (const ipa_call_context &ctx) -{ - if (m_node != ctx.m_node - || m_possible_truths != ctx.m_possible_truths - || m_nonspec_possible_truths != ctx.m_nonspec_possible_truths) - return false; - - class ipa_node_params *params_summary = IPA_NODE_REF (m_node); - unsigned int nargs = params_summary - ? ipa_get_param_count (params_summary) : 0; - - if (m_inline_param_summary.exists () || ctx.m_inline_param_summary.exists ()) - { - for (unsigned int i = 0; i < nargs; i++) - { - if (!ipa_is_param_used_by_ipa_predicates (params_summary, i)) - continue; - if (i >= m_inline_param_summary.length () - || m_inline_param_summary[i].useless_p ()) - { - if (i < ctx.m_inline_param_summary.length () - && !ctx.m_inline_param_summary[i].useless_p ()) - return false; - continue; - } - if (i >= ctx.m_inline_param_summary.length () - || ctx.m_inline_param_summary[i].useless_p ()) - { - if (i < m_inline_param_summary.length () - && !m_inline_param_summary[i].useless_p ()) - return false; - continue; - } - if (!m_inline_param_summary[i].equal_to - (ctx.m_inline_param_summary[i])) - return false; - } - } - if (m_known_vals.exists () || ctx.m_known_vals.exists ()) - { - for (unsigned int i = 0; i < nargs; i++) - { - if (!ipa_is_param_used_by_indirect_call (params_summary, i)) - continue; - if (i >= m_known_vals.length () || !m_known_vals[i]) - { - if (i < ctx.m_known_vals.length () && ctx.m_known_vals[i]) - return false; - continue; - } - if (i >= ctx.m_known_vals.length () || !ctx.m_known_vals[i]) - { - if (i < m_known_vals.length () && m_known_vals[i]) - return false; - continue; - } - if (m_known_vals[i] != ctx.m_known_vals[i]) - return false; - } - } - if (m_known_contexts.exists () || ctx.m_known_contexts.exists ()) - { - for (unsigned int i = 0; i < nargs; i++) - { - if (!ipa_is_param_used_by_polymorphic_call (params_summary, i)) - continue; - if (i >= m_known_contexts.length () - || m_known_contexts[i].useless_p ()) - { - if (i < ctx.m_known_contexts.length () - && !ctx.m_known_contexts[i].useless_p ()) - return false; - continue; - } - if (i >= ctx.m_known_contexts.length () - || ctx.m_known_contexts[i].useless_p ()) - { - if (i < m_known_contexts.length () - && !m_known_contexts[i].useless_p ()) - return false; - continue; - } - if (!m_known_contexts[i].equal_to - (ctx.m_known_contexts[i])) - return false; - } - } - if (m_known_aggs.exists () || ctx.m_known_aggs.exists ()) - { - for (unsigned int i = 0; i < nargs; i++) - { - if (!ipa_is_param_used_by_indirect_call (params_summary, i)) - continue; - if (i >= m_known_aggs.length () || m_known_aggs[i].is_empty ()) - { - if (i < ctx.m_known_aggs.length () - && !ctx.m_known_aggs[i].is_empty ()) - return false; - continue; - } - if (i >= ctx.m_known_aggs.length () - || ctx.m_known_aggs[i].is_empty ()) - { - if (i < m_known_aggs.length () - && !m_known_aggs[i].is_empty ()) - return false; - continue; - } - if (!m_known_aggs[i].equal_to (ctx.m_known_aggs[i])) - return false; - } - } - return true; -} - /* Estimate size and time needed to execute call in the given context. Additionally determine hints determined by the context. Finally compute minimal size needed for the call that is independent on the call context and @@ -3574,7 +3324,7 @@ ipa_call_context::estimate_size_and_time (int *ret_size, estimate_calls_size_and_time (m_node, &size, &min_size, ret_time ? &time : NULL, ret_hints ? &hints : NULL, m_possible_truths, - m_known_vals, m_known_contexts, m_known_aggs); + m_avals); sreal nonspecialized_time = time; @@ -3681,22 +3431,17 @@ ipa_call_context::estimate_size_and_time (int *ret_size, void estimate_ipcp_clone_size_and_time (struct cgraph_node *node, - vec<tree> known_vals, - vec<ipa_polymorphic_call_context> - known_contexts, - vec<ipa_agg_value_set> known_aggs, + ipa_call_arg_values *avals, int *ret_size, sreal *ret_time, sreal *ret_nonspec_time, ipa_hints *hints) { clause_t clause, nonspec_clause; - /* TODO: Also pass known value ranges. */ - evaluate_conditions_for_known_args (node, false, known_vals, vNULL, - known_aggs, &clause, &nonspec_clause); + evaluate_conditions_for_known_args (node, false, avals, &clause, + &nonspec_clause); ipa_call_context ctx (node, clause, nonspec_clause, - known_vals, known_contexts, - known_aggs, vNULL); + vNULL, avals); ctx.estimate_size_and_time (ret_size, NULL, ret_time, ret_nonspec_time, hints); } @@ -3914,10 +3659,9 @@ ipa_merge_fn_summary_after_inlining (struct cgraph_edge *edge) if (callee_info->conds) { - auto_vec<tree, 32> known_vals; - auto_vec<ipa_agg_value_set, 32> known_aggs; + ipa_call_arg_values avals; evaluate_properties_for_edge (edge, true, &clause, NULL, - &known_vals, NULL, &known_aggs); + &avals, false); } if (ipa_node_params_sum && callee_info->conds) { @@ -4011,8 +3755,7 @@ ipa_merge_fn_summary_after_inlining (struct cgraph_edge *edge) int edge_size = 0; sreal edge_time = 0; - estimate_edge_size_and_time (edge, &edge_size, NULL, &edge_time, vNULL, - vNULL, vNULL, 0); + estimate_edge_size_and_time (edge, &edge_size, NULL, &edge_time, NULL, 0); /* Unaccount size and time of the optimized out call. */ info->account_size_time (-edge_size, -edge_time, es->predicate ? *es->predicate : true, @@ -4054,7 +3797,7 @@ ipa_update_overall_fn_summary (struct cgraph_node *node, bool reset) estimate_calls_size_and_time (node, &size_info->size, &info->min_size, &info->time, NULL, ~(clause_t) (1 << predicate::false_condition), - vNULL, vNULL, vNULL); + NULL); size_info->size = RDIV (size_info->size, ipa_fn_summary::size_scale); info->min_size = RDIV (info->min_size, ipa_fn_summary::size_scale); } diff --git a/gcc/ipa-fnsummary.h b/gcc/ipa-fnsummary.h index c6ddc9f3199..34050516d91 100644 --- a/gcc/ipa-fnsummary.h +++ b/gcc/ipa-fnsummary.h @@ -287,6 +287,8 @@ public: ipa_call_summary *dst_data); }; +class ipa_node_context_cache_entry; + /* This object describe a context of call. That is a summary of known information about its parameters. Main purpose of this context is to give more realistic estimations of function runtime, size and @@ -297,10 +299,8 @@ public: ipa_call_context (cgraph_node *node, clause_t possible_truths, clause_t nonspec_possible_truths, - vec<tree> known_vals, - vec<ipa_polymorphic_call_context> known_contexts, - vec<ipa_agg_value_set> known_aggs, - vec<inline_param_summary> m_inline_param_summary); + vec<inline_param_summary> inline_param_summary, + ipa_call_arg_values *arg_values); ipa_call_context () : m_node(NULL) { @@ -309,9 +309,8 @@ public: sreal *ret_time, sreal *ret_nonspecialized_time, ipa_hints *ret_hints); - void duplicate_from (const ipa_call_context &ctx); - void release (bool all = false); - bool equal_to (const ipa_call_context &); + void store_to_cache (ipa_node_context_cache_entry *cache) const; + bool equivalent_to_p (const ipa_node_context_cache_entry &cache) const; bool exists_p () { return m_node != NULL; @@ -328,14 +327,9 @@ private: /* Inline summary maintains info about change probabilities. */ vec<inline_param_summary> m_inline_param_summary; - /* The following is used only to resolve indirect calls. */ - - /* Vector describing known values of parameters. */ - vec<tree> m_known_vals; - /* Vector describing known polymorphic call contexts. */ - vec<ipa_polymorphic_call_context> m_known_contexts; - /* Vector describing known aggregate values. */ - vec<ipa_agg_value_set> m_known_aggs; + /* Even after having calculated clauses, the information about argument + values is used to resolve indirect calls. */ + ipa_call_arg_values *m_avals; }; extern fast_call_summary <ipa_call_summary *, va_heap> *ipa_call_summaries; @@ -349,9 +343,7 @@ void ipa_free_fn_summary (void); void ipa_free_size_summary (void); void inline_analyze_function (struct cgraph_node *node); void estimate_ipcp_clone_size_and_time (struct cgraph_node *, - vec<tree>, - vec<ipa_polymorphic_call_context>, - vec<ipa_agg_value_set>, + ipa_call_arg_values *, int *, sreal *, sreal *, ipa_hints *); void ipa_merge_fn_summary_after_inlining (struct cgraph_edge *edge); @@ -363,10 +355,8 @@ void evaluate_properties_for_edge (struct cgraph_edge *e, bool inline_p, clause_t *clause_ptr, clause_t *nonspec_clause_ptr, - vec<tree> *known_vals_ptr, - vec<ipa_polymorphic_call_context> - *known_contexts_ptr, - vec<ipa_agg_value_set> *); + ipa_call_arg_values *avals, + bool compute_contexts); void ipa_fnsummary_c_finalize (void); HOST_WIDE_INT ipa_get_stack_frame_offset (struct cgraph_node *node); diff --git a/gcc/ipa-inline-analysis.c b/gcc/ipa-inline-analysis.c index 148efbc09ef..d8cb9294c36 100644 --- a/gcc/ipa-inline-analysis.c +++ b/gcc/ipa-inline-analysis.c @@ -52,31 +52,253 @@ along with GCC; see the file COPYING3. If not see /* Cached node/edge growths. */ fast_call_summary<edge_growth_cache_entry *, va_heap> *edge_growth_cache = NULL; -/* The context cache remembers estimated time/size and hints for given - ipa_call_context of a call. */ -class node_context_cache_entry +/* This class caches estimated times, size and hints for an ipa_call_context + together with all information necessary to figure out whether the original + context is th same as another one for the purposes of these estimations. */ + +class ipa_node_context_cache_entry { public: - ipa_call_context ctx; - sreal time, nonspec_time; - int size; - ipa_hints hints; + friend class ipa_call_context; - node_context_cache_entry () - : ctx () + ~ipa_node_context_cache_entry () { + release (); } - ~node_context_cache_entry () - { - ctx.release (); - } + /* Free all memory held by member vectors of this class. */ + void release (); + + /* Estimated time and nonspecialized time. */ + sreal m_time = 0; + sreal m_nonspec_time = 0; + /* Estimated size. */ + int m_size = 0; + /* Estimated hints. */ + ipa_hints m_hints = 0; + +private: + /* m_possible_truths copied from the initial ipa_call_context. */ + clause_t m_possible_truths = 0; + /* m_nonspec_possible_truths copied from the initial ipa_call_context. */ + clause_t m_nonspec_possible_truths = 0; + /* m_inline_param_summary copied from the initial ipa_call_context. */ + vec<inline_param_summary> m_inline_param_summary = vNULL; + /* Copy of m_known_vals of m_avals of the initial ipa_call_context. */ + vec<tree> m_known_vals = vNULL; + /* Copy of m_known_contexts of m_avals of the initial ipa_call_context. */ + vec<ipa_polymorphic_call_context> m_known_contexts = vNULL; + /* Copy of m_known_vals of m_avals of the initial ipa_call_context. */ + vec<ipa_agg_value_set> m_known_aggs = vNULL; + /* m_known_value_ranges from m_avals of the initial ipa_call_context is not + cached (yet), it turns out not to be necessary any more as it is not used + for discovering targets of indirect call graph edges. */ + }; +void +ipa_node_context_cache_entry::release () +{ + ipa_release_agg_values (m_known_aggs, true); + m_known_vals.release (); + m_known_contexts.release (); + m_inline_param_summary.release (); +} + +/* Copy all information necessary to subsequently identify equivalent contexts + to CACHE. */ + +void +ipa_call_context::store_to_cache (ipa_node_context_cache_entry *cache) const +{ + cache->m_possible_truths = m_possible_truths; + cache->m_nonspec_possible_truths = m_nonspec_possible_truths; + class ipa_node_params *params_summary = IPA_NODE_REF (m_node); + unsigned int nargs = params_summary + ? ipa_get_param_count (params_summary) : 0; + + cache->m_inline_param_summary = vNULL; + /* Copy the info only if there is at least one useful entry. */ + if (m_inline_param_summary.exists ()) + { + unsigned int n = MIN (m_inline_param_summary.length (), nargs); + + for (unsigned int i = 0; i < n; i++) + if (ipa_is_param_used_by_ipa_predicates (params_summary, i) + && !m_inline_param_summary[i].useless_p ()) + { + cache->m_inline_param_summary = m_inline_param_summary.copy (); + break; + } + } + cache->m_known_vals = vNULL; + if (m_avals->m_known_vals.exists ()) + { + unsigned int n = MIN (m_avals->m_known_vals.length (), nargs); + + for (unsigned int i = 0; i < n; i++) + if (ipa_is_param_used_by_indirect_call (params_summary, i) + && m_avals->m_known_vals[i]) + { + cache->m_known_vals = m_avals->m_known_vals.copy (); + break; + } + } + + cache->m_known_contexts = vNULL; + if (m_avals->m_known_contexts.exists ()) + { + unsigned int n = MIN (m_avals->m_known_contexts.length (), nargs); + + for (unsigned int i = 0; i < n; i++) + if (ipa_is_param_used_by_polymorphic_call (params_summary, i) + && !m_avals->m_known_contexts[i].useless_p ()) + { + cache->m_known_contexts = m_avals->m_known_contexts.copy (); + break; + } + } + + cache->m_known_aggs = vNULL; + if (m_avals->m_known_aggs.exists ()) + { + unsigned int n = MIN (m_avals->m_known_aggs.length (), nargs); + + for (unsigned int i = 0; i < n; i++) + if (ipa_is_param_used_by_indirect_call (params_summary, i) + && !m_avals->m_known_aggs[i].is_empty ()) + { + cache->m_known_aggs = ipa_copy_agg_values (m_avals->m_known_aggs); + break; + } + } +} + +/* Return true if CACHE describes the same call context as THIS for the + purposes of context caching. */ + +bool +ipa_call_context::equivalent_to_p (const ipa_node_context_cache_entry &cache) + const +{ + if (m_possible_truths != cache.m_possible_truths + || m_nonspec_possible_truths != cache.m_nonspec_possible_truths) + return false; + + class ipa_node_params *params_summary = IPA_NODE_REF (m_node); + unsigned int nargs = params_summary + ? ipa_get_param_count (params_summary) : 0; + + if (m_inline_param_summary.exists () || cache.m_inline_param_summary.exists ()) + { + for (unsigned int i = 0; i < nargs; i++) + { + if (!ipa_is_param_used_by_ipa_predicates (params_summary, i)) + continue; + if (i >= m_inline_param_summary.length () + || m_inline_param_summary[i].useless_p ()) + { + if (i < cache.m_inline_param_summary.length () + && !cache.m_inline_param_summary[i].useless_p ()) + return false; + continue; + } + if (i >= cache.m_inline_param_summary.length () + || cache.m_inline_param_summary[i].useless_p ()) + { + if (i < m_inline_param_summary.length () + && !m_inline_param_summary[i].useless_p ()) + return false; + continue; + } + if (!m_inline_param_summary[i].equal_to + (cache.m_inline_param_summary[i])) + return false; + } + } + if (m_avals->m_known_vals.exists () || cache.m_known_vals.exists ()) + { + for (unsigned int i = 0; i < nargs; i++) + { + if (!ipa_is_param_used_by_indirect_call (params_summary, i)) + continue; + if (i >= m_avals->m_known_vals.length () || !m_avals->m_known_vals[i]) + { + if (i < cache.m_known_vals.length () && cache.m_known_vals[i]) + return false; + continue; + } + if (i >= cache.m_known_vals.length () || !cache.m_known_vals[i]) + { + if (i < m_avals->m_known_vals.length () + && m_avals->m_known_vals[i]) + return false; + continue; + } + if (m_avals->m_known_vals[i] != cache.m_known_vals[i]) + return false; + } + } + if (m_avals->m_known_contexts.exists () || cache.m_known_contexts.exists ()) + { + for (unsigned int i = 0; i < nargs; i++) + { + if (!ipa_is_param_used_by_polymorphic_call (params_summary, i)) + continue; + if (i >= m_avals->m_known_contexts.length () + || m_avals->m_known_contexts[i].useless_p ()) + { + if (i < cache.m_known_contexts.length () + && !cache.m_known_contexts[i].useless_p ()) + return false; + continue; + } + if (i >= cache.m_known_contexts.length () + || cache.m_known_contexts[i].useless_p ()) + { + if (i < m_avals->m_known_contexts.length () + && !m_avals->m_known_contexts[i].useless_p ()) + return false; + continue; + } + if (!m_avals->m_known_contexts[i].equal_to + (cache.m_known_contexts[i])) + return false; + } + } + if (m_avals->m_known_aggs.exists () || cache.m_known_aggs.exists ()) + { + for (unsigned int i = 0; i < nargs; i++) + { + if (!ipa_is_param_used_by_indirect_call (params_summary, i)) + continue; + if (i >= m_avals->m_known_aggs.length () + || m_avals->m_known_aggs[i].is_empty ()) + { + if (i < cache.m_known_aggs.length () + && !cache.m_known_aggs[i].is_empty ()) + return false; + continue; + } + if (i >= cache.m_known_aggs.length () + || cache.m_known_aggs[i].is_empty ()) + { + if (i < m_avals->m_known_aggs.length () + && !m_avals->m_known_aggs[i].is_empty ()) + return false; + continue; + } + if (!m_avals->m_known_aggs[i].equal_to (cache.m_known_aggs[i])) + return false; + } + } + return true; +} + /* At the moment we implement primitive single entry LRU cache. */ class node_context_summary { public: - node_context_cache_entry entry; + ipa_node_context_cache_entry entry; node_context_summary () : entry () @@ -184,30 +406,26 @@ do_estimate_edge_time (struct cgraph_edge *edge, sreal *ret_nonspec_time) ipa_hints hints; struct cgraph_node *callee; clause_t clause, nonspec_clause; - auto_vec<tree, 32> known_vals; - auto_vec<ipa_polymorphic_call_context, 32> known_contexts; - auto_vec<ipa_agg_value_set, 32> known_aggs; + ipa_call_arg_values avals; class ipa_call_summary *es = ipa_call_summaries->get (edge); int min_size = -1; callee = edge->callee->ultimate_alias_target (); gcc_checking_assert (edge->inline_failed); - evaluate_properties_for_edge (edge, true, - &clause, &nonspec_clause, &known_vals, - &known_contexts, &known_aggs); - ipa_call_context ctx (callee, clause, nonspec_clause, known_vals, - known_contexts, known_aggs, es->param); + evaluate_properties_for_edge (edge, true, &clause, &nonspec_clause, + &avals, true); + ipa_call_context ctx (callee, clause, nonspec_clause, es->param, &avals); if (node_context_cache != NULL) { - node_context_summary *e = node_context_cache->get_create (callee); - if (e->entry.ctx.equal_to (ctx)) + node_context_summary *e = node_context_cache->get (callee); + if (e && ctx.equivalent_to_p (e->entry)) { node_context_cache_hit++; - size = e->entry.size; - time = e->entry.time; - nonspec_time = e->entry.nonspec_time; - hints = e->entry.hints; + size = e->entry.m_size; + time = e->entry.m_time; + nonspec_time = e->entry.m_nonspec_time; + hints = e->entry.m_hints; if (flag_checking && !opt_for_fn (callee->decl, flag_profile_partial_training) && !callee->count.ipa_p ()) @@ -226,18 +444,21 @@ do_estimate_edge_time (struct cgraph_edge *edge, sreal *ret_nonspec_time) } else { - if (e->entry.ctx.exists_p ()) - node_context_cache_miss++; + if (e) + { + node_context_cache_miss++; + e->entry.release (); + } else - node_context_cache_clear++; - e->entry.ctx.release (true); + e = node_context_cache->get_create (callee); + ctx.estimate_size_and_time (&size, &min_size, &time, &nonspec_time, &hints); - e->entry.size = size; - e->entry.time = time; - e->entry.nonspec_time = nonspec_time; - e->entry.hints = hints; - e->entry.ctx.duplicate_from (ctx); + ctx.store_to_cache (&e->entry); + e->entry.m_size = size; + e->entry.m_time = time; + e->entry.m_nonspec_time = nonspec_time; + e->entry.m_hints = hints; } } else @@ -255,7 +476,6 @@ do_estimate_edge_time (struct cgraph_edge *edge, sreal *ret_nonspec_time) : edge->caller->count.ipa ()))) hints |= INLINE_HINT_known_hot; - ctx.release (); gcc_checking_assert (size >= 0); gcc_checking_assert (time >= 0); @@ -272,6 +492,8 @@ do_estimate_edge_time (struct cgraph_edge *edge, sreal *ret_nonspec_time) entry->size = size + (size >= 0); hints |= simple_edge_hints (edge); + /* This is how we distinguish valid entries, consumer must decrement hint + to get the real value. */ entry->hints = hints + 1; } if (ret_nonspec_time) @@ -307,9 +529,7 @@ do_estimate_edge_size (struct cgraph_edge *edge) int size; struct cgraph_node *callee; clause_t clause, nonspec_clause; - auto_vec<tree, 32> known_vals; - auto_vec<ipa_polymorphic_call_context, 32> known_contexts; - auto_vec<ipa_agg_value_set, 32> known_aggs; + ipa_call_arg_values avals; /* When we do caching, use do_estimate_edge_time to populate the entry. */ @@ -325,14 +545,10 @@ do_estimate_edge_size (struct cgraph_edge *edge) /* Early inliner runs without caching, go ahead and do the dirty work. */ gcc_checking_assert (edge->inline_failed); - evaluate_properties_for_edge (edge, true, - &clause, &nonspec_clause, - &known_vals, &known_contexts, - &known_aggs); - ipa_call_context ctx (callee, clause, nonspec_clause, known_vals, - known_contexts, known_aggs, vNULL); + evaluate_properties_for_edge (edge, true, &clause, &nonspec_clause, &avals, + true); + ipa_call_context ctx (callee, clause, nonspec_clause, vNULL, &avals); ctx.estimate_size_and_time (&size, NULL, NULL, NULL, NULL); - ctx.release (); return size; } @@ -346,9 +562,7 @@ do_estimate_edge_hints (struct cgraph_edge *edge) ipa_hints hints; struct cgraph_node *callee; clause_t clause, nonspec_clause; - auto_vec<tree, 32> known_vals; - auto_vec<ipa_polymorphic_call_context, 32> known_contexts; - auto_vec<ipa_agg_value_set, 32> known_aggs; + ipa_call_arg_values avals; /* When we do caching, use do_estimate_edge_time to populate the entry. */ @@ -364,14 +578,10 @@ do_estimate_edge_hints (struct cgraph_edge *edge) /* Early inliner runs without caching, go ahead and do the dirty work. */ gcc_checking_assert (edge->inline_failed); - evaluate_properties_for_edge (edge, true, - &clause, &nonspec_clause, - &known_vals, &known_contexts, - &known_aggs); - ipa_call_context ctx (callee, clause, nonspec_clause, known_vals, - known_contexts, known_aggs, vNULL); + evaluate_properties_for_edge (edge, true, &clause, &nonspec_clause, &avals, + true); + ipa_call_context ctx (callee, clause, nonspec_clause, vNULL, &avals); ctx.estimate_size_and_time (NULL, NULL, NULL, NULL, &hints); - ctx.release (); hints |= simple_edge_hints (edge); return hints; } diff --git a/gcc/ipa-prop.c b/gcc/ipa-prop.c index da50f0837fd..1220ba9351a 100644 --- a/gcc/ipa-prop.c +++ b/gcc/ipa-prop.c @@ -5795,4 +5795,14 @@ ipa_agg_value::equal_to (const ipa_agg_value &other) return offset == other.offset && operand_equal_p (value, other.value, 0); } + +/* Clean-up also */ + +ipa_call_arg_values::~ipa_call_arg_values () +{ + ipa_release_agg_values (m_known_aggs, false); +} + + + #include "gt-ipa-prop.h" diff --git a/gcc/ipa-prop.h b/gcc/ipa-prop.h index 23fcf905ef3..cc35848f53e 100644 --- a/gcc/ipa-prop.h +++ b/gcc/ipa-prop.h @@ -433,6 +433,48 @@ ipa_get_jf_ancestor_type_preserved (struct ipa_jump_func *jfunc) return jfunc->value.ancestor.agg_preserved; } +/* Class bundling the various potentially known properties about actual + arguments of a particular call. */ + +class ipa_call_arg_values +{ +public: + ~ipa_call_arg_values (); + + /* If m_known_vals (vector of known "scalar" values) is sufficiantly long, + return its element at INDEX, otherwise return NULL. */ + tree safe_sval_at (int index) + { + /* TODO: Assert non-negative index here and test. */ + if ((unsigned) index < m_known_vals.length ()) + return m_known_vals[index]; + return NULL; + } + + /* If m_known_aggs is sufficiantly long, return the pointer rto its element + at INDEX, otherwise return NULL. */ + ipa_agg_value_set *safe_aggval_at (int index) + { + /* TODO: Assert non-negative index here and test. */ + if ((unsigned) index < m_known_aggs.length ()) + return &m_known_aggs[index]; + return NULL; + } + + /* Vector describing known values of parameters. */ + auto_vec<tree, 32> m_known_vals; + + /* Vector describing known polymorphic call contexts. */ + auto_vec<ipa_polymorphic_call_context, 32> m_known_contexts; + + /* Vector describing known aggregate values. */ + auto_vec<ipa_agg_value_set, 32> m_known_aggs; + + /* Vector describing known value ranges of arguments. */ + auto_vec<value_range, 32> m_known_value_ranges; +}; + + /* Summary describing a single formal parameter. */ struct GTY(()) ipa_param_descriptor @@ -970,12 +1012,10 @@ void ipa_initialize_node_params (struct cgraph_node *node); bool ipa_propagate_indirect_call_infos (struct cgraph_edge *cs, vec<cgraph_edge *> *new_edges); -/* Indirect edge and binfo processing. */ +/* Indirect edge processing and target discovery. */ tree ipa_get_indirect_edge_target (struct cgraph_edge *ie, - vec<tree>, - vec<ipa_polymorphic_call_context>, - vec<ipa_agg_value_set>, - bool *); + ipa_call_arg_values *avals, + bool *speculative); struct cgraph_edge *ipa_make_edge_direct_to_target (struct cgraph_edge *, tree, bool speculative = false); tree ipa_impossible_devirt_target (struct cgraph_edge *, tree);