From patchwork Wed Feb 22 10:11:14 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Martin Jambor X-Patchwork-Id: 731037 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3vStSR5vxdz9s7N for ; Wed, 22 Feb 2017 21:11:34 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.b="RMoAK5To"; dkim-atps=neutral DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:date :from:to:cc:subject:message-id:mime-version:content-type; q=dns; s=default; b=aIwnLiwMd5AefJ3nWP37TqzFDhNh5zYewv3P4uPuwKMaYRWe3/ aAuWN8EGJgjHZqH4wlerJpCTEXjqHh1hFDcROqOQTreiOvz+AfFToH6nR9xdPV2+ kGgC8AYrbWKhIYd12jpJk2eICczE2vrTjuSkq7X6HfNeo/3Uxn+iVtWgQ= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:date :from:to:cc:subject:message-id:mime-version:content-type; s= default; bh=HcGVyLH/7g7ZV6lbv0IX1rgfJYo=; b=RMoAK5ToNRTsAWqFMP3q huARzY5p4XwhLUNJNAZIzbehN8xLM+0brvVLAy0imWbb5VzFcONODmi6aBAsF1nN 5H8UB3aUC7Tz4H9b986MSsFXuxhdRhQpSoq0ZdJwdlKUwpP5CMA/SvSdOUrKMA+n 0K2ea+wzOOLyp+UtGp0Kmdw= Received: (qmail 100294 invoked by alias); 22 Feb 2017 10:11:21 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 100282 invoked by uid 89); 22 Feb 2017 10:11:20 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-24.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, SPF_PASS, UNSUBSCRIBE_BODY autolearn=ham version=3.3.2 spammy=Grow, replacements, martins, Martins X-HELO: mx2.suse.de Received: from mx2.suse.de (HELO mx2.suse.de) (195.135.220.15) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 22 Feb 2017 10:11:16 +0000 Received: from relay2.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id ED8BAAAC7 for ; Wed, 22 Feb 2017 10:11:14 +0000 (UTC) Date: Wed, 22 Feb 2017 11:11:14 +0100 From: Martin Jambor To: GCC Patches Cc: Jan Hubicka Subject: [PR 78140] Reuse same IPA bits and VR info Message-ID: <20170222101114.quuz7doforc36s3o@virgil.suse.cz> Mail-Followup-To: GCC Patches , Jan Hubicka MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.6.2 (2016-07-01) X-IsSubscribed: yes Hello, this is a fix for PR 78140 which is about LTO WPA of Firefox taking 1GB memory more than gcc 6. It works by reusing the ipa_bits and value_range that we previously had directly in jump functions and which are just too big to be allocated for each actual argument in all of Firefox. Reusing is achieved by two hash table traits derived from ggc_cache_remove which apparently has been created just for this purpose and once I understood them they made my life a lot easier. In future, I will have a look at applying this method to other parts of jump functions as well. According to my measurements, the patch saves about 1.2 GB of memory. The problem is that some change last week (between revision 245382 and 245595) has more than invalidated this: | compiler | WPA mem (GB) | |---------------------+--------------| | gcc 6 branch | 3.86 | | trunk rev. 245382 | 5.21 | | patched rev. 245382 | 4.06 | | trunk rev. 245595 | 6.59 | | patched rev. 245595 | 5.25 | (I have verified this by Martin's way of measuring things.) I will try to bisect what commit has caused the increase. Still, the patch helps a lot. There is one thing in the patch that intrigues me, I do not understand why I had to mark value_range with GTY((for_user)) - as opposed to just GTY(()) that was there before - whereas ipa_bits does not need it. If anyone could enlighten me, that would be great. But I suppose this is not an indication of anything being wrong under the hood. I have bootstrapped and LTO-bootstrapped the patch on x86_64-linux and also bootstrapped (C, C++ and Fortran) on an aarch64 and i686 cfarm machine. I have also LTO-built Firefox with the patch and used it to browse for a while and it seemed fine. OK for trunk? Thanks, Martin 2017-02-20 Martin Jambor PR lto/78140 * ipa-prop.h (ipa_bits): Removed field known. (ipa_jump_func): Removed field vr_known. Changed fields bits and m_vr to pointers. Adjusted their comments to warn about their sharing. (ipcp_transformation_summary): Change bits to a vector of pointers. (ipa_check_create_edge_args): Moved to ipa-prop.c, declare. (ipa_get_ipa_bits_for_value): Declare. * tree-vrp.h (value_range): Mark as GTY((for_user)). * ipa-prop.c (ipa_bit_ggc_hash_traits): New. (ipa_bits_hash_table): Likewise. (ipa_vr_ggc_hash_traits): Likewise. (ipa_vr_hash_table): Likewise. (ipa_print_node_jump_functions_for_edge): Adjust for bits and m_vr being pointers and vr_known being removed. (ipa_set_jf_unknown): Likewise. (ipa_get_ipa_bits_for_value): New function. (ipa_set_jfunc_bits): Likewise. (ipa_get_value_range): New overloaded functions. (ipa_set_jfunc_vr): Likewise. (ipa_compute_jump_functions_for_edge): Use the above functions to construct bits and vr parts of jump functions. (ipa_check_create_edge_args): Move here from ipa-prop.h, also allocate ipa_bits_hash_table and ipa_vr_hash_table if they do not already exist. (ipcp_grow_transformations_if_necessary): Also allocate ipa_bits_hash_table and ipa_vr_hash_table if they do not already exist. (ipa_node_params_t::duplicate): Do not copy bits, just pointers to them. Fix too long lines. (ipa_write_jump_function): Adjust for bits and m_vr being pointers and vr_known being removed. (ipa_read_jump_function): Use new setter functions to construct bits and vr parts of jump functions or set them to NULL. (write_ipcp_transformation_info): Adjust for bits being pointers. (read_ipcp_transformation_info): Likewise. (ipcp_update_bits): Likewise. Fix excessively long lines a trailing space. Include gt-ipa-prop.h. * ipa-cp.c (propagate_bits_across_jump_function): Adjust for bits being pointers. (ipcp_store_bits_results): Likewise. (propagate_vr_across_jump_function): Adjust for m_vr being a pointer. Do not write to existing jump functions but use a temporary instead. --- gcc/ipa-cp.c | 49 ++++---- gcc/ipa-prop.c | 381 ++++++++++++++++++++++++++++++++++++++++++--------------- gcc/ipa-prop.h | 32 ++--- gcc/tree-vrp.h | 2 +- 4 files changed, 319 insertions(+), 145 deletions(-) diff --git a/gcc/ipa-cp.c b/gcc/ipa-cp.c index aa3c9973a66..16e7e2216ab 100644 --- a/gcc/ipa-cp.c +++ b/gcc/ipa-cp.c @@ -1819,8 +1819,8 @@ propagate_bits_across_jump_function (cgraph_edge *cs, int idx, and we store it in jump function during analysis stage. */ if (src_lats->bits_lattice.bottom_p () - && jfunc->bits.known) - return dest_lattice->meet_with (jfunc->bits.value, jfunc->bits.mask, + && jfunc->bits) + return dest_lattice->meet_with (jfunc->bits->value, jfunc->bits->mask, precision); else return dest_lattice->meet_with (src_lats->bits_lattice, precision, sgn, @@ -1829,10 +1829,9 @@ propagate_bits_across_jump_function (cgraph_edge *cs, int idx, else if (jfunc->type == IPA_JF_ANCESTOR) return dest_lattice->set_to_bottom (); - - else if (jfunc->bits.known) - return dest_lattice->meet_with (jfunc->bits.value, jfunc->bits.mask, precision); - + else if (jfunc->bits) + return dest_lattice->meet_with (jfunc->bits->value, jfunc->bits->mask, + precision); else return dest_lattice->set_to_bottom (); } @@ -1903,19 +1902,21 @@ propagate_vr_across_jump_function (cgraph_edge *cs, ipa_jump_func *jfunc, val = fold_convert (param_type, val); if (TREE_OVERFLOW_P (val)) val = drop_tree_overflow (val); - jfunc->vr_known = true; - jfunc->m_vr.type = VR_RANGE; - jfunc->m_vr.min = val; - jfunc->m_vr.max = val; - return dest_lat->meet_with (&jfunc->m_vr); + + value_range tmpvr; + memset (&tmpvr, 0, sizeof (tmpvr)); + tmpvr.type = VR_RANGE; + tmpvr.min = val; + tmpvr.max = val; + return dest_lat->meet_with (&tmpvr); } } value_range vr; - if (jfunc->vr_known - && ipa_vr_operation_and_type_effects (&vr, &jfunc->m_vr, NOP_EXPR, + if (jfunc->m_vr + && ipa_vr_operation_and_type_effects (&vr, jfunc->m_vr, NOP_EXPR, param_type, - TREE_TYPE (jfunc->m_vr.min))) + TREE_TYPE (jfunc->m_vr->min))) return dest_lat->meet_with (&vr); else return dest_lat->set_to_bottom (); @@ -4870,19 +4871,17 @@ ipcp_store_bits_results (void) for (unsigned i = 0; i < count; i++) { ipcp_param_lattices *plats = ipa_get_parm_lattices (info, i); - ipa_bits bits_jfunc; + ipa_bits *jfbits; if (plats->bits_lattice.constant_p ()) - { - bits_jfunc.known = true; - bits_jfunc.value = plats->bits_lattice.get_value (); - bits_jfunc.mask = plats->bits_lattice.get_mask (); - } + jfbits + = ipa_get_ipa_bits_for_value (plats->bits_lattice.get_value (), + plats->bits_lattice.get_mask ()); else - bits_jfunc.known = false; + jfbits = NULL; - ts->bits->quick_push (bits_jfunc); - if (!dump_file || !bits_jfunc.known) + ts->bits->quick_push (jfbits); + if (!dump_file || !jfbits) continue; if (!dumped_sth) { @@ -4891,9 +4890,9 @@ ipcp_store_bits_results (void) dumped_sth = true; } fprintf (dump_file, " param %i: value = ", i); - print_hex (bits_jfunc.value, dump_file); + print_hex (jfbits->value, dump_file); fprintf (dump_file, ", mask = "); - print_hex (bits_jfunc.mask, dump_file); + print_hex (jfbits->mask, dump_file); fprintf (dump_file, "\n"); } } diff --git a/gcc/ipa-prop.c b/gcc/ipa-prop.c index e4e44ce20c6..358e439f51b 100644 --- a/gcc/ipa-prop.c +++ b/gcc/ipa-prop.c @@ -60,6 +60,93 @@ vec *ipcp_transformations; /* Vector where the parameter infos are actually stored. */ vec *ipa_edge_args_vector; +/* Traits for a hash table for reusing already existing ipa_bits. */ + +struct ipa_bit_ggc_hash_traits : public ggc_cache_remove +{ + typedef ipa_bits *value_type; + typedef ipa_bits *compare_type; + static hashval_t + hash (const ipa_bits *p) + { + hashval_t t = (hashval_t) p->value.to_shwi (); + return iterative_hash_host_wide_int (p->mask.to_shwi (), t); + } + static bool + equal (const ipa_bits *a, const ipa_bits *b) + { + return a->value == b->value && a->mask == b->mask; + } + static void + mark_empty (ipa_bits *&p) + { + p = NULL; + } + static bool + is_empty (const ipa_bits *p) + { + return p == NULL; + } + static bool + is_deleted (const ipa_bits *p) + { + return p == reinterpret_cast (1); + } + static void + mark_deleted (ipa_bits *&p) + { + p = reinterpret_cast (1); + } +}; + +/* Hash table for avoid repeated allocations of equal ipa_bits. */ +static GTY ((cache)) hash_table *ipa_bits_hash_table; + +/* Traits for a hash table for reusing value_ranges used for IPA. Note that + the equiv bitmap is not hashed and is expected to be NULL. */ + +struct ipa_vr_ggc_hash_traits : public ggc_cache_remove +{ + typedef value_range *value_type; + typedef value_range *compare_type; + static hashval_t + hash (const value_range *p) + { + gcc_checking_assert (!p->equiv); + hashval_t t = (hashval_t) p->type; + t = iterative_hash_expr (p->min, t); + return iterative_hash_expr (p->max, t); + } + static bool + equal (const value_range *a, const value_range *b) + { + return a->type == b->type && a->min == b->min && a->max == b->max; + } + static void + mark_empty (value_range *&p) + { + p = NULL; + } + static bool + is_empty (const value_range *p) + { + return p == NULL; + } + static bool + is_deleted (const value_range *p) + { + return p == reinterpret_cast (1); + } + static void + mark_deleted (value_range *&p) + { + p = reinterpret_cast (1); + } +}; + +/* Hash table for avoid repeated allocations of equal value_ranges. */ +static GTY ((cache)) hash_table *ipa_vr_hash_table; + /* Holders of ipa cgraph hooks: */ static struct cgraph_edge_hook_list *edge_removal_hook_holder; static struct cgraph_2edge_hook_list *edge_duplication_hook_holder; @@ -298,23 +385,25 @@ ipa_print_node_jump_functions_for_edge (FILE *f, struct cgraph_edge *cs) ctx->dump (dump_file); } - if (jump_func->bits.known) + if (jump_func->bits) { - fprintf (f, " value: "); print_hex (jump_func->bits.value, f); - fprintf (f, ", mask: "); print_hex (jump_func->bits.mask, f); + fprintf (f, " value: "); + print_hex (jump_func->bits->value, f); + fprintf (f, ", mask: "); + print_hex (jump_func->bits->mask, f); fprintf (f, "\n"); } else fprintf (f, " Unknown bits\n"); - if (jump_func->vr_known) + if (jump_func->m_vr) { fprintf (f, " VR "); fprintf (f, "%s[", - (jump_func->m_vr.type == VR_ANTI_RANGE) ? "~" : ""); - print_decs (jump_func->m_vr.min, f); + (jump_func->m_vr->type == VR_ANTI_RANGE) ? "~" : ""); + print_decs (jump_func->m_vr->min, f); fprintf (f, ", "); - print_decs (jump_func->m_vr.max, f); + print_decs (jump_func->m_vr->max, f); fprintf (f, "]\n"); } else @@ -397,8 +486,8 @@ static void ipa_set_jf_unknown (struct ipa_jump_func *jfunc) { jfunc->type = IPA_JF_UNKNOWN; - jfunc->bits.known = false; - jfunc->vr_known = false; + jfunc->bits = NULL; + jfunc->m_vr = NULL; } /* Set JFUNC to be a copy of another jmp (to be used by jump function @@ -1658,6 +1747,91 @@ ipa_get_callee_param_type (struct cgraph_edge *e, int i) return NULL; } +/* Return ipa_bits with VALUE and MASK values, which can be either a newly + allocated structure or a previously existing one shared with other jump + functions and/or transformation summaries. */ + +ipa_bits * +ipa_get_ipa_bits_for_value (const widest_int &value, const widest_int &mask) +{ + ipa_bits tmp; + tmp.value = value; + tmp.mask = mask; + + ipa_bits **slot = ipa_bits_hash_table->find_slot (&tmp, INSERT); + if (*slot) + return *slot; + + ipa_bits *res = ggc_alloc (); + res->value = value; + res->mask = mask; + *slot = res; + + return res; +} + +/* Assign to JF a pointer to ipa_bits structure with VALUE and MASK. Use hash + table in order to avoid creating multiple same ipa_bits structures. */ + +static void +ipa_set_jfunc_bits (ipa_jump_func *jf, const widest_int &value, + const widest_int &mask) +{ + jf->bits = ipa_get_ipa_bits_for_value (value, mask); +} + +/* Return a pointer to a value_range just like *TMP, but either find it in + ipa_vr_hash_table or allocate it in GC memory. TMP->equiv must be NULL. */ + +static value_range * +ipa_get_value_range (value_range *tmp) +{ + value_range **slot = ipa_vr_hash_table->find_slot (tmp, INSERT); + if (*slot) + return *slot; + + value_range *vr = ggc_alloc (); + *vr = *tmp; + *slot = vr; + + return vr; +} + +/* Return a pointer to a value range consisting of TYPE, MIN, MAX and an empty + equiv set. Use hash table in order to avoid creating multiple same copies of + value_ranges. */ + +static value_range * +ipa_get_value_range (enum value_range_type type, tree min, tree max) +{ + value_range tmp; + tmp.type = type; + tmp.min = min; + tmp.max = max; + tmp.equiv = NULL; + return ipa_get_value_range (&tmp); +} + +/* Assign to JF a pointer to a value_range structure with TYPE, MIN and MAX and + a NULL equiv bitmap. Use hash table in order to avoid creating multiple + same value_range structures. */ + +static void +ipa_set_jfunc_vr (ipa_jump_func *jf, enum value_range_type type, + tree min, tree max) +{ + jf->m_vr = ipa_get_value_range (type, min, max); +} + +/* Assign to JF a pointer to a value_range just liek TMP but either fetch a + copy from ipa_vr_hash_table or allocate a new on in GC memory. */ + +static void +ipa_set_jfunc_vr (ipa_jump_func *jf, value_range *tmp) +{ + jf->m_vr = ipa_get_value_range (tmp); +} + /* Compute jump function for all arguments of callsite CS and insert the information in the jump_functions array in the ipa_edge_args corresponding to this callsite. */ @@ -1714,14 +1888,11 @@ ipa_compute_jump_functions_for_edge (struct ipa_func_body_info *fbi, if (addr_nonzero) { - jfunc->vr_known = true; - jfunc->m_vr.type = VR_ANTI_RANGE; - jfunc->m_vr.min = build_int_cst (TREE_TYPE (arg), 0); - jfunc->m_vr.max = build_int_cst (TREE_TYPE (arg), 0); - jfunc->m_vr.equiv = NULL; + tree z = build_int_cst (TREE_TYPE (arg), 0); + ipa_set_jfunc_vr (jfunc, VR_ANTI_RANGE, z, z); } else - gcc_assert (!jfunc->vr_known); + gcc_assert (!jfunc->m_vr); } else { @@ -1732,56 +1903,48 @@ ipa_compute_jump_functions_for_edge (struct ipa_func_body_info *fbi, && (type = get_range_info (arg, &min, &max)) && (type == VR_RANGE || type == VR_ANTI_RANGE)) { - value_range vr; - - vr.type = type; - vr.min = wide_int_to_tree (TREE_TYPE (arg), min); - vr.max = wide_int_to_tree (TREE_TYPE (arg), max); - vr.equiv = NULL; - extract_range_from_unary_expr (&jfunc->m_vr, - NOP_EXPR, - param_type, - &vr, TREE_TYPE (arg)); - if (jfunc->m_vr.type == VR_RANGE - || jfunc->m_vr.type == VR_ANTI_RANGE) - jfunc->vr_known = true; + value_range tmpvr,resvr; + + tmpvr.type = type; + tmpvr.min = wide_int_to_tree (TREE_TYPE (arg), min); + tmpvr.max = wide_int_to_tree (TREE_TYPE (arg), max); + tmpvr.equiv = NULL; + memset (&resvr, 0, sizeof (resvr)); + extract_range_from_unary_expr (&resvr, NOP_EXPR, param_type, + &tmpvr, TREE_TYPE (arg)); + if (resvr.type == VR_RANGE || resvr.type == VR_ANTI_RANGE) + ipa_set_jfunc_vr (jfunc, &resvr); else - jfunc->vr_known = false; + gcc_assert (!jfunc->m_vr); } else - gcc_assert (!jfunc->vr_known); + gcc_assert (!jfunc->m_vr); } if (INTEGRAL_TYPE_P (TREE_TYPE (arg)) && (TREE_CODE (arg) == SSA_NAME || TREE_CODE (arg) == INTEGER_CST)) { - jfunc->bits.known = true; - if (TREE_CODE (arg) == SSA_NAME) - { - jfunc->bits.value = 0; - jfunc->bits.mask = widest_int::from (get_nonzero_bits (arg), - TYPE_SIGN (TREE_TYPE (arg))); - } + ipa_set_jfunc_bits (jfunc, 0, + widest_int::from (get_nonzero_bits (arg), + TYPE_SIGN (TREE_TYPE (arg)))); else - { - jfunc->bits.value = wi::to_widest (arg); - jfunc->bits.mask = 0; - } + ipa_set_jfunc_bits (jfunc, wi::to_widest (arg), 0); } else if (POINTER_TYPE_P (TREE_TYPE (arg))) { unsigned HOST_WIDE_INT bitpos; unsigned align; - jfunc->bits.known = true; get_pointer_alignment_1 (arg, &align, &bitpos); - jfunc->bits.mask = wi::mask(TYPE_PRECISION (TREE_TYPE (arg)), false) - .and_not (align / BITS_PER_UNIT - 1); - jfunc->bits.value = bitpos / BITS_PER_UNIT; + widest_int mask + = wi::mask(TYPE_PRECISION (TREE_TYPE (arg)), false) + .and_not (align / BITS_PER_UNIT - 1); + widest_int value = bitpos / BITS_PER_UNIT; + ipa_set_jfunc_bits (jfunc, value, mask); } else - gcc_assert (!jfunc->bits.known); + gcc_assert (!jfunc->bits); if (is_gimple_ip_invariant (arg) || (VAR_P (arg) @@ -3545,6 +3708,22 @@ ipa_propagate_indirect_call_infos (struct cgraph_edge *cs, return changed; } +/* Ensure that array of edge arguments infos is big enough to accommodate a + structure for all edges and reallocates it if not. Also, allocate + associated hash tables is they do not already exist. */ + +void +ipa_check_create_edge_args (void) +{ + if (vec_safe_length (ipa_edge_args_vector) + <= (unsigned) symtab->edges_max_uid) + vec_safe_grow_cleared (ipa_edge_args_vector, symtab->edges_max_uid + 1); + if (!ipa_bits_hash_table) + ipa_bits_hash_table = hash_table::create_ggc (37); + if (!ipa_vr_hash_table) + ipa_vr_hash_table = hash_table::create_ggc (37); +} + /* Frees all dynamically allocated structures that the argument info points to. */ @@ -3581,7 +3760,8 @@ ipa_free_all_node_params (void) ipa_node_params_sum = NULL; } -/* Grow ipcp_transformations if necessary. */ +/* Grow ipcp_transformations if necessary. Also allocate any necessary hash + tables if they do not already exist. */ void ipcp_grow_transformations_if_necessary (void) @@ -3589,6 +3769,10 @@ ipcp_grow_transformations_if_necessary (void) if (vec_safe_length (ipcp_transformations) <= (unsigned) symtab->cgraph_max_uid) vec_safe_grow_cleared (ipcp_transformations, symtab->cgraph_max_uid + 1); + if (!ipa_bits_hash_table) + ipa_bits_hash_table = hash_table::create_ggc (37); + if (!ipa_vr_hash_table) + ipa_vr_hash_table = hash_table::create_ggc (37); } /* Set the aggregate replacements of NODE to be AGGVALS. */ @@ -3775,12 +3959,18 @@ ipa_node_params_t::duplicate(cgraph_node *src, cgraph_node *dst, ipa_set_node_agg_value_chain (dst, new_av); } - ipcp_transformation_summary *src_trans = ipcp_get_transformation_summary (src); + ipcp_transformation_summary *src_trans + = ipcp_get_transformation_summary (src); if (src_trans) { ipcp_grow_transformations_if_necessary (); src_trans = ipcp_get_transformation_summary (src); + ipcp_transformation_summary *dst_trans + = ipcp_get_transformation_summary (dst); + + dst_trans->bits = vec_safe_copy (src_trans->bits); + const vec *src_vr = src_trans->m_vr; vec *&dst_vr = ipcp_get_transformation_summary (dst)->m_vr; @@ -3791,18 +3981,6 @@ ipa_node_params_t::duplicate(cgraph_node *src, cgraph_node *dst, dst_vr->quick_push ((*src_vr)[i]); } } - - if (src_trans && vec_safe_length (src_trans->bits) > 0) - { - ipcp_grow_transformations_if_necessary (); - src_trans = ipcp_get_transformation_summary (src); - const vec *src_bits = src_trans->bits; - vec *&dst_bits - = ipcp_get_transformation_summary (dst)->bits; - vec_safe_reserve_exact (dst_bits, src_bits->length ()); - for (unsigned i = 0; i < src_bits->length (); ++i) - dst_bits->quick_push ((*src_bits)[i]); - } } /* Register our cgraph hooks if they are not already there. */ @@ -4718,21 +4896,21 @@ ipa_write_jump_function (struct output_block *ob, } bp = bitpack_create (ob->main_stream); - bp_pack_value (&bp, jump_func->bits.known, 1); + bp_pack_value (&bp, !!jump_func->bits, 1); streamer_write_bitpack (&bp); - if (jump_func->bits.known) + if (jump_func->bits) { - streamer_write_widest_int (ob, jump_func->bits.value); - streamer_write_widest_int (ob, jump_func->bits.mask); + streamer_write_widest_int (ob, jump_func->bits->value); + streamer_write_widest_int (ob, jump_func->bits->mask); } - bp_pack_value (&bp, jump_func->vr_known, 1); + bp_pack_value (&bp, !!jump_func->m_vr, 1); streamer_write_bitpack (&bp); - if (jump_func->vr_known) + if (jump_func->m_vr) { streamer_write_enum (ob->main_stream, value_rang_type, - VR_LAST, jump_func->m_vr.type); - stream_write_tree (ob, jump_func->m_vr.min, true); - stream_write_tree (ob, jump_func->m_vr.max, true); + VR_LAST, jump_func->m_vr->type); + stream_write_tree (ob, jump_func->m_vr->min, true); + stream_write_tree (ob, jump_func->m_vr->max, true); } } @@ -4809,26 +4987,25 @@ ipa_read_jump_function (struct lto_input_block *ib, bool bits_known = bp_unpack_value (&bp, 1); if (bits_known) { - jump_func->bits.known = true; - jump_func->bits.value = streamer_read_widest_int (ib); - jump_func->bits.mask = streamer_read_widest_int (ib); + widest_int value = streamer_read_widest_int (ib); + widest_int mask = streamer_read_widest_int (ib); + ipa_set_jfunc_bits (jump_func, value, mask); } else - jump_func->bits.known = false; + jump_func->bits = NULL; struct bitpack_d vr_bp = streamer_read_bitpack (ib); bool vr_known = bp_unpack_value (&vr_bp, 1); if (vr_known) { - jump_func->vr_known = true; - jump_func->m_vr.type = streamer_read_enum (ib, - value_range_type, - VR_LAST); - jump_func->m_vr.min = stream_read_tree (ib, data_in); - jump_func->m_vr.max = stream_read_tree (ib, data_in); + enum value_range_type type = streamer_read_enum (ib, value_range_type, + VR_LAST); + tree min = stream_read_tree (ib, data_in); + tree max = stream_read_tree (ib, data_in); + ipa_set_jfunc_vr (jump_func, type, min, max); } else - jump_func->vr_known = false; + jump_func->m_vr = NULL; } /* Stream out parts of cgraph_indirect_call_info corresponding to CS that are @@ -5207,14 +5384,14 @@ write_ipcp_transformation_info (output_block *ob, cgraph_node *node) for (unsigned i = 0; i < count; ++i) { - const ipa_bits& bits_jfunc = (*ts->bits)[i]; + const ipa_bits *bits_jfunc = (*ts->bits)[i]; struct bitpack_d bp = bitpack_create (ob->main_stream); - bp_pack_value (&bp, bits_jfunc.known, 1); + bp_pack_value (&bp, !!bits_jfunc, 1); streamer_write_bitpack (&bp); - if (bits_jfunc.known) + if (bits_jfunc) { - streamer_write_widest_int (ob, bits_jfunc.value); - streamer_write_widest_int (ob, bits_jfunc.mask); + streamer_write_widest_int (ob, bits_jfunc->value); + streamer_write_widest_int (ob, bits_jfunc->mask); } } } @@ -5281,13 +5458,14 @@ read_ipcp_transformation_info (lto_input_block *ib, cgraph_node *node, for (i = 0; i < count; i++) { - ipa_bits& bits_jfunc = (*ts->bits)[i]; struct bitpack_d bp = streamer_read_bitpack (ib); - bits_jfunc.known = bp_unpack_value (&bp, 1); - if (bits_jfunc.known) + bool known = bp_unpack_value (&bp, 1); + if (known) { - bits_jfunc.value = streamer_read_widest_int (ib); - bits_jfunc.mask = streamer_read_widest_int (ib); + ipa_bits *bits + = ipa_get_ipa_bits_for_value (streamer_read_widest_int (ib), + streamer_read_widest_int (ib)); + (*ts->bits)[i] = bits; } } } @@ -5554,7 +5732,7 @@ ipcp_update_bits (struct cgraph_node *node) if (!ts || vec_safe_length (ts->bits) == 0) return; - vec &bits = *ts->bits; + vec &bits = *ts->bits; unsigned count = bits.length (); for (unsigned i = 0; i < count; ++i, parm = next_parm) @@ -5566,10 +5744,11 @@ ipcp_update_bits (struct cgraph_node *node) gcc_checking_assert (parm); next_parm = DECL_CHAIN (parm); - if (!bits[i].known - || !(INTEGRAL_TYPE_P (TREE_TYPE (parm)) || POINTER_TYPE_P (TREE_TYPE (parm))) + if (!bits[i] + || !(INTEGRAL_TYPE_P (TREE_TYPE (parm)) + || POINTER_TYPE_P (TREE_TYPE (parm))) || !is_gimple_reg (parm)) - continue; + continue; tree ddef = ssa_default_def (DECL_STRUCT_FUNCTION (node->decl), parm); if (!ddef) @@ -5577,8 +5756,8 @@ ipcp_update_bits (struct cgraph_node *node) if (dump_file) { - fprintf (dump_file, "Adjusting mask for param %u to ", i); - print_hex (bits[i].mask, dump_file); + fprintf (dump_file, "Adjusting mask for param %u to ", i); + print_hex (bits[i]->mask, dump_file); fprintf (dump_file, "\n"); } @@ -5587,14 +5766,14 @@ ipcp_update_bits (struct cgraph_node *node) unsigned prec = TYPE_PRECISION (TREE_TYPE (ddef)); signop sgn = TYPE_SIGN (TREE_TYPE (ddef)); - wide_int nonzero_bits = wide_int::from (bits[i].mask, prec, UNSIGNED) - | wide_int::from (bits[i].value, prec, sgn); + wide_int nonzero_bits = wide_int::from (bits[i]->mask, prec, UNSIGNED) + | wide_int::from (bits[i]->value, prec, sgn); set_nonzero_bits (ddef, nonzero_bits); } else { - unsigned tem = bits[i].mask.to_uhwi (); - unsigned HOST_WIDE_INT bitpos = bits[i].value.to_uhwi (); + unsigned tem = bits[i]->mask.to_uhwi (); + unsigned HOST_WIDE_INT bitpos = bits[i]->value.to_uhwi (); unsigned align = tem & -tem; unsigned misalign = bitpos & (align - 1); @@ -5757,3 +5936,5 @@ ipcp_transform_function (struct cgraph_node *node) else return TODO_update_ssa_only_virtuals; } + +#include "gt-ipa-prop.h" diff --git a/gcc/ipa-prop.h b/gcc/ipa-prop.h index 8f7eb088813..74234eb470d 100644 --- a/gcc/ipa-prop.h +++ b/gcc/ipa-prop.h @@ -152,11 +152,10 @@ struct GTY(()) ipa_bits Similar to ccp_lattice_t, if xth bit of mask is 0, implies xth bit of value is constant. */ widest_int mask; - /* True if jump function is known. */ - bool known; }; /* Info about value ranges. */ + struct GTY(()) ipa_vr { /* The data fields below are valid only if known is true. */ @@ -175,13 +174,15 @@ struct GTY (()) ipa_jump_func description. */ struct ipa_agg_jump_function agg; - /* Information about zero/non-zero bits. */ - struct ipa_bits bits; + /* Information about zero/non-zero bits. The pointed to structure is shared + betweed different jump functions. Use ipa_set_jfunc_bits to set this + field. */ + struct ipa_bits *bits; /* Information about value range, containing valid data only when vr_known is - true. */ - value_range m_vr; - bool vr_known; + true. The pointed to structure is shared betweed different jump + functions. Use ipa_set_jfunc_vr to set this field. */ + struct value_range *m_vr; enum jump_func_type type; /* Represents a value of a jump function. pass_through is used only in jump @@ -547,7 +548,7 @@ struct GTY(()) ipcp_transformation_summary /* Linked list of known aggregate values. */ ipa_agg_replacement_value *agg_values; /* Known bits information. */ - vec *bits; + vec *bits; /* Value range information. */ vec *m_vr; }; @@ -630,6 +631,7 @@ extern GTY(()) vec *ipa_edge_args_vector; /* Creating and freeing ipa_node_params and ipa_edge_args. */ void ipa_create_all_node_params (void); void ipa_create_all_edge_args (void); +void ipa_check_create_edge_args (void); void ipa_free_edge_args_substructures (struct ipa_edge_args *); void ipa_free_all_node_params (void); void ipa_free_all_edge_args (void); @@ -651,17 +653,6 @@ ipa_check_create_node_params (void) ipa_node_params_t (symtab, true)); } -/* This function ensures the array of edge arguments infos is big enough to - accommodate a structure for all edges and reallocates it if not. */ - -static inline void -ipa_check_create_edge_args (void) -{ - if (vec_safe_length (ipa_edge_args_vector) - <= (unsigned) symtab->edges_max_uid) - vec_safe_grow_cleared (ipa_edge_args_vector, symtab->edges_max_uid + 1); -} - /* Returns true if the array of edge infos is large enough to accommodate an info for EDGE. The main purpose of this function is that debug dumping function can check info availability without causing reallocations. */ @@ -703,6 +694,9 @@ tree ipa_get_indirect_edge_target (struct cgraph_edge *ie, struct cgraph_edge *ipa_make_edge_direct_to_target (struct cgraph_edge *, tree, bool speculative = false); tree ipa_impossible_devirt_target (struct cgraph_edge *, tree); +ipa_bits *ipa_get_ipa_bits_for_value (const widest_int &value, + const widest_int &mask); + /* Functions related to both. */ void ipa_analyze_node (struct cgraph_node *); diff --git a/gcc/tree-vrp.h b/gcc/tree-vrp.h index 4fed978d0aa..ef2c68a752b 100644 --- a/gcc/tree-vrp.h +++ b/gcc/tree-vrp.h @@ -24,7 +24,7 @@ enum value_range_type { VR_UNDEFINED, VR_RANGE, /* Range of values that can be associated with an SSA_NAME after VRP has executed. */ -struct GTY(()) value_range +struct GTY((for_user)) value_range { /* Lattice value represented by this range. */ enum value_range_type type;