From patchwork Thu Aug 24 13:14:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Hubicka X-Patchwork-Id: 1825388 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=ZcrszUff; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RWk8Y2ySWz1yg8 for ; Thu, 24 Aug 2023 23:15:08 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 85C2A3856261 for ; Thu, 24 Aug 2023 13:15:04 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 85C2A3856261 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1692882904; bh=9xAvBnC+m2b5XAnA40Xbec1QaiVSB9wC4eQmlR3HtVI=; h=Date:To:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=ZcrszUffF3bilMBZi3I3HbY2Mko5H026EkoXrYmQa1YrO3OyuaJIi37E0vBdJkAHW nL2sm+6CA+zRNexakst9VIeKTIV5MmXyP4G/9wFqiGMK7SssD3KlLNJGqh5k1AkBWx x82ivxEA2qu89md1ak0Bxvx40bfStfv692Gan4M4= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from nikam.ms.mff.cuni.cz (nikam.ms.mff.cuni.cz [195.113.20.16]) by sourceware.org (Postfix) with ESMTPS id E68053858426 for ; Thu, 24 Aug 2023 13:14:37 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org E68053858426 Received: by nikam.ms.mff.cuni.cz (Postfix, from userid 16202) id E148E281986; Thu, 24 Aug 2023 15:14:32 +0200 (CEST) Date: Thu, 24 Aug 2023 15:14:32 +0200 To: gcc-patches@gcc.gnu.org Subject: Check that passes do not forget to define profile Message-ID: MIME-Version: 1.0 Content-Disposition: inline X-Spam-Status: No, score=-11.2 required=5.0 tests=BAYES_00, DKIM_INVALID, DKIM_SIGNED, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, JMQ_SPF_NEUTRAL, KAM_DMARC_STATUS, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jan Hubicka via Gcc-patches From: Jan Hubicka Reply-To: Jan Hubicka Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" Hi, this patch extends verifier to check that all probabilities and counts are initialized if profile is supposed to be present. This is a bit complicated by the posibility that we inline !flag_guess_branch_probability function into function with profile defined and in this case we need to stop verification. For this reason I added flag to cfg structure tracking this. Bootstrapped/regtested x86_64-linux, comitted. gcc/ChangeLog: * cfg.h (struct control_flow_graph): New field full_profile. * auto-profile.cc (afdo_annotate_cfg): Set full_profile to true. * cfg.cc (init_flow): Set full_profile to false. * graphite.cc (graphite_transform_loops): Set full_profile to false. * lto-streamer-in.cc (input_cfg): Initialize full_profile flag. * predict.cc (pass_profile::execute): Set full_profile to true. * symtab-thunks.cc (expand_thunk): Set full_profile to true. * tree-cfg.cc (gimple_verify_flow_info): Verify that profile is full if full_profile is set. * tree-inline.cc (initialize_cfun): Initialize full_profile. (expand_call_inline): Combine full_profile. diff --git a/gcc/auto-profile.cc b/gcc/auto-profile.cc index e3af3555e75..ff3b763945c 100644 --- a/gcc/auto-profile.cc +++ b/gcc/auto-profile.cc @@ -1578,6 +1578,7 @@ afdo_annotate_cfg (const stmt_set &promoted_stmts) } update_max_bb_count (); profile_status_for_fn (cfun) = PROFILE_READ; + cfun->cfg->full_profile = true; if (flag_value_profile_transformations) { gimple_value_profile_transformations (); diff --git a/gcc/cfg.cc b/gcc/cfg.cc index 9eb9916f61a..b7865f14e7f 100644 --- a/gcc/cfg.cc +++ b/gcc/cfg.cc @@ -81,6 +81,7 @@ init_flow (struct function *the_fun) = ENTRY_BLOCK_PTR_FOR_FN (the_fun); the_fun->cfg->edge_flags_allocated = EDGE_ALL_FLAGS; the_fun->cfg->bb_flags_allocated = BB_ALL_FLAGS; + the_fun->cfg->full_profile = false; } /* Helper function for remove_edge and free_cffg. Frees edge structure diff --git a/gcc/cfg.h b/gcc/cfg.h index a0e944979c8..53e2553012c 100644 --- a/gcc/cfg.h +++ b/gcc/cfg.h @@ -78,6 +78,9 @@ struct GTY(()) control_flow_graph { /* Dynamically allocated edge/bb flags. */ int edge_flags_allocated; int bb_flags_allocated; + + /* Set if the profile is computed on every edge and basic block. */ + bool full_profile; }; diff --git a/gcc/graphite.cc b/gcc/graphite.cc index 19f8975ffa2..2b387d5b016 100644 --- a/gcc/graphite.cc +++ b/gcc/graphite.cc @@ -512,6 +512,8 @@ graphite_transform_loops (void) if (changed) { + /* FIXME: Graphite does not update profile meaningfully currently. */ + cfun->cfg->full_profile = false; cleanup_tree_cfg (); profile_status_for_fn (cfun) = PROFILE_ABSENT; release_recorded_exits (cfun); diff --git a/gcc/lto-streamer-in.cc b/gcc/lto-streamer-in.cc index 0cce14414ca..d3128fcebe4 100644 --- a/gcc/lto-streamer-in.cc +++ b/gcc/lto-streamer-in.cc @@ -1030,6 +1030,7 @@ input_cfg (class lto_input_block *ib, class data_in *data_in, basic_block p_bb; unsigned int i; int index; + bool full_profile = false; init_empty_tree_cfg_for_function (fn); @@ -1071,6 +1072,8 @@ input_cfg (class lto_input_block *ib, class data_in *data_in, data_in->location_cache.input_location_and_block (&e->goto_locus, &bp, ib, data_in); e->probability = profile_probability::stream_in (ib); + if (!e->probability.initialized_p ()) + full_profile = false; } @@ -1145,6 +1148,7 @@ input_cfg (class lto_input_block *ib, class data_in *data_in, /* Rebuild the loop tree. */ flow_loops_find (loops); + cfun->cfg->full_profile = full_profile; } diff --git a/gcc/predict.cc b/gcc/predict.cc index 5a1a561cc24..396746cbfd1 100644 --- a/gcc/predict.cc +++ b/gcc/predict.cc @@ -4131,6 +4131,7 @@ pass_profile::execute (function *fun) scev_initialize (); tree_estimate_probability (false); + cfun->cfg->full_profile = true; if (nb_loops > 1) scev_finalize (); diff --git a/gcc/symtab-thunks.cc b/gcc/symtab-thunks.cc index 4c04235c41b..23ead0d2138 100644 --- a/gcc/symtab-thunks.cc +++ b/gcc/symtab-thunks.cc @@ -648,6 +648,7 @@ expand_thunk (cgraph_node *node, bool output_asm_thunks, ? PROFILE_READ : PROFILE_GUESSED; /* FIXME: C++ FE should stop setting TREE_ASM_WRITTEN on thunks. */ TREE_ASM_WRITTEN (thunk_fndecl) = false; + cfun->cfg->full_profile = true; delete_unreachable_blocks (); update_ssa (TODO_update_ssa); checking_verify_flow_info (); diff --git a/gcc/tree-cfg.cc b/gcc/tree-cfg.cc index 272d5ce321e..ffab7518b15 100644 --- a/gcc/tree-cfg.cc +++ b/gcc/tree-cfg.cc @@ -5684,6 +5684,26 @@ gimple_verify_flow_info (void) error ("fallthru to exit from bb %d", e->src->index); err = true; } + if (cfun->cfg->full_profile + && !ENTRY_BLOCK_PTR_FOR_FN (cfun)->count.initialized_p ()) + { + error ("entry block count not initialized"); + err = true; + } + if (cfun->cfg->full_profile + && !EXIT_BLOCK_PTR_FOR_FN (cfun)->count.initialized_p ()) + { + error ("exit block count not initialized"); + err = true; + } + if (cfun->cfg->full_profile + && !single_succ_edge + (ENTRY_BLOCK_PTR_FOR_FN (cfun))->probability.initialized_p ()) + { + error ("probability of edge from entry block not initialized"); + err = true; + } + FOR_EACH_BB_FN (bb, cfun) { @@ -5691,6 +5711,22 @@ gimple_verify_flow_info (void) stmt = NULL; + if (cfun->cfg->full_profile) + { + if (!bb->count.initialized_p ()) + { + error ("count of bb %d not initialized", bb->index); + err = true; + } + FOR_EACH_EDGE (e, ei, bb->succs) + if (!e->probability.initialized_p ()) + { + error ("probability of edge %d->%d not initialized", + bb->index, e->dest->index); + err = true; + } + } + /* Skip labels on the start of basic block. */ for (gsi = gsi_start_bb (bb); !gsi_end_p (gsi); gsi_next (&gsi)) { diff --git a/gcc/tree-inline.cc b/gcc/tree-inline.cc index 954b39ae1c6..1d98d96df71 100644 --- a/gcc/tree-inline.cc +++ b/gcc/tree-inline.cc @@ -2815,6 +2815,7 @@ initialize_cfun (tree new_fndecl, tree callee_fndecl, profile_count count) init_empty_tree_cfg (); profile_status_for_fn (cfun) = profile_status_for_fn (src_cfun); + cfun->cfg->full_profile = src_cfun->cfg->full_profile; profile_count num = count; profile_count den = ENTRY_BLOCK_PTR_FOR_FN (src_cfun)->count; @@ -4953,6 +4954,7 @@ expand_call_inline (basic_block bb, gimple *stmt, copy_body_data *id, id->src_cfun = DECL_STRUCT_FUNCTION (fn); id->reset_location = DECL_IGNORED_P (fn); id->call_stmt = call_stmt; + cfun->cfg->full_profile &= id->src_cfun->cfg->full_profile; /* When inlining into an OpenMP SIMD-on-SIMT loop, arrange for new automatic variables to be added to IFN_GOMP_SIMT_ENTER argument list. */ From patchwork Tue Sep 5 07:16:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Jelinek X-Patchwork-Id: 1829779 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=sGNIlawk; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Rfxf23mncz1yh1 for ; Tue, 5 Sep 2023 17:17:14 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 8CDC2385842E for ; Tue, 5 Sep 2023 07:17:12 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 8CDC2385842E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1693898232; bh=CTDwOkg0Fopr9mSXvGa93tXJiABq1/g+6P7m07Vhpdk=; h=Date:To:Cc:Subject:References:In-Reply-To:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=sGNIlawkFW4s9SMvUCWApajZrIFM+AJ3Ez7K/fZswkdqpE2eePtXtlVaQgtseHCIu FkV6pyKFc1Zq4FjV2lvOvNy1l4ye6+YeZXq56aVrDhxsvd4WDAIl/j6YzsZazG9EP4 pOSRXgABdwZXJdax7EqPvXB3dbQ2LZKpJpSp29XE= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id CBD40385843A for ; Tue, 5 Sep 2023 07:16:51 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org CBD40385843A Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-201-bBOB_dhjMUu9gEFrLlY2MA-1; Tue, 05 Sep 2023 03:16:50 -0400 X-MC-Unique: bBOB_dhjMUu9gEFrLlY2MA-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C8B9C8015AA; Tue, 5 Sep 2023 07:16:49 +0000 (UTC) Received: from tucnak.zalov.cz (unknown [10.45.224.16]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8CF37493110; Tue, 5 Sep 2023 07:16:49 +0000 (UTC) Received: from tucnak.zalov.cz (localhost [127.0.0.1]) by tucnak.zalov.cz (8.17.1/8.17.1) with ESMTPS id 3857GlrA2876942 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Tue, 5 Sep 2023 09:16:48 +0200 Received: (from jakub@localhost) by tucnak.zalov.cz (8.17.1/8.17.1/Submit) id 3857GlsF2876941; Tue, 5 Sep 2023 09:16:47 +0200 Date: Tue, 5 Sep 2023 09:16:47 +0200 To: Jan Hubicka Cc: gcc-patches@gcc.gnu.org Subject: [PATCH 16/12] _BitInt profile fixes [PR102989] Message-ID: References: MIME-Version: 1.0 In-Reply-To: X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-Spam-Status: No, score=-3.4 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jakub Jelinek via Gcc-patches From: Jakub Jelinek Reply-To: Jakub Jelinek Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" On Thu, Aug 24, 2023 at 03:14:32PM +0200, Jan Hubicka via Gcc-patches wrote: > this patch extends verifier to check that all probabilities and counts are > initialized if profile is supposed to be present. This is a bit complicated > by the posibility that we inline !flag_guess_branch_probability function > into function with profile defined and in this case we need to stop > verification. For this reason I added flag to cfg structure tracking this. This patch broke a couple of _BitInt tests (in the admittedly still uncommitted series - still waiting for review of the C FE bits). Here is a minimal patch to make it work again, though I'm not sure if in the if_then_else and if_then_if_then_else cases I shouldn't scale count of the other bbs as well. if_then method creates if (COND) new_bb1; in a middle of some pre-existing bb (with PROB that COND is true), if_then_else if (COND) new_bb1; else new_bb2; and if_then_if_then_else if (COND1) { if (COND2) new_bb2; else new_bb1; } with PROB1 and PROB2 probabilities that COND1 and COND2 are true. The lowering happens shortly after IPA. Ok for trunk with rest of the series? 2023-09-05 Jakub Jelinek PR c/102989 * gimple-lower-bitint.cc (bitint_large_huge::if_then_else, bitint_large_huge::if_then_if_then_else): Use make_single_succ_edge rather than make_edge, initialize bb->count. Jakub --- gcc/gimple-lower-bitint.cc.jj 2023-09-04 09:45:37.357004452 +0200 +++ gcc/gimple-lower-bitint.cc 2023-09-04 22:53:30.343756938 +0200 @@ -683,9 +683,10 @@ bitint_large_huge::if_then_else (gimple e1->flags = EDGE_FALSE_VALUE; e3->probability = prob; e1->probability = prob.invert (); + bb->count = e1->src->count.apply_probability (prob); set_immediate_dominator (CDI_DOMINATORS, bb, e1->src); set_immediate_dominator (CDI_DOMINATORS, e2->dest, e1->src); - edge_true = make_edge (bb, e2->dest, EDGE_FALLTHRU); + edge_true = make_single_succ_edge (bb, e2->dest, EDGE_FALLTHRU); edge_false = e2; m_gsi = gsi_after_labels (bb); } @@ -741,7 +742,8 @@ bitint_large_huge::if_then_if_then_else e4->probability = prob2; e2->flags = EDGE_FALSE_VALUE; e2->probability = prob2.invert (); - e4 = make_edge (bb, e3->dest, EDGE_FALLTHRU); + bb->count = e2->src->count.apply_probability (prob2); + e4 = make_single_succ_edge (bb, e3->dest, EDGE_FALLTHRU); e2 = find_edge (e2->dest, e3->dest); edge_true_true = e4; edge_true_false = e2;