From patchwork Sun May 12 22:57:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?6ZKf5bGF5ZOy?= X-Patchwork-Id: 1934351 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Vcyk32qh2z1yfq for ; Mon, 13 May 2024 08:59:39 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 9CB113858D3C for ; Sun, 12 May 2024 22:59:37 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbg150.qq.com (smtpbg150.qq.com [18.132.163.193]) by sourceware.org (Postfix) with ESMTPS id 185953858CDB for ; Sun, 12 May 2024 22:58:10 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 185953858CDB Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 185953858CDB Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=18.132.163.193 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1715554694; cv=none; b=oQT3hP+WwtgYUhr5VlRqcTRv1xCC35KfFdT3efrvlTn6jLBEcgWDuKsuzn3djGwjQqTqYyWf7DV0ZwlJUcwurh/63kUkOfxaUjHj0LSkzw2xVPy8ni8EsWjgpkopE2qP3TTbEyxTKU2htcurU58O5gQ6q7CH6PP3u/BhgxpyqiM= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1715554694; c=relaxed/simple; bh=Pt89L+q/bSxDypBfpkLpHCcKaKCPluVqKtQh0Tu60ZY=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=Dzd6H+XEz3TR1bI2YY2agZh9MbPdx75i0ebwy8pGZONU3nNQ2bgyUwDEB+DiHC5O5LaAjG6QEoKhUhqzRTS9DWxHN7SvoExq+xgn0H/3BNU14Fg81gbhdTj8s9IewWrYnacT8HPkFCprSqz20pn4DGx5+3D+1T2Sj5+VmkjxDmA= ARC-Authentication-Results: i=1; server2.sourceware.org X-QQ-mid: bizesmtpsz6t1715554665t07rmkp X-QQ-Originating-IP: IdqFDcpjn1ZcSQ3jmkiApRmXC9Qhe82cJr7YxX3qu+4= Received: from rios-cad121.hadoop.rioslab.org ( [58.60.1.28]) by bizesmtp.qq.com (ESMTP) with id ; Mon, 13 May 2024 06:57:45 +0800 (CST) X-QQ-SSF: 01400000000000H0V000000A0000000 X-QQ-FEAT: WmhCOH6N4IkJWQzvm9nxLL64DkWWOUM4jDKB9Wrg57kDwFDRNR+L0TADMl7II oq+U02r3Eq3k9lqlPYdHJjl3vM5j7LwqzcxYgqiuhjo23omkZo6MB2G215Cce4HmH8cceud 9XuiLngJoORfOat+sPiXWXBOEPTi8EbfXrixHzqawkMVKJL0fsylovqRUrJm/6CToo2zyfn 4a62Yi5P7ar8kFdG5RTY94tD1vosOnXLjkLPta5jF7ntYnF9De6TyhouIq9mtdajneiAsbO 4a9Dxl/7hM0ST0MZQXhaRhDizKBQAsRr884uCEeFJbwxF20H6+jxzhx3HwXiJQ9DcJQ94sN vDUpKr/WdyvdxNAy+65Z/NUKgTZKxUTe6NneafRH710tXdvmcNKucrnrCusF4TUYzMct9x8 X-QQ-GoodBg: 2 X-BIZMAIL-ID: 18435693362927522032 From: Juzhe-Zhong To: gcc-patches@gcc.gnu.org Cc: vmakarov@redhat.com, richard.sandiford@arm.com, jin.xia@rivai.ai, dimitar@dinux.eu, Juzhe-Zhong Subject: [SUBREG V4 1/4] DF: Add -ftrack-subreg-liveness option Date: Mon, 13 May 2024 06:57:35 +0800 Message-Id: <20240512225738.528917-2-juzhe.zhong@rivai.ai> X-Mailer: git-send-email 2.36.3 In-Reply-To: <20240512225738.528917-1-juzhe.zhong@rivai.ai> References: <20240512225738.528917-1-juzhe.zhong@rivai.ai> MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtpsz:rivai.ai:qybglogicsvrgz:qybglogicsvrgz7a-one-0 X-Spam-Status: No, score=-10.3 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org --- gcc/common.opt | 4 ++++ gcc/common.opt.urls | 3 +++ gcc/doc/invoke.texi | 8 ++++++++ gcc/opts.cc | 1 + 4 files changed, 16 insertions(+) diff --git a/gcc/common.opt b/gcc/common.opt index 40cab3cb36a..5710e817abe 100644 --- a/gcc/common.opt +++ b/gcc/common.opt @@ -2163,6 +2163,10 @@ fira-share-spill-slots Common Var(flag_ira_share_spill_slots) Init(1) Optimization Share stack slots for spilled pseudo-registers. +ftrack-subreg-liveness +Common Var(flag_track_subreg_liveness) Init(0) Optimization +Track subreg liveness information. + fira-verbose= Common RejectNegative Joined UInteger Var(flag_ira_verbose) Init(5) -fira-verbose= Control IRA's level of diagnostic messages. diff --git a/gcc/common.opt.urls b/gcc/common.opt.urls index f71ed80a34b..59f27a6f7c6 100644 --- a/gcc/common.opt.urls +++ b/gcc/common.opt.urls @@ -880,6 +880,9 @@ UrlSuffix(gcc/Optimize-Options.html#index-fira-share-save-slots) fira-share-spill-slots UrlSuffix(gcc/Optimize-Options.html#index-fira-share-spill-slots) +ftrack-subreg-liveness +UrlSuffix(gcc/Optimize-Options.html#index-ftrack-subreg-liveness) + fira-verbose= UrlSuffix(gcc/Developer-Options.html#index-fira-verbose) diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi index ddcd5213f06..fbcde8aa745 100644 --- a/gcc/doc/invoke.texi +++ b/gcc/doc/invoke.texi @@ -13188,6 +13188,14 @@ Disable sharing of stack slots allocated for pseudo-registers. Each pseudo-register that does not get a hard register gets a separate stack slot, and as a result function stack frames are larger. +@opindex ftrack-subreg-liveness +@item -ftrack-subreg-liveness +Enable tracking subreg liveness information. This infomation allows IRA +and LRA to support subreg coalesce feature which can improve the quality +of register allocation. + +This option is enabled at level @option{-O3} for all targets. + @opindex flra-remat @item -flra-remat Enable CFG-sensitive rematerialization in LRA. Instead of loading diff --git a/gcc/opts.cc b/gcc/opts.cc index 14d1767e48f..8fe3a213807 100644 --- a/gcc/opts.cc +++ b/gcc/opts.cc @@ -698,6 +698,7 @@ static const struct default_options default_options_table[] = { OPT_LEVELS_3_PLUS, OPT_funswitch_loops, NULL, 1 }, { OPT_LEVELS_3_PLUS, OPT_fvect_cost_model_, NULL, VECT_COST_MODEL_DYNAMIC }, { OPT_LEVELS_3_PLUS, OPT_fversion_loops_for_strides, NULL, 1 }, + { OPT_LEVELS_3_PLUS, OPT_ftrack_subreg_liveness, NULL, 1 }, /* -O3 parameters. */ { OPT_LEVELS_3_PLUS, OPT__param_max_inline_insns_auto_, NULL, 30 }, From patchwork Sun May 12 22:57:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?6ZKf5bGF5ZOy?= X-Patchwork-Id: 1934352 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VcykD53YVz1yfq for ; Mon, 13 May 2024 08:59:48 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id ED2373860766 for ; Sun, 12 May 2024 22:59:46 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbg154.qq.com (smtpbg154.qq.com [15.184.224.54]) by sourceware.org (Postfix) with ESMTPS id 797573858C31 for ; Sun, 12 May 2024 22:58:16 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 797573858C31 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 797573858C31 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=15.184.224.54 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1715554707; cv=none; b=kc+Alg0bhcJiGhOfN5zRK2y0uUSO90t5XaOY5uYlf8bef7IsS+uXFI4LwcFRBnccMqQ/aenBQKTmAV/baIW0SGUaRoxm/fAkbTYye2+dMnqBBCoyB/lGbsulCMsSUG3i5rz3jDagPXUCZ1GFBEg4gaJ1VP61C2bE82HoUFIdNHI= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1715554707; c=relaxed/simple; bh=nYZzd8wvjkrZtNcwJrtN7Z4uSeL1dZcxSsw97rJc2Ts=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=K6V4u6O0FjKEXkVHXjx1jiwS3CHYS6Dp5uuGOBTkWZbP3r58YOxtItLpD0jwpPTnuiJ+LfDxm1AkiBlD5tWjxpxibZoES5gQKEkav2qdJvhSu4oN+u6ODSR5rqzetWhaEbXdXi2ZPbvrQimQ58TpIoUEZUs2bRylOgY4wccxUXg= ARC-Authentication-Results: i=1; server2.sourceware.org X-QQ-mid: bizesmtpsz6t1715554669togmzb0 X-QQ-Originating-IP: gsHyq/l9TZ/Quk2oFtcCmLxCB+LoNh4C5hteXhRcjUo= Received: from rios-cad121.hadoop.rioslab.org ( [58.60.1.28]) by bizesmtp.qq.com (ESMTP) with id ; Mon, 13 May 2024 06:57:48 +0800 (CST) X-QQ-SSF: 01400000000000H0V000000A0000000 X-QQ-FEAT: WKaXkSLX6oOlSkNTBjP0wwpHrjVEpq27SzW/rTM8KMZHpVxmCNZ4uEl9tnOpl ryoLZhpC0FeNa7xOdGYrrdCv6nOJHn8MDNJ+Z4k4mcyLdd9alEjQqN34wZc+C9j6jt00cFW Ssl3VZPoV3/WWTAqVwfcThMoSYhoufvL04I0/BWMCEJrK+dn6Mn60WcAHApoRzBy/IW8kyW pv6PClWoRMc7B5ed5wtdSuK3tw9OJeARgiAwTx7CYPav2kUft4tTjQOenobimzdCRwOOrk7 rHyRcrLOSfBMUJCusu8amSdz8cQzo5EpMfQiHIWUAmsy84NLUAnXKYI+YwTVL2Pf4iTCmMV /NW3/OMZhTOfUEYc5bvf64gt6QSJ1JUwov5rhXY0IPiHXTcTdugJ23dzfmz8jDTLijriMrb bdQLkU/tM6g= X-QQ-GoodBg: 2 X-BIZMAIL-ID: 10383953195413339322 From: Juzhe-Zhong To: gcc-patches@gcc.gnu.org Cc: vmakarov@redhat.com, richard.sandiford@arm.com, jin.xia@rivai.ai, dimitar@dinux.eu, Juzhe-Zhong Subject: [SUBREG V4 2/4] DF: Add DF_LIVE_SUBREG problem Date: Mon, 13 May 2024 06:57:36 +0800 Message-Id: <20240512225738.528917-3-juzhe.zhong@rivai.ai> X-Mailer: git-send-email 2.36.3 In-Reply-To: <20240512225738.528917-1-juzhe.zhong@rivai.ai> References: <20240512225738.528917-1-juzhe.zhong@rivai.ai> MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtpsz:rivai.ai:qybglogicsvrgz:qybglogicsvrgz7a-one-0 X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_ASCII_DIVIDERS, KAM_DMARC_STATUS, KAM_SHORT, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org --- gcc/Makefile.in | 1 + gcc/df-problems.cc | 886 ++++++++++++++++++++++++++++++++++++++- gcc/df.h | 159 +++++++ gcc/regs.h | 5 + gcc/sbitmap.cc | 98 +++++ gcc/sbitmap.h | 2 + gcc/subreg-live-range.cc | 233 ++++++++++ gcc/subreg-live-range.h | 60 +++ gcc/timevar.def | 1 + 9 files changed, 1444 insertions(+), 1 deletion(-) create mode 100644 gcc/subreg-live-range.cc create mode 100644 gcc/subreg-live-range.h diff --git a/gcc/Makefile.in b/gcc/Makefile.in index a7f15694c34..67d2e3ca1bc 100644 --- a/gcc/Makefile.in +++ b/gcc/Makefile.in @@ -1684,6 +1684,7 @@ OBJS = \ store-motion.o \ streamer-hooks.o \ stringpool.o \ + subreg-live-range.o \ substring-locations.o \ target-globals.o \ targhooks.o \ diff --git a/gcc/df-problems.cc b/gcc/df-problems.cc index 88ee0dd67fc..01f1f850925 100644 --- a/gcc/df-problems.cc +++ b/gcc/df-problems.cc @@ -28,6 +28,7 @@ along with GCC; see the file COPYING3. If not see #include "target.h" #include "rtl.h" #include "df.h" +#include "subreg-live-range.h" #include "memmodel.h" #include "tm_p.h" #include "insn-config.h" @@ -1344,8 +1345,891 @@ df_lr_verify_transfer_functions (void) bitmap_clear (&all_blocks); } +/*---------------------------------------------------------------------------- + REGISTER AND SUBREGS LIVES + Like DF_LR, but include tracking subreg liveness. Currently used to provide + subreg liveness related information to the register allocator. The subreg + information is currently tracked for registers that satisfy the following + conditions: + 1. REG is a pseudo register + 2. MODE_SIZE > UNIT_SIZE + 3. MODE_SIZE is a multiple of UNIT_SIZE + 4. REG is used via subreg pattern + Assuming: MODE = the machine mode of the REG + MODE_SIZE = GET_MODE_SIZE (MODE) + UNIT_SIZE = REGMODE_NATURAL_SIZE (MODE) + Condition 3 is currently strict, maybe it can be removed in the future, but + for now it is sufficient. +----------------------------------------------------------------------------*/ + +/* These two empty data are used as default data in case the user does not turn + * on the track-subreg-liveness feature. */ +bitmap_head df_subreg_empty_bitmap; +subregs_live df_subreg_empty_live; + +/* Private data for live_subreg problem. */ +struct df_live_subreg_problem_data +{ + /* Record registers that need to track subreg liveness. */ + bitmap_head tracked_regs; + /* An obstack for the bitmaps we need for this problem. */ + bitmap_obstack live_subreg_bitmaps; +}; + +/* Helper functions. */ + +static df_live_subreg_bb_info * +df_live_subreg_get_bb_info (unsigned int index) +{ + if (index < df_live_subreg->block_info_size) + return &static_cast ( + df_live_subreg->block_info)[index]; + else + return nullptr; +} + +static df_live_subreg_local_bb_info * +get_live_subreg_local_bb_info (unsigned int bb_index) +{ + return df_live_subreg_get_bb_info (bb_index); +} + +/* Return true if regno is a multireg. */ +bool +multireg_p (int regno) +{ + if (regno < FIRST_PSEUDO_REGISTER) + return false; + rtx regno_rtx = regno_reg_rtx[regno]; + machine_mode reg_mode = GET_MODE (regno_rtx); + poly_int64 total_size = GET_MODE_SIZE (reg_mode); + poly_int64 natural_size = REGMODE_NATURAL_SIZE (reg_mode); + return maybe_gt (total_size, natural_size) + && multiple_p (total_size, natural_size); +} + +/* Return true if the REGNO need be track with subreg liveness. */ + +static bool +need_track_subreg_p (unsigned regno) +{ + auto problem_data + = (struct df_live_subreg_problem_data *) df_live_subreg->problem_data; + return bitmap_bit_p (&problem_data->tracked_regs, regno); +} + +/* Fill RANGE with the subreg range for OP in REGMODE_NATURAL_SIZE granularity. + */ +void +init_range (rtx op, sbitmap range) +{ + rtx reg = SUBREG_P (op) ? SUBREG_REG (op) : op; + machine_mode reg_mode = GET_MODE (reg); + + if (!read_modify_subreg_p (op)) + { + bitmap_set_range (range, 0, get_nblocks (reg_mode)); + return; + } + + rtx subreg = op; + machine_mode subreg_mode = GET_MODE (subreg); + poly_int64 offset = SUBREG_BYTE (subreg); + int nblocks = get_nblocks (reg_mode); + poly_int64 unit_size = REGMODE_NATURAL_SIZE (reg_mode); + poly_int64 subreg_size = GET_MODE_SIZE (subreg_mode); + poly_int64 left = offset + subreg_size; + + int subreg_start = -1; + int subreg_nblocks = -1; + for (int i = 0; i < nblocks; i += 1) + { + poly_int64 right = unit_size * (i + 1); + if (subreg_start < 0 && maybe_lt (offset, right)) + subreg_start = i; + if (subreg_nblocks < 0 && maybe_le (left, right)) + { + subreg_nblocks = i + 1 - subreg_start; + break; + } + } + gcc_assert (subreg_start >= 0 && subreg_nblocks > 0); + + bitmap_set_range (range, subreg_start, subreg_nblocks); +} + +/* Remove RANGE from BB_INFO's use data. */ +void +remove_subreg_range (df_live_subreg_local_bb_info *bb_info, unsigned int regno, + const_sbitmap range) +{ + bitmap partial = &bb_info->partial_use; + + bb_info->range_use->remove_range (regno, range); + if (bb_info->range_use->empty_p (regno)) + bitmap_clear_bit (partial, regno); +} + +/* Remove RANGE of REF from BB_INFO's use data. */ +static void +remove_subreg_range (df_live_subreg_local_bb_info *bb_info, df_ref ref) +{ + unsigned int regno = DF_REF_REGNO (ref); + machine_mode reg_mode = GET_MODE (DF_REF_REAL_REG (ref)); + + if (need_track_subreg_p (regno)) + { + auto_sbitmap range (get_nblocks (reg_mode)); + init_range (DF_REF_REG (ref), range); + remove_subreg_range (bb_info, regno, range); + } + else + { + bitmap_clear_bit (&bb_info->full_use, regno); + gcc_assert (!bitmap_bit_p (&bb_info->partial_use, regno)); + gcc_assert (!bitmap_bit_p (&bb_info->partial_def, regno)); + } +} + +/* add RANGE to BB_INFO's def. */ +void +add_subreg_range_to_def (df_live_subreg_local_bb_info *bb_info, + unsigned int regno, const_sbitmap range) +{ + bitmap partial = &bb_info->partial_def; + subregs_live *range_live = bb_info->range_def; + + bitmap_set_bit (partial, regno); + range_live->add_range (regno, range); +} + +/* add RANGE to BB_INFO's use. */ +void +add_subreg_range_to_use (df_live_subreg_local_bb_info *bb_info, + unsigned int regno, const_sbitmap range) +{ + bitmap partial = &bb_info->partial_use; + subregs_live *range_live = bb_info->range_use; + + bitmap_set_bit (partial, regno); + range_live->add_range (regno, range); +} + +/* add RANGE of REF to BB_INFO def. */ +static void +add_subreg_range_to_def (df_live_subreg_local_bb_info *bb_info, df_ref ref) +{ + unsigned int regno = DF_REF_REGNO (ref); + machine_mode reg_mode = GET_MODE (DF_REF_REAL_REG (ref)); + + if (need_track_subreg_p (regno)) + { + auto_sbitmap range (get_nblocks (reg_mode)); + init_range (DF_REF_REG (ref), range); + add_subreg_range_to_def (bb_info, regno, range); + } + else + { + bitmap full = &bb_info->full_def; + bitmap partial = &bb_info->partial_def; + bitmap_set_bit (full, regno); + gcc_assert (!bitmap_bit_p (partial, regno)); + } +} + +/* add RANGE of REF to BB_INFO use. */ +static void +add_subreg_range_to_use (df_live_subreg_local_bb_info *bb_info, df_ref ref) +{ + unsigned int regno = DF_REF_REGNO (ref); + machine_mode reg_mode = GET_MODE (DF_REF_REAL_REG (ref)); + + if (need_track_subreg_p (regno)) + { + auto_sbitmap range (get_nblocks (reg_mode)); + init_range (DF_REF_REG (ref), range); + add_subreg_range_to_use (bb_info, regno, range); + } + else + { + bitmap full = &bb_info->full_use; + bitmap partial = &bb_info->partial_use; + bitmap_set_bit (full, regno); + gcc_assert (!bitmap_bit_p (partial, regno)); + } +} + +/* Free basic block info. */ + +static void +df_live_subreg_free_bb_info (basic_block bb ATTRIBUTE_UNUSED, void *vbb_info) +{ + df_live_subreg_bb_info *bb_info = (df_live_subreg_bb_info *) vbb_info; + if (bb_info) + { + delete bb_info->range_in; + bb_info->range_in = NULL; + delete bb_info->range_out; + bb_info->range_out = NULL; + + bitmap_clear (&bb_info->all_in); + bitmap_clear (&bb_info->full_in); + bitmap_clear (&bb_info->partial_in); + bitmap_clear (&bb_info->all_out); + bitmap_clear (&bb_info->full_out); + bitmap_clear (&bb_info->partial_out); + } +} + +/* Allocate or reset bitmaps for DF_LIVE_SUBREG blocks. The solution bits are + not touched unless the block is new. */ + +static void +df_live_subreg_alloc (bitmap all_blocks ATTRIBUTE_UNUSED) +{ + struct df_live_subreg_problem_data *problem_data; + df_grow_bb_info (df_live_subreg); + if (df_live_subreg->problem_data) + problem_data + = (struct df_live_subreg_problem_data *) df_live_subreg->problem_data; + else + { + problem_data = XNEW (struct df_live_subreg_problem_data); + df_live_subreg->problem_data = problem_data; + + bitmap_obstack_initialize (&problem_data->live_subreg_bitmaps); + bitmap_initialize (&problem_data->tracked_regs, + &problem_data->live_subreg_bitmaps); + } + + bitmap_clear (&problem_data->tracked_regs); + + basic_block bb; + FOR_EACH_BB_FN (bb, cfun) + bitmap_set_bit (df_live_subreg->out_of_date_transfer_functions, bb->index); + + bitmap_set_bit (df_live_subreg->out_of_date_transfer_functions, ENTRY_BLOCK); + bitmap_set_bit (df_live_subreg->out_of_date_transfer_functions, EXIT_BLOCK); + + unsigned int bb_index; + bitmap_iterator bi; + EXECUTE_IF_SET_IN_BITMAP (df_live_subreg->out_of_date_transfer_functions, 0, + bb_index, bi) + { + /* Find the regs which we need to track it's subreg liveness. */ + rtx_insn *insn; + df_ref use; + FOR_BB_INSNS (bb, insn) + { + if (!NONDEBUG_INSN_P (insn)) + continue; + + df_insn_info *insn_info = DF_INSN_INFO_GET (insn); + + FOR_EACH_INSN_INFO_USE (use, insn_info) + { + unsigned int regno = DF_REF_REGNO (use); + /* A multireg which is used via subreg pattern. */ + if (multireg_p (regno) && DF_REF_FLAGS (use) & (DF_REF_SUBREG)) + bitmap_set_bit (&problem_data->tracked_regs, regno); + } + } + } + + if (dump_file) + { + fprintf (dump_file, ";; regs need to be tracked subreg liveness: "); + df_print_regset (dump_file, &problem_data->tracked_regs); + } + + size_t n = bitmap_count_bits (&problem_data->tracked_regs); + + EXECUTE_IF_SET_IN_BITMAP (df_live_subreg->out_of_date_transfer_functions, 0, + bb_index, bi) + { + /* Clean global infos. */ + df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb_index); + + /* When bitmaps are already initialized, just clear them. */ + if (bb_info->all_in.obstack) + { + bitmap_clear (&bb_info->all_in); + bitmap_clear (&bb_info->full_in); + bitmap_clear (&bb_info->partial_in); + bitmap_clear (&bb_info->all_out); + bitmap_clear (&bb_info->full_out); + bitmap_clear (&bb_info->partial_out); + } + else + { + bitmap_initialize (&bb_info->all_in, + &problem_data->live_subreg_bitmaps); + bitmap_initialize (&bb_info->full_in, + &problem_data->live_subreg_bitmaps); + bitmap_initialize (&bb_info->partial_in, + &problem_data->live_subreg_bitmaps); + bitmap_initialize (&bb_info->all_out, + &problem_data->live_subreg_bitmaps); + bitmap_initialize (&bb_info->full_out, + &problem_data->live_subreg_bitmaps); + bitmap_initialize (&bb_info->partial_out, + &problem_data->live_subreg_bitmaps); + } + + if (bb_info->range_in) + { + bb_info->range_in->clear (n); + bb_info->range_out->clear (n); + } + else + { + bb_info->range_in = new subregs_live (n); + bb_info->range_out = new subregs_live (n); + } + + /* Clean local infos. */ + df_live_subreg_local_bb_info *local_bb_info + = get_live_subreg_local_bb_info (bb_index); + + /* When bitmaps are already initialized, just clear them. */ + if (local_bb_info->full_use.obstack) + { + bitmap_clear (&local_bb_info->full_def); + bitmap_clear (&local_bb_info->partial_def); + bitmap_clear (&local_bb_info->full_use); + bitmap_clear (&local_bb_info->partial_use); + } + else + { + bitmap_initialize (&local_bb_info->full_def, + &problem_data->live_subreg_bitmaps); + bitmap_initialize (&local_bb_info->partial_def, + &problem_data->live_subreg_bitmaps); + bitmap_initialize (&local_bb_info->full_use, + &problem_data->live_subreg_bitmaps); + bitmap_initialize (&local_bb_info->partial_use, + &problem_data->live_subreg_bitmaps); + } + + if (local_bb_info->range_def) + { + local_bb_info->range_def->clear (n); + local_bb_info->range_use->clear (n); + } + else + { + local_bb_info->range_def = new subregs_live (n); + local_bb_info->range_use = new subregs_live (n); + } + } + + df_live_subreg->optional_p = true; +} + +/* Reset the global solution for recalculation. */ + +static void +df_live_subreg_reset (bitmap all_blocks) +{ + unsigned int bb_index; + bitmap_iterator bi; + + EXECUTE_IF_SET_IN_BITMAP (all_blocks, 0, bb_index, bi) + { + df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb_index); + gcc_assert (bb_info); + bitmap_clear (&bb_info->all_in); + bitmap_clear (&bb_info->full_in); + bitmap_clear (&bb_info->partial_in); + bitmap_clear (&bb_info->all_out); + bitmap_clear (&bb_info->full_out); + bitmap_clear (&bb_info->partial_out); + bb_info->range_in->clear (); + bb_info->range_out->clear (); + } +} + +/* Compute local live register info for basic block BB. */ + +static void +df_live_subreg_bb_local_compute (unsigned int bb_index) +{ + basic_block bb = BASIC_BLOCK_FOR_FN (cfun, bb_index); + df_live_subreg_local_bb_info *local_bb_info + = get_live_subreg_local_bb_info (bb_index); + + rtx_insn *insn; + df_ref def, use; + + /* Process the registers set in an exception handler. */ + FOR_EACH_ARTIFICIAL_DEF (def, bb_index) + if ((DF_REF_FLAGS (def) & DF_REF_AT_TOP) == 0) + { + add_subreg_range_to_def (local_bb_info, def); + remove_subreg_range (local_bb_info, def); + } + + /* Process the hardware registers that are always live. */ + FOR_EACH_ARTIFICIAL_USE (use, bb_index) + if ((DF_REF_FLAGS (use) & DF_REF_AT_TOP) == 0) + add_subreg_range_to_use (local_bb_info, use); + + FOR_BB_INSNS_REVERSE (bb, insn) + { + if (!NONDEBUG_INSN_P (insn)) + continue; + + df_insn_info *insn_info = DF_INSN_INFO_GET (insn); + FOR_EACH_INSN_INFO_DEF (def, insn_info) + { + remove_subreg_range (local_bb_info, def); + add_subreg_range_to_def (local_bb_info, def); + } + + FOR_EACH_INSN_INFO_USE (use, insn_info) + { + unsigned int regno = DF_REF_REGNO (use); + /* Ignore the use of subreg which is used as dest operand. */ + if (need_track_subreg_p (regno) + && DF_REF_FLAGS (use) & (DF_REF_READ_WRITE | DF_REF_SUBREG)) + continue; + add_subreg_range_to_use (local_bb_info, use); + } + } + + /* Process the registers set in an exception handler or the hard + frame pointer if this block is the target of a non local + goto. */ + FOR_EACH_ARTIFICIAL_DEF (def, bb_index) + if (DF_REF_FLAGS (def) & DF_REF_AT_TOP) + { + add_subreg_range_to_def (local_bb_info, def); + remove_subreg_range (local_bb_info, def); + } + +#ifdef EH_USES + /* Process the uses that are live into an exception handler. */ + FOR_EACH_ARTIFICIAL_USE (use, bb_index) + /* Add use to set of uses in this BB. */ + if (DF_REF_FLAGS (use) & DF_REF_AT_TOP) + add_subreg_range_to_use (local_bb_info, use); +#endif +} + +/* Compute local live register info for each basic block within BLOCKS. */ + +static void +df_live_subreg_local_compute (bitmap all_blocks ATTRIBUTE_UNUSED) +{ + unsigned int bb_index, i; + bitmap_iterator bi; + + bitmap_clear (&df->hardware_regs_used); + + /* The all-important stack pointer must always be live. */ + bitmap_set_bit (&df->hardware_regs_used, STACK_POINTER_REGNUM); + + /* Global regs are always live, too. */ + for (i = 0; i < FIRST_PSEUDO_REGISTER; i++) + if (global_regs[i]) + bitmap_set_bit (&df->hardware_regs_used, i); + + /* Before reload, there are a few registers that must be forced + live everywhere -- which might not already be the case for + blocks within infinite loops. */ + if (!reload_completed) + { + unsigned int pic_offset_table_regnum = PIC_OFFSET_TABLE_REGNUM; + /* Any reference to any pseudo before reload is a potential + reference of the frame pointer. */ + bitmap_set_bit (&df->hardware_regs_used, FRAME_POINTER_REGNUM); + + /* Pseudos with argument area equivalences may require + reloading via the argument pointer. */ + if (FRAME_POINTER_REGNUM != ARG_POINTER_REGNUM + && fixed_regs[ARG_POINTER_REGNUM]) + bitmap_set_bit (&df->hardware_regs_used, ARG_POINTER_REGNUM); + + /* Any constant, or pseudo with constant equivalences, may + require reloading from memory using the pic register. */ + if (pic_offset_table_regnum != INVALID_REGNUM + && fixed_regs[pic_offset_table_regnum]) + bitmap_set_bit (&df->hardware_regs_used, pic_offset_table_regnum); + } + + EXECUTE_IF_SET_IN_BITMAP (df_live_subreg->out_of_date_transfer_functions, 0, + bb_index, bi) + { + if (bb_index == EXIT_BLOCK) + { + /* The exit block is special for this problem and its bits are + computed from thin air. */ + df_live_subreg_local_bb_info *local_bb_info + = get_live_subreg_local_bb_info (EXIT_BLOCK); + bitmap_copy (&local_bb_info->full_use, df->exit_block_uses); + } + else + df_live_subreg_bb_local_compute (bb_index); + } + + bitmap_clear (df_live_subreg->out_of_date_transfer_functions); +} + +/* Initialize the solution vectors. */ + +static void +df_live_subreg_init (bitmap all_blocks) +{ + unsigned int bb_index; + bitmap_iterator bi; + + EXECUTE_IF_SET_IN_BITMAP (all_blocks, 0, bb_index, bi) + { + df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb_index); + df_live_subreg_local_bb_info *local_bb_info + = get_live_subreg_local_bb_info (bb_index); + bitmap_copy (&bb_info->full_in, &local_bb_info->full_use); + bitmap_copy (&bb_info->partial_in, &local_bb_info->partial_use); + bb_info->range_in->copy_lives (*local_bb_info->range_use); + bitmap_clear (&bb_info->full_out); + bitmap_clear (&bb_info->partial_out); + bb_info->range_out->clear (); + } +} + +/* Check that the status of the final result is correct. */ + +void +df_live_subreg_check_result (bitmap full, bitmap partial, + subregs_live *partial_live) +{ + unsigned int regno; + bitmap_iterator bi; + /* Make sure that full and partial do not overlap. */ + gcc_assert (!bitmap_intersect_p (full, partial)); + /* Ensure that registers belonging to full are not partially live. */ + EXECUTE_IF_SET_IN_BITMAP (full, 0, regno, bi) + gcc_assert (partial_live->empty_p (regno)); + /* Ensure that registers belonging to partial are partially live. */ + EXECUTE_IF_SET_IN_BITMAP (partial, 0, regno, bi) + gcc_assert (!partial_live->empty_p (regno)); +} + +/* Confluence function that processes infinite loops. This might be a + noreturn function that throws. And even if it isn't, getting the + unwind info right helps debugging. */ +static void +df_live_subreg_confluence_0 (basic_block bb) +{ + bitmap full_out = &df_live_subreg_get_bb_info (bb->index)->full_out; + if (bb != EXIT_BLOCK_PTR_FOR_FN (cfun)) + bitmap_copy (full_out, &df->hardware_regs_used); +} + +/* Confluence function that ignores fake edges. */ + +static bool +df_live_subreg_confluence_n (edge e) +{ + class df_live_subreg_bb_info *src_bb_info + = df_live_subreg_get_bb_info (e->src->index); + class df_live_subreg_bb_info *dest_bb_info + = df_live_subreg_get_bb_info (e->dest->index); + + bool changed = false; + + /* Call-clobbered registers die across exception and call edges. + Conservatively treat partially-clobbered registers as surviving + across the edges; they might or might not, depending on what + mode they have. */ + /* ??? Abnormal call edges ignored for the moment, as this gets + confused by sibling call edges, which crashes reg-stack. */ + if (e->flags & EDGE_EH) + { + bitmap_view eh_kills (eh_edge_abi.full_reg_clobbers ()); + changed = bitmap_ior_and_compl_into (&src_bb_info->full_out, + &dest_bb_info->full_in, eh_kills); + } + else + changed = bitmap_ior_into (&src_bb_info->full_out, &dest_bb_info->full_in); + + changed |= bitmap_ior_into (&src_bb_info->full_out, &df->hardware_regs_used); + + /* Handle partial live case. */ + if (!bitmap_empty_p (&dest_bb_info->partial_in)) + { + unsigned int regno; + bitmap_iterator bi; + EXECUTE_IF_SET_IN_BITMAP (&dest_bb_info->partial_in, + FIRST_PSEUDO_REGISTER, regno, bi) + { + sbitmap dest_range = dest_bb_info->range_in->get_range (regno); + changed |= src_bb_info->range_out->add_range (regno, dest_range); + } + changed |= bitmap_ior_into (&src_bb_info->partial_out, + &dest_bb_info->partial_in); + + df_live_subreg_check_result (&src_bb_info->full_out, + &src_bb_info->partial_out, + src_bb_info->range_out); + } + + return changed; +} + +/* Transfer function. IN = USE | (OUT & ~DEF). */ + +static bool +df_live_subreg_transfer_function (int bb_index) +{ + class df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb_index); + class df_live_subreg_local_bb_info *local_bb_info + = get_live_subreg_local_bb_info (bb_index); + + bool changed = false; + + changed + |= bitmap_ior_and_compl (&bb_info->full_in, &local_bb_info->full_use, + &bb_info->full_out, &local_bb_info->full_def); + + /* Handle partial live case. */ + if (!bitmap_empty_p (&bb_info->partial_out) + || !bitmap_empty_p (&local_bb_info->partial_use)) + { + unsigned int regno; + bitmap_iterator bi; + bitmap_head temp_partial_out; + subregs_live temp_range_out; + + /* TEMP = (OUT & ~DEF) */ + bitmap_initialize (&temp_partial_out, &bitmap_default_obstack); + EXECUTE_IF_SET_IN_BITMAP (&bb_info->partial_out, FIRST_PSEUDO_REGISTER, + regno, bi) + { + sbitmap out_range = bb_info->range_out->get_range (regno); + temp_range_out.add_range (regno, out_range); + if (bitmap_bit_p (&local_bb_info->partial_def, regno)) + { + sbitmap def_range = local_bb_info->range_def->get_range (regno); + temp_range_out.remove_range (regno, def_range); + if (!temp_range_out.empty_p (regno)) + bitmap_set_bit (&temp_partial_out, regno); + } + else + bitmap_set_bit (&temp_partial_out, regno); + } + + /* TEMP = USE | TEMP. */ + EXECUTE_IF_SET_IN_BITMAP (&local_bb_info->partial_use, + FIRST_PSEUDO_REGISTER, regno, bi) + { + sbitmap use_range = local_bb_info->range_use->get_range (regno); + temp_range_out.add_range (regno, use_range); + } + bitmap_ior_into (&temp_partial_out, &local_bb_info->partial_use); + + /* IN = TEMP. */ + changed |= bb_info->range_in->copy_lives (temp_range_out); + bitmap_copy (&bb_info->partial_in, &temp_partial_out); + df_live_subreg_check_result (&bb_info->full_in, &bb_info->partial_in, + bb_info->range_in); + } + else if (!bitmap_empty_p (&bb_info->partial_in)) + { + changed = true; + bitmap_clear (&bb_info->partial_in); + bb_info->range_in->clear (); + } + + return changed; +} + +/* Calculating ALL_IN from FULL_IN and PARTIAL_IN + and ALL_OUT from FULL_OUT and PARTIAL_OUT. */ + +void +df_live_subreg_finalize (bitmap all_blocks) +{ + unsigned int bb_index; + bitmap_iterator bi; + EXECUTE_IF_SET_IN_BITMAP (all_blocks, 0, bb_index, bi) + { + class df_live_subreg_bb_info *bb_info + = df_live_subreg_get_bb_info (bb_index); + gcc_assert (bb_info); + bitmap_ior (&bb_info->all_in, &bb_info->full_in, &bb_info->partial_in); + bitmap_ior (&bb_info->all_out, &bb_info->full_out, &bb_info->partial_out); + + /* Move full live reg in partial_in/partial_out to full_in/full_out. */ + unsigned int regno; + bitmap_iterator ri; + bool changed = false; + EXECUTE_IF_SET_IN_BITMAP (&bb_info->partial_in, FIRST_PSEUDO_REGISTER, + regno, ri) + { + if (bb_info->range_in->full_p (regno)) + { + bitmap_set_bit (&bb_info->full_in, regno); + bb_info->range_in->remove_range (regno); + changed = true; + } + } + if (changed) + bitmap_and_compl_into (&bb_info->partial_in, &bb_info->full_in); + + changed = false; + EXECUTE_IF_SET_IN_BITMAP (&bb_info->partial_out, FIRST_PSEUDO_REGISTER, + regno, ri) + { + if (bb_info->range_out->full_p (regno)) + { + bitmap_set_bit (&bb_info->full_out, regno); + bb_info->range_out->remove_range (regno); + changed = true; + } + } + if (changed) + bitmap_and_compl_into (&bb_info->partial_out, &bb_info->full_out); + } +} + +/* Free all storage associated with the problem. */ + +static void +df_live_subreg_free (void) +{ + df_live_subreg_problem_data *problem_data + = (df_live_subreg_problem_data *) df_live_subreg->problem_data; + if (df_live_subreg->block_info) + { + bitmap_obstack_release (&problem_data->live_subreg_bitmaps); + basic_block bb; + FOR_ALL_BB_FN (bb, cfun) + { + df_live_subreg_bb_info *bb_info + = df_live_subreg_get_bb_info (bb->index); + df_live_subreg_local_bb_info *local_bb_info + = get_live_subreg_local_bb_info (bb->index); + delete bb_info->range_in; + delete bb_info->range_out; + delete local_bb_info->range_def; + delete local_bb_info->range_use; + } + free (problem_data); + df_live_subreg->problem_data = NULL; + + df_live_subreg->block_info_size = 0; + free (df_live_subreg->block_info); + df_live_subreg->block_info = NULL; + } + + BITMAP_FREE (df_live_subreg->out_of_date_transfer_functions); + free (df_live_subreg); +} + +/* Debugging info at top of bb. */ + +static void +df_live_subreg_top_dump (basic_block bb, FILE *file) +{ + df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb->index); + df_live_subreg_local_bb_info *local_bb_info + = get_live_subreg_local_bb_info (bb->index); + + if (!bb_info) + return; + + fprintf (file, ";; subreg live all in \t"); + df_print_regset (file, &bb_info->all_in); + fprintf (file, ";; subreg live full in \t"); + df_print_regset (file, &bb_info->full_in); + fprintf (file, ";; subreg live partial in \t"); + df_print_regset (file, &bb_info->partial_in); + fprintf (file, ";; subreg live range in \t"); + bb_info->range_in->dump (file, ""); + + fprintf (file, "\n;; subreg live full use \t"); + df_print_regset (file, &local_bb_info->full_use); + fprintf (file, ";; subreg live partial use \t"); + df_print_regset (file, &local_bb_info->partial_use); + fprintf (file, ";; subreg live range use \t"); + local_bb_info->range_use->dump (file, ""); + + fprintf (file, "\n;; subreg live full def \t"); + df_print_regset (file, &local_bb_info->full_def); + fprintf (file, ";; subreg live partial def \t"); + df_print_regset (file, &local_bb_info->partial_def); + fprintf (file, ";; subreg live range def \t"); + local_bb_info->range_def->dump (file, ""); +} + +/* Debugging info at bottom of bb. */ + +static void +df_live_subreg_bottom_dump (basic_block bb, FILE *file) +{ + df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb->index); + if (!bb_info) + return; + + fprintf (file, ";; subreg live all out \t"); + df_print_regset (file, &bb_info->all_out); + fprintf (file, ";; subreg live full out \t"); + df_print_regset (file, &bb_info->full_out); + fprintf (file, ";; subreg live partial out \t"); + df_print_regset (file, &bb_info->partial_out); + fprintf (file, ";; subreg live range out \t"); + bb_info->range_out->dump (file, ""); +} + +/* All of the information associated with every instance of the problem. */ + +static const struct df_problem problem_LIVE_SUBREG = { + DF_LIVE_SUBREG, /* Problem id. */ + DF_BACKWARD, /* Direction. */ + df_live_subreg_alloc, /* Allocate the problem data. */ + df_live_subreg_reset, /* Reset global information. */ + df_live_subreg_free_bb_info, /* Free basic block info. */ + df_live_subreg_local_compute, /* Local compute function. */ + df_live_subreg_init, /* Init the solution data. */ + df_worklist_dataflow, /* Worklist solver. */ + df_live_subreg_confluence_0, /* Confluence operator 0. */ + df_live_subreg_confluence_n, /* Confluence operator n. */ + df_live_subreg_transfer_function, /* Transfer function. */ + df_live_subreg_finalize, /* Finalize function. */ + df_live_subreg_free, /* Free problem information. */ + df_live_subreg_free, /* Remove this problem from the stack + of dataflow problem. */ + NULL, /* Debugging. */ + df_live_subreg_top_dump, /* Debugging start block. */ + df_live_subreg_bottom_dump, /* Debugging end block. */ + NULL, /* Debugging start insn. */ + NULL, /* Debugging end insn. */ + NULL, /* Incremental solution start. */ + NULL, /* Incremental solution end. */ + &problem_LR, /* Dependent problem. */ + sizeof (df_live_subreg_bb_info), /* Size of entry of block_info array. */ + TV_DF_LIVE_SUBREG, /* Timing variable. */ + false /* Reset blocks on dropping out of blocks_to_analyze. */ +}; + +/* Create a new DATAFLOW instance and add it to an existing instance + of DF. The returned structure is what is used to get at the + solution. */ + +void +df_live_subreg_add_problem (void) +{ + gcc_assert (flag_track_subreg_liveness); + + df_add_problem (&problem_LIVE_SUBREG); + + /* These will be initialized when df_scan_blocks processes each + block. */ + df_live_subreg->out_of_date_transfer_functions + = BITMAP_ALLOC (&df_bitmap_obstack); +} - /*---------------------------------------------------------------------------- LIVE AND MAY-INITIALIZED REGISTERS. diff --git a/gcc/df.h b/gcc/df.h index 84e5aa8b524..7a3876dec8c 100644 --- a/gcc/df.h +++ b/gcc/df.h @@ -47,6 +47,7 @@ enum df_problem_id { DF_SCAN, DF_LR, /* Live Registers backward. */ + DF_LIVE_SUBREG, /* Live Regs and Subregs. */ DF_LIVE, /* Live Registers & Uninitialized Registers */ DF_RD, /* Reaching Defs. */ DF_CHAIN, /* Def-Use and/or Use-Def Chains. */ @@ -878,6 +879,33 @@ public: bitmap_head out; /* At the bottom of the block. */ }; +class subregs_live; /* Defined in subreg-range.{h,cc} */ + +/* Local live info in basic block. Use by live_subreg problem and LRA pass. */ +class df_live_subreg_local_bb_info +{ +public: + bitmap_head full_def; + bitmap_head full_use; + /* Only for pseudo registers. */ + bitmap_head partial_def; + bitmap_head partial_use; + subregs_live *range_def = nullptr; + subregs_live *range_use = nullptr; +}; + +/* Live in/out infos of each basic. */ +class df_live_subreg_bb_info : public df_live_subreg_local_bb_info +{ +public: + bitmap_head all_in, full_in; + bitmap_head all_out, full_out; + /* Only for pseudo registers. */ + bitmap_head partial_in; + bitmap_head partial_out; + subregs_live *range_in = nullptr; + subregs_live *range_out = nullptr; +}; /* Uninitialized registers. All bitmaps are referenced by the register number. Anded results of the forwards and backward live @@ -940,6 +968,7 @@ extern class df_d *df; #define df_scan (df->problems_by_index[DF_SCAN]) #define df_rd (df->problems_by_index[DF_RD]) #define df_lr (df->problems_by_index[DF_LR]) +#define df_live_subreg (df->problems_by_index[DF_LIVE_SUBREG]) #define df_live (df->problems_by_index[DF_LIVE]) #define df_chain (df->problems_by_index[DF_CHAIN]) #define df_word_lr (df->problems_by_index[DF_WORD_LR]) @@ -1031,6 +1060,8 @@ extern void df_lr_add_problem (void); extern void df_lr_verify_transfer_functions (void); extern void df_live_verify_transfer_functions (void); extern void df_live_add_problem (void); +extern void df_live_subreg_add_problem (); +extern void df_live_subreg_finalize (bitmap all_blocks); extern void df_live_set_all_dirty (void); extern void df_chain_add_problem (unsigned int); extern void df_word_lr_add_problem (void); @@ -1059,6 +1090,19 @@ extern bool can_move_insns_across (rtx_insn *, rtx_insn *, rtx_insn *, rtx_insn *, basic_block, regset, regset, rtx_insn **); +extern void +df_live_subreg_check_result (bitmap, bitmap, subregs_live *); +extern bool multireg_p (int); +extern void init_range (rtx, sbitmap); +extern void +add_subreg_range_to_def (df_live_subreg_local_bb_info *, unsigned int, + const_sbitmap); +extern void +add_subreg_range_to_use (df_live_subreg_local_bb_info *, unsigned int, + const_sbitmap); +extern void +remove_subreg_range (df_live_subreg_local_bb_info *, unsigned int, + const_sbitmap); /* Functions defined in df-scan.cc. */ extern void df_scan_alloc (bitmap); @@ -1192,6 +1236,121 @@ df_get_live_in (basic_block bb) return DF_LR_IN (bb); } +/* Get the subreg live at in set for BB. The live set include full and partial + * live. We only track and use subreg liveness when -ftrack-subreg-liveness, + * otherwise use DF_LR_IN. This function is used by the register allocators. + */ + +inline bitmap +df_get_subreg_live_in (basic_block bb) +{ + if (flag_track_subreg_liveness) + { + df_live_subreg_bb_info *bb_info = &( + (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index]; + return &(bb_info->all_in); + } + return DF_LR_IN (bb); +} + +/* Get the subreg live at out set for BB. The live set include full and + * partial live. We only track and use subreg liveness when + * -ftrack-subreg-liveness, otherwise use DF_LR_OUT. This function is used by + * the register allocators. */ + +inline bitmap +df_get_subreg_live_out (basic_block bb) +{ + if (flag_track_subreg_liveness) + { + df_live_subreg_bb_info *bb_info = &( + (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index]; + return &(bb_info->all_out); + } + return DF_LR_OUT (bb); +} + +/* Get the subreg live at in set for BB. The live set only include full and + * partial live. We only track and use subreg liveness when + * -ftrack-subreg-liveness, otherwise use DF_LR_OUT. This function is used by + * the register allocators. */ + +inline bitmap +df_get_subreg_live_full_in (basic_block bb) +{ + if (flag_track_subreg_liveness) + { + df_live_subreg_bb_info *bb_info = &( + (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index]; + return &(bb_info->full_in); + } + return DF_LR_IN (bb); +} + +inline bitmap +df_get_subreg_live_full_out (basic_block bb) +{ + if (flag_track_subreg_liveness) + { + df_live_subreg_bb_info *bb_info = &( + (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index]; + return &(bb_info->full_out); + } + return DF_LR_OUT (bb); +} + +/* Define in df-problems.cc, used when disable track-subreg-liveness. */ +extern bitmap_head df_subreg_empty_bitmap; +extern subregs_live df_subreg_empty_live; + +inline bitmap +df_get_subreg_live_partial_in (basic_block bb) +{ + if (flag_track_subreg_liveness) + { + df_live_subreg_bb_info *bb_info = &( + (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index]; + return &(bb_info->partial_in); + } + return &df_subreg_empty_bitmap; +} + +inline bitmap +df_get_subreg_live_partial_out (basic_block bb) +{ + if (flag_track_subreg_liveness) + { + df_live_subreg_bb_info *bb_info = &( + (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index]; + return &(bb_info->partial_out); + } + return &df_subreg_empty_bitmap; +} + +inline subregs_live * +df_get_subreg_live_range_in (basic_block bb) +{ + if (flag_track_subreg_liveness) + { + df_live_subreg_bb_info *bb_info = &( + (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index]; + return bb_info->range_in; + } + return &df_subreg_empty_live; +} + +inline subregs_live * +df_get_subreg_live_range_out (basic_block bb) +{ + if (flag_track_subreg_liveness) + { + df_live_subreg_bb_info *bb_info = &( + (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index]; + return bb_info->range_out; + } + return &df_subreg_empty_live; +} + /* Get basic block info. */ /* Get the artificial defs for a basic block. */ diff --git a/gcc/regs.h b/gcc/regs.h index 16189c58fd1..b0e9fd0c344 100644 --- a/gcc/regs.h +++ b/gcc/regs.h @@ -389,4 +389,9 @@ range_in_hard_reg_set_p (const_hard_reg_set set, unsigned regno, int nregs) return true; } +/* Return the number of blocks the MODE overlap. One block equal to mode's + natural size. */ +#define get_nblocks(mode) \ + (exact_div (GET_MODE_SIZE (mode), REGMODE_NATURAL_SIZE (mode)).to_constant ()) + #endif /* GCC_REGS_H */ diff --git a/gcc/sbitmap.cc b/gcc/sbitmap.cc index 03bb2c6d44b..9e35d6f6736 100644 --- a/gcc/sbitmap.cc +++ b/gcc/sbitmap.cc @@ -208,6 +208,29 @@ bitmap_empty_p (const_sbitmap bmap) return true; } +/* Return true if the bitmap is full, i.e. all bits are set. */ + +bool +bitmap_full_p (const_sbitmap bmap) +{ + unsigned int end_word = bmap->n_bits / SBITMAP_ELT_BITS; + unsigned int i = 0; + for (; i < end_word; i++) + if (bmap->elms[i] != (SBITMAP_ELT_TYPE) -1) + return false; + + unsigned int end_bitno = bmap->n_bits % SBITMAP_ELT_BITS; + + if (end_bitno == 0) + return true; + + gcc_checking_assert (i + 1 == bmap->size); + + SBITMAP_ELT_TYPE mask = ((SBITMAP_ELT_TYPE) 1 << end_bitno) - 1; + /* Make sure the tail bits are set. */ + return (bmap->elms[i] & mask) == mask; +} + /* Clear COUNT bits from START in BMAP. */ void @@ -662,6 +685,36 @@ bitmap_subset_p (const_sbitmap a, const_sbitmap b) return true; } +/* Return true if the bits in A and B are set the same and the number of bits is + the same. */ + +bool +bitmap_same_p (const_sbitmap a, const_sbitmap b) +{ + if (a->n_bits != b->n_bits) + return false; + + gcc_checking_assert (a->size == b->size); + + unsigned int end_word = a->n_bits / SBITMAP_ELT_BITS; + unsigned int i = 0; + for (; i < end_word; i++) + if (a->elms[i] != b->elms[i]) + return false; + + unsigned int end_bitno = a->n_bits % SBITMAP_ELT_BITS; + + if (end_bitno == 0) + return true; + + gcc_checking_assert (i + 1 == a->size); + + SBITMAP_ELT_TYPE mask = ((SBITMAP_ELT_TYPE) 1 << end_bitno) - 1; + + /* Make sure the tail bits are same. */ + return (a->elms[i] & mask) == (b->elms[i] & mask); +} + /* Set DST to be (A or (B and C)). Return nonzero if any change is made. */ @@ -994,6 +1047,49 @@ test_bit_in_range () sbitmap_free (s); } +/* Verify bitmap_full_p functions for sbitmap. */ + +static void +test_full () +{ + sbitmap s = sbitmap_alloc (193); + + bitmap_clear (s); + ASSERT_FALSE (bitmap_full_p (s)); + + bitmap_ones (s); + ASSERT_TRUE (bitmap_full_p (s)); + + bitmap_clear_bit (s, 192); + ASSERT_FALSE (bitmap_full_p (s)); + + bitmap_ones (s); + bitmap_clear_bit (s, 17); + ASSERT_FALSE (bitmap_full_p (s)); +} + +/* Verify bitmap_same_p functions for sbitmap. */ + +static void +test_same () +{ + sbitmap s1 = sbitmap_alloc (193); + sbitmap s2 = sbitmap_alloc (193); + sbitmap s3 = sbitmap_alloc (192); + + ASSERT_FALSE (bitmap_same_p (s1, s3)); + + bitmap_clear (s1); + bitmap_clear (s2); + ASSERT_TRUE (bitmap_same_p (s1, s2)); + + bitmap_set_bit (s2, 192); + ASSERT_FALSE (bitmap_same_p (s1, s2)); + + bitmap_set_bit (s1, 192); + ASSERT_TRUE (bitmap_same_p (s1, s2)); +} + /* Run all of the selftests within this file. */ void @@ -1001,6 +1097,8 @@ sbitmap_cc_tests () { test_set_range (); test_bit_in_range (); + test_full (); + test_same (); } } // namespace selftest diff --git a/gcc/sbitmap.h b/gcc/sbitmap.h index da6116ce925..71cfded9fb2 100644 --- a/gcc/sbitmap.h +++ b/gcc/sbitmap.h @@ -267,6 +267,7 @@ extern void bitmap_copy (sbitmap, const_sbitmap); extern bool bitmap_equal_p (const_sbitmap, const_sbitmap); extern unsigned int bitmap_count_bits (const_sbitmap); extern bool bitmap_empty_p (const_sbitmap); +extern bool bitmap_full_p (const_sbitmap); extern void bitmap_clear (sbitmap); extern void bitmap_clear_range (sbitmap, unsigned, unsigned); extern void bitmap_set_range (sbitmap, unsigned, unsigned); @@ -287,6 +288,7 @@ extern bool bitmap_and (sbitmap, const_sbitmap, const_sbitmap); extern bool bitmap_ior (sbitmap, const_sbitmap, const_sbitmap); extern bool bitmap_xor (sbitmap, const_sbitmap, const_sbitmap); extern bool bitmap_subset_p (const_sbitmap, const_sbitmap); +extern bool bitmap_same_p (const_sbitmap, const_sbitmap); extern bool bitmap_bit_in_range_p (const_sbitmap, unsigned int, unsigned int); extern int bitmap_first_set_bit (const_sbitmap); diff --git a/gcc/subreg-live-range.cc b/gcc/subreg-live-range.cc new file mode 100644 index 00000000000..4181463eefa --- /dev/null +++ b/gcc/subreg-live-range.cc @@ -0,0 +1,233 @@ +/* SUBREG liveness tracking classes for DF & IRA & LRA. + Copyright (C) 2024 Free Software Foundation, Inc. + Contributed by RiVAI Technologies Ltd. + + This file is part of GCC. + + GCC is free software; you can redistribute it and/or modify it + under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3, or (at your option) + any later version. + + GCC is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + + You should have received a copy of the GNU General Public License + along with GCC; see the file COPYING3. If not see + . */ + +#include "subreg-live-range.h" + +subregs_live::~subregs_live () +{ + for (auto &kv : lives) + sbitmap_free (kv.second); +} + +void +subregs_live::clear () +{ + for (auto &kv : lives) + sbitmap_free (kv.second); + lives.clear (); +} + +void +subregs_live::clear (size_t n) +{ + clear (); + lives.rehash (n); +} + +bool +subregs_live::find_p (unsigned int regno) const +{ + return lives.find (regno) != lives.end (); +} + +sbitmap +subregs_live::get_range (unsigned int regno) +{ + gcc_assert (find_p (regno)); + return lives.at (regno); +} + +const_sbitmap +subregs_live::get_range (unsigned int regno) const +{ + gcc_assert (find_p (regno)); + return lives.at (regno); +} + +/* Added RANGE to regno's ranges. Return true if leads to a change. */ +bool +subregs_live::add_range (unsigned int regno, const_sbitmap range) +{ + if (find_p (regno)) + { + sbitmap curr = get_range (regno); + gcc_assert (range->n_bits == curr->n_bits); + return bitmap_ior (curr, curr, range); + } + else + { + sbitmap a = sbitmap_alloc (range->n_bits); + lives.insert ({regno, a}); + sbitmap curr = get_range (regno); + bitmap_copy (curr, range); + return !bitmap_empty_p (range); + } +} + +/* Removed RANGE from regno's ranges. Return true if leads to a change. */ +bool +subregs_live::remove_range (unsigned int regno, const_sbitmap range) +{ + if (find_p (regno)) + { + sbitmap curr = get_range (regno); + bitmap_check_sizes (curr, range); + if (bitmap_subset_p (curr, range)) + return remove_range (regno); + + auto_sbitmap a (range->n_bits); + bitmap_not (a, range); + return bitmap_and (curr, curr, a); + } + return false; +} + +/* Removed the whole range of REGNO. Return true if leads to a change. */ +bool +subregs_live::remove_range (unsigned int regno) +{ + if (find_p (regno)) + { + sbitmap curr = get_range (regno); + bool changed = !bitmap_empty_p (curr); + sbitmap_free (curr); + lives.erase (regno); + return changed; + } + return false; +} + +/* Replace the range of REGNO with RANGE. Return true if leads to a change. + */ +bool +subregs_live::replace_range (unsigned int regno, const_sbitmap range) +{ + if (find_p (regno)) + { + sbitmap curr = get_range (regno); + if (!bitmap_same_p (curr, range)) + { + bitmap_copy (curr, range); + return true; + } + else + return false; + } + else + return add_range (regno, range); +} + +/* Copy subregs_live SL. Return true if leads to a change. */ +bool +subregs_live::copy_lives (const subregs_live &sl) +{ + bool changed = false; + for (auto &kv : sl.lives) + { + unsigned int regno = kv.first; + const_sbitmap range = kv.second; + if (bitmap_empty_p (range)) + continue; + if (find_p (regno)) + changed |= replace_range (regno, range); + else + changed |= add_range (regno, range); + } + + for (auto it = lives.cbegin (); it != lives.cend ();) + { + unsigned int regno = it->first; + auto prev_it = it; + it++; + if (sl.empty_p (regno)) + { + changed |= bitmap_empty_p (it->second); + lives.erase (prev_it); + } + } + + return changed; +} + +/* Added subregs_live SL. Return true if leads to a change. */ +bool +subregs_live::add_lives (const subregs_live &sl) +{ + bool changed = false; + for (auto &kv : sl.lives) + { + unsigned int regno = kv.first; + const_sbitmap range = kv.second; + if (find_p (regno)) + { + sbitmap curr = get_range (regno); + changed |= bitmap_ior (curr, curr, range); + } + else + changed |= add_range (regno, range); + } + return changed; +} + +/* Return true if regno's live range is full. */ +bool +subregs_live::full_p (unsigned int regno) const +{ + return find_p (regno) && bitmap_full_p (get_range (regno)); +} + +/* Return true if regno's live range is empty. */ +bool +subregs_live::empty_p (unsigned int regno) const +{ + return !find_p (regno) || bitmap_empty_p (get_range (regno)); +} + +void +subregs_live::dump (FILE *file, const char *indent) const +{ + if (lives.empty ()) + { + fprintf (file, "%sempty\n", indent); + return; + } + fprintf (file, "%s", indent); + for (auto &kv : lives) + { + const_sbitmap range = kv.second; + if (bitmap_empty_p (range)) + continue; + fprintf (file, "%d: ", kv.first); + if (!bitmap_full_p (range)) + { + dump_bitmap_file (file, range); + fprintf (file, ", "); + } + else + fprintf (file, "full, "); + } + fprintf (file, "\n"); +} + +DEBUG_FUNCTION void +debug (const subregs_live &l) +{ + l.dump (stderr, ""); +} diff --git a/gcc/subreg-live-range.h b/gcc/subreg-live-range.h new file mode 100644 index 00000000000..f51d20c43fe --- /dev/null +++ b/gcc/subreg-live-range.h @@ -0,0 +1,60 @@ +/* SUBREG liveness tracking classes for DF & IRA & LRA. + Copyright (C) 2024 Free Software Foundation, Inc. + Contributed by RiVAI Technologies Ltd. + + This file is part of GCC. + + GCC is free software; you can redistribute it and/or modify it + under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3, or (at your option) + any later version. + + GCC is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + + You should have received a copy of the GNU General Public License + along with GCC; see the file COPYING3. If not see + . */ + +#ifndef GCC_SUBREG_LIVE_RANGE_H +#define GCC_SUBREG_LIVE_RANGE_H + +#include "config.h" +#include "system.h" +#include "coretypes.h" +#include +#include "sbitmap.h" + +/* class subregs_live record the live subreg_ranges of registers. */ +class subregs_live +{ +public: + /* The key is usually the register's regno. */ + std::unordered_map lives; + + subregs_live () : lives () {} + + subregs_live (size_t n) : lives (n) {} + + ~subregs_live (); + void clear (); + void clear (size_t); + bool find_p (unsigned int) const; + sbitmap get_range (unsigned int); + const_sbitmap get_range (unsigned int) const; + bool add_range (unsigned int, const_sbitmap); + bool remove_range (unsigned int, const_sbitmap); + bool remove_range (unsigned int); + bool replace_range (unsigned int, const_sbitmap); + bool copy_lives (const subregs_live &); + bool add_lives (const subregs_live &); + bool full_p (unsigned int) const; + bool empty_p (unsigned int) const; + + /* Debug helper. */ + void dump (FILE *file, const char *indent = ";; ") const; +}; + +#endif /* GCC_SUBREG_LIVE_RANGE_H */ diff --git a/gcc/timevar.def b/gcc/timevar.def index 8e2168e0817..337a2b7225b 100644 --- a/gcc/timevar.def +++ b/gcc/timevar.def @@ -120,6 +120,7 @@ DEFTIMEVAR (TV_DF_SCAN , "df scan insns") DEFTIMEVAR (TV_DF_MD , "df multiple defs") DEFTIMEVAR (TV_DF_RD , "df reaching defs") DEFTIMEVAR (TV_DF_LR , "df live regs") +DEFTIMEVAR (TV_DF_LIVE_SUBREG , "df live subregs") DEFTIMEVAR (TV_DF_LIVE , "df live&initialized regs") DEFTIMEVAR (TV_DF_MIR , "df must-initialized regs") DEFTIMEVAR (TV_DF_CHAIN , "df use-def / def-use chains") From patchwork Sun May 12 22:57:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?6ZKf5bGF5ZOy?= X-Patchwork-Id: 1934353 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Vcykx17VXz20d6 for ; Mon, 13 May 2024 09:00:25 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 74D2D3860766 for ; Sun, 12 May 2024 23:00:23 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbguseast2.qq.com (smtpbguseast2.qq.com [54.204.34.130]) by sourceware.org (Postfix) with ESMTPS id 333483858C53 for ; Sun, 12 May 2024 22:58:12 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 333483858C53 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 333483858C53 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=54.204.34.130 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1715554696; cv=none; b=s8Zjw76NgL66tyCAFekQRP6FtOB7YsUgP+QKWzu0pS8cUGvO9MWL9ZZzk3EgqZzY8vqF4HDrDkNCHIgqbtRqLC3ut0ZuTEgEM0XY5fHJEQIYkZNaeUcfyaVo8/wuMJDiW5zVHs0+Idgcma87LscBeRnJv3gJnmtzhs53FEtHiEM= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1715554696; c=relaxed/simple; bh=bqxFXIGGCaS5tsIQHCIaJJB79onVHyC1IjvSxqpIt9g=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=Hgu/ngEbzE71kFLKUXdSJovPl0Z4/7RwzO1Ui2NzIkfo1Pk4E4JHJffRGXWqCB2Z8OqnXgOeQJ990n3M15yHlEh+wsWzxZWDbuq45cMSuav6k12KWxdbJhpNMPscNBab8AbLYg+e9RpHtdGJwu7WwrzOzHOoRWYLxdnXdiGmUV0= ARC-Authentication-Results: i=1; server2.sourceware.org X-QQ-mid: bizesmtpsz6t1715554673tuoln7d X-QQ-Originating-IP: 9oQebiybzNkdh2tky3GOI6OC2Ykw0LXO8sPriFoV2OA= Received: from rios-cad121.hadoop.rioslab.org ( [58.60.1.28]) by bizesmtp.qq.com (ESMTP) with id ; Mon, 13 May 2024 06:57:52 +0800 (CST) X-QQ-SSF: 01400000002000H0V000B00A0000000 X-QQ-FEAT: o819FCS+0JSW5iv4pITnj19sZWrN17W2nfEx06czjc+eThWvNlGDva4+jbTX4 5Y/RH8KTjhAYcQ/lg9usqlJZiS7lDJadB26CbEJ3+jOqDt48ZDPGpho8nGU3M43phSejP41 87JROjpdq0WQdRb7kRXJqghhz1w9TW4bn4cLP3206/3tBQOLG1QuqpF/knbf28ypgzvvYA+ ZJsCGQe6xO5Ms+CF9+j+FVitiU9+pQsCcmUYuqWtBA0kRhaKwJQEP+S65kyd/1rsS/vYb63 QSlh59j4o9qrjHRmZfs+0MOqOGu8eFN2rIouymnM1FApAN1yLGRIkm2I33me3lvpEaJG5nN kYvz4Tfb2vQKumy7R4LUfR19AXEVaXI0C4B+jDeaKCVrHyOXW/snorYZC/c1svnl+zz6uqm X-QQ-GoodBg: 2 X-BIZMAIL-ID: 6868734298631044390 From: Juzhe-Zhong To: gcc-patches@gcc.gnu.org Cc: vmakarov@redhat.com, richard.sandiford@arm.com, jin.xia@rivai.ai, dimitar@dinux.eu, Juzhe-Zhong Subject: [SUBREG V4 3/4] IRA: Apply DF_LIVE_SUBREG data Date: Mon, 13 May 2024 06:57:37 +0800 Message-Id: <20240512225738.528917-4-juzhe.zhong@rivai.ai> X-Mailer: git-send-email 2.36.3 In-Reply-To: <20240512225738.528917-1-juzhe.zhong@rivai.ai> References: <20240512225738.528917-1-juzhe.zhong@rivai.ai> MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtpsz:rivai.ai:qybglogicsvrgz:qybglogicsvrgz7a-one-0 X-Spam-Status: No, score=-9.5 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP, URIBL_BLACK autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org --- gcc/ira-build.cc | 7 ++++--- gcc/ira-color.cc | 8 ++++---- gcc/ira-emit.cc | 12 ++++++------ gcc/ira-lives.cc | 7 ++++--- gcc/ira.cc | 19 ++++++++++++------- 5 files changed, 30 insertions(+), 23 deletions(-) diff --git a/gcc/ira-build.cc b/gcc/ira-build.cc index ea593d5a087..283ff36d3dd 100644 --- a/gcc/ira-build.cc +++ b/gcc/ira-build.cc @@ -1921,7 +1921,8 @@ create_bb_allocnos (ira_loop_tree_node_t bb_node) create_insn_allocnos (PATTERN (insn), NULL, false); /* It might be a allocno living through from one subloop to another. */ - EXECUTE_IF_SET_IN_REG_SET (df_get_live_in (bb), FIRST_PSEUDO_REGISTER, i, bi) + EXECUTE_IF_SET_IN_REG_SET (df_get_subreg_live_in (bb), FIRST_PSEUDO_REGISTER, + i, bi) if (ira_curr_regno_allocno_map[i] == NULL) ira_create_allocno (i, false, ira_curr_loop_tree_node); } @@ -1937,9 +1938,9 @@ create_loop_allocnos (edge e) bitmap_iterator bi; ira_loop_tree_node_t parent; - live_in_regs = df_get_live_in (e->dest); + live_in_regs = df_get_subreg_live_in (e->dest); border_allocnos = ira_curr_loop_tree_node->border_allocnos; - EXECUTE_IF_SET_IN_REG_SET (df_get_live_out (e->src), + EXECUTE_IF_SET_IN_REG_SET (df_get_subreg_live_out (e->src), FIRST_PSEUDO_REGISTER, i, bi) if (bitmap_bit_p (live_in_regs, i)) { diff --git a/gcc/ira-color.cc b/gcc/ira-color.cc index b9ae32d1b4d..bfebc48ef83 100644 --- a/gcc/ira-color.cc +++ b/gcc/ira-color.cc @@ -2786,8 +2786,8 @@ ira_loop_edge_freq (ira_loop_tree_node_t loop_node, int regno, bool exit_p) FOR_EACH_EDGE (e, ei, loop_node->loop->header->preds) if (e->src != loop_node->loop->latch && (regno < 0 - || (bitmap_bit_p (df_get_live_out (e->src), regno) - && bitmap_bit_p (df_get_live_in (e->dest), regno)))) + || (bitmap_bit_p (df_get_subreg_live_out (e->src), regno) + && bitmap_bit_p (df_get_subreg_live_in (e->dest), regno)))) freq += EDGE_FREQUENCY (e); } else @@ -2795,8 +2795,8 @@ ira_loop_edge_freq (ira_loop_tree_node_t loop_node, int regno, bool exit_p) auto_vec edges = get_loop_exit_edges (loop_node->loop); FOR_EACH_VEC_ELT (edges, i, e) if (regno < 0 - || (bitmap_bit_p (df_get_live_out (e->src), regno) - && bitmap_bit_p (df_get_live_in (e->dest), regno))) + || (bitmap_bit_p (df_get_subreg_live_out (e->src), regno) + && bitmap_bit_p (df_get_subreg_live_in (e->dest), regno))) freq += EDGE_FREQUENCY (e); } diff --git a/gcc/ira-emit.cc b/gcc/ira-emit.cc index d347f11fa02..8075b082e36 100644 --- a/gcc/ira-emit.cc +++ b/gcc/ira-emit.cc @@ -510,8 +510,8 @@ generate_edge_moves (edge e) return; src_map = src_loop_node->regno_allocno_map; dest_map = dest_loop_node->regno_allocno_map; - regs_live_in_dest = df_get_live_in (e->dest); - regs_live_out_src = df_get_live_out (e->src); + regs_live_in_dest = df_get_subreg_live_in (e->dest); + regs_live_out_src = df_get_subreg_live_out (e->src); EXECUTE_IF_SET_IN_REG_SET (regs_live_in_dest, FIRST_PSEUDO_REGISTER, regno, bi) if (bitmap_bit_p (regs_live_out_src, regno)) @@ -1229,16 +1229,16 @@ add_ranges_and_copies (void) destination block) to use for searching allocnos by their regnos because of subsequent IR flattening. */ node = IRA_BB_NODE (bb)->parent; - bitmap_copy (live_through, df_get_live_in (bb)); + bitmap_copy (live_through, df_get_subreg_live_in (bb)); add_range_and_copies_from_move_list (at_bb_start[bb->index], node, live_through, REG_FREQ_FROM_BB (bb)); - bitmap_copy (live_through, df_get_live_out (bb)); + bitmap_copy (live_through, df_get_subreg_live_out (bb)); add_range_and_copies_from_move_list (at_bb_end[bb->index], node, live_through, REG_FREQ_FROM_BB (bb)); FOR_EACH_EDGE (e, ei, bb->succs) { - bitmap_and (live_through, - df_get_live_in (e->dest), df_get_live_out (bb)); + bitmap_and (live_through, df_get_subreg_live_in (e->dest), + df_get_subreg_live_out (bb)); add_range_and_copies_from_move_list ((move_t) e->aux, node, live_through, REG_FREQ_FROM_EDGE_FREQ (EDGE_FREQUENCY (e))); diff --git a/gcc/ira-lives.cc b/gcc/ira-lives.cc index e07d3dc3e89..7641184069d 100644 --- a/gcc/ira-lives.cc +++ b/gcc/ira-lives.cc @@ -1254,7 +1254,8 @@ process_out_of_region_eh_regs (basic_block bb) if (! eh_p) return; - EXECUTE_IF_SET_IN_BITMAP (df_get_live_out (bb), FIRST_PSEUDO_REGISTER, i, bi) + EXECUTE_IF_SET_IN_BITMAP (df_get_subreg_live_out (bb), FIRST_PSEUDO_REGISTER, + i, bi) { ira_allocno_t a = ira_curr_regno_allocno_map[i]; for (int n = ALLOCNO_NUM_OBJECTS (a) - 1; n >= 0; n--) @@ -1288,7 +1289,7 @@ add_conflict_from_region_landing_pads (eh_region region, ira_object_t obj, if ((landing_label = lp->landing_pad) != NULL && (landing_bb = BLOCK_FOR_INSN (landing_label)) != NULL && (region->type != ERT_CLEANUP - || bitmap_bit_p (df_get_live_in (landing_bb), + || bitmap_bit_p (df_get_subreg_live_in (landing_bb), ALLOCNO_REGNO (a)))) { HARD_REG_SET new_conflict_regs @@ -1325,7 +1326,7 @@ process_bb_node_lives (ira_loop_tree_node_t loop_tree_node) high_pressure_start_point[ira_pressure_classes[i]] = -1; } curr_bb_node = loop_tree_node; - reg_live_out = df_get_live_out (bb); + reg_live_out = df_get_subreg_live_out (bb); sparseset_clear (objects_live); REG_SET_TO_HARD_REG_SET (hard_regs_live, reg_live_out); hard_regs_live &= ~(eliminable_regset | ira_no_alloc_regs); diff --git a/gcc/ira.cc b/gcc/ira.cc index 5642aea3caa..bebf6bb3e9b 100644 --- a/gcc/ira.cc +++ b/gcc/ira.cc @@ -4735,8 +4735,8 @@ find_moveable_pseudos (void) bitmap_initialize (local, 0); bitmap_initialize (transp, 0); bitmap_initialize (moveable, 0); - bitmap_copy (live, df_get_live_out (bb)); - bitmap_and_into (live, df_get_live_in (bb)); + bitmap_copy (live, df_get_subreg_live_out (bb)); + bitmap_and_into (live, df_get_subreg_live_in (bb)); bitmap_copy (transp, live); bitmap_clear (moveable); bitmap_clear (live); @@ -5036,7 +5036,8 @@ interesting_dest_for_shprep_1 (rtx set, basic_block call_dom) rtx dest = SET_DEST (set); if (!REG_P (src) || !HARD_REGISTER_P (src) || !REG_P (dest) || HARD_REGISTER_P (dest) - || (call_dom && !bitmap_bit_p (df_get_live_in (call_dom), REGNO (dest)))) + || (call_dom + && !bitmap_bit_p (df_get_subreg_live_in (call_dom), REGNO (dest)))) return NULL; return dest; } @@ -5514,10 +5515,12 @@ allocate_initial_values (void) /* Update global register liveness information. */ FOR_EACH_BB_FN (bb, cfun) { - if (REGNO_REG_SET_P (df_get_live_in (bb), regno)) - SET_REGNO_REG_SET (df_get_live_in (bb), new_regno); - if (REGNO_REG_SET_P (df_get_live_out (bb), regno)) - SET_REGNO_REG_SET (df_get_live_out (bb), new_regno); + if (REGNO_REG_SET_P (df_get_subreg_live_in (bb), regno)) + SET_REGNO_REG_SET (df_get_subreg_live_in (bb), + new_regno); + if (REGNO_REG_SET_P (df_get_subreg_live_out (bb), regno)) + SET_REGNO_REG_SET (df_get_subreg_live_out (bb), + new_regno); } } } @@ -5679,6 +5682,8 @@ ira (FILE *f) if (optimize > 1) df_remove_problem (df_live); gcc_checking_assert (df_live == NULL); + if (flag_track_subreg_liveness) + df_live_subreg_add_problem (); if (flag_checking) df->changeable_flags |= DF_VERIFY_SCHEDULED; From patchwork Sun May 12 22:57:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?6ZKf5bGF5ZOy?= X-Patchwork-Id: 1934350 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VcyjB6g01z1yfq for ; Mon, 13 May 2024 08:58:54 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 29C6D3858D3C for ; Sun, 12 May 2024 22:58:52 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbguseast1.qq.com (smtpbguseast1.qq.com [54.204.34.129]) by sourceware.org (Postfix) with ESMTPS id BB107385841C for ; Sun, 12 May 2024 22:58:21 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org BB107385841C Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai ARC-Filter: OpenARC Filter v1.0.0 sourceware.org BB107385841C Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=54.204.34.129 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1715554707; cv=none; b=dqnxk245Hm+bCR+DuiTE4JlC94FE6gNG48vioVRLmPSfUvZSAC6jqyU9Zd+7096TftAPqIkXCI/e39IQSjLAff2FjMJ3Zlk7MbpapkJrZoozpomKzsc3PxTAKQTcbCJkmEKeczbB6rbnWHEw5B9UvYPDPyRfJQ+cr9qg4hXmBpY= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1715554707; c=relaxed/simple; bh=ahMA2nCgE18WCNqq6nA+k2/PV8Scw50qcg66EHXYSZQ=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=Wtwaj+Ip45/yH/70RO0yuZE8PW66JN19vMpimF4fVWzBI2RgXGmqWnUJlMZKRefIwJyDnAYzRvSKyLhTkCbx784+8ItP+WpSKrEEVLeRwp9qNuSpwfoURYux+CVE/4bNhx+XLSbh6KaYuIDXjj0nL2outKikTfsVh34BD0xpfMI= ARC-Authentication-Results: i=1; server2.sourceware.org X-QQ-mid: bizesmtpsz6t1715554677tix6pnc X-QQ-Originating-IP: ySUCLxz6wgjU8kieZo2p2n4Pzh+vB6uraFrfIDK8xa4= Received: from rios-cad121.hadoop.rioslab.org ( [58.60.1.28]) by bizesmtp.qq.com (ESMTP) with id ; Mon, 13 May 2024 06:57:56 +0800 (CST) X-QQ-SSF: 01400000002000H0V000B00A0000000 X-QQ-FEAT: 0AHoP94G71XrfcxgByxieKKsyO9lsR1cJ55zGZpgcszGNzyuQJ47SZh2nCTUJ wj3G/no5VcPZ4+L0189YbYEF8IJAXPlrAtcFlsSezM/evcg7xUpjfLKXtjFYXKbjF63RjwR pQOfRq0afqMSyokurCspRDjvE+1VV/do2rDPlwAtSZJqUVg2Kje6YRE8LLOEDPI6Mjev2Ou VvwXv1K/1Ws1BOU8xskNAup4gx2BkR16DWw7qiXOPmXi7YPc9lmwQsyGMAYDJ5qHtoTFiE4 EW2DPbRoKoutnIYxghx8iT+hRBDVP2R4SuxTZ6nQDy5agZaOBA/vhiHBiCxdsEOcZpcyYHG yT+3l0Xi4Vn6H+e16tsqdteDb6OGTr3jZFtVEy091VzHKu3ELLCT54dgoPw7vxHYVdjCRP9 hFR+dSLG498= X-QQ-GoodBg: 2 X-BIZMAIL-ID: 17939451675758472568 From: Juzhe-Zhong To: gcc-patches@gcc.gnu.org Cc: vmakarov@redhat.com, richard.sandiford@arm.com, jin.xia@rivai.ai, dimitar@dinux.eu, Juzhe-Zhong Subject: [SUBREG V4 4/4] LRA: Apply DF_LIVE_SUBREG data Date: Mon, 13 May 2024 06:57:38 +0800 Message-Id: <20240512225738.528917-5-juzhe.zhong@rivai.ai> X-Mailer: git-send-email 2.36.3 In-Reply-To: <20240512225738.528917-1-juzhe.zhong@rivai.ai> References: <20240512225738.528917-1-juzhe.zhong@rivai.ai> MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtpsz:rivai.ai:qybglogicsvrgz:qybglogicsvrgz7a-one-0 X-Spam-Status: No, score=-10.3 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org --- gcc/lra-coalesce.cc | 27 +++- gcc/lra-constraints.cc | 109 ++++++++++--- gcc/lra-int.h | 4 + gcc/lra-lives.cc | 357 ++++++++++++++++++++++++++++++++--------- gcc/lra-remat.cc | 8 +- gcc/lra-spills.cc | 27 +++- gcc/lra.cc | 10 +- 7 files changed, 430 insertions(+), 112 deletions(-) diff --git a/gcc/lra-coalesce.cc b/gcc/lra-coalesce.cc index a9b5b51cb3f..9416775a009 100644 --- a/gcc/lra-coalesce.cc +++ b/gcc/lra-coalesce.cc @@ -186,19 +186,28 @@ static bitmap_head used_pseudos_bitmap; /* Set up USED_PSEUDOS_BITMAP, and update LR_BITMAP (a BB live info bitmap). */ static void -update_live_info (bitmap lr_bitmap) +update_live_info (bitmap all, bitmap full, bitmap partial) { unsigned int j; bitmap_iterator bi; bitmap_clear (&used_pseudos_bitmap); - EXECUTE_IF_AND_IN_BITMAP (&coalesced_pseudos_bitmap, lr_bitmap, + EXECUTE_IF_AND_IN_BITMAP (&coalesced_pseudos_bitmap, all, FIRST_PSEUDO_REGISTER, j, bi) bitmap_set_bit (&used_pseudos_bitmap, first_coalesced_pseudo[j]); - if (! bitmap_empty_p (&used_pseudos_bitmap)) + if (!bitmap_empty_p (&used_pseudos_bitmap)) { - bitmap_and_compl_into (lr_bitmap, &coalesced_pseudos_bitmap); - bitmap_ior_into (lr_bitmap, &used_pseudos_bitmap); + bitmap_and_compl_into (all, &coalesced_pseudos_bitmap); + bitmap_ior_into (all, &used_pseudos_bitmap); + + if (flag_track_subreg_liveness) + { + bitmap_and_compl_into (full, &coalesced_pseudos_bitmap); + bitmap_ior_and_compl_into (full, &used_pseudos_bitmap, partial); + + bitmap_and_compl_into (partial, &coalesced_pseudos_bitmap); + bitmap_ior_and_compl_into (partial, &used_pseudos_bitmap, full); + } } } @@ -301,8 +310,12 @@ lra_coalesce (void) bitmap_initialize (&used_pseudos_bitmap, ®_obstack); FOR_EACH_BB_FN (bb, cfun) { - update_live_info (df_get_live_in (bb)); - update_live_info (df_get_live_out (bb)); + update_live_info (df_get_subreg_live_in (bb), + df_get_subreg_live_full_in (bb), + df_get_subreg_live_partial_in (bb)); + update_live_info (df_get_subreg_live_out (bb), + df_get_subreg_live_full_out (bb), + df_get_subreg_live_partial_out (bb)); FOR_BB_INSNS_SAFE (bb, insn, next) if (INSN_P (insn) && bitmap_bit_p (&involved_insns_bitmap, INSN_UID (insn))) diff --git a/gcc/lra-constraints.cc b/gcc/lra-constraints.cc index e945a4da451..c9246e6be58 100644 --- a/gcc/lra-constraints.cc +++ b/gcc/lra-constraints.cc @@ -6565,34 +6565,86 @@ update_ebb_live_info (rtx_insn *head, rtx_insn *tail) { if (prev_bb != NULL) { - /* Update df_get_live_in (prev_bb): */ + /* Update subreg live (prev_bb): */ + bitmap subreg_all_in = df_get_subreg_live_in (prev_bb); + bitmap subreg_full_in = df_get_subreg_live_full_in (prev_bb); + bitmap subreg_partial_in = df_get_subreg_live_partial_in (prev_bb); + subregs_live *range_in = df_get_subreg_live_range_in (prev_bb); EXECUTE_IF_SET_IN_BITMAP (&check_only_regs, 0, j, bi) if (bitmap_bit_p (&live_regs, j)) - bitmap_set_bit (df_get_live_in (prev_bb), j); - else - bitmap_clear_bit (df_get_live_in (prev_bb), j); + { + bitmap_set_bit (subreg_all_in, j); + if (flag_track_subreg_liveness) + { + bitmap_set_bit (subreg_full_in, j); + if (bitmap_bit_p (subreg_partial_in, j)) + { + bitmap_clear_bit (subreg_partial_in, j); + range_in->remove_range (j); + } + } + } + else if (bitmap_bit_p (subreg_all_in, j)) + { + bitmap_clear_bit (subreg_all_in, j); + if (flag_track_subreg_liveness) + { + bitmap_clear_bit (subreg_full_in, j); + if (bitmap_bit_p (subreg_partial_in, j)) + { + bitmap_clear_bit (subreg_partial_in, j); + range_in->remove_range (j); + } + } + } } + bitmap subreg_all_out = df_get_subreg_live_out (curr_bb); if (curr_bb != last_bb) { - /* Update df_get_live_out (curr_bb): */ + /* Update subreg live (curr_bb): */ + bitmap subreg_full_out = df_get_subreg_live_full_out (curr_bb); + bitmap subreg_partial_out = df_get_subreg_live_partial_out (curr_bb); + subregs_live *range_out = df_get_subreg_live_range_out (curr_bb); EXECUTE_IF_SET_IN_BITMAP (&check_only_regs, 0, j, bi) { live_p = bitmap_bit_p (&live_regs, j); if (! live_p) FOR_EACH_EDGE (e, ei, curr_bb->succs) - if (bitmap_bit_p (df_get_live_in (e->dest), j)) + if (bitmap_bit_p (df_get_subreg_live_in (e->dest), j)) { live_p = true; break; } if (live_p) - bitmap_set_bit (df_get_live_out (curr_bb), j); - else - bitmap_clear_bit (df_get_live_out (curr_bb), j); + { + bitmap_set_bit (subreg_all_out, j); + if (flag_track_subreg_liveness) + { + bitmap_set_bit (subreg_full_out, j); + if (bitmap_bit_p (subreg_partial_out, j)) + { + bitmap_clear_bit (subreg_partial_out, j); + range_out->remove_range (j); + } + } + } + else if (bitmap_bit_p (subreg_all_out, j)) + { + bitmap_clear_bit (subreg_all_out, j); + if (flag_track_subreg_liveness) + { + bitmap_clear_bit (subreg_full_out, j); + if (bitmap_bit_p (subreg_partial_out, j)) + { + bitmap_clear_bit (subreg_partial_out, j); + range_out->remove_range (j); + } + } + } } } prev_bb = curr_bb; - bitmap_and (&live_regs, &check_only_regs, df_get_live_out (curr_bb)); + bitmap_and (&live_regs, &check_only_regs, subreg_all_out); } if (! NONDEBUG_INSN_P (curr_insn)) continue; @@ -6709,7 +6761,7 @@ get_live_on_other_edges (basic_block from, basic_block to, bitmap res) bitmap_clear (res); FOR_EACH_EDGE (e, ei, from->succs) if (e->dest != to) - bitmap_ior_into (res, df_get_live_in (e->dest)); + bitmap_ior_into (res, df_get_subreg_live_in (e->dest)); last = get_last_insertion_point (from); if (! JUMP_P (last)) return; @@ -6781,7 +6833,7 @@ inherit_in_ebb (rtx_insn *head, rtx_insn *tail) { /* We are at the end of BB. Add qualified living pseudos for potential splitting. */ - to_process = df_get_live_out (curr_bb); + to_process = df_get_subreg_live_out (curr_bb); if (last_processed_bb != NULL) { /* We are somewhere in the middle of EBB. */ @@ -7153,7 +7205,7 @@ inherit_in_ebb (rtx_insn *head, rtx_insn *tail) { /* We reached the beginning of the current block -- do rest of spliting in the current BB. */ - to_process = df_get_live_in (curr_bb); + to_process = df_get_subreg_live_in (curr_bb); if (BLOCK_FOR_INSN (head) != curr_bb) { /* We are somewhere in the middle of EBB. */ @@ -7230,7 +7282,7 @@ lra_inheritance (void) fprintf (lra_dump_file, "EBB"); /* Form a EBB starting with BB. */ bitmap_clear (&ebb_global_regs); - bitmap_ior_into (&ebb_global_regs, df_get_live_in (bb)); + bitmap_ior_into (&ebb_global_regs, df_get_subreg_live_in (bb)); for (;;) { if (lra_dump_file != NULL) @@ -7246,7 +7298,7 @@ lra_inheritance (void) break; bb = bb->next_bb; } - bitmap_ior_into (&ebb_global_regs, df_get_live_out (bb)); + bitmap_ior_into (&ebb_global_regs, df_get_subreg_live_out (bb)); if (lra_dump_file != NULL) fprintf (lra_dump_file, "\n"); if (inherit_in_ebb (BB_HEAD (start_bb), BB_END (bb))) @@ -7275,15 +7327,26 @@ int lra_undo_inheritance_iter; /* Fix BB live info LIVE after removing pseudos created on pass doing inheritance/split which are REMOVED_PSEUDOS. */ static void -fix_bb_live_info (bitmap live, bitmap removed_pseudos) +fix_bb_live_info (bitmap all, bitmap full, bitmap partial, + bitmap removed_pseudos) { unsigned int regno; bitmap_iterator bi; EXECUTE_IF_SET_IN_BITMAP (removed_pseudos, 0, regno, bi) - if (bitmap_clear_bit (live, regno) - && REG_P (lra_reg_info[regno].restore_rtx)) - bitmap_set_bit (live, REGNO (lra_reg_info[regno].restore_rtx)); + { + if (bitmap_clear_bit (all, regno) + && REG_P (lra_reg_info[regno].restore_rtx)) + { + bitmap_set_bit (all, REGNO (lra_reg_info[regno].restore_rtx)); + if (flag_track_subreg_liveness) + { + bitmap_clear_bit (full, regno); + bitmap_set_bit (full, REGNO (lra_reg_info[regno].restore_rtx)); + gcc_assert (!bitmap_bit_p (partial, regno)); + } + } + } } /* Return regno of the (subreg of) REG. Otherwise, return a negative @@ -7349,8 +7412,12 @@ remove_inheritance_pseudos (bitmap remove_pseudos) constraint pass. */ FOR_EACH_BB_FN (bb, cfun) { - fix_bb_live_info (df_get_live_in (bb), remove_pseudos); - fix_bb_live_info (df_get_live_out (bb), remove_pseudos); + fix_bb_live_info (df_get_subreg_live_in (bb), + df_get_subreg_live_full_in (bb), + df_get_subreg_live_partial_in (bb), remove_pseudos); + fix_bb_live_info (df_get_subreg_live_out (bb), + df_get_subreg_live_full_out (bb), + df_get_subreg_live_partial_out (bb), remove_pseudos); FOR_BB_INSNS_REVERSE (bb, curr_insn) { if (! INSN_P (curr_insn)) diff --git a/gcc/lra-int.h b/gcc/lra-int.h index 5f605c3ae41..d73ff4e31b5 100644 --- a/gcc/lra-int.h +++ b/gcc/lra-int.h @@ -21,6 +21,8 @@ along with GCC; see the file COPYING3. If not see #ifndef GCC_LRA_INT_H #define GCC_LRA_INT_H +#include "subreg-live-range.h" + #define lra_assert(c) gcc_checking_assert (c) /* The parameter used to prevent infinite reloading for an insn. Each @@ -161,6 +163,8 @@ struct lra_insn_reg int regno; /* Next reg info of the same insn. */ struct lra_insn_reg *next; + /* The corresponding reg or subreg RTX. */ + rtx op; }; /* Static part (common info for insns with the same ICODE) of LRA diff --git a/gcc/lra-lives.cc b/gcc/lra-lives.cc index 66c6577e5d6..27897fdd3e1 100644 --- a/gcc/lra-lives.cc +++ b/gcc/lra-lives.cc @@ -272,8 +272,34 @@ update_pseudo_point (int regno, int point, enum point_type type) } } -/* The corresponding bitmaps of BB currently being processed. */ -static bitmap bb_killed_pseudos, bb_gen_pseudos; +/* Structure describing local BB data used for pseudo + live-analysis. */ +class bb_data_pseudos : public df_live_subreg_local_bb_info +{ +public: + /* Basic block about which the below data are. */ + basic_block bb; +}; + +/* Array for all BB data. Indexed by the corresponding BB index. */ +typedef class bb_data_pseudos *bb_data_t; + +/* All basic block data are referred through the following array. */ +static bb_data_t bb_data; + +/* The corresponding df local data of BB currently being processed. */ +static bb_data_t curr_bb_info; + +/* The regs which need to be tracked it's subreg liveness. */ +static bitmap_head tracked_regs; + +/* Return true if the REGNO need be track with subreg liveness. */ + +static bool +need_track_subreg_p (unsigned regno) +{ + return bitmap_bit_p (&tracked_regs, regno); +} /* Record hard register REGNO as now being live. It updates living hard regs and START_LIVING. */ @@ -287,7 +313,7 @@ make_hard_regno_live (int regno) SET_HARD_REG_BIT (hard_regs_live, regno); sparseset_set_bit (start_living, regno); if (fixed_regs[regno] || TEST_HARD_REG_BIT (hard_regs_spilled_into, regno)) - bitmap_set_bit (bb_gen_pseudos, regno); + bitmap_set_bit (&curr_bb_info->full_use, regno); } /* Process the definition of hard register REGNO. This updates @@ -310,8 +336,8 @@ make_hard_regno_dead (int regno) sparseset_set_bit (start_dying, regno); if (fixed_regs[regno] || TEST_HARD_REG_BIT (hard_regs_spilled_into, regno)) { - bitmap_clear_bit (bb_gen_pseudos, regno); - bitmap_set_bit (bb_killed_pseudos, regno); + bitmap_clear_bit (&curr_bb_info->full_use, regno); + bitmap_set_bit (&curr_bb_info->full_def, regno); } } @@ -343,7 +369,7 @@ mark_pseudo_dead (int regno) /* Mark register REGNO (pseudo or hard register) in MODE as being live and update BB_GEN_PSEUDOS. */ static void -mark_regno_live (int regno, machine_mode mode) +mark_regno_live (int regno, machine_mode mode, struct lra_insn_reg *reg) { int last; @@ -355,15 +381,23 @@ mark_regno_live (int regno, machine_mode mode) else { mark_pseudo_live (regno); - bitmap_set_bit (bb_gen_pseudos, regno); + if (!need_track_subreg_p (regno)) + bitmap_set_bit (&curr_bb_info->full_use, regno); + else + { + machine_mode reg_mode + = GET_MODE (SUBREG_P (reg->op) ? SUBREG_REG (reg->op) : reg->op); + auto_sbitmap range (get_nblocks (reg_mode)); + init_range (reg->op, range); + add_subreg_range_to_use (curr_bb_info, regno, range); + } } } - /* Mark register REGNO (pseudo or hard register) in MODE as being dead and update BB_GEN_PSEUDOS and BB_KILLED_PSEUDOS. */ static void -mark_regno_dead (int regno, machine_mode mode) +mark_regno_dead (int regno, machine_mode mode, struct lra_insn_reg *reg) { int last; @@ -375,8 +409,20 @@ mark_regno_dead (int regno, machine_mode mode) else { mark_pseudo_dead (regno); - bitmap_clear_bit (bb_gen_pseudos, regno); - bitmap_set_bit (bb_killed_pseudos, regno); + if (!need_track_subreg_p (regno)) + { + bitmap_clear_bit (&curr_bb_info->full_use, regno); + bitmap_set_bit (&curr_bb_info->full_def, regno); + } + else + { + machine_mode reg_mode + = GET_MODE (SUBREG_P (reg->op) ? SUBREG_REG (reg->op) : reg->op); + auto_sbitmap range (get_nblocks (reg_mode)); + init_range (reg->op, range); + remove_subreg_range (curr_bb_info, regno, range); + add_subreg_range_to_def (curr_bb_info, regno, range); + } } } @@ -387,23 +433,6 @@ mark_regno_dead (int regno, machine_mode mode) border. That might be a consequence of some global transformations in LRA, e.g. PIC pseudo reuse or rematerialization. */ -/* Structure describing local BB data used for pseudo - live-analysis. */ -class bb_data_pseudos -{ -public: - /* Basic block about which the below data are. */ - basic_block bb; - bitmap_head killed_pseudos; /* pseudos killed in the BB. */ - bitmap_head gen_pseudos; /* pseudos generated in the BB. */ -}; - -/* Array for all BB data. Indexed by the corresponding BB index. */ -typedef class bb_data_pseudos *bb_data_t; - -/* All basic block data are referred through the following array. */ -static bb_data_t bb_data; - /* Two small functions for access to the bb data. */ static inline bb_data_t get_bb_data (basic_block bb) @@ -429,14 +458,73 @@ static bitmap_head all_hard_regs_bitmap; static bool live_trans_fun (int bb_index) { - basic_block bb = get_bb_data_by_index (bb_index)->bb; - bitmap bb_liveout = df_get_live_out (bb); - bitmap bb_livein = df_get_live_in (bb); - bb_data_t bb_info = get_bb_data (bb); - - bitmap_and_compl (&temp_bitmap, bb_liveout, &all_hard_regs_bitmap); - return bitmap_ior_and_compl (bb_livein, &bb_info->gen_pseudos, - &temp_bitmap, &bb_info->killed_pseudos); + bb_data_t local_bb_info = get_bb_data_by_index (bb_index); + bitmap full_in = df_get_subreg_live_full_in (local_bb_info->bb); + bitmap full_out = df_get_subreg_live_full_out (local_bb_info->bb); + + bitmap_and_compl (&temp_bitmap, full_out, &all_hard_regs_bitmap); + bool changed = bitmap_ior_and_compl (full_in, &local_bb_info->full_use, + &temp_bitmap, &local_bb_info->full_def); + + /* Handle partial live case. */ + if (flag_track_subreg_liveness) + { + bitmap partial_in = df_get_subreg_live_partial_in (local_bb_info->bb); + bitmap partial_out = df_get_subreg_live_partial_out (local_bb_info->bb); + subregs_live *range_in = df_get_subreg_live_range_in (local_bb_info->bb); + subregs_live *range_out + = df_get_subreg_live_range_out (local_bb_info->bb); + + if (!bitmap_empty_p (partial_out) + || !bitmap_empty_p (&local_bb_info->partial_use)) + { + unsigned int regno; + bitmap_iterator bi; + bitmap_head temp_partial_out; + subregs_live temp_range_out; + + /* TEMP = (OUT & ~DEF) */ + bitmap_initialize (&temp_partial_out, &bitmap_default_obstack); + EXECUTE_IF_SET_IN_BITMAP (partial_out, FIRST_PSEUDO_REGISTER, regno, + bi) + { + sbitmap out_range = range_out->get_range (regno); + temp_range_out.add_range (regno, out_range); + if (bitmap_bit_p (&local_bb_info->partial_def, regno)) + { + sbitmap def_range + = local_bb_info->range_def->get_range (regno); + temp_range_out.remove_range (regno, def_range); + if (!temp_range_out.empty_p (regno)) + bitmap_set_bit (&temp_partial_out, regno); + } + else + bitmap_set_bit (&temp_partial_out, regno); + } + + /* TEMP = USE | TEMP */ + EXECUTE_IF_SET_IN_BITMAP (&local_bb_info->partial_use, + FIRST_PSEUDO_REGISTER, regno, bi) + { + sbitmap use_range = local_bb_info->range_use->get_range (regno); + temp_range_out.add_range (regno, use_range); + } + bitmap_ior_into (&temp_partial_out, &local_bb_info->partial_use); + + /* IN = TEMP */ + changed |= range_in->copy_lives (temp_range_out); + bitmap_copy (partial_in, &temp_partial_out); + df_live_subreg_check_result (full_in, partial_in, range_in); + } + else if (!bitmap_empty_p (partial_in)) + { + changed = true; + bitmap_clear (partial_in); + range_in->clear (); + } + } + + return changed; } /* The confluence function used by the DF equation solver to set up @@ -444,7 +532,9 @@ live_trans_fun (int bb_index) static void live_con_fun_0 (basic_block bb) { - bitmap_and_into (df_get_live_out (bb), &all_hard_regs_bitmap); + bitmap_and_into (df_get_subreg_live_out (bb), &all_hard_regs_bitmap); + if (flag_track_subreg_liveness) + bitmap_and_into (df_get_subreg_live_full_out (bb), &all_hard_regs_bitmap); } /* The confluence function used by the DF equation solver to propagate @@ -456,13 +546,37 @@ live_con_fun_0 (basic_block bb) static bool live_con_fun_n (edge e) { - basic_block bb = e->src; - basic_block dest = e->dest; - bitmap bb_liveout = df_get_live_out (bb); - bitmap dest_livein = df_get_live_in (dest); + bitmap src_full_out = df_get_subreg_live_full_out (e->src); + bitmap dest_full_in = df_get_subreg_live_full_in (e->dest); + + bool changed = bitmap_ior_and_compl_into (src_full_out, dest_full_in, + &all_hard_regs_bitmap); + /* Handle partial live case. */ + if (flag_track_subreg_liveness) + { + bitmap src_partial_out = df_get_subreg_live_partial_out (e->src); + subregs_live *src_range_out = df_get_subreg_live_range_out (e->src); + bitmap dest_partial_in = df_get_subreg_live_partial_in (e->dest); + subregs_live *dest_range_in = df_get_subreg_live_range_in (e->dest); + + if (bitmap_empty_p (dest_partial_in)) + return changed; + + unsigned int regno; + bitmap_iterator bi; + EXECUTE_IF_SET_IN_BITMAP (dest_partial_in, FIRST_PSEUDO_REGISTER, regno, + bi) + { + sbitmap dest_range = dest_range_in->get_range (regno); + changed |= src_range_out->add_range (regno, dest_range); + } + changed |= bitmap_ior_into (src_partial_out, dest_partial_in); - return bitmap_ior_and_compl_into (bb_liveout, - dest_livein, &all_hard_regs_bitmap); + df_live_subreg_check_result (src_full_out, src_partial_out, + src_range_out); + } + + return changed; } /* Indexes of all function blocks. */ @@ -479,12 +593,47 @@ initiate_live_solver (void) bitmap_initialize (&all_blocks, ®_obstack); basic_block bb; + if (flag_track_subreg_liveness) + { + bitmap_initialize (&tracked_regs, ®_obstack); + FOR_ALL_BB_FN (bb, cfun) + { + rtx_insn *insn; + df_ref use; + FOR_BB_INSNS (bb, insn) + { + if (!NONDEBUG_INSN_P (insn)) + continue; + + df_insn_info *insn_info = DF_INSN_INFO_GET (insn); + + FOR_EACH_INSN_INFO_USE (use, insn_info) + { + unsigned int regno = DF_REF_REGNO (use); + /* A multireg which is used via subreg pattern. */ + if (multireg_p (regno) + && DF_REF_FLAGS (use) & (DF_REF_SUBREG)) + bitmap_set_bit (&tracked_regs, regno); + } + } + } + } + + FOR_ALL_BB_FN (bb, cfun) { bb_data_t bb_info = get_bb_data (bb); bb_info->bb = bb; - bitmap_initialize (&bb_info->killed_pseudos, ®_obstack); - bitmap_initialize (&bb_info->gen_pseudos, ®_obstack); + bitmap_initialize (&bb_info->full_def, ®_obstack); + bitmap_initialize (&bb_info->full_use, ®_obstack); + if (flag_track_subreg_liveness) + { + bitmap_initialize (&bb_info->partial_def, ®_obstack); + bitmap_initialize (&bb_info->partial_use, ®_obstack); + size_t num_regs = bitmap_count_bits (&tracked_regs); + bb_info->range_def = new subregs_live (num_regs); + bb_info->range_use = new subregs_live (num_regs); + } bitmap_set_bit (&all_blocks, bb->index); } } @@ -496,11 +645,19 @@ finish_live_solver (void) basic_block bb; bitmap_clear (&all_blocks); + bitmap_clear (&tracked_regs); FOR_ALL_BB_FN (bb, cfun) { bb_data_t bb_info = get_bb_data (bb); - bitmap_clear (&bb_info->killed_pseudos); - bitmap_clear (&bb_info->gen_pseudos); + bitmap_clear (&bb_info->full_def); + bitmap_clear (&bb_info->full_use); + if (flag_track_subreg_liveness) + { + bitmap_clear (&bb_info->partial_def); + bitmap_clear (&bb_info->partial_use); + delete bb_info->range_def; + delete bb_info->range_use; + } } free (bb_data); bitmap_clear (&all_hard_regs_bitmap); @@ -663,7 +820,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) /* Only has a meaningful value once we've seen a call. */ function_abi last_call_abi = default_function_abi; - reg_live_out = df_get_live_out (bb); + reg_live_out = df_get_subreg_live_out (bb); sparseset_clear (pseudos_live); sparseset_clear (pseudos_live_through_calls); sparseset_clear (pseudos_live_through_setjumps); @@ -675,10 +832,16 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) mark_pseudo_live (j); } - bb_gen_pseudos = &get_bb_data (bb)->gen_pseudos; - bb_killed_pseudos = &get_bb_data (bb)->killed_pseudos; - bitmap_clear (bb_gen_pseudos); - bitmap_clear (bb_killed_pseudos); + curr_bb_info = get_bb_data (bb); + bitmap_clear (&curr_bb_info->full_use); + bitmap_clear (&curr_bb_info->full_def); + if (flag_track_subreg_liveness) + { + bitmap_clear (&curr_bb_info->partial_use); + bitmap_clear (&curr_bb_info->partial_def); + curr_bb_info->range_use->clear (); + curr_bb_info->range_def->clear (); + } freq = REG_FREQ_FROM_BB (bb); if (lra_dump_file != NULL) @@ -860,7 +1023,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) if (reg->type != OP_IN) { update_pseudo_point (reg->regno, curr_point, USE_POINT); - mark_regno_live (reg->regno, reg->biggest_mode); + mark_regno_live (reg->regno, reg->biggest_mode, reg); /* ??? Should be a no-op for unused registers. */ check_pseudos_live_through_calls (reg->regno, last_call_abi); } @@ -886,7 +1049,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) { if (reg->type == OP_OUT) update_pseudo_point (reg->regno, curr_point, DEF_POINT); - mark_regno_dead (reg->regno, reg->biggest_mode); + mark_regno_dead (reg->regno, reg->biggest_mode, reg); } for (reg = curr_static_id->hard_regs; reg != NULL; reg = reg->next) @@ -945,8 +1108,12 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) { if (reg->type == OP_IN) update_pseudo_point (reg->regno, curr_point, USE_POINT); - mark_regno_live (reg->regno, reg->biggest_mode); check_pseudos_live_through_calls (reg->regno, last_call_abi); + /* Ignore the use of subreg which is used as dest operand. */ + if (need_track_subreg_p (reg->regno) && reg->subreg_p + && reg->type == OP_INOUT) + continue; + mark_regno_live (reg->regno, reg->biggest_mode, reg); } for (reg = curr_static_id->hard_regs; reg != NULL; reg = reg->next) @@ -970,12 +1137,12 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) { if (reg->type == OP_OUT) update_pseudo_point (reg->regno, curr_point, DEF_POINT); - mark_regno_dead (reg->regno, reg->biggest_mode); + mark_regno_dead (reg->regno, reg->biggest_mode, reg); /* We're done processing inputs, so make sure early clobber operands that are both inputs and outputs are still live. */ if (reg->type == OP_INOUT) - mark_regno_live (reg->regno, reg->biggest_mode); + mark_regno_live (reg->regno, reg->biggest_mode, reg); } for (reg = curr_static_id->hard_regs; reg != NULL; reg = reg->next) @@ -1099,8 +1266,8 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) bool live_change_p = false; /* Check if bb border live info was changed. */ unsigned int live_pseudos_num = 0; - EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb), - FIRST_PSEUDO_REGISTER, j, bi) + EXECUTE_IF_SET_IN_BITMAP (df_get_subreg_live_in (bb), FIRST_PSEUDO_REGISTER, + j, bi) { live_pseudos_num++; if (! sparseset_bit_p (pseudos_live, j)) @@ -1118,7 +1285,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) live_change_p = true; if (lra_dump_file != NULL) EXECUTE_IF_SET_IN_SPARSESET (pseudos_live, j) - if (! bitmap_bit_p (df_get_live_in (bb), j)) + if (! bitmap_bit_p (df_get_subreg_live_in (bb), j)) fprintf (lra_dump_file, " r%d is added to live at bb%d start\n", j, bb->index); } @@ -1133,7 +1300,8 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) mark_pseudo_dead (i); } - EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb), FIRST_PSEUDO_REGISTER, j, bi) + EXECUTE_IF_SET_IN_BITMAP (df_get_subreg_live_in (bb), FIRST_PSEUDO_REGISTER, + j, bi) { if (sparseset_cardinality (pseudos_live_through_calls) == 0) break; @@ -1149,7 +1317,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) if (!TEST_HARD_REG_BIT (hard_regs_spilled_into, i)) continue; - if (bitmap_bit_p (df_get_live_in (bb), i)) + if (bitmap_bit_p (df_get_subreg_live_in (bb), i)) continue; live_change_p = true; @@ -1157,7 +1325,9 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) fprintf (lra_dump_file, " hard reg r%d is added to live at bb%d start\n", i, bb->index); - bitmap_set_bit (df_get_live_in (bb), i); + bitmap_set_bit (df_get_subreg_live_in (bb), i); + if (flag_track_subreg_liveness) + bitmap_set_bit (df_get_subreg_live_full_in (bb), i); } if (need_curr_point_incr) @@ -1421,12 +1591,30 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p) { /* We need to clear pseudo live info as some pseudos can disappear, e.g. pseudos with used equivalences. */ - FOR_EACH_BB_FN (bb, cfun) + FOR_BB_BETWEEN (bb, ENTRY_BLOCK_PTR_FOR_FN (cfun), + EXIT_BLOCK_PTR_FOR_FN (cfun), next_bb) { - bitmap_clear_range (df_get_live_in (bb), FIRST_PSEUDO_REGISTER, + bitmap_clear_range (df_get_subreg_live_in (bb), FIRST_PSEUDO_REGISTER, max_regno - FIRST_PSEUDO_REGISTER); - bitmap_clear_range (df_get_live_out (bb), FIRST_PSEUDO_REGISTER, + bitmap_clear_range (df_get_subreg_live_out (bb), FIRST_PSEUDO_REGISTER, max_regno - FIRST_PSEUDO_REGISTER); + if (flag_track_subreg_liveness) + { + bitmap_clear_range (df_get_subreg_live_full_in (bb), + FIRST_PSEUDO_REGISTER, + max_regno - FIRST_PSEUDO_REGISTER); + bitmap_clear_range (df_get_subreg_live_partial_in (bb), + FIRST_PSEUDO_REGISTER, + max_regno - FIRST_PSEUDO_REGISTER); + bitmap_clear_range (df_get_subreg_live_full_out (bb), + FIRST_PSEUDO_REGISTER, + max_regno - FIRST_PSEUDO_REGISTER); + bitmap_clear_range (df_get_subreg_live_partial_out (bb), + FIRST_PSEUDO_REGISTER, + max_regno - FIRST_PSEUDO_REGISTER); + df_get_subreg_live_range_in (bb)->clear (); + df_get_subreg_live_range_out (bb)->clear (); + } } /* As we did not change CFG since LRA start we can use DF-infrastructure solver to solve live data flow problem. */ @@ -1439,6 +1627,10 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p) (DF_BACKWARD, NULL, live_con_fun_0, live_con_fun_n, live_trans_fun, &all_blocks, df_get_postorder (DF_BACKWARD), df_get_n_blocks (DF_BACKWARD)); + + if (flag_track_subreg_liveness) + df_live_subreg_finalize (&all_blocks); + if (lra_dump_file != NULL) { fprintf (lra_dump_file, @@ -1447,16 +1639,33 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p) FOR_EACH_BB_FN (bb, cfun) { bb_data_t bb_info = get_bb_data (bb); - bitmap bb_livein = df_get_live_in (bb); - bitmap bb_liveout = df_get_live_out (bb); fprintf (lra_dump_file, "\nBB %d:\n", bb->index); - lra_dump_bitmap_with_title (" gen:", - &bb_info->gen_pseudos, bb->index); - lra_dump_bitmap_with_title (" killed:", - &bb_info->killed_pseudos, bb->index); - lra_dump_bitmap_with_title (" livein:", bb_livein, bb->index); - lra_dump_bitmap_with_title (" liveout:", bb_liveout, bb->index); + lra_dump_bitmap_with_title (" full use", &bb_info->full_use, + bb->index); + lra_dump_bitmap_with_title (" full def", &bb_info->full_def, + bb->index); + lra_dump_bitmap_with_title (" live in full", + df_get_subreg_live_full_in (bb), + bb->index); + lra_dump_bitmap_with_title (" live out full", + df_get_subreg_live_full_out (bb), + bb->index); + if (flag_track_subreg_liveness) + { + lra_dump_bitmap_with_title (" partial use", + &bb_info->partial_use, bb->index); + lra_dump_bitmap_with_title (" partial def", + &bb_info->partial_def, bb->index); + lra_dump_bitmap_with_title (" live in partial", + df_get_subreg_live_partial_in ( + bb), + bb->index); + lra_dump_bitmap_with_title (" live out partial", + df_get_subreg_live_partial_out ( + bb), + bb->index); + } } } } diff --git a/gcc/lra-remat.cc b/gcc/lra-remat.cc index c84bf3c9938..ef0157513b0 100644 --- a/gcc/lra-remat.cc +++ b/gcc/lra-remat.cc @@ -556,11 +556,11 @@ dump_candidates_and_remat_bb_data (void) fprintf (lra_dump_file, "\nBB %d:\n", bb->index); /* Livein */ fprintf (lra_dump_file, " register live in:"); - dump_regset (df_get_live_in (bb), lra_dump_file); + dump_regset (df_get_subreg_live_in (bb), lra_dump_file); putc ('\n', lra_dump_file); /* Liveout */ fprintf (lra_dump_file, " register live out:"); - dump_regset (df_get_live_out (bb), lra_dump_file); + dump_regset (df_get_subreg_live_out (bb), lra_dump_file); putc ('\n', lra_dump_file); /* Changed/dead regs: */ fprintf (lra_dump_file, " changed regs:"); @@ -727,7 +727,7 @@ calculate_livein_cands (void) FOR_EACH_BB_FN (bb, cfun) { - bitmap livein_regs = df_get_live_in (bb); + bitmap livein_regs = df_get_subreg_live_in (bb); bitmap livein_cands = &get_remat_bb_data (bb)->livein_cands; for (unsigned int i = 0; i < cands_num; i++) { @@ -1064,7 +1064,7 @@ do_remat (void) FOR_EACH_BB_FN (bb, cfun) { CLEAR_HARD_REG_SET (live_hard_regs); - EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb), 0, regno, bi) + EXECUTE_IF_SET_IN_BITMAP (df_get_subreg_live_in (bb), 0, regno, bi) { int hard_regno = regno < FIRST_PSEUDO_REGISTER ? regno diff --git a/gcc/lra-spills.cc b/gcc/lra-spills.cc index 8b1a945d632..08ec247bc00 100644 --- a/gcc/lra-spills.cc +++ b/gcc/lra-spills.cc @@ -566,8 +566,31 @@ spill_pseudos (void) "Debug insn #%u is reset because it referenced " "removed pseudo\n", INSN_UID (insn)); } - bitmap_and_compl_into (df_get_live_in (bb), spilled_pseudos); - bitmap_and_compl_into (df_get_live_out (bb), spilled_pseudos); + bitmap_and_compl_into (df_get_subreg_live_in (bb), spilled_pseudos); + bitmap_and_compl_into (df_get_subreg_live_out (bb), spilled_pseudos); + + if (flag_track_subreg_liveness) + { + bitmap_and_compl_into (df_get_subreg_live_full_in (bb), + spilled_pseudos); + bitmap partial_in = df_get_subreg_live_partial_in (bb); + subregs_live *range_in = df_get_subreg_live_range_in (bb); + unsigned int regno; + bitmap_iterator bi; + EXECUTE_IF_AND_IN_BITMAP (partial_in, spilled_pseudos, + FIRST_PSEUDO_REGISTER, regno, bi) + range_in->remove_range (regno); + bitmap_and_compl_into (partial_in, spilled_pseudos); + + bitmap_and_compl_into (df_get_subreg_live_full_out (bb), + spilled_pseudos); + bitmap partial_out = df_get_subreg_live_partial_out (bb); + subregs_live *range_out = df_get_subreg_live_range_out (bb); + EXECUTE_IF_AND_IN_BITMAP (partial_out, spilled_pseudos, + FIRST_PSEUDO_REGISTER, regno, bi) + range_out->remove_range (regno); + bitmap_and_compl_into (partial_out, spilled_pseudos); + } } } } diff --git a/gcc/lra.cc b/gcc/lra.cc index fb32e134004..a6fabe7d118 100644 --- a/gcc/lra.cc +++ b/gcc/lra.cc @@ -574,7 +574,7 @@ object_allocator lra_insn_reg_pool ("insn regs"); EARLY_CLOBBER_ALTS. */ static struct lra_insn_reg * new_insn_reg (rtx_insn *insn, int regno, enum op_type type, - machine_mode mode, bool subreg_p, + machine_mode mode, bool subreg_p, rtx op, alternative_mask early_clobber_alts, struct lra_insn_reg *next) { @@ -586,6 +586,7 @@ new_insn_reg (rtx_insn *insn, int regno, enum op_type type, ir->subreg_p = subreg_p; ir->early_clobber_alts = early_clobber_alts; ir->regno = regno; + ir->op = op; ir->next = next; return ir; } @@ -926,7 +927,7 @@ collect_non_operand_hard_regs (rtx_insn *insn, rtx *x, && ! (FIRST_STACK_REG <= regno && regno <= LAST_STACK_REG)); #endif - list = new_insn_reg (data->insn, regno, type, mode, subreg_p, + list = new_insn_reg (data->insn, regno, type, mode, subreg_p, *x, early_clobber ? ALL_ALTERNATIVES : 0, list); } } @@ -1484,6 +1485,7 @@ add_regs_to_insn_regno_info (lra_insn_recog_data_t data, rtx x, code = GET_CODE (x); mode = GET_MODE (x); subreg_p = false; + rtx op = x; if (GET_CODE (x) == SUBREG) { mode = wider_subreg_mode (x); @@ -1501,7 +1503,7 @@ add_regs_to_insn_regno_info (lra_insn_recog_data_t data, rtx x, if (bitmap_set_bit (&lra_reg_info[regno].insn_bitmap, INSN_UID (insn))) { data->regs = new_insn_reg (data->insn, regno, type, mode, subreg_p, - early_clobber_alts, data->regs); + op, early_clobber_alts, data->regs); return; } else @@ -1513,7 +1515,7 @@ add_regs_to_insn_regno_info (lra_insn_recog_data_t data, rtx x, /* The info cannot be integrated into the found structure. */ data->regs = new_insn_reg (data->insn, regno, type, mode, - subreg_p, early_clobber_alts, + subreg_p, op, early_clobber_alts, data->regs); else {