From patchwork Wed Jul 10 15:44:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Sandiford X-Patchwork-Id: 1958905 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WK2HW43qKz1xpd for ; Thu, 11 Jul 2024 01:45:11 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id C7DE3384AB59 for ; Wed, 10 Jul 2024 15:45:09 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 574B63858432 for ; Wed, 10 Jul 2024 15:44:42 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 574B63858432 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 574B63858432 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1720626285; cv=none; b=ldvLxGpK0HBiGdFiDNFbEbHCAoBMIIn1PhwucFvXOLh/CaY/LvRsrF6X+UPIz4PgcjZAPejDtgnwXDkDxTUXKVwa5Hboflv4CckZIsqt205lfc7IdLG/3EryK0c7YzAETi3jWU4tFaMw5yw5nO2KhVC4bVMzXf2cuajPkp2F56I= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1720626285; c=relaxed/simple; bh=mY3p0QOZJ/KWbVBkQjKXCkfnDOOLIvoGDPDfdeuzXzU=; h=From:To:Subject:Date:Message-ID:MIME-Version; b=q9+rF/JHbS6s2MluKo0gt4jLgZslNTdk3cf+8pZRxGS+s2DpWX7+gfjrmb17afCcE9nMGRAC2EEvBMs79aMHrJpL+Q2+78n/Xq9cwPd/K0i9ps7r0fE6sxRP7YjC6QiCmWHXg5Oiy85sH45zGclanxgM548+R8yFHBo/SpKeGSw= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4B0E6106F for ; Wed, 10 Jul 2024 08:45:07 -0700 (PDT) Received: from localhost (e121540-lin.manchester.arm.com [10.32.110.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9415C3F766 for ; Wed, 10 Jul 2024 08:44:41 -0700 (PDT) From: Richard Sandiford To: gcc-patches@gcc.gnu.org Mail-Followup-To: gcc-patches@gcc.gnu.org, richard.sandiford@arm.com Subject: [PATCH] internal-fn: Reuse SUBREG_PROMOTED_VAR_P handling Date: Wed, 10 Jul 2024 16:44:40 +0100 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 X-Spam-Status: No, score=-19.5 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org expand_fn_using_insn has code to handle SUBREG_PROMOTED_VAR_P destinations. Specifically, for: (subreg/v:M1 (reg:M2 R) ...) it creates a new temporary register T, uses it for the output operand, then sign- or zero-extends the M1 lowpart of T to M2, storing the result in R. This patch splits this handling out into helper routines and uses them for other instances of: if (!rtx_equal_p (target, ops[0].value)) emit_move_insn (target, ops[0].value); It's quite probable that this doesn't help any of the other cases; in particular, it shouldn't affect vectors. But I think it could be useful for the CRC work. Bootstrapped & regression-tested on aarch64-linux-gnu. OK to install? Richard gcc/ * internal-fn.cc (create_call_lhs_operand, assign_call_lhs): New functions, split out from... (expand_fn_using_insn): ...here. (expand_load_lanes_optab_fn): Use them. (expand_GOMP_SIMT_ENTER_ALLOC): Likewise. (expand_GOMP_SIMT_LAST_LANE): Likewise. (expand_GOMP_SIMT_ORDERED_PRED): Likewise. (expand_GOMP_SIMT_VOTE_ANY): Likewise. (expand_GOMP_SIMT_XCHG_BFLY): Likewise. (expand_GOMP_SIMT_XCHG_IDX): Likewise. (expand_partial_load_optab_fn): Likewise. (expand_vec_cond_optab_fn): Likewise. (expand_vec_cond_mask_optab_fn): Likewise. (expand_RAWMEMCHR): Likewise. (expand_gather_load_optab_fn): Likewise. (expand_while_optab_fn): Likewise. (expand_SPACESHIP): Likewise. --- gcc/internal-fn.cc | 162 +++++++++++++++++++++++---------------------- 1 file changed, 84 insertions(+), 78 deletions(-) diff --git a/gcc/internal-fn.cc b/gcc/internal-fn.cc index 4948b48bde8..95946bfd683 100644 --- a/gcc/internal-fn.cc +++ b/gcc/internal-fn.cc @@ -199,6 +199,58 @@ const direct_internal_fn_info direct_internal_fn_array[IFN_LAST + 1] = { not_direct }; +/* Like create_output_operand, but for callers that will use + assign_call_lhs afterwards. */ + +static void +create_call_lhs_operand (expand_operand *op, rtx lhs_rtx, machine_mode mode) +{ + /* Do not assign directly to a promoted subreg, since there is no + guarantee that the instruction will leave the upper bits of the + register in the state required by SUBREG_PROMOTED_SIGN. */ + rtx dest = lhs_rtx; + if (dest && GET_CODE (dest) == SUBREG && SUBREG_PROMOTED_VAR_P (dest)) + dest = NULL_RTX; + create_output_operand (op, dest, mode); +} + +/* Move the result of an expanded instruction into the lhs of a gimple call. + LHS is the lhs of the call, LHS_RTX is its expanded form, and OP is the + result of the expanded instruction. OP should have been set up by + create_call_lhs_operand. */ + +static void +assign_call_lhs (tree lhs, rtx lhs_rtx, expand_operand *op) +{ + if (rtx_equal_p (lhs_rtx, op->value)) + return; + + /* If the return value has an integral type, convert the instruction + result to that type. This is useful for things that return an + int regardless of the size of the input. If the instruction result + is smaller than required, assume that it is signed. + + If the return value has a nonintegral type, its mode must match + the instruction result. */ + if (GET_CODE (lhs_rtx) == SUBREG && SUBREG_PROMOTED_VAR_P (lhs_rtx)) + { + /* If this is a scalar in a register that is stored in a wider + mode than the declared mode, compute the result into its + declared mode and then convert to the wider mode. */ + gcc_checking_assert (INTEGRAL_TYPE_P (TREE_TYPE (lhs))); + rtx tmp = convert_to_mode (GET_MODE (lhs_rtx), op->value, 0); + convert_move (SUBREG_REG (lhs_rtx), tmp, + SUBREG_PROMOTED_SIGN (lhs_rtx)); + } + else if (GET_MODE (lhs_rtx) == GET_MODE (op->value)) + emit_move_insn (lhs_rtx, op->value); + else + { + gcc_checking_assert (INTEGRAL_TYPE_P (TREE_TYPE (lhs))); + convert_move (lhs_rtx, op->value, 0); + } +} + /* Expand STMT using instruction ICODE. The instruction has NOUTPUTS output operands and NINPUTS input operands, where NOUTPUTS is either 0 or 1. The output operand (if any) comes first, followed by the @@ -220,15 +272,8 @@ expand_fn_using_insn (gcall *stmt, insn_code icode, unsigned int noutputs, gcc_assert (noutputs == 1); if (lhs) lhs_rtx = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); - - /* Do not assign directly to a promoted subreg, since there is no - guarantee that the instruction will leave the upper bits of the - register in the state required by SUBREG_PROMOTED_SIGN. */ - rtx dest = lhs_rtx; - if (dest && GET_CODE (dest) == SUBREG && SUBREG_PROMOTED_VAR_P (dest)) - dest = NULL_RTX; - create_output_operand (&ops[opno], dest, - insn_data[icode].operand[opno].mode); + create_call_lhs_operand (&ops[opno], lhs_rtx, + insn_data[icode].operand[opno].mode); opno += 1; } else @@ -266,33 +311,8 @@ expand_fn_using_insn (gcall *stmt, insn_code icode, unsigned int noutputs, gcc_assert (opno == noutputs + ninputs); expand_insn (icode, opno, ops); - if (lhs_rtx && !rtx_equal_p (lhs_rtx, ops[0].value)) - { - /* If the return value has an integral type, convert the instruction - result to that type. This is useful for things that return an - int regardless of the size of the input. If the instruction result - is smaller than required, assume that it is signed. - - If the return value has a nonintegral type, its mode must match - the instruction result. */ - if (GET_CODE (lhs_rtx) == SUBREG && SUBREG_PROMOTED_VAR_P (lhs_rtx)) - { - /* If this is a scalar in a register that is stored in a wider - mode than the declared mode, compute the result into its - declared mode and then convert to the wider mode. */ - gcc_checking_assert (INTEGRAL_TYPE_P (TREE_TYPE (lhs))); - rtx tmp = convert_to_mode (GET_MODE (lhs_rtx), ops[0].value, 0); - convert_move (SUBREG_REG (lhs_rtx), tmp, - SUBREG_PROMOTED_SIGN (lhs_rtx)); - } - else if (GET_MODE (lhs_rtx) == GET_MODE (ops[0].value)) - emit_move_insn (lhs_rtx, ops[0].value); - else - { - gcc_checking_assert (INTEGRAL_TYPE_P (TREE_TYPE (lhs))); - convert_move (lhs_rtx, ops[0].value, 0); - } - } + if (lhs_rtx) + assign_call_lhs (lhs, lhs_rtx, &ops[0]); } /* ARRAY_TYPE is an array of vector modes. Return the associated insn @@ -376,11 +396,10 @@ expand_load_lanes_optab_fn (internal_fn, gcall *stmt, convert_optab optab) gcc_assert (MEM_P (mem)); PUT_MODE (mem, TYPE_MODE (type)); - create_output_operand (&ops[0], target, TYPE_MODE (type)); + create_call_lhs_operand (&ops[0], target, TYPE_MODE (type)); create_fixed_operand (&ops[1], mem); expand_insn (get_multi_vector_move (type, optab), 2, ops); - if (!rtx_equal_p (target, ops[0].value)) - emit_move_insn (target, ops[0].value); + assign_call_lhs (lhs, target, &ops[0]); } /* Expand STORE_LANES call STMT using optab OPTAB. */ @@ -443,13 +462,12 @@ expand_GOMP_SIMT_ENTER_ALLOC (internal_fn, gcall *stmt) rtx size = expand_normal (gimple_call_arg (stmt, 0)); rtx align = expand_normal (gimple_call_arg (stmt, 1)); class expand_operand ops[3]; - create_output_operand (&ops[0], target, Pmode); + create_call_lhs_operand (&ops[0], target, Pmode); create_input_operand (&ops[1], size, Pmode); create_input_operand (&ops[2], align, Pmode); gcc_assert (targetm.have_omp_simt_enter ()); expand_insn (targetm.code_for_omp_simt_enter, 3, ops); - if (!rtx_equal_p (target, ops[0].value)) - emit_move_insn (target, ops[0].value); + assign_call_lhs (lhs, target, &ops[0]); } /* Deallocate per-lane storage and leave non-uniform execution region. */ @@ -511,12 +529,11 @@ expand_GOMP_SIMT_LAST_LANE (internal_fn, gcall *stmt) rtx cond = expand_normal (gimple_call_arg (stmt, 0)); machine_mode mode = TYPE_MODE (TREE_TYPE (lhs)); class expand_operand ops[2]; - create_output_operand (&ops[0], target, mode); + create_call_lhs_operand (&ops[0], target, mode); create_input_operand (&ops[1], cond, mode); gcc_assert (targetm.have_omp_simt_last_lane ()); expand_insn (targetm.code_for_omp_simt_last_lane, 2, ops); - if (!rtx_equal_p (target, ops[0].value)) - emit_move_insn (target, ops[0].value); + assign_call_lhs (lhs, target, &ops[0]); } /* Non-transparent predicate used in SIMT lowering of OpenMP "ordered". */ @@ -532,12 +549,11 @@ expand_GOMP_SIMT_ORDERED_PRED (internal_fn, gcall *stmt) rtx ctr = expand_normal (gimple_call_arg (stmt, 0)); machine_mode mode = TYPE_MODE (TREE_TYPE (lhs)); class expand_operand ops[2]; - create_output_operand (&ops[0], target, mode); + create_call_lhs_operand (&ops[0], target, mode); create_input_operand (&ops[1], ctr, mode); gcc_assert (targetm.have_omp_simt_ordered ()); expand_insn (targetm.code_for_omp_simt_ordered, 2, ops); - if (!rtx_equal_p (target, ops[0].value)) - emit_move_insn (target, ops[0].value); + assign_call_lhs (lhs, target, &ops[0]); } /* "Or" boolean reduction across SIMT lanes: return non-zero in all lanes if @@ -554,12 +570,11 @@ expand_GOMP_SIMT_VOTE_ANY (internal_fn, gcall *stmt) rtx cond = expand_normal (gimple_call_arg (stmt, 0)); machine_mode mode = TYPE_MODE (TREE_TYPE (lhs)); class expand_operand ops[2]; - create_output_operand (&ops[0], target, mode); + create_call_lhs_operand (&ops[0], target, mode); create_input_operand (&ops[1], cond, mode); gcc_assert (targetm.have_omp_simt_vote_any ()); expand_insn (targetm.code_for_omp_simt_vote_any, 2, ops); - if (!rtx_equal_p (target, ops[0].value)) - emit_move_insn (target, ops[0].value); + assign_call_lhs (lhs, target, &ops[0]); } /* Exchange between SIMT lanes with a "butterfly" pattern: source lane index @@ -577,13 +592,12 @@ expand_GOMP_SIMT_XCHG_BFLY (internal_fn, gcall *stmt) rtx idx = expand_normal (gimple_call_arg (stmt, 1)); machine_mode mode = TYPE_MODE (TREE_TYPE (lhs)); class expand_operand ops[3]; - create_output_operand (&ops[0], target, mode); + create_call_lhs_operand (&ops[0], target, mode); create_input_operand (&ops[1], src, mode); create_input_operand (&ops[2], idx, SImode); gcc_assert (targetm.have_omp_simt_xchg_bfly ()); expand_insn (targetm.code_for_omp_simt_xchg_bfly, 3, ops); - if (!rtx_equal_p (target, ops[0].value)) - emit_move_insn (target, ops[0].value); + assign_call_lhs (lhs, target, &ops[0]); } /* Exchange between SIMT lanes according to given source lane index. */ @@ -600,13 +614,12 @@ expand_GOMP_SIMT_XCHG_IDX (internal_fn, gcall *stmt) rtx idx = expand_normal (gimple_call_arg (stmt, 1)); machine_mode mode = TYPE_MODE (TREE_TYPE (lhs)); class expand_operand ops[3]; - create_output_operand (&ops[0], target, mode); + create_call_lhs_operand (&ops[0], target, mode); create_input_operand (&ops[1], src, mode); create_input_operand (&ops[2], idx, SImode); gcc_assert (targetm.have_omp_simt_xchg_idx ()); expand_insn (targetm.code_for_omp_simt_xchg_idx, 3, ops); - if (!rtx_equal_p (target, ops[0].value)) - emit_move_insn (target, ops[0].value); + assign_call_lhs (lhs, target, &ops[0]); } /* This should get expanded in adjust_simduid_builtins. */ @@ -3029,13 +3042,12 @@ expand_partial_load_optab_fn (internal_fn ifn, gcall *stmt, convert_optab optab) set_mem_expr (mem, NULL_TREE); clear_mem_offset (mem); target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); - create_output_operand (&ops[i++], target, TYPE_MODE (type)); + create_call_lhs_operand (&ops[i++], target, TYPE_MODE (type)); create_fixed_operand (&ops[i++], mem); i = add_mask_and_len_args (ops, i, stmt); expand_insn (icode, i, ops); - if (!rtx_equal_p (target, ops[0].value)) - emit_move_insn (target, ops[0].value); + assign_call_lhs (lhs, target, &ops[0]); } #define expand_mask_load_optab_fn expand_partial_load_optab_fn @@ -3129,15 +3141,14 @@ expand_vec_cond_optab_fn (internal_fn, gcall *stmt, convert_optab optab) rtx_op2 = expand_normal (op2); rtx target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); - create_output_operand (&ops[0], target, mode); + create_call_lhs_operand (&ops[0], target, mode); create_input_operand (&ops[1], rtx_op1, mode); create_input_operand (&ops[2], rtx_op2, mode); create_fixed_operand (&ops[3], comparison); create_fixed_operand (&ops[4], XEXP (comparison, 0)); create_fixed_operand (&ops[5], XEXP (comparison, 1)); expand_insn (icode, 6, ops); - if (!rtx_equal_p (ops[0].value, target)) - emit_move_insn (target, ops[0].value); + assign_call_lhs (lhs, target, &ops[0]); } /* Expand VCOND_MASK optab internal function. @@ -3166,13 +3177,12 @@ expand_vec_cond_mask_optab_fn (internal_fn, gcall *stmt, convert_optab optab) rtx_op2 = expand_normal (op2); rtx target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); - create_output_operand (&ops[0], target, mode); + create_call_lhs_operand (&ops[0], target, mode); create_input_operand (&ops[1], rtx_op1, mode); create_input_operand (&ops[2], rtx_op2, mode); create_input_operand (&ops[3], mask, mask_mode); expand_insn (icode, 4, ops); - if (!rtx_equal_p (ops[0].value, target)) - emit_move_insn (target, ops[0].value); + assign_call_lhs (lhs, target, &ops[0]); } /* Expand VEC_SET internal functions. */ @@ -3266,7 +3276,7 @@ expand_RAWMEMCHR (internal_fn, gcall *stmt) return; machine_mode lhs_mode = TYPE_MODE (TREE_TYPE (lhs)); rtx lhs_rtx = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); - create_output_operand (&ops[0], lhs_rtx, lhs_mode); + create_call_lhs_operand (&ops[0], lhs_rtx, lhs_mode); tree mem = gimple_call_arg (stmt, 0); rtx mem_rtx = get_memory_rtx (mem, NULL); @@ -3280,8 +3290,7 @@ expand_RAWMEMCHR (internal_fn, gcall *stmt) insn_code icode = direct_optab_handler (rawmemchr_optab, mode); expand_insn (icode, 3, ops); - if (!rtx_equal_p (lhs_rtx, ops[0].value)) - emit_move_insn (lhs_rtx, ops[0].value); + assign_call_lhs (lhs, lhs_rtx, &ops[0]); } /* Expand the IFN_UNIQUE function according to its first argument. */ @@ -3691,7 +3700,7 @@ expand_gather_load_optab_fn (internal_fn, gcall *stmt, direct_optab optab) int i = 0; class expand_operand ops[8]; - create_output_operand (&ops[i++], lhs_rtx, TYPE_MODE (TREE_TYPE (lhs))); + create_call_lhs_operand (&ops[i++], lhs_rtx, TYPE_MODE (TREE_TYPE (lhs))); create_address_operand (&ops[i++], base_rtx); create_input_operand (&ops[i++], offset_rtx, TYPE_MODE (TREE_TYPE (offset))); create_integer_operand (&ops[i++], TYPE_UNSIGNED (TREE_TYPE (offset))); @@ -3700,8 +3709,7 @@ expand_gather_load_optab_fn (internal_fn, gcall *stmt, direct_optab optab) insn_code icode = convert_optab_handler (optab, TYPE_MODE (TREE_TYPE (lhs)), TYPE_MODE (TREE_TYPE (offset))); expand_insn (icode, i, ops); - if (!rtx_equal_p (lhs_rtx, ops[0].value)) - emit_move_insn (lhs_rtx, ops[0].value); + assign_call_lhs (lhs, lhs_rtx, &ops[0]); } /* Helper for expand_DIVMOD. Return true if the sequence starting with @@ -3909,7 +3917,7 @@ expand_while_optab_fn (internal_fn, gcall *stmt, convert_optab optab) tree lhs = gimple_call_lhs (stmt); tree lhs_type = TREE_TYPE (lhs); rtx lhs_rtx = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); - create_output_operand (&ops[0], lhs_rtx, TYPE_MODE (lhs_type)); + create_call_lhs_operand (&ops[0], lhs_rtx, TYPE_MODE (lhs_type)); for (unsigned int i = 0; i < 2; ++i) { @@ -3937,8 +3945,7 @@ expand_while_optab_fn (internal_fn, gcall *stmt, convert_optab optab) TYPE_MODE (lhs_type)); expand_insn (icode, opcnt, ops); - if (!rtx_equal_p (lhs_rtx, ops[0].value)) - emit_move_insn (lhs_rtx, ops[0].value); + assign_call_lhs (lhs, lhs_rtx, &ops[0]); } /* Expand a call to a convert-like optab using the operands in STMT. @@ -5067,13 +5074,12 @@ expand_SPACESHIP (internal_fn, gcall *stmt) rtx op2 = expand_normal (rhs2); class expand_operand ops[3]; - create_output_operand (&ops[0], target, TYPE_MODE (TREE_TYPE (lhs))); + create_call_lhs_operand (&ops[0], target, TYPE_MODE (TREE_TYPE (lhs))); create_input_operand (&ops[1], op1, TYPE_MODE (type)); create_input_operand (&ops[2], op2, TYPE_MODE (type)); insn_code icode = optab_handler (spaceship_optab, TYPE_MODE (type)); expand_insn (icode, 3, ops); - if (!rtx_equal_p (target, ops[0].value)) - emit_move_insn (target, ops[0].value); + assign_call_lhs (lhs, target, &ops[0]); } void