From patchwork Thu Jul 11 21:33:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Roger Sayle X-Patchwork-Id: 1959541 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=nextmovesoftware.com header.i=@nextmovesoftware.com header.a=rsa-sha256 header.s=default header.b=K1ZP4BsE; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WKnz56Cptz1xqx for ; Fri, 12 Jul 2024 07:33:36 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 034F63875DD7 for ; Thu, 11 Jul 2024 21:33:35 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from server.nextmovesoftware.com (server.nextmovesoftware.com [69.48.154.134]) by sourceware.org (Postfix) with ESMTPS id D7FD63858D34 for ; Thu, 11 Jul 2024 21:33:12 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org D7FD63858D34 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=nextmovesoftware.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nextmovesoftware.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org D7FD63858D34 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=69.48.154.134 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1720733595; cv=none; b=o1EFZZ2z1xTxXgb6rI1vr6ht0D8wEufHE2CNa3fKdvph3CGD0G53w6Hec5U3Im5QwPavXb7A3tGp4YTDLDhIrFSyNrFEDaECHUvd0BW1eYnbXSDYOgeVaRnSboAjEqztwDuJr0fojNdpksoMNwbm0OkL3MnEl9nWhmQ4QolYau8= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1720733595; c=relaxed/simple; bh=XiCtm48H9H6+Lie/KtsWrVnePot2EeCiGpcO+11lT9U=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=W+hOlwK5E3SPWjbDgHS+i+I6ur6Hm3DY5wtck+LTxorsBNHoaODTWJ3lnnbChKDcrxjCDwlF51PmaMjLt3R81C/3f48AvgsOa7uI52IiTvKbLcFmoj7rio9267q7RUOT+VLZuy7SHGUggsf1dnBkUludZn8QBrkkdaRsWtvVxTY= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=nextmovesoftware.com; s=default; h=Content-Type:MIME-Version:Message-ID: Date:Subject:In-Reply-To:References:Cc:To:From:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=X5a8teXrWM5eobtcGe4oGZnH0euhPQsNlifS9sPXo3E=; b=K1ZP4BsEIehTV0MfpXI4EN5k4i 5XYlYUi4lSQSn51GL0MdT/6016/y81oAMiSjNb8PAo4N++9fJnIkjlq7WyfRohkl3uQrzats98UBc ZECfAF+J4OvDPcQOpDKx/qp+g8i508dW4pwSL3SHw/Ta+nRjtUE9Uih6APzzvrFew94dDsLJZK/BR aJBWlJcuSDWGfA+GCEN93anskevYdylk5gJBQqbfWisXDz1Uz9Jtz+dqJRYjopLI4QLzSLLaFfdwW CS/TgGkrGvJUSMjTRpoR1/juS7TPV7qP3N1LtU8l85FaQ/4CVdFVjMJR23DmzfPWycSNgmYtF2smd DHdXcsXw==; Received: from [168.86.198.82] (port=57418 helo=Dell) by server.nextmovesoftware.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.97.1) (envelope-from ) id 1sS1Pa-00000000rWv-1ZoZ; Thu, 11 Jul 2024 17:33:10 -0400 From: "Roger Sayle" To: "'Hongtao Liu'" Cc: References: <007701dad04c$240bb590$6c2320b0$@nextmovesoftware.com> In-Reply-To: Subject: [x86 SSE PATCH] Some AVX512 ternlog expansion refinements (take #2) Date: Thu, 11 Jul 2024 22:33:02 +0100 Message-ID: <009501dad3d9$ed09d720$c71d8560$@nextmovesoftware.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 16.0 Thread-Index: AdrT2J25JLMYPWZTSKSbSY0vyYOxiQ== Content-Language: en-gb X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server.nextmovesoftware.com X-AntiAbuse: Original Domain - gcc.gnu.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - nextmovesoftware.com X-Get-Message-Sender-Via: server.nextmovesoftware.com: authenticated_id: roger@nextmovesoftware.com X-Authenticated-Sender: server.nextmovesoftware.com: roger@nextmovesoftware.com X-Source: X-Source-Args: X-Source-Dir: X-Spam-Status: No, score=-12.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org Hi Hongtao, Thanks for the review and pointing out the remaining uses of force_reg that I'd overlooked. Here's a revised version of the patch that incorporates your feedback. One minor change was that rather than using memory_operand, which as you point out also needs to include bcst_mem_operand, it's simpler to invert the logic to check for register_operand [i.e. the first operand must be a register]. This patch has been tested on x86_64-pc-linux-gnu with make bootstrap and make -k check, both with and without --target_board=unix{-m32} with no new failures. Ok for mainline? 2024-07-11 Roger Sayle Hongtao Liu gcc/ChangeLog * config/i386/i386-expand.cc (ix86_broadcast_from_constant): Use CONST_VECTOR_P instead of comparison against GET_CODE. (ix86_gen_bcst_mem): Likewise. (ix86_ternlog_leaf_p): Likewise. (ix86_ternlog_operand_p): ix86_ternlog_leaf_p is always true for vector_all_ones_operand. (ix86_expand_ternlog_bin_op): Use CONST_VECTOR_P instead of equality comparison against GET_CODE. Replace call to force_reg with gen_reg_rtx and emit_move_insn (for VEC_DUPLICATE broadcast). Check for !register_operand instead of memory_operand. Support CONST_VECTORs by calling force_const_mem. (ix86_expand_ternlog): Fix indentation whitespace. Allow ix86_ternlog_leaf_p as ix86_expand_ternlog_andnot's second operand. Use CONST_VECTOR_P instead of equality against GET_CODE. Use gen_reg_rtx and emit_move_insn for ~a, ~b and ~c cases. Thanks again, Roger > -----Original Message----- > From: Hongtao Liu > Sent: 08 July 2024 02:55 > To: Roger Sayle > Cc: gcc-patches@gcc.gnu.org; Uros Bizjak > Subject: Re: [x86 SSE PATCH] Some AVX512 ternlog expansion refinements. > > On Sun, Jul 7, 2024 at 5:00 PM Roger Sayle > wrote: > > Hi Hongtao, > > This should address concerns about the remaining use of force_reg. > > > 51@@ -25793,15 +25792,20 @@ ix86_expand_ternlog_binop (enum rtx_code > code, machine_mode mode, > 52 if (GET_MODE (op1) != mode) > 53 op1 = gen_lowpart (mode, op1); > 54 > 55- if (GET_CODE (op0) == CONST_VECTOR) 56+ if (CONST_VECTOR_P (op0)) > 57 op0 = validize_mem (force_const_mem (mode, op0)); > 58- if (GET_CODE (op1) == CONST_VECTOR) 59+ if (CONST_VECTOR_P (op1)) > 60 op1 = validize_mem (force_const_mem (mode, op1)); > 61 > 62 if (memory_operand (op0, mode)) > 63 { > 64 if (memory_operand (op1, mode)) > 65- op0 = force_reg (mode, op0); > 66+ { > 67+ /* We can't use force_reg (op0, mode). */ > 68+ rtx reg = gen_reg_rtx (mode); > 69+ emit_move_insn (reg, op0); > 70+ op0 = reg; > 71+ } > Shouldn't we handle bcst_mem_operand instead of > memory_operand(bcst_memory_operand is not a memory_operand)? > so maybe > if (memory_operand (op0, mode0) || bcst_mem_operand (op0, mode0) > if (memory_operand (op1, mode) || bcst_mem_operand (op1, mode0)? > 72 else > 73 std::swap (op0, op1); > 74 } > > Also there's force_reg in below 3 cases, are there any restrictions to avoid > bcst_mem_operand into them? > case 0x0f: /* ~a */ > case 0x33: /* ~b */ > case 0x33: /* ~b */ > .. > if (!TARGET_64BIT && !register_operand (op2, mode)) > op2 = force_reg (mode, op2); > > -- > BR, > Hongtao diff --git a/gcc/config/i386/i386-expand.cc b/gcc/config/i386/i386-expand.cc index dd2c3a8..7818d40 100644 --- a/gcc/config/i386/i386-expand.cc +++ b/gcc/config/i386/i386-expand.cc @@ -612,7 +612,7 @@ ix86_broadcast_from_constant (machine_mode mode, rtx op) return nullptr; rtx constant = get_pool_constant (XEXP (op, 0)); - if (GET_CODE (constant) != CONST_VECTOR) + if (!CONST_VECTOR_P (constant)) return nullptr; /* There could be some rtx like @@ -622,7 +622,7 @@ ix86_broadcast_from_constant (machine_mode mode, rtx op) { constant = simplify_subreg (mode, constant, GET_MODE (constant), 0); - if (constant == nullptr || GET_CODE (constant) != CONST_VECTOR) + if (constant == nullptr || !CONST_VECTOR_P (constant)) return nullptr; } @@ -25532,7 +25532,7 @@ static rtx ix86_gen_bcst_mem (machine_mode mode, rtx x) { if (!TARGET_AVX512F - || GET_CODE (x) != CONST_VECTOR + || !CONST_VECTOR_P (x) || (!TARGET_AVX512VL && (GET_MODE_SIZE (mode) != 64 || !TARGET_EVEX512)) || !VALID_BCST_MODE_P (GET_MODE_INNER (mode)) @@ -25722,7 +25722,7 @@ ix86_ternlog_leaf_p (rtx op, machine_mode mode) problems splitting instructions. */ return register_operand (op, mode) || MEM_P (op) - || GET_CODE (op) == CONST_VECTOR + || CONST_VECTOR_P (op) || bcst_mem_operand (op, mode); } @@ -25772,8 +25772,7 @@ ix86_ternlog_operand_p (rtx op) op1 = XEXP (op, 1); /* Prefer pxor, or one_cmpl2. */ if (ix86_ternlog_leaf_p (XEXP (op, 0), mode) - && (ix86_ternlog_leaf_p (op1, mode) - || vector_all_ones_operand (op1, mode))) + && ix86_ternlog_leaf_p (XEXP (op, 1), mode)) return false; break; @@ -25793,15 +25792,20 @@ ix86_expand_ternlog_binop (enum rtx_code code, machine_mode mode, if (GET_MODE (op1) != mode) op1 = gen_lowpart (mode, op1); - if (GET_CODE (op0) == CONST_VECTOR) + if (CONST_VECTOR_P (op0)) op0 = validize_mem (force_const_mem (mode, op0)); - if (GET_CODE (op1) == CONST_VECTOR) + if (CONST_VECTOR_P (op1)) op1 = validize_mem (force_const_mem (mode, op1)); - if (memory_operand (op0, mode)) + if (!register_operand (op0, mode)) { - if (memory_operand (op1, mode)) - op0 = force_reg (mode, op0); + if (!register_operand (op1, mode)) + { + /* We can't use force_reg (op0, mode). */ + rtx reg = gen_reg_rtx (mode); + emit_move_insn (reg, op0); + op0 = reg; + } else std::swap (op0, op1); } @@ -25820,6 +25824,8 @@ ix86_expand_ternlog_andnot (machine_mode mode, rtx op0, rtx op1, rtx target) op0 = gen_rtx_NOT (mode, op0); if (GET_MODE (op1) != mode) op1 = gen_lowpart (mode, op1); + if (CONST_VECTOR_P (op1)) + op1 = validize_mem (force_const_mem (mode, op1)); emit_move_insn (target, gen_rtx_AND (mode, op0, op1)); return target; } @@ -25856,9 +25862,9 @@ ix86_expand_ternlog (machine_mode mode, rtx op0, rtx op1, rtx op2, int idx, { case 0x00: if ((!op0 || !side_effects_p (op0)) - && (!op1 || !side_effects_p (op1)) - && (!op2 || !side_effects_p (op2))) - { + && (!op1 || !side_effects_p (op1)) + && (!op2 || !side_effects_p (op2))) + { emit_move_insn (target, CONST0_RTX (mode)); return target; } @@ -25867,26 +25873,31 @@ ix86_expand_ternlog (machine_mode mode, rtx op0, rtx op1, rtx op2, int idx, case 0x0a: /* ~a&c */ if ((!op1 || !side_effects_p (op1)) && op0 && register_operand (op0, mode) - && op2 && register_operand (op2, mode)) + && op2 && ix86_ternlog_leaf_p (op2, mode)) return ix86_expand_ternlog_andnot (mode, op0, op2, target); break; case 0x0c: /* ~a&b */ if ((!op2 || !side_effects_p (op2)) && op0 && register_operand (op0, mode) - && op1 && register_operand (op1, mode)) + && op1 && ix86_ternlog_leaf_p (op1, mode)) return ix86_expand_ternlog_andnot (mode, op0, op1, target); break; case 0x0f: /* ~a */ if ((!op1 || !side_effects_p (op1)) && (!op2 || !side_effects_p (op2)) - && op0) + && op0) { if (GET_MODE (op0) != mode) op0 = gen_lowpart (mode, op0); if (!TARGET_64BIT && !register_operand (op0, mode)) - op0 = force_reg (mode, op0); + { + /* Avoid force_reg (mode, op0). */ + rtx reg = gen_reg_rtx (mode); + emit_move_insn (reg, op0); + op0 = reg; + } emit_move_insn (target, gen_rtx_XOR (mode, op0, CONSTM1_RTX (mode))); return target; } @@ -25895,13 +25906,13 @@ ix86_expand_ternlog (machine_mode mode, rtx op0, rtx op1, rtx op2, int idx, case 0x22: /* ~b&c */ if ((!op0 || !side_effects_p (op0)) && op1 && register_operand (op1, mode) - && op2 && register_operand (op2, mode)) + && op2 && ix86_ternlog_leaf_p (op2, mode)) return ix86_expand_ternlog_andnot (mode, op1, op2, target); break; case 0x30: /* ~b&a */ if ((!op2 || !side_effects_p (op2)) - && op0 && register_operand (op0, mode) + && op0 && ix86_ternlog_leaf_p (op0, mode) && op1 && register_operand (op1, mode)) return ix86_expand_ternlog_andnot (mode, op1, op0, target); break; @@ -25909,33 +25920,39 @@ ix86_expand_ternlog (machine_mode mode, rtx op0, rtx op1, rtx op2, int idx, case 0x33: /* ~b */ if ((!op0 || !side_effects_p (op0)) && (!op2 || !side_effects_p (op2)) - && op1) + && op1) { if (GET_MODE (op1) != mode) op1 = gen_lowpart (mode, op1); if (!TARGET_64BIT && !register_operand (op1, mode)) - op1 = force_reg (mode, op1); + { + /* Avoid force_reg (mode, op1). */ + rtx reg = gen_reg_rtx (mode); + emit_move_insn (reg, op1); + op1 = reg; + } emit_move_insn (target, gen_rtx_XOR (mode, op1, CONSTM1_RTX (mode))); return target; } break; case 0x3c: /* a^b */ - if (op0 && op1 - && (!op2 || !side_effects_p (op2))) + if (op0 && ix86_ternlog_leaf_p (op0, mode) + && op1 && ix86_ternlog_leaf_p (op1, mode) + && (!op2 || !side_effects_p (op2))) return ix86_expand_ternlog_binop (XOR, mode, op0, op1, target); break; case 0x44: /* ~c&b */ if ((!op0 || !side_effects_p (op0)) - && op1 && register_operand (op1, mode) + && op1 && ix86_ternlog_leaf_p (op1, mode) && op2 && register_operand (op2, mode)) return ix86_expand_ternlog_andnot (mode, op2, op1, target); break; case 0x50: /* ~c&a */ if ((!op1 || !side_effects_p (op1)) - && op0 && register_operand (op0, mode) + && op0 && ix86_ternlog_leaf_p (op0, mode) && op2 && register_operand (op2, mode)) return ix86_expand_ternlog_andnot (mode, op2, op0, target); break; @@ -25943,45 +25960,54 @@ ix86_expand_ternlog (machine_mode mode, rtx op0, rtx op1, rtx op2, int idx, case 0x55: /* ~c */ if ((!op0 || !side_effects_p (op0)) && (!op1 || !side_effects_p (op1)) - && op2) + && op2) { if (GET_MODE (op2) != mode) op2 = gen_lowpart (mode, op2); if (!TARGET_64BIT && !register_operand (op2, mode)) - op2 = force_reg (mode, op2); + { + /* Avoid force_reg (mode, op2). */ + rtx reg = gen_reg_rtx (mode); + emit_move_insn (reg, op2); + op2 = reg; + } emit_move_insn (target, gen_rtx_XOR (mode, op2, CONSTM1_RTX (mode))); return target; } break; case 0x5a: /* a^c */ - if (op0 && op2 - && (!op1 || !side_effects_p (op1))) + if (op0 && ix86_ternlog_leaf_p (op0, mode) + && op2 && ix86_ternlog_leaf_p (op2, mode) + && (!op1 || !side_effects_p (op1))) return ix86_expand_ternlog_binop (XOR, mode, op0, op2, target); break; case 0x66: /* b^c */ if ((!op0 || !side_effects_p (op0)) - && op1 && op2) + && op1 && ix86_ternlog_leaf_p (op1, mode) + && op2 && ix86_ternlog_leaf_p (op2, mode)) return ix86_expand_ternlog_binop (XOR, mode, op1, op2, target); break; case 0x88: /* b&c */ if ((!op0 || !side_effects_p (op0)) - && op1 && op2) + && op1 && ix86_ternlog_leaf_p (op1, mode) + && op2 && ix86_ternlog_leaf_p (op2, mode)) return ix86_expand_ternlog_binop (AND, mode, op1, op2, target); break; case 0xa0: /* a&c */ if ((!op1 || !side_effects_p (op1)) - && op0 && op2) + && op0 && ix86_ternlog_leaf_p (op0, mode) + && op2 && ix86_ternlog_leaf_p (op2, mode)) return ix86_expand_ternlog_binop (AND, mode, op0, op2, target); break; case 0xaa: /* c */ if ((!op0 || !side_effects_p (op0)) && (!op1 || !side_effects_p (op1)) - && op2) + && op2) { if (GET_MODE (op2) != mode) op2 = gen_lowpart (mode, op2); @@ -25991,14 +26017,15 @@ ix86_expand_ternlog (machine_mode mode, rtx op0, rtx op1, rtx op2, int idx, break; case 0xc0: /* a&b */ - if (op0 && op1 - && (!op2 || !side_effects_p (op2))) + if (op0 && ix86_ternlog_leaf_p (op0, mode) + && op1 && ix86_ternlog_leaf_p (op1, mode) + && (!op2 || !side_effects_p (op2))) return ix86_expand_ternlog_binop (AND, mode, op0, op1, target); break; case 0xcc: /* b */ if ((!op0 || !side_effects_p (op0)) - && op1 + && op1 && (!op2 || !side_effects_p (op2))) { if (GET_MODE (op1) != mode) @@ -26010,13 +26037,14 @@ ix86_expand_ternlog (machine_mode mode, rtx op0, rtx op1, rtx op2, int idx, case 0xee: /* b|c */ if ((!op0 || !side_effects_p (op0)) - && op1 && op2) + && op1 && ix86_ternlog_leaf_p (op1, mode) + && op2 && ix86_ternlog_leaf_p (op2, mode)) return ix86_expand_ternlog_binop (IOR, mode, op1, op2, target); break; case 0xf0: /* a */ if (op0 - && (!op1 || !side_effects_p (op1)) + && (!op1 || !side_effects_p (op1)) && (!op2 || !side_effects_p (op2))) { if (GET_MODE (op0) != mode) @@ -26027,23 +26055,25 @@ ix86_expand_ternlog (machine_mode mode, rtx op0, rtx op1, rtx op2, int idx, break; case 0xfa: /* a|c */ - if (op0 && op2 - && (!op1 || !side_effects_p (op1))) + if (op0 && ix86_ternlog_leaf_p (op0, mode) + && op2 && ix86_ternlog_leaf_p (op2, mode) + && (!op1 || !side_effects_p (op1))) return ix86_expand_ternlog_binop (IOR, mode, op0, op2, target); break; case 0xfc: /* a|b */ - if (op0 && op1 - && (!op2 || !side_effects_p (op2))) + if (op0 && ix86_ternlog_leaf_p (op0, mode) + && op1 && ix86_ternlog_leaf_p (op1, mode) + && (!op2 || !side_effects_p (op2))) return ix86_expand_ternlog_binop (IOR, mode, op0, op1, target); break; case 0xff: if ((!op0 || !side_effects_p (op0)) - && (!op1 || !side_effects_p (op1)) - && (!op2 || !side_effects_p (op2))) - { - emit_move_insn (target, CONSTM1_RTX (mode)); + && (!op1 || !side_effects_p (op1)) + && (!op2 || !side_effects_p (op2))) + { + emit_move_insn (target, CONSTM1_RTX (mode)); return target; } break; @@ -26066,7 +26096,7 @@ ix86_expand_ternlog (machine_mode mode, rtx op0, rtx op1, rtx op2, int idx, tmp2 = copy_rtx (tmp0); else if (rtx_equal_p (op1, op2)) tmp2 = copy_rtx (tmp1); - else if (GET_CODE (op2) == CONST_VECTOR) + else if (CONST_VECTOR_P (op2)) { if (GET_MODE (op2) != mode) op2 = gen_lowpart (mode, op2);