From patchwork Tue Oct 15 03:29:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liu, Hongtao" X-Patchwork-Id: 1997173 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=RbEKBPzG; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XSKP12Pdgz1xvy for ; Tue, 15 Oct 2024 14:30:28 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 0DCCE3857C7F for ; Tue, 15 Oct 2024 03:30:27 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.15]) by sourceware.org (Postfix) with ESMTPS id 150C43858D28 for ; Tue, 15 Oct 2024 03:29:58 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 150C43858D28 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=intel.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 150C43858D28 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=192.198.163.15 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1728963004; cv=none; b=ONQ2s1PB0B9qIjm9GDZqRQ6MPLJUx9bOfZhz0Fnawg8bjLF0yym08k2ATkXfdbCXu5cH+HkKnTahqmJRMVL/BzuBT+v385HI1srFs/TWm7GGkeE3cAMHYPiLER/J4Aa+dX6Q2tItWberPvNBm3FetPZcTrC7Mleh9p55tcxm1vo= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1728963004; c=relaxed/simple; bh=ZclcXPJcP0YpKu0Oy5OEhSU3Od+lqBvRXJ6cGKBuw3Y=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=UYdl60LUxGolKedlqhMgfGvrYRRQe9WTziBoyco8+60E+rKQJqv6rV4umIN09h+E/XYa/NR48h+ZinYShEHF3S309nFzgJ8d+svqBFu2HjDZ6CdEbyP6zjcqAo2xW/PQMUuYZ4asvZrvXa27O0UoG+l2c2bS55Uf37wsPK3EiRU= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1728962999; x=1760498999; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZclcXPJcP0YpKu0Oy5OEhSU3Od+lqBvRXJ6cGKBuw3Y=; b=RbEKBPzGac8qd8WVkBNHRYiCpTWUsm21xKMDqM2Qta7buaj52MZ+iP6D h5LrHUbzv1rn8XCaa5Kl5szphiR9CQCzOfhfhjE8brmgdv/e+eoQLT11l HHmvclkrqNG2WgXUNpBeCAxJs3sp2xFLadrpWBgFVpyc93o5AXWojKFGt woBPPs6zuIBNUktuz9Fd8VcUPF3Vml0go5M7H0h9eXkCHSXyGjuZIiPmF YoW8LsukrSmZlch5wSpaJWk4ZW3bdSK9at68tT7T/6N6dURfZ74ssj8Uc NKxN2/YYQkcVoYQC3bVJ5nnrELKYOoVOCw6Ovi8+kQ3SpHfRKyy9Mhmhg A==; X-CSE-ConnectionGUID: ONTWwQQ+RoKd5vb2xPkOfw== X-CSE-MsgGUID: cKD7dPodTnyLZj5170mGpw== X-IronPort-AV: E=McAfee;i="6700,10204,11225"; a="28465090" X-IronPort-AV: E=Sophos;i="6.11,204,1725346800"; d="scan'208";a="28465090" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by fmvoesa109.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Oct 2024 20:29:58 -0700 X-CSE-ConnectionGUID: LXsm0kFETR6Dz7OpRSmpOg== X-CSE-MsgGUID: h/rx8fovSk+w6bwpIQU8+w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,204,1725346800"; d="scan'208";a="82381270" Received: from shliclel4217.sh.intel.com ([10.239.240.127]) by fmviesa004.fm.intel.com with ESMTP; 14 Oct 2024 20:29:57 -0700 From: liuhongt To: gcc-patches@gcc.gnu.org Cc: segher@kernel.crashing.org, jeffreyalaw@gmail.com Subject: [PATCH 1/2] [Middle-end] Canonicalize (vec_merge (fma op2 op1 op3) op1 mask) to (vec_merge (fma op1 op2 op3) op1 mask). Date: Tue, 15 Oct 2024 11:29:54 +0800 Message-Id: <20241015032955.3677006-2-hongtao.liu@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20241015032955.3677006-1-hongtao.liu@intel.com> References: <7p8r18n6-668q-37pr-7o70-74rr02673919@fhfr.qr> <20241015032955.3677006-1-hongtao.liu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.2 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org For x86 masked fma, there're 2 rtl representations 1) (vec_merge (fma op2 op1 op3) op1 mask) 2) (vec_merge (fma op1 op2 op3) op1 mask). 5894(define_insn "_fmadd__mask" 5895 [(set (match_operand:VFH_AVX512VL 0 "register_operand" "=v,v") 5896 (vec_merge:VFH_AVX512VL 5897 (fma:VFH_AVX512VL 5898 (match_operand:VFH_AVX512VL 1 "nonimmediate_operand" "0,0") 5899 (match_operand:VFH_AVX512VL 2 "" ",v") 5900 (match_operand:VFH_AVX512VL 3 "" "v,")) 5901 (match_dup 1) 5902 (match_operand: 4 "register_operand" "Yk,Yk")))] 5903 "TARGET_AVX512F && " 5904 "@ 5905 vfmadd132\t{%2, %3, %0%{%4%}|%0%{%4%}, %3, %2} 5906 vfmadd213\t{%3, %2, %0%{%4%}|%0%{%4%}, %2, %3}" 5907 [(set_attr "type" "ssemuladd") 5908 (set_attr "prefix" "evex") 5909 (set_attr "mode" "")]) Here op1 has constraint "0", and the scecond op1 is (match_dup 1), we once tried to replace it with (match_operand:M 5 "nonimmediate_operand" "0")) to enable more flexibility for pattern match and recog, but it triggered an ICE in reload(reload can handle at most one perand with "0" constraint). So we need either add 2 patterns in the backend or just do the canonicalization in the middle-end. gcc/ChangeLog: * combine.cc (maybe_swap_commutative_operands): Canonicalize (vec_merge (fma op2 op1 op3) op1 mask) to (vec_merge (fma op1 op2 op3) op1 mask). --- gcc/combine.cc | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/gcc/combine.cc b/gcc/combine.cc index fef06a6cdc0..aa40fdcc50d 100644 --- a/gcc/combine.cc +++ b/gcc/combine.cc @@ -5656,6 +5656,31 @@ maybe_swap_commutative_operands (rtx x) SUBST (XEXP (x, 1), temp); } + /* Canonicalize (vec_merge (fma op2 op1 op3) op1 mask) to + (vec_merge (fma op1 op2 op3) op1 mask). */ + if (GET_CODE (x) == VEC_MERGE + && GET_CODE (XEXP (x, 0)) == FMA) + { + rtx fma_op1 = XEXP (XEXP (x, 0), 0); + rtx fma_op2 = XEXP (XEXP (x, 0), 1); + rtx masked_op = XEXP (x, 1); + if (rtx_equal_p (masked_op, fma_op2)) + { + if (GET_CODE (fma_op1) == NEG) + { + fma_op1 = XEXP (fma_op1, 0); + SUBST (XEXP (XEXP (XEXP (x, 0), 0), 0), fma_op2); + SUBST (XEXP (XEXP (x, 0), 1), fma_op1); + } + else + { + SUBST (XEXP (XEXP (x, 0), 0), fma_op2); + SUBST (XEXP (XEXP (x, 0), 1), fma_op1); + } + + } + } + unsigned n_elts = 0; if (GET_CODE (x) == VEC_MERGE && CONST_INT_P (XEXP (x, 2)) From patchwork Tue Oct 15 03:29:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liu, Hongtao" X-Patchwork-Id: 1997175 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=aXgUGpOv; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XSKP81LTKz1xv6 for ; Tue, 15 Oct 2024 14:30:34 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id CE08B3858C53 for ; Tue, 15 Oct 2024 03:30:32 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.15]) by sourceware.org (Postfix) with ESMTPS id 903C23858D33 for ; Tue, 15 Oct 2024 03:30:00 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 903C23858D33 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=intel.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 903C23858D33 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=192.198.163.15 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1728963004; cv=none; b=oaYuIIf59RA3ekzxOrvRCY9Y8AatOeWl60p9JfDMlW2vBWoC/R8u6CoVXq246hr0L01u1usftAWx5rxkv53STMx2qTUYWPEHL0iQ/t0VycAO1XlqlGVhooAjG1sEYQkf62jm2CFGmAeQXHfypKZ/7Ln2nbEw6yMmjOYmeQOmXAQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1728963004; c=relaxed/simple; bh=G7wXm7z78fJ26aGo5RU6en/S9R7I2EgAQbimxnuq/es=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=nvoE/kdo8goa4/esVYBs1gLzjzbRwYB6HjsIF3TMldVsxngYEB5v7wn3p2YeJe7uJzd9CN3wkT/4EFNvD3W32PvwaQmb1BRKxy7c6xn4phLxA/y8wbBvhcNxPMvzCXBPq9MOXy/f2GXeyEjwPnawK5ifyJajJIvSjdXgHPleP60= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1728963000; x=1760499000; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=G7wXm7z78fJ26aGo5RU6en/S9R7I2EgAQbimxnuq/es=; b=aXgUGpOvjTYwyMVoBCR5UBEuEDQetAoLMYCXfgbfIaQvR+ApuuPqmnnM hNtOq72wfD3Wdd+Mm1wUHiM1Zu8T4ntuCdK7lHaBeNgyUdtu2I4A0uOwx Eh00u9cVLvAIYxPYsU7qtCO/CDiEbPy2ANSRal1lzoP2SfnZgWAcCQ1hK 57eYGMVWuP3biDWrn4ht0zWwyyFD/qy2YDNJuKkbbaPTvnYyzu7xmHm3r h4e1Uj8z3BQv2Ecc62X5mhk6vu5nIadWEPSxH8ylNTNt9FDCcmsKZfG6x BeC0wy5NBAXzra96FlfFUnXnyOrQ7xzmhEYvRAx9VvMiBiUnME/gsP1UE g==; X-CSE-ConnectionGUID: nYPwVQiPQ4a40q3mTiDQRg== X-CSE-MsgGUID: u9xjoc7hSZKEzUOXZRx3WQ== X-IronPort-AV: E=McAfee;i="6700,10204,11225"; a="28465098" X-IronPort-AV: E=Sophos;i="6.11,204,1725346800"; d="scan'208";a="28465098" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by fmvoesa109.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Oct 2024 20:29:59 -0700 X-CSE-ConnectionGUID: MNVfrlAxQQSxbshToO4YGw== X-CSE-MsgGUID: Zv6dmxV2TSafSWHjizWMHQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,204,1725346800"; d="scan'208";a="82381280" Received: from shliclel4217.sh.intel.com ([10.239.240.127]) by fmviesa004.fm.intel.com with ESMTP; 14 Oct 2024 20:29:58 -0700 From: liuhongt To: gcc-patches@gcc.gnu.org Cc: segher@kernel.crashing.org, jeffreyalaw@gmail.com Subject: [PATCH 2/2] [x86] Canonicalize (vec_merge (fma: op2 op1 op3) (match_dup 1)) mask) to (vec_merge (fma: op1 op2 op3) (match_dup 1)) mask) Date: Tue, 15 Oct 2024 11:29:55 +0800 Message-Id: <20241015032955.3677006-3-hongtao.liu@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20241015032955.3677006-1-hongtao.liu@intel.com> References: <7p8r18n6-668q-37pr-7o70-74rr02673919@fhfr.qr> <20241015032955.3677006-1-hongtao.liu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.2 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org For masked FMA, there're 2 forms of RTL representation 1) (vec_merge (fma: op2 op1 op3) op1) mask) 2) (vec_merge (fma: op1 op2 op3) op1) mask) It's because op1 op2 are communatative in RTL(the second op1 is written as (match_dup 1)) we once tried to replace (match_dup 1) with (match_operand:VFH_AVX512VL 5 "nonimmediate_operand" "0,0")), but trigger an ICE in reload(reload can handle at most one operand with "0" constraint). So the patch do the canonicalizaton for the backend part. gcc/ChangeLog: PR target/117072 (_fmadd__mask): Relax predicates of fma operands from register_operand to nonimmediate_operand (_fmadd__mask3): Ditto. (_fmsub__mask): Ditto. (_fmsub__mask3): Ditto. (_fnmadd__mask): Ditto. (_fnmadd__mask3): Ditto. (_fnmsub__mask): Ditto. (_fnmsub__mask3): Ditto. (_fmaddsub__mask3): Ditto. (_fmsubadd__mask): Ditto. (_fmsubadd__mask3): Ditto. (avx512f_vmfmadd__mask): Ditto. (avx512f_vmfmadd__mask3): Ditto. (avx512f_vmfmadd__maskz_1): Ditto. (*avx512f_vmfmsub__mask): Ditto. (avx512f_vmfmsub__mask3): Ditto. (*avx512f_vmfmsub__maskz_1): Ditto. (avx512f_vmfnmadd__mask): Ditto. (avx512f_vmfnmadd__mask3): Ditto. (avx512f_vmfnmadd__maskz_1): Ditto. (*avx512f_vmfnmsub__mask): Ditto. (*avx512f_vmfnmsub__mask3): Ditto. (*avx512f_vmfnmsub__maskz_1): Ditto. (avx10_2_fmaddnepbf16__mask3): Ditto. (avx10_2_fnmaddnepbf16__mask3): Ditto. (avx10_2_fmsubnepbf16__mask3): Ditto. (avx10_2_fnmsubnepbf16__mask3): Ditto. (fmai_vmfmadd_): Swap operands[1] and operands[2]. (fmai_vmfmsub_): Ditto. (fmai_vmfnmadd_): Ditto. (fmai_vmfnmsub_): Ditto. (*fmai_fmadd_): Swap operands[1] and operands[2] adjust operands[1] predicates from register_operand to nonimmediate_operand. (*fmai_fmsub_): Ditto. (*fmai_fnmadd_): Ditto. (*fmai_fnmsub_): Ditto. --- gcc/config/i386/sse.md | 86 +++++++++++++++++++++--------------------- 1 file changed, 43 insertions(+), 43 deletions(-) diff --git a/gcc/config/i386/sse.md b/gcc/config/i386/sse.md index a45b50ad732..9201b1a0782 100644 --- a/gcc/config/i386/sse.md +++ b/gcc/config/i386/sse.md @@ -5895,7 +5895,7 @@ (define_insn "_fmadd__mask" [(set (match_operand:VFH_AVX512VL 0 "register_operand" "=v,v") (vec_merge:VFH_AVX512VL (fma:VFH_AVX512VL - (match_operand:VFH_AVX512VL 1 "register_operand" "0,0") + (match_operand:VFH_AVX512VL 1 "nonimmediate_operand" "0,0") (match_operand:VFH_AVX512VL 2 "" ",v") (match_operand:VFH_AVX512VL 3 "" "v,")) (match_dup 1) @@ -5914,7 +5914,7 @@ (define_insn "_fmadd__mask3" (fma:VFH_AVX512VL (match_operand:VFH_AVX512VL 1 "" "%v") (match_operand:VFH_AVX512VL 2 "" "") - (match_operand:VFH_AVX512VL 3 "register_operand" "0")) + (match_operand:VFH_AVX512VL 3 "nonimmediate_operand" "0")) (match_dup 3) (match_operand: 4 "register_operand" "Yk")))] "TARGET_AVX512F && " @@ -5999,7 +5999,7 @@ (define_insn "_fmsub__mask" [(set (match_operand:VFH_AVX512VL 0 "register_operand" "=v,v") (vec_merge:VFH_AVX512VL (fma:VFH_AVX512VL - (match_operand:VFH_AVX512VL 1 "register_operand" "0,0") + (match_operand:VFH_AVX512VL 1 "nonimmediate_operand" "0,0") (match_operand:VFH_AVX512VL 2 "" ",v") (neg:VFH_AVX512VL (match_operand:VFH_AVX512VL 3 "" "v,"))) @@ -6020,7 +6020,7 @@ (define_insn "_fmsub__mask3" (match_operand:VFH_AVX512VL 1 "" "%v") (match_operand:VFH_AVX512VL 2 "" "") (neg:VFH_AVX512VL - (match_operand:VFH_AVX512VL 3 "register_operand" "0"))) + (match_operand:VFH_AVX512VL 3 "nonimmediate_operand" "0"))) (match_dup 3) (match_operand: 4 "register_operand" "Yk")))] "TARGET_AVX512F && " @@ -6106,7 +6106,7 @@ (define_insn "_fnmadd__mask" (vec_merge:VFH_AVX512VL (fma:VFH_AVX512VL (neg:VFH_AVX512VL - (match_operand:VFH_AVX512VL 1 "register_operand" "0,0")) + (match_operand:VFH_AVX512VL 1 "nonimmediate_operand" "0,0")) (match_operand:VFH_AVX512VL 2 "" ",v") (match_operand:VFH_AVX512VL 3 "" "v,")) (match_dup 1) @@ -6126,7 +6126,7 @@ (define_insn "_fnmadd__mask3" (neg:VFH_AVX512VL (match_operand:VFH_AVX512VL 1 "" "%v")) (match_operand:VFH_AVX512VL 2 "" "") - (match_operand:VFH_AVX512VL 3 "register_operand" "0")) + (match_operand:VFH_AVX512VL 3 "nonimmediate_operand" "0")) (match_dup 3) (match_operand: 4 "register_operand" "Yk")))] "TARGET_AVX512F && " @@ -6215,7 +6215,7 @@ (define_insn "_fnmsub__mask" (vec_merge:VFH_AVX512VL (fma:VFH_AVX512VL (neg:VFH_AVX512VL - (match_operand:VFH_AVX512VL 1 "register_operand" "0,0")) + (match_operand:VFH_AVX512VL 1 "nonimmediate_operand" "0,0")) (match_operand:VFH_AVX512VL 2 "" ",v") (neg:VFH_AVX512VL (match_operand:VFH_AVX512VL 3 "" "v,"))) @@ -6237,7 +6237,7 @@ (define_insn "_fnmsub__mask3" (match_operand:VFH_AVX512VL 1 "" "%v")) (match_operand:VFH_AVX512VL 2 "" "") (neg:VFH_AVX512VL - (match_operand:VFH_AVX512VL 3 "register_operand" "0"))) + (match_operand:VFH_AVX512VL 3 "nonimmediate_operand" "0"))) (match_dup 3) (match_operand: 4 "register_operand" "Yk")))] "TARGET_AVX512F && " @@ -6369,9 +6369,9 @@ (define_insn "_fmaddsub__mask3" [(set (match_operand:VFH_AVX512VL 0 "register_operand" "=v") (vec_merge:VFH_AVX512VL (unspec:VFH_AVX512VL - [(match_operand:VFH_AVX512VL 1 "register_operand" "v") + [(match_operand:VFH_AVX512VL 1 "nonimmediate_operand" "v") (match_operand:VFH_AVX512VL 2 "" "") - (match_operand:VFH_AVX512VL 3 "register_operand" "0")] + (match_operand:VFH_AVX512VL 3 "nonimmediate_operand" "0")] UNSPEC_FMADDSUB) (match_dup 3) (match_operand: 4 "register_operand" "Yk")))] @@ -6421,7 +6421,7 @@ (define_insn "_fmsubadd__mask" [(set (match_operand:VFH_AVX512VL 0 "register_operand" "=v,v") (vec_merge:VFH_AVX512VL (unspec:VFH_AVX512VL - [(match_operand:VFH_AVX512VL 1 "register_operand" "0,0") + [(match_operand:VFH_AVX512VL 1 "nonimmediate_operand" "0,0") (match_operand:VFH_AVX512VL 2 "" ",v") (neg:VFH_AVX512VL (match_operand:VFH_AVX512VL 3 "" "v,"))] @@ -6440,10 +6440,10 @@ (define_insn "_fmsubadd__mask3" [(set (match_operand:VFH_AVX512VL 0 "register_operand" "=v") (vec_merge:VFH_AVX512VL (unspec:VFH_AVX512VL - [(match_operand:VFH_AVX512VL 1 "register_operand" "v") + [(match_operand:VFH_AVX512VL 1 "nonimmediate_operand" "v") (match_operand:VFH_AVX512VL 2 "" "") (neg:VFH_AVX512VL - (match_operand:VFH_AVX512VL 3 "register_operand" "0"))] + (match_operand:VFH_AVX512VL 3 "nonimmediate_operand" "0"))] UNSPEC_FMADDSUB) (match_dup 3) (match_operand: 4 "register_operand" "Yk")))] @@ -6460,7 +6460,7 @@ (define_expand "fmai_vmfmadd_" [(set (match_operand:VFH_128 0 "register_operand") (vec_merge:VFH_128 (fma:VFH_128 - (match_operand:VFH_128 1 "register_operand") + (match_operand:VFH_128 1 "nonimmediate_operand") (match_operand:VFH_128 2 "") (match_operand:VFH_128 3 "")) (match_dup 1) @@ -6471,7 +6471,7 @@ (define_expand "fmai_vmfmsub_" [(set (match_operand:VFH_128 0 "register_operand") (vec_merge:VFH_128 (fma:VFH_128 - (match_operand:VFH_128 1 "register_operand") + (match_operand:VFH_128 1 "nonimmediate_operand") (match_operand:VFH_128 2 "") (neg:VFH_128 (match_operand:VFH_128 3 ""))) @@ -6484,8 +6484,8 @@ (define_expand "fmai_vmfnmadd_" (vec_merge:VFH_128 (fma:VFH_128 (neg:VFH_128 - (match_operand:VFH_128 2 "")) - (match_operand:VFH_128 1 "register_operand") + (match_operand:VFH_128 1 "nonimmediate_operand")) + (match_operand:VFH_128 2 "") (match_operand:VFH_128 3 "")) (match_dup 1) (const_int 1)))] @@ -6496,8 +6496,8 @@ (define_expand "fmai_vmfnmsub_" (vec_merge:VFH_128 (fma:VFH_128 (neg:VFH_128 - (match_operand:VFH_128 2 "")) - (match_operand:VFH_128 1 "register_operand") + (match_operand:VFH_128 1 "nonimmediate_operand")) + (match_operand:VFH_128 2 "") (neg:VFH_128 (match_operand:VFH_128 3 ""))) (match_dup 1) @@ -6508,7 +6508,7 @@ (define_insn "*fmai_fmadd_" [(set (match_operand:VFH_128 0 "register_operand" "=v,v") (vec_merge:VFH_128 (fma:VFH_128 - (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 1 "nonimmediate_operand" "0,0") (match_operand:VFH_128 2 "" ", v") (match_operand:VFH_128 3 "" "v,")) (match_dup 1) @@ -6525,7 +6525,7 @@ (define_insn "*fmai_fmsub_" [(set (match_operand:VFH_128 0 "register_operand" "=v,v") (vec_merge:VFH_128 (fma:VFH_128 - (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 1 "nonimmediate_operand" "0,0") (match_operand:VFH_128 2 "" ",v") (neg:VFH_128 (match_operand:VFH_128 3 "" "v,"))) @@ -6544,8 +6544,8 @@ (define_insn "*fmai_fnmadd_" (vec_merge:VFH_128 (fma:VFH_128 (neg:VFH_128 - (match_operand:VFH_128 2 "" ",v")) - (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 1 "nonimmediate_operand" "0,0")) + (match_operand:VFH_128 2 "" ",v") (match_operand:VFH_128 3 "" "v,")) (match_dup 1) (const_int 1)))] @@ -6562,8 +6562,8 @@ (define_insn "*fmai_fnmsub_" (vec_merge:VFH_128 (fma:VFH_128 (neg:VFH_128 - (match_operand:VFH_128 2 "" ",v")) - (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 1 "nonimmediate_operand" "0,0")) + (match_operand:VFH_128 2 "" ",v") (neg:VFH_128 (match_operand:VFH_128 3 "" "v,"))) (match_dup 1) @@ -6581,7 +6581,7 @@ (define_insn "avx512f_vmfmadd__mask" (vec_merge:VFH_128 (vec_merge:VFH_128 (fma:VFH_128 - (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 1 "nonimmediate_operand" "0,0") (match_operand:VFH_128 2 "" ",v") (match_operand:VFH_128 3 "" "v,")) (match_dup 1) @@ -6603,7 +6603,7 @@ (define_insn "avx512f_vmfmadd__mask3" (fma:VFH_128 (match_operand:VFH_128 1 "" "%v") (match_operand:VFH_128 2 "" "") - (match_operand:VFH_128 3 "register_operand" "0")) + (match_operand:VFH_128 3 "nonimmediate_operand" "0")) (match_dup 3) (match_operand:QI 4 "register_operand" "Yk")) (match_dup 3) @@ -6633,7 +6633,7 @@ (define_insn "avx512f_vmfmadd__maskz_1" (vec_merge:VFH_128 (vec_merge:VFH_128 (fma:VFH_128 - (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 1 "nonimmediate_operand" "0,0") (match_operand:VFH_128 2 "" ",v") (match_operand:VFH_128 3 "" "v,")) (match_operand:VFH_128 4 "const0_operand") @@ -6653,7 +6653,7 @@ (define_insn "*avx512f_vmfmsub__mask" (vec_merge:VFH_128 (vec_merge:VFH_128 (fma:VFH_128 - (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 1 "nonimmediate_operand" "0,0") (match_operand:VFH_128 2 "" ",v") (neg:VFH_128 (match_operand:VFH_128 3 "" "v,"))) @@ -6677,7 +6677,7 @@ (define_insn "avx512f_vmfmsub__mask3" (match_operand:VFH_128 1 "" "%v") (match_operand:VFH_128 2 "" "") (neg:VFH_128 - (match_operand:VFH_128 3 "register_operand" "0"))) + (match_operand:VFH_128 3 "nonimmediate_operand" "0"))) (match_dup 3) (match_operand:QI 4 "register_operand" "Yk")) (match_dup 3) @@ -6693,7 +6693,7 @@ (define_insn "*avx512f_vmfmsub__maskz_1" (vec_merge:VFH_128 (vec_merge:VFH_128 (fma:VFH_128 - (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 1 "nonimmediate_operand" "0,0") (match_operand:VFH_128 2 "" ",v") (neg:VFH_128 (match_operand:VFH_128 3 "" "v,"))) @@ -6715,8 +6715,8 @@ (define_insn "avx512f_vmfnmadd__mask" (vec_merge:VFH_128 (fma:VFH_128 (neg:VFH_128 - (match_operand:VFH_128 2 "" ",v")) - (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 1 "nonimmediate_operand" "0,0")) + (match_operand:VFH_128 2 "" ",v") (match_operand:VFH_128 3 "" "v,")) (match_dup 1) (match_operand:QI 4 "register_operand" "Yk,Yk")) @@ -6738,7 +6738,7 @@ (define_insn "avx512f_vmfnmadd__mask3" (neg:VFH_128 (match_operand:VFH_128 2 "" "")) (match_operand:VFH_128 1 "" "%v") - (match_operand:VFH_128 3 "register_operand" "0")) + (match_operand:VFH_128 3 "nonimmediate_operand" "0")) (match_dup 3) (match_operand:QI 4 "register_operand" "Yk")) (match_dup 3) @@ -6770,7 +6770,7 @@ (define_insn "avx512f_vmfnmadd__maskz_1" (fma:VFH_128 (neg:VFH_128 (match_operand:VFH_128 2 "" ",v")) - (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 1 "nonimmediate_operand" "0,0") (match_operand:VFH_128 3 "" "v,")) (match_operand:VFH_128 4 "const0_operand") (match_operand:QI 5 "register_operand" "Yk,Yk")) @@ -6790,8 +6790,8 @@ (define_insn "*avx512f_vmfnmsub__mask" (vec_merge:VFH_128 (fma:VFH_128 (neg:VFH_128 - (match_operand:VFH_128 2 "" ",v")) - (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 1 "nonimmediate_operand" "0,0")) + (match_operand:VFH_128 2 "" ",v") (neg:VFH_128 (match_operand:VFH_128 3 "" "v,"))) (match_dup 1) @@ -6815,7 +6815,7 @@ (define_insn "*avx512f_vmfnmsub__mask3" (match_operand:VFH_128 2 "" "")) (match_operand:VFH_128 1 "" "%v") (neg:VFH_128 - (match_operand:VFH_128 3 "register_operand" "0"))) + (match_operand:VFH_128 3 "nonimmediate_operand" "0"))) (match_dup 3) (match_operand:QI 4 "register_operand" "Yk")) (match_dup 3) @@ -6833,7 +6833,7 @@ (define_insn "*avx512f_vmfnmsub__maskz_1" (fma:VFH_128 (neg:VFH_128 (match_operand:VFH_128 2 "" ",v")) - (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 1 "nonimmediate_operand" "0,0") (neg:VFH_128 (match_operand:VFH_128 3 "" "v,"))) (match_operand:VFH_128 4 "const0_operand") @@ -32048,7 +32048,7 @@ (define_insn "avx10_2_fmaddnepbf16__mask3" (fma:VBF_AVX10_2 (match_operand:VBF_AVX10_2 1 "nonimmediate_operand" "%v") (match_operand:VBF_AVX10_2 2 "nonimmediate_operand" "vm") - (match_operand:VBF_AVX10_2 3 "register_operand" "0")) + (match_operand:VBF_AVX10_2 3 "nonimmediate_operand" "0")) (match_dup 3) (match_operand: 4 "register_operand" "Yk")))] "TARGET_AVX10_2_256" @@ -32113,7 +32113,7 @@ (define_insn "avx10_2_fnmaddnepbf16__mask3" (neg:VBF_AVX10_2 (match_operand:VBF_AVX10_2 1 "nonimmediate_operand" "%v")) (match_operand:VBF_AVX10_2 2 "nonimmediate_operand" "vm") - (match_operand:VBF_AVX10_2 3 "register_operand" "0")) + (match_operand:VBF_AVX10_2 3 "nonimmediate_operand" "0")) (match_dup 3) (match_operand: 4 "register_operand" "Yk")))] "TARGET_AVX10_2_256" @@ -32178,7 +32178,7 @@ (define_insn "avx10_2_fmsubnepbf16__mask3" (match_operand:VBF_AVX10_2 1 "nonimmediate_operand" "%v") (match_operand:VBF_AVX10_2 2 "nonimmediate_operand" "vm") (neg:VBF_AVX10_2 - (match_operand:VBF_AVX10_2 3 "register_operand" "0"))) + (match_operand:VBF_AVX10_2 3 "nonimmediate_operand" "0"))) (match_dup 3) (match_operand: 4 "register_operand" "Yk")))] "TARGET_AVX10_2_256" @@ -32246,7 +32246,7 @@ (define_insn "avx10_2_fnmsubnepbf16__mask3" (match_operand:VBF_AVX10_2 1 "nonimmediate_operand" "%v")) (match_operand:VBF_AVX10_2 2 "nonimmediate_operand" "vm") (neg:VBF_AVX10_2 - (match_operand:VBF_AVX10_2 3 "register_operand" "0"))) + (match_operand:VBF_AVX10_2 3 "nonimmediate_operand" "0"))) (match_dup 3) (match_operand: 4 "register_operand" "Yk")))] "TARGET_AVX10_2_256"