From patchwork Fri Oct 18 14:22:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robin Dapp X-Patchwork-Id: 1999202 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20230601 header.b=IXUFGVHC; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XVRlD2025z1xwD for ; Sat, 19 Oct 2024 01:23:36 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 7A4C63857C4F for ; Fri, 18 Oct 2024 14:23:34 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) by sourceware.org (Postfix) with ESMTPS id 352893858401 for ; Fri, 18 Oct 2024 14:22:33 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 352893858401 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 352893858401 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a00:1450:4864:20::629 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729261362; cv=none; b=S3OKa/VC0/+wSFg2mEpID0txzpsLOw70VaOl7cY1qxfKFVocEfCxxnQRCpKiIzXc0dQPPpZINK4xxfSCnQh8vqC5wjvXdCdZ5X2E3d78qWcNlRbSQp2Aqr14+v1E5qanMeJEJXwYcX0BDzNfGNuYZ4/OsruSg7fpABzNifeYnA4= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1729261362; c=relaxed/simple; bh=mLb2jSsfmmQ5Gvy/MjJAfx5FRQxjy6KevEOR7JlTwpw=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=Oum0NrvNmVyV8wAwuwOkysEdGxWeXrbDri3L1jd9S0kT5rsxLl1K611wXGzmp0OttnM/4sy232PVlz4RfKdKvIoJNBBSWeSxt7LhpCNoK6bEflzhDXWulRjmU8iPcb8Oexeq8al4XlnkNnBi0leztgTvrFhytI6C7DyhtE7//Ck= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-ej1-x629.google.com with SMTP id a640c23a62f3a-a9a1b71d7ffso307239166b.1 for ; Fri, 18 Oct 2024 07:22:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729261351; x=1729866151; darn=gcc.gnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zr0mtnAPYzL1RfI8/t1hQILRzJ3B0W5QMPOiZNhaoCY=; b=IXUFGVHC5X5JZd/rztg1vweP/6rbW1keLTKJe4fi6tINxtdz49wJ+37Koebv6n7+D8 /Enu3a3TuEKljKIskwKKVCNk/AfK+D/9fKC97j5/aJoLLtqYqfVVvTWM1QLxlphemo7S eVwApLHt2LS9TaCxT2LTsYCDvD8J709cCuQycfV4phHIS5hobMJ34RNOH02LS81yFlWK 4ozFcIAEMvAIOdXEfn30A7AWWT3Uv00ML7FfsHbu31BATRbPSUw0Oa1lYCrp7EskynLK nWIHVotvwHST2wYBAsxq/RBB/Knt1EpgB02xzKDJJmVjZVFyZ5GKdq9I8F1lxfEe6h6h aCtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729261351; x=1729866151; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zr0mtnAPYzL1RfI8/t1hQILRzJ3B0W5QMPOiZNhaoCY=; b=OiOsLiMZ7m1w44AxZ9x8uNMspR9fluZ/WEtmBSOKm66NFqsWUm17oVdPcfRIE888ao FId8DWvqgTtvomhNGwtCwBvH79L05lVvNd61/+GgZKXJj5OLKP1x8uMV2xNaiLtM+Lki 9Tjp6XtYKdkjn0egLq4oIYTaGXYSLPkOEc8zuKnI5VTP52Pv2ZQjrtGTEdPR4WX0knZd Dkr1i6cZ42UeDTmzePqizve6ElLhRBgsTQ/CynvpkHqTOMEtipHHX9Fqy5hmE1IKkQtt 49yZJVFu7vOSPEpYtTDczxR0IRxpcV3AmX8iH4qr2AgOyS7+h8dDjgyAKwvHIE2EUcXp ZtaA== X-Gm-Message-State: AOJu0YwlZ8tQ1NH2UE9vP32YK/CRb0dtqewyf3SQjCaT3POP2NMEurAQ VloPc0TWvKNbnh7VsQkUCWwnh0D55MxbQTewCuds2K1ZrwVGYzAi6U50EA== X-Google-Smtp-Source: AGHT+IFmNEIWy/GnB8fIwAu/+PvlLY0oIaaCTdhsSoGioKbG0pxg/qOse7W7GOBwhdaLNKpmFMTRxQ== X-Received: by 2002:a17:907:728c:b0:a9a:6855:1820 with SMTP id a640c23a62f3a-a9a69a7616cmr254291466b.15.1729261349304; Fri, 18 Oct 2024 07:22:29 -0700 (PDT) Received: from x1c10.dc1.ventanamicro.com (ip-149-172-150-237.um42.pools.vodafone-ip.de. [149.172.150.237]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9a68c2677esm102812166b.188.2024.10.18.07.22.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Oct 2024 07:22:28 -0700 (PDT) From: Robin Dapp X-Google-Original-From: Robin Dapp To: gcc-patches@gcc.gnu.org Cc: rdapp.gcc@gmail.com, rguenther@suse.de, richard.sandiford@arm.com, jeffreyalaw@gmail.com, ams@baylibre.com Subject: [PATCH v2 7/8] i386: Add else operand to masked loads. Date: Fri, 18 Oct 2024 16:22:19 +0200 Message-ID: <20241018142220.173482-8-rdapp@ventanamicro.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241018142220.173482-1-rdapp@ventanamicro.com> References: <20241018142220.173482-1-rdapp@ventanamicro.com> MIME-Version: 1.0 X-Spam-Status: No, score=-9.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org This patch adds a zero else operand to masked loads, in particular the masked gather load builtins that are used for gather vectorization. gcc/ChangeLog: * config/i386/i386-expand.cc (ix86_expand_special_args_builtin): Add else-operand handling. (ix86_expand_builtin): Ditto. * config/i386/predicates.md (vcvtne2ps2bf_parallel): New predicate. (maskload_else_operand): Ditto. * config/i386/sse.md: Use predicate. --- gcc/config/i386/i386-expand.cc | 26 +++++-- gcc/config/i386/predicates.md | 4 ++ gcc/config/i386/sse.md | 124 ++++++++++++++++++++------------- 3 files changed, 101 insertions(+), 53 deletions(-) diff --git a/gcc/config/i386/i386-expand.cc b/gcc/config/i386/i386-expand.cc index 63f5e348d64..f6a2c2d65b8 100644 --- a/gcc/config/i386/i386-expand.cc +++ b/gcc/config/i386/i386-expand.cc @@ -12994,10 +12994,11 @@ ix86_expand_special_args_builtin (const struct builtin_description *d, { tree arg; rtx pat, op; - unsigned int i, nargs, arg_adjust, memory; + unsigned int i, nargs, arg_adjust, memory = -1; unsigned int constant = 100; bool aligned_mem = false; - rtx xops[4]; + rtx xops[4] = {}; + bool add_els = false; enum insn_code icode = d->icode; const struct insn_data_d *insn_p = &insn_data[icode]; machine_mode tmode = insn_p->operand[0].mode; @@ -13124,6 +13125,9 @@ ix86_expand_special_args_builtin (const struct builtin_description *d, case V4DI_FTYPE_PCV4DI_V4DI: case V4SI_FTYPE_PCV4SI_V4SI: case V2DI_FTYPE_PCV2DI_V2DI: + /* Two actual args but an additional else operand. */ + add_els = true; + /* Fallthru. */ case VOID_FTYPE_INT_INT64: nargs = 2; klass = load; @@ -13396,6 +13400,12 @@ ix86_expand_special_args_builtin (const struct builtin_description *d, xops[i]= op; } + if (add_els) + { + xops[i] = CONST0_RTX (GET_MODE (xops[0])); + nargs++; + } + switch (nargs) { case 0: @@ -13652,7 +13662,7 @@ ix86_expand_builtin (tree exp, rtx target, rtx subtarget, enum insn_code icode, icode2; tree fndecl = TREE_OPERAND (CALL_EXPR_FN (exp), 0); tree arg0, arg1, arg2, arg3, arg4; - rtx op0, op1, op2, op3, op4, pat, pat2, insn; + rtx op0, op1, op2, op3, op4, opels, pat, pat2, insn; machine_mode mode0, mode1, mode2, mode3, mode4; unsigned int fcode = DECL_MD_FUNCTION_CODE (fndecl); HOST_WIDE_INT bisa, bisa2; @@ -15559,12 +15569,15 @@ rdseed_step: op3 = copy_to_reg (op3); op3 = lowpart_subreg (mode3, op3, GET_MODE (op3)); } + if (!insn_data[icode].operand[5].predicate (op4, mode4)) { - error ("the last argument must be scale 1, 2, 4, 8"); - return const0_rtx; + error ("the last argument must be scale 1, 2, 4, 8"); + return const0_rtx; } + opels = CONST0_RTX (GET_MODE (subtarget)); + /* Optimize. If mask is known to have all high bits set, replace op0 with pc_rtx to signal that the instruction overwrites the whole destination and doesn't use its @@ -15633,7 +15646,8 @@ rdseed_step: } } - pat = GEN_FCN (icode) (subtarget, op0, op1, op2, op3, op4); + pat = GEN_FCN (icode) (subtarget, op0, op1, op2, op3, op4, opels); + if (! pat) return const0_rtx; emit_insn (pat); diff --git a/gcc/config/i386/predicates.md b/gcc/config/i386/predicates.md index 053312bbe27..7c7d8f61f11 100644 --- a/gcc/config/i386/predicates.md +++ b/gcc/config/i386/predicates.md @@ -2346,3 +2346,7 @@ (define_predicate "apx_evex_add_memory_operand" return true; }) + +(define_predicate "maskload_else_operand" + (and (match_code "const_int,const_vector") + (match_test "op == CONST0_RTX (GET_MODE (op))"))) diff --git a/gcc/config/i386/sse.md b/gcc/config/i386/sse.md index a45b50ad732..83955eee5a0 100644 --- a/gcc/config/i386/sse.md +++ b/gcc/config/i386/sse.md @@ -1575,7 +1575,8 @@ (define_expand "_load_mask" } else if (MEM_P (operands[1])) operands[1] = gen_rtx_UNSPEC (mode, - gen_rtvec(1, operands[1]), + gen_rtvec(2, operands[1], + CONST0_RTX (mode)), UNSPEC_MASKLOAD); }) @@ -1583,7 +1584,8 @@ (define_insn "*_load_mask" [(set (match_operand:V48_AVX512VL 0 "register_operand" "=v") (vec_merge:V48_AVX512VL (unspec:V48_AVX512VL - [(match_operand:V48_AVX512VL 1 "memory_operand" "m")] + [(match_operand:V48_AVX512VL 1 "memory_operand" "m") + (match_operand:V48_AVX512VL 4 "maskload_else_operand")] UNSPEC_MASKLOAD) (match_operand:V48_AVX512VL 2 "nonimm_or_0_operand" "0C") (match_operand: 3 "register_operand" "Yk")))] @@ -1611,7 +1613,8 @@ (define_insn "*_load_mask" (define_insn_and_split "*_load" [(set (match_operand:V48_AVX512VL 0 "register_operand") (unspec:V48_AVX512VL - [(match_operand:V48_AVX512VL 1 "memory_operand")] + [(match_operand:V48_AVX512VL 1 "memory_operand") + (match_operand:V48_AVX512VL 2 "maskload_else_operand")] UNSPEC_MASKLOAD))] "TARGET_AVX512F" "#" @@ -1633,7 +1636,8 @@ (define_expand "_load_mask" } else if (MEM_P (operands[1])) operands[1] = gen_rtx_UNSPEC (mode, - gen_rtvec(1, operands[1]), + gen_rtvec(2, operands[1], + CONST0_RTX (mode)), UNSPEC_MASKLOAD); }) @@ -1642,7 +1646,8 @@ (define_insn "*_load_mask" [(set (match_operand:VI12HFBF_AVX512VL 0 "register_operand" "=v") (vec_merge:VI12HFBF_AVX512VL (unspec:VI12HFBF_AVX512VL - [(match_operand:VI12HFBF_AVX512VL 1 "memory_operand" "m")] + [(match_operand:VI12HFBF_AVX512VL 1 "memory_operand" "m") + (match_operand:VI12HFBF_AVX512VL 4 "maskload_else_operand")] UNSPEC_MASKLOAD) (match_operand:VI12HFBF_AVX512VL 2 "nonimm_or_0_operand" "0C") (match_operand: 3 "register_operand" "Yk")))] @@ -1655,7 +1660,8 @@ (define_insn "*_load_mask" (define_insn_and_split "*_load" [(set (match_operand:VI12HFBF_AVX512VL 0 "register_operand" "=v") (unspec:VI12HFBF_AVX512VL - [(match_operand:VI12HFBF_AVX512VL 1 "memory_operand" "m")] + [(match_operand:VI12HFBF_AVX512VL 1 "memory_operand" "m") + (match_operand:VI12HFBF_AVX512VL 2 "maskload_else_operand")] UNSPEC_MASKLOAD))] "TARGET_AVX512BW" "#" @@ -28586,7 +28592,8 @@ (define_insn "_maskload" [(set (match_operand:V48_128_256 0 "register_operand" "=x") (unspec:V48_128_256 [(match_operand: 2 "register_operand" "x") - (match_operand:V48_128_256 1 "memory_operand" "jm")] + (match_operand:V48_128_256 1 "memory_operand" "jm") + (match_operand:V48_128_256 3 "maskload_else_operand")] UNSPEC_MASKMOV))] "TARGET_AVX" { @@ -28627,7 +28634,8 @@ (define_expand "maskload" [(set (match_operand:V48_128_256 0 "register_operand") (unspec:V48_128_256 [(match_operand: 2 "register_operand") - (match_operand:V48_128_256 1 "memory_operand")] + (match_operand:V48_128_256 1 "memory_operand") + (match_operand:V48_128_256 3 "maskload_else_operand")] UNSPEC_MASKMOV))] "TARGET_AVX") @@ -28635,20 +28643,24 @@ (define_expand "maskload" [(set (match_operand:V48_AVX512VL 0 "register_operand") (vec_merge:V48_AVX512VL (unspec:V48_AVX512VL - [(match_operand:V48_AVX512VL 1 "memory_operand")] + [(match_operand:V48_AVX512VL 1 "memory_operand") + (match_operand:V48_AVX512VL 3 "maskload_else_operand")] UNSPEC_MASKLOAD) (match_dup 0) - (match_operand: 2 "register_operand")))] + (match_operand: 2 "register_operand"))) + ] "TARGET_AVX512F") (define_expand "maskload" [(set (match_operand:VI12HFBF_AVX512VL 0 "register_operand") (vec_merge:VI12HFBF_AVX512VL (unspec:VI12HFBF_AVX512VL - [(match_operand:VI12HFBF_AVX512VL 1 "memory_operand")] + [(match_operand:VI12HFBF_AVX512VL 1 "memory_operand") + (match_operand:VI12HFBF_AVX512VL 3 "maskload_else_operand")] UNSPEC_MASKLOAD) (match_dup 0) - (match_operand: 2 "register_operand")))] + (match_operand: 2 "register_operand"))) + ] "TARGET_AVX512BW") (define_expand "maskstore" @@ -29214,20 +29226,22 @@ (define_expand "avx2_gathersi" (unspec:VEC_GATHER_MODE [(match_operand:VEC_GATHER_MODE 1 "register_operand") (mem: - (match_par_dup 6 + (match_par_dup 7 [(match_operand 2 "vsib_address_operand") (match_operand: 3 "register_operand") - (match_operand:SI 5 "const1248_operand ")])) + (match_operand:SI 5 "const1248_operand ") + (match_operand:VEC_GATHER_MODE 6 "maskload_else_operand")])) (mem:BLK (scratch)) (match_operand:VEC_GATHER_MODE 4 "register_operand")] UNSPEC_GATHER)) - (clobber (match_scratch:VEC_GATHER_MODE 7))])] + (clobber (match_scratch:VEC_GATHER_MODE 8))])] "TARGET_AVX2" { - operands[6] - = gen_rtx_UNSPEC (Pmode, gen_rtvec (3, operands[2], operands[3], - operands[5]), UNSPEC_VSIBADDR); + operands[7] + = gen_rtx_UNSPEC (Pmode, gen_rtvec (4, operands[2], operands[3], + operands[5], operands[6]), + UNSPEC_VSIBADDR); }) (define_insn "*avx2_gathersi" @@ -29238,7 +29252,8 @@ (define_insn "*avx2_gathersi" [(unspec:P [(match_operand:P 3 "vsib_address_operand" "jb") (match_operand: 4 "register_operand" "x") - (match_operand:SI 6 "const1248_operand")] + (match_operand:SI 6 "const1248_operand") + (match_operand:VEC_GATHER_MODE 8 "maskload_else_operand")] UNSPEC_VSIBADDR)]) (mem:BLK (scratch)) (match_operand:VEC_GATHER_MODE 5 "register_operand" "1")] @@ -29259,7 +29274,8 @@ (define_insn "*avx2_gathersi_2" [(unspec:P [(match_operand:P 2 "vsib_address_operand" "jb") (match_operand: 3 "register_operand" "x") - (match_operand:SI 5 "const1248_operand")] + (match_operand:SI 5 "const1248_operand") + (match_operand:VEC_GATHER_MODE 7 "maskload_else_operand")] UNSPEC_VSIBADDR)]) (mem:BLK (scratch)) (match_operand:VEC_GATHER_MODE 4 "register_operand" "1")] @@ -29277,20 +29293,22 @@ (define_expand "avx2_gatherdi" (unspec:VEC_GATHER_MODE [(match_operand: 1 "register_operand") (mem: - (match_par_dup 6 + (match_par_dup 7 [(match_operand 2 "vsib_address_operand") (match_operand: 3 "register_operand") - (match_operand:SI 5 "const1248_operand ")])) + (match_operand:SI 5 "const1248_operand ") + (match_operand:VEC_GATHER_MODE 6 "maskload_else_operand")])) (mem:BLK (scratch)) (match_operand: 4 "register_operand")] UNSPEC_GATHER)) - (clobber (match_scratch:VEC_GATHER_MODE 7))])] + (clobber (match_scratch:VEC_GATHER_MODE 8))])] "TARGET_AVX2" { - operands[6] - = gen_rtx_UNSPEC (Pmode, gen_rtvec (3, operands[2], operands[3], - operands[5]), UNSPEC_VSIBADDR); + operands[7] + = gen_rtx_UNSPEC (Pmode, gen_rtvec (4, operands[2], operands[3], + operands[5], operands[6]), + UNSPEC_VSIBADDR); }) (define_insn "*avx2_gatherdi" @@ -29301,7 +29319,8 @@ (define_insn "*avx2_gatherdi" [(unspec:P [(match_operand:P 3 "vsib_address_operand" "jb") (match_operand: 4 "register_operand" "x") - (match_operand:SI 6 "const1248_operand")] + (match_operand:SI 6 "const1248_operand") + (match_operand:VEC_GATHER_MODE 8 "maskload_else_operand")] UNSPEC_VSIBADDR)]) (mem:BLK (scratch)) (match_operand: 5 "register_operand" "1")] @@ -29322,7 +29341,8 @@ (define_insn "*avx2_gatherdi_2" [(unspec:P [(match_operand:P 2 "vsib_address_operand" "jb") (match_operand: 3 "register_operand" "x") - (match_operand:SI 5 "const1248_operand")] + (match_operand:SI 5 "const1248_operand") + (match_operand:VEC_GATHER_MODE 7 "maskload_else_operand")] UNSPEC_VSIBADDR)]) (mem:BLK (scratch)) (match_operand: 4 "register_operand" "1")] @@ -29348,7 +29368,8 @@ (define_insn "*avx2_gatherdi_3" [(unspec:P [(match_operand:P 3 "vsib_address_operand" "jb") (match_operand: 4 "register_operand" "x") - (match_operand:SI 6 "const1248_operand")] + (match_operand:SI 6 "const1248_operand") + (match_operand:VI4F_256 8 "maskload_else_operand")] UNSPEC_VSIBADDR)]) (mem:BLK (scratch)) (match_operand: 5 "register_operand" "1")] @@ -29372,7 +29393,8 @@ (define_insn "*avx2_gatherdi_4" [(unspec:P [(match_operand:P 2 "vsib_address_operand" "jb") (match_operand: 3 "register_operand" "x") - (match_operand:SI 5 "const1248_operand")] + (match_operand:SI 5 "const1248_operand") + (match_operand:VI4F_256 7 "maskload_else_operand")] UNSPEC_VSIBADDR)]) (mem:BLK (scratch)) (match_operand: 4 "register_operand" "1")] @@ -29393,17 +29415,19 @@ (define_expand "_gathersi" [(match_operand:VI48F 1 "register_operand") (match_operand: 4 "register_operand") (mem: - (match_par_dup 6 + (match_par_dup 7 [(match_operand 2 "vsib_address_operand") (match_operand: 3 "register_operand") - (match_operand:SI 5 "const1248_operand")]))] + (match_operand:SI 5 "const1248_operand") + (match_operand:VI48F 6 "maskload_else_operand")]))] UNSPEC_GATHER)) - (clobber (match_scratch: 7))])] + (clobber (match_scratch: 8))])] "TARGET_AVX512F" { - operands[6] - = gen_rtx_UNSPEC (Pmode, gen_rtvec (3, operands[2], operands[3], - operands[5]), UNSPEC_VSIBADDR); + operands[7] + = gen_rtx_UNSPEC (Pmode, gen_rtvec (4, operands[2], operands[3], + operands[5], operands[6]), + UNSPEC_VSIBADDR); }) (define_insn "*avx512f_gathersi" @@ -29415,7 +29439,8 @@ (define_insn "*avx512f_gathersi" [(unspec:P [(match_operand:P 4 "vsib_address_operand" "Tv") (match_operand: 3 "register_operand" "v") - (match_operand:SI 5 "const1248_operand")] + (match_operand:SI 5 "const1248_operand") + (match_operand:VI48F 8 "maskload_else_operand")] UNSPEC_VSIBADDR)])] UNSPEC_GATHER)) (clobber (match_scratch: 2 "=&Yk"))] @@ -29436,7 +29461,8 @@ (define_insn "*avx512f_gathersi_2" [(unspec:P [(match_operand:P 3 "vsib_address_operand" "Tv") (match_operand: 2 "register_operand" "v") - (match_operand:SI 4 "const1248_operand")] + (match_operand:SI 4 "const1248_operand") + (match_operand:VI48F 7 "maskload_else_operand")] UNSPEC_VSIBADDR)])] UNSPEC_GATHER)) (clobber (match_scratch: 1 "=&Yk"))] @@ -29455,17 +29481,19 @@ (define_expand "_gatherdi" [(match_operand: 1 "register_operand") (match_operand:QI 4 "register_operand") (mem: - (match_par_dup 6 + (match_par_dup 7 [(match_operand 2 "vsib_address_operand") (match_operand: 3 "register_operand") - (match_operand:SI 5 "const1248_operand")]))] + (match_operand:SI 5 "const1248_operand") + (match_operand:VI48F 6 "maskload_else_operand")]))] UNSPEC_GATHER)) - (clobber (match_scratch:QI 7))])] + (clobber (match_scratch:QI 8))])] "TARGET_AVX512F" { - operands[6] - = gen_rtx_UNSPEC (Pmode, gen_rtvec (3, operands[2], operands[3], - operands[5]), UNSPEC_VSIBADDR); + operands[7] + = gen_rtx_UNSPEC (Pmode, gen_rtvec (4, operands[2], operands[3], + operands[5], operands[6]), + UNSPEC_VSIBADDR); }) (define_insn "*avx512f_gatherdi" @@ -29477,7 +29505,8 @@ (define_insn "*avx512f_gatherdi" [(unspec:P [(match_operand:P 4 "vsib_address_operand" "Tv") (match_operand: 3 "register_operand" "v") - (match_operand:SI 5 "const1248_operand")] + (match_operand:SI 5 "const1248_operand") + (match_operand:VI48F 8 "maskload_else_operand")] UNSPEC_VSIBADDR)])] UNSPEC_GATHER)) (clobber (match_scratch:QI 2 "=&Yk"))] @@ -29498,7 +29527,8 @@ (define_insn "*avx512f_gatherdi_2" [(unspec:P [(match_operand:P 3 "vsib_address_operand" "Tv") (match_operand: 2 "register_operand" "v") - (match_operand:SI 4 "const1248_operand")] + (match_operand:SI 4 "const1248_operand") + (match_operand:VI48F 7 "maskload_else_operand")] UNSPEC_VSIBADDR)])] UNSPEC_GATHER)) (clobber (match_scratch:QI 1 "=&Yk"))] @@ -29535,7 +29565,7 @@ (define_expand "_scattersi" operands[5] = gen_rtx_UNSPEC (Pmode, gen_rtvec (4, operands[0], operands[2], operands[4], operands[1]), - UNSPEC_VSIBADDR); + UNSPEC_VSIBADDR); }) (define_insn "*avx512f_scattersi"