| Message ID | patch-18357-tamar@arm.com |
|---|---|
| State | New |
| Headers | show |
| Series | None | expand |
Tamar Christina <tamar.christina@arm.com> writes: > Hi All, > > This patch adds new alternatives to the patterns which are affected. The new > alternatives with the conditional early clobbers are added before the normal > ones in order for LRA to prefer them in the event that we have enough free > registers to accommodate them. > > In case register pressure is too high the normal alternatives will be preferred > before a reload is considered as we rather have the tie than a spill. > > Tests are in the next patch. > > Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. > > Ok for master? > > Thanks, > Tamar > > gcc/ChangeLog: > > * config/aarch64/aarch64-sve.md (and<mode>3, > @aarch64_pred_<optab><mode>_z, *<optab><mode>3_cc, > *<optab><mode>3_ptest, aarch64_pred_<nlogical><mode>_z, > *<nlogical><mode>3_cc, *<nlogical><mode>3_ptest, > aarch64_pred_<logical_nn><mode>_z, *<logical_nn><mode>3_cc, > *<logical_nn><mode>3_ptest, @aarch64_pred_cmp<cmp_op><mode>, > *cmp<cmp_op><mode>_cc, *cmp<cmp_op><mode>_ptest, > @aarch64_pred_cmp<cmp_op><mode>_wide, > *aarch64_pred_cmp<cmp_op><mode>_wide_cc, > *aarch64_pred_cmp<cmp_op><mode>_wide_ptest, @aarch64_brk<brk_op>, > *aarch64_brk<brk_op>_cc, *aarch64_brk<brk_op>_ptest, > @aarch64_brk<brk_op>, *aarch64_brkn_cc, *aarch64_brkn_ptest, > *aarch64_brk<brk_op>_cc, *aarch64_brk<brk_op>_ptest, > aarch64_rdffr_z, *aarch64_rdffr_z_ptest, *aarch64_rdffr_ptest, > *aarch64_rdffr_z_cc, *aarch64_rdffr_cc): Add new early clobber > alternative. > * config/aarch64/aarch64-sve2.md > (@aarch64_pred_<sve_int_op><mode>): Likewise. > > --- > diff --git a/gcc/config/aarch64/aarch64-sve.md b/gcc/config/aarch64/aarch64-sve.md > index e3085c0c636f1317409bbf3b5fbaf5342a2df1f6..8fdc1bc3cd43acfcd675a18350c297428c85fe46 100644 > --- a/gcc/config/aarch64/aarch64-sve.md > +++ b/gcc/config/aarch64/aarch64-sve.md > @@ -1161,8 +1161,10 @@ (define_insn "aarch64_rdffr_z" > (reg:VNx16BI FFRT_REGNUM) > (match_operand:VNx16BI 1 "register_operand")))] > "TARGET_SVE && TARGET_NON_STREAMING" > - {@ [ cons: =0, 1 ] > - [ Upa , Upa ] rdffr\t%0.b, %1/z > + {@ [ cons: =0, 1 ; attrs: pred_clobber ] > + [ &Upa , Upa; yes ] rdffr\t%0.b, %1/z > + [ ?Upa , Upa; yes ] ^ > + [ Upa , Upa; * ] ^ > } > ) Sorry for not explaining it very well, but in the previous review I suggested: > The gather-like approach would be something like: > > [ &Upa , Upl , w , <sve_imm_con>; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, #%4 > [ ?Upl , 0 , w , <sve_imm_con>; yes ] ^ > [ Upa , Upl , w , <sve_imm_con>; no ] ^ > [ &Upa , Upl , w , w ; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.<Vetype> > [ ?Upl , 0 , w , w ; yes ] ^ > [ Upa , Upl , w , w ; no ] ^ > > with: > > (define_attr "pred_clobber" "any,no,yes" (const_string "any")) (with emphasis on the last line). What I didn't say explicitly is that "no" should require !TARGET_SVE_PRED_CLOBBER. The premise of that review was that we shouldn't enable things like: [ Upa , Upl , w , w ; no ] ^ for TARGET_SVE_PRED_CLOBBER since it contradicts the earlyclobber alternative. So we should enable either the pred_clobber=yes alternatives or the pred_clobber=no alternatives, but not both. The default "any" is then for other non-predicate instructions that don't care about TARGET_SVE_PRED_CLOBBER either way. In contrast, this patch makes pred_clobber=yes enable the alternatives that correctly describe the restriction (good!) but then also enables the normal alternatives too, which IMO makes the semantics unclear. Thanks, Richard > > @@ -1179,8 +1181,10 @@ (define_insn "*aarch64_rdffr_z_ptest" > UNSPEC_PTEST)) > (clobber (match_scratch:VNx16BI 0))] > "TARGET_SVE && TARGET_NON_STREAMING" > - {@ [ cons: =0, 1 ] > - [ Upa , Upa ] rdffrs\t%0.b, %1/z > + {@ [ cons: =0, 1 ; attrs: pred_clobber ] > + [ &Upa , Upa; yes ] rdffrs\t%0.b, %1/z > + [ ?Upa , Upa; yes ] ^ > + [ Upa , Upa; * ] ^ > } > ) > > @@ -1195,8 +1199,10 @@ (define_insn "*aarch64_rdffr_ptest" > UNSPEC_PTEST)) > (clobber (match_scratch:VNx16BI 0))] > "TARGET_SVE && TARGET_NON_STREAMING" > - {@ [ cons: =0, 1 ] > - [ Upa , Upa ] rdffrs\t%0.b, %1/z > + {@ [ cons: =0, 1 ; attrs: pred_clobber ] > + [ &Upa , Upa; yes ] rdffrs\t%0.b, %1/z > + [ ?Upa , Upa; yes ] ^ > + [ Upa , Upa; * ] ^ > } > ) > > @@ -1216,8 +1222,10 @@ (define_insn "*aarch64_rdffr_z_cc" > (reg:VNx16BI FFRT_REGNUM) > (match_dup 1)))] > "TARGET_SVE && TARGET_NON_STREAMING" > - {@ [ cons: =0, 1 ] > - [ Upa , Upa ] rdffrs\t%0.b, %1/z > + {@ [ cons: =0, 1 ; attrs: pred_clobber ] > + [ &Upa , Upa; yes ] rdffrs\t%0.b, %1/z > + [ ?Upa , Upa; yes ] ^ > + [ Upa , Upa; * ] ^ > } > ) > > @@ -1233,8 +1241,10 @@ (define_insn "*aarch64_rdffr_cc" > (set (match_operand:VNx16BI 0 "register_operand") > (reg:VNx16BI FFRT_REGNUM))] > "TARGET_SVE && TARGET_NON_STREAMING" > - {@ [ cons: =0, 1 ] > - [ Upa , Upa ] rdffrs\t%0.b, %1/z > + {@ [ cons: =0, 1 ; attrs: pred_clobber ] > + [ &Upa , Upa; yes ] rdffrs\t%0.b, %1/z > + [ ?Upa , Upa; yes ] ^ > + [ Upa , Upa; * ] ^ > } > ) > > @@ -6651,8 +6661,10 @@ (define_insn "and<mode>3" > (and:PRED_ALL (match_operand:PRED_ALL 1 "register_operand") > (match_operand:PRED_ALL 2 "register_operand")))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 ] > - [ Upa , Upa, Upa ] and\t%0.b, %1/z, %2.b, %2.b > + {@ [ cons: =0, 1 , 2 ; attrs: pred_clobber ] > + [ &Upa , Upa, Upa; yes ] and\t%0.b, %1/z, %2.b, %2.b > + [ ?Upa , Upa, Upa; yes ] ^ > + [ Upa , Upa, Upa; * ] ^ > } > ) > > @@ -6679,8 +6691,10 @@ (define_insn "@aarch64_pred_<optab><mode>_z" > (match_operand:PRED_ALL 3 "register_operand")) > (match_operand:PRED_ALL 1 "register_operand")))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 , 3 ] > - [ Upa , Upa, Upa, Upa ] <logical>\t%0.b, %1/z, %2.b, %3.b > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > + [ &Upa , Upa, Upa, Upa; yes ] <logical>\t%0.b, %1/z, %2.b, %3.b > + [ ?Upa , Upa, Upa, Upa; yes ] ^ > + [ Upa , Upa, Upa, Upa; * ] ^ > } > ) > > @@ -6703,8 +6717,10 @@ (define_insn "*<optab><mode>3_cc" > (and:PRED_ALL (LOGICAL:PRED_ALL (match_dup 2) (match_dup 3)) > (match_dup 4)))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 , 3 ] > - [ Upa , Upa, Upa, Upa ] <logical>s\t%0.b, %1/z, %2.b, %3.b > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > + [ &Upa , Upa, Upa, Upa; yes ] <logical>s\t%0.b, %1/z, %2.b, %3.b > + [ ?Upa , Upa, Upa, Upa; yes ] ^ > + [ Upa , Upa, Upa, Upa; * ] ^ > } > ) > > @@ -6723,8 +6739,10 @@ (define_insn "*<optab><mode>3_ptest" > UNSPEC_PTEST)) > (clobber (match_scratch:VNx16BI 0))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 , 3 ] > - [ Upa , Upa, Upa, Upa ] <logical>s\t%0.b, %1/z, %2.b, %3.b > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > + [ &Upa , Upa, Upa, Upa; yes ] <logical>s\t%0.b, %1/z, %2.b, %3.b > + [ ?Upa , Upa, Upa, Upa; yes ] ^ > + [ Upa , Upa, Upa, Upa; * ] ^ > } > ) > > @@ -6745,8 +6763,10 @@ (define_insn "aarch64_pred_<nlogical><mode>_z" > (match_operand:PRED_ALL 2 "register_operand")) > (match_operand:PRED_ALL 1 "register_operand")))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 , 3 ] > - [ Upa , Upa, Upa, Upa ] <nlogical>\t%0.b, %1/z, %2.b, %3.b > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > + [ &Upa , Upa, Upa, Upa; yes ] <nlogical>\t%0.b, %1/z, %2.b, %3.b > + [ ?Upa , Upa, Upa, Upa; yes ] ^ > + [ Upa , Upa, Upa, Upa; * ] ^ > } > ) > > @@ -6770,8 +6790,10 @@ (define_insn "*<nlogical><mode>3_cc" > (match_dup 2)) > (match_dup 4)))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 , 3 ] > - [ Upa , Upa, Upa, Upa ] <nlogical>s\t%0.b, %1/z, %2.b, %3.b > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > + [ &Upa , Upa, Upa, Upa; yes ] <nlogical>s\t%0.b, %1/z, %2.b, %3.b > + [ ?Upa , Upa, Upa, Upa; yes ] ^ > + [ Upa , Upa, Upa, Upa; * ] ^ > } > ) > > @@ -6791,8 +6813,10 @@ (define_insn "*<nlogical><mode>3_ptest" > UNSPEC_PTEST)) > (clobber (match_scratch:VNx16BI 0))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 , 3 ] > - [ Upa , Upa, Upa, Upa ] <nlogical>s\t%0.b, %1/z, %2.b, %3.b > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > + [ &Upa , Upa, Upa, Upa; yes ] <nlogical>s\t%0.b, %1/z, %2.b, %3.b > + [ ?Upa , Upa, Upa, Upa; yes ] ^ > + [ Upa , Upa, Upa, Upa; * ] ^ > } > ) > > @@ -6813,8 +6837,10 @@ (define_insn "aarch64_pred_<logical_nn><mode>_z" > (not:PRED_ALL (match_operand:PRED_ALL 3 "register_operand"))) > (match_operand:PRED_ALL 1 "register_operand")))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 , 3 ] > - [ Upa , Upa, Upa, Upa ] <logical_nn>\t%0.b, %1/z, %2.b, %3.b > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > + [ &Upa , Upa, Upa, Upa; yes ] <logical_nn>\t%0.b, %1/z, %2.b, %3.b > + [ ?Upa , Upa, Upa, Upa; yes ] ^ > + [ Upa , Upa, Upa, Upa; * ] ^ > } > ) > > @@ -6839,8 +6865,10 @@ (define_insn "*<logical_nn><mode>3_cc" > (not:PRED_ALL (match_dup 3))) > (match_dup 4)))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 , 3 ] > - [ Upa , Upa, Upa, Upa ] <logical_nn>s\t%0.b, %1/z, %2.b, %3.b > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > + [ &Upa , Upa, Upa, Upa; yes ] <logical_nn>s\t%0.b, %1/z, %2.b, %3.b > + [ ?Upa , Upa, Upa, Upa; yes ] ^ > + [ Upa , Upa, Upa, Upa; * ] ^ > } > ) > > @@ -6861,8 +6889,10 @@ (define_insn "*<logical_nn><mode>3_ptest" > UNSPEC_PTEST)) > (clobber (match_scratch:VNx16BI 0))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 , 3 ] > - [ Upa , Upa, Upa, Upa ] <logical_nn>s\t%0.b, %1/z, %2.b, %3.b > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > + [ &Upa , Upa, Upa, Upa; yes ] <logical_nn>s\t%0.b, %1/z, %2.b, %3.b > + [ ?Upa , Upa, Upa, Upa; yes ] ^ > + [ Upa , Upa, Upa, Upa; * ] ^ > } > ) > > @@ -8104,9 +8134,13 @@ (define_insn "@aarch64_pred_cmp<cmp_op><mode>" > UNSPEC_PRED_Z)) > (clobber (reg:CC_NZC CC_REGNUM))] > "TARGET_SVE" > - {@ [ cons: =0 , 1 , 3 , 4 ] > - [ Upa , Upl , w , <sve_imm_con> ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, #%4 > - [ Upa , Upl , w , w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.<Vetype> > + {@ [ cons: =0 , 1 , 3 , 4 ; attrs: pred_clobber ] > + [ &Upa , Upl , w , <sve_imm_con>; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, #%4 > + [ ?Upa , Upl , w , <sve_imm_con>; yes ] ^ > + [ Upa , Upl , w , <sve_imm_con>; * ] ^ > + [ &Upa , Upl , w , w ; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.<Vetype> > + [ ?Upa , Upl , w , w ; yes ] ^ > + [ Upa , Upl , w , w ; * ] ^ > } > ) > > @@ -8136,9 +8170,13 @@ (define_insn_and_rewrite "*cmp<cmp_op><mode>_cc" > UNSPEC_PRED_Z))] > "TARGET_SVE > && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" > - {@ [ cons: =0 , 1 , 2 , 3 ] > - [ Upa , Upl , w , <sve_imm_con> ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, #%3 > - [ Upa , Upl , w , w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.<Vetype> > + {@ [ cons: =0 , 1 , 2 , 3 ; attrs: pred_clobber ] > + [ &Upa , Upl , w , <sve_imm_con>; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, #%3 > + [ ?Upa , Upl , w , <sve_imm_con>; yes ] ^ > + [ Upa , Upl , w , <sve_imm_con>; * ] ^ > + [ &Upa , Upl , w , w ; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.<Vetype> > + [ ?Upa , Upl , w , w ; yes ] ^ > + [ Upa , Upl , w , w ; * ] ^ > } > "&& !rtx_equal_p (operands[4], operands[6])" > { > @@ -8166,9 +8204,13 @@ (define_insn_and_rewrite "*cmp<cmp_op><mode>_ptest" > (clobber (match_scratch:<VPRED> 0))] > "TARGET_SVE > && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" > - {@ [ cons: =0, 1 , 2 , 3 ] > - [ Upa , Upl, w , <sve_imm_con> ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, #%3 > - [ Upa , Upl, w , w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.<Vetype> > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > + [ &Upa , Upl, w , <sve_imm_con>; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, #%3 > + [ ?Upa , Upl, w , <sve_imm_con>; yes ] ^ > + [ Upa , Upl, w , <sve_imm_con>; * ] ^ > + [ &Upa , Upl, w , w ; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.<Vetype> > + [ ?Upa , Upl, w , w ; yes ] ^ > + [ Upa , Upl, w , w ; * ] ^ > } > "&& !rtx_equal_p (operands[4], operands[6])" > { > @@ -8221,8 +8263,10 @@ (define_insn "@aarch64_pred_cmp<cmp_op><mode>_wide" > UNSPEC_PRED_Z)) > (clobber (reg:CC_NZC CC_REGNUM))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2, 3, 4 ] > - [ Upa , Upl, , w, w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.d > + {@ [ cons: =0, 1 , 2, 3, 4; attrs: pred_clobber ] > + [ &Upa , Upl, , w, w; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.d > + [ ?Upa , Upl, , w, w; yes ] ^ > + [ Upa , Upl, , w, w; * ] ^ > } > ) > > @@ -8254,8 +8298,10 @@ (define_insn "*aarch64_pred_cmp<cmp_op><mode>_wide_cc" > UNSPEC_PRED_Z))] > "TARGET_SVE > && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" > - {@ [ cons: =0, 1 , 2, 3, 6 ] > - [ Upa , Upl, w, w, Upl ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.d > + {@ [ cons: =0, 1 , 2, 3, 6 ; attrs: pred_clobber ] > + [ &Upa , Upl, w, w, Upl; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.d > + [ ?Upa , Upl, w, w, Upl; yes ] ^ > + [ Upa , Upl, w, w, Upl; * ] ^ > } > ) > > @@ -8279,8 +8325,10 @@ (define_insn "*aarch64_pred_cmp<cmp_op><mode>_wide_ptest" > (clobber (match_scratch:<VPRED> 0))] > "TARGET_SVE > && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" > - {@ [ cons: =0, 1 , 2, 3, 6 ] > - [ Upa , Upl, w, w, Upl ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.d > + {@ [ cons: =0, 1 , 2, 3, 6 ; attrs: pred_clobber ] > + [ &Upa , Upl, w, w, Upl; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.d > + [ ?Upa , Upl, w, w, Upl; yes ] ^ > + [ Upa , Upl, w, w, Upl; * ] ^ > } > ) > > @@ -9948,9 +9996,13 @@ (define_insn "@aarch64_brk<brk_op>" > (match_operand:VNx16BI 3 "aarch64_simd_reg_or_zero")] > SVE_BRK_UNARY))] > "TARGET_SVE" > - {@ [ cons: =0 , 1 , 2 , 3 ] > - [ Upa , Upa , Upa , Dz ] brk<brk_op>\t%0.b, %1/z, %2.b > - [ Upa , Upa , Upa , 0 ] brk<brk_op>\t%0.b, %1/m, %2.b > + {@ [ cons: =0 , 1 , 2 , 3 ; attrs: pred_clobber ] > + [ &Upa , Upa , Upa , Dz; yes ] brk<brk_op>\t%0.b, %1/z, %2.b > + [ ?Upa , Upa , Upa , Dz; yes ] ^ > + [ Upa , Upa , Upa , Dz; * ] ^ > + [ &Upa , Upa , Upa , 0 ; yes ] brk<brk_op>\t%0.b, %1/m, %2.b > + [ ?Upa , Upa , Upa , 0 ; yes ] ^ > + [ Upa , Upa , Upa , 0 ; * ] ^ > } > ) > > @@ -9974,8 +10026,10 @@ (define_insn "*aarch64_brk<brk_op>_cc" > (match_dup 3)] > SVE_BRK_UNARY))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 ] > - [ Upa , Upa, Upa ] brk<brk_op>s\t%0.b, %1/z, %2.b > + {@ [ cons: =0, 1 , 2 ; attrs: pred_clobber ] > + [ &Upa , Upa, Upa; yes ] brk<brk_op>s\t%0.b, %1/z, %2.b > + [ ?Upa , Upa, Upa; yes ] ^ > + [ Upa , Upa, Upa; * ] ^ > } > ) > > @@ -9994,8 +10048,10 @@ (define_insn "*aarch64_brk<brk_op>_ptest" > UNSPEC_PTEST)) > (clobber (match_scratch:VNx16BI 0))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 ] > - [ Upa , Upa, Upa ] brk<brk_op>s\t%0.b, %1/z, %2.b > + {@ [ cons: =0, 1 , 2 ; attrs: pred_clobber ] > + [ &Upa , Upa, Upa; yes ] brk<brk_op>s\t%0.b, %1/z, %2.b > + [ ?Upa , Upa, Upa; yes ] ^ > + [ Upa , Upa, Upa; * ] ^ > } > ) > > @@ -10020,8 +10076,10 @@ (define_insn "@aarch64_brk<brk_op>" > (match_operand:VNx16BI 3 "register_operand")] > SVE_BRK_BINARY))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 , 3 ] > - [ Upa , Upa, Upa, <brk_reg_con> ] brk<brk_op>\t%0.b, %1/z, %2.b, %<brk_reg_opno>.b > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > + [ &Upa , Upa, Upa, <brk_reg_con>; yes ] brk<brk_op>\t%0.b, %1/z, %2.b, %<brk_reg_opno>.b > + [ ?Upa , Upa, Upa, <brk_reg_con>; yes ] ^ > + [ Upa , Upa, Upa, <brk_reg_con>; * ] ^ > } > ) > > @@ -10046,8 +10104,10 @@ (define_insn_and_rewrite "*aarch64_brkn_cc" > (match_dup 3)] > UNSPEC_BRKN))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 , 3 ] > - [ Upa , Upa, Upa, 0 ] brkns\t%0.b, %1/z, %2.b, %0.b > + {@ [ cons: =0, 1 , 2 , 3; attrs: pred_clobber ] > + [ &Upa , Upa, Upa, 0; yes ] brkns\t%0.b, %1/z, %2.b, %0.b > + [ ?Upa , Upa, Upa, 0; yes ] ^ > + [ Upa , Upa, Upa, 0; * ] ^ > } > "&& (operands[4] != CONST0_RTX (VNx16BImode) > || operands[5] != CONST0_RTX (VNx16BImode))" > @@ -10072,8 +10132,10 @@ (define_insn_and_rewrite "*aarch64_brkn_ptest" > UNSPEC_PTEST)) > (clobber (match_scratch:VNx16BI 0))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 , 3 ] > - [ Upa , Upa, Upa, 0 ] brkns\t%0.b, %1/z, %2.b, %0.b > + {@ [ cons: =0, 1 , 2 , 3; attrs: pred_clobber ] > + [ &Upa , Upa, Upa, 0; yes ] brkns\t%0.b, %1/z, %2.b, %0.b > + [ ?Upa , Upa, Upa, 0; yes ] ^ > + [ Upa , Upa, Upa, 0; * ] ^ > } > "&& (operands[4] != CONST0_RTX (VNx16BImode) > || operands[5] != CONST0_RTX (VNx16BImode))" > @@ -10103,8 +10165,10 @@ (define_insn "*aarch64_brk<brk_op>_cc" > (match_dup 3)] > SVE_BRKP))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 , 3 ] > - [ Upa , Upa, Upa, Upa ] brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b > + {@ [ cons: =0, 1 , 2 , 3 , 4; attrs: pred_clobber ] > + [ &Upa , Upa, Upa, Upa, ; yes ] brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b > + [ ?Upa , Upa, Upa, Upa, ; yes ] ^ > + [ Upa , Upa, Upa, Upa, ; * ] ^ > } > ) > > @@ -10123,8 +10187,10 @@ (define_insn "*aarch64_brk<brk_op>_ptest" > UNSPEC_PTEST)) > (clobber (match_scratch:VNx16BI 0))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 , 3 ] > - [ Upa , Upa, Upa, Upa ] brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > + [ &Upa , Upa, Upa, Upa; yes ] brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b > + [ ?Upa , Upa, Upa, Upa; yes ] ^ > + [ Upa , Upa, Upa, Upa; * ] ^ > } > ) > > diff --git a/gcc/config/aarch64/aarch64-sve2.md b/gcc/config/aarch64/aarch64-sve2.md > index aa12baf48355358ca4fefe88157df3aac6eb09bd..1a49494a69d8335e5f7d3ef4bd3a90d0805bba84 100644 > --- a/gcc/config/aarch64/aarch64-sve2.md > +++ b/gcc/config/aarch64/aarch64-sve2.md > @@ -3349,8 +3349,10 @@ (define_insn "@aarch64_pred_<sve_int_op><mode>" > UNSPEC_PRED_Z)) > (clobber (reg:CC_NZC CC_REGNUM))] > "TARGET_SVE2 && TARGET_NON_STREAMING" > - {@ [ cons: =0, 1 , 2, 3, 4 ] > - [ Upa , Upl, , w, w ] <sve_int_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.<Vetype> > + {@ [ cons: =0, 1 , 2, 3, 4; attrs: pred_clobber ] > + [ &Upa , Upl, , w, w; yes ] <sve_int_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.<Vetype> > + [ ?Upa , Upl, , w, w; yes ] ^ > + [ Upa , Upl, , w, w; * ] ^ > } > )
> -----Original Message----- > From: Richard Sandiford <richard.sandiford@arm.com> > Sent: Wednesday, May 22, 2024 10:48 AM > To: Tamar Christina <Tamar.Christina@arm.com> > Cc: gcc-patches@gcc.gnu.org; nd <nd@arm.com>; Richard Earnshaw > <Richard.Earnshaw@arm.com>; Marcus Shawcroft > <Marcus.Shawcroft@arm.com>; ktkachov@gcc.gnu.org > Subject: Re: [PATCH 3/4]AArch64: add new alternative with early clobber to > patterns > > Tamar Christina <tamar.christina@arm.com> writes: > > Hi All, > > > > This patch adds new alternatives to the patterns which are affected. The new > > alternatives with the conditional early clobbers are added before the normal > > ones in order for LRA to prefer them in the event that we have enough free > > registers to accommodate them. > > > > In case register pressure is too high the normal alternatives will be preferred > > before a reload is considered as we rather have the tie than a spill. > > > > Tests are in the next patch. > > > > Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. > > > > Ok for master? > > > > Thanks, > > Tamar > > > > gcc/ChangeLog: > > > > * config/aarch64/aarch64-sve.md (and<mode>3, > > @aarch64_pred_<optab><mode>_z, *<optab><mode>3_cc, > > *<optab><mode>3_ptest, aarch64_pred_<nlogical><mode>_z, > > *<nlogical><mode>3_cc, *<nlogical><mode>3_ptest, > > aarch64_pred_<logical_nn><mode>_z, *<logical_nn><mode>3_cc, > > *<logical_nn><mode>3_ptest, @aarch64_pred_cmp<cmp_op><mode>, > > *cmp<cmp_op><mode>_cc, *cmp<cmp_op><mode>_ptest, > > @aarch64_pred_cmp<cmp_op><mode>_wide, > > *aarch64_pred_cmp<cmp_op><mode>_wide_cc, > > *aarch64_pred_cmp<cmp_op><mode>_wide_ptest, > @aarch64_brk<brk_op>, > > *aarch64_brk<brk_op>_cc, *aarch64_brk<brk_op>_ptest, > > @aarch64_brk<brk_op>, *aarch64_brkn_cc, *aarch64_brkn_ptest, > > *aarch64_brk<brk_op>_cc, *aarch64_brk<brk_op>_ptest, > > aarch64_rdffr_z, *aarch64_rdffr_z_ptest, *aarch64_rdffr_ptest, > > *aarch64_rdffr_z_cc, *aarch64_rdffr_cc): Add new early clobber > > alternative. > > * config/aarch64/aarch64-sve2.md > > (@aarch64_pred_<sve_int_op><mode>): Likewise. > > > > --- > > diff --git a/gcc/config/aarch64/aarch64-sve.md b/gcc/config/aarch64/aarch64- > sve.md > > index > e3085c0c636f1317409bbf3b5fbaf5342a2df1f6..8fdc1bc3cd43acfcd675a18350c > 297428c85fe46 100644 > > --- a/gcc/config/aarch64/aarch64-sve.md > > +++ b/gcc/config/aarch64/aarch64-sve.md > > @@ -1161,8 +1161,10 @@ (define_insn "aarch64_rdffr_z" > > (reg:VNx16BI FFRT_REGNUM) > > (match_operand:VNx16BI 1 "register_operand")))] > > "TARGET_SVE && TARGET_NON_STREAMING" > > - {@ [ cons: =0, 1 ] > > - [ Upa , Upa ] rdffr\t%0.b, %1/z > > + {@ [ cons: =0, 1 ; attrs: pred_clobber ] > > + [ &Upa , Upa; yes ] rdffr\t%0.b, %1/z > > + [ ?Upa , Upa; yes ] ^ > > + [ Upa , Upa; * ] ^ > > } > > ) > > Sorry for not explaining it very well, but in the previous review I suggested: > > > The gather-like approach would be something like: > > > > [ &Upa , Upl , w , <sve_imm_con>; yes ] > cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, #%4 > > [ ?Upl , 0 , w , <sve_imm_con>; yes ] ^ > > [ Upa , Upl , w , <sve_imm_con>; no ] ^ > > [ &Upa , Upl , w , w ; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, > %3.<Vetype>, %4.<Vetype> > > [ ?Upl , 0 , w , w ; yes ] ^ > > [ Upa , Upl , w , w ; no ] ^ > > > > with: > > > > (define_attr "pred_clobber" "any,no,yes" (const_string "any")) > > (with emphasis on the last line). What I didn't say explicitly is > that "no" should require !TARGET_SVE_PRED_CLOBBER. > > The premise of that review was that we shouldn't enable things like: > > [ Upa , Upl , w , w ; no ] ^ > > for TARGET_SVE_PRED_CLOBBER since it contradicts the earlyclobber > alternative. So we should enable either the pred_clobber=yes > alternatives or the pred_clobber=no alternatives, but not both. > > The default "any" is then for other non-predicate instructions that > don't care about TARGET_SVE_PRED_CLOBBER either way. > > In contrast, this patch makes pred_clobber=yes enable the alternatives > that correctly describe the restriction (good!) but then also enables > the normal alternatives too, which IMO makes the semantics unclear. Sure, the reason I still had that is because this ICEs under high register pressure: {@ [ cons: =0 , 1 , 3 , 4 ; attrs: pred_clobber ] [ &Upa , Upl , w , <sve_imm_con>; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, #%4 [ ?Upa , 0 , w , <sve_imm_con>; yes ] ^ [ Upa , Upl , w , <sve_imm_con>; no ] ^ [ &Upa , Upl , w , w ; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.<Vetype> [ ?Upa , 0 , w , w ; yes ] ^ [ Upa , Upl , w , w ; no ] ^ } So now in the `yes` case reload does: Considering alt=0 of insn 10: (0) =&Upa (1) Upl (3) w (4) vsd 0 Small class reload: reject+=3 0 Non input pseudo reload: reject++ 0 Early clobber: reject++ Bad operand -- refuse Considering alt=1 of insn 10: (0) ?Upa (1) 0 (3) w (4) vsd Staticly defined alt reject+=6 0 Small class reload: reject+=3 0 Non input pseudo reload: reject++ 1 Dying matched operand reload: reject++ 1 Small class reload: reject+=3 Bad operand -- refuse Considering alt=3 of insn 10: (0) &Upa (1) Upl (3) w (4) w 0 Small class reload: reject+=3 0 Non input pseudo reload: reject++ 0 Early clobber: reject++ overall=11,losers=1,rld_nregs=1 Considering alt=4 of insn 10: (0) ?Upa (1) 0 (3) w (4) w Staticly defined alt reject+=6 0 Small class reload: reject+=3 0 Non input pseudo reload: reject++ overall=16,losers=1 -- refuse Choosing alt 3 in insn 10: (0) &Upa (1) Upl (3) w (4) w {aarch64_pred_cmplovnx8hi} And the penalty of alt=4 makes it pick alt=3 even though it doesn't have the free registers for it. alt=4 would have worked. I believe this now follows exactly what was suggested: 1. provide an early clobber alternative 2. provide an explicit tie alternative with an increase in cost for using it 3. provide a general/normal alternative that is only enabled when the first two aren't. Having read the email a number of times.. did I somehow miss something? Tamar > > Thanks, > Richard > > > > > @@ -1179,8 +1181,10 @@ (define_insn "*aarch64_rdffr_z_ptest" > > UNSPEC_PTEST)) > > (clobber (match_scratch:VNx16BI 0))] > > "TARGET_SVE && TARGET_NON_STREAMING" > > - {@ [ cons: =0, 1 ] > > - [ Upa , Upa ] rdffrs\t%0.b, %1/z > > + {@ [ cons: =0, 1 ; attrs: pred_clobber ] > > + [ &Upa , Upa; yes ] rdffrs\t%0.b, %1/z > > + [ ?Upa , Upa; yes ] ^ > > + [ Upa , Upa; * ] ^ > > } > > ) > > > > @@ -1195,8 +1199,10 @@ (define_insn "*aarch64_rdffr_ptest" > > UNSPEC_PTEST)) > > (clobber (match_scratch:VNx16BI 0))] > > "TARGET_SVE && TARGET_NON_STREAMING" > > - {@ [ cons: =0, 1 ] > > - [ Upa , Upa ] rdffrs\t%0.b, %1/z > > + {@ [ cons: =0, 1 ; attrs: pred_clobber ] > > + [ &Upa , Upa; yes ] rdffrs\t%0.b, %1/z > > + [ ?Upa , Upa; yes ] ^ > > + [ Upa , Upa; * ] ^ > > } > > ) > > > > @@ -1216,8 +1222,10 @@ (define_insn "*aarch64_rdffr_z_cc" > > (reg:VNx16BI FFRT_REGNUM) > > (match_dup 1)))] > > "TARGET_SVE && TARGET_NON_STREAMING" > > - {@ [ cons: =0, 1 ] > > - [ Upa , Upa ] rdffrs\t%0.b, %1/z > > + {@ [ cons: =0, 1 ; attrs: pred_clobber ] > > + [ &Upa , Upa; yes ] rdffrs\t%0.b, %1/z > > + [ ?Upa , Upa; yes ] ^ > > + [ Upa , Upa; * ] ^ > > } > > ) > > > > @@ -1233,8 +1241,10 @@ (define_insn "*aarch64_rdffr_cc" > > (set (match_operand:VNx16BI 0 "register_operand") > > (reg:VNx16BI FFRT_REGNUM))] > > "TARGET_SVE && TARGET_NON_STREAMING" > > - {@ [ cons: =0, 1 ] > > - [ Upa , Upa ] rdffrs\t%0.b, %1/z > > + {@ [ cons: =0, 1 ; attrs: pred_clobber ] > > + [ &Upa , Upa; yes ] rdffrs\t%0.b, %1/z > > + [ ?Upa , Upa; yes ] ^ > > + [ Upa , Upa; * ] ^ > > } > > ) > > > > @@ -6651,8 +6661,10 @@ (define_insn "and<mode>3" > > (and:PRED_ALL (match_operand:PRED_ALL 1 "register_operand") > > (match_operand:PRED_ALL 2 "register_operand")))] > > "TARGET_SVE" > > - {@ [ cons: =0, 1 , 2 ] > > - [ Upa , Upa, Upa ] and\t%0.b, %1/z, %2.b, %2.b > > + {@ [ cons: =0, 1 , 2 ; attrs: pred_clobber ] > > + [ &Upa , Upa, Upa; yes ] and\t%0.b, %1/z, %2.b, %2.b > > + [ ?Upa , Upa, Upa; yes ] ^ > > + [ Upa , Upa, Upa; * ] ^ > > } > > ) > > > > @@ -6679,8 +6691,10 @@ (define_insn "@aarch64_pred_<optab><mode>_z" > > (match_operand:PRED_ALL 3 "register_operand")) > > (match_operand:PRED_ALL 1 "register_operand")))] > > "TARGET_SVE" > > - {@ [ cons: =0, 1 , 2 , 3 ] > > - [ Upa , Upa, Upa, Upa ] <logical>\t%0.b, %1/z, %2.b, %3.b > > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > > + [ &Upa , Upa, Upa, Upa; yes ] <logical>\t%0.b, %1/z, %2.b, %3.b > > + [ ?Upa , Upa, Upa, Upa; yes ] ^ > > + [ Upa , Upa, Upa, Upa; * ] ^ > > } > > ) > > > > @@ -6703,8 +6717,10 @@ (define_insn "*<optab><mode>3_cc" > > (and:PRED_ALL (LOGICAL:PRED_ALL (match_dup 2) (match_dup 3)) > > (match_dup 4)))] > > "TARGET_SVE" > > - {@ [ cons: =0, 1 , 2 , 3 ] > > - [ Upa , Upa, Upa, Upa ] <logical>s\t%0.b, %1/z, %2.b, %3.b > > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > > + [ &Upa , Upa, Upa, Upa; yes ] <logical>s\t%0.b, %1/z, %2.b, %3.b > > + [ ?Upa , Upa, Upa, Upa; yes ] ^ > > + [ Upa , Upa, Upa, Upa; * ] ^ > > } > > ) > > > > @@ -6723,8 +6739,10 @@ (define_insn "*<optab><mode>3_ptest" > > UNSPEC_PTEST)) > > (clobber (match_scratch:VNx16BI 0))] > > "TARGET_SVE" > > - {@ [ cons: =0, 1 , 2 , 3 ] > > - [ Upa , Upa, Upa, Upa ] <logical>s\t%0.b, %1/z, %2.b, %3.b > > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > > + [ &Upa , Upa, Upa, Upa; yes ] <logical>s\t%0.b, %1/z, %2.b, %3.b > > + [ ?Upa , Upa, Upa, Upa; yes ] ^ > > + [ Upa , Upa, Upa, Upa; * ] ^ > > } > > ) > > > > @@ -6745,8 +6763,10 @@ (define_insn "aarch64_pred_<nlogical><mode>_z" > > (match_operand:PRED_ALL 2 "register_operand")) > > (match_operand:PRED_ALL 1 "register_operand")))] > > "TARGET_SVE" > > - {@ [ cons: =0, 1 , 2 , 3 ] > > - [ Upa , Upa, Upa, Upa ] <nlogical>\t%0.b, %1/z, %2.b, %3.b > > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > > + [ &Upa , Upa, Upa, Upa; yes ] <nlogical>\t%0.b, %1/z, %2.b, %3.b > > + [ ?Upa , Upa, Upa, Upa; yes ] ^ > > + [ Upa , Upa, Upa, Upa; * ] ^ > > } > > ) > > > > @@ -6770,8 +6790,10 @@ (define_insn "*<nlogical><mode>3_cc" > > (match_dup 2)) > > (match_dup 4)))] > > "TARGET_SVE" > > - {@ [ cons: =0, 1 , 2 , 3 ] > > - [ Upa , Upa, Upa, Upa ] <nlogical>s\t%0.b, %1/z, %2.b, %3.b > > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > > + [ &Upa , Upa, Upa, Upa; yes ] <nlogical>s\t%0.b, %1/z, %2.b, %3.b > > + [ ?Upa , Upa, Upa, Upa; yes ] ^ > > + [ Upa , Upa, Upa, Upa; * ] ^ > > } > > ) > > > > @@ -6791,8 +6813,10 @@ (define_insn "*<nlogical><mode>3_ptest" > > UNSPEC_PTEST)) > > (clobber (match_scratch:VNx16BI 0))] > > "TARGET_SVE" > > - {@ [ cons: =0, 1 , 2 , 3 ] > > - [ Upa , Upa, Upa, Upa ] <nlogical>s\t%0.b, %1/z, %2.b, %3.b > > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > > + [ &Upa , Upa, Upa, Upa; yes ] <nlogical>s\t%0.b, %1/z, %2.b, %3.b > > + [ ?Upa , Upa, Upa, Upa; yes ] ^ > > + [ Upa , Upa, Upa, Upa; * ] ^ > > } > > ) > > > > @@ -6813,8 +6837,10 @@ (define_insn > "aarch64_pred_<logical_nn><mode>_z" > > (not:PRED_ALL (match_operand:PRED_ALL 3 "register_operand"))) > > (match_operand:PRED_ALL 1 "register_operand")))] > > "TARGET_SVE" > > - {@ [ cons: =0, 1 , 2 , 3 ] > > - [ Upa , Upa, Upa, Upa ] <logical_nn>\t%0.b, %1/z, %2.b, %3.b > > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > > + [ &Upa , Upa, Upa, Upa; yes ] <logical_nn>\t%0.b, %1/z, %2.b, > %3.b > > + [ ?Upa , Upa, Upa, Upa; yes ] ^ > > + [ Upa , Upa, Upa, Upa; * ] ^ > > } > > ) > > > > @@ -6839,8 +6865,10 @@ (define_insn "*<logical_nn><mode>3_cc" > > (not:PRED_ALL (match_dup 3))) > > (match_dup 4)))] > > "TARGET_SVE" > > - {@ [ cons: =0, 1 , 2 , 3 ] > > - [ Upa , Upa, Upa, Upa ] <logical_nn>s\t%0.b, %1/z, %2.b, %3.b > > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > > + [ &Upa , Upa, Upa, Upa; yes ] <logical_nn>s\t%0.b, %1/z, %2.b, > %3.b > > + [ ?Upa , Upa, Upa, Upa; yes ] ^ > > + [ Upa , Upa, Upa, Upa; * ] ^ > > } > > ) > > > > @@ -6861,8 +6889,10 @@ (define_insn "*<logical_nn><mode>3_ptest" > > UNSPEC_PTEST)) > > (clobber (match_scratch:VNx16BI 0))] > > "TARGET_SVE" > > - {@ [ cons: =0, 1 , 2 , 3 ] > > - [ Upa , Upa, Upa, Upa ] <logical_nn>s\t%0.b, %1/z, %2.b, %3.b > > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > > + [ &Upa , Upa, Upa, Upa; yes ] <logical_nn>s\t%0.b, %1/z, %2.b, > %3.b > > + [ ?Upa , Upa, Upa, Upa; yes ] ^ > > + [ Upa , Upa, Upa, Upa; * ] ^ > > } > > ) > > > > @@ -8104,9 +8134,13 @@ (define_insn > "@aarch64_pred_cmp<cmp_op><mode>" > > UNSPEC_PRED_Z)) > > (clobber (reg:CC_NZC CC_REGNUM))] > > "TARGET_SVE" > > - {@ [ cons: =0 , 1 , 3 , 4 ] > > - [ Upa , Upl , w , <sve_imm_con> ] cmp<cmp_op>\t%0.<Vetype>, %1/z, > %3.<Vetype>, #%4 > > - [ Upa , Upl , w , w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, > %3.<Vetype>, %4.<Vetype> > > + {@ [ cons: =0 , 1 , 3 , 4 ; attrs: pred_clobber ] > > + [ &Upa , Upl , w , <sve_imm_con>; yes ] > cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, #%4 > > + [ ?Upa , Upl , w , <sve_imm_con>; yes ] ^ > > + [ Upa , Upl , w , <sve_imm_con>; * ] ^ > > + [ &Upa , Upl , w , w ; yes ] cmp<cmp_op>\t%0.<Vetype>, > %1/z, %3.<Vetype>, %4.<Vetype> > > + [ ?Upa , Upl , w , w ; yes ] ^ > > + [ Upa , Upl , w , w ; * ] ^ > > } > > ) > > > > @@ -8136,9 +8170,13 @@ (define_insn_and_rewrite > "*cmp<cmp_op><mode>_cc" > > UNSPEC_PRED_Z))] > > "TARGET_SVE > > && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" > > - {@ [ cons: =0 , 1 , 2 , 3 ] > > - [ Upa , Upl , w , <sve_imm_con> ] cmp<cmp_op>\t%0.<Vetype>, %1/z, > %2.<Vetype>, #%3 > > - [ Upa , Upl , w , w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, > %2.<Vetype>, %3.<Vetype> > > + {@ [ cons: =0 , 1 , 2 , 3 ; attrs: pred_clobber ] > > + [ &Upa , Upl , w , <sve_imm_con>; yes ] > cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, #%3 > > + [ ?Upa , Upl , w , <sve_imm_con>; yes ] ^ > > + [ Upa , Upl , w , <sve_imm_con>; * ] ^ > > + [ &Upa , Upl , w , w ; yes ] cmp<cmp_op>\t%0.<Vetype>, > %1/z, %2.<Vetype>, %3.<Vetype> > > + [ ?Upa , Upl , w , w ; yes ] ^ > > + [ Upa , Upl , w , w ; * ] ^ > > } > > "&& !rtx_equal_p (operands[4], operands[6])" > > { > > @@ -8166,9 +8204,13 @@ (define_insn_and_rewrite > "*cmp<cmp_op><mode>_ptest" > > (clobber (match_scratch:<VPRED> 0))] > > "TARGET_SVE > > && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" > > - {@ [ cons: =0, 1 , 2 , 3 ] > > - [ Upa , Upl, w , <sve_imm_con> ] cmp<cmp_op>\t%0.<Vetype>, %1/z, > %2.<Vetype>, #%3 > > - [ Upa , Upl, w , w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, > %2.<Vetype>, %3.<Vetype> > > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > > + [ &Upa , Upl, w , <sve_imm_con>; yes ] > cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, #%3 > > + [ ?Upa , Upl, w , <sve_imm_con>; yes ] ^ > > + [ Upa , Upl, w , <sve_imm_con>; * ] ^ > > + [ &Upa , Upl, w , w ; yes ] cmp<cmp_op>\t%0.<Vetype>, > %1/z, %2.<Vetype>, %3.<Vetype> > > + [ ?Upa , Upl, w , w ; yes ] ^ > > + [ Upa , Upl, w , w ; * ] ^ > > } > > "&& !rtx_equal_p (operands[4], operands[6])" > > { > > @@ -8221,8 +8263,10 @@ (define_insn > "@aarch64_pred_cmp<cmp_op><mode>_wide" > > UNSPEC_PRED_Z)) > > (clobber (reg:CC_NZC CC_REGNUM))] > > "TARGET_SVE" > > - {@ [ cons: =0, 1 , 2, 3, 4 ] > > - [ Upa , Upl, , w, w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, > %4.d > > + {@ [ cons: =0, 1 , 2, 3, 4; attrs: pred_clobber ] > > + [ &Upa , Upl, , w, w; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, > %3.<Vetype>, %4.d > > + [ ?Upa , Upl, , w, w; yes ] ^ > > + [ Upa , Upl, , w, w; * ] ^ > > } > > ) > > > > @@ -8254,8 +8298,10 @@ (define_insn > "*aarch64_pred_cmp<cmp_op><mode>_wide_cc" > > UNSPEC_PRED_Z))] > > "TARGET_SVE > > && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" > > - {@ [ cons: =0, 1 , 2, 3, 6 ] > > - [ Upa , Upl, w, w, Upl ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, > %3.d > > + {@ [ cons: =0, 1 , 2, 3, 6 ; attrs: pred_clobber ] > > + [ &Upa , Upl, w, w, Upl; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, > %2.<Vetype>, %3.d > > + [ ?Upa , Upl, w, w, Upl; yes ] ^ > > + [ Upa , Upl, w, w, Upl; * ] ^ > > } > > ) > > > > @@ -8279,8 +8325,10 @@ (define_insn > "*aarch64_pred_cmp<cmp_op><mode>_wide_ptest" > > (clobber (match_scratch:<VPRED> 0))] > > "TARGET_SVE > > && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" > > - {@ [ cons: =0, 1 , 2, 3, 6 ] > > - [ Upa , Upl, w, w, Upl ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, > %3.d > > + {@ [ cons: =0, 1 , 2, 3, 6 ; attrs: pred_clobber ] > > + [ &Upa , Upl, w, w, Upl; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, > %2.<Vetype>, %3.d > > + [ ?Upa , Upl, w, w, Upl; yes ] ^ > > + [ Upa , Upl, w, w, Upl; * ] ^ > > } > > ) > > > > @@ -9948,9 +9996,13 @@ (define_insn "@aarch64_brk<brk_op>" > > (match_operand:VNx16BI 3 "aarch64_simd_reg_or_zero")] > > SVE_BRK_UNARY))] > > "TARGET_SVE" > > - {@ [ cons: =0 , 1 , 2 , 3 ] > > - [ Upa , Upa , Upa , Dz ] brk<brk_op>\t%0.b, %1/z, %2.b > > - [ Upa , Upa , Upa , 0 ] brk<brk_op>\t%0.b, %1/m, %2.b > > + {@ [ cons: =0 , 1 , 2 , 3 ; attrs: pred_clobber ] > > + [ &Upa , Upa , Upa , Dz; yes ] brk<brk_op>\t%0.b, %1/z, %2.b > > + [ ?Upa , Upa , Upa , Dz; yes ] ^ > > + [ Upa , Upa , Upa , Dz; * ] ^ > > + [ &Upa , Upa , Upa , 0 ; yes ] brk<brk_op>\t%0.b, %1/m, %2.b > > + [ ?Upa , Upa , Upa , 0 ; yes ] ^ > > + [ Upa , Upa , Upa , 0 ; * ] ^ > > } > > ) > > > > @@ -9974,8 +10026,10 @@ (define_insn "*aarch64_brk<brk_op>_cc" > > (match_dup 3)] > > SVE_BRK_UNARY))] > > "TARGET_SVE" > > - {@ [ cons: =0, 1 , 2 ] > > - [ Upa , Upa, Upa ] brk<brk_op>s\t%0.b, %1/z, %2.b > > + {@ [ cons: =0, 1 , 2 ; attrs: pred_clobber ] > > + [ &Upa , Upa, Upa; yes ] brk<brk_op>s\t%0.b, %1/z, %2.b > > + [ ?Upa , Upa, Upa; yes ] ^ > > + [ Upa , Upa, Upa; * ] ^ > > } > > ) > > > > @@ -9994,8 +10048,10 @@ (define_insn "*aarch64_brk<brk_op>_ptest" > > UNSPEC_PTEST)) > > (clobber (match_scratch:VNx16BI 0))] > > "TARGET_SVE" > > - {@ [ cons: =0, 1 , 2 ] > > - [ Upa , Upa, Upa ] brk<brk_op>s\t%0.b, %1/z, %2.b > > + {@ [ cons: =0, 1 , 2 ; attrs: pred_clobber ] > > + [ &Upa , Upa, Upa; yes ] brk<brk_op>s\t%0.b, %1/z, %2.b > > + [ ?Upa , Upa, Upa; yes ] ^ > > + [ Upa , Upa, Upa; * ] ^ > > } > > ) > > > > @@ -10020,8 +10076,10 @@ (define_insn "@aarch64_brk<brk_op>" > > (match_operand:VNx16BI 3 "register_operand")] > > SVE_BRK_BINARY))] > > "TARGET_SVE" > > - {@ [ cons: =0, 1 , 2 , 3 ] > > - [ Upa , Upa, Upa, <brk_reg_con> ] brk<brk_op>\t%0.b, %1/z, %2.b, > %<brk_reg_opno>.b > > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > > + [ &Upa , Upa, Upa, <brk_reg_con>; yes ] brk<brk_op>\t%0.b, %1/z, > %2.b, %<brk_reg_opno>.b > > + [ ?Upa , Upa, Upa, <brk_reg_con>; yes ] ^ > > + [ Upa , Upa, Upa, <brk_reg_con>; * ] ^ > > } > > ) > > > > @@ -10046,8 +10104,10 @@ (define_insn_and_rewrite "*aarch64_brkn_cc" > > (match_dup 3)] > > UNSPEC_BRKN))] > > "TARGET_SVE" > > - {@ [ cons: =0, 1 , 2 , 3 ] > > - [ Upa , Upa, Upa, 0 ] brkns\t%0.b, %1/z, %2.b, %0.b > > + {@ [ cons: =0, 1 , 2 , 3; attrs: pred_clobber ] > > + [ &Upa , Upa, Upa, 0; yes ] brkns\t%0.b, %1/z, %2.b, %0.b > > + [ ?Upa , Upa, Upa, 0; yes ] ^ > > + [ Upa , Upa, Upa, 0; * ] ^ > > } > > "&& (operands[4] != CONST0_RTX (VNx16BImode) > > || operands[5] != CONST0_RTX (VNx16BImode))" > > @@ -10072,8 +10132,10 @@ (define_insn_and_rewrite > "*aarch64_brkn_ptest" > > UNSPEC_PTEST)) > > (clobber (match_scratch:VNx16BI 0))] > > "TARGET_SVE" > > - {@ [ cons: =0, 1 , 2 , 3 ] > > - [ Upa , Upa, Upa, 0 ] brkns\t%0.b, %1/z, %2.b, %0.b > > + {@ [ cons: =0, 1 , 2 , 3; attrs: pred_clobber ] > > + [ &Upa , Upa, Upa, 0; yes ] brkns\t%0.b, %1/z, %2.b, %0.b > > + [ ?Upa , Upa, Upa, 0; yes ] ^ > > + [ Upa , Upa, Upa, 0; * ] ^ > > } > > "&& (operands[4] != CONST0_RTX (VNx16BImode) > > || operands[5] != CONST0_RTX (VNx16BImode))" > > @@ -10103,8 +10165,10 @@ (define_insn "*aarch64_brk<brk_op>_cc" > > (match_dup 3)] > > SVE_BRKP))] > > "TARGET_SVE" > > - {@ [ cons: =0, 1 , 2 , 3 ] > > - [ Upa , Upa, Upa, Upa ] brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b > > + {@ [ cons: =0, 1 , 2 , 3 , 4; attrs: pred_clobber ] > > + [ &Upa , Upa, Upa, Upa, ; yes ] brk<brk_op>s\t%0.b, %1/z, %2.b, > %3.b > > + [ ?Upa , Upa, Upa, Upa, ; yes ] ^ > > + [ Upa , Upa, Upa, Upa, ; * ] ^ > > } > > ) > > > > @@ -10123,8 +10187,10 @@ (define_insn "*aarch64_brk<brk_op>_ptest" > > UNSPEC_PTEST)) > > (clobber (match_scratch:VNx16BI 0))] > > "TARGET_SVE" > > - {@ [ cons: =0, 1 , 2 , 3 ] > > - [ Upa , Upa, Upa, Upa ] brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b > > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > > + [ &Upa , Upa, Upa, Upa; yes ] brk<brk_op>s\t%0.b, %1/z, %2.b, > %3.b > > + [ ?Upa , Upa, Upa, Upa; yes ] ^ > > + [ Upa , Upa, Upa, Upa; * ] ^ > > } > > ) > > > > diff --git a/gcc/config/aarch64/aarch64-sve2.md > b/gcc/config/aarch64/aarch64-sve2.md > > index > aa12baf48355358ca4fefe88157df3aac6eb09bd..1a49494a69d8335e5f7d3ef4b > d3a90d0805bba84 100644 > > --- a/gcc/config/aarch64/aarch64-sve2.md > > +++ b/gcc/config/aarch64/aarch64-sve2.md > > @@ -3349,8 +3349,10 @@ (define_insn > "@aarch64_pred_<sve_int_op><mode>" > > UNSPEC_PRED_Z)) > > (clobber (reg:CC_NZC CC_REGNUM))] > > "TARGET_SVE2 && TARGET_NON_STREAMING" > > - {@ [ cons: =0, 1 , 2, 3, 4 ] > > - [ Upa , Upl, , w, w ] <sve_int_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, > %4.<Vetype> > > + {@ [ cons: =0, 1 , 2, 3, 4; attrs: pred_clobber ] > > + [ &Upa , Upl, , w, w; yes ] <sve_int_op>\t%0.<Vetype>, %1/z, > %3.<Vetype>, %4.<Vetype> > > + [ ?Upa , Upl, , w, w; yes ] ^ > > + [ Upa , Upl, , w, w; * ] ^ > > } > > )
Tamar Christina <Tamar.Christina@arm.com> writes: >> -----Original Message----- >> From: Richard Sandiford <richard.sandiford@arm.com> >> Sent: Wednesday, May 22, 2024 10:48 AM >> To: Tamar Christina <Tamar.Christina@arm.com> >> Cc: gcc-patches@gcc.gnu.org; nd <nd@arm.com>; Richard Earnshaw >> <Richard.Earnshaw@arm.com>; Marcus Shawcroft >> <Marcus.Shawcroft@arm.com>; ktkachov@gcc.gnu.org >> Subject: Re: [PATCH 3/4]AArch64: add new alternative with early clobber to >> patterns >> >> Tamar Christina <tamar.christina@arm.com> writes: >> > Hi All, >> > >> > This patch adds new alternatives to the patterns which are affected. The new >> > alternatives with the conditional early clobbers are added before the normal >> > ones in order for LRA to prefer them in the event that we have enough free >> > registers to accommodate them. >> > >> > In case register pressure is too high the normal alternatives will be preferred >> > before a reload is considered as we rather have the tie than a spill. >> > >> > Tests are in the next patch. >> > >> > Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. >> > >> > Ok for master? >> > >> > Thanks, >> > Tamar >> > >> > gcc/ChangeLog: >> > >> > * config/aarch64/aarch64-sve.md (and<mode>3, >> > @aarch64_pred_<optab><mode>_z, *<optab><mode>3_cc, >> > *<optab><mode>3_ptest, aarch64_pred_<nlogical><mode>_z, >> > *<nlogical><mode>3_cc, *<nlogical><mode>3_ptest, >> > aarch64_pred_<logical_nn><mode>_z, *<logical_nn><mode>3_cc, >> > *<logical_nn><mode>3_ptest, @aarch64_pred_cmp<cmp_op><mode>, >> > *cmp<cmp_op><mode>_cc, *cmp<cmp_op><mode>_ptest, >> > @aarch64_pred_cmp<cmp_op><mode>_wide, >> > *aarch64_pred_cmp<cmp_op><mode>_wide_cc, >> > *aarch64_pred_cmp<cmp_op><mode>_wide_ptest, >> @aarch64_brk<brk_op>, >> > *aarch64_brk<brk_op>_cc, *aarch64_brk<brk_op>_ptest, >> > @aarch64_brk<brk_op>, *aarch64_brkn_cc, *aarch64_brkn_ptest, >> > *aarch64_brk<brk_op>_cc, *aarch64_brk<brk_op>_ptest, >> > aarch64_rdffr_z, *aarch64_rdffr_z_ptest, *aarch64_rdffr_ptest, >> > *aarch64_rdffr_z_cc, *aarch64_rdffr_cc): Add new early clobber >> > alternative. >> > * config/aarch64/aarch64-sve2.md >> > (@aarch64_pred_<sve_int_op><mode>): Likewise. >> > >> > --- >> > diff --git a/gcc/config/aarch64/aarch64-sve.md b/gcc/config/aarch64/aarch64- >> sve.md >> > index >> e3085c0c636f1317409bbf3b5fbaf5342a2df1f6..8fdc1bc3cd43acfcd675a18350c >> 297428c85fe46 100644 >> > --- a/gcc/config/aarch64/aarch64-sve.md >> > +++ b/gcc/config/aarch64/aarch64-sve.md >> > @@ -1161,8 +1161,10 @@ (define_insn "aarch64_rdffr_z" >> > (reg:VNx16BI FFRT_REGNUM) >> > (match_operand:VNx16BI 1 "register_operand")))] >> > "TARGET_SVE && TARGET_NON_STREAMING" >> > - {@ [ cons: =0, 1 ] >> > - [ Upa , Upa ] rdffr\t%0.b, %1/z >> > + {@ [ cons: =0, 1 ; attrs: pred_clobber ] >> > + [ &Upa , Upa; yes ] rdffr\t%0.b, %1/z >> > + [ ?Upa , Upa; yes ] ^ >> > + [ Upa , Upa; * ] ^ >> > } >> > ) >> >> Sorry for not explaining it very well, but in the previous review I suggested: >> >> > The gather-like approach would be something like: >> > >> > [ &Upa , Upl , w , <sve_imm_con>; yes ] >> cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, #%4 >> > [ ?Upl , 0 , w , <sve_imm_con>; yes ] ^ >> > [ Upa , Upl , w , <sve_imm_con>; no ] ^ >> > [ &Upa , Upl , w , w ; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, >> %3.<Vetype>, %4.<Vetype> >> > [ ?Upl , 0 , w , w ; yes ] ^ >> > [ Upa , Upl , w , w ; no ] ^ >> > >> > with: >> > >> > (define_attr "pred_clobber" "any,no,yes" (const_string "any")) >> >> (with emphasis on the last line). What I didn't say explicitly is >> that "no" should require !TARGET_SVE_PRED_CLOBBER. >> >> The premise of that review was that we shouldn't enable things like: >> >> [ Upa , Upl , w , w ; no ] ^ >> >> for TARGET_SVE_PRED_CLOBBER since it contradicts the earlyclobber >> alternative. So we should enable either the pred_clobber=yes >> alternatives or the pred_clobber=no alternatives, but not both. >> >> The default "any" is then for other non-predicate instructions that >> don't care about TARGET_SVE_PRED_CLOBBER either way. >> >> In contrast, this patch makes pred_clobber=yes enable the alternatives >> that correctly describe the restriction (good!) but then also enables >> the normal alternatives too, which IMO makes the semantics unclear. > > Sure, the reason I still had that is because this ICEs under high register > pressure: > > {@ [ cons: =0 , 1 , 3 , 4 ; attrs: pred_clobber ] > [ &Upa , Upl , w , <sve_imm_con>; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, #%4 > [ ?Upa , 0 , w , <sve_imm_con>; yes ] ^ > [ Upa , Upl , w , <sve_imm_con>; no ] ^ > [ &Upa , Upl , w , w ; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.<Vetype> > [ ?Upa , 0 , w , w ; yes ] ^ > [ Upa , Upl , w , w ; no ] ^ > } > > So now in the `yes` case reload does: > > Considering alt=0 of insn 10: (0) =&Upa (1) Upl (3) w (4) vsd > 0 Small class reload: reject+=3 > 0 Non input pseudo reload: reject++ > 0 Early clobber: reject++ > Bad operand -- refuse > Considering alt=1 of insn 10: (0) ?Upa (1) 0 (3) w (4) vsd > Staticly defined alt reject+=6 > 0 Small class reload: reject+=3 > 0 Non input pseudo reload: reject++ > 1 Dying matched operand reload: reject++ > 1 Small class reload: reject+=3 > Bad operand -- refuse > Considering alt=3 of insn 10: (0) &Upa (1) Upl (3) w (4) w > 0 Small class reload: reject+=3 > 0 Non input pseudo reload: reject++ > 0 Early clobber: reject++ > overall=11,losers=1,rld_nregs=1 > Considering alt=4 of insn 10: (0) ?Upa (1) 0 (3) w (4) w > Staticly defined alt reject+=6 > 0 Small class reload: reject+=3 > 0 Non input pseudo reload: reject++ > overall=16,losers=1 -- refuse > Choosing alt 3 in insn 10: (0) &Upa (1) Upl (3) w (4) w {aarch64_pred_cmplovnx8hi} > > And the penalty of alt=4 makes it pick alt=3 even though it doesn't have the free registers > for it. alt=4 would have worked. By "high register pressure", do you mean if predicate registers are disabled using -ffixed? If so, that's ok in itself. That kind of ICE shouldn't occur in real use. > I believe this now follows exactly what was suggested: > > 1. provide an early clobber alternative > 2. provide an explicit tie alternative with an increase in cost for using it > 3. provide a general/normal alternative that is only enabled when the first two aren't. > > Having read the email a number of times.. did I somehow miss something? But how is (3) arranged? It looks like the normal alternative is enabled unconditionally, in the sense that the "enabled" attribute is always "yes". Thanks, Richard
> -----Original Message----- > From: Richard Sandiford <richard.sandiford@arm.com> > Sent: Wednesday, May 22, 2024 12:24 PM > To: Tamar Christina <Tamar.Christina@arm.com> > Cc: gcc-patches@gcc.gnu.org; nd <nd@arm.com>; Richard Earnshaw > <Richard.Earnshaw@arm.com>; Marcus Shawcroft > <Marcus.Shawcroft@arm.com>; ktkachov@gcc.gnu.org > Subject: Re: [PATCH 3/4]AArch64: add new alternative with early clobber to > patterns > > Tamar Christina <Tamar.Christina@arm.com> writes: > >> -----Original Message----- > >> From: Richard Sandiford <richard.sandiford@arm.com> > >> Sent: Wednesday, May 22, 2024 10:48 AM > >> To: Tamar Christina <Tamar.Christina@arm.com> > >> Cc: gcc-patches@gcc.gnu.org; nd <nd@arm.com>; Richard Earnshaw > >> <Richard.Earnshaw@arm.com>; Marcus Shawcroft > >> <Marcus.Shawcroft@arm.com>; ktkachov@gcc.gnu.org > >> Subject: Re: [PATCH 3/4]AArch64: add new alternative with early clobber to > >> patterns > >> > >> Tamar Christina <tamar.christina@arm.com> writes: > >> > Hi All, > >> > > >> > This patch adds new alternatives to the patterns which are affected. The new > >> > alternatives with the conditional early clobbers are added before the normal > >> > ones in order for LRA to prefer them in the event that we have enough free > >> > registers to accommodate them. > >> > > >> > In case register pressure is too high the normal alternatives will be preferred > >> > before a reload is considered as we rather have the tie than a spill. > >> > > >> > Tests are in the next patch. > >> > > >> > Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. > >> > > >> > Ok for master? > >> > > >> > Thanks, > >> > Tamar > >> > > >> > gcc/ChangeLog: > >> > > >> > * config/aarch64/aarch64-sve.md (and<mode>3, > >> > @aarch64_pred_<optab><mode>_z, *<optab><mode>3_cc, > >> > *<optab><mode>3_ptest, aarch64_pred_<nlogical><mode>_z, > >> > *<nlogical><mode>3_cc, *<nlogical><mode>3_ptest, > >> > aarch64_pred_<logical_nn><mode>_z, *<logical_nn><mode>3_cc, > >> > *<logical_nn><mode>3_ptest, @aarch64_pred_cmp<cmp_op><mode>, > >> > *cmp<cmp_op><mode>_cc, *cmp<cmp_op><mode>_ptest, > >> > @aarch64_pred_cmp<cmp_op><mode>_wide, > >> > *aarch64_pred_cmp<cmp_op><mode>_wide_cc, > >> > *aarch64_pred_cmp<cmp_op><mode>_wide_ptest, > >> @aarch64_brk<brk_op>, > >> > *aarch64_brk<brk_op>_cc, *aarch64_brk<brk_op>_ptest, > >> > @aarch64_brk<brk_op>, *aarch64_brkn_cc, *aarch64_brkn_ptest, > >> > *aarch64_brk<brk_op>_cc, *aarch64_brk<brk_op>_ptest, > >> > aarch64_rdffr_z, *aarch64_rdffr_z_ptest, *aarch64_rdffr_ptest, > >> > *aarch64_rdffr_z_cc, *aarch64_rdffr_cc): Add new early clobber > >> > alternative. > >> > * config/aarch64/aarch64-sve2.md > >> > (@aarch64_pred_<sve_int_op><mode>): Likewise. > >> > > >> > --- > >> > diff --git a/gcc/config/aarch64/aarch64-sve.md > b/gcc/config/aarch64/aarch64- > >> sve.md > >> > index > >> > e3085c0c636f1317409bbf3b5fbaf5342a2df1f6..8fdc1bc3cd43acfcd675a18350c > >> 297428c85fe46 100644 > >> > --- a/gcc/config/aarch64/aarch64-sve.md > >> > +++ b/gcc/config/aarch64/aarch64-sve.md > >> > @@ -1161,8 +1161,10 @@ (define_insn "aarch64_rdffr_z" > >> > (reg:VNx16BI FFRT_REGNUM) > >> > (match_operand:VNx16BI 1 "register_operand")))] > >> > "TARGET_SVE && TARGET_NON_STREAMING" > >> > - {@ [ cons: =0, 1 ] > >> > - [ Upa , Upa ] rdffr\t%0.b, %1/z > >> > + {@ [ cons: =0, 1 ; attrs: pred_clobber ] > >> > + [ &Upa , Upa; yes ] rdffr\t%0.b, %1/z > >> > + [ ?Upa , Upa; yes ] ^ > >> > + [ Upa , Upa; * ] ^ > >> > } > >> > ) > >> > >> Sorry for not explaining it very well, but in the previous review I suggested: > >> > >> > The gather-like approach would be something like: > >> > > >> > [ &Upa , Upl , w , <sve_imm_con>; yes ] > >> cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, #%4 > >> > [ ?Upl , 0 , w , <sve_imm_con>; yes ] ^ > >> > [ Upa , Upl , w , <sve_imm_con>; no ] ^ > >> > [ &Upa , Upl , w , w ; yes ] cmp<cmp_op>\t%0.<Vetype>, > %1/z, > >> %3.<Vetype>, %4.<Vetype> > >> > [ ?Upl , 0 , w , w ; yes ] ^ > >> > [ Upa , Upl , w , w ; no ] ^ > >> > > >> > with: > >> > > >> > (define_attr "pred_clobber" "any,no,yes" (const_string "any")) > >> > >> (with emphasis on the last line). What I didn't say explicitly is > >> that "no" should require !TARGET_SVE_PRED_CLOBBER. > >> > >> The premise of that review was that we shouldn't enable things like: > >> > >> [ Upa , Upl , w , w ; no ] ^ > >> > >> for TARGET_SVE_PRED_CLOBBER since it contradicts the earlyclobber > >> alternative. So we should enable either the pred_clobber=yes > >> alternatives or the pred_clobber=no alternatives, but not both. > >> > >> The default "any" is then for other non-predicate instructions that > >> don't care about TARGET_SVE_PRED_CLOBBER either way. > >> > >> In contrast, this patch makes pred_clobber=yes enable the alternatives > >> that correctly describe the restriction (good!) but then also enables > >> the normal alternatives too, which IMO makes the semantics unclear. > > > > Sure, the reason I still had that is because this ICEs under high register > > pressure: > > > > {@ [ cons: =0 , 1 , 3 , 4 ; attrs: pred_clobber ] > > [ &Upa , Upl , w , <sve_imm_con>; yes ] > cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, #%4 > > [ ?Upa , 0 , w , <sve_imm_con>; yes ] ^ > > [ Upa , Upl , w , <sve_imm_con>; no ] ^ > > [ &Upa , Upl , w , w ; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, > %3.<Vetype>, %4.<Vetype> > > [ ?Upa , 0 , w , w ; yes ] ^ > > [ Upa , Upl , w , w ; no ] ^ > > } > > > > So now in the `yes` case reload does: > > > > Considering alt=0 of insn 10: (0) =&Upa (1) Upl (3) w (4) vsd > > 0 Small class reload: reject+=3 > > 0 Non input pseudo reload: reject++ > > 0 Early clobber: reject++ > > Bad operand -- refuse > > Considering alt=1 of insn 10: (0) ?Upa (1) 0 (3) w (4) vsd > > Staticly defined alt reject+=6 > > 0 Small class reload: reject+=3 > > 0 Non input pseudo reload: reject++ > > 1 Dying matched operand reload: reject++ > > 1 Small class reload: reject+=3 > > Bad operand -- refuse > > Considering alt=3 of insn 10: (0) &Upa (1) Upl (3) w (4) w > > 0 Small class reload: reject+=3 > > 0 Non input pseudo reload: reject++ > > 0 Early clobber: reject++ > > overall=11,losers=1,rld_nregs=1 > > Considering alt=4 of insn 10: (0) ?Upa (1) 0 (3) w (4) w > > Staticly defined alt reject+=6 > > 0 Small class reload: reject+=3 > > 0 Non input pseudo reload: reject++ > > overall=16,losers=1 -- refuse > > Choosing alt 3 in insn 10: (0) &Upa (1) Upl (3) w (4) w > {aarch64_pred_cmplovnx8hi} > > > > And the penalty of alt=4 makes it pick alt=3 even though it doesn't have the free > registers > > for it. alt=4 would have worked. > > By "high register pressure", do you mean if predicate registers are > disabled using -ffixed? If so, that's ok in itself. That kind of > ICE shouldn't occur in real use. > > > I believe this now follows exactly what was suggested: > > > > 1. provide an early clobber alternative > > 2. provide an explicit tie alternative with an increase in cost for using it > > 3. provide a general/normal alternative that is only enabled when the first two > aren't. > > > > Having read the email a number of times.. did I somehow miss something? > > But how is (3) arranged? It looks like the normal alternative is enabled > unconditionally, in the sense that the "enabled" attribute is always "yes". > Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. Ok for master? Thanks, Tamar gcc/ChangeLog: * config/aarch64/aarch64-sve.md (and<mode>3, @aarch64_pred_<optab><mode>_z, *<optab><mode>3_cc, *<optab><mode>3_ptest, aarch64_pred_<nlogical><mode>_z, *<nlogical><mode>3_cc, *<nlogical><mode>3_ptest, aarch64_pred_<logical_nn><mode>_z, *<logical_nn><mode>3_cc, *<logical_nn><mode>3_ptest, @aarch64_pred_cmp<cmp_op><mode>, *cmp<cmp_op><mode>_cc, *cmp<cmp_op><mode>_ptest, @aarch64_pred_cmp<cmp_op><mode>_wide, *aarch64_pred_cmp<cmp_op><mode>_wide_cc, *aarch64_pred_cmp<cmp_op><mode>_wide_ptest, @aarch64_brk<brk_op>, *aarch64_brk<brk_op>_cc, *aarch64_brk<brk_op>_ptest, @aarch64_brk<brk_op>, *aarch64_brkn_cc, *aarch64_brkn_ptest, *aarch64_brk<brk_op>_cc, *aarch64_brk<brk_op>_ptest, aarch64_rdffr_z, *aarch64_rdffr_z_ptest, *aarch64_rdffr_ptest, *aarch64_rdffr_z_cc, *aarch64_rdffr_cc): Add new early clobber alternative. * config/aarch64/aarch64-sve2.md (@aarch64_pred_<sve_int_op><mode>): Likewise. -- inline copy of patch -- diff --git a/gcc/config/aarch64/aarch64-sve.md b/gcc/config/aarch64/aarch64-sve.md index e3085c0c636f1317409bbf3b5fbaf5342a2df1f6..ca97750f5c3549bbb3a89aa41acb4edfac3f1b85 100644 --- a/gcc/config/aarch64/aarch64-sve.md +++ b/gcc/config/aarch64/aarch64-sve.md @@ -1161,8 +1161,10 @@ (define_insn "aarch64_rdffr_z" (reg:VNx16BI FFRT_REGNUM) (match_operand:VNx16BI 1 "register_operand")))] "TARGET_SVE && TARGET_NON_STREAMING" - {@ [ cons: =0, 1 ] - [ Upa , Upa ] rdffr\t%0.b, %1/z + {@ [ cons: =0, 1 ; attrs: pred_clobber ] + [ &Upa , Upa; yes ] rdffr\t%0.b, %1/z + [ ?Upa , 0 ; yes ] ^ + [ Upa , Upa; no ] ^ } ) @@ -1179,8 +1181,10 @@ (define_insn "*aarch64_rdffr_z_ptest" UNSPEC_PTEST)) (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE && TARGET_NON_STREAMING" - {@ [ cons: =0, 1 ] - [ Upa , Upa ] rdffrs\t%0.b, %1/z + {@ [ cons: =0, 1 ; attrs: pred_clobber ] + [ &Upa , Upa; yes ] rdffrs\t%0.b, %1/z + [ ?Upa , 0 ; yes ] ^ + [ Upa , Upa; no ] ^ } ) @@ -1195,8 +1199,10 @@ (define_insn "*aarch64_rdffr_ptest" UNSPEC_PTEST)) (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE && TARGET_NON_STREAMING" - {@ [ cons: =0, 1 ] - [ Upa , Upa ] rdffrs\t%0.b, %1/z + {@ [ cons: =0, 1 ; attrs: pred_clobber ] + [ &Upa , Upa; yes ] rdffrs\t%0.b, %1/z + [ ?Upa , 0 ; yes ] ^ + [ Upa , Upa; no ] ^ } ) @@ -1216,8 +1222,10 @@ (define_insn "*aarch64_rdffr_z_cc" (reg:VNx16BI FFRT_REGNUM) (match_dup 1)))] "TARGET_SVE && TARGET_NON_STREAMING" - {@ [ cons: =0, 1 ] - [ Upa , Upa ] rdffrs\t%0.b, %1/z + {@ [ cons: =0, 1 ; attrs: pred_clobber ] + [ &Upa , Upa; yes ] rdffrs\t%0.b, %1/z + [ ?Upa , 0 ; yes ] ^ + [ Upa , Upa; no ] ^ } ) @@ -1233,8 +1241,10 @@ (define_insn "*aarch64_rdffr_cc" (set (match_operand:VNx16BI 0 "register_operand") (reg:VNx16BI FFRT_REGNUM))] "TARGET_SVE && TARGET_NON_STREAMING" - {@ [ cons: =0, 1 ] - [ Upa , Upa ] rdffrs\t%0.b, %1/z + {@ [ cons: =0, 1 ; attrs: pred_clobber ] + [ &Upa , Upa; yes ] rdffrs\t%0.b, %1/z + [ ?Upa , 0 ; yes ] ^ + [ Upa , Upa; no ] ^ } ) @@ -6651,8 +6661,10 @@ (define_insn "and<mode>3" (and:PRED_ALL (match_operand:PRED_ALL 1 "register_operand") (match_operand:PRED_ALL 2 "register_operand")))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 ] - [ Upa , Upa, Upa ] and\t%0.b, %1/z, %2.b, %2.b + {@ [ cons: =0, 1 , 2 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa; yes ] and\t%0.b, %1/z, %2.b, %2.b + [ ?Upa , 0 , Upa; yes ] ^ + [ Upa , Upa, Upa; no ] ^ } ) @@ -6679,8 +6691,10 @@ (define_insn "@aarch64_pred_<optab><mode>_z" (match_operand:PRED_ALL 3 "register_operand")) (match_operand:PRED_ALL 1 "register_operand")))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] <logical>\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa; yes ] <logical>\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , 0 , Upa, Upa; yes ] ^ + [ Upa , Upa, Upa, Upa; no ] ^ } ) @@ -6703,8 +6717,10 @@ (define_insn "*<optab><mode>3_cc" (and:PRED_ALL (LOGICAL:PRED_ALL (match_dup 2) (match_dup 3)) (match_dup 4)))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] <logical>s\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa; yes ] <logical>s\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , 0 , Upa, Upa; yes ] ^ + [ Upa , Upa, Upa, Upa; no ] ^ } ) @@ -6723,8 +6739,10 @@ (define_insn "*<optab><mode>3_ptest" UNSPEC_PTEST)) (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] <logical>s\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa; yes ] <logical>s\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , 0 , Upa, Upa; yes ] ^ + [ Upa , Upa, Upa, Upa; no ] ^ } ) @@ -6745,8 +6763,10 @@ (define_insn "aarch64_pred_<nlogical><mode>_z" (match_operand:PRED_ALL 2 "register_operand")) (match_operand:PRED_ALL 1 "register_operand")))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] <nlogical>\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa; yes ] <nlogical>\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , 0 , Upa, Upa; yes ] ^ + [ Upa , Upa, Upa, Upa; no ] ^ } ) @@ -6770,8 +6790,10 @@ (define_insn "*<nlogical><mode>3_cc" (match_dup 2)) (match_dup 4)))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] <nlogical>s\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa; yes ] <nlogical>s\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , 0 , Upa, Upa; yes ] ^ + [ Upa , Upa, Upa, Upa; no ] ^ } ) @@ -6791,8 +6813,10 @@ (define_insn "*<nlogical><mode>3_ptest" UNSPEC_PTEST)) (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] <nlogical>s\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa; yes ] <nlogical>s\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , 0 , Upa, Upa; yes ] ^ + [ Upa , Upa, Upa, Upa; no ] ^ } ) @@ -6813,8 +6837,10 @@ (define_insn "aarch64_pred_<logical_nn><mode>_z" (not:PRED_ALL (match_operand:PRED_ALL 3 "register_operand"))) (match_operand:PRED_ALL 1 "register_operand")))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] <logical_nn>\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa; yes ] <logical_nn>\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , 0 , Upa, Upa; yes ] ^ + [ Upa , Upa, Upa, Upa; no ] ^ } ) @@ -6839,8 +6865,10 @@ (define_insn "*<logical_nn><mode>3_cc" (not:PRED_ALL (match_dup 3))) (match_dup 4)))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] <logical_nn>s\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa; yes ] <logical_nn>s\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , 0 , Upa, Upa; yes ] ^ + [ Upa , Upa, Upa, Upa; no ] ^ } ) @@ -6861,8 +6889,10 @@ (define_insn "*<logical_nn><mode>3_ptest" UNSPEC_PTEST)) (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] <logical_nn>s\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa; yes ] <logical_nn>s\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , 0 , Upa, Upa; yes ] ^ + [ Upa , Upa, Upa, Upa; no ] ^ } ) @@ -8104,9 +8134,13 @@ (define_insn "@aarch64_pred_cmp<cmp_op><mode>" UNSPEC_PRED_Z)) (clobber (reg:CC_NZC CC_REGNUM))] "TARGET_SVE" - {@ [ cons: =0 , 1 , 3 , 4 ] - [ Upa , Upl , w , <sve_imm_con> ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, #%4 - [ Upa , Upl , w , w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.<Vetype> + {@ [ cons: =0 , 1 , 3 , 4 ; attrs: pred_clobber ] + [ &Upa , Upl , w , <sve_imm_con>; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, #%4 + [ ?Upa , 0 , w , <sve_imm_con>; yes ] ^ + [ Upa , Upl , w , <sve_imm_con>; no ] ^ + [ &Upa , Upl , w , w ; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.<Vetype> + [ ?Upa , 0 , w , w ; yes ] ^ + [ Upa , Upl , w , w ; no ] ^ } ) @@ -8136,9 +8170,13 @@ (define_insn_and_rewrite "*cmp<cmp_op><mode>_cc" UNSPEC_PRED_Z))] "TARGET_SVE && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" - {@ [ cons: =0 , 1 , 2 , 3 ] - [ Upa , Upl , w , <sve_imm_con> ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, #%3 - [ Upa , Upl , w , w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.<Vetype> + {@ [ cons: =0 , 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upl , w , <sve_imm_con>; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, #%3 + [ ?Upa , 0 , w , <sve_imm_con>; yes ] ^ + [ Upa , Upl , w , <sve_imm_con>; no ] ^ + [ &Upa , Upl , w , w ; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.<Vetype> + [ ?Upa , 0 , w , w ; yes ] ^ + [ Upa , Upl , w , w ; no ] ^ } "&& !rtx_equal_p (operands[4], operands[6])" { @@ -8166,9 +8204,13 @@ (define_insn_and_rewrite "*cmp<cmp_op><mode>_ptest" (clobber (match_scratch:<VPRED> 0))] "TARGET_SVE && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upl, w , <sve_imm_con> ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, #%3 - [ Upa , Upl, w , w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.<Vetype> + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upl, w , <sve_imm_con>; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, #%3 + [ ?Upa , 0 , w , <sve_imm_con>; yes ] ^ + [ Upa , Upl, w , <sve_imm_con>; no ] ^ + [ &Upa , Upl, w , w ; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.<Vetype> + [ ?Upa , 0 , w , w ; yes ] ^ + [ Upa , Upl, w , w ; no ] ^ } "&& !rtx_equal_p (operands[4], operands[6])" { @@ -8221,8 +8263,10 @@ (define_insn "@aarch64_pred_cmp<cmp_op><mode>_wide" UNSPEC_PRED_Z)) (clobber (reg:CC_NZC CC_REGNUM))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2, 3, 4 ] - [ Upa , Upl, , w, w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.d + {@ [ cons: =0, 1 , 2, 3, 4; attrs: pred_clobber ] + [ &Upa , Upl, , w, w; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.d + [ ?Upa , 0 , , w, w; yes ] ^ + [ Upa , Upl, , w, w; no ] ^ } ) @@ -8254,8 +8298,10 @@ (define_insn "*aarch64_pred_cmp<cmp_op><mode>_wide_cc" UNSPEC_PRED_Z))] "TARGET_SVE && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" - {@ [ cons: =0, 1 , 2, 3, 6 ] - [ Upa , Upl, w, w, Upl ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.d + {@ [ cons: =0, 1 , 2, 3, 6 ; attrs: pred_clobber ] + [ &Upa , Upl, w, w, Upl; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.d + [ ?Upa , 0 , w, w, Upl; yes ] ^ + [ Upa , Upl, w, w, Upl; no ] ^ } ) @@ -8279,8 +8325,10 @@ (define_insn "*aarch64_pred_cmp<cmp_op><mode>_wide_ptest" (clobber (match_scratch:<VPRED> 0))] "TARGET_SVE && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" - {@ [ cons: =0, 1 , 2, 3, 6 ] - [ Upa , Upl, w, w, Upl ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.d + {@ [ cons: =0, 1 , 2, 3, 6 ; attrs: pred_clobber ] + [ &Upa , Upl, w, w, Upl; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.d + [ ?Upa , 0 , w, w, Upl; yes ] ^ + [ Upa , Upl, w, w, Upl; no ] ^ } ) @@ -9948,9 +9996,13 @@ (define_insn "@aarch64_brk<brk_op>" (match_operand:VNx16BI 3 "aarch64_simd_reg_or_zero")] SVE_BRK_UNARY))] "TARGET_SVE" - {@ [ cons: =0 , 1 , 2 , 3 ] - [ Upa , Upa , Upa , Dz ] brk<brk_op>\t%0.b, %1/z, %2.b - [ Upa , Upa , Upa , 0 ] brk<brk_op>\t%0.b, %1/m, %2.b + {@ [ cons: =0 , 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa , Upa , Dz; yes ] brk<brk_op>\t%0.b, %1/z, %2.b + [ ?Upa , 0 , Upa , Dz; yes ] ^ + [ Upa , Upa , Upa , Dz; no ] ^ + [ &Upa , Upa , Upa , 0 ; yes ] brk<brk_op>\t%0.b, %1/m, %2.b + [ ?Upa , 0 , Upa , 0 ; yes ] ^ + [ Upa , Upa , Upa , 0 ; no ] ^ } ) @@ -9974,8 +10026,10 @@ (define_insn "*aarch64_brk<brk_op>_cc" (match_dup 3)] SVE_BRK_UNARY))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 ] - [ Upa , Upa, Upa ] brk<brk_op>s\t%0.b, %1/z, %2.b + {@ [ cons: =0, 1 , 2 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa; yes ] brk<brk_op>s\t%0.b, %1/z, %2.b + [ ?Upa , 0 , Upa; yes ] ^ + [ Upa , Upa, Upa; no ] ^ } ) @@ -9994,8 +10048,10 @@ (define_insn "*aarch64_brk<brk_op>_ptest" UNSPEC_PTEST)) (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 ] - [ Upa , Upa, Upa ] brk<brk_op>s\t%0.b, %1/z, %2.b + {@ [ cons: =0, 1 , 2 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa; yes ] brk<brk_op>s\t%0.b, %1/z, %2.b + [ ?Upa , 0 , Upa; yes ] ^ + [ Upa , Upa, Upa; no ] ^ } ) @@ -10020,8 +10076,10 @@ (define_insn "@aarch64_brk<brk_op>" (match_operand:VNx16BI 3 "register_operand")] SVE_BRK_BINARY))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, <brk_reg_con> ] brk<brk_op>\t%0.b, %1/z, %2.b, %<brk_reg_opno>.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, <brk_reg_con>; yes ] brk<brk_op>\t%0.b, %1/z, %2.b, %<brk_reg_opno>.b + [ ?Upa , 0 , Upa, <brk_reg_con>; yes ] ^ + [ Upa , Upa, Upa, <brk_reg_con>; no ] ^ } ) @@ -10046,8 +10104,10 @@ (define_insn_and_rewrite "*aarch64_brkn_cc" (match_dup 3)] UNSPEC_BRKN))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, 0 ] brkns\t%0.b, %1/z, %2.b, %0.b + {@ [ cons: =0, 1 , 2 , 3; attrs: pred_clobber ] + [ &Upa , Upa, Upa, 0; yes ] brkns\t%0.b, %1/z, %2.b, %0.b + [ ?Upa , 0 , Upa, 0; yes ] ^ + [ Upa , Upa, Upa, 0; no ] ^ } "&& (operands[4] != CONST0_RTX (VNx16BImode) || operands[5] != CONST0_RTX (VNx16BImode))" @@ -10072,8 +10132,10 @@ (define_insn_and_rewrite "*aarch64_brkn_ptest" UNSPEC_PTEST)) (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, 0 ] brkns\t%0.b, %1/z, %2.b, %0.b + {@ [ cons: =0, 1 , 2 , 3; attrs: pred_clobber ] + [ &Upa , Upa, Upa, 0; yes ] brkns\t%0.b, %1/z, %2.b, %0.b + [ ?Upa , 0 , Upa, 0; yes ] ^ + [ Upa , Upa, Upa, 0; no ] ^ } "&& (operands[4] != CONST0_RTX (VNx16BImode) || operands[5] != CONST0_RTX (VNx16BImode))" @@ -10103,8 +10165,10 @@ (define_insn "*aarch64_brk<brk_op>_cc" (match_dup 3)] SVE_BRKP))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 , 4; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa, ; yes ] brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , 0 , Upa, Upa, ; yes ] ^ + [ Upa , Upa, Upa, Upa, ; no ] ^ } ) @@ -10123,8 +10187,10 @@ (define_insn "*aarch64_brk<brk_op>_ptest" UNSPEC_PTEST)) (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa; yes ] brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , 0 , Upa, Upa; yes ] ^ + [ Upa , Upa, Upa, Upa; no ] ^ } ) diff --git a/gcc/config/aarch64/aarch64-sve2.md b/gcc/config/aarch64/aarch64-sve2.md index aa12baf48355358ca4fefe88157df3aac6eb09bd..62634e16daad577c5de710c1fc5c8fa542282265 100644 --- a/gcc/config/aarch64/aarch64-sve2.md +++ b/gcc/config/aarch64/aarch64-sve2.md @@ -3349,8 +3349,10 @@ (define_insn "@aarch64_pred_<sve_int_op><mode>" UNSPEC_PRED_Z)) (clobber (reg:CC_NZC CC_REGNUM))] "TARGET_SVE2 && TARGET_NON_STREAMING" - {@ [ cons: =0, 1 , 2, 3, 4 ] - [ Upa , Upl, , w, w ] <sve_int_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.<Vetype> + {@ [ cons: =0, 1 , 2, 3, 4; attrs: pred_clobber ] + [ &Upa , Upl, , w, w; yes ] <sve_int_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.<Vetype> + [ ?Upa , 0 , , w, w; yes ] ^ + [ Upa , Upl, , w, w; no ] ^ } )
Tamar Christina <Tamar.Christina@arm.com> writes: > [...] > @@ -6651,8 +6661,10 @@ (define_insn "and<mode>3" > (and:PRED_ALL (match_operand:PRED_ALL 1 "register_operand") > (match_operand:PRED_ALL 2 "register_operand")))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 ] > - [ Upa , Upa, Upa ] and\t%0.b, %1/z, %2.b, %2.b > + {@ [ cons: =0, 1 , 2 ; attrs: pred_clobber ] > + [ &Upa , Upa, Upa; yes ] and\t%0.b, %1/z, %2.b, %2.b > + [ ?Upa , 0 , Upa; yes ] ^ > + [ Upa , Upa, Upa; no ] ^ I think this ought to be: > + {@ [ cons: =0, 1 , 2 ; attrs: pred_clobber ] > + [ &Upa , Upa, Upa ; yes ] and\t%0.b, %1/z, %2.b, %2.b > + [ ?Upa , 0Upa, 0Upa; yes ] ^ > + [ Upa , Upa, Upa ; no ] ^ so that operand 2 can be tied to operand 0 in the worst case. Similarly: > } > ) > > @@ -6679,8 +6691,10 @@ (define_insn "@aarch64_pred_<optab><mode>_z" > (match_operand:PRED_ALL 3 "register_operand")) > (match_operand:PRED_ALL 1 "register_operand")))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 , 3 ] > - [ Upa , Upa, Upa, Upa ] <logical>\t%0.b, %1/z, %2.b, %3.b > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > + [ &Upa , Upa, Upa, Upa; yes ] <logical>\t%0.b, %1/z, %2.b, %3.b > + [ ?Upa , 0 , Upa, Upa; yes ] ^ > + [ Upa , Upa, Upa, Upa; no ] ^ > } > ) this would be: {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] [ &Upa , Upa , Upa , Upa ; yes ] <logical>\t%0.b, %1/z, %2.b, %3.b [ ?Upa , 0Upa, 0Upa, 0Upa; yes ] ^ [ Upa , Upa , Upa, Upa ; no ] ^ } Same idea for the rest. I tried this on: ---------------------------------------------------------------------- #include <arm_sve.h> void use (svbool_t, svbool_t, svbool_t); void f1 (svbool_t p0, svbool_t p1, svbool_t p2, int n, svbool_t *ptr) { while (n--) p2 = svand_z (p0, p1, p2); *ptr = p2; } void f2 (svbool_t p0, svbool_t p1, svbool_t p2, svbool_t *ptr) { *ptr = svand_z (p0, p1, p2); } void f3 (svbool_t p0, svbool_t p1, svbool_t p2, svbool_t *ptr) { use (svand_z (p0, p1, p2), p1, p2); } void f4 (svbool_t p0, svbool_t p1, svbool_t p2, svbool_t *ptr) { use (p0, svand_z (p0, p1, p2), p2); } void f5 (svbool_t p0, svbool_t p1, svbool_t p2, svbool_t *ptr) { use (p0, p1, svand_z (p0, p1, p2)); } ---------------------------------------------------------------------- and it seemed to produce the right output: ---------------------------------------------------------------------- f1: cbz w0, .L2 sub w0, w0, #1 .p2align 5,,15 .L3: and p2.b, p0/z, p1.b, p2.b sub w0, w0, #1 cmn w0, #1 bne .L3 .L2: str p2, [x1] ret f2: and p3.b, p0/z, p1.b, p2.b str p3, [x0] ret f3: and p0.b, p0/z, p1.b, p2.b b use f4: and p1.b, p0/z, p1.b, p2.b b use f5: and p2.b, p0/z, p1.b, p2.b b use ---------------------------------------------------------------------- (with that coming directly from RA, rather than being cleaned up later) > [...] > @@ -10046,8 +10104,10 @@ (define_insn_and_rewrite "*aarch64_brkn_cc" > (match_dup 3)] > UNSPEC_BRKN))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 , 3 ] > - [ Upa , Upa, Upa, 0 ] brkns\t%0.b, %1/z, %2.b, %0.b > + {@ [ cons: =0, 1 , 2 , 3; attrs: pred_clobber ] > + [ &Upa , Upa, Upa, 0; yes ] brkns\t%0.b, %1/z, %2.b, %0.b > + [ ?Upa , 0 , Upa, 0; yes ] ^ > + [ Upa , Upa, Upa, 0; no ] ^ > } > "&& (operands[4] != CONST0_RTX (VNx16BImode) > || operands[5] != CONST0_RTX (VNx16BImode))" Probably best to leave this out. All alternatives require operand 3 to match operand 0. So operands 1 and 2 will only match operand 0 if they're the same as operand 3. In that case it'd be better to allow the sharing rather than force the same value to be stored in two registers. That is, if op1 != op3 && op2 != op3 then we get what we want naturally, regardless of tuning. The same thing would apply to the BRKN instances of <brk_reg_con>: > @@ -10020,8 +10076,10 @@ (define_insn "@aarch64_brk<brk_op>" > (match_operand:VNx16BI 3 "register_operand")] > SVE_BRK_BINARY))] > "TARGET_SVE" > - {@ [ cons: =0, 1 , 2 , 3 ] > - [ Upa , Upa, Upa, <brk_reg_con> ] brk<brk_op>\t%0.b, %1/z, %2.b, %<brk_reg_opno>.b > + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] > + [ &Upa , Upa, Upa, <brk_reg_con>; yes ] brk<brk_op>\t%0.b, %1/z, %2.b, %<brk_reg_opno>.b > + [ ?Upa , 0 , Upa, <brk_reg_con>; yes ] ^ > + [ Upa , Upa, Upa, <brk_reg_con>; no ] ^ > } > ) but I think we should keep this factoring/abstraction and just add the extra alternatives regardless. I.e.: {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] [ &Upa , Upa , Upa , <brk_reg_con> ; yes ] brk<brk_op>\t%0.b, %1/z, %2.b, %<brk_reg_opno>.b [ ?Upa , 0Upa, 0Upa, 0<brk_reg_con>; yes ] ^ [ Upa , Upa , Upa , <brk_reg_con> ; no ] ^ (even though this gives "00", which is valid but redundant). OK with those changes, thanks. Richard
diff --git a/gcc/config/aarch64/aarch64-sve.md b/gcc/config/aarch64/aarch64-sve.md index e3085c0c636f1317409bbf3b5fbaf5342a2df1f6..8fdc1bc3cd43acfcd675a18350c297428c85fe46 100644 --- a/gcc/config/aarch64/aarch64-sve.md +++ b/gcc/config/aarch64/aarch64-sve.md @@ -1161,8 +1161,10 @@ (define_insn "aarch64_rdffr_z" (reg:VNx16BI FFRT_REGNUM) (match_operand:VNx16BI 1 "register_operand")))] "TARGET_SVE && TARGET_NON_STREAMING" - {@ [ cons: =0, 1 ] - [ Upa , Upa ] rdffr\t%0.b, %1/z + {@ [ cons: =0, 1 ; attrs: pred_clobber ] + [ &Upa , Upa; yes ] rdffr\t%0.b, %1/z + [ ?Upa , Upa; yes ] ^ + [ Upa , Upa; * ] ^ } ) @@ -1179,8 +1181,10 @@ (define_insn "*aarch64_rdffr_z_ptest" UNSPEC_PTEST)) (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE && TARGET_NON_STREAMING" - {@ [ cons: =0, 1 ] - [ Upa , Upa ] rdffrs\t%0.b, %1/z + {@ [ cons: =0, 1 ; attrs: pred_clobber ] + [ &Upa , Upa; yes ] rdffrs\t%0.b, %1/z + [ ?Upa , Upa; yes ] ^ + [ Upa , Upa; * ] ^ } ) @@ -1195,8 +1199,10 @@ (define_insn "*aarch64_rdffr_ptest" UNSPEC_PTEST)) (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE && TARGET_NON_STREAMING" - {@ [ cons: =0, 1 ] - [ Upa , Upa ] rdffrs\t%0.b, %1/z + {@ [ cons: =0, 1 ; attrs: pred_clobber ] + [ &Upa , Upa; yes ] rdffrs\t%0.b, %1/z + [ ?Upa , Upa; yes ] ^ + [ Upa , Upa; * ] ^ } ) @@ -1216,8 +1222,10 @@ (define_insn "*aarch64_rdffr_z_cc" (reg:VNx16BI FFRT_REGNUM) (match_dup 1)))] "TARGET_SVE && TARGET_NON_STREAMING" - {@ [ cons: =0, 1 ] - [ Upa , Upa ] rdffrs\t%0.b, %1/z + {@ [ cons: =0, 1 ; attrs: pred_clobber ] + [ &Upa , Upa; yes ] rdffrs\t%0.b, %1/z + [ ?Upa , Upa; yes ] ^ + [ Upa , Upa; * ] ^ } ) @@ -1233,8 +1241,10 @@ (define_insn "*aarch64_rdffr_cc" (set (match_operand:VNx16BI 0 "register_operand") (reg:VNx16BI FFRT_REGNUM))] "TARGET_SVE && TARGET_NON_STREAMING" - {@ [ cons: =0, 1 ] - [ Upa , Upa ] rdffrs\t%0.b, %1/z + {@ [ cons: =0, 1 ; attrs: pred_clobber ] + [ &Upa , Upa; yes ] rdffrs\t%0.b, %1/z + [ ?Upa , Upa; yes ] ^ + [ Upa , Upa; * ] ^ } ) @@ -6651,8 +6661,10 @@ (define_insn "and<mode>3" (and:PRED_ALL (match_operand:PRED_ALL 1 "register_operand") (match_operand:PRED_ALL 2 "register_operand")))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 ] - [ Upa , Upa, Upa ] and\t%0.b, %1/z, %2.b, %2.b + {@ [ cons: =0, 1 , 2 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa; yes ] and\t%0.b, %1/z, %2.b, %2.b + [ ?Upa , Upa, Upa; yes ] ^ + [ Upa , Upa, Upa; * ] ^ } ) @@ -6679,8 +6691,10 @@ (define_insn "@aarch64_pred_<optab><mode>_z" (match_operand:PRED_ALL 3 "register_operand")) (match_operand:PRED_ALL 1 "register_operand")))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] <logical>\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa; yes ] <logical>\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , Upa, Upa, Upa; yes ] ^ + [ Upa , Upa, Upa, Upa; * ] ^ } ) @@ -6703,8 +6717,10 @@ (define_insn "*<optab><mode>3_cc" (and:PRED_ALL (LOGICAL:PRED_ALL (match_dup 2) (match_dup 3)) (match_dup 4)))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] <logical>s\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa; yes ] <logical>s\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , Upa, Upa, Upa; yes ] ^ + [ Upa , Upa, Upa, Upa; * ] ^ } ) @@ -6723,8 +6739,10 @@ (define_insn "*<optab><mode>3_ptest" UNSPEC_PTEST)) (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] <logical>s\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa; yes ] <logical>s\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , Upa, Upa, Upa; yes ] ^ + [ Upa , Upa, Upa, Upa; * ] ^ } ) @@ -6745,8 +6763,10 @@ (define_insn "aarch64_pred_<nlogical><mode>_z" (match_operand:PRED_ALL 2 "register_operand")) (match_operand:PRED_ALL 1 "register_operand")))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] <nlogical>\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa; yes ] <nlogical>\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , Upa, Upa, Upa; yes ] ^ + [ Upa , Upa, Upa, Upa; * ] ^ } ) @@ -6770,8 +6790,10 @@ (define_insn "*<nlogical><mode>3_cc" (match_dup 2)) (match_dup 4)))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] <nlogical>s\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa; yes ] <nlogical>s\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , Upa, Upa, Upa; yes ] ^ + [ Upa , Upa, Upa, Upa; * ] ^ } ) @@ -6791,8 +6813,10 @@ (define_insn "*<nlogical><mode>3_ptest" UNSPEC_PTEST)) (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] <nlogical>s\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa; yes ] <nlogical>s\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , Upa, Upa, Upa; yes ] ^ + [ Upa , Upa, Upa, Upa; * ] ^ } ) @@ -6813,8 +6837,10 @@ (define_insn "aarch64_pred_<logical_nn><mode>_z" (not:PRED_ALL (match_operand:PRED_ALL 3 "register_operand"))) (match_operand:PRED_ALL 1 "register_operand")))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] <logical_nn>\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa; yes ] <logical_nn>\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , Upa, Upa, Upa; yes ] ^ + [ Upa , Upa, Upa, Upa; * ] ^ } ) @@ -6839,8 +6865,10 @@ (define_insn "*<logical_nn><mode>3_cc" (not:PRED_ALL (match_dup 3))) (match_dup 4)))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] <logical_nn>s\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa; yes ] <logical_nn>s\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , Upa, Upa, Upa; yes ] ^ + [ Upa , Upa, Upa, Upa; * ] ^ } ) @@ -6861,8 +6889,10 @@ (define_insn "*<logical_nn><mode>3_ptest" UNSPEC_PTEST)) (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] <logical_nn>s\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa; yes ] <logical_nn>s\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , Upa, Upa, Upa; yes ] ^ + [ Upa , Upa, Upa, Upa; * ] ^ } ) @@ -8104,9 +8134,13 @@ (define_insn "@aarch64_pred_cmp<cmp_op><mode>" UNSPEC_PRED_Z)) (clobber (reg:CC_NZC CC_REGNUM))] "TARGET_SVE" - {@ [ cons: =0 , 1 , 3 , 4 ] - [ Upa , Upl , w , <sve_imm_con> ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, #%4 - [ Upa , Upl , w , w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.<Vetype> + {@ [ cons: =0 , 1 , 3 , 4 ; attrs: pred_clobber ] + [ &Upa , Upl , w , <sve_imm_con>; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, #%4 + [ ?Upa , Upl , w , <sve_imm_con>; yes ] ^ + [ Upa , Upl , w , <sve_imm_con>; * ] ^ + [ &Upa , Upl , w , w ; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.<Vetype> + [ ?Upa , Upl , w , w ; yes ] ^ + [ Upa , Upl , w , w ; * ] ^ } ) @@ -8136,9 +8170,13 @@ (define_insn_and_rewrite "*cmp<cmp_op><mode>_cc" UNSPEC_PRED_Z))] "TARGET_SVE && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" - {@ [ cons: =0 , 1 , 2 , 3 ] - [ Upa , Upl , w , <sve_imm_con> ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, #%3 - [ Upa , Upl , w , w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.<Vetype> + {@ [ cons: =0 , 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upl , w , <sve_imm_con>; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, #%3 + [ ?Upa , Upl , w , <sve_imm_con>; yes ] ^ + [ Upa , Upl , w , <sve_imm_con>; * ] ^ + [ &Upa , Upl , w , w ; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.<Vetype> + [ ?Upa , Upl , w , w ; yes ] ^ + [ Upa , Upl , w , w ; * ] ^ } "&& !rtx_equal_p (operands[4], operands[6])" { @@ -8166,9 +8204,13 @@ (define_insn_and_rewrite "*cmp<cmp_op><mode>_ptest" (clobber (match_scratch:<VPRED> 0))] "TARGET_SVE && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upl, w , <sve_imm_con> ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, #%3 - [ Upa , Upl, w , w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.<Vetype> + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upl, w , <sve_imm_con>; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, #%3 + [ ?Upa , Upl, w , <sve_imm_con>; yes ] ^ + [ Upa , Upl, w , <sve_imm_con>; * ] ^ + [ &Upa , Upl, w , w ; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.<Vetype> + [ ?Upa , Upl, w , w ; yes ] ^ + [ Upa , Upl, w , w ; * ] ^ } "&& !rtx_equal_p (operands[4], operands[6])" { @@ -8221,8 +8263,10 @@ (define_insn "@aarch64_pred_cmp<cmp_op><mode>_wide" UNSPEC_PRED_Z)) (clobber (reg:CC_NZC CC_REGNUM))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2, 3, 4 ] - [ Upa , Upl, , w, w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.d + {@ [ cons: =0, 1 , 2, 3, 4; attrs: pred_clobber ] + [ &Upa , Upl, , w, w; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.d + [ ?Upa , Upl, , w, w; yes ] ^ + [ Upa , Upl, , w, w; * ] ^ } ) @@ -8254,8 +8298,10 @@ (define_insn "*aarch64_pred_cmp<cmp_op><mode>_wide_cc" UNSPEC_PRED_Z))] "TARGET_SVE && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" - {@ [ cons: =0, 1 , 2, 3, 6 ] - [ Upa , Upl, w, w, Upl ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.d + {@ [ cons: =0, 1 , 2, 3, 6 ; attrs: pred_clobber ] + [ &Upa , Upl, w, w, Upl; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.d + [ ?Upa , Upl, w, w, Upl; yes ] ^ + [ Upa , Upl, w, w, Upl; * ] ^ } ) @@ -8279,8 +8325,10 @@ (define_insn "*aarch64_pred_cmp<cmp_op><mode>_wide_ptest" (clobber (match_scratch:<VPRED> 0))] "TARGET_SVE && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" - {@ [ cons: =0, 1 , 2, 3, 6 ] - [ Upa , Upl, w, w, Upl ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.d + {@ [ cons: =0, 1 , 2, 3, 6 ; attrs: pred_clobber ] + [ &Upa , Upl, w, w, Upl; yes ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.d + [ ?Upa , Upl, w, w, Upl; yes ] ^ + [ Upa , Upl, w, w, Upl; * ] ^ } ) @@ -9948,9 +9996,13 @@ (define_insn "@aarch64_brk<brk_op>" (match_operand:VNx16BI 3 "aarch64_simd_reg_or_zero")] SVE_BRK_UNARY))] "TARGET_SVE" - {@ [ cons: =0 , 1 , 2 , 3 ] - [ Upa , Upa , Upa , Dz ] brk<brk_op>\t%0.b, %1/z, %2.b - [ Upa , Upa , Upa , 0 ] brk<brk_op>\t%0.b, %1/m, %2.b + {@ [ cons: =0 , 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa , Upa , Dz; yes ] brk<brk_op>\t%0.b, %1/z, %2.b + [ ?Upa , Upa , Upa , Dz; yes ] ^ + [ Upa , Upa , Upa , Dz; * ] ^ + [ &Upa , Upa , Upa , 0 ; yes ] brk<brk_op>\t%0.b, %1/m, %2.b + [ ?Upa , Upa , Upa , 0 ; yes ] ^ + [ Upa , Upa , Upa , 0 ; * ] ^ } ) @@ -9974,8 +10026,10 @@ (define_insn "*aarch64_brk<brk_op>_cc" (match_dup 3)] SVE_BRK_UNARY))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 ] - [ Upa , Upa, Upa ] brk<brk_op>s\t%0.b, %1/z, %2.b + {@ [ cons: =0, 1 , 2 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa; yes ] brk<brk_op>s\t%0.b, %1/z, %2.b + [ ?Upa , Upa, Upa; yes ] ^ + [ Upa , Upa, Upa; * ] ^ } ) @@ -9994,8 +10048,10 @@ (define_insn "*aarch64_brk<brk_op>_ptest" UNSPEC_PTEST)) (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 ] - [ Upa , Upa, Upa ] brk<brk_op>s\t%0.b, %1/z, %2.b + {@ [ cons: =0, 1 , 2 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa; yes ] brk<brk_op>s\t%0.b, %1/z, %2.b + [ ?Upa , Upa, Upa; yes ] ^ + [ Upa , Upa, Upa; * ] ^ } ) @@ -10020,8 +10076,10 @@ (define_insn "@aarch64_brk<brk_op>" (match_operand:VNx16BI 3 "register_operand")] SVE_BRK_BINARY))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, <brk_reg_con> ] brk<brk_op>\t%0.b, %1/z, %2.b, %<brk_reg_opno>.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, <brk_reg_con>; yes ] brk<brk_op>\t%0.b, %1/z, %2.b, %<brk_reg_opno>.b + [ ?Upa , Upa, Upa, <brk_reg_con>; yes ] ^ + [ Upa , Upa, Upa, <brk_reg_con>; * ] ^ } ) @@ -10046,8 +10104,10 @@ (define_insn_and_rewrite "*aarch64_brkn_cc" (match_dup 3)] UNSPEC_BRKN))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, 0 ] brkns\t%0.b, %1/z, %2.b, %0.b + {@ [ cons: =0, 1 , 2 , 3; attrs: pred_clobber ] + [ &Upa , Upa, Upa, 0; yes ] brkns\t%0.b, %1/z, %2.b, %0.b + [ ?Upa , Upa, Upa, 0; yes ] ^ + [ Upa , Upa, Upa, 0; * ] ^ } "&& (operands[4] != CONST0_RTX (VNx16BImode) || operands[5] != CONST0_RTX (VNx16BImode))" @@ -10072,8 +10132,10 @@ (define_insn_and_rewrite "*aarch64_brkn_ptest" UNSPEC_PTEST)) (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, 0 ] brkns\t%0.b, %1/z, %2.b, %0.b + {@ [ cons: =0, 1 , 2 , 3; attrs: pred_clobber ] + [ &Upa , Upa, Upa, 0; yes ] brkns\t%0.b, %1/z, %2.b, %0.b + [ ?Upa , Upa, Upa, 0; yes ] ^ + [ Upa , Upa, Upa, 0; * ] ^ } "&& (operands[4] != CONST0_RTX (VNx16BImode) || operands[5] != CONST0_RTX (VNx16BImode))" @@ -10103,8 +10165,10 @@ (define_insn "*aarch64_brk<brk_op>_cc" (match_dup 3)] SVE_BRKP))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 , 4; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa, ; yes ] brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , Upa, Upa, Upa, ; yes ] ^ + [ Upa , Upa, Upa, Upa, ; * ] ^ } ) @@ -10123,8 +10187,10 @@ (define_insn "*aarch64_brk<brk_op>_ptest" UNSPEC_PTEST)) (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE" - {@ [ cons: =0, 1 , 2 , 3 ] - [ Upa , Upa, Upa, Upa ] brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b + {@ [ cons: =0, 1 , 2 , 3 ; attrs: pred_clobber ] + [ &Upa , Upa, Upa, Upa; yes ] brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b + [ ?Upa , Upa, Upa, Upa; yes ] ^ + [ Upa , Upa, Upa, Upa; * ] ^ } ) diff --git a/gcc/config/aarch64/aarch64-sve2.md b/gcc/config/aarch64/aarch64-sve2.md index aa12baf48355358ca4fefe88157df3aac6eb09bd..1a49494a69d8335e5f7d3ef4bd3a90d0805bba84 100644 --- a/gcc/config/aarch64/aarch64-sve2.md +++ b/gcc/config/aarch64/aarch64-sve2.md @@ -3349,8 +3349,10 @@ (define_insn "@aarch64_pred_<sve_int_op><mode>" UNSPEC_PRED_Z)) (clobber (reg:CC_NZC CC_REGNUM))] "TARGET_SVE2 && TARGET_NON_STREAMING" - {@ [ cons: =0, 1 , 2, 3, 4 ] - [ Upa , Upl, , w, w ] <sve_int_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.<Vetype> + {@ [ cons: =0, 1 , 2, 3, 4; attrs: pred_clobber ] + [ &Upa , Upl, , w, w; yes ] <sve_int_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.<Vetype> + [ ?Upa , Upl, , w, w; yes ] ^ + [ Upa , Upl, , w, w; * ] ^ } )