| Message ID | 20240827070840.1214913-1-pan2.li@intel.com |
|---|---|
| State | New |
| Headers | show |
| Series | [v2] Vect: Reconcile the const_int operand type of unsigned .SAT_ADD | expand |
On Tue, Aug 27, 2024 at 9:09 AM <pan2.li@intel.com> wrote: > > From: Pan Li <pan2.li@intel.com> > > The .SAT_ADD has 2 operand, when one of the operand may be INTEGER_CST. > For example _1 = .SAT_ADD (_2, 9) comes from below sample code. > > Form 3: > #define DEF_VEC_SAT_U_ADD_IMM_FMT_3(T, IMM) \ > T __attribute__((noinline)) \ > vec_sat_u_add_imm##IMM##_##T##_fmt_3 (T *out, T *in, unsigned limit) \ > { \ > unsigned i; \ > T ret; \ > for (i = 0; i < limit; i++) \ > { \ > out[i] = __builtin_add_overflow (in[i], IMM, &ret) ? -1 : ret; \ > } \ > } > > DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint64_t, 9) > > It will fail to vectorize as the vectorizable_call will check the > operands is type_compatiable but the imm will be (const_int 9) with > the SImode, which is different from _2 (DImode). Aka: > > uint64_t _1; > uint64_t _2; > _1 = .SAT_ADD (_2, 9); > > This patch would like to reconcile the imm operand to the operand type > mode of _2 if and only if there is no precision/data loss. Aka convert > the imm 9 to the DImode for above example. > > The below test suites are passed for this patch: > 1. The rv64gcv fully regression tests. > 2. The rv64gcv build with glibc. > 3. The x86 bootstrap tests. > 4. The x86 fully regression tests. > > gcc/ChangeLog: > > * tree-vect-patterns.cc (vect_recog_reconcile_cst_to_unsigned): > Add new func impl to reconcile the cst int type to given TREE type. > (vect_recog_sat_add_pattern): Reconcile the ops of .SAT_ADD > before building the gimple call. > > gcc/testsuite/ChangeLog: > > * gcc.target/riscv/rvv/autovec/vec_sat_arith.h: Add test helper macros. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-1.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-10.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-11.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-12.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-13.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-14.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-15.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-2.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-3.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-4.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-5.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-6.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-7.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-8.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-9.c: New test. > > Signed-off-by: Pan Li <pan2.li@intel.com> > --- > .../binop/vec_sat_u_add_imm_reconcile-1.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-10.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-11.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-12.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-13.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-14.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-15.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-2.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-3.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-4.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-5.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-6.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-7.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-8.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-9.c | 9 +++++ > .../riscv/rvv/autovec/vec_sat_arith.h | 20 ++++++++++ > gcc/tree-vect-patterns.cc | 38 +++++++++++++++++++ > 17 files changed, 193 insertions(+) > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-1.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-10.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-11.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-12.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-13.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-14.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-15.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-2.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-3.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-4.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-5.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-6.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-7.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-8.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-9.c > > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-1.c > new file mode 100644 > index 00000000000..74feae0db03 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-1.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint8_t, 9u) > + > +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-10.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-10.c > new file mode 100644 > index 00000000000..41140955032 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-10.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint32_t, 4294967205u) > + > +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-11.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-11.c > new file mode 100644 > index 00000000000..d2b1e315fbf > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-11.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint32_t, 4294967495ll) > + > +/* { dg-final { scan-rtl-dump-not ".SAT_ADD " "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-12.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-12.c > new file mode 100644 > index 00000000000..b260d706f59 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-12.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint32_t, 9294967495ull) > + > +/* { dg-final { scan-rtl-dump-not ".SAT_ADD " "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-13.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-13.c > new file mode 100644 > index 00000000000..ff1136a2d31 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-13.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint64_t, 119u) > + > +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-14.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-14.c > new file mode 100644 > index 00000000000..2157ed71fab > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-14.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint64_t, 18446744073709551615ull) > + > +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-15.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-15.c > new file mode 100644 > index 00000000000..2fcf951a5eb > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-15.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint64_t, 9223372036854775807ull) > + > +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-2.c > new file mode 100644 > index 00000000000..46dc6dc9606 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-2.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint8_t, 219) > + > +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-3.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-3.c > new file mode 100644 > index 00000000000..ba48f484142 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-3.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint8_t, 299) > + > +/* { dg-final { scan-rtl-dump-not ".SAT_ADD " "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-4.c > new file mode 100644 > index 00000000000..1a41a6da155 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-4.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint8_t, 301u) > + > +/* { dg-final { scan-rtl-dump-not ".SAT_ADD " "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-5.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-5.c > new file mode 100644 > index 00000000000..4b4f70da15e > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-5.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint16_t, 9u) > + > +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-6.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-6.c > new file mode 100644 > index 00000000000..ce6c46b7242 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-6.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint16_t, 65530) > + > +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-7.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-7.c > new file mode 100644 > index 00000000000..bd72b2471d4 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-7.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint16_t, 65559) > + > +/* { dg-final { scan-rtl-dump-not ".SAT_ADD " "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-8.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-8.c > new file mode 100644 > index 00000000000..678aa972077 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-8.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint16_t, 75559u) > + > +/* { dg-final { scan-rtl-dump-not ".SAT_ADD " "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-9.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-9.c > new file mode 100644 > index 00000000000..ef552b94830 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-9.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint32_t, 911u) > + > +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec_sat_arith.h b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec_sat_arith.h > index f28cb7e30a1..deb6bb82eba 100644 > --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec_sat_arith.h > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec_sat_arith.h > @@ -169,6 +169,20 @@ vec_sat_u_add_imm##IMM##_##T##_fmt_2 (T *out, T *in, unsigned limit) \ > #define DEF_VEC_SAT_U_ADD_IMM_FMT_2_WRAP(T, IMM) \ > DEF_VEC_SAT_U_ADD_IMM_FMT_2(T, IMM) > > +#define DEF_VEC_SAT_U_ADD_IMM_FMT_3(T, IMM) \ > +T __attribute__((noinline)) \ > +vec_sat_u_add_imm##IMM##_##T##_fmt_3 (T *out, T *in, unsigned limit) \ > +{ \ > + unsigned i; \ > + T ret; \ > + for (i = 0; i < limit; i++) \ > + { \ > + out[i] = __builtin_add_overflow (in[i], IMM, &ret) ? -1 : ret; \ > + } \ > +} > +#define DEF_VEC_SAT_U_ADD_IMM_FMT_3_WRAP(T, IMM) \ > + DEF_VEC_SAT_U_ADD_IMM_FMT_3(T, IMM) > + > #define RUN_VEC_SAT_U_ADD_IMM_FMT_1(T, out, op_1, expect, IMM, N) \ > vec_sat_u_add_imm##IMM##_##T##_fmt_1(out, op_1, N); \ > VALIDATE_RESULT (out, expect, N) > @@ -181,6 +195,12 @@ vec_sat_u_add_imm##IMM##_##T##_fmt_2 (T *out, T *in, unsigned limit) \ > #define RUN_VEC_SAT_U_ADD_IMM_FMT_2_WRAP(T, out, op_1, expect, IMM, N) \ > RUN_VEC_SAT_U_ADD_IMM_FMT_2(T, out, op_1, expect, IMM, N) > > +#define RUN_VEC_SAT_U_ADD_IMM_FMT_3(T, out, op_1, expect, IMM, N) \ > + vec_sat_u_add_imm##IMM##_##T##_fmt_3(out, op_1, N); \ > + VALIDATE_RESULT (out, expect, N) > +#define RUN_VEC_SAT_U_ADD_IMM_FMT_3_WRAP(T, out, op_1, expect, IMM, N) \ > + RUN_VEC_SAT_U_ADD_IMM_FMT_3(T, out, op_1, expect, IMM, N) > + > /******************************************************************************/ > /* Saturation Sub (Unsigned and Signed) */ > /******************************************************************************/ > diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc > index 18b322c63b8..80a98c50847 100644 > --- a/gcc/tree-vect-patterns.cc > +++ b/gcc/tree-vect-patterns.cc > @@ -4532,6 +4532,41 @@ vect_recog_build_binary_gimple_stmt (vec_info *vinfo, stmt_vec_info stmt_info, > return NULL; > } > > +/* Reconcile the INTEGER_CST tree to the given TREE type. The INTEGER_CST tree > + * will be returned directly if > + * 1. Not INTEGER_CST code or not unsigned TREE type. > + * 2. The precision of INTEGER_CST is the same as the precision of TREE type. > + * 3. The INTEGER_CST op exceeds the max value of TREE type. > + * > + * If precision > cst_precision, the INTEGER_CST op will widen to precision > + * of TREE type. > + * > + * If precision < cst_precision, the INTEGER_CST op will narrow to precision if > + * the precision of TREE type is able to hold the value of INTEGER_CST op. > + */ > + > +static tree > +vect_recog_reconcile_cst_to_unsigned (tree op, tree type) > +{ > + if (TREE_CODE (op) != INTEGER_CST || !TYPE_UNSIGNED (type)) > + return op; > + > + unsigned precision = TYPE_PRECISION (type); > + unsigned cst_precision = TYPE_PRECISION (TREE_TYPE (op)); > + > + if (cst_precision == precision) > + return op; > + > + wide_int cst = wi::to_wide (op); > + > + if (cst_precision < precision) > + return wide_int_to_tree (type, cst); > + > + wide_int max = wi::mask (precision, false, cst_precision); > + > + return wi::leu_p (cst, max) ? wide_int_to_tree (type, cst) : op; > +} > + > /* > * Try to detect saturation add pattern (SAT_ADD), aka below gimple: > * _7 = _4 + _6; > @@ -4558,6 +4593,9 @@ vect_recog_sat_add_pattern (vec_info *vinfo, stmt_vec_info stmt_vinfo, > > if (gimple_unsigned_integer_sat_add (lhs, ops, NULL)) > { > + ops[0] = vect_recog_reconcile_cst_to_unsigned (ops[0], TREE_TYPE (ops[1])); > + ops[1] = vect_recog_reconcile_cst_to_unsigned (ops[1], TREE_TYPE (ops[0])); > + Err, can you please simply do if (TREE_CODE (ops[1]) == INTEGER_CST) ops[1] = fold_convert (TREE_TYPE (ops[0]), ops[1]); ? you are always matching the constant to @1 IIRC. > gimple *stmt = vect_recog_build_binary_gimple_stmt (vinfo, stmt_vinfo, > IFN_SAT_ADD, type_out, > lhs, ops[0], ops[1]); > -- > 2.43.0 >
Thanks Richard for comments. > Err, can you please simply do > if (TREE_CODE (ops[1]) == INTEGER_CST) > ops[1] = fold_convert (TREE_TYPE (ops[0]), ops[1]) > ? you are always matching the constant to @1 IIRC. That would be much more simple, will have a try in v3. Pan -----Original Message----- From: Richard Biener <richard.guenther@gmail.com> Sent: Tuesday, August 27, 2024 5:09 PM To: Li, Pan2 <pan2.li@intel.com> Cc: gcc-patches@gcc.gnu.org; Tamar.Christina@arm.com; juzhe.zhong@rivai.ai; kito.cheng@gmail.com; jeffreyalaw@gmail.com; rdapp.gcc@gmail.com Subject: Re: [PATCH v2] Vect: Reconcile the const_int operand type of unsigned .SAT_ADD On Tue, Aug 27, 2024 at 9:09 AM <pan2.li@intel.com> wrote: > > From: Pan Li <pan2.li@intel.com> > > The .SAT_ADD has 2 operand, when one of the operand may be INTEGER_CST. > For example _1 = .SAT_ADD (_2, 9) comes from below sample code. > > Form 3: > #define DEF_VEC_SAT_U_ADD_IMM_FMT_3(T, IMM) \ > T __attribute__((noinline)) \ > vec_sat_u_add_imm##IMM##_##T##_fmt_3 (T *out, T *in, unsigned limit) \ > { \ > unsigned i; \ > T ret; \ > for (i = 0; i < limit; i++) \ > { \ > out[i] = __builtin_add_overflow (in[i], IMM, &ret) ? -1 : ret; \ > } \ > } > > DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint64_t, 9) > > It will fail to vectorize as the vectorizable_call will check the > operands is type_compatiable but the imm will be (const_int 9) with > the SImode, which is different from _2 (DImode). Aka: > > uint64_t _1; > uint64_t _2; > _1 = .SAT_ADD (_2, 9); > > This patch would like to reconcile the imm operand to the operand type > mode of _2 if and only if there is no precision/data loss. Aka convert > the imm 9 to the DImode for above example. > > The below test suites are passed for this patch: > 1. The rv64gcv fully regression tests. > 2. The rv64gcv build with glibc. > 3. The x86 bootstrap tests. > 4. The x86 fully regression tests. > > gcc/ChangeLog: > > * tree-vect-patterns.cc (vect_recog_reconcile_cst_to_unsigned): > Add new func impl to reconcile the cst int type to given TREE type. > (vect_recog_sat_add_pattern): Reconcile the ops of .SAT_ADD > before building the gimple call. > > gcc/testsuite/ChangeLog: > > * gcc.target/riscv/rvv/autovec/vec_sat_arith.h: Add test helper macros. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-1.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-10.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-11.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-12.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-13.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-14.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-15.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-2.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-3.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-4.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-5.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-6.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-7.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-8.c: New test. > * gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-9.c: New test. > > Signed-off-by: Pan Li <pan2.li@intel.com> > --- > .../binop/vec_sat_u_add_imm_reconcile-1.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-10.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-11.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-12.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-13.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-14.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-15.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-2.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-3.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-4.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-5.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-6.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-7.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-8.c | 9 +++++ > .../binop/vec_sat_u_add_imm_reconcile-9.c | 9 +++++ > .../riscv/rvv/autovec/vec_sat_arith.h | 20 ++++++++++ > gcc/tree-vect-patterns.cc | 38 +++++++++++++++++++ > 17 files changed, 193 insertions(+) > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-1.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-10.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-11.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-12.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-13.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-14.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-15.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-2.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-3.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-4.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-5.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-6.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-7.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-8.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-9.c > > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-1.c > new file mode 100644 > index 00000000000..74feae0db03 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-1.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint8_t, 9u) > + > +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-10.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-10.c > new file mode 100644 > index 00000000000..41140955032 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-10.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint32_t, 4294967205u) > + > +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-11.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-11.c > new file mode 100644 > index 00000000000..d2b1e315fbf > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-11.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint32_t, 4294967495ll) > + > +/* { dg-final { scan-rtl-dump-not ".SAT_ADD " "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-12.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-12.c > new file mode 100644 > index 00000000000..b260d706f59 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-12.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint32_t, 9294967495ull) > + > +/* { dg-final { scan-rtl-dump-not ".SAT_ADD " "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-13.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-13.c > new file mode 100644 > index 00000000000..ff1136a2d31 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-13.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint64_t, 119u) > + > +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-14.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-14.c > new file mode 100644 > index 00000000000..2157ed71fab > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-14.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint64_t, 18446744073709551615ull) > + > +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-15.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-15.c > new file mode 100644 > index 00000000000..2fcf951a5eb > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-15.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint64_t, 9223372036854775807ull) > + > +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-2.c > new file mode 100644 > index 00000000000..46dc6dc9606 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-2.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint8_t, 219) > + > +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-3.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-3.c > new file mode 100644 > index 00000000000..ba48f484142 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-3.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint8_t, 299) > + > +/* { dg-final { scan-rtl-dump-not ".SAT_ADD " "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-4.c > new file mode 100644 > index 00000000000..1a41a6da155 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-4.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint8_t, 301u) > + > +/* { dg-final { scan-rtl-dump-not ".SAT_ADD " "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-5.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-5.c > new file mode 100644 > index 00000000000..4b4f70da15e > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-5.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint16_t, 9u) > + > +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-6.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-6.c > new file mode 100644 > index 00000000000..ce6c46b7242 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-6.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint16_t, 65530) > + > +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-7.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-7.c > new file mode 100644 > index 00000000000..bd72b2471d4 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-7.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint16_t, 65559) > + > +/* { dg-final { scan-rtl-dump-not ".SAT_ADD " "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-8.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-8.c > new file mode 100644 > index 00000000000..678aa972077 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-8.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint16_t, 75559u) > + > +/* { dg-final { scan-rtl-dump-not ".SAT_ADD " "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-9.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-9.c > new file mode 100644 > index 00000000000..ef552b94830 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-9.c > @@ -0,0 +1,9 @@ > +/* { dg-do compile } */ > +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ > +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ > + > +#include "../vec_sat_arith.h" > + > +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint32_t, 911u) > + > +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec_sat_arith.h b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec_sat_arith.h > index f28cb7e30a1..deb6bb82eba 100644 > --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec_sat_arith.h > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec_sat_arith.h > @@ -169,6 +169,20 @@ vec_sat_u_add_imm##IMM##_##T##_fmt_2 (T *out, T *in, unsigned limit) \ > #define DEF_VEC_SAT_U_ADD_IMM_FMT_2_WRAP(T, IMM) \ > DEF_VEC_SAT_U_ADD_IMM_FMT_2(T, IMM) > > +#define DEF_VEC_SAT_U_ADD_IMM_FMT_3(T, IMM) \ > +T __attribute__((noinline)) \ > +vec_sat_u_add_imm##IMM##_##T##_fmt_3 (T *out, T *in, unsigned limit) \ > +{ \ > + unsigned i; \ > + T ret; \ > + for (i = 0; i < limit; i++) \ > + { \ > + out[i] = __builtin_add_overflow (in[i], IMM, &ret) ? -1 : ret; \ > + } \ > +} > +#define DEF_VEC_SAT_U_ADD_IMM_FMT_3_WRAP(T, IMM) \ > + DEF_VEC_SAT_U_ADD_IMM_FMT_3(T, IMM) > + > #define RUN_VEC_SAT_U_ADD_IMM_FMT_1(T, out, op_1, expect, IMM, N) \ > vec_sat_u_add_imm##IMM##_##T##_fmt_1(out, op_1, N); \ > VALIDATE_RESULT (out, expect, N) > @@ -181,6 +195,12 @@ vec_sat_u_add_imm##IMM##_##T##_fmt_2 (T *out, T *in, unsigned limit) \ > #define RUN_VEC_SAT_U_ADD_IMM_FMT_2_WRAP(T, out, op_1, expect, IMM, N) \ > RUN_VEC_SAT_U_ADD_IMM_FMT_2(T, out, op_1, expect, IMM, N) > > +#define RUN_VEC_SAT_U_ADD_IMM_FMT_3(T, out, op_1, expect, IMM, N) \ > + vec_sat_u_add_imm##IMM##_##T##_fmt_3(out, op_1, N); \ > + VALIDATE_RESULT (out, expect, N) > +#define RUN_VEC_SAT_U_ADD_IMM_FMT_3_WRAP(T, out, op_1, expect, IMM, N) \ > + RUN_VEC_SAT_U_ADD_IMM_FMT_3(T, out, op_1, expect, IMM, N) > + > /******************************************************************************/ > /* Saturation Sub (Unsigned and Signed) */ > /******************************************************************************/ > diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc > index 18b322c63b8..80a98c50847 100644 > --- a/gcc/tree-vect-patterns.cc > +++ b/gcc/tree-vect-patterns.cc > @@ -4532,6 +4532,41 @@ vect_recog_build_binary_gimple_stmt (vec_info *vinfo, stmt_vec_info stmt_info, > return NULL; > } > > +/* Reconcile the INTEGER_CST tree to the given TREE type. The INTEGER_CST tree > + * will be returned directly if > + * 1. Not INTEGER_CST code or not unsigned TREE type. > + * 2. The precision of INTEGER_CST is the same as the precision of TREE type. > + * 3. The INTEGER_CST op exceeds the max value of TREE type. > + * > + * If precision > cst_precision, the INTEGER_CST op will widen to precision > + * of TREE type. > + * > + * If precision < cst_precision, the INTEGER_CST op will narrow to precision if > + * the precision of TREE type is able to hold the value of INTEGER_CST op. > + */ > + > +static tree > +vect_recog_reconcile_cst_to_unsigned (tree op, tree type) > +{ > + if (TREE_CODE (op) != INTEGER_CST || !TYPE_UNSIGNED (type)) > + return op; > + > + unsigned precision = TYPE_PRECISION (type); > + unsigned cst_precision = TYPE_PRECISION (TREE_TYPE (op)); > + > + if (cst_precision == precision) > + return op; > + > + wide_int cst = wi::to_wide (op); > + > + if (cst_precision < precision) > + return wide_int_to_tree (type, cst); > + > + wide_int max = wi::mask (precision, false, cst_precision); > + > + return wi::leu_p (cst, max) ? wide_int_to_tree (type, cst) : op; > +} > + > /* > * Try to detect saturation add pattern (SAT_ADD), aka below gimple: > * _7 = _4 + _6; > @@ -4558,6 +4593,9 @@ vect_recog_sat_add_pattern (vec_info *vinfo, stmt_vec_info stmt_vinfo, > > if (gimple_unsigned_integer_sat_add (lhs, ops, NULL)) > { > + ops[0] = vect_recog_reconcile_cst_to_unsigned (ops[0], TREE_TYPE (ops[1])); > + ops[1] = vect_recog_reconcile_cst_to_unsigned (ops[1], TREE_TYPE (ops[0])); > + Err, can you please simply do if (TREE_CODE (ops[1]) == INTEGER_CST) ops[1] = fold_convert (TREE_TYPE (ops[0]), ops[1]); ? you are always matching the constant to @1 IIRC. > gimple *stmt = vect_recog_build_binary_gimple_stmt (vinfo, stmt_vinfo, > IFN_SAT_ADD, type_out, > lhs, ops[0], ops[1]); > -- > 2.43.0 >
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-1.c new file mode 100644 index 00000000000..74feae0db03 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-1.c @@ -0,0 +1,9 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ + +#include "../vec_sat_arith.h" + +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint8_t, 9u) + +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-10.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-10.c new file mode 100644 index 00000000000..41140955032 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-10.c @@ -0,0 +1,9 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ + +#include "../vec_sat_arith.h" + +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint32_t, 4294967205u) + +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-11.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-11.c new file mode 100644 index 00000000000..d2b1e315fbf --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-11.c @@ -0,0 +1,9 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ + +#include "../vec_sat_arith.h" + +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint32_t, 4294967495ll) + +/* { dg-final { scan-rtl-dump-not ".SAT_ADD " "expand" } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-12.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-12.c new file mode 100644 index 00000000000..b260d706f59 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-12.c @@ -0,0 +1,9 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ + +#include "../vec_sat_arith.h" + +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint32_t, 9294967495ull) + +/* { dg-final { scan-rtl-dump-not ".SAT_ADD " "expand" } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-13.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-13.c new file mode 100644 index 00000000000..ff1136a2d31 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-13.c @@ -0,0 +1,9 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ + +#include "../vec_sat_arith.h" + +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint64_t, 119u) + +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-14.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-14.c new file mode 100644 index 00000000000..2157ed71fab --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-14.c @@ -0,0 +1,9 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ + +#include "../vec_sat_arith.h" + +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint64_t, 18446744073709551615ull) + +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-15.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-15.c new file mode 100644 index 00000000000..2fcf951a5eb --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-15.c @@ -0,0 +1,9 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ + +#include "../vec_sat_arith.h" + +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint64_t, 9223372036854775807ull) + +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-2.c new file mode 100644 index 00000000000..46dc6dc9606 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-2.c @@ -0,0 +1,9 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ + +#include "../vec_sat_arith.h" + +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint8_t, 219) + +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-3.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-3.c new file mode 100644 index 00000000000..ba48f484142 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-3.c @@ -0,0 +1,9 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ + +#include "../vec_sat_arith.h" + +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint8_t, 299) + +/* { dg-final { scan-rtl-dump-not ".SAT_ADD " "expand" } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-4.c new file mode 100644 index 00000000000..1a41a6da155 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-4.c @@ -0,0 +1,9 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ + +#include "../vec_sat_arith.h" + +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint8_t, 301u) + +/* { dg-final { scan-rtl-dump-not ".SAT_ADD " "expand" } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-5.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-5.c new file mode 100644 index 00000000000..4b4f70da15e --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-5.c @@ -0,0 +1,9 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ + +#include "../vec_sat_arith.h" + +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint16_t, 9u) + +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-6.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-6.c new file mode 100644 index 00000000000..ce6c46b7242 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-6.c @@ -0,0 +1,9 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ + +#include "../vec_sat_arith.h" + +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint16_t, 65530) + +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-7.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-7.c new file mode 100644 index 00000000000..bd72b2471d4 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-7.c @@ -0,0 +1,9 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ + +#include "../vec_sat_arith.h" + +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint16_t, 65559) + +/* { dg-final { scan-rtl-dump-not ".SAT_ADD " "expand" } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-8.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-8.c new file mode 100644 index 00000000000..678aa972077 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-8.c @@ -0,0 +1,9 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ + +#include "../vec_sat_arith.h" + +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint16_t, 75559u) + +/* { dg-final { scan-rtl-dump-not ".SAT_ADD " "expand" } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-9.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-9.c new file mode 100644 index 00000000000..ef552b94830 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vec_sat_u_add_imm_reconcile-9.c @@ -0,0 +1,9 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fdump-rtl-expand-details" } */ +/* { dg-skip-if "" { *-*-* } { "-flto" } } */ + +#include "../vec_sat_arith.h" + +DEF_VEC_SAT_U_ADD_IMM_FMT_3(uint32_t, 911u) + +/* { dg-final { scan-rtl-dump-times ".SAT_ADD " 2 "expand" } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec_sat_arith.h b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec_sat_arith.h index f28cb7e30a1..deb6bb82eba 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec_sat_arith.h +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vec_sat_arith.h @@ -169,6 +169,20 @@ vec_sat_u_add_imm##IMM##_##T##_fmt_2 (T *out, T *in, unsigned limit) \ #define DEF_VEC_SAT_U_ADD_IMM_FMT_2_WRAP(T, IMM) \ DEF_VEC_SAT_U_ADD_IMM_FMT_2(T, IMM) +#define DEF_VEC_SAT_U_ADD_IMM_FMT_3(T, IMM) \ +T __attribute__((noinline)) \ +vec_sat_u_add_imm##IMM##_##T##_fmt_3 (T *out, T *in, unsigned limit) \ +{ \ + unsigned i; \ + T ret; \ + for (i = 0; i < limit; i++) \ + { \ + out[i] = __builtin_add_overflow (in[i], IMM, &ret) ? -1 : ret; \ + } \ +} +#define DEF_VEC_SAT_U_ADD_IMM_FMT_3_WRAP(T, IMM) \ + DEF_VEC_SAT_U_ADD_IMM_FMT_3(T, IMM) + #define RUN_VEC_SAT_U_ADD_IMM_FMT_1(T, out, op_1, expect, IMM, N) \ vec_sat_u_add_imm##IMM##_##T##_fmt_1(out, op_1, N); \ VALIDATE_RESULT (out, expect, N) @@ -181,6 +195,12 @@ vec_sat_u_add_imm##IMM##_##T##_fmt_2 (T *out, T *in, unsigned limit) \ #define RUN_VEC_SAT_U_ADD_IMM_FMT_2_WRAP(T, out, op_1, expect, IMM, N) \ RUN_VEC_SAT_U_ADD_IMM_FMT_2(T, out, op_1, expect, IMM, N) +#define RUN_VEC_SAT_U_ADD_IMM_FMT_3(T, out, op_1, expect, IMM, N) \ + vec_sat_u_add_imm##IMM##_##T##_fmt_3(out, op_1, N); \ + VALIDATE_RESULT (out, expect, N) +#define RUN_VEC_SAT_U_ADD_IMM_FMT_3_WRAP(T, out, op_1, expect, IMM, N) \ + RUN_VEC_SAT_U_ADD_IMM_FMT_3(T, out, op_1, expect, IMM, N) + /******************************************************************************/ /* Saturation Sub (Unsigned and Signed) */ /******************************************************************************/ diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc index 18b322c63b8..80a98c50847 100644 --- a/gcc/tree-vect-patterns.cc +++ b/gcc/tree-vect-patterns.cc @@ -4532,6 +4532,41 @@ vect_recog_build_binary_gimple_stmt (vec_info *vinfo, stmt_vec_info stmt_info, return NULL; } +/* Reconcile the INTEGER_CST tree to the given TREE type. The INTEGER_CST tree + * will be returned directly if + * 1. Not INTEGER_CST code or not unsigned TREE type. + * 2. The precision of INTEGER_CST is the same as the precision of TREE type. + * 3. The INTEGER_CST op exceeds the max value of TREE type. + * + * If precision > cst_precision, the INTEGER_CST op will widen to precision + * of TREE type. + * + * If precision < cst_precision, the INTEGER_CST op will narrow to precision if + * the precision of TREE type is able to hold the value of INTEGER_CST op. + */ + +static tree +vect_recog_reconcile_cst_to_unsigned (tree op, tree type) +{ + if (TREE_CODE (op) != INTEGER_CST || !TYPE_UNSIGNED (type)) + return op; + + unsigned precision = TYPE_PRECISION (type); + unsigned cst_precision = TYPE_PRECISION (TREE_TYPE (op)); + + if (cst_precision == precision) + return op; + + wide_int cst = wi::to_wide (op); + + if (cst_precision < precision) + return wide_int_to_tree (type, cst); + + wide_int max = wi::mask (precision, false, cst_precision); + + return wi::leu_p (cst, max) ? wide_int_to_tree (type, cst) : op; +} + /* * Try to detect saturation add pattern (SAT_ADD), aka below gimple: * _7 = _4 + _6; @@ -4558,6 +4593,9 @@ vect_recog_sat_add_pattern (vec_info *vinfo, stmt_vec_info stmt_vinfo, if (gimple_unsigned_integer_sat_add (lhs, ops, NULL)) { + ops[0] = vect_recog_reconcile_cst_to_unsigned (ops[0], TREE_TYPE (ops[1])); + ops[1] = vect_recog_reconcile_cst_to_unsigned (ops[1], TREE_TYPE (ops[0])); + gimple *stmt = vect_recog_build_binary_gimple_stmt (vinfo, stmt_vinfo, IFN_SAT_ADD, type_out, lhs, ops[0], ops[1]);