diff mbox series

[x86,SSE] Some AVX512 ternlog expansion refinements (take #2)

Message ID 009501dad3d9$ed09d720$c71d8560$@nextmovesoftware.com
State New
Headers show
Series [x86,SSE] Some AVX512 ternlog expansion refinements (take #2) | expand

Commit Message

Roger Sayle July 11, 2024, 9:33 p.m. UTC 
Hi Hongtao,
Thanks for the review and pointing out the remaining uses of force_reg
that I'd overlooked.  Here's a revised version of the patch that incorporates
your feedback.  One minor change was that rather than using
memory_operand, which as you point out also needs to include
bcst_mem_operand, it's simpler to invert the logic to check for
register_operand [i.e. the first operand must be a register].

This patch has been tested on x86_64-pc-linux-gnu with make bootstrap
and make -k check, both with and without --target_board=unix{-m32}
with no new failures.  Ok for mainline?


2024-07-11  Roger Sayle  <roger@nextmovesoftware.com>
            Hongtao Liu  <hongtao.liu@intel.com>

gcc/ChangeLog
        * config/i386/i386-expand.cc (ix86_broadcast_from_constant):
        Use CONST_VECTOR_P instead of comparison against GET_CODE.
        (ix86_gen_bcst_mem): Likewise.
        (ix86_ternlog_leaf_p): Likewise.
        (ix86_ternlog_operand_p): ix86_ternlog_leaf_p is always true for
        vector_all_ones_operand.
        (ix86_expand_ternlog_bin_op): Use CONST_VECTOR_P instead of
        equality comparison against GET_CODE.  Replace call to force_reg
        with gen_reg_rtx and emit_move_insn (for VEC_DUPLICATE broadcast).
        Check for !register_operand instead of memory_operand.
        Support CONST_VECTORs by calling force_const_mem.
        (ix86_expand_ternlog): Fix indentation whitespace.
        Allow ix86_ternlog_leaf_p as ix86_expand_ternlog_andnot's second
        operand. Use CONST_VECTOR_P instead of equality against GET_CODE.
        Use gen_reg_rtx and emit_move_insn for ~a, ~b and ~c cases.


Thanks again,
Roger

> -----Original Message-----
> From: Hongtao Liu <crazylht@gmail.com>
> Sent: 08 July 2024 02:55
> To: Roger Sayle <roger@nextmovesoftware.com>
> Cc: gcc-patches@gcc.gnu.org; Uros Bizjak <ubizjak@gmail.com>
> Subject: Re: [x86 SSE PATCH] Some AVX512 ternlog expansion refinements.
> 
> On Sun, Jul 7, 2024 at 5:00 PM Roger Sayle <roger@nextmovesoftware.com>
> wrote:
> > Hi Hongtao,
> > This should address concerns about the remaining use of force_reg.
> >
>  51@@ -25793,15 +25792,20 @@ ix86_expand_ternlog_binop (enum rtx_code
> code, machine_mode mode,
>  52   if (GET_MODE (op1) != mode)
>  53     op1 = gen_lowpart (mode, op1);
>  54
>  55-  if (GET_CODE (op0) == CONST_VECTOR)  56+  if (CONST_VECTOR_P (op0))
>  57     op0 = validize_mem (force_const_mem (mode, op0));
>  58-  if (GET_CODE (op1) == CONST_VECTOR)  59+  if (CONST_VECTOR_P (op1))
>  60     op1 = validize_mem (force_const_mem (mode, op1));
>  61
>  62   if (memory_operand (op0, mode))
>  63     {
>  64       if (memory_operand (op1, mode))
>  65-       op0 = force_reg (mode, op0);
>  66+       {
>  67+         /* We can't use force_reg (op0, mode).  */
>  68+         rtx reg = gen_reg_rtx (mode);
>  69+         emit_move_insn (reg, op0);
>  70+         op0 = reg;
>  71+       }
> Shouldn't we handle bcst_mem_operand instead of
> memory_operand(bcst_memory_operand is not a memory_operand)?
> so maybe
> if (memory_operand (op0, mode0) || bcst_mem_operand (op0, mode0)
>   if (memory_operand (op1, mode) || bcst_mem_operand (op1, mode0)?
>  72       else
>  73        std::swap (op0, op1);
>  74     }
> 
> Also there's force_reg in below 3 cases, are there any restrictions to avoid
> bcst_mem_operand into them?
> case 0x0f:  /* ~a */
> case 0x33:  /* ~b */
> case 0x33:  /* ~b */
> ..
>          if (!TARGET_64BIT && !register_operand (op2, mode))
>            op2 = force_reg (mode, op2);
> 
> --
> BR,
> Hongtao

Comments

Hongtao Liu July 12, 2024, 2:20 a.m. UTC | #1 
On Fri, Jul 12, 2024 at 5:33 AM Roger Sayle <roger@nextmovesoftware.com> wrote:
>
>
> Hi Hongtao,
> Thanks for the review and pointing out the remaining uses of force_reg
> that I'd overlooked.  Here's a revised version of the patch that incorporates
> your feedback.  One minor change was that rather than using
> memory_operand, which as you point out also needs to include
> bcst_mem_operand, it's simpler to invert the logic to check for
> register_operand [i.e. the first operand must be a register].
>
> This patch has been tested on x86_64-pc-linux-gnu with make bootstrap
> and make -k check, both with and without --target_board=unix{-m32}
> with no new failures.  Ok for mainline?
Ok.
>
>
> 2024-07-11  Roger Sayle  <roger@nextmovesoftware.com>
>             Hongtao Liu  <hongtao.liu@intel.com>
>
> gcc/ChangeLog
>         * config/i386/i386-expand.cc (ix86_broadcast_from_constant):
>         Use CONST_VECTOR_P instead of comparison against GET_CODE.
>         (ix86_gen_bcst_mem): Likewise.
>         (ix86_ternlog_leaf_p): Likewise.
>         (ix86_ternlog_operand_p): ix86_ternlog_leaf_p is always true for
>         vector_all_ones_operand.
>         (ix86_expand_ternlog_bin_op): Use CONST_VECTOR_P instead of
>         equality comparison against GET_CODE.  Replace call to force_reg
>         with gen_reg_rtx and emit_move_insn (for VEC_DUPLICATE broadcast).
>         Check for !register_operand instead of memory_operand.
>         Support CONST_VECTORs by calling force_const_mem.
>         (ix86_expand_ternlog): Fix indentation whitespace.
>         Allow ix86_ternlog_leaf_p as ix86_expand_ternlog_andnot's second
>         operand. Use CONST_VECTOR_P instead of equality against GET_CODE.
>         Use gen_reg_rtx and emit_move_insn for ~a, ~b and ~c cases.
>
>
> Thanks again,
> Roger
>
> > -----Original Message-----
> > From: Hongtao Liu <crazylht@gmail.com>
> > Sent: 08 July 2024 02:55
> > To: Roger Sayle <roger@nextmovesoftware.com>
> > Cc: gcc-patches@gcc.gnu.org; Uros Bizjak <ubizjak@gmail.com>
> > Subject: Re: [x86 SSE PATCH] Some AVX512 ternlog expansion refinements.
> >
> > On Sun, Jul 7, 2024 at 5:00 PM Roger Sayle <roger@nextmovesoftware.com>
> > wrote:
> > > Hi Hongtao,
> > > This should address concerns about the remaining use of force_reg.
> > >
> >  51@@ -25793,15 +25792,20 @@ ix86_expand_ternlog_binop (enum rtx_code
> > code, machine_mode mode,
> >  52   if (GET_MODE (op1) != mode)
> >  53     op1 = gen_lowpart (mode, op1);
> >  54
> >  55-  if (GET_CODE (op0) == CONST_VECTOR)  56+  if (CONST_VECTOR_P (op0))
> >  57     op0 = validize_mem (force_const_mem (mode, op0));
> >  58-  if (GET_CODE (op1) == CONST_VECTOR)  59+  if (CONST_VECTOR_P (op1))
> >  60     op1 = validize_mem (force_const_mem (mode, op1));
> >  61
> >  62   if (memory_operand (op0, mode))
> >  63     {
> >  64       if (memory_operand (op1, mode))
> >  65-       op0 = force_reg (mode, op0);
> >  66+       {
> >  67+         /* We can't use force_reg (op0, mode).  */
> >  68+         rtx reg = gen_reg_rtx (mode);
> >  69+         emit_move_insn (reg, op0);
> >  70+         op0 = reg;
> >  71+       }
> > Shouldn't we handle bcst_mem_operand instead of
> > memory_operand(bcst_memory_operand is not a memory_operand)?
> > so maybe
> > if (memory_operand (op0, mode0) || bcst_mem_operand (op0, mode0)
> >   if (memory_operand (op1, mode) || bcst_mem_operand (op1, mode0)?
> >  72       else
> >  73        std::swap (op0, op1);
> >  74     }
> >
> > Also there's force_reg in below 3 cases, are there any restrictions to avoid
> > bcst_mem_operand into them?
> > case 0x0f:  /* ~a */
> > case 0x33:  /* ~b */
> > case 0x33:  /* ~b */
> > ..
> >          if (!TARGET_64BIT && !register_operand (op2, mode))
> >            op2 = force_reg (mode, op2);
> >
> > --
> > BR,
> > Hongtao
diff mbox series

Patch

diff --git a/gcc/config/i386/i386-expand.cc b/gcc/config/i386/i386-expand.cc
index dd2c3a8..7818d40 100644
--- a/gcc/config/i386/i386-expand.cc
+++ b/gcc/config/i386/i386-expand.cc
@@ -612,7 +612,7 @@  ix86_broadcast_from_constant (machine_mode mode, rtx op)
     return nullptr;
 
   rtx constant = get_pool_constant (XEXP (op, 0));
-  if (GET_CODE (constant) != CONST_VECTOR)
+  if (!CONST_VECTOR_P (constant))
     return nullptr;
 
   /* There could be some rtx like
@@ -622,7 +622,7 @@  ix86_broadcast_from_constant (machine_mode mode, rtx op)
     {
       constant = simplify_subreg (mode, constant, GET_MODE (constant),
 				  0);
-      if (constant == nullptr || GET_CODE (constant) != CONST_VECTOR)
+      if (constant == nullptr || !CONST_VECTOR_P (constant))
 	return nullptr;
     }
 
@@ -25532,7 +25532,7 @@  static rtx
 ix86_gen_bcst_mem (machine_mode mode, rtx x)
 {
   if (!TARGET_AVX512F
-      || GET_CODE (x) != CONST_VECTOR
+      || !CONST_VECTOR_P (x)
       || (!TARGET_AVX512VL
 	  && (GET_MODE_SIZE (mode) != 64 || !TARGET_EVEX512))
       || !VALID_BCST_MODE_P (GET_MODE_INNER (mode))
@@ -25722,7 +25722,7 @@  ix86_ternlog_leaf_p (rtx op, machine_mode mode)
      problems splitting instructions.  */
   return register_operand (op, mode)
 	 || MEM_P (op)
-	 || GET_CODE (op) == CONST_VECTOR
+	 || CONST_VECTOR_P (op)
 	 || bcst_mem_operand (op, mode);
 }
 
@@ -25772,8 +25772,7 @@  ix86_ternlog_operand_p (rtx op)
       op1 = XEXP (op, 1);
       /* Prefer pxor, or one_cmpl<vmode>2.  */
       if (ix86_ternlog_leaf_p (XEXP (op, 0), mode)
-	  && (ix86_ternlog_leaf_p (op1, mode)
-	      || vector_all_ones_operand (op1, mode)))
+	  && ix86_ternlog_leaf_p (XEXP (op, 1), mode))
 	return false;
       break;
 
@@ -25793,15 +25792,20 @@  ix86_expand_ternlog_binop (enum rtx_code code, machine_mode mode,
   if (GET_MODE (op1) != mode)
     op1 = gen_lowpart (mode, op1);
 
-  if (GET_CODE (op0) == CONST_VECTOR)
+  if (CONST_VECTOR_P (op0))
     op0 = validize_mem (force_const_mem (mode, op0));
-  if (GET_CODE (op1) == CONST_VECTOR)
+  if (CONST_VECTOR_P (op1))
     op1 = validize_mem (force_const_mem (mode, op1));
 
-  if (memory_operand (op0, mode))
+  if (!register_operand (op0, mode))
     {
-      if (memory_operand (op1, mode))
-	op0 = force_reg (mode, op0);
+      if (!register_operand (op1, mode))
+	{
+	  /* We can't use force_reg (op0, mode).  */
+	  rtx reg = gen_reg_rtx (mode);
+	  emit_move_insn (reg, op0);
+	  op0 = reg;
+	}
       else
 	std::swap (op0, op1);
     }
@@ -25820,6 +25824,8 @@  ix86_expand_ternlog_andnot (machine_mode mode, rtx op0, rtx op1, rtx target)
   op0 = gen_rtx_NOT (mode, op0);
   if (GET_MODE (op1) != mode)
     op1 = gen_lowpart (mode, op1);
+  if (CONST_VECTOR_P (op1))
+    op1 = validize_mem (force_const_mem (mode, op1));
   emit_move_insn (target, gen_rtx_AND (mode, op0, op1));
   return target;
 }
@@ -25856,9 +25862,9 @@  ix86_expand_ternlog (machine_mode mode, rtx op0, rtx op1, rtx op2, int idx,
     {
     case 0x00:
       if ((!op0 || !side_effects_p (op0))
-          && (!op1 || !side_effects_p (op1))
-          && (!op2 || !side_effects_p (op2)))
-        {
+	  && (!op1 || !side_effects_p (op1))
+	  && (!op2 || !side_effects_p (op2)))
+	{
 	  emit_move_insn (target, CONST0_RTX (mode));
 	  return target;
 	}
@@ -25867,26 +25873,31 @@  ix86_expand_ternlog (machine_mode mode, rtx op0, rtx op1, rtx op2, int idx,
     case 0x0a: /* ~a&c */
       if ((!op1 || !side_effects_p (op1))
 	  && op0 && register_operand (op0, mode)
-	  && op2 && register_operand (op2, mode))
+	  && op2 && ix86_ternlog_leaf_p (op2, mode))
 	return ix86_expand_ternlog_andnot (mode, op0, op2, target);
       break;
 
     case 0x0c: /* ~a&b */
       if ((!op2 || !side_effects_p (op2))
 	  && op0 && register_operand (op0, mode)
-	  && op1 && register_operand (op1, mode))
+	  && op1 && ix86_ternlog_leaf_p (op1, mode))
 	return ix86_expand_ternlog_andnot (mode, op0, op1, target);
       break;
 
     case 0x0f:  /* ~a */
       if ((!op1 || !side_effects_p (op1))
 	  && (!op2 || !side_effects_p (op2))
-          && op0)
+	  && op0)
 	{
 	  if (GET_MODE (op0) != mode)
 	    op0 = gen_lowpart (mode, op0);
 	  if (!TARGET_64BIT && !register_operand (op0, mode))
-	    op0 = force_reg (mode, op0);
+	    {
+	      /* Avoid force_reg (mode, op0).  */
+	      rtx reg = gen_reg_rtx (mode);
+	      emit_move_insn (reg, op0);
+	      op0 = reg;
+	    }
 	  emit_move_insn (target, gen_rtx_XOR (mode, op0, CONSTM1_RTX (mode)));
 	  return target;
 	}
@@ -25895,13 +25906,13 @@  ix86_expand_ternlog (machine_mode mode, rtx op0, rtx op1, rtx op2, int idx,
     case 0x22: /* ~b&c */
       if ((!op0 || !side_effects_p (op0))
 	  && op1 && register_operand (op1, mode)
-	  && op2 && register_operand (op2, mode))
+	  && op2 && ix86_ternlog_leaf_p (op2, mode))
 	return ix86_expand_ternlog_andnot (mode, op1, op2, target);
       break;
 
     case 0x30: /* ~b&a */
       if ((!op2 || !side_effects_p (op2))
-	  && op0 && register_operand (op0, mode)
+	  && op0 && ix86_ternlog_leaf_p (op0, mode)
 	  && op1 && register_operand (op1, mode))
 	return ix86_expand_ternlog_andnot (mode, op1, op0, target);
       break;
@@ -25909,33 +25920,39 @@  ix86_expand_ternlog (machine_mode mode, rtx op0, rtx op1, rtx op2, int idx,
     case 0x33:  /* ~b */
       if ((!op0 || !side_effects_p (op0))
 	  && (!op2 || !side_effects_p (op2))
-          && op1)
+	  && op1)
 	{
 	  if (GET_MODE (op1) != mode)
 	    op1 = gen_lowpart (mode, op1);
 	  if (!TARGET_64BIT && !register_operand (op1, mode))
-	    op1 = force_reg (mode, op1);
+	    {
+	      /* Avoid force_reg (mode, op1).  */
+	      rtx reg = gen_reg_rtx (mode);
+	      emit_move_insn (reg, op1);
+	      op1 = reg;
+	    }
 	  emit_move_insn (target, gen_rtx_XOR (mode, op1, CONSTM1_RTX (mode)));
 	  return target;
 	}
       break;
 
     case 0x3c:  /* a^b */
-      if (op0 && op1
-          && (!op2 || !side_effects_p (op2)))
+      if (op0 && ix86_ternlog_leaf_p (op0, mode)
+	  && op1 && ix86_ternlog_leaf_p (op1, mode)
+	  && (!op2 || !side_effects_p (op2)))
 	return ix86_expand_ternlog_binop (XOR, mode, op0, op1, target);
       break;
 
     case 0x44: /* ~c&b */
       if ((!op0 || !side_effects_p (op0))
-	  && op1 && register_operand (op1, mode)
+	  && op1 && ix86_ternlog_leaf_p (op1, mode)
 	  && op2 && register_operand (op2, mode))
 	return ix86_expand_ternlog_andnot (mode, op2, op1, target);
       break;
 
     case 0x50: /* ~c&a */
       if ((!op1 || !side_effects_p (op1))
-	  && op0 && register_operand (op0, mode)
+	  && op0 && ix86_ternlog_leaf_p (op0, mode)
 	  && op2 && register_operand (op2, mode))
 	return ix86_expand_ternlog_andnot (mode, op2, op0, target);
       break;
@@ -25943,45 +25960,54 @@  ix86_expand_ternlog (machine_mode mode, rtx op0, rtx op1, rtx op2, int idx,
     case 0x55:  /* ~c */
       if ((!op0 || !side_effects_p (op0))
 	  && (!op1 || !side_effects_p (op1))
-          && op2)
+	  && op2)
 	{
 	  if (GET_MODE (op2) != mode)
 	    op2 = gen_lowpart (mode, op2);
 	  if (!TARGET_64BIT && !register_operand (op2, mode))
-	    op2 = force_reg (mode, op2);
+	    {
+	      /* Avoid force_reg (mode, op2).  */
+	      rtx reg = gen_reg_rtx (mode);
+	      emit_move_insn (reg, op2);
+	      op2 = reg;
+	    }
 	  emit_move_insn (target, gen_rtx_XOR (mode, op2, CONSTM1_RTX (mode)));
 	  return target;
 	}
       break;
   
     case 0x5a:  /* a^c */
-      if (op0 && op2
-          && (!op1 || !side_effects_p (op1)))
+      if (op0 && ix86_ternlog_leaf_p (op0, mode)
+	  && op2 && ix86_ternlog_leaf_p (op2, mode)
+	  && (!op1 || !side_effects_p (op1)))
 	return ix86_expand_ternlog_binop (XOR, mode, op0, op2, target);
       break;
 
     case 0x66:  /* b^c */
       if ((!op0 || !side_effects_p (op0))
-          && op1 && op2)
+	  && op1 && ix86_ternlog_leaf_p (op1, mode)
+	  && op2 && ix86_ternlog_leaf_p (op2, mode))
 	return ix86_expand_ternlog_binop (XOR, mode, op1, op2, target);
       break;
 
     case 0x88:  /* b&c */
       if ((!op0 || !side_effects_p (op0))
-          && op1 && op2)
+	  && op1 && ix86_ternlog_leaf_p (op1, mode)
+	  && op2 && ix86_ternlog_leaf_p (op2, mode))
 	return ix86_expand_ternlog_binop (AND, mode, op1, op2, target);
       break;
 
     case 0xa0:  /* a&c */
       if ((!op1 || !side_effects_p (op1))
-          && op0 && op2)
+	  && op0 && ix86_ternlog_leaf_p (op0, mode)
+	  && op2 && ix86_ternlog_leaf_p (op2, mode))
 	return ix86_expand_ternlog_binop (AND, mode, op0, op2, target);
       break;
 
     case 0xaa:  /* c */
       if ((!op0 || !side_effects_p (op0))
 	  && (!op1 || !side_effects_p (op1))
-          && op2)
+	  && op2)
 	{
 	  if (GET_MODE (op2) != mode)
 	    op2 = gen_lowpart (mode, op2);
@@ -25991,14 +26017,15 @@  ix86_expand_ternlog (machine_mode mode, rtx op0, rtx op1, rtx op2, int idx,
       break;
 
     case 0xc0:  /* a&b */
-      if (op0 && op1
-          && (!op2 || !side_effects_p (op2)))
+      if (op0 && ix86_ternlog_leaf_p (op0, mode)
+	  && op1 && ix86_ternlog_leaf_p (op1, mode)
+	  && (!op2 || !side_effects_p (op2)))
 	return ix86_expand_ternlog_binop (AND, mode, op0, op1, target);
       break;
 
     case 0xcc:  /* b */
       if ((!op0 || !side_effects_p (op0))
-          && op1
+	  && op1
 	  && (!op2 || !side_effects_p (op2)))
 	{
 	  if (GET_MODE (op1) != mode)
@@ -26010,13 +26037,14 @@  ix86_expand_ternlog (machine_mode mode, rtx op0, rtx op1, rtx op2, int idx,
 
     case 0xee:  /* b|c */
       if ((!op0 || !side_effects_p (op0))
-          && op1 && op2)
+	  && op1 && ix86_ternlog_leaf_p (op1, mode)
+	  && op2 && ix86_ternlog_leaf_p (op2, mode))
 	return ix86_expand_ternlog_binop (IOR, mode, op1, op2, target);
       break;
 
     case 0xf0:  /* a */
       if (op0
-          && (!op1 || !side_effects_p (op1))
+	  && (!op1 || !side_effects_p (op1))
 	  && (!op2 || !side_effects_p (op2)))
 	{
 	  if (GET_MODE (op0) != mode)
@@ -26027,23 +26055,25 @@  ix86_expand_ternlog (machine_mode mode, rtx op0, rtx op1, rtx op2, int idx,
       break;
 
     case 0xfa:  /* a|c */
-      if (op0 && op2
-          && (!op1 || !side_effects_p (op1)))
+      if (op0 && ix86_ternlog_leaf_p (op0, mode)
+	  && op2 && ix86_ternlog_leaf_p (op2, mode)
+	  && (!op1 || !side_effects_p (op1)))
 	return ix86_expand_ternlog_binop (IOR, mode, op0, op2, target);
       break;
 
     case 0xfc:  /* a|b */
-      if (op0 && op1
-          && (!op2 || !side_effects_p (op2)))
+      if (op0 && ix86_ternlog_leaf_p (op0, mode)
+	  && op1 && ix86_ternlog_leaf_p (op1, mode)
+	  && (!op2 || !side_effects_p (op2)))
 	return ix86_expand_ternlog_binop (IOR, mode, op0, op1, target);
       break;
 
     case 0xff:
       if ((!op0 || !side_effects_p (op0))
-          && (!op1 || !side_effects_p (op1))
-          && (!op2 || !side_effects_p (op2)))
-        {
-          emit_move_insn (target, CONSTM1_RTX (mode));
+	  && (!op1 || !side_effects_p (op1))
+	  && (!op2 || !side_effects_p (op2)))
+	{
+	  emit_move_insn (target, CONSTM1_RTX (mode));
 	  return target;
 	}
       break;
@@ -26066,7 +26096,7 @@  ix86_expand_ternlog (machine_mode mode, rtx op0, rtx op1, rtx op2, int idx,
     tmp2 = copy_rtx (tmp0);
   else if (rtx_equal_p (op1, op2))
     tmp2 = copy_rtx (tmp1);
-  else if (GET_CODE (op2) == CONST_VECTOR)
+  else if (CONST_VECTOR_P (op2))
     {
       if (GET_MODE (op2) != mode)
 	op2 = gen_lowpart (mode, op2);