diff mbox series

[v2,3/3] RISC-V: Add Zalrsc amo-op patterns

Message ID 20240603215329.2664893-4-patrick@rivosinc.com
State New
Headers show
Series RISC-V: Add basic Zaamo and Zalrsc support | expand

Commit Message

Patrick O'Neill June 3, 2024, 9:53 p.m. UTC 
All amo<op> patterns can be represented with lrsc sequences.
Add these patterns as a fallback when Zaamo is not enabled.

gcc/ChangeLog:

	* config/riscv/sync.md (atomic_<atomic_optab><mode>): New expand pattern.
	(amo_atomic_<atomic_optab><mode>): Rename amo pattern.
	(atomic_fetch_<atomic_optab><mode>): New lrsc sequence pattern.
	(lrsc_atomic_<atomic_optab><mode>): New expand pattern.
	(amo_atomic_fetch_<atomic_optab><mode>): Rename amo pattern.
	(lrsc_atomic_fetch_<atomic_optab><mode>): New lrsc sequence pattern.
	(atomic_exchange<mode>): New expand pattern.
	(amo_atomic_exchange<mode>): Rename amo pattern.
	(lrsc_atomic_exchange<mode>): New lrsc sequence pattern.

gcc/testsuite/ChangeLog:

	* gcc.target/riscv/amo-zaamo-preferred-over-zalrsc.c: New test.
	* gcc.target/riscv/amo-zalrsc-amo-add-1.c: New test.
	* gcc.target/riscv/amo-zalrsc-amo-add-2.c: New test.
	* gcc.target/riscv/amo-zalrsc-amo-add-3.c: New test.
	* gcc.target/riscv/amo-zalrsc-amo-add-4.c: New test.
	* gcc.target/riscv/amo-zalrsc-amo-add-5.c: New test.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
------
rv64imfdc_zalrsc has the same testsuite results as rv64imafdc after this
patch is applied.
---
AFAIK there isn't a way to subtract an extension similar to dg-add-options.
As a result I needed to specify a -march string for
amo-zaamo-preferred-over-zalrsc.c instead of using testsuite infra.
---
 gcc/config/riscv/sync.md                      | 124 +++++++++++++++++-
 .../riscv/amo-zaamo-preferred-over-zalrsc.c   |  17 +++
 .../gcc.target/riscv/amo-zalrsc-amo-add-1.c   |  19 +++
 .../gcc.target/riscv/amo-zalrsc-amo-add-2.c   |  19 +++
 .../gcc.target/riscv/amo-zalrsc-amo-add-3.c   |  19 +++
 .../gcc.target/riscv/amo-zalrsc-amo-add-4.c   |  19 +++
 .../gcc.target/riscv/amo-zalrsc-amo-add-5.c   |  19 +++
 7 files changed, 231 insertions(+), 5 deletions(-)
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-zaamo-preferred-over-zalrsc.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-5.c

--
2.34.1

Comments

Jeff Law June 7, 2024, 11:11 p.m. UTC | #1 
On 6/3/24 3:53 PM, Patrick O'Neill wrote:
> All amo<op> patterns can be represented with lrsc sequences.
> Add these patterns as a fallback when Zaamo is not enabled.
> 
> gcc/ChangeLog:
> 
> 	* config/riscv/sync.md (atomic_<atomic_optab><mode>): New expand pattern.
> 	(amo_atomic_<atomic_optab><mode>): Rename amo pattern.
> 	(atomic_fetch_<atomic_optab><mode>): New lrsc sequence pattern.
> 	(lrsc_atomic_<atomic_optab><mode>): New expand pattern.
> 	(amo_atomic_fetch_<atomic_optab><mode>): Rename amo pattern.
> 	(lrsc_atomic_fetch_<atomic_optab><mode>): New lrsc sequence pattern.
> 	(atomic_exchange<mode>): New expand pattern.
> 	(amo_atomic_exchange<mode>): Rename amo pattern.
> 	(lrsc_atomic_exchange<mode>): New lrsc sequence pattern.
> 
> gcc/testsuite/ChangeLog:
> 
> 	* gcc.target/riscv/amo-zaamo-preferred-over-zalrsc.c: New test.
> 	* gcc.target/riscv/amo-zalrsc-amo-add-1.c: New test.
> 	* gcc.target/riscv/amo-zalrsc-amo-add-2.c: New test.
> 	* gcc.target/riscv/amo-zalrsc-amo-add-3.c: New test.
> 	* gcc.target/riscv/amo-zalrsc-amo-add-4.c: New test.
> 	* gcc.target/riscv/amo-zalrsc-amo-add-5.c: New test.
> 
> Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
> ------
> rv64imfdc_zalrsc has the same testsuite results as rv64imafdc after this
> patch is applied.
> ---
> AFAIK there isn't a way to subtract an extension similar to dg-add-options.
> As a result I needed to specify a -march string for
> amo-zaamo-preferred-over-zalrsc.c instead of using testsuite infra.
I believe you are correct.



> diff --git a/gcc/testsuite/gcc.target/riscv/amo-zaamo-preferred-over-zalrsc.c b/gcc/testsuite/gcc.target/riscv/amo-zaamo-preferred-over-zalrsc.c
> new file mode 100644
> index 00000000000..1c124c2b8b1
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-zaamo-preferred-over-zalrsc.c
[ ... ]
Not a big fan of the function-bodies tests.  If we're going to use them, 
we need to be especially careful about requiring specific registers so 
that we're not stuck adjusting them all the time due to changes in the 
regsiter allocator, optimizers, etc.

> diff --git a/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-1.c b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-1.c
> new file mode 100644
> index 00000000000..3cd6ce04830
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-1.c
> @@ -0,0 +1,19 @@
> +/* { dg-do compile } */
> +/* Verify that lrsc atomic op mappings match Table A.6's recommended mapping.  */
> +/* { dg-options "-O3 -march=rv64id_zalrsc" } */
> +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
> +/* { dg-final { check-function-bodies "**" "" } } */
> +
> +/*
> +** foo:
> +**	1:
> +**	lr.w\ta5, 0\(a0\)
> +**	add\ta5, a5, a1
> +**	sc.w\ta5, a5, 0\(a0\)
> +**      bnez\ta5, 1b
> +**	ret
> +*/
> +void foo (int* bar, int* baz)
> +{
> +  __atomic_add_fetch(bar, baz, __ATOMIC_RELAXED);
> +}
This one is a good example.  We could just as easily use a variety of 
registers other than a5 for the temporary.

Obviously for registers that hold the incoming argument or an outgoing 
result, we can be more strict.

If you could take a look at the added tests and generalize the registers 
it'd be appreciated.  OK with that adjustment.

jeff
diff mbox series

Patch

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index c9544176ead..4df9d0b5a5f 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -86,7 +86,24 @@ 
     DONE;
   })

-(define_insn "atomic_<atomic_optab><mode>"
+;; AMO ops
+
+(define_expand "atomic_<atomic_optab><mode>"
+  [(any_atomic:GPR (match_operand:GPR 0 "memory_operand")    ;; mem location
+		   (match_operand:GPR 1 "reg_or_0_operand")) ;; value for op
+   (match_operand:SI 2 "const_int_operand")]		     ;; model
+  "TARGET_ZAAMO || TARGET_ZALRSC"
+{
+  if (TARGET_ZAAMO)
+    emit_insn (gen_amo_atomic_<atomic_optab><mode> (operands[0], operands[1],
+						    operands[2]));
+  else
+    emit_insn (gen_lrsc_atomic_<atomic_optab><mode> (operands[0], operands[1],
+						     operands[2]));
+  DONE;
+})
+
+(define_insn "amo_atomic_<atomic_optab><mode>"
   [(set (match_operand:GPR 0 "memory_operand" "+A")
 	(unspec_volatile:GPR
 	  [(any_atomic:GPR (match_dup 0)
@@ -98,7 +115,44 @@ 
   [(set_attr "type" "atomic")
    (set (attr "length") (const_int 4))])

-(define_insn "atomic_fetch_<atomic_optab><mode>"
+(define_insn "lrsc_atomic_<atomic_optab><mode>"
+  [(set (match_operand:GPR 0 "memory_operand" "+A")
+	(unspec_volatile:GPR
+	  [(any_atomic:GPR (match_dup 0)
+		     (match_operand:GPR 1 "reg_or_0_operand" "rJ"))
+	   (match_operand:SI 2 "const_int_operand")] ;; model
+	 UNSPEC_SYNC_OLD_OP))
+   (clobber (match_scratch:GPR 3 "=&r"))]	     ;; tmp_1
+  "!TARGET_ZAAMO && TARGET_ZALRSC"
+  {
+    return "1:\;"
+	   "lr.<amo>%I2\t%3, %0\;"
+	   "<insn>\t%3, %3, %1\;"
+	   "sc.<amo>%J2\t%3, %3, %0\;"
+	   "bnez\t%3, 1b";
+  }
+  [(set_attr "type" "atomic")
+   (set (attr "length") (const_int 16))])
+
+;; AMO fetch ops
+
+(define_expand "atomic_fetch_<atomic_optab><mode>"
+  [(match_operand:GPR 0 "register_operand")		     ;; old value at mem
+   (any_atomic:GPR (match_operand:GPR 1 "memory_operand")    ;; mem location
+		   (match_operand:GPR 2 "reg_or_0_operand")) ;; value for op
+   (match_operand:SI 3 "const_int_operand")]		     ;; model
+  "TARGET_ZAAMO || TARGET_ZALRSC"
+  {
+    if (TARGET_ZAAMO)
+      emit_insn (gen_amo_atomic_fetch_<atomic_optab><mode> (operands[0], operands[1],
+							    operands[2], operands[3]));
+    else
+      emit_insn (gen_lrsc_atomic_fetch_<atomic_optab><mode> (operands[0], operands[1],
+							     operands[2], operands[3]));
+    DONE;
+  })
+
+(define_insn "amo_atomic_fetch_<atomic_optab><mode>"
   [(set (match_operand:GPR 0 "register_operand" "=&r")
 	(match_operand:GPR 1 "memory_operand" "+A"))
    (set (match_dup 1)
@@ -112,6 +166,27 @@ 
   [(set_attr "type" "atomic")
    (set (attr "length") (const_int 4))])

+(define_insn "lrsc_atomic_fetch_<atomic_optab><mode>"
+  [(set (match_operand:GPR 0 "register_operand" "=&r")
+	(match_operand:GPR 1 "memory_operand" "+A"))
+   (set (match_dup 1)
+	(unspec_volatile:GPR
+	  [(any_atomic:GPR (match_dup 1)
+		     (match_operand:GPR 2 "reg_or_0_operand" "rJ"))
+	   (match_operand:SI 3 "const_int_operand")] ;; model
+	 UNSPEC_SYNC_OLD_OP))
+   (clobber (match_scratch:GPR 4 "=&r"))]	  ;; tmp_1
+  "!TARGET_ZAAMO && TARGET_ZALRSC"
+  {
+    return "1:\;"
+	   "lr.<amo>%I3\t%0, %1\;"
+	   "<insn>\t%4, %0, %2\;"
+	   "sc.<amo>%J3\t%4, %4, %1\;"
+	   "bnez\t%4, 1b";
+  }
+  [(set_attr "type" "atomic")
+   (set (attr "length") (const_int 20))])
+
 (define_insn "subword_atomic_fetch_strong_<atomic_optab>"
   [(set (match_operand:SI 0 "register_operand" "=&r")		   ;; old value at mem
 	(match_operand:SI 1 "memory_operand" "+A"))		   ;; mem location
@@ -248,7 +323,23 @@ 
   DONE;
 })

-(define_insn "atomic_exchange<mode>"
+(define_expand "atomic_exchange<mode>"
+  [(match_operand:GPR 0 "register_operand")  ;; old value at mem
+   (match_operand:GPR 1 "memory_operand")    ;; mem location
+   (match_operand:GPR 2 "register_operand")  ;; value for op
+   (match_operand:SI 3 "const_int_operand")] ;; model
+  "TARGET_ZAAMO || TARGET_ZALRSC"
+  {
+    if (TARGET_ZAAMO)
+      emit_insn (gen_amo_atomic_exchange<mode> (operands[0], operands[1],
+					    operands[2], operands[3]));
+    else
+      emit_insn (gen_lrsc_atomic_exchange<mode> (operands[0], operands[1],
+					     operands[2], operands[3]));
+    DONE;
+  })
+
+(define_insn "amo_atomic_exchange<mode>"
   [(set (match_operand:GPR 0 "register_operand" "=&r")
 	(unspec_volatile:GPR
 	  [(match_operand:GPR 1 "memory_operand" "+A")
@@ -261,6 +352,26 @@ 
   [(set_attr "type" "atomic")
    (set (attr "length") (const_int 4))])

+(define_insn "lrsc_atomic_exchange<mode>"
+  [(set (match_operand:GPR 0 "register_operand" "=&r")
+	(unspec_volatile:GPR
+	  [(match_operand:GPR 1 "memory_operand" "+A")
+	   (match_operand:SI 3 "const_int_operand")] ;; model
+	  UNSPEC_SYNC_EXCHANGE))
+   (set (match_dup 1)
+	(match_operand:GPR 2 "register_operand" "0"))
+   (clobber (match_scratch:GPR 4 "=&r"))]	  ;; tmp_1
+  "!TARGET_ZAAMO && TARGET_ZALRSC"
+  {
+    return "1:\;"
+	   "lr.<amo>%I3\t%4, %1\;"
+	   "sc.<amo>%J3\t%0, %0, %1\;"
+	   "bnez\t%0, 1b\;"
+	   "mv\t%0, %4";
+  }
+  [(set_attr "type" "atomic")
+   (set (attr "length") (const_int 20))])
+
 (define_expand "atomic_exchange<mode>"
   [(match_operand:SHORT 0 "register_operand") ;; old value at mem
    (match_operand:SHORT 1 "memory_operand")   ;; mem location
@@ -516,7 +627,7 @@ 
   [(match_operand:QI 0 "register_operand" "")    ;; bool output
    (match_operand:QI 1 "memory_operand" "+A")    ;; memory
    (match_operand:SI 2 "const_int_operand" "")]  ;; model
-  "TARGET_ZALRSC"
+  "TARGET_ZAAMO || TARGET_ZALRSC"
 {
   /* We have no QImode atomics, so use the address LSBs to form a mask,
      then use an aligned SImode atomic.  */
@@ -537,7 +648,10 @@ 
   rtx shifted_set = gen_reg_rtx (SImode);
   riscv_lshift_subword (QImode, set, shift, &shifted_set);

-  emit_insn (gen_atomic_fetch_orsi (old, aligned_mem, shifted_set, model));
+  if (TARGET_ZAAMO)
+    emit_insn (gen_amo_atomic_fetch_orsi (old, aligned_mem, shifted_set, model));
+  else if (TARGET_ZALRSC)
+    emit_insn (gen_lrsc_atomic_fetch_orsi (old, aligned_mem, shifted_set, model));

   emit_move_insn (old, gen_rtx_ASHIFTRT (SImode, old,
 					 gen_lowpart (QImode, shift)));
diff --git a/gcc/testsuite/gcc.target/riscv/amo-zaamo-preferred-over-zalrsc.c b/gcc/testsuite/gcc.target/riscv/amo-zaamo-preferred-over-zalrsc.c
new file mode 100644
index 00000000000..1c124c2b8b1
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-zaamo-preferred-over-zalrsc.c
@@ -0,0 +1,17 @@ 
+/* { dg-do compile } */
+/* Ensure that AMO ops are emitted when both zalrsc and zaamo are enabled.  */
+/* { dg-options "-O3" } */
+/* { dg-add-options riscv_zalrsc } */
+/* { dg-add-options riscv_zaamo } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	amoadd\.w\tzero,a1,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-1.c b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-1.c
new file mode 100644
index 00000000000..3cd6ce04830
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-1.c
@@ -0,0 +1,19 @@ 
+/* { dg-do compile } */
+/* Verify that lrsc atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3 -march=rv64id_zalrsc" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	1:
+**	lr.w\ta5, 0\(a0\)
+**	add\ta5, a5, a1
+**	sc.w\ta5, a5, 0\(a0\)
+**      bnez\ta5, 1b
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-2.c b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-2.c
new file mode 100644
index 00000000000..d7371dac301
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-2.c
@@ -0,0 +1,19 @@ 
+/* { dg-do compile } */
+/* Verify that lrsc atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3 -march=rv64id_zalrsc" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	1:
+**	lr.w.aq\ta5, 0\(a0\)
+**	add\ta5, a5, a1
+**	sc.w\ta5, a5, 0\(a0\)
+**      bnez\ta5, 1b
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-3.c b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-3.c
new file mode 100644
index 00000000000..25060b0894f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-3.c
@@ -0,0 +1,19 @@ 
+/* { dg-do compile } */
+/* Verify that lrsc atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3 -march=rv64id_zalrsc" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	1:
+**	lr.w\ta5, 0\(a0\)
+**	add\ta5, a5, a1
+**	sc.w.rl\ta5, a5, 0\(a0\)
+**      bnez\ta5, 1b
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-4.c b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-4.c
new file mode 100644
index 00000000000..7cf9357fdbc
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-4.c
@@ -0,0 +1,19 @@ 
+/* { dg-do compile } */
+/* Verify that lrsc atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3 -march=rv64id_zalrsc" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	1:
+**	lr.w.aq\ta5, 0\(a0\)
+**	add\ta5, a5, a1
+**	sc.w.rl\ta5, a5, 0\(a0\)
+**      bnez\ta5, 1b
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQ_REL);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-5.c b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-5.c
new file mode 100644
index 00000000000..df761892493
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-5.c
@@ -0,0 +1,19 @@ 
+/* { dg-do compile } */
+/* Verify that lrsc atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3 -march=rv64id_zalrsc" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	1:
+**	lr.w.aqrl\ta5, 0\(a0\)
+**	add\ta5, a5, a1
+**	sc.w.rl\ta5, a5, 0\(a0\)
+**      bnez\ta5, 1b
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_SEQ_CST);
+}