diff mbox series

[rs6000] Tweak modulo define_insns to eliminate register copy

Message ID 3cad2a5e-dd68-2fbe-d52b-e077a7405623@linux.ibm.com
State New
Headers show
Series [rs6000] Tweak modulo define_insns to eliminate register copy | expand

Commit Message

Pat Haugen Feb. 27, 2023, 3:11 p.m. UTC 
Don't force target of modulo into a distinct register.

The define_insns for the modulo operation currently force the target 
register
to a distinct reg in preparation for a possible future peephole combining
div/mod. But this can lead to cases of a needless copy being inserted. Fixed
with the following patch.

Bootstrapped and regression tested on powerpc64le.
Ok for master?

-Pat


2023-02-27  Pat Haugen  <pthaugen@linux.ibm.com>

gcc/
	* config/rs6000/rs6000.md (*mod<mode>3, umod<mode>3): Add
	non-earlyclobber alternative.

gcc/testsuite/
	* gcc.target/powerpc/mod-no_copy.c: New.

Comments

Segher Boessenkool Feb. 27, 2023, 5:08 p.m. UTC | #1 
Hi!

On Mon, Feb 27, 2023 at 09:11:37AM -0600, Pat Haugen wrote:
> The define_insns for the modulo operation currently force the target 
> register
> to a distinct reg in preparation for a possible future peephole combining
> div/mod. But this can lead to cases of a needless copy being inserted. Fixed
> with the following patch.

Have you verified those peepholes still match?

Do those peepholes actually improve performance?  On new CPUs?  The code
here says
;; On machines with modulo support, do a combined div/mod the old fashioned
;; method, since the multiply/subtract is faster than doing the mod instruction
;; after a divide.
but that really should not be true: we can do the div and mod in
parallel (except in SMT4 perhaps, which we never schedule for anyway),
so that should always be strictly faster.

> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/powerpc/mod-no_copy.c
> @@ -0,0 +1,17 @@
> +/* { dg-do compile { target { powerpc*-*-*  } } } */

All files in gcc.target/powerpc/ test for this already.  Just leave off
the target clause here?

> +/* { dg-require-effective-target powerpc_p9modulo_ok } */

Leave out this line, because ...

> +/* { dg-options "-mdejagnu-cpu=power9 -O2" } */

... the -mcpu= forces it to true always.

> +/* Verify r3 is used as source and target, no copy inserted. */

> +/* { dg-final { scan-assembler-not {\mmr\M} } } */

That is probably good enough, yeah, since the test results in only a
handful of insns.


Segher
Pat Haugen Feb. 27, 2023, 8:12 p.m. UTC | #2 
On 2/27/23 11:08 AM, Segher Boessenkool wrote:
> Hi!
> 
> On Mon, Feb 27, 2023 at 09:11:37AM -0600, Pat Haugen wrote:
>> The define_insns for the modulo operation currently force the target
>> register
>> to a distinct reg in preparation for a possible future peephole combining
>> div/mod. But this can lead to cases of a needless copy being inserted. Fixed
>> with the following patch.
> 
> Have you verified those peepholes still match?

Yes, I verified the peepholes still match and transform the sequence.

> 
> Do those peepholes actually improve performance?  On new CPUs?  The code
> here says
> ;; On machines with modulo support, do a combined div/mod the old fashioned
> ;; method, since the multiply/subtract is faster than doing the mod instruction
> ;; after a divide.
> but that really should not be true: we can do the div and mod in
> parallel (except in SMT4 perhaps, which we never schedule for anyway),
> so that should always be strictly faster.
> 
Since the modulo insns were introduced in Power9, we're just talking 
Power9/Power10. On paper, I would agree that separate div/mod could be 
slightly faster to get the mod result, but if you throw in another 
independent div or mod in the insn stream then doing the peephole should 
be a clear win since that 3rd insn can execute in parallel with the 
initial divide as opposed to waiting for the one of the first div/mod to 
clear the exclusive stage of the pipe.

>> --- /dev/null
>> +++ b/gcc/testsuite/gcc.target/powerpc/mod-no_copy.c
>> @@ -0,0 +1,17 @@
>> +/* { dg-do compile { target { powerpc*-*-*  } } } */
> 
> All files in gcc.target/powerpc/ test for this already.  Just leave off
> the target clause here?
> 
>> +/* { dg-require-effective-target powerpc_p9modulo_ok } */
> 
> Leave out this line, because ...
> 
>> +/* { dg-options "-mdejagnu-cpu=power9 -O2" } */
> 
> ... the -mcpu= forces it to true always.

Will update.

-Pat

> 
>> +/* Verify r3 is used as source and target, no copy inserted. */
> 
>> +/* { dg-final { scan-assembler-not {\mmr\M} } } */
> 
> That is probably good enough, yeah, since the test results in only a
> handful of insns.
> 
> 
> Segher
Segher Boessenkool Feb. 27, 2023, 8:53 p.m. UTC | #3 
Hi!

On Mon, Feb 27, 2023 at 02:12:23PM -0600, Pat Haugen wrote:
> On 2/27/23 11:08 AM, Segher Boessenkool wrote:
> >On Mon, Feb 27, 2023 at 09:11:37AM -0600, Pat Haugen wrote:
> >>The define_insns for the modulo operation currently force the target
> >>register
> >>to a distinct reg in preparation for a possible future peephole combining
> >>div/mod. But this can lead to cases of a needless copy being inserted. 
> >>Fixed
> >>with the following patch.
> >
> >Have you verified those peepholes still match?
> 
> Yes, I verified the peepholes still match and transform the sequence.

Please add the testcases for that then?  Or do we have tests for it
already :-)

> >Do those peepholes actually improve performance?  On new CPUs?  The code
> >here says
> >;; On machines with modulo support, do a combined div/mod the old fashioned
> >;; method, since the multiply/subtract is faster than doing the mod 
> >instruction
> >;; after a divide.
> >but that really should not be true: we can do the div and mod in
> >parallel (except in SMT4 perhaps, which we never schedule for anyway),
> >so that should always be strictly faster.
> >
> Since the modulo insns were introduced in Power9, we're just talking 
> Power9/Power10. On paper, I would agree that separate div/mod could be 
> slightly faster to get the mod result,

"Slightly".  It takes 12 cycles for the two in parallel (64-bit, p9),
but 17 cycles for the "cheaper" sequence (divd+mulld+subf, 12+5+2).  It
is all worse if the units are busy of course, or if there are other
problems.

> but if you throw in another 
> independent div or mod in the insn stream then doing the peephole should 
> be a clear win since that 3rd insn can execute in parallel with the 
> initial divide as opposed to waiting for the one of the first div/mod to 
> clear the exclusive stage of the pipe.

That is the SMT4 case, the one we do not optimise for.  SMT2 and ST can
do four in parallel.  This means you can start a div or mod every 2nd
cycle on average, so it is very unlikely you will ever be limited by
this on real code.


Segher
Pat Haugen Feb. 27, 2023, 10:03 p.m. UTC | #4 
On 2/27/23 2:53 PM, Segher Boessenkool wrote:
> Hi!
> 
> On Mon, Feb 27, 2023 at 02:12:23PM -0600, Pat Haugen wrote:
>> On 2/27/23 11:08 AM, Segher Boessenkool wrote:
>>> On Mon, Feb 27, 2023 at 09:11:37AM -0600, Pat Haugen wrote:
>>>> The define_insns for the modulo operation currently force the target
>>>> register
>>>> to a distinct reg in preparation for a possible future peephole combining
>>>> div/mod. But this can lead to cases of a needless copy being inserted.
>>>> Fixed
>>>> with the following patch.
>>>
>>> Have you verified those peepholes still match?
>>
>> Yes, I verified the peepholes still match and transform the sequence.
> 
> Please add the testcases for that then?  Or do we have tests for it
> already :-)

I don't see one, but can add one.

>>> Do those peepholes actually improve performance?  On new CPUs?  The code
>>> here says
>>> ;; On machines with modulo support, do a combined div/mod the old fashioned
>>> ;; method, since the multiply/subtract is faster than doing the mod
>>> instruction
>>> ;; after a divide.
>>> but that really should not be true: we can do the div and mod in
>>> parallel (except in SMT4 perhaps, which we never schedule for anyway),
>>> so that should always be strictly faster.
>>>
>> Since the modulo insns were introduced in Power9, we're just talking
>> Power9/Power10. On paper, I would agree that separate div/mod could be
>> slightly faster to get the mod result,
> 
> "Slightly".  It takes 12 cycles for the two in parallel (64-bit, p9),
> but 17 cycles for the "cheaper" sequence (divd+mulld+subf, 12+5+2).  It
> is all worse if the units are busy of course, or if there are other
> problems.
> 
>> but if you throw in another
>> independent div or mod in the insn stream then doing the peephole should
>> be a clear win since that 3rd insn can execute in parallel with the
>> initial divide as opposed to waiting for the one of the first div/mod to
>> clear the exclusive stage of the pipe.
> 
> That is the SMT4 case, the one we do not optimise for.  SMT2 and ST can
> do four in parallel.  This means you can start a div or mod every 2nd
> cycle on average, so it is very unlikely you will ever be limited by
> this on real code.

Power9/Power10 only have 2 fixed-point divide units, and are able to 
issue 2 divides every 9/11 cycles (they aren't fully pipelined), with 
latencies of 12-24/12-25. Not saying that changes the "best case" 
scenario, just pointing out a lot of variables in play.

-Pat
Segher Boessenkool Feb. 27, 2023, 10:30 p.m. UTC | #5 
On Mon, Feb 27, 2023 at 04:03:56PM -0600, Pat Haugen wrote:
> On 2/27/23 2:53 PM, Segher Boessenkool wrote:
> >"Slightly".  It takes 12 cycles for the two in parallel (64-bit, p9),
> >but 17 cycles for the "cheaper" sequence (divd+mulld+subf, 12+5+2).  It
> >is all worse if the units are busy of course, or if there are other
> >problems.
> >
> >>but if you throw in another
> >>independent div or mod in the insn stream then doing the peephole should
> >>be a clear win since that 3rd insn can execute in parallel with the
> >>initial divide as opposed to waiting for the one of the first div/mod to
> >>clear the exclusive stage of the pipe.
> >
> >That is the SMT4 case, the one we do not optimise for.  SMT2 and ST can
> >do four in parallel.  This means you can start a div or mod every 2nd
> >cycle on average, so it is very unlikely you will ever be limited by
> >this on real code.
> 
> Power9/Power10 only have 2 fixed-point divide units, and are able to 
> issue 2 divides every 9/11 cycles (they aren't fully pipelined), with 
> latencies of 12-24/12-25. Not saying that changes the "best case" 
> scenario, just pointing out a lot of variables in play.

The p9 UM says in no uncertain terms there are four integer dividers
(four fixed-point execution pipelines, all four capable of divides).
Is that wrong then?

Let's do actual tests on actual hardware :-)


Segher
diff mbox series

Patch

diff --git a/gcc/config/rs6000/rs6000.md b/gcc/config/rs6000/rs6000.md
index 81bffb04ceb..44f7dd509cb 100644
--- a/gcc/config/rs6000/rs6000.md
+++ b/gcc/config/rs6000/rs6000.md
@@ -3437,9 +3437,9 @@  (define_expand "mod<mode>3"
  ;; In order to enable using a peephole2 for combining div/mod to 
eliminate the
  ;; mod, prefer putting the result of mod into a different register
  (define_insn "*mod<mode>3"
-  [(set (match_operand:GPR 0 "gpc_reg_operand" "=&r")
-        (mod:GPR (match_operand:GPR 1 "gpc_reg_operand" "r")
-		 (match_operand:GPR 2 "gpc_reg_operand" "r")))]
+  [(set (match_operand:GPR 0 "gpc_reg_operand" "=&r,r")
+        (mod:GPR (match_operand:GPR 1 "gpc_reg_operand" "r,r")
+		 (match_operand:GPR 2 "gpc_reg_operand" "r,r")))]
    "TARGET_MODULO"
    "mods<wd> %0,%1,%2"
    [(set_attr "type" "div")
@@ -3447,9 +3447,9 @@  (define_insn "*mod<mode>3"


  (define_insn "umod<mode>3"
-  [(set (match_operand:GPR 0 "gpc_reg_operand" "=&r")
-        (umod:GPR (match_operand:GPR 1 "gpc_reg_operand" "r")
-		  (match_operand:GPR 2 "gpc_reg_operand" "r")))]
+  [(set (match_operand:GPR 0 "gpc_reg_operand" "=&r,r")
+        (umod:GPR (match_operand:GPR 1 "gpc_reg_operand" "r,r")
+		  (match_operand:GPR 2 "gpc_reg_operand" "r,r")))]
    "TARGET_MODULO"
    "modu<wd> %0,%1,%2"
    [(set_attr "type" "div")
diff --git a/gcc/testsuite/gcc.target/powerpc/mod-no_copy.c 
b/gcc/testsuite/gcc.target/powerpc/mod-no_copy.c
new file mode 100644
index 00000000000..91e3003b3fc
--- /dev/null
+++ b/gcc/testsuite/gcc.target/powerpc/mod-no_copy.c
@@ -0,0 +1,17 @@ 
+/* { dg-do compile { target { powerpc*-*-*  } } } */
+/* { dg-require-effective-target powerpc_p9modulo_ok } */
+/* { dg-options "-mdejagnu-cpu=power9 -O2" } */
+
+/* Verify r3 is used as source and target, no copy inserted. */
+
+long foo (long a, long b)
+{
+  return (a % b);
+}
+
+unsigned long foo2 (unsigned long a, unsigned long b)
+{
+  return (a % b);
+}
+
+/* { dg-final { scan-assembler-not {\mmr\M} } } */