[committed] Set num_threads to 50 on 32-bit hppa in two libgomp loop tests

Tested on hppa-unknown-linux-gnu.  Committed to trunk.

Dave
---

Set num_threads to 50 on 32-bit hppa in two libgomp loop tests

We support a maximum of 50 threads on 32-bit hppa.

2024-02-01  John David Anglin  <danglin@gcc.gnu.org>

libgomp/ChangeLog:

	* testsuite/libgomp.c++/loop-3.C: Set num_threads to 50
	on 32-bit hppa.
	* testsuite/libgomp.c/omp-loop03.c: Likewise.

Hi!

On 2024-02-01T19:20:57+0000, John David Anglin <dave@parisc-linux.org> wrote:
> Tested on hppa-unknown-linux-gnu.  Committed to trunk.

> Set num_threads to 50 on 32-bit hppa in two libgomp loop tests
>
> We support a maximum of 50 threads on 32-bit hppa.

What happens if you go higher?  Curious, what/why is that architectural
limit of 50 threads?

I wonder: shouldn't that cap at 50 threads happen inside libgomp,
generally, instead of per test case and user code (!)?  Per my
understanding, OpenMP 'num_threads' specifies a *desired* number of
threads; the implementation may limit that value.


Grüße
 Thomas


> --- a/libgomp/testsuite/libgomp.c++/loop-3.C
> +++ b/libgomp/testsuite/libgomp.c++/loop-3.C
> @@ -1,3 +1,9 @@
> +#if defined(__hppa__) && !defined(__LP64__)
> +#define NUM_THREADS 50
> +#else
> +#define NUM_THREADS 64
> +#endif
> +
>  extern "C" void abort (void);
>  int a;
>  
> @@ -19,7 +25,7 @@ foo ()
>  int
>  main (void)
>  {
> -#pragma omp parallel num_threads (64)
> +#pragma omp parallel num_threads (NUM_THREADS)
>    foo ();
>  
>    return 0;

> --- a/libgomp/testsuite/libgomp.c/omp-loop03.c
> +++ b/libgomp/testsuite/libgomp.c/omp-loop03.c
> @@ -1,3 +1,9 @@
> +#if defined(__hppa__) && !defined(__LP64__)
> +#define NUM_THREADS 50
> +#else
> +#define NUM_THREADS 64
> +#endif
> +
>  extern void abort (void);
>  int a;
>  
> @@ -19,7 +25,7 @@ foo ()
>  int
>  main (void)
>  {
> -#pragma omp parallel num_threads (64)
> +#pragma omp parallel num_threads (NUM_THREADS)
>    foo ();
>  
>    return 0;

On 2024-02-29 6:02 p.m., Thomas Schwinge wrote:
> Hi!
>
> On 2024-02-01T19:20:57+0000, John David Anglin <dave@parisc-linux.org> wrote:
>> Tested on hppa-unknown-linux-gnu.  Committed to trunk.
>> Set num_threads to 50 on 32-bit hppa in two libgomp loop tests
>>
>> We support a maximum of 50 threads on 32-bit hppa.
> What happens if you go higher?  Curious, what/why is that architectural
> limit of 50 threads?
One gets an EAGAIN error at 51.  I don't know why 50 is the architectural limit on hppa-linux.
I had asked Helge previously but didn't get an answer.  As far as I can tell, limit isn't set by glibc.

It seems 64 is supported on all other targets.
>
> I wonder: shouldn't that cap at 50 threads happen inside libgomp,
> generally, instead of per test case and user code (!)?  Per my
> understanding, OpenMP 'num_threads' specifies a *desired* number of
> threads; the implementation may limit that value.
Sounds like a good suggestion.

Dave

Hi all, hi John & Thomas

John David Anglin wrote:
> On 2024-02-29 6:02 p.m., Thomas Schwinge wrote:
>> I wonder: shouldn't that cap at 50 threads happen inside libgomp,
>> generally, instead of per test case and user code (!)?

>> Per my
>> understanding, OpenMP 'num_threads' specifies a *desired* number of
>> threads; the implementation may limit that value.
> Sounds like a good suggestion.

I concur – if the hardware/OS doesn't support more.

* * *

However – for completeness and to correct a statement: While num_threads 
specifies the desired number of threads, 'strict' will turn this into 
error termination if the implementation cannot fulfilled the request.

Namely, "if prescriptiveness is specified as 'strict' and Algorithm 11.1 
would result in a number of threads other than the value of the first 
item of the _nthreads_ list then runtime error termination is performed."

Note that 'strict' for num_threads is new in/since the OpenMP 6.0 draft 
(TR11, I think) and not yet implemented in GCC.

However, I guess that the thread limit also affects 'teams' and nested 
parallelization. And for teams 'num_teams(n)' sets lower = upper value 
to 'n' — Thus, this enforces this number of teams. (While 
'num_teams(m:n)' sets both limits and 'omp_set_num_teams(n)' or 
OMP_NUM_TEAMS=n only set the upper bound).

[As far as I can see, OpenACC always permits an implementation to use 
fewer gangs/workers/vectors if the hardware doesn't support the 
requested number.]

Tobias

On Fri, Mar 01, 2024 at 09:29:01AM +0100, Tobias Burnus wrote:
> John David Anglin wrote:
> > On 2024-02-29 6:02 p.m., Thomas Schwinge wrote:
> > > I wonder: shouldn't that cap at 50 threads happen inside libgomp,
> > > generally, instead of per test case and user code (!)?
> 
> > > Per my
> > > understanding, OpenMP 'num_threads' specifies a *desired* number of
> > > threads; the implementation may limit that value.
> > Sounds like a good suggestion.
> 
> I concur – if the hardware/OS doesn't support more.
> 
> * * *
> 
> However – for completeness and to correct a statement: While num_threads
> specifies the desired number of threads, 'strict' will turn this into error
> termination if the implementation cannot fulfilled the request.
> 
> Namely, "if prescriptiveness is specified as 'strict' and Algorithm 11.1
> would result in a number of threads other than the value of the first item
> of the _nthreads_ list then runtime error termination is performed."
> 
> Note that 'strict' for num_threads is new in/since the OpenMP 6.0 draft
> (TR11, I think) and not yet implemented in GCC.

Also note that if hppa-linux really has such low thread limits, we can't
simply try to add some hack in gomp_resolve_num_threads where it would lower
the result if larger than 50, because if the limit is max 50 threads per
process, just doing nested parallelism and asking for 25 threads in the
outer and 4 in the inner in each will run over that limit, or teams 4
with max 25 threads in parallel inside of it, or user can use pthread_create
in the process as well, etc.

So, if at all possible, the best thing would be to change kernel so that
the number of threads is limited just by available memory for stacks and
user tweakable limit.
E.g. on my ws I have
cat /proc/sys/kernel/threads-max
253865

Isn't this just that you have 50 in there?

	Jakub

On 3/1/24 09:44, Jakub Jelinek wrote:
> On Fri, Mar 01, 2024 at 09:29:01AM +0100, Tobias Burnus wrote:
>> John David Anglin wrote:
>>> On 2024-02-29 6:02 p.m., Thomas Schwinge wrote:
>>>> I wonder: shouldn't that cap at 50 threads happen inside libgomp,
>>>> generally, instead of per test case and user code (!)?
>>
>>>> Per my
>>>> understanding, OpenMP 'num_threads' specifies a *desired* number of
>>>> threads; the implementation may limit that value.
>>> Sounds like a good suggestion.
>>
>> I concur – if the hardware/OS doesn't support more.
>>
>> * * *
>>
>> However – for completeness and to correct a statement: While num_threads
>> specifies the desired number of threads, 'strict' will turn this into error
>> termination if the implementation cannot fulfilled the request.
>>
>> Namely, "if prescriptiveness is specified as 'strict' and Algorithm 11.1
>> would result in a number of threads other than the value of the first item
>> of the _nthreads_ list then runtime error termination is performed."
>>
>> Note that 'strict' for num_threads is new in/since the OpenMP 6.0 draft
>> (TR11, I think) and not yet implemented in GCC.
>
> Also note that if hppa-linux really has such low thread limits, we can't
> simply try to add some hack in gomp_resolve_num_threads where it would lower
> the result if larger than 50, because if the limit is max 50 threads per
> process, just doing nested parallelism and asking for 25 threads in the
> outer and 4 in the inner in each will run over that limit, or teams 4
> with max 25 threads in parallel inside of it, or user can use pthread_create
> in the process as well, etc.
>
> So, if at all possible, the best thing would be to change kernel so that
> the number of threads is limited just by available memory for stacks and
> user tweakable limit.
> E.g. on my ws I have
> cat /proc/sys/kernel/threads-max
> 253865
>
> Isn't this just that you have 50 in there?

On my physical parisc machines I see:

root@panama(4GB RAM):~# cat /proc/sys/kernel/threads-max
39696

root@parisc(8GM RAM):~# cat /proc/sys/kernel/threads-max
63861

So, I wonder where the 50 comes from.
On parisc stacks grow upwards and the initial maximum stack
size is limited, so maybe this influences the 50 ?

Helge

On 2024-03-01 3:44 a.m., Jakub Jelinek wrote:
> Isn't this just that you have 50 in there?
No.  It's okay.

The problem is we run out of memory caused by a "ulimit -s 81920" statement that I had
in .bashrc.  The test pass with default stack allocation.

clone(child_stack=0x3191040, 
flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, 
parent_tid=[1108], tls=0x81918c0, child_tidptr=0x8191468) = 1108
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
mmap2(NULL, 83890176, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = -1 ENOMEM (Cannot allocate memory)

Will revert change to tests.

Dave