| Message ID | 20230610040528.1058420420@pchp3.se.axis.com |
|---|---|
| State | New |
| Headers | show |
| Series | (Re: Splitting up 27_io/basic_istream/ignore/wchar_t/94749.cc (takes too long)) | expand |
On Sat, 10 Jun 2023, 06:18 Hans-Peter Nilsson via Libstdc++, < libstdc++@gcc.gnu.org> wrote: > Thank you for your consideration. (Or is that phrase only used > negatively?) > > > From: Jonathan Wakely <jwakely@redhat.com> > > Date: Fri, 9 Jun 2023 21:40:15 +0100 > > > test01, test02, test03 and test04 should run almost instantly. On my > system > > they take about 5 microseconds each. So I don't think splitting those up > > will help. > > Right. > > > I thought it would help to avoid re-allocating the buffer and zeroing it > > again. If we reuse the same buffer, then we just have to loop until we > > overflow the 32-bit counter. That would make the whole test run much > > faster, which would reduce the total time for a testsuite run. Splitting > > the file up into smaller files would not decrease the total time, only > > decrease the time for that single test so it doesn't time out. > > > > I've attached a patch that does that. I makes very little difference for > > me, probably because allocating zero-filled pages isn't actually > expensive > > on linux. Maybe it will make a differene for your simulator though? > > Nope, just some five seconds down (from about 10min 21s). > Bah, worth a try :) > > You could also try reducing the size of the buffer: > > +#ifdef SIMULATOR_TEST > > + static const streamsize bufsz = 16 << limits::digits10; > > +#else > > static const streamsize bufsz = 2048 << limits::digits10; > > +#endif > > Was that supposed to be with or without the patch? Anyway; > both: 606s. Only smaller bufsz: 614s. (All numbers subject > to usual system jitter.) > > > test06 is the really slow part, that takes 10+ seconds for me. But that > > entire function should already be skipped for simulators. > > Yep, we may have been here before... I certainly get a > deja-vu feeling here, but visiting old email conversations > of ours, it seems I easily conflate several similar ones. > I see that here, test06 was always #ifndef SIMULATOR_TEST. > > > We can probably skip test05 for simulators too, none of the code it tests > > is platform-specific, so as long as it's being tested on x86 we don't > > really need to test it on cris-elf too. > > Thanks. Let's do that, then. The similar s/wchar_t/char/ > test clocks in at "only" 3m30s, but I suggest treating it > the same, if nothing else than for symmetry. > > Ok as below? > OK for trunk, and all branches you care about. > -- >8 -- > Subject: [PATCH] testsuite: Cut down 27_io/basic_istream/.../94749.cc for > simulators > > The test wchar_t/94749.cc can take about 10 minutes on some > simulator/host combinations with char/94749.cc at a third of > that time. The cause is test05 which is quite heavy and > includes wrapping a 32-bit counter. Run it only for native > setups. > > * testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc (main) > [! SIMULATOR_TEST]: Also exclude running test05. > * testsuite/27_io/basic_istream/ignore/char/94749.cc: Ditto. > --- > libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc | 2 +- > .../testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc | 2 +- > 2 files changed, 2 insertions(+), 2 deletions(-) > > diff --git > a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc > b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc > index 6416863983b7..9160995c05ec 100644 > --- a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc > +++ b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc > @@ -221,8 +221,8 @@ main() > test02(); > test03(); > test04(); > - test05(); > #ifndef SIMULATOR_TEST > + test05(); > test06(); > #endif > } > diff --git > a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc > b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc > index 65e0a326c109..a5b9eb71a389 100644 > --- a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc > +++ b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc > @@ -221,8 +221,8 @@ main() > test02(); > test03(); > test04(); > - test05(); > #ifndef SIMULATOR_TEST > + test05(); > test06(); > #endif > } > -- > 2.30.2 > >
Hi, I have noticed that in gcc-13, test05 (in the 94749.cc testcase) is still enabled for simulators, and I have noticed that because of test05, the 27_io/basic_istream/ignore/char/94749.cc execution test is not terminating on our simulator for armv8.1-m.main+mve, even after 3 hours. The execution test was passing before this commit : https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=e30211cb0b3a2b88959e9bc40626a17461de52de Could you please provide some hints or ideas as to what might be causing this regression? I imagine that the issue could be with the simulator or with code-gen. However, could this also highlight a different issue in test05? Is this testing a commonly used feature or area of the compiler? And would it be worth re-including it for simulators? Kind regards, Vasee
> From: Vaseeharan Vinayagamoorthy <Vaseeharan.Vinayagamoorthy@arm.com> > Date: Sat, 22 Jun 2024 01:38:09 +0000 Sorry for the late reply. I sort of hoped somebody else would chime in. Maybe the issue has resolved itself in the meantime? > Hi, > > I have noticed that in gcc-13, test05 (in the 94749.cc > testcase) is still enabled for simulators, and I have > noticed that because of test05, the > 27_io/basic_istream/ignore/char/94749.cc execution test is > not terminating on our simulator for armv8.1-m.main+mve, > even after 3 hours. > > The execution test was passing before this commit : > https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=e30211cb0b3a2b88959e9bc40626a17461de52de > > Could you please provide some hints or ideas as to what > might be causing this regression? No, sorry. If I were you and in this situation: IIUC with a noticeable codegen regression around a specific commit (not "just" say a percent pushing it over a timeout), I'd analyze it with regards to the actual code regression around that commit. Though, that's just the usual leg-work when this kind of regression happens: I have no insight specific to this test. I see no easy way around that hard work here. > I imagine that the issue could be with the simulator or > with code-gen. However, could this also highlight a > different issue in test05? It could, but I'm guessing that commit just caused a codegen regression, perhaps even generating incorrect code to the effect of an infinite loop. Or is somehow it that usual ARM caveat: default unsigned char? > Is this testing a commonly used > feature or area of the compiler? All I know is that the intent is specific to functionality in libstdc++-v3. > And would it be worth > re-including it for simulators? IMHO: only if you somehow make it ARM-specific. It'd be bad practice to "re-enable" it for all simulator targets only because it exposes an uninvestigated issue for one specific configuration, and a timeout at that. Alternatively (after analysis), the SOP is to put a derived minimal testcase in the *generic* parts of the test-suite (C or C++, as a runtime test) unless the compiled code really only runs on an ARM, in which case it goes in gcc.target/arm or g++.target/arm. HTH. brgds, H-P > > Kind regards, > Vasee > > ________________________________________ > From: Libstdc++ <libstdc++-bounces+vvinayag=arm.com@gcc.gnu.org> on behalf of Jonathan Wakely via Libstdc++ <libstdc++@gcc.gnu.org> > Sent: 10 June 2023 08:12 > To: Hans-Peter Nilsson > Cc: Jonathan Wakely; libstdc++; gcc-patches > Subject: Re: [PATCH] (Re: Splitting up 27_io/basic_istream/ignore/wchar_t/94749.cc (takes too long)) > > On Sat, 10 Jun 2023, 06:18 Hans-Peter Nilsson via Libstdc++, < > libstdc++@gcc.gnu.org> wrote: > > > Thank you for your consideration. (Or is that phrase only used > > negatively?) > > > > > From: Jonathan Wakely <jwakely@redhat.com> > > > Date: Fri, 9 Jun 2023 21:40:15 +0100 > > > > > test01, test02, test03 and test04 should run almost instantly. On my > > system > > > they take about 5 microseconds each. So I don't think splitting those up > > > will help. > > > > Right. > > > > > I thought it would help to avoid re-allocating the buffer and zeroing it > > > again. If we reuse the same buffer, then we just have to loop until we > > > overflow the 32-bit counter. That would make the whole test run much > > > faster, which would reduce the total time for a testsuite run. Splitting > > > the file up into smaller files would not decrease the total time, only > > > decrease the time for that single test so it doesn't time out. > > > > > > I've attached a patch that does that. I makes very little difference for > > > me, probably because allocating zero-filled pages isn't actually > > expensive > > > on linux. Maybe it will make a differene for your simulator though? > > > > Nope, just some five seconds down (from about 10min 21s). > > > > Bah, worth a try :) > > > > > You could also try reducing the size of the buffer: > > > +#ifdef SIMULATOR_TEST > > > + static const streamsize bufsz = 16 << limits::digits10; > > > +#else > > > static const streamsize bufsz = 2048 << limits::digits10; > > > +#endif > > > > Was that supposed to be with or without the patch? Anyway; > > both: 606s. Only smaller bufsz: 614s. (All numbers subject > > to usual system jitter.) > > > > > test06 is the really slow part, that takes 10+ seconds for me. But that > > > entire function should already be skipped for simulators. > > > > Yep, we may have been here before... I certainly get a > > deja-vu feeling here, but visiting old email conversations > > of ours, it seems I easily conflate several similar ones. > > I see that here, test06 was always #ifndef SIMULATOR_TEST. > > > > > We can probably skip test05 for simulators too, none of the code it tests > > > is platform-specific, so as long as it's being tested on x86 we don't > > > really need to test it on cris-elf too. > > > > Thanks. Let's do that, then. The similar s/wchar_t/char/ > > test clocks in at "only" 3m30s, but I suggest treating it > > the same, if nothing else than for symmetry. > > > > Ok as below? > > > > OK for trunk, and all branches you care about. > > > > > -- >8 -- > > Subject: [PATCH] testsuite: Cut down 27_io/basic_istream/.../94749.cc for > > simulators > > > > The test wchar_t/94749.cc can take about 10 minutes on some > > simulator/host combinations with char/94749.cc at a third of > > that time. The cause is test05 which is quite heavy and > > includes wrapping a 32-bit counter. Run it only for native > > setups. > > > > * testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc (main) > > [! SIMULATOR_TEST]: Also exclude running test05. > > * testsuite/27_io/basic_istream/ignore/char/94749.cc: Ditto. > > --- > > libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc | 2 +- > > .../testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc | 2 +- > > 2 files changed, 2 insertions(+), 2 deletions(-) > > > > diff --git > > a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc > > b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc > > index 6416863983b7..9160995c05ec 100644 > > --- a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc > > +++ b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc > > @@ -221,8 +221,8 @@ main() > > test02(); > > test03(); > > test04(); > > - test05(); > > #ifndef SIMULATOR_TEST > > + test05(); > > test06(); > > #endif > > } > > diff --git > > a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc > > b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc > > index 65e0a326c109..a5b9eb71a389 100644 > > --- a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc > > +++ b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc > > @@ -221,8 +221,8 @@ main() > > test02(); > > test03(); > > test04(); > > - test05(); > > #ifndef SIMULATOR_TEST > > + test05(); > > test06(); > > #endif > > } > > -- > > 2.30.2 > > > > >
On Wed, 14 Aug 2024 at 15:47, Hans-Peter Nilsson <hp@axis.com> wrote: > > > From: Vaseeharan Vinayagamoorthy <Vaseeharan.Vinayagamoorthy@arm.com> > > Date: Sat, 22 Jun 2024 01:38:09 +0000 > > Sorry for the late reply. I sort of hoped somebody else > would chime in. Maybe the issue has resolved itself in the > meantime? > > > Hi, > > > > I have noticed that in gcc-13, test05 (in the 94749.cc > > testcase) is still enabled for simulators, and I have > > noticed that because of test05, the > > 27_io/basic_istream/ignore/char/94749.cc execution test is > > not terminating on our simulator for armv8.1-m.main+mve, > > even after 3 hours. > > > > The execution test was passing before this commit : > > https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=e30211cb0b3a2b88959e9bc40626a17461de52de > > > > Could you please provide some hints or ideas as to what > > might be causing this regression? > > No, sorry. If I were you and in this situation: IIUC with a > noticeable codegen regression around a specific commit (not > "just" say a percent pushing it over a timeout), I'd analyze > it with regards to the actual code regression around that > commit. Though, that's just the usual leg-work when this > kind of regression happens: I have no insight specific to > this test. I see no easy way around that hard work here. > > > I imagine that the issue could be with the simulator or > > with code-gen. However, could this also highlight a > > different issue in test05? > > It could, but I'm guessing that commit just caused a codegen > regression, perhaps even generating incorrect code to the > effect of an infinite loop. Or is somehow it that usual ARM > caveat: default unsigned char? > > > Is this testing a commonly used > > feature or area of the compiler? > > All I know is that the intent is specific to functionality > in libstdc++-v3. Right, it should not be testing *anything* in the compiler, beyond relying on "the compiler can compile C++ code correctly". > > > And would it be worth > > re-including it for simulators? > > IMHO: only if you somehow make it ARM-specific. It'd be bad > practice to "re-enable" it for all simulator targets only > because it exposes an uninvestigated issue for one specific > configuration, and a timeout at that. > > Alternatively (after analysis), the SOP is to put a derived > minimal testcase in the *generic* parts of the test-suite (C > or C++, as a runtime test) unless the compiled code really > only runs on an ARM, in which case it goes in gcc.target/arm > or g++.target/arm. > > HTH. > > brgds, H-P > > > > > > Kind regards, > > Vasee > > > > ________________________________________ > > From: Libstdc++ <libstdc++-bounces+vvinayag=arm.com@gcc.gnu.org> on behalf of Jonathan Wakely via Libstdc++ <libstdc++@gcc.gnu.org> > > Sent: 10 June 2023 08:12 > > To: Hans-Peter Nilsson > > Cc: Jonathan Wakely; libstdc++; gcc-patches > > Subject: Re: [PATCH] (Re: Splitting up 27_io/basic_istream/ignore/wchar_t/94749.cc (takes too long)) > > > > On Sat, 10 Jun 2023, 06:18 Hans-Peter Nilsson via Libstdc++, < > > libstdc++@gcc.gnu.org> wrote: > > > > > Thank you for your consideration. (Or is that phrase only used > > > negatively?) > > > > > > > From: Jonathan Wakely <jwakely@redhat.com> > > > > Date: Fri, 9 Jun 2023 21:40:15 +0100 > > > > > > > test01, test02, test03 and test04 should run almost instantly. On my > > > system > > > > they take about 5 microseconds each. So I don't think splitting those up > > > > will help. > > > > > > Right. > > > > > > > I thought it would help to avoid re-allocating the buffer and zeroing it > > > > again. If we reuse the same buffer, then we just have to loop until we > > > > overflow the 32-bit counter. That would make the whole test run much > > > > faster, which would reduce the total time for a testsuite run. Splitting > > > > the file up into smaller files would not decrease the total time, only > > > > decrease the time for that single test so it doesn't time out. > > > > > > > > I've attached a patch that does that. I makes very little difference for > > > > me, probably because allocating zero-filled pages isn't actually > > > expensive > > > > on linux. Maybe it will make a differene for your simulator though? > > > > > > Nope, just some five seconds down (from about 10min 21s). > > > > > > > Bah, worth a try :) > > > > > > > > You could also try reducing the size of the buffer: > > > > +#ifdef SIMULATOR_TEST > > > > + static const streamsize bufsz = 16 << limits::digits10; > > > > +#else > > > > static const streamsize bufsz = 2048 << limits::digits10; > > > > +#endif > > > > > > Was that supposed to be with or without the patch? Anyway; > > > both: 606s. Only smaller bufsz: 614s. (All numbers subject > > > to usual system jitter.) > > > > > > > test06 is the really slow part, that takes 10+ seconds for me. But that > > > > entire function should already be skipped for simulators. > > > > > > Yep, we may have been here before... I certainly get a > > > deja-vu feeling here, but visiting old email conversations > > > of ours, it seems I easily conflate several similar ones. > > > I see that here, test06 was always #ifndef SIMULATOR_TEST. > > > > > > > We can probably skip test05 for simulators too, none of the code it tests > > > > is platform-specific, so as long as it's being tested on x86 we don't > > > > really need to test it on cris-elf too. > > > > > > Thanks. Let's do that, then. The similar s/wchar_t/char/ > > > test clocks in at "only" 3m30s, but I suggest treating it > > > the same, if nothing else than for symmetry. > > > > > > Ok as below? > > > > > > > OK for trunk, and all branches you care about. > > > > > > > > > -- >8 -- > > > Subject: [PATCH] testsuite: Cut down 27_io/basic_istream/.../94749.cc for > > > simulators > > > > > > The test wchar_t/94749.cc can take about 10 minutes on some > > > simulator/host combinations with char/94749.cc at a third of > > > that time. The cause is test05 which is quite heavy and > > > includes wrapping a 32-bit counter. Run it only for native > > > setups. > > > > > > * testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc (main) > > > [! SIMULATOR_TEST]: Also exclude running test05. > > > * testsuite/27_io/basic_istream/ignore/char/94749.cc: Ditto. > > > --- > > > libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc | 2 +- > > > .../testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc | 2 +- > > > 2 files changed, 2 insertions(+), 2 deletions(-) > > > > > > diff --git > > > a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc > > > b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc > > > index 6416863983b7..9160995c05ec 100644 > > > --- a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc > > > +++ b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc > > > @@ -221,8 +221,8 @@ main() > > > test02(); > > > test03(); > > > test04(); > > > - test05(); > > > #ifndef SIMULATOR_TEST > > > + test05(); > > > test06(); > > > #endif > > > } > > > diff --git > > > a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc > > > b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc > > > index 65e0a326c109..a5b9eb71a389 100644 > > > --- a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc > > > +++ b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc > > > @@ -221,8 +221,8 @@ main() > > > test02(); > > > test03(); > > > test04(); > > > - test05(); > > > #ifndef SIMULATOR_TEST > > > + test05(); > > > test06(); > > > #endif > > > } > > > -- > > > 2.30.2 > > > > > > > > >
diff --git a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc index 6416863983b7..9160995c05ec 100644 --- a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc +++ b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc @@ -221,8 +221,8 @@ main() test02(); test03(); test04(); - test05(); #ifndef SIMULATOR_TEST + test05(); test06(); #endif } diff --git a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc index 65e0a326c109..a5b9eb71a389 100644 --- a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc +++ b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc @@ -221,8 +221,8 @@ main() test02(); test03(); test04(); - test05(); #ifndef SIMULATOR_TEST + test05(); test06(); #endif }