| Message ID | 20231121100515.1831979-1-dceara@redhat.com |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | [ovs-dev] ci: cirrus: Fix test ranges. | expand |
| Context | Check | Description |
|---|---|---|
| ovsrobot/apply-robot | success | apply and check: success |
| ovsrobot/github-robot-_Build_and_Test | success | github build: passed |
| ovsrobot/github-robot-_ovn-kubernetes | success | github build: passed |
On Tue, Nov 21, 2023 at 11:05 AM Dumitru Ceara <dceara@redhat.com> wrote: > The CirrusCI configuration file was not updated back when the size of the > test matrix was reduced. > > CirrusCI jobs were complaining of invalid test ranges for a while but > that slipped through the cracks until commit 0224e45a6a10 ("ci: Remove > '--recheck' in CI.") uncovered the failure: > > make distcheck 'CFLAGS= ' -j4 'TESTSUITEFLAGS=-j4 501-1000' RECHECK=no > SKIP_UNSTABLE=yes > [...] > invalid test group: 1000 > > At the same time, make sure --recheck is passed to ARM jobs (on > CirrusCI). These have been always more unstable than other CI jobs. > > Fixes: 937a9b59e262 ("ci, tests: Use parallelization permutations for few > jobs") > Fixes: 0224e45a6a10 ("ci: Remove '--recheck' in CI.") > Signed-off-by: Dumitru Ceara <dceara@redhat.com> > --- > Hi Dumitru, thank you for fixing this. It was indeed silently failing. I have one comment, see down below. > .cirrus.yml | 14 ++++++++------ > 1 file changed, 8 insertions(+), 6 deletions(-) > > diff --git a/.cirrus.yml b/.cirrus.yml > index bd4cd08aaf..ba4b73d73e 100644 > --- a/.cirrus.yml > +++ b/.cirrus.yml > @@ -9,25 +9,27 @@ arm_unit_tests_task: > ARCH: aarch64 > CIRRUS_CLONE_SUBMODULES: true > PATH: ${HOME}/bin:${HOME}/.local/bin:${PATH} > + SKIP_UNSTABLE: no > This won't be propagated correctly. We should do the same thing as we do for GH actions, that is to add the "UNSTABLE" to the last job of the specified group. > + RECHECK: yes > matrix: > - CC: gcc > TESTSUITE: test > - TEST_RANGE: -500 > + TEST_RANGE: -300 > - CC: gcc > TESTSUITE: test > - TEST_RANGE: 501-1000 > + TEST_RANGE: 301-600 > - CC: gcc > TESTSUITE: test > - TEST_RANGE: 1001- > + TEST_RANGE: 601- > - CC: clang > TESTSUITE: test > - TEST_RANGE: -500 > + TEST_RANGE: -300 > - CC: clang > TESTSUITE: test > - TEST_RANGE: 501-1000 > + TEST_RANGE: 301-600 > - CC: clang > TESTSUITE: test > - TEST_RANGE: 1001- > + TEST_RANGE: 601- > > name: ARM64 ${CC} ${TESTSUITE} ${TEST_RANGE} > > -- > 2.39.3 > > Thanks, Ales
On 11/21/23 12:00, Ales Musil wrote: > On Tue, Nov 21, 2023 at 11:05 AM Dumitru Ceara <dceara@redhat.com> wrote: > >> The CirrusCI configuration file was not updated back when the size of the >> test matrix was reduced. >> >> CirrusCI jobs were complaining of invalid test ranges for a while but >> that slipped through the cracks until commit 0224e45a6a10 ("ci: Remove >> '--recheck' in CI.") uncovered the failure: >> >> make distcheck 'CFLAGS= ' -j4 'TESTSUITEFLAGS=-j4 501-1000' RECHECK=no >> SKIP_UNSTABLE=yes >> [...] >> invalid test group: 1000 >> >> At the same time, make sure --recheck is passed to ARM jobs (on >> CirrusCI). These have been always more unstable than other CI jobs. >> >> Fixes: 937a9b59e262 ("ci, tests: Use parallelization permutations for few >> jobs") >> Fixes: 0224e45a6a10 ("ci: Remove '--recheck' in CI.") >> Signed-off-by: Dumitru Ceara <dceara@redhat.com> >> --- >> > > Hi Dumitru, > > thank you for fixing this. It was indeed silently failing. I have one > comment, see down below. > > >> .cirrus.yml | 14 ++++++++------ >> 1 file changed, 8 insertions(+), 6 deletions(-) >> >> diff --git a/.cirrus.yml b/.cirrus.yml >> index bd4cd08aaf..ba4b73d73e 100644 >> --- a/.cirrus.yml >> +++ b/.cirrus.yml >> @@ -9,25 +9,27 @@ arm_unit_tests_task: >> ARCH: aarch64 >> CIRRUS_CLONE_SUBMODULES: true >> PATH: ${HOME}/bin:${HOME}/.local/bin:${PATH} >> + SKIP_UNSTABLE: no >> > > This won't be propagated correctly. We should do the same thing as we do > for GH actions, that is to add the "UNSTABLE" to the last job of the > specified group. > You're right, my bad, thanks for catching this. It was generating: make distcheck 'CFLAGS= ' -j4 'TESTSUITEFLAGS=-j4 301-600' RECHECK=yes SKIP_UNSTABLE=yes which would just skip all tests tagged as "unstable". I'll post v2 soon. > >> + RECHECK: yes >> matrix: >> - CC: gcc >> TESTSUITE: test >> - TEST_RANGE: -500 >> + TEST_RANGE: -300 >> - CC: gcc >> TESTSUITE: test >> - TEST_RANGE: 501-1000 >> + TEST_RANGE: 301-600 >> - CC: gcc >> TESTSUITE: test >> - TEST_RANGE: 1001- >> + TEST_RANGE: 601- >> - CC: clang >> TESTSUITE: test >> - TEST_RANGE: -500 >> + TEST_RANGE: -300 >> - CC: clang >> TESTSUITE: test >> - TEST_RANGE: 501-1000 >> + TEST_RANGE: 301-600 >> - CC: clang >> TESTSUITE: test >> - TEST_RANGE: 1001- >> + TEST_RANGE: 601- >> >> name: ARM64 ${CC} ${TESTSUITE} ${TEST_RANGE} >> >> -- >> 2.39.3 >> >> > Thanks, > Ales Regards, Dumitru
diff --git a/.cirrus.yml b/.cirrus.yml index bd4cd08aaf..ba4b73d73e 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -9,25 +9,27 @@ arm_unit_tests_task: ARCH: aarch64 CIRRUS_CLONE_SUBMODULES: true PATH: ${HOME}/bin:${HOME}/.local/bin:${PATH} + SKIP_UNSTABLE: no + RECHECK: yes matrix: - CC: gcc TESTSUITE: test - TEST_RANGE: -500 + TEST_RANGE: -300 - CC: gcc TESTSUITE: test - TEST_RANGE: 501-1000 + TEST_RANGE: 301-600 - CC: gcc TESTSUITE: test - TEST_RANGE: 1001- + TEST_RANGE: 601- - CC: clang TESTSUITE: test - TEST_RANGE: -500 + TEST_RANGE: -300 - CC: clang TESTSUITE: test - TEST_RANGE: 501-1000 + TEST_RANGE: 301-600 - CC: clang TESTSUITE: test - TEST_RANGE: 1001- + TEST_RANGE: 601- name: ARM64 ${CC} ${TESTSUITE} ${TEST_RANGE}
The CirrusCI configuration file was not updated back when the size of the test matrix was reduced. CirrusCI jobs were complaining of invalid test ranges for a while but that slipped through the cracks until commit 0224e45a6a10 ("ci: Remove '--recheck' in CI.") uncovered the failure: make distcheck 'CFLAGS= ' -j4 'TESTSUITEFLAGS=-j4 501-1000' RECHECK=no SKIP_UNSTABLE=yes [...] invalid test group: 1000 At the same time, make sure --recheck is passed to ARM jobs (on CirrusCI). These have been always more unstable than other CI jobs. Fixes: 937a9b59e262 ("ci, tests: Use parallelization permutations for few jobs") Fixes: 0224e45a6a10 ("ci: Remove '--recheck' in CI.") Signed-off-by: Dumitru Ceara <dceara@redhat.com> --- .cirrus.yml | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-)