| Message ID | 20211103160128.1613127-1-numans@ovn.org |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | [ovs-dev] ci: Don't run unit tests for system-test job. | expand |
| Context | Check | Description |
|---|---|---|
| ovsrobot/apply-robot | success | apply and check: success |
| ovsrobot/github-robot-_Build_and_Test | success | github build: passed |
| ovsrobot/github-robot-_ovn-kubernetes | success | github build: passed |
On 11/3/21 17:01, numans@ovn.org wrote: > From: Numan Siddique <numans@ovn.org> > > For the system-test matrix, running unit tests is not > required and if a unit test fails, then system tests > are not run. Also test matrix covers unit tests anyway. Hmm. I don't think that is correct. IIUC, the only remaining gcc unit test run is with jemalloc. The one within a system-test job is the only normal test run we have. Best regards, Ilya Maximets.
On Wed, Nov 3, 2021 at 12:09 PM Ilya Maximets <i.maximets@ovn.org> wrote: > > On 11/3/21 17:01, numans@ovn.org wrote: > > From: Numan Siddique <numans@ovn.org> > > > > For the system-test matrix, running unit tests is not > > required and if a unit test fails, then system tests > > are not run. Also test matrix covers unit tests anyway. > > Hmm. I don't think that is correct. > IIUC, the only remaining gcc unit test run is with jemalloc. > The one within a system-test job is the only normal test run > we have. Thanks for pointing this out. I missed it. I'll add another test matrix to run unit tests with gcc. The main reason for this patch is to run system tests even if unit tests fail (either due to a flake or due to genuine failure). Thanks Numan > > Best regards, Ilya Maximets. > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev >
On 11/3/21 5:39 PM, Numan Siddique wrote: > On Wed, Nov 3, 2021 at 12:09 PM Ilya Maximets <i.maximets@ovn.org> wrote: >> >> On 11/3/21 17:01, numans@ovn.org wrote: >>> From: Numan Siddique <numans@ovn.org> >>> >>> For the system-test matrix, running unit tests is not >>> required and if a unit test fails, then system tests >>> are not run. Also test matrix covers unit tests anyway. >> >> Hmm. I don't think that is correct. >> IIUC, the only remaining gcc unit test run is with jemalloc. >> The one within a system-test job is the only normal test run >> we have. > > Thanks for pointing this out. I missed it. > > I'll add another test matrix to run unit tests with gcc. > The main reason for this patch is to run system tests even if unit > tests fail (either due to a flake or due to genuine failure). +1 for this. In the patch that originally enabled system tests in CI the goal was to save some CI time/resources but indeed the flakiness of the unit tests makes it more reliable to have a separate test matrix entry for system tests. Regards, Dumitru
diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index 76a2ff459f..b0975685d4 100755 --- a/.ci/linux-build.sh +++ b/.ci/linux-build.sh @@ -49,17 +49,6 @@ else fi if [ "$TESTSUITE" ]; then - # 'distcheck' will reconfigure with required options. - # Now we only need to prepare the Makefile without sparse-wrapped CC. - configure_ovn - - export DISTCHECK_CONFIGURE_FLAGS="$OPTS" - if ! make distcheck -j4 TESTSUITEFLAGS="-j4" RECHECK=yes; then - # testsuite.log is necessary for debugging. - cat */_build/sub/tests/testsuite.log - exit 1 - fi - if [ "$TESTSUITE" = "system-test" ]; then # Reconfigure build with required OPTS, rebuild and run system tests. configure_ovn $OPTS @@ -69,6 +58,17 @@ if [ "$TESTSUITE" ]; then cat tests/system-kmod-testsuite.log exit 1 fi + else + # 'distcheck' will reconfigure with required options. + # Now we only need to prepare the Makefile without sparse-wrapped CC. + configure_ovn + + export DISTCHECK_CONFIGURE_FLAGS="$OPTS" + if ! make distcheck -j4 TESTSUITEFLAGS="-j4" RECHECK=yes; then + # testsuite.log is necessary for debugging. + cat */_build/sub/tests/testsuite.log + exit 1 + fi fi else configure_ovn $OPTS