| Message ID | 20210924185506.2542588-1-crosa@redhat.com |
|---|---|
| Headers | show |
| Series | Acceptance Tests: use Avocado 91.0 features and other improvements | expand |
Hi, Cleber! What about record/replay tests from 25.06? On 24.09.2021 21:54, Cleber Rosa wrote: > This is a collection of patches for the Acceptance Tests to leverage > some of the features of Avocado 91.0. With the Avocado version bump > by itself, there would be a change in the default "test runner" > implementation that Avocado uses, from the one simply known as > "runner" to the new one called "nrunner". > > Among the changes from one implementation to the other, is the fact > that "nrunner" will run tests in parallel by default. This is *not > yet* enabled by default on "make check-acceptance", but users can > choose to use simply by setting the "AVOCADO_RUNNER" variable, that > is: > > make AVOCADO_RUNNER=nrunner check-acceptance > > If you are curious about the architectural differences of the nrunner, > please refer to: > > https://avocado-framework.readthedocs.io/en/91.0/guides/contributor/chapters/runners.html > > One other noteworthy proposal is a convention to tag tests that either > have known issues, or that touch on QEMU features that have known > issues. By tagging those tests accordingly, they will be > automatically excluded from the regular execution with "make > check-acceptance". > > Finally, some updates to assets locations and some tests refactors and > cleanups. > > Cleber Rosa (16): > Acceptance Tests: bump Avocado requirement to 91.0 > Acceptance Tests: improve check-acceptance description > Acceptance Tests: add mechanism for listing tests > Acceptance Tests: keep track and disable tests with known issues > Acceptance Tests: add standard clean up at test tearDown() > Acceptance Tests: use extract from package from avocado.utils > Acceptance Tests: workaround expired mipsdistros.mips.com HTTPS cert > acceptance/tests/vnc.py: use explicit syntax for enabling passwords > tests/acceptance/boot_xen.py: merge base classes > tests/acceptance/boot_xen.py: unify tags > tests/acceptance/boot_xen.py: fetch kernel during test setUp() > tests/acceptance/boot_xen.py: removed unused import > tests/acceptance/boot_xen.py: use class attribute > tests/acceptance/ppc_prep_40p.py: NetBSD 7.1.2 location update > tests/acceptance/ppc_prep_40p.py: clean up unused import > tests/acceptance/ppc_prep_40p.py: unify tags > > docs/devel/testing.rst | 40 ++++++++++ > tests/Makefile.include | 15 +++- > tests/acceptance/avocado_qemu/__init__.py | 1 + > tests/acceptance/boot_linux_console.py | 93 +++++++++-------------- > tests/acceptance/boot_xen.py | 54 ++++--------- > tests/acceptance/machine_rx_gdbsim.py | 3 + > tests/acceptance/ppc_prep_40p.py | 17 ++--- > tests/acceptance/replay_kernel.py | 18 ++--- > tests/acceptance/tcg_plugins.py | 2 +- > tests/acceptance/vnc.py | 2 +- > tests/requirements.txt | 2 +- > 11 files changed, 128 insertions(+), 119 deletions(-) >
On 9/24/21 20:54, Cleber Rosa wrote: > This is a collection of patches for the Acceptance Tests to leverage > some of the features of Avocado 91.0. With the Avocado version bump > by itself, there would be a change in the default "test runner" > implementation that Avocado uses, from the one simply known as > "runner" to the new one called "nrunner". > > Among the changes from one implementation to the other, is the fact > that "nrunner" will run tests in parallel by default. This is *not > yet* enabled by default on "make check-acceptance", but users can > choose to use simply by setting the "AVOCADO_RUNNER" variable, that > is: > > make AVOCADO_RUNNER=nrunner check-acceptance > > If you are curious about the architectural differences of the nrunner, > please refer to: > > https://avocado-framework.readthedocs.io/en/91.0/guides/contributor/chapters/runners.html > > One other noteworthy proposal is a convention to tag tests that either > have known issues, or that touch on QEMU features that have known > issues. By tagging those tests accordingly, they will be > automatically excluded from the regular execution with "make > check-acceptance". > > Finally, some updates to assets locations and some tests refactors and > cleanups. > > Cleber Rosa (16): > Acceptance Tests: bump Avocado requirement to 91.0 > Acceptance Tests: improve check-acceptance description > Acceptance Tests: add mechanism for listing tests > Acceptance Tests: keep track and disable tests with known issues > Acceptance Tests: add standard clean up at test tearDown() > Acceptance Tests: use extract from package from avocado.utils > Acceptance Tests: workaround expired mipsdistros.mips.com HTTPS cert > acceptance/tests/vnc.py: use explicit syntax for enabling passwords > tests/acceptance/boot_xen.py: merge base classes > tests/acceptance/boot_xen.py: unify tags > tests/acceptance/boot_xen.py: fetch kernel during test setUp() > tests/acceptance/boot_xen.py: removed unused import > tests/acceptance/boot_xen.py: use class attribute > tests/acceptance/ppc_prep_40p.py: NetBSD 7.1.2 location update > tests/acceptance/ppc_prep_40p.py: clean up unused import > tests/acceptance/ppc_prep_40p.py: unify tags Thanks. Trivial patches 2, 8, 12, 14, 15 applied to my integration-testing tree.