mbox series

[00/13] Bump Avocado to 103.0 LTS and update tests for compatibility and new features

Message ID 20240726134438.14720-1-crosa@redhat.com
Headers show
Series Bump Avocado to 103.0 LTS and update tests for compatibility and new features | expand

Message

Cleber Rosa July 26, 2024, 1:44 p.m. UTC
This is a *long* overdue update of the Avocado version used in QEMU.
It comes a time where the role of the runner and the libraries are
being discussed and questioned.

These exact commits have been staging on my side for over 30 days now,
and I was exceeding what I should in terms of testing before posting.
I apologize for the miscalculation.

Nevertheless, as pointed out, on the ML, these changes are needed NOW.

Some examples of runs in the CI can be seen below:

* Serial with 103.0 LTS (https://gitlab.com/cleber.gnu/qemu/-/jobs/7074346143#L220):
   RESULTS    : PASS 46 | ERROR 0 | FAIL 0 | SKIP 2 | WARN 0 | INTERRUPT 0 | CANCEL 0
   JOB TIME   : 432.63 s

* Parallel with 103.0 LTS (https://gitlab.com/cleber.gnu/qemu/-/jobs/7085879478#L222)
   RESULTS    : PASS 46 | ERROR 0 | FAIL 0 | SKIP 2 | WARN 0 | INTERRUPT 0 | CANCEL 0
   JOB TIME   : 148.99 s

Cleber Rosa (13):
  tests/avocado: mips: fallback to HTTP given certificate expiration
  tests/avocado: mips: add hint for fetchasset plugin
  tests/avocado/intel_iommu.py: increase timeout
  tests/avocado: add cdrom permission related tests
  tests/avocado: machine aarch64: standardize location and RO access
  tests/avocado: use more distinct names for assets
  tests/avocado/kvm_xen_guest.py: cope with asset RW requirements
  testa/avocado: test_arm_emcraft_sf2: handle RW requirements for asset
  tests/avocado/boot_xen.py: fetch kernel during test setUp()
  tests/avocado/tuxrun_baselines.py: use Avocado's zstd support
  tests/avocado/machine_aarch64_sbsaref.py: allow for rw usage of image
  Bump avocado to 103.0
  Avocado tests: allow for parallel execution of tests

 docs/devel/testing.rst                   | 12 +++++++
 pythondeps.toml                          |  2 +-
 tests/Makefile.include                   |  6 +++-
 tests/avocado/boot_linux_console.py      | 24 ++++++++------
 tests/avocado/boot_xen.py                | 13 ++++----
 tests/avocado/cdrom.py                   | 41 ++++++++++++++++++++++++
 tests/avocado/intel_iommu.py             |  2 ++
 tests/avocado/kvm_xen_guest.py           | 30 +++++++++++------
 tests/avocado/machine_aarch64_sbsaref.py | 11 +++++--
 tests/avocado/machine_aarch64_virt.py    | 14 ++++----
 tests/avocado/netdev-ethtool.py          |  3 +-
 tests/avocado/tuxrun_baselines.py        | 16 ++++-----
 12 files changed, 125 insertions(+), 49 deletions(-)
 create mode 100644 tests/avocado/cdrom.py

Comments

Akihiko Odaki July 28, 2024, 3:27 p.m. UTC | #1
On 2024/07/26 22:44, Cleber Rosa wrote:
> This is a *long* overdue update of the Avocado version used in QEMU.
> It comes a time where the role of the runner and the libraries are
> being discussed and questioned.
> 
> These exact commits have been staging on my side for over 30 days now,
> and I was exceeding what I should in terms of testing before posting.
> I apologize for the miscalculation.
> 
> Nevertheless, as pointed out, on the ML, these changes are needed NOW.
> 
> Some examples of runs in the CI can be seen below:
> 
> * Serial with 103.0 LTS (https://gitlab.com/cleber.gnu/qemu/-/jobs/7074346143#L220):
>     RESULTS    : PASS 46 | ERROR 0 | FAIL 0 | SKIP 2 | WARN 0 | INTERRUPT 0 | CANCEL 0
>     JOB TIME   : 432.63 s
> 
> * Parallel with 103.0 LTS (https://gitlab.com/cleber.gnu/qemu/-/jobs/7085879478#L222)
>     RESULTS    : PASS 46 | ERROR 0 | FAIL 0 | SKIP 2 | WARN 0 | INTERRUPT 0 | CANCEL 0
>     JOB TIME   : 148.99 s
> 
> Cleber Rosa (13):
>    tests/avocado: mips: fallback to HTTP given certificate expiration
>    tests/avocado: mips: add hint for fetchasset plugin
>    tests/avocado/intel_iommu.py: increase timeout
>    tests/avocado: add cdrom permission related tests
>    tests/avocado: machine aarch64: standardize location and RO access
>    tests/avocado: use more distinct names for assets
>    tests/avocado/kvm_xen_guest.py: cope with asset RW requirements
>    testa/avocado: test_arm_emcraft_sf2: handle RW requirements for asset
>    tests/avocado/boot_xen.py: fetch kernel during test setUp()
>    tests/avocado/tuxrun_baselines.py: use Avocado's zstd support
>    tests/avocado/machine_aarch64_sbsaref.py: allow for rw usage of image
>    Bump avocado to 103.0
>    Avocado tests: allow for parallel execution of tests
> 
>   docs/devel/testing.rst                   | 12 +++++++
>   pythondeps.toml                          |  2 +-
>   tests/Makefile.include                   |  6 +++-
>   tests/avocado/boot_linux_console.py      | 24 ++++++++------
>   tests/avocado/boot_xen.py                | 13 ++++----
>   tests/avocado/cdrom.py                   | 41 ++++++++++++++++++++++++
>   tests/avocado/intel_iommu.py             |  2 ++
>   tests/avocado/kvm_xen_guest.py           | 30 +++++++++++------
>   tests/avocado/machine_aarch64_sbsaref.py | 11 +++++--
>   tests/avocado/machine_aarch64_virt.py    | 14 ++++----
>   tests/avocado/netdev-ethtool.py          |  3 +-
>   tests/avocado/tuxrun_baselines.py        | 16 ++++-----
>   12 files changed, 125 insertions(+), 49 deletions(-)
>   create mode 100644 tests/avocado/cdrom.py
> 

Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Philippe Mathieu-Daudé July 29, 2024, 2:40 p.m. UTC | #2
On 26/7/24 15:44, Cleber Rosa wrote:

> Cleber Rosa (13):
>    tests/avocado: mips: fallback to HTTP given certificate expiration
>    tests/avocado: mips: add hint for fetchasset plugin
>    tests/avocado/intel_iommu.py: increase timeout
>    tests/avocado: add cdrom permission related tests
>    tests/avocado: machine aarch64: standardize location and RO access
>    tests/avocado: use more distinct names for assets
>    tests/avocado/kvm_xen_guest.py: cope with asset RW requirements
>    testa/avocado: test_arm_emcraft_sf2: handle RW requirements for asset
>    tests/avocado/boot_xen.py: fetch kernel during test setUp()
>    tests/avocado/tuxrun_baselines.py: use Avocado's zstd support
>    tests/avocado/machine_aarch64_sbsaref.py: allow for rw usage of image
>    Bump avocado to 103.0
>    Avocado tests: allow for parallel execution of tests

Queuing patches #1, #2 and #8. Unfortunately #10 depends on Avocado
101.0 so can't be merged at this point.