From patchwork Wed Sep 4 10:39:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 1980637 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Yhqc5SaN; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WzK2k0krkz1yhr for ; Wed, 4 Sep 2024 20:47:58 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slnRX-0005Uj-PQ; Wed, 04 Sep 2024 06:40:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slnRU-0005Bo-HZ for qemu-devel@nongnu.org; Wed, 04 Sep 2024 06:40:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slnRR-0000Nu-PB for qemu-devel@nongnu.org; Wed, 04 Sep 2024 06:40:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725446449; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Nt9AzICbk4gZxdVkFY9UjvkgXQM8tAMN6F8SghD1MO8=; b=Yhqc5SaNav93nUikT2fvZKJDXwUoZ7mRf3JEKhSBhq8AkI7S+vg6Mx0AdqSveahuGYPdcY uqis6w/8VTXYXOZsYKq1qvzZvBcWtOEq3sV3E53olNN3WxvdlQ4/dVfLj0+JE0pIf8SuKP eRqbDMtF3tgWMEpMgUaVVIGMEQY51/A= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-360--KiR017JPT-okYgTX7bd-Q-1; Wed, 04 Sep 2024 06:40:47 -0400 X-MC-Unique: -KiR017JPT-okYgTX7bd-Q-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0C00A1955F0D; Wed, 4 Sep 2024 10:40:46 +0000 (UTC) Received: from thuth-p1g4.redhat.com (unknown [10.39.194.48]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D8972195605A; Wed, 4 Sep 2024 10:40:43 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org Cc: Richard Henderson , Stefan Hajnoczi , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?q?=C3=A9?= Subject: [PULL 28/42] tests/functional: Convert the acpi-bits test into a standalone test Date: Wed, 4 Sep 2024 12:39:03 +0200 Message-ID: <20240904103923.451847-29-thuth@redhat.com> In-Reply-To: <20240904103923.451847-1-thuth@redhat.com> References: <20240904103923.451847-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 Received-SPF: pass client-ip=170.10.133.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 11 X-Spam_score: 1.1 X-Spam_bar: + X-Spam_report: (1.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.142, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Mostly a straight-forward conversion. Looks like we can simply drop the avocado datadrainer stuff when not using the avocado framework anymore. Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Message-ID: <20240830133841.142644-30-thuth@redhat.com> Signed-off-by: Thomas Huth --- MAINTAINERS | 4 +- docs/devel/acpi-bits.rst | 62 ++++++-------- .../acpi-bits/bits-config/bits-cfg.txt | 0 .../acpi-bits/bits-tests/smbios.py2 | 0 .../acpi-bits/bits-tests/smilatency.py2 | 0 .../acpi-bits/bits-tests/testacpi.py2 | 0 .../acpi-bits/bits-tests/testcpuid.py2 | 0 tests/functional/meson.build | 2 + .../test_acpi_bits.py} | 83 ++++++++++--------- 9 files changed, 71 insertions(+), 80 deletions(-) rename tests/{avocado => functional}/acpi-bits/bits-config/bits-cfg.txt (100%) rename tests/{avocado => functional}/acpi-bits/bits-tests/smbios.py2 (100%) rename tests/{avocado => functional}/acpi-bits/bits-tests/smilatency.py2 (100%) rename tests/{avocado => functional}/acpi-bits/bits-tests/testacpi.py2 (100%) rename tests/{avocado => functional}/acpi-bits/bits-tests/testcpuid.py2 (100%) rename tests/{avocado/acpi-bits.py => functional/test_acpi_bits.py} (86%) mode change 100644 => 100755 diff --git a/MAINTAINERS b/MAINTAINERS index 70dd2430db..f234464d11 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2074,8 +2074,8 @@ ACPI/AVOCADO/BIOSBITS M: Ani Sinha M: Michael S. Tsirkin S: Supported -F: tests/avocado/acpi-bits/* -F: tests/avocado/acpi-bits.py +F: tests/functional/acpi-bits/* +F: tests/functional/test_acpi_bits.py F: docs/devel/acpi-bits.rst ACPI/HEST/GHES diff --git a/docs/devel/acpi-bits.rst b/docs/devel/acpi-bits.rst index 1ec394f5fb..78aeb6aa3c 100644 --- a/docs/devel/acpi-bits.rst +++ b/docs/devel/acpi-bits.rst @@ -1,6 +1,6 @@ -============================================================================= -ACPI/SMBIOS avocado tests using biosbits -============================================================================= +================================== +ACPI/SMBIOS testing using biosbits +================================== ************ Introduction ************ @@ -35,7 +35,7 @@ for developing biosbits and its real life uses can be found in [#a]_ and [#b]_. For QEMU, we maintain a fork of bios bits in gitlab along with all the dependent submodules `here `__. This fork contains numerous fixes, a newer acpica and changes specific to -running this avocado QEMU tests using bits. The author of this document +running these functional QEMU tests using bits. The author of this document is the sole maintainer of the QEMU fork of bios bits repository. For more information, please see author's `FOSDEM talk on this bios-bits based test framework `__. @@ -44,12 +44,12 @@ framework diff --git a/tests/avocado/acpi-bits/bits-config/bits-cfg.txt b/tests/functional/acpi-bits/bits-config/bits-cfg.txt similarity index 100% rename from tests/avocado/acpi-bits/bits-config/bits-cfg.txt rename to tests/functional/acpi-bits/bits-config/bits-cfg.txt diff --git a/tests/avocado/acpi-bits/bits-tests/smbios.py2 b/tests/functional/acpi-bits/bits-tests/smbios.py2 similarity index 100% rename from tests/avocado/acpi-bits/bits-tests/smbios.py2 rename to tests/functional/acpi-bits/bits-tests/smbios.py2 diff --git a/tests/avocado/acpi-bits/bits-tests/smilatency.py2 b/tests/functional/acpi-bits/bits-tests/smilatency.py2 similarity index 100% rename from tests/avocado/acpi-bits/bits-tests/smilatency.py2 rename to tests/functional/acpi-bits/bits-tests/smilatency.py2 diff --git a/tests/avocado/acpi-bits/bits-tests/testacpi.py2 b/tests/functional/acpi-bits/bits-tests/testacpi.py2 similarity index 100% rename from tests/avocado/acpi-bits/bits-tests/testacpi.py2 rename to tests/functional/acpi-bits/bits-tests/testacpi.py2 diff --git a/tests/avocado/acpi-bits/bits-tests/testcpuid.py2 b/tests/functional/acpi-bits/bits-tests/testcpuid.py2 similarity index 100% rename from tests/avocado/acpi-bits/bits-tests/testcpuid.py2 rename to tests/functional/acpi-bits/bits-tests/testcpuid.py2 diff --git a/tests/functional/meson.build b/tests/functional/meson.build index 61cdd1d598..3bbe80b05d 100644 --- a/tests/functional/meson.build +++ b/tests/functional/meson.build @@ -11,6 +11,7 @@ endif # Timeouts for individual tests that can be slow e.g. with debugging enabled test_timeouts = { + 'acpi_bits' : 240, 'netdev_ethtool' : 180, 'ppc_40p' : 240, 'ppc64_hv' : 1000, @@ -96,6 +97,7 @@ tests_x86_64_system_quick = [ ] tests_x86_64_system_thorough = [ + 'acpi_bits', 'netdev_ethtool', 'virtio_gpu', ] diff --git a/tests/avocado/acpi-bits.py b/tests/functional/test_acpi_bits.py old mode 100644 new mode 100755 similarity index 86% rename from tests/avocado/acpi-bits.py rename to tests/functional/test_acpi_bits.py index efe4f52ee0..ee40647d5b --- a/tests/avocado/acpi-bits.py +++ b/tests/functional/test_acpi_bits.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -# group: rw quick +# # Exercise QEMU generated ACPI/SMBIOS tables using biosbits, # https://biosbits.org/ # @@ -24,7 +24,7 @@ # pylint: disable=consider-using-f-string """ -This is QEMU ACPI/SMBIOS avocado tests using biosbits. +This is QEMU ACPI/SMBIOS functional tests using biosbits. Biosbits is available originally at https://biosbits.org/. This test uses a fork of the upstream bits and has numerous fixes including an upgraded acpica. The fork is located here: @@ -41,15 +41,16 @@ import tempfile import time import zipfile + +from pathlib import Path from typing import ( List, Optional, Sequence, ) from qemu.machine import QEMUMachine -from avocado import skipIf -from avocado.utils import datadrainer as drainer -from avocado_qemu import QemuBaseTest +from unittest import skipIf +from qemu_test import QemuBaseTest, Asset deps = ["xorriso", "mformat"] # dependent tools needed in the test setup/box. supported_platforms = ['x86_64'] # supported test platforms. @@ -129,34 +130,32 @@ def base_args(self): class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes """ ACPI and SMBIOS tests using biosbits. - - :avocado: tags=arch:x86_64 - :avocado: tags=acpi - """ # in slower systems the test can take as long as 3 minutes to complete. timeout = BITS_TIMEOUT + # following are some standard configuration constants + # gitlab CI does shallow clones of depth 20 + BITS_INTERNAL_VER = 2020 + # commit hash must match the artifact tag below + BITS_COMMIT_HASH = 'c7920d2b' + # this is the latest bits release as of today. + BITS_TAG = "qemu-bits-10262023" + + ASSET_BITS = Asset(("https://gitlab.com/qemu-project/" + "biosbits-bits/-/jobs/artifacts/%s/" + "download?job=qemu-bits-build" % BITS_TAG), + '1b8dd612c6831a6b491716a77acc486666aaa867051cdc34f7ce169c2e25f487') + def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self._vm = None self._workDir = None self._baseDir = None - # following are some standard configuration constants - self._bitsInternalVer = 2020 # gitlab CI does shallow clones of depth 20 - self._bitsCommitHash = 'c7920d2b' # commit hash must match - # the artifact tag below - self._bitsTag = "qemu-bits-10262023" # this is the latest bits - # release as of today. - self._bitsArtSHA1Hash = 'b22cdfcfc7453875297d06d626f5474ee36a343f' - self._bitsArtURL = ("https://gitlab.com/qemu-project/" - "biosbits-bits/-/jobs/artifacts/%s/" - "download?job=qemu-bits-build" %self._bitsTag) self._debugcon_addr = '0x403' self._debugcon_log = 'debugcon-log.txt' - logging.basicConfig(level=logging.INFO) - self.logger = logging.getLogger('acpi-bits') + self.logger = self.log def _print_log(self, log): self.logger.info('\nlogs from biosbits follows:') @@ -171,7 +170,7 @@ def copy_bits_config(self): bits_config_dir = os.path.join(self._baseDir, 'acpi-bits', 'bits-config') target_config_dir = os.path.join(self._workDir, - 'bits-%d' %self._bitsInternalVer, + 'bits-%d' %self.BITS_INTERNAL_VER, 'boot') self.assertTrue(os.path.exists(bits_config_dir)) self.assertTrue(os.path.exists(target_config_dir)) @@ -188,7 +187,7 @@ def copy_test_scripts(self): bits_test_dir = os.path.join(self._baseDir, 'acpi-bits', 'bits-tests') target_test_dir = os.path.join(self._workDir, - 'bits-%d' %self._bitsInternalVer, + 'bits-%d' %self.BITS_INTERNAL_VER, 'boot', 'python') self.assertTrue(os.path.exists(bits_test_dir)) @@ -248,9 +247,9 @@ def generate_bits_iso(self): test scripts """ bits_dir = os.path.join(self._workDir, - 'bits-%d' %self._bitsInternalVer) + 'bits-%d' %self.BITS_INTERNAL_VER) iso_file = os.path.join(self._workDir, - 'bits-%d.iso' %self._bitsInternalVer) + 'bits-%d.iso' %self.BITS_INTERNAL_VER) mkrescue_script = os.path.join(self._workDir, 'grub-inst-x86_64-efi', 'bin', 'grub-mkrescue') @@ -264,8 +263,12 @@ def generate_bits_iso(self): try: if os.getenv('V') or os.getenv('BITS_DEBUG'): - subprocess.check_call([mkrescue_script, '-o', iso_file, - bits_dir], stderr=subprocess.STDOUT) + proc = subprocess.run([mkrescue_script, '-o', iso_file, + bits_dir], + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + check=True) + self.logger.info("grub-mkrescue output %s" % proc.stdout) else: subprocess.check_call([mkrescue_script, '-o', iso_file, bits_dir], @@ -282,8 +285,9 @@ def generate_bits_iso(self): def setUp(self): # pylint: disable=arguments-differ super().setUp('qemu-system-') + self.logger = self.log - self._baseDir = os.getenv('AVOCADO_TEST_BASEDIR') + self._baseDir = Path(__file__).parent # workdir could also be avocado's own workdir in self.workdir. # At present, I prefer to maintain my own temporary working @@ -300,15 +304,14 @@ def setUp(self): # pylint: disable=arguments-differ os.mkdir(prebuiltDir, mode=0o775) bits_zip_file = os.path.join(prebuiltDir, 'bits-%d-%s.zip' - %(self._bitsInternalVer, - self._bitsCommitHash)) + %(self.BITS_INTERNAL_VER, + self.BITS_COMMIT_HASH)) grub_tar_file = os.path.join(prebuiltDir, 'bits-%d-%s-grub.tar.gz' - %(self._bitsInternalVer, - self._bitsCommitHash)) + %(self.BITS_INTERNAL_VER, + self.BITS_COMMIT_HASH)) - bitsLocalArtLoc = self.fetch_asset(self._bitsArtURL, - asset_hash=self._bitsArtSHA1Hash) + bitsLocalArtLoc = self.ASSET_BITS.fetch() self.logger.info("downloaded bits artifacts to %s", bitsLocalArtLoc) # extract the bits artifact in the temp working directory @@ -369,7 +372,7 @@ def test_acpi_smbios_bits(self): """The main test case implementation.""" iso_file = os.path.join(self._workDir, - 'bits-%d.iso' %self._bitsInternalVer) + 'bits-%d.iso' %self.BITS_INTERNAL_VER) self.assertTrue(os.access(iso_file, os.R_OK)) @@ -393,12 +396,6 @@ def test_acpi_smbios_bits(self): self._vm.set_console() self._vm.launch() - self.logger.debug("Console output from bits VM follows ...") - c_drainer = drainer.LineLogger(self._vm.console_socket.fileno(), - logger=self.logger.getChild("console"), - stop_check=(lambda : - not self._vm.is_running())) - c_drainer.start() # biosbits has been configured to run all the specified test suites # in batch mode and then automatically initiate a vm shutdown. @@ -406,4 +403,8 @@ def test_acpi_smbios_bits(self): # with the avocado test timeout. self._vm.event_wait('SHUTDOWN', timeout=BITS_TIMEOUT) self._vm.wait(timeout=None) + self.logger.debug("Checking console output ...") self.parse_log() + +if __name__ == '__main__': + QemuBaseTest.main()