From patchwork Fri Jan 20 22:14:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1729853 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=aUqZSu9+; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=KM0ifDVR; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4NzDQz3vTFz23hB for ; Sat, 21 Jan 2023 09:18:18 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pIzhD-0005Zl-NJ; Fri, 20 Jan 2023 17:17:15 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pIzhA-0005ZL-GY for qemu-devel@nongnu.org; Fri, 20 Jan 2023 17:17:12 -0500 Received: from smtp-out2.suse.de ([2001:67c:2178:6::1d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pIzh8-000797-4H for qemu-devel@nongnu.org; Fri, 20 Jan 2023 17:17:11 -0500 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id DFB195CA28; Fri, 20 Jan 2023 22:17:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1674253026; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZoaSAQghUA4A5BJwi7VckEZenr6FJ88giTsaofH5gIw=; b=aUqZSu9+kgx3sr2e0rDNB3u16jfpLt43GqrjkZGKb2BgQw2Ezr/asaY/3FpoGB8MupmzFJ 8TmH9h0WlUQTvPiY4ktfq5KAq6wnn8P26CDurBOdCsyU50EzCVeu7HmFm//a4A814S5TCQ JlAXcHab2BmpEK1FOzUcaYxFUipSPCs= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1674253026; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZoaSAQghUA4A5BJwi7VckEZenr6FJ88giTsaofH5gIw=; b=KM0ifDVR6otteA64w8lysyB/SkxsHNg/qZBsBXHBFBRTpi69jNki3HzsavCKnJnyrDd8uP pUCkJerdKIyYJDCg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 0743513251; Fri, 20 Jan 2023 22:17:04 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 6PB0L+ASy2P6RwAAMHmgww (envelope-from ); Fri, 20 Jan 2023 22:17:04 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Cleber Rosa , Wainer dos Santos Moschetta , Beraldo Leal , Daniel Henrique Barboza , =?utf-8?q?Philippe_Mat?= =?utf-8?q?hieu-Daud=C3=A9?= Subject: [PATCH 1/2] tests/avocado: Invert parameter vs. tag precedence during setUp Date: Fri, 20 Jan 2023 19:14:59 -0300 Message-Id: <20230120221500.4194-2-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230120221500.4194-1-farosas@suse.de> References: <20230120221500.4194-1-farosas@suse.de> MIME-Version: 1.0 Received-SPF: pass client-ip=2001:67c:2178:6::1d; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org We currently never pass parameters to the avocado process via Makefile. To start doing so we need to invert the precedence between command line parameters and tags, otherwise a command line parameter would override values for all the tests, which is unlikely to be a common use-case. A more likely use-case is to force certain values for the tests that have no tags. For instance, if a test has no 'arch' tags and therefore can run for all targets, one could possibly force it to run on a certain target with an arch=foo parameter. This applies to the variables set during setUp(): arch, machine, cpu, distro_name, distro_version. Parameters used directly in tests or read via self.params.get are left unchanged. Signed-off-by: Fabiano Rosas Reviewed-by: Daniel Henrique Barboza --- tests/avocado/avocado_qemu/__init__.py | 32 +++++++++++++++----------- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/tests/avocado/avocado_qemu/__init__.py b/tests/avocado/avocado_qemu/__init__.py index 910f3ba1ea..a181cac383 100644 --- a/tests/avocado/avocado_qemu/__init__.py +++ b/tests/avocado/avocado_qemu/__init__.py @@ -240,12 +240,23 @@ def _get_unique_tag_val(self, tag_name): return vals.pop() return None + def _get_prop(self, name): + """ + Infer test properties based on tags. If no tag is present, + look for a command line parameter of the same name. + """ + val = self._get_unique_tag_val(name) + if not val: + # If there's no tag, look for a command line + # parameter. This allows the user to override any defaults + # the caller of this function would choose if we were to + # return None. + val = self.params.get(name) + return val + def setUp(self, bin_prefix): - self.arch = self.params.get('arch', - default=self._get_unique_tag_val('arch')) - - self.cpu = self.params.get('cpu', - default=self._get_unique_tag_val('cpu')) + self.arch = self._get_prop('arch') + self.cpu = self._get_prop('cpu') default_qemu_bin = pick_default_qemu_bin(bin_prefix, arch=self.arch) self.qemu_bin = self.params.get('qemu_bin', @@ -274,8 +285,7 @@ def setUp(self): super().setUp('qemu-system-') - self.machine = self.params.get('machine', - default=self._get_unique_tag_val('machine')) + self.machine = self._get_prop('machine') def require_accelerator(self, accelerator): """ @@ -529,15 +539,11 @@ class LinuxTest(LinuxSSHMixIn, QemuSystemTest): memory = '1024' def _set_distro(self): - distro_name = self.params.get( - 'distro', - default=self._get_unique_tag_val('distro')) + distro_name = self._get_prop('distro') if not distro_name: distro_name = 'fedora' - distro_version = self.params.get( - 'distro_version', - default=self._get_unique_tag_val('distro_version')) + distro_version = self._get_prop('distro_version') if not distro_version: distro_version = '31' From patchwork Fri Jan 20 22:15:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1729855 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=UJk+SN79; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=szGqJxZ7; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4NzDR45wycz23gV for ; Sat, 21 Jan 2023 09:18:24 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pIzhI-0005aM-1g; Fri, 20 Jan 2023 17:17:20 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pIzhE-0005a7-7e for qemu-devel@nongnu.org; Fri, 20 Jan 2023 17:17:16 -0500 Received: from smtp-out1.suse.de ([2001:67c:2178:6::1c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pIzhA-00079L-Ed for qemu-devel@nongnu.org; Fri, 20 Jan 2023 17:17:14 -0500 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id B785D3389B; Fri, 20 Jan 2023 22:17:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1674253029; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=702jP5ex4ZxJiqdr56IM9l5rrWzdFVZCbS0jspohVFU=; b=UJk+SN79X6Mi52SyPf4UIc+c9ZA0LQdqez5UlIZizDYu6G6CbPCKYkF+xl7APS/17EshBj wmzPg/A8tI/uWAk0iP1kbiiDa3KLLnnlhcKzHcOU0Iyo4zTs8wm+7CKE4A4DI68fpB84Hv k1G7LkUbSIm7gj5KuTIp6vbZTWKV+VQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1674253029; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=702jP5ex4ZxJiqdr56IM9l5rrWzdFVZCbS0jspohVFU=; b=szGqJxZ7M3x6lOVYRVsyQ8rEbPxKinM3DNNlBAsLd3IsuA5T1iF1jQIuy3I5jX2nZNpPRZ x4vVGcZc3mQRQKAw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 604F113251; Fri, 20 Jan 2023 22:17:07 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id QFCnCeMSy2P6RwAAMHmgww (envelope-from ); Fri, 20 Jan 2023 22:17:07 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Cleber Rosa , Wainer dos Santos Moschetta , Beraldo Leal , Daniel Henrique Barboza , Paolo Bonzini , =?utf-8?q?Alex_Benn=C3=A9e?= , Thomas Huth Subject: [PATCH 2/2] tests/avocado: Allow passing command line parameters via Makefile Date: Fri, 20 Jan 2023 19:15:00 -0300 Message-Id: <20230120221500.4194-3-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230120221500.4194-1-farosas@suse.de> References: <20230120221500.4194-1-farosas@suse.de> MIME-Version: 1.0 Received-SPF: pass client-ip=2001:67c:2178:6::1c; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Add support for the 'avocado run' "-p" option, which allows us to pass parameters in the form key=value to be applied to all tests selected for a given run. This is useful to force generic tests to use a specific machine, cpu or qemu-binary where otherwise the defaults would be used. E.g.: $ make check-avocado AVOCADO_PARAMS="machine=virt arch=riscv64" Signed-off-by: Fabiano Rosas Reviewed-by: Daniel Henrique Barboza --- tests/Makefile.include | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tests/Makefile.include b/tests/Makefile.include index 9422ddaece..f92e730aa0 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -107,6 +107,10 @@ else AVOCADO_CMDLINE_TAGS=$(addprefix -t , $(AVOCADO_TAGS)) endif +ifdef AVOCADO_PARAMS + AVOCADO_CMDLINE_PARAMS=$(addprefix -p , $(AVOCADO_PARAMS)) +endif + quiet-venv-pip = $(quiet-@)$(call quiet-command-run, \ $(TESTS_PYTHON) -m pip -q --disable-pip-version-check $1, \ "VENVPIP","$1") @@ -144,7 +148,7 @@ check-avocado: check-venv $(TESTS_RESULTS_DIR) get-vm-images --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \ $(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \ --filter-by-tags-include-empty-key) \ - $(AVOCADO_CMDLINE_TAGS) \ + $(AVOCADO_CMDLINE_TAGS) $(AVOCADO_CMDLINE_PARAMS) \ $(if $(GITLAB_CI),,--failfast) $(AVOCADO_TESTS), \ "AVOCADO", "tests/avocado")