From patchwork Wed May 30 18:41:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 922964 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40wzzl5Qsdz9s0q for ; Thu, 31 May 2018 04:45:07 +1000 (AEST) Received: from localhost ([::1]:40255 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fO65p-0000qD-Cv for incoming@patchwork.ozlabs.org; Wed, 30 May 2018 14:45:05 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42026) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fO634-0007LN-9v for qemu-devel@nongnu.org; Wed, 30 May 2018 14:42:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fO631-0004Au-H0 for qemu-devel@nongnu.org; Wed, 30 May 2018 14:42:14 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:56110 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fO631-0004Ak-BW for qemu-devel@nongnu.org; Wed, 30 May 2018 14:42:11 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E2EB4BB41A; Wed, 30 May 2018 18:42:10 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-125-239.rdu2.redhat.com [10.10.125.239]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 57B8F83B7E; Wed, 30 May 2018 18:42:10 +0000 (UTC) From: Cleber Rosa To: qemu-devel@nongnu.org Date: Wed, 30 May 2018 14:41:52 -0400 Message-Id: <20180530184156.15634-2-crosa@redhat.com> In-Reply-To: <20180530184156.15634-1-crosa@redhat.com> References: <20180530184156.15634-1-crosa@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 30 May 2018 18:42:10 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 30 May 2018 18:42:10 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'crosa@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v4 1/5] Add functional/acceptance tests infrastructure X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Eduardo Habkost , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Amador Pahim , Stefan Hajnoczi , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch adds the very minimum infrastructure necessary for writing and running functional/acceptance tests, including: * Documentation * The avocado_qemu.Test base test class * One example tests (version.py) Additional functionality is expected to be added along the tests that require them. Signed-off-by: Cleber Rosa Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Reviewed-by: Stefan Hajnoczi --- docs/devel/testing.rst | 192 ++++++++++++++++++++++ tests/acceptance/README.rst | 10 ++ tests/acceptance/avocado_qemu/__init__.py | 54 ++++++ tests/acceptance/version.py | 24 +++ 4 files changed, 280 insertions(+) create mode 100644 tests/acceptance/README.rst create mode 100644 tests/acceptance/avocado_qemu/__init__.py create mode 100644 tests/acceptance/version.py diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index 0ca1a2d4b5..6cc04c6d78 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -484,3 +484,195 @@ supported. To start the fuzzer, run Alternatively, some command different from "qemu-img info" can be tested, by changing the ``-c`` option. + +Acceptance tests using the Avocado Framework +============================================ + +The ``tests/acceptance`` directory hosts functional tests, also known +as acceptance level tests. They're usually higher level tests, and +may interact with external resources and with various guest operating +systems. + +These tests are written using the Avocado Testing Framework (which must +be installed separately) in conjunction with a the ``avocado_qemu.Test`` +class, implemented at ``tests/acceptance/avocado_qemu``. + +Tests based on ``avocado_qemu.Test`` can easily: + + * Customize the command line arguments given to the convenience + ``self.vm`` attribute (a QEMUMachine instance) + + * Interact with the QEMU monitor, send QMP commands and check + their results + + * Interact with the guest OS, using the convenience console device + (which may be useful to assert the effectiveness and correctness of + command line arguments or QMP commands) + + * Interact with external data files that accompany the test itself + (see ``self.get_data()``) + + * Donwload (and cache) remote data files, such as firmware and kernel + images + + * Have access to a library of guest OS images (by means of the + ``avocado.utils.vmimage`` library) + + * Make use of various other test related utilities available at the + test class itself and at the utility library: + + - http://avocado-framework.readthedocs.io/en/latest/api/test/avocado.html#avocado.Test + - http://avocado-framework.readthedocs.io/en/latest/api/utils/avocado.utils.html + +Installation +------------ + +To install Avocado and its dependencies, run: + +.. code:: + + pip install --user avocado-framework + +Alternatively, follow the instructions on this link: + + http://avocado-framework.readthedocs.io/en/latest/GetStartedGuide.html#installing-avocado + +Overview +-------- + +This directory provides the ``avocado_qemu`` Python module, containing +the ``avocado_qemu.Test`` class. Here's a simple usage example: + +.. code:: + + from avocado_qemu import Test + + + class Version(Test): + """ + :avocado: enable + :avocado: tags=quick + """ + def test_qmp_human_info_version(self): + self.vm.launch() + res = self.vm.command('human-monitor-command', + command_line='info version') + self.assertRegexpMatches(res, r'^(\d+\.\d+\.\d)') + +To execute your test, run: + +.. code:: + + avocado run version.py + +Tests may be classified according to a convention by using docstring +directives such as ``:avocado: tags=TAG1,TAG2``. To run all tests +in the current directory, tagged as "quick", run: + +.. code:: + + avocado run -t quick . + +The ``avocado_qemu.Test`` base test class +----------------------------------------- + +The ``avocado_qemu.Test`` class has a number of characteristics that +are worth being mentioned right away. + +First of all, it attempts to give each test a ready to use QEMUMachine +instance, available at ``self.vm``. Because many tests will tweak the +QEMU command line, launching the QEMUMachine (by using ``self.vm.launch()``) +is left to the test writer. + +At test "tear down", ``avocado_qemu.Test`` handles the QEMUMachine +shutdown. + +QEMUMachine +~~~~~~~~~~~ + +The QEMUMachine API is already widely used in the Python iotests, +device-crash-test and other Python scripts. It's a wrapper around the +execution of a QEMU binary, giving its users: + + * the ability to set command line arguments to be given to the QEMU + binary + + * a ready to use QMP connection and interface, which can be used to + send commands and inspect its results, as well as asynchronous + events + + * convenience methods to set commonly used command line arguments in + a more succinct and intuitive way + +QEMU binary selection +~~~~~~~~~~~~~~~~~~~~~ + +The QEMU binary used for the ``self.vm`` QEMUMachine instance will +primarily depend on the value of the ``qemu_bin`` parameter. If it's +not explicitly set, its default value will be the result of a dynamic +probe in the same source tree. A suitable binary will be one that +targets the architecture matching host machine. + +Based on this description, test writers will usually rely on one of +the following approaches: + +1) Set ``qemu_bin``, and use the given binary + +2) Do not set ``qemu_bin``, and use a QEMU binary named like + "${arch}-softmmu/qemu-system-${arch}", either in the current + working directory, or in the current source tree. + +The resulting ``qemu_bin`` value will be preserved in the +``avocado_qemu.Test`` as an attribute with the same name. + +Attribute reference +------------------- + +Besides the attributes and methods that are part of the base +``avocado.Test`` class, the following attributes are available on any +``avocado_qemu.Test`` instance. + +vm +~~ + +A QEMUMachine instance, initially configured according to the given +``qemu_bin`` parameter. + +qemu_bin +~~~~~~~~ + +The preserved value of the ``qemu_bin`` parameter or the result of the +dynamic probe for a QEMU binary in the current working directory or +source tree. + +Parameter reference +------------------- + +To understand how Avocado parameters are accessed by tests, and how +they can be passed to tests, please refer to:: + + http://avocado-framework.readthedocs.io/en/latest/WritingTests.html#accessing-test-parameters + +Parameter values can be easily seen in the log files, and will look +like the following: + +.. code:: + + PARAMS (key=qemu_bin, path=*, default=x86_64-softmmu/qemu-system-x86_64) => 'x86_64-softmmu/qemu-system-x86_64 + +qemu_bin +~~~~~~~~ + +The exact QEMU binary to be used on QEMUMachine. + +Uninstalling Avocado +-------------------- + +If you've followed the installation instructions above, you can easily +uninstall Avocado. Start by listing the packages you have installed:: + + pip list --user + +And remove any package you want with:: + + pip uninstall diff --git a/tests/acceptance/README.rst b/tests/acceptance/README.rst new file mode 100644 index 0000000000..89260faed6 --- /dev/null +++ b/tests/acceptance/README.rst @@ -0,0 +1,10 @@ +============================================ +Acceptance tests using the Avocado Framework +============================================ + +This directory contains functional tests, also known as acceptance +level tests. They're usually higher level, and may interact with +external resources and with various guest operating systems. + +For more information, please refer to ``docs/devel/testing.rst``, +section "Acceptance tests using the Avocado Framework". diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py new file mode 100644 index 0000000000..1e54fd5932 --- /dev/null +++ b/tests/acceptance/avocado_qemu/__init__.py @@ -0,0 +1,54 @@ +# Test class and utilities for functional tests +# +# Copyright (c) 2018 Red Hat, Inc. +# +# Author: +# Cleber Rosa +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import os +import sys + +import avocado + +SRC_ROOT_DIR = os.path.dirname(os.path.dirname(os.path.dirname(__file__))) +SRC_ROOT_DIR = os.path.abspath(os.path.dirname(SRC_ROOT_DIR)) +sys.path.append(os.path.join(SRC_ROOT_DIR, 'scripts')) + +from qemu import QEMUMachine + +def is_readable_executable_file(path): + return os.path.isfile(path) and os.access(path, os.R_OK | os.X_OK) + + +def pick_default_qemu_bin(): + """ + Picks the path of a QEMU binary, starting either in the current working + directory or in the source tree root directory. + """ + arch = os.uname()[4] + qemu_bin_relative_path = os.path.join("%s-softmmu" % arch, + "qemu-system-%s" % arch) + if is_readable_executable_file(qemu_bin_relative_path): + return qemu_bin_relative_path + + qemu_bin_from_src_dir_path = os.path.join(SRC_ROOT_DIR, + qemu_bin_relative_path) + if is_readable_executable_file(qemu_bin_from_src_dir_path): + return qemu_bin_from_src_dir_path + + +class Test(avocado.Test): + def setUp(self): + self.vm = None + self.qemu_bin = self.params.get('qemu_bin', + default=pick_default_qemu_bin()) + if self.qemu_bin is None: + self.cancel("No QEMU binary defined or found in the source tree") + self.vm = QEMUMachine(self.qemu_bin) + + def tearDown(self): + if self.vm is not None: + self.vm.shutdown() diff --git a/tests/acceptance/version.py b/tests/acceptance/version.py new file mode 100644 index 0000000000..13b0a7440d --- /dev/null +++ b/tests/acceptance/version.py @@ -0,0 +1,24 @@ +# Version check example test +# +# Copyright (c) 2018 Red Hat, Inc. +# +# Author: +# Cleber Rosa +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + + +from avocado_qemu import Test + + +class Version(Test): + """ + :avocado: enable + :avocado: tags=quick + """ + def test_qmp_human_info_version(self): + self.vm.launch() + res = self.vm.command('human-monitor-command', + command_line='info version') + self.assertRegexpMatches(res, r'^(\d+\.\d+\.\d)') From patchwork Wed May 30 18:41:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 922959 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40wzx94wclz9s01 for ; Thu, 31 May 2018 04:42:52 +1000 (AEST) Received: from localhost ([::1]:40244 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fO63d-0007M8-05 for incoming@patchwork.ozlabs.org; Wed, 30 May 2018 14:42:49 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42031) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fO634-0007LQ-Cu for qemu-devel@nongnu.org; Wed, 30 May 2018 14:42:15 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fO632-0004BH-1v for qemu-devel@nongnu.org; Wed, 30 May 2018 14:42:14 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:53626 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fO631-0004B4-Tc for qemu-devel@nongnu.org; Wed, 30 May 2018 14:42:11 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7E3B04022414; Wed, 30 May 2018 18:42:11 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-125-239.rdu2.redhat.com [10.10.125.239]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 041B683B81; Wed, 30 May 2018 18:42:10 +0000 (UTC) From: Cleber Rosa To: qemu-devel@nongnu.org Date: Wed, 30 May 2018 14:41:53 -0400 Message-Id: <20180530184156.15634-3-crosa@redhat.com> In-Reply-To: <20180530184156.15634-1-crosa@redhat.com> References: <20180530184156.15634-1-crosa@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 30 May 2018 18:42:11 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 30 May 2018 18:42:11 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'crosa@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v4 2/5] scripts/qemu.py: allow adding to the list of extra arguments X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Eduardo Habkost , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Amador Pahim , Stefan Hajnoczi , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Tests will often need to add extra arguments to QEMU command line arguments. Signed-off-by: Cleber Rosa Reviewed-by: Stefan Hajnoczi Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé --- scripts/qemu.py | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/scripts/qemu.py b/scripts/qemu.py index 08a3e9af5a..7cd8193df8 100644 --- a/scripts/qemu.py +++ b/scripts/qemu.py @@ -359,3 +359,9 @@ class QEMUMachine(object): of the qemu process. ''' return self._iolog + + def add_args(self, *args): + ''' + Adds to the list of extra arguments to be given to the QEMU binary + ''' + self._args.extend(args) From patchwork Wed May 30 18:41:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 922960 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40wzx966wyz9s0w for ; Thu, 31 May 2018 04:42:53 +1000 (AEST) Received: from localhost ([::1]:40245 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fO63f-0007Mk-Dc for incoming@patchwork.ozlabs.org; Wed, 30 May 2018 14:42:51 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42028) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fO634-0007LO-BJ for qemu-devel@nongnu.org; Wed, 30 May 2018 14:42:15 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fO632-0004Br-SR for qemu-devel@nongnu.org; Wed, 30 May 2018 14:42:14 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:35006 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fO632-0004BW-NV for qemu-devel@nongnu.org; Wed, 30 May 2018 14:42:12 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3CFA3818BAF3; Wed, 30 May 2018 18:42:12 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-125-239.rdu2.redhat.com [10.10.125.239]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 942AD83B7F; Wed, 30 May 2018 18:42:11 +0000 (UTC) From: Cleber Rosa To: qemu-devel@nongnu.org Date: Wed, 30 May 2018 14:41:54 -0400 Message-Id: <20180530184156.15634-4-crosa@redhat.com> In-Reply-To: <20180530184156.15634-1-crosa@redhat.com> References: <20180530184156.15634-1-crosa@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 30 May 2018 18:42:12 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 30 May 2018 18:42:12 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'crosa@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v4 3/5] Acceptance tests: add quick VNC tests X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Eduardo Habkost , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Amador Pahim , Stefan Hajnoczi , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch adds a few simple behavior tests for VNC. Signed-off-by: Cleber Rosa Reviewed-by: Stefan Hajnoczi Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé --- tests/acceptance/vnc.py | 60 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) create mode 100644 tests/acceptance/vnc.py diff --git a/tests/acceptance/vnc.py b/tests/acceptance/vnc.py new file mode 100644 index 0000000000..b1ef9d71b1 --- /dev/null +++ b/tests/acceptance/vnc.py @@ -0,0 +1,60 @@ +# Simple functional tests for VNC functionality +# +# Copyright (c) 2018 Red Hat, Inc. +# +# Author: +# Cleber Rosa +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +from avocado_qemu import Test + + +class Vnc(Test): + """ + :avocado: enable + :avocado: tags=vnc,quick + """ + def test_no_vnc(self): + self.vm.add_args('-nodefaults', '-S') + self.vm.launch() + self.assertFalse(self.vm.qmp('query-vnc')['return']['enabled']) + + def test_no_vnc_change_password(self): + self.vm.add_args('-nodefaults', '-S') + self.vm.launch() + self.assertFalse(self.vm.qmp('query-vnc')['return']['enabled']) + set_password_response = self.vm.qmp('change', + device='vnc', + target='password', + arg='new_password') + self.assertIn('error', set_password_response) + self.assertEqual(set_password_response['error']['class'], + 'GenericError') + self.assertEqual(set_password_response['error']['desc'], + 'Could not set password') + + def test_vnc_change_password_requires_a_password(self): + self.vm.add_args('-nodefaults', '-S', '-vnc', ':0') + self.vm.launch() + self.assertTrue(self.vm.qmp('query-vnc')['return']['enabled']) + set_password_response = self.vm.qmp('change', + device='vnc', + target='password', + arg='new_password') + self.assertIn('error', set_password_response) + self.assertEqual(set_password_response['error']['class'], + 'GenericError') + self.assertEqual(set_password_response['error']['desc'], + 'Could not set password') + + def test_vnc_change_password(self): + self.vm.add_args('-nodefaults', '-S', '-vnc', ':0,password') + self.vm.launch() + self.assertTrue(self.vm.qmp('query-vnc')['return']['enabled']) + set_password_response = self.vm.qmp('change', + device='vnc', + target='password', + arg='new_password') + self.assertEqual(set_password_response['return'], {}) From patchwork Wed May 30 18:41:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 922963 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40wzzk5Tlmz9s01 for ; Thu, 31 May 2018 04:45:06 +1000 (AEST) Received: from localhost ([::1]:40254 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fO65o-0000oM-G1 for incoming@patchwork.ozlabs.org; Wed, 30 May 2018 14:45:04 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42047) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fO634-0007LX-Vc for qemu-devel@nongnu.org; Wed, 30 May 2018 14:42:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fO633-0004CH-EY for qemu-devel@nongnu.org; Wed, 30 May 2018 14:42:14 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:55792 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fO633-0004C2-9p for qemu-devel@nongnu.org; Wed, 30 May 2018 14:42:13 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id CEEB340BC042; Wed, 30 May 2018 18:42:12 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-125-239.rdu2.redhat.com [10.10.125.239]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5388683B7F; Wed, 30 May 2018 18:42:12 +0000 (UTC) From: Cleber Rosa To: qemu-devel@nongnu.org Date: Wed, 30 May 2018 14:41:55 -0400 Message-Id: <20180530184156.15634-5-crosa@redhat.com> In-Reply-To: <20180530184156.15634-1-crosa@redhat.com> References: <20180530184156.15634-1-crosa@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Wed, 30 May 2018 18:42:12 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Wed, 30 May 2018 18:42:12 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'crosa@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v4 4/5] scripts/qemu.py: introduce set_console() method X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Eduardo Habkost , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Amador Pahim , Stefan Hajnoczi , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" The set_console() method is intended to ease higher level use cases that require a console device. The amount of intelligence is limited on purpose, requiring either the device type explicitly, or the existence of a machine (pattern) definition. Because of the console device type selection criteria (by machine type), users should also be able to define that. It'll then be used for both '-machine' and for the console device type selection. Users of the set_console() method will certainly be interested in accessing the console device, and for that a console_socket property has been added. Signed-off-by: Cleber Rosa Tested-by: Philippe Mathieu-Daudé Reviewed-by: Stefan Hajnoczi --- scripts/qemu.py | 97 ++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 96 insertions(+), 1 deletion(-) diff --git a/scripts/qemu.py b/scripts/qemu.py index 7cd8193df8..f099ce7278 100644 --- a/scripts/qemu.py +++ b/scripts/qemu.py @@ -17,19 +17,41 @@ import logging import os import subprocess import qmp.qmp +import re import shutil +import socket import tempfile LOG = logging.getLogger(__name__) +#: Maps machine types to the preferred console device types +CONSOLE_DEV_TYPES = { + r'^clipper$': 'isa-serial', + r'^malta': 'isa-serial', + r'^(pc.*|q35.*|isapc)$': 'isa-serial', + r'^(40p|powernv|prep)$': 'isa-serial', + r'^pseries.*': 'spapr-vty', + r'^s390-ccw-virtio.*': 'sclpconsole', + } + + class QEMUMachineError(Exception): """ Exception called when an error in QEMUMachine happens. """ +class QEMUMachineAddDeviceError(QEMUMachineError): + """ + Exception raised when a request to add a device can not be fulfilled + + The failures are caused by limitations, lack of information or conflicting + requests on the QEMUMachine methods. This exception does not represent + failures reported by the QEMU binary itself. + """ + class MonitorResponseError(qmp.qmp.QMPError): ''' Represents erroneous QMP monitor reply @@ -91,6 +113,10 @@ class QEMUMachine(object): self._test_dir = test_dir self._temp_dir = None self._launched = False + self._machine = None + self._console_device_type = None + self._console_address = None + self._console_socket = None # just in case logging wasn't configured by the main script: logging.basicConfig() @@ -175,9 +201,19 @@ class QEMUMachine(object): self._monitor_address[1]) else: moncdev = 'socket,id=mon,path=%s' % self._vm_monitor - return ['-chardev', moncdev, + args = ['-chardev', moncdev, '-mon', 'chardev=mon,mode=control', '-display', 'none', '-vga', 'none'] + if self._machine is not None: + args.extend(['-machine', self._machine]) + if self._console_device_type is not None: + self._console_address = os.path.join(self._temp_dir, + self._name + "-console.sock") + chardev = ('socket,id=console,path=%s,server,nowait' % + self._console_address) + device = '%s,chardev=console' % self._console_device_type + args.extend(['-chardev', chardev, '-device', device]) + return args def _pre_launch(self): self._temp_dir = tempfile.mkdtemp(dir=self._test_dir) @@ -202,6 +238,10 @@ class QEMUMachine(object): self._qemu_log_path = None + if self._console_socket is not None: + self._console_socket.close() + self._console_socket = None + if self._temp_dir is not None: shutil.rmtree(self._temp_dir) self._temp_dir = None @@ -365,3 +405,58 @@ class QEMUMachine(object): Adds to the list of extra arguments to be given to the QEMU binary ''' self._args.extend(args) + + def set_machine(self, machine_type): + ''' + Sets the machine type + + If set, the machine type will be added to the base arguments + of the resulting QEMU command line. + ''' + self._machine = machine_type + + def set_console(self, device_type=None): + ''' + Sets the device type for a console device + + If set, the console device and a backing character device will + be added to the base arguments of the resulting QEMU command + line. + + This is a convenience method that will either use the provided + device type, of if not given, it will used the device type set + on CONSOLE_DEV_TYPES. + + The actual setting of command line arguments will be be done at + machine launch time, as it depends on the temporary directory + to be created. + + @param device_type: the device type, such as "isa-serial" + @raises: QEMUMachineAddDeviceError if the device type is not given + and can not be determined. + ''' + if device_type is None: + if self._machine is None: + raise QEMUMachineAddDeviceError("Can not add a console device:" + " QEMU instance without a " + "defined machine type") + for regex, device in CONSOLE_DEV_TYPES.items(): + if re.match(regex, self._machine): + device_type = device + break + if device_type is None: + raise QEMUMachineAddDeviceError("Can not add a console device:" + " no matching console device " + "type definition") + self._console_device_type = device_type + + @property + def console_socket(self): + """ + Returns a socket connected to the console + """ + if self._console_socket is None: + self._console_socket = socket.socket(socket.AF_UNIX, + socket.SOCK_STREAM) + self._console_socket.connect(self._console_address) + return self._console_socket From patchwork Wed May 30 18:41:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 922961 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40wzx94wgXz9s0q for ; Thu, 31 May 2018 04:42:53 +1000 (AEST) Received: from localhost ([::1]:40246 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fO63e-0007Mv-Pq for incoming@patchwork.ozlabs.org; Wed, 30 May 2018 14:42:50 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42053) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fO635-0007Lj-BV for qemu-devel@nongnu.org; Wed, 30 May 2018 14:42:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fO633-0004D2-WA for qemu-devel@nongnu.org; Wed, 30 May 2018 14:42:15 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:56124 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fO633-0004Ck-RH for qemu-devel@nongnu.org; Wed, 30 May 2018 14:42:13 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6E383BB41A; Wed, 30 May 2018 18:42:13 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-125-239.rdu2.redhat.com [10.10.125.239]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E551B83B7F; Wed, 30 May 2018 18:42:12 +0000 (UTC) From: Cleber Rosa To: qemu-devel@nongnu.org Date: Wed, 30 May 2018 14:41:56 -0400 Message-Id: <20180530184156.15634-6-crosa@redhat.com> In-Reply-To: <20180530184156.15634-1-crosa@redhat.com> References: <20180530184156.15634-1-crosa@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 30 May 2018 18:42:13 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 30 May 2018 18:42:13 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'crosa@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v4 5/5] Acceptance tests: add Linux kernel boot and console checking test X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Eduardo Habkost , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Amador Pahim , Stefan Hajnoczi , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This test boots a Linux kernel, and checks that the given command line was effective in two ways: * It makes the kernel use the set "console device" as a console * The kernel records the command line as expected in the console Given that way too many error conditions may occur, and detecting the kernel boot progress status may not be trivial, this test relies on a timeout to handle unexpected situations. Also, it's *not* tagged as a quick test for obvious reasons. It may be useful, while interactively running/debugging this test, or tests similar to this one, to show some of the logging channels. Example: $ avocado --show=QMP,console run boot_linux_console.py Signed-off-by: Cleber Rosa Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Reviewed-by: Stefan Hajnoczi --- tests/acceptance/boot_linux_console.py | 47 ++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) create mode 100644 tests/acceptance/boot_linux_console.py diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py new file mode 100644 index 0000000000..98324f7591 --- /dev/null +++ b/tests/acceptance/boot_linux_console.py @@ -0,0 +1,47 @@ +# Functional test that boots a Linux kernel and checks the console +# +# Copyright (c) 2018 Red Hat, Inc. +# +# Author: +# Cleber Rosa +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import logging + +from avocado_qemu import Test + + +class BootLinuxConsole(Test): + """ + Boots a x86_64 Linux kernel and checks that the console is operational + and the kernel command line is properly passed from QEMU to the kernel + + :avocado: enable + :avocado: tags=x86_64 + """ + + timeout = 60 + + def test(self): + kernel_url = ('https://mirrors.kernel.org/fedora/releases/28/' + 'Everything/x86_64/os/images/pxeboot/vmlinuz') + kernel_hash = '238e083e114c48200f80d889f7e32eeb2793e02a' + kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash) + + self.vm.set_machine('pc') + self.vm.set_console() + kernel_command_line = 'console=ttyS0' + self.vm.add_args('-kernel', kernel_path, + '-append', kernel_command_line) + self.vm.launch() + console = self.vm.console_socket.makefile() + console_logger = logging.getLogger('console') + while True: + msg = console.readline() + console_logger.debug(msg.strip()) + if 'Kernel command line: %s' % kernel_command_line in msg: + break + if 'Kernel panic - not syncing' in msg: + self.fail("Kernel panic reached")