From patchwork Tue Oct 16 23:51:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brendan Higgins X-Patchwork-Id: 985064 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.infradead.org (client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=linux-um-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="BCSbRRjV"; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.b="BBrFvPu6"; dkim-atps=neutral Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42ZXJ66NPzz9s9J for ; Wed, 17 Oct 2018 10:55:50 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=pafe3KCwddzxlY5GqbWo4eYkOcA1NPNuKbXhP5JgCVA=; b=BCSbRRjVuLFFtj mYva7fpSf/8i4acO6TWScxWw2+6fA5eq37B2LIZKoXKrkh3q2uxt4/L2vwINb8iqIqnkU5TVmwNc5 zf2LuFJE1lMc2qiiu+oblwfHuGLgjvC96RcMM35m1TT3xF5k/86m+OT7OzRlfAk9UwgJsUYk/sUdQ WFKutVK5KHmsiNG7XAbozR/DSQp3EzJ+z/GvE1oR02Lt89ZG56fKUpM6V+sgO4ul8Q32szb/4vrxo ZOkzgHz2zK+asuvOpmaBOpUjn+mUbzx2z+xgtiDPIjKGnL5mUtMyoVfftZgPCKTaomVmgcusN9HTY e4qW4hhyMpxIcW4QP4FQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1gCZBf-00069m-GS; Tue, 16 Oct 2018 23:55:43 +0000 Received: from mail-qk1-x74a.google.com ([2607:f8b0:4864:20::74a]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1gCZB4-0004bo-H7 for linux-um@lists.infradead.org; Tue, 16 Oct 2018 23:55:38 +0000 Received: by mail-qk1-x74a.google.com with SMTP id w6-v6so22529519qka.15 for ; Tue, 16 Oct 2018 16:54:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=4xqp6vFemfXW8THZIer615fzQIJFwGYlGopIVPlWEZU=; b=BBrFvPu6rMmaVi30GrjwrXOOL8LtR+BGznO8B6Lu7/D0K7Rxmkg7bSDYdO0W37ggd6 KUNSubXNekqIeyP7oqpcPSpGI0mcv6Vtw2I5Z3936kap9Ya6W1ZImCRBoHm8wueyJwFs n3DXvsdpVwzqIG6E5uoEIgwtyv44VHSz4lAto6aCW+jm0k6tzCeeR55BZmH2pd3RGt4g JcdwDvt19oZbFzthf6WjznwYRdU7Wo4RSSdCEfrnbjAXW9y9Kzjl6Y+tfZ7jDte0Y1Zr 7s2y2yF6SPkihHyj/U3+03bMRJrvXI/JB2rc/uNM0cpzs9K2a3FRYrGnf1rSvlzbT2SQ qbkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=4xqp6vFemfXW8THZIer615fzQIJFwGYlGopIVPlWEZU=; b=Ybd2FBVOC/EiWutKa+yXkLpyAakBLDmtBb5wEhMSbFm2WPxyQ4gBqFLmqK1xyM4gJB gHe6gDJCdtWL8p236cnEVuwAhttSJykpcTClwhCXEaT+WR8JA4tvm6mYBh/kEn03pQ7T InhdSLgGnenWzyx1cTrV1bwnc1jzUlbb2DKXKVPBxnNo7pVBXtLMvIUWxEa8D1XhoiJr aqoNCXffG6MjdSRbPgL3z8JTcKXZchFEPa9jiQGq1M0zkJC2VZvTnROmieR0k5PhKT37 pPosc5PthTJCr6vYY4pzneiBaM3G/9Zzq8CCs1XKW4Y+rEbota9NDjVVn+z/bYON52Lh WngA== X-Gm-Message-State: ABuFfoipr/XVJRdblcKoo3uU06RXllWHxjMjqBoNJ0moJ7i2+gfLYZBh eM7XfkkirdJtwnWQDuu8pWMcRbdYs3OTG0HE2Vbyzg== X-Google-Smtp-Source: ACcGV61ZHBmusrNthEsvEz3NbEV4FbUz7HloQ+nhCM0P9R1x78s+c2LlynbGJcX007Ruj/zTWK2lQyTYOoSWK9tR8t8v0Q== X-Received: by 2002:a0c:c13b:: with SMTP id f56mr20378309qvh.11.1539734094801; Tue, 16 Oct 2018 16:54:54 -0700 (PDT) Date: Tue, 16 Oct 2018 16:51:17 -0700 In-Reply-To: <20181016235120.138227-1-brendanhiggins@google.com> Message-Id: <20181016235120.138227-29-brendanhiggins@google.com> Mime-Version: 1.0 References: <20181016235120.138227-1-brendanhiggins@google.com> X-Mailer: git-send-email 2.19.1.331.ge82ca0e54c-goog Subject: [RFC v1 28/31] kunit: added Python libraries for handing KUnit config and kernel From: Brendan Higgins To: gregkh@linuxfoundation.org, keescook@google.com, mcgrof@kernel.org, shuah@kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20181016_165506_668977_988FA90F X-CRM114-Status: GOOD ( 13.28 ) X-Spam-Score: -7.6 (-------) X-Spam-Report: SpamAssassin version 3.4.1 on bombadil.infradead.org summary: Content analysis details: (-7.6 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:74a listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.0 DKIMWL_WL_MED DKIMwl.org - Whitelisted Medium sender X-BeenThere: linux-um@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: brakmo@fb.com, richard@nod.at, mpe@ellerman.id.au, Tim.Bird@sony.com, linux-um@lists.infradead.org, linux-kernel@vger.kernel.org, rostedt@goodmis.org, julia.lawall@lip6.fr, Felix Guo , joel@jms.id.au, linux-kselftest@vger.kernel.org, khilman@baylibre.com, joe@perches.com, jdike@addtoit.com, Brendan Higgins , kunit-dev@googlegroups.com Sender: "linux-um" Errors-To: linux-um-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org kunit_config.py: - parses .config and Kconfig files kunit_kernel.py: provides helper functions to: - configure the kernel using kunitconfig - builds the kernel with the correct architecture - provides function to invoke the kernel and stream the output back The kernel invocation is wrapped in a subprocess call within the module because regular invocation of the kernel (./linux) may modify TTY settings. Signed-off-by: Felix Guo Signed-off-by: Brendan Higgins --- tools/testing/kunit/.gitignore | 3 + tools/testing/kunit/kunit_config.py | 60 ++++++++++++++ tools/testing/kunit/kunit_kernel.py | 123 ++++++++++++++++++++++++++++ 3 files changed, 186 insertions(+) create mode 100644 tools/testing/kunit/.gitignore create mode 100644 tools/testing/kunit/kunit_config.py create mode 100644 tools/testing/kunit/kunit_kernel.py diff --git a/tools/testing/kunit/.gitignore b/tools/testing/kunit/.gitignore new file mode 100644 index 0000000000000..c791ff59a37a9 --- /dev/null +++ b/tools/testing/kunit/.gitignore @@ -0,0 +1,3 @@ +# Byte-compiled / optimized / DLL files +__pycache__/ +*.py[cod] \ No newline at end of file diff --git a/tools/testing/kunit/kunit_config.py b/tools/testing/kunit/kunit_config.py new file mode 100644 index 0000000000000..183bd5e758762 --- /dev/null +++ b/tools/testing/kunit/kunit_config.py @@ -0,0 +1,60 @@ +# SPDX-License-Identifier: GPL-2.0 + +import collections +import re + +CONFIG_IS_NOT_SET_PATTERN = r'^# CONFIG_\w+ is not set$' +CONFIG_PATTERN = r'^CONFIG_\w+=\S+$' + +KconfigEntryBase = collections.namedtuple('KconfigEntry', ['raw_entry']) + + +class KconfigEntry(KconfigEntryBase): + + def __str__(self) -> str: + return self.raw_entry + + +class KconfigParseError(Exception): + """Error parsing Kconfig defconfig or .config.""" + + +class Kconfig(object): + """Represents defconfig or .config specified using the Kconfig language.""" + + def __init__(self): + self._entries = [] + + def entries(self): + return set(self._entries) + + def add_entry(self, entry: KconfigEntry) -> None: + self._entries.append(entry) + + def is_subset_of(self, other: "Kconfig") -> bool: + return self.entries().issubset(other.entries()) + + def write_to_file(self, path: str) -> None: + with open(path, 'w') as f: + for entry in self.entries(): + f.write(str(entry) + '\n') + + def parse_from_string(self, blob: str) -> None: + """Parses a string containing KconfigEntrys and populates this Kconfig.""" + self._entries = [] + is_not_set_matcher = re.compile(CONFIG_IS_NOT_SET_PATTERN) + config_matcher = re.compile(CONFIG_PATTERN) + for line in blob.split('\n'): + line = line.strip() + if not line: + continue + elif config_matcher.match(line) or is_not_set_matcher.match(line): + self._entries.append(KconfigEntry(line)) + elif line[0] == '#': + continue + else: + raise KconfigParseError('Failed to parse: ' + line) + + def read_from_file(self, path: str) -> None: + with open(path, 'r') as f: + self.parse_from_string(f.read()) diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py new file mode 100644 index 0000000000000..87abaede50513 --- /dev/null +++ b/tools/testing/kunit/kunit_kernel.py @@ -0,0 +1,123 @@ +# SPDX-License-Identifier: GPL-2.0 + +import logging +import subprocess +import os + +import kunit_config + +KCONFIG_PATH = '.config' + +class ConfigError(Exception): + """Represents an error trying to configure the Linux kernel.""" + + +class BuildError(Exception): + """Represents an error trying to build the Linux kernel.""" + + +class LinuxSourceTreeOperations(object): + """An abstraction over command line operations performed on a source tree.""" + + def make_mrproper(self): + try: + subprocess.check_output(['make', 'mrproper']) + except OSError as e: + raise ConfigError('Could not call make command: ' + e) + except subprocess.CalledProcessError as e: + raise ConfigError(e.output) + + def make_olddefconfig(self): + try: + subprocess.check_output(['make', 'ARCH=um', 'olddefconfig']) + except OSError as e: + raise ConfigError('Could not call make command: ' + e) + except subprocess.CalledProcessError as e: + raise ConfigError(e.output) + + def make(self): + try: + subprocess.check_output(['make', 'ARCH=um']) + except OSError as e: + raise BuildError('Could not call execute make: ' + e) + except subprocess.CalledProcessError as e: + raise BuildError(e.output) + + def linux_bin(self, params, timeout): + """Runs the Linux UML binary. Must be named 'linux'.""" + process = subprocess.Popen( + ['./linux'] + params, + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + process.wait(timeout=timeout) + return process + + +class LinuxSourceTree(object): + """Represents a Linux kernel source tree with KUnit tests.""" + + def __init__(self): + self._kconfig = kunit_config.Kconfig() + self._kconfig.read_from_file('kunitconfig') + self._ops = LinuxSourceTreeOperations() + + def clean(self): + try: + self._ops.make_mrproper() + except ConfigError as e: + logging.error(e) + return False + return True + + def build_config(self): + self._kconfig.write_to_file(KCONFIG_PATH) + try: + self._ops.make_olddefconfig() + except ConfigError as e: + logging.error(e) + return False + validated_kconfig = kunit_config.Kconfig() + validated_kconfig.read_from_file(KCONFIG_PATH) + if not self._kconfig.is_subset_of(validated_kconfig): + logging.error('Provided Kconfig is not contained in validated .config!') + return False + return True + + def build_reconfig(self): + """Creates a new .config if it is not a subset of the kunitconfig.""" + if os.path.exists(KCONFIG_PATH): + existing_kconfig = kunit_config.Kconfig() + existing_kconfig.read_from_file(KCONFIG_PATH) + if not self._kconfig.is_subset_of(existing_kconfig): + print('Regenerating .config ...') + os.remove(KCONFIG_PATH) + return self.build_config() + else: + return True + else: + print('Generating .config ...') + return self.build_config() + + def build_um_kernel(self): + try: + self._ops.make_olddefconfig() + self._ops.make() + except (ConfigError, BuildError) as e: + logging.error(e) + return False + used_kconfig = kunit_config.Kconfig() + used_kconfig.read_from_file(KCONFIG_PATH) + if not self._kconfig.is_subset_of(used_kconfig): + logging.error('Provided Kconfig is not contained in final config!') + return False + return True + + def run_kernel(self, args=[]): + timeout = None + args.extend(['mem=256M']) + process = self._ops.linux_bin(args, timeout) + with open('test.log', 'w') as f: + for line in process.stdout: + f.write(line.rstrip().decode('ascii') + '\n') + yield line.rstrip().decode('ascii')