From patchwork Thu Nov 22 04:26:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Weber X-Patchwork-Id: 1001524 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=rockwellcollins.com Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 430mbq00vFz9s29 for ; Thu, 22 Nov 2018 15:26:30 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id C36D98808F; Thu, 22 Nov 2018 04:26:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 63gP9E5MCGmi; Thu, 22 Nov 2018 04:26:23 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id BD25288076; Thu, 22 Nov 2018 04:26:23 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 9DD091C290C for ; Thu, 22 Nov 2018 04:26:22 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 99B5488423 for ; Thu, 22 Nov 2018 04:26:22 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EuSPQ3K9VkmG for ; Thu, 22 Nov 2018 04:26:21 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from ch3vs04.rockwellcollins.com (ch3vs04.rockwellcollins.com [205.175.226.52]) by whitealder.osuosl.org (Postfix) with ESMTPS id 37F3D88415 for ; Thu, 22 Nov 2018 04:26:21 +0000 (UTC) Received: from ofwch3n02.rockwellcollins.com (HELO dtulimr01.rockwellcollins.com) ([205.175.226.14]) by ch3vs04.rockwellcollins.com with ESMTP; 21 Nov 2018 22:26:20 -0600 X-Received: from largo.rockwellcollins.com (unknown [192.168.140.76]) by dtulimr01.rockwellcollins.com (Postfix) with ESMTP id 0E0E4603AE; Wed, 21 Nov 2018 22:26:20 -0600 (CST) From: Matt Weber To: buildroot@buildroot.org Date: Wed, 21 Nov 2018 22:26:11 -0600 Message-Id: <1542860777-57909-1-git-send-email-matthew.weber@rockwellcollins.com> X-Mailer: git-send-email 1.9.1 Subject: [Buildroot] [PATCH v4 1/7] package/compiler-rt: new package X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Romain Naour , Ricardo Martincoski , Valentin Korenblit MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" This patch adds support for the compiler-rt (CLANG runtime) library. It builds a set of static libraries and installs them into the CLANG/LLVM toolchain resource folder. Signed-off-by: Matthew Weber Cc: Romain Naour Cc: Ricardo Martincoski Cc: Valentin Korenblit --- Changes v1 -> v2 [Romain - Removed unnecessary host-cmake dependency v2 -> v3 [Romain - Update to use COMPILER_RT_INSTALL_PATH for library install [Ricardo - Fixed check-package extra line - Spelling in commit message v3 -> v4 [Valentin - Pointed out that non-glibc libraries won't build. Further investigation shows that the lib asan support in compiler-rt doesn't have the libc checks yet to support multiple libraries. (https://patchwork.ozlabs.org/patch/339938/ was the best example I could find and it doesn't look like anything has changed in the recent codebase) --- DEVELOPERS | 1 + package/Config.in | 1 + package/compiler-rt/Config.in | 14 ++++++++++++++ package/compiler-rt/compiler-rt.hash | 3 +++ package/compiler-rt/compiler-rt.mk | 33 +++++++++++++++++++++++++++++++++ 5 files changed, 52 insertions(+) create mode 100644 package/compiler-rt/Config.in create mode 100644 package/compiler-rt/compiler-rt.hash create mode 100644 package/compiler-rt/compiler-rt.mk diff --git a/DEVELOPERS b/DEVELOPERS index 53467da..e78d649 100644 --- a/DEVELOPERS +++ b/DEVELOPERS @@ -1360,6 +1360,7 @@ F: package/bridge-utils/ F: package/checkpolicy/ F: package/checksec/ F: package/cgroupfs-mount/ +F: package/compiler-rt/ F: package/crda/ F: package/devmem2/ F: package/dnsmasq/ diff --git a/package/Config.in b/package/Config.in index b60e770..73ddc2d 100644 --- a/package/Config.in +++ b/package/Config.in @@ -1602,6 +1602,7 @@ menu "Other" source "package/clapack/Config.in" source "package/classpath/Config.in" source "package/cmocka/Config.in" + source "package/compiler-rt/Config.in" source "package/cppcms/Config.in" source "package/cracklib/Config.in" source "package/dawgdic/Config.in" diff --git a/package/compiler-rt/Config.in b/package/compiler-rt/Config.in new file mode 100644 index 0000000..9afc0d9 --- /dev/null +++ b/package/compiler-rt/Config.in @@ -0,0 +1,14 @@ +config BR2_PACKAGE_COMPILER_RT + bool "compiler-rt" + depends on BR2_PACKAGE_LLVM + depends on BR2_TOOLCHAIN_USES_GLIBC # asan lib requires + help + A collection of runtime libraries primarily used by clang and + llvm to provide builtins, sanitizer runtimes, and profiling + at runtime. + + https://compiler-rt.llvm.org/ + +comment "compiler-rt requires llvm to be enabled and a glibc toolchain" + depends on !BR2_PACKAGE_LLVM + depends on !BR2_TOOLCHAIN_USES_GLIBC diff --git a/package/compiler-rt/compiler-rt.hash b/package/compiler-rt/compiler-rt.hash new file mode 100644 index 0000000..df6ed19 --- /dev/null +++ b/package/compiler-rt/compiler-rt.hash @@ -0,0 +1,3 @@ +# Locally computed: +sha256 bdec7fe3cf2c85f55656c07dfb0bd93ae46f2b3dd8f33ff3ad6e7586f4c670d6 compiler-rt-7.0.0.src.tar.xz +sha256 417541d990edb3f96327ac03cb67e52eac80fc5c3e7afc69213cd04d7c3b9b27 LICENSE.TXT diff --git a/package/compiler-rt/compiler-rt.mk b/package/compiler-rt/compiler-rt.mk new file mode 100644 index 0000000..fc64707 --- /dev/null +++ b/package/compiler-rt/compiler-rt.mk @@ -0,0 +1,33 @@ +################################################################################ +# +# compiler-rt +# +################################################################################ + +# Compiler-RT should be bumped together with LLVM and Clang as the run-time is +# tied to the version of those tools +COMPILER_RT_VERSION = 7.0.0 +COMPILER_RT_SOURCE = compiler-rt-$(COMPILER_RT_VERSION).src.tar.xz +COMPILER_RT_SITE = http://llvm.org/releases/$(COMPILER_RT_VERSION) +COMPILER_RT_LICENSE = NCSA MIT +COMPILER_RT_LICENSE_FILES = LICENSE.TXT +COMPILER_RT_DEPENDENCIES = host-clang llvm + +COMPILER_RT_INSTALL_STAGING = YES +COMPILER_RT_INSTALL_TARGET = NO + +# The installation of the target runtime libraries is currently into the +# host-{clang,llvm} resources directory. The "resources" directory is loosely +# document and this location will probably need revisited when making the +# llvm/clang tools SDK relocatable. +COMPILER_RT_CONF_OPTS=-DCOMPILER_RT_INCLUDE_TESTS=ON \ + -DCOMPILER_RT_STANDALONE_BUILD=ON \ + -DCOMPILER_RT_DEFAULT_TARGET_TRIPLE=$(GNU_TARGET_NAME) \ + -DLLVM_CONFIG_PATH=$(HOST_DIR)/usr/bin/llvm-config \ + -DCOMPILER_RT_INSTALL_PATH=$(HOST_DIR)/lib/clang/$(HOST_CLANG_VERSION)/ + +define COMPILER_RT_INSTALL_STAGING_CMDS + $(TARGET_MAKE_ENV) $(MAKE) -C $(@D) install/fast +endef + +$(eval $(cmake-package)) From patchwork Thu Nov 22 04:26:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Weber X-Patchwork-Id: 1001527 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=rockwellcollins.com Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 430mbt5pyvz9s29 for ; Thu, 22 Nov 2018 15:26:34 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 079A287503; Thu, 22 Nov 2018 04:26:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JePjMcIrRnc7; Thu, 22 Nov 2018 04:26:30 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 253EC874E7; Thu, 22 Nov 2018 04:26:30 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id CA7F21C290C for ; Thu, 22 Nov 2018 04:26:25 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id C60C688423 for ; Thu, 22 Nov 2018 04:26:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bZZXMBomnUdS for ; Thu, 22 Nov 2018 04:26:23 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from da1vs03.rockwellcollins.com (da1vs03.rockwellcollins.com [205.175.227.47]) by whitealder.osuosl.org (Postfix) with ESMTPS id 1D1B388415 for ; Thu, 22 Nov 2018 04:26:23 +0000 (UTC) Received: from ofwda1n02.rockwellcollins.com (HELO dtulimr01.rockwellcollins.com) ([205.175.227.14]) by da1vs03.rockwellcollins.com with ESMTP; 21 Nov 2018 22:26:22 -0600 X-Received: from largo.rockwellcollins.com (unknown [192.168.140.76]) by dtulimr01.rockwellcollins.com (Postfix) with ESMTP id 1E639603D9; Wed, 21 Nov 2018 22:26:22 -0600 (CST) From: Matt Weber To: buildroot@buildroot.org Date: Wed, 21 Nov 2018 22:26:13 -0600 Message-Id: <1542860777-57909-3-git-send-email-matthew.weber@rockwellcollins.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1542860777-57909-1-git-send-email-matthew.weber@rockwellcollins.com> References: <1542860777-57909-1-git-send-email-matthew.weber@rockwellcollins.com> Subject: [Buildroot] [PATCH v4 3/7] testing/infra/builder: build with target and environment X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ricardo Martincoski MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" From: Ricardo Martincoski Make the builder able to call 'VAR1=1 make VAR2=2 target'. Allow to send extra parameters to be added to the end of make command line. It can be used for these purposes: - to configure a br2-external, by passing 'BR2_EXTERNAL="dir"'; - to specify a make target, such as 'foo-source'. Allow to add variables to the environment in which make runs. It can be used to override values from environment, such as 'BR2_DL_DIR="dl"'. This change will be needed when adding support for br2-externals to the test infra. Signed-off-by: Ricardo Martincoski Cc: Arnout Vandecappelle Signed-off-by: Matt Weber --- Changes v2 - New from: http://patchwork.ozlabs.org/patch/912351/ v2 -> v3 [Ricardo - Updated patch commit description context. v4 - None --- support/testing/infra/builder.py | 38 ++++++++++++++++++++++++++++++++++---- 1 file changed, 34 insertions(+), 4 deletions(-) diff --git a/support/testing/infra/builder.py b/support/testing/infra/builder.py index fc318fe..4d9c512 100644 --- a/support/testing/infra/builder.py +++ b/support/testing/infra/builder.py @@ -12,7 +12,18 @@ class Builder(object): self.builddir = builddir self.logfile = infra.open_log_file(builddir, "build", logtofile) - def configure(self): + def configure(self, make_extra_opts=[], make_extra_env={}): + """ + Configure the build. + + make_extra_opts: a list of arguments to be passed to the make + command. + e.g. make_extra_opts=["BR2_EXTERNAL=/path"] + + make_extra_env: a dict of variables to be appended (or replaced) + in the environment that calls make. + e.g. make_extra_env={"BR2_DL_DIR": "/path"} + """ if not os.path.isdir(self.builddir): os.makedirs(self.builddir) @@ -25,22 +36,41 @@ class Builder(object): self.logfile.flush() env = {"PATH": os.environ["PATH"]} + env.update(make_extra_env) + cmd = ["make", - "O={}".format(self.builddir), - "olddefconfig"] + "O={}".format(self.builddir)] + cmd += make_extra_opts + cmd += ["olddefconfig"] + ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile, env=env) if ret != 0: raise SystemError("Cannot olddefconfig") - def build(self): + def build(self, make_extra_opts=[], make_extra_env={}): + """ + Perform the build. + + make_extra_opts: a list of arguments to be passed to the make + command. It can include a make target. + e.g. make_extra_opts=["foo-source"] + + make_extra_env: a dict of variables to be appended (or replaced) + in the environment that calls make. + e.g. make_extra_env={"BR2_DL_DIR": "/path"} + """ env = {"PATH": os.environ["PATH"]} if "http_proxy" in os.environ: self.logfile.write("Using system proxy: " + os.environ["http_proxy"] + "\n") env['http_proxy'] = os.environ["http_proxy"] env['https_proxy'] = os.environ["http_proxy"] + env.update(make_extra_env) + cmd = ["make", "-C", self.builddir] + cmd += make_extra_opts + ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile, env=env) if ret != 0: From patchwork Thu Nov 22 04:26:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Weber X-Patchwork-Id: 1001530 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=rockwellcollins.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 430mc053Bpz9sC7 for ; Thu, 22 Nov 2018 15:26:40 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 13644306A4; Thu, 22 Nov 2018 04:26:37 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EfsXOlUdw3FO; Thu, 22 Nov 2018 04:26:31 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 07AF530936; Thu, 22 Nov 2018 04:26:31 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 313931C290C for ; Thu, 22 Nov 2018 04:26:26 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 2C06188098 for ; Thu, 22 Nov 2018 04:26:26 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TwB3uV-+yf-J for ; Thu, 22 Nov 2018 04:26:23 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from secvs02.rockwellcollins.com (secvs02.rockwellcollins.com [205.175.225.241]) by hemlock.osuosl.org (Postfix) with ESMTPS id 95E7E87B9D for ; Thu, 22 Nov 2018 04:26:23 +0000 (UTC) Received: from ofwgwc03.rockwellcollins.com (HELO dtulimr01.rockwellcollins.com) ([205.175.225.12]) by secvs02.rockwellcollins.com with ESMTP; 21 Nov 2018 22:26:22 -0600 X-Received: from largo.rockwellcollins.com (unknown [192.168.140.76]) by dtulimr01.rockwellcollins.com (Postfix) with ESMTP id 49D65600DE; Wed, 21 Nov 2018 22:26:22 -0600 (CST) From: Matt Weber To: buildroot@buildroot.org Date: Wed, 21 Nov 2018 22:26:14 -0600 Message-Id: <1542860777-57909-4-git-send-email-matthew.weber@rockwellcollins.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1542860777-57909-1-git-send-email-matthew.weber@rockwellcollins.com> References: <1542860777-57909-1-git-send-email-matthew.weber@rockwellcollins.com> Subject: [Buildroot] [PATCH v4 4/7] testing/infra/basetest: support br2-external X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ricardo Martincoski MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" From: Ricardo Martincoski Some upcoming test cases can use one or more br2-external trees as fixtures that provide packages used only in runtime tests. Add support for br2-external into the BRTest class. Any test case can then provide a list of paths for be used as br2-external trees during the build of the image to be tested. Signed-off-by: Ricardo Martincoski Cc: Arnout Vandecappelle Signed-off-by: Matt Weber --- Changes v2 - New from: http://patchwork.ozlabs.org/patch/912354/ v2 -> v3 [Ricardo - Updated patch commit description context. v4 - None --- support/testing/infra/basetest.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/support/testing/infra/basetest.py b/support/testing/infra/basetest.py index 5014fef..8413943 100644 --- a/support/testing/infra/basetest.py +++ b/support/testing/infra/basetest.py @@ -30,6 +30,7 @@ MINIMAL_CONFIG = \ class BRTest(unittest.TestCase): config = None + br2_external = list() downloaddir = None outputdir = None logtofile = True @@ -58,7 +59,7 @@ class BRTest(unittest.TestCase): if not self.b.is_finished(): self.show_msg("Building") - self.b.configure() + self.b.configure(["BR2_EXTERNAL={}".format(":".join(self.br2_external))]) self.b.build() self.show_msg("Building done") From patchwork Thu Nov 22 04:26:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Weber X-Patchwork-Id: 1001526 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=rockwellcollins.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 430mbs2vjCz9s29 for ; Thu, 22 Nov 2018 15:26:33 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id EB3FC30319; Thu, 22 Nov 2018 04:26:30 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K9qzZZLd14NE; Thu, 22 Nov 2018 04:26:29 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 610A8305A6; Thu, 22 Nov 2018 04:26:29 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 92A341C290C for ; Thu, 22 Nov 2018 04:26:25 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 206F3883CF for ; Thu, 22 Nov 2018 04:26:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 89Snq+jS+Una for ; Thu, 22 Nov 2018 04:26:24 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from secvs01.rockwellcollins.com (secvs01.rockwellcollins.com [205.175.225.240]) by hemlock.osuosl.org (Postfix) with ESMTPS id 9F6D287E53 for ; Thu, 22 Nov 2018 04:26:23 +0000 (UTC) Received: from ofwgwc03.rockwellcollins.com (HELO dtulimr01.rockwellcollins.com) ([205.175.225.12]) by secvs01.rockwellcollins.com with ESMTP; 21 Nov 2018 22:26:22 -0600 X-Received: from largo.rockwellcollins.com (unknown [192.168.140.76]) by dtulimr01.rockwellcollins.com (Postfix) with ESMTP id 751F1603B0; Wed, 21 Nov 2018 22:26:22 -0600 (CST) From: Matt Weber To: buildroot@buildroot.org Date: Wed, 21 Nov 2018 22:26:15 -0600 Message-Id: <1542860777-57909-5-git-send-email-matthew.weber@rockwellcollins.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1542860777-57909-1-git-send-email-matthew.weber@rockwellcollins.com> References: <1542860777-57909-1-git-send-email-matthew.weber@rockwellcollins.com> Subject: [Buildroot] [PATCH v4 5/7] testing/tests: CLANG compiler-rt runtime test X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ricardo Martincoski MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" This patch adds a test case that 1) Builds the complete LLVM and CLANG set of host tools 2) Cross-compiles the compiler-rt runtime using CLANG 3) Builds a cross-compiled application using CLANG and the libfuzzer compiler-rt library. 4) Executes the fuzz application (part of the libfuzzer package) on target and checks expected output Signed-off-by: Matthew Weber Cc: Ricardo Martincoski Reviewed-by: Ricardo Martincoski [with the entire series applied on next branch: https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/121908178 NOTE: this test case takes longer than test_rust, ~2 hours in my host machine with all tarballs previously downloaded] Tested-by: Ricardo Martincoski --- Changes v1 -> v2 [Ricardo - updated yml with test case - moved emulator launch cmd to test case from infra - defconfig in the order provided by savedefconfig - indent defconfig - add full package for test code to the test br2-external - consolidated to one class only v2 -> v3 - Added Review/tested by v4 - None --- .gitlab-ci.yml | 1 + .../br2-external/clang-compiler-rt/Config.in | 1 + .../br2-external/clang-compiler-rt/external.desc | 1 + .../br2-external/clang-compiler-rt/external.mk | 1 + .../clang-compiler-rt/package/libfuzzer/Config.in | 7 ++++ .../package/libfuzzer/libfuzzer.hash | 2 + .../package/libfuzzer/libfuzzer.mk | 24 +++++++++++ support/testing/tests/package/test_clang.py | 46 ++++++++++++++++++++++ 8 files changed, 83 insertions(+) create mode 100644 support/testing/tests/package/br2-external/clang-compiler-rt/Config.in create mode 100644 support/testing/tests/package/br2-external/clang-compiler-rt/external.desc create mode 100644 support/testing/tests/package/br2-external/clang-compiler-rt/external.mk create mode 100644 support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/Config.in create mode 100644 support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.hash create mode 100644 support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.mk create mode 100644 support/testing/tests/package/test_clang.py diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index c78f2ea..2f2125f 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -309,6 +309,7 @@ tests.init.test_systemd.TestInitSystemSystemdRoNetworkd: *runtime_test tests.init.test_systemd.TestInitSystemSystemdRwFull: *runtime_test tests.init.test_systemd.TestInitSystemSystemdRwIfupdown: *runtime_test tests.init.test_systemd.TestInitSystemSystemdRwNetworkd: *runtime_test +tests.package.test_clang.TestClangCompilerRT: *runtime_test tests.package.test_dropbear.TestDropbear: *runtime_test tests.package.test_ipython.TestIPythonPy2: *runtime_test tests.package.test_ipython.TestIPythonPy3: *runtime_test diff --git a/support/testing/tests/package/br2-external/clang-compiler-rt/Config.in b/support/testing/tests/package/br2-external/clang-compiler-rt/Config.in new file mode 100644 index 0000000..e1f9f8c --- /dev/null +++ b/support/testing/tests/package/br2-external/clang-compiler-rt/Config.in @@ -0,0 +1 @@ +source "$BR2_EXTERNAL_CLANG_COMPILER_RT_PATH/package/libfuzzer/Config.in" diff --git a/support/testing/tests/package/br2-external/clang-compiler-rt/external.desc b/support/testing/tests/package/br2-external/clang-compiler-rt/external.desc new file mode 100644 index 0000000..92df859 --- /dev/null +++ b/support/testing/tests/package/br2-external/clang-compiler-rt/external.desc @@ -0,0 +1 @@ +name: CLANG_COMPILER_RT diff --git a/support/testing/tests/package/br2-external/clang-compiler-rt/external.mk b/support/testing/tests/package/br2-external/clang-compiler-rt/external.mk new file mode 100644 index 0000000..6fa55c1 --- /dev/null +++ b/support/testing/tests/package/br2-external/clang-compiler-rt/external.mk @@ -0,0 +1 @@ +include $(sort $(wildcard $(BR2_EXTERNAL_CLANG_COMPILER_RT_PATH)/package/*/*.mk)) diff --git a/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/Config.in b/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/Config.in new file mode 100644 index 0000000..5af5d1b --- /dev/null +++ b/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/Config.in @@ -0,0 +1,7 @@ +config BR2_PACKAGE_LIBFUZZER + bool "libfuzzer" + help + This is a set of tests (benchmarks) for fuzzing + engines (fuzzers). + + https://github.com/google/fuzzer-test-suite diff --git a/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.hash b/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.hash new file mode 100644 index 0000000..6baf576 --- /dev/null +++ b/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.hash @@ -0,0 +1,2 @@ +sha256 c0addb4d7f0447fc9fd7c80e5721fafe4c137f29a8ebd94c5fef7e1d6a2c944c libfuzzer-64e4d9aa19a8d33b61882154addbf8419d7416e1.tar.gz +sha256 cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 LICENSE diff --git a/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.mk b/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.mk new file mode 100644 index 0000000..7d6f83b --- /dev/null +++ b/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.mk @@ -0,0 +1,24 @@ +################################################################################ +# +# libfuzzer +# +################################################################################ + +LIBFUZZER_VERSION = 64e4d9aa19a8d33b61882154addbf8419d7416e1 +LIBFUZZER_SITE = $(call github,google,fuzzer-test-suite,$(LIBFUZZER_VERSION)) +LIBFUZZER_LICENSE = Apache-2.0 +LIBFUZZER_LICENSE_FILES = LICENSE +LIBFUZZER_DEPENDENCIES = compiler-rt + +define LIBFUZZER_BUILD_CMDS + $(HOST_DIR)/bin/clang++ -mcpu=cortex-a53 --sysroot=$(STAGING_DIR) \ + -B $(HOST_DIR)/opt/ext-toolchain -fsanitize=address,fuzzer \ + $(@D)/tutorial/fuzz_me.cc \ + -o $(@D)/fuzz_me +endef + +define LIBFUZZER_INSTALL_TARGET_CMDS + $(INSTALL) -D -m 755 $(@D)/fuzz_me $(TARGET_DIR)/usr/bin/fuzz_me +endef + +$(eval $(generic-package)) diff --git a/support/testing/tests/package/test_clang.py b/support/testing/tests/package/test_clang.py new file mode 100644 index 0000000..84a2528 --- /dev/null +++ b/support/testing/tests/package/test_clang.py @@ -0,0 +1,46 @@ +import os + +import infra.basetest + +FUZZ_TIMEOUT = 120 + + +class TestClangCompilerRT(infra.basetest.BRTest): + br2_external = [infra.filepath("tests/package/br2-external/clang-compiler-rt")] + config = \ + """ + BR2_aarch64=y + BR2_TOOLCHAIN_EXTERNAL=y + BR2_TARGET_GENERIC_GETTY_PORT="ttyAMA0" + BR2_LINUX_KERNEL=y + BR2_LINUX_KERNEL_CUSTOM_VERSION=y + BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.16.7" + BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y + BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/aarch64-virt/linux.config" + BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL=y + BR2_PACKAGE_COMPILER_RT=y + BR2_PACKAGE_LLVM=y + BR2_TARGET_ROOTFS_CPIO=y + BR2_TARGET_ROOTFS_CPIO_GZIP=y + # BR2_TARGET_ROOTFS_TAR is not set + BR2_PACKAGE_LIBFUZZER=y + """ + + def login(self): + img = os.path.join(self.builddir, "images", "rootfs.cpio.gz") + kern = os.path.join(self.builddir, "images", "Image") + # Sanitizers overallocate memory and the minimum that seemed to work was 512MB + self.emulator.boot(arch="aarch64", + kernel=kern, + kernel_cmdline=["console=ttyAMA0"], + options=["-M", "virt", "-cpu", "cortex-a53", "-m", "512", "-initrd", img]) + self.emulator.login() + + def test_run(self): + self.login() + + # The test case verifies both that the application executes and that + # the symbolizer is working to decode the stack trace + cmd = "fuzz_me 2>&1 | grep _M_replace" + _, exit_code = self.emulator.run(cmd, FUZZ_TIMEOUT) + self.assertEqual(exit_code, 0) From patchwork Thu Nov 22 04:26:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Weber X-Patchwork-Id: 1001529 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=rockwellcollins.com Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 430mc00rShz9s8J for ; Thu, 22 Nov 2018 15:26:40 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 90D038843A; Thu, 22 Nov 2018 04:26:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dVLiFb5cxEhd; Thu, 22 Nov 2018 04:26:37 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id 63B9A88444; Thu, 22 Nov 2018 04:26:37 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id E5CB81C290C for ; Thu, 22 Nov 2018 04:26:26 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id E0B92883EB for ; Thu, 22 Nov 2018 04:26:26 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EkjPpBeSykaw for ; Thu, 22 Nov 2018 04:26:24 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from ch3vs02.rockwellcollins.com (ch3vs02.rockwellcollins.com [205.175.226.29]) by hemlock.osuosl.org (Postfix) with ESMTPS id AB18D87FF8 for ; Thu, 22 Nov 2018 04:26:23 +0000 (UTC) Received: from ofwch3n02.rockwellcollins.com (HELO dtulimr01.rockwellcollins.com) ([205.175.226.14]) by ch3vs02.rockwellcollins.com with ESMTP; 21 Nov 2018 22:26:23 -0600 X-Received: from largo.rockwellcollins.com (unknown [192.168.140.76]) by dtulimr01.rockwellcollins.com (Postfix) with ESMTP id 95FBF603A3; Wed, 21 Nov 2018 22:26:22 -0600 (CST) From: Matt Weber To: buildroot@buildroot.org Date: Wed, 21 Nov 2018 22:26:16 -0600 Message-Id: <1542860777-57909-6-git-send-email-matthew.weber@rockwellcollins.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1542860777-57909-1-git-send-email-matthew.weber@rockwellcollins.com> References: <1542860777-57909-1-git-send-email-matthew.weber@rockwellcollins.com> Subject: [Buildroot] [PATCH v4 6/7] llvm/clang: add note about version bumping together X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ricardo Martincoski MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Signed-off-by: Matthew Weber Acked-by: Romain Naour Cc: Ricardo Martincoski --- Changes v1 -> v2 - None v2 -> v3 [Ricardo - Added compiler-rt to list of items to bump. I originaly had kept this patch independent of the compiler-rt series but since compiler-rt is first in the series, this one may as well include it in the statements v4 - None --- package/clang/clang.mk | 1 + package/llvm/llvm.mk | 1 + 2 files changed, 2 insertions(+) diff --git a/package/clang/clang.mk b/package/clang/clang.mk index f9d4ae3..461d5f9 100644 --- a/package/clang/clang.mk +++ b/package/clang/clang.mk @@ -4,6 +4,7 @@ # ################################################################################ +# LLVM, Clang and Compiler-RT should be version bumped together CLANG_VERSION = 7.0.0 CLANG_SITE = http://llvm.org/releases/$(CLANG_VERSION) CLANG_SOURCE = cfe-$(CLANG_VERSION).src.tar.xz diff --git a/package/llvm/llvm.mk b/package/llvm/llvm.mk index 79477f0..14f317d 100644 --- a/package/llvm/llvm.mk +++ b/package/llvm/llvm.mk @@ -4,6 +4,7 @@ # ################################################################################ +# LLVM, Clang and Compiler-RT should be version bumped together LLVM_VERSION = 7.0.0 LLVM_SITE = http://llvm.org/releases/$(LLVM_VERSION) LLVM_SOURCE = llvm-$(LLVM_VERSION).src.tar.xz From patchwork Thu Nov 22 04:26:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Weber X-Patchwork-Id: 1001528 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=rockwellcollins.com Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 430mbw4lgFz9s29 for ; Thu, 22 Nov 2018 15:26:36 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 0034D8843E; Thu, 22 Nov 2018 04:26:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xoHfTD4ZfFEg; Thu, 22 Nov 2018 04:26:28 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id 9686088423; Thu, 22 Nov 2018 04:26:28 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id A96FE1C290C for ; Thu, 22 Nov 2018 04:26:24 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id A6FC787336 for ; Thu, 22 Nov 2018 04:26:24 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mxhZXZ9UzJk1 for ; Thu, 22 Nov 2018 04:26:23 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from da1vs04.rockwellcollins.com (da1vs04.rockwellcollins.com [205.175.227.52]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 5F86C872FF for ; Thu, 22 Nov 2018 04:26:23 +0000 (UTC) Received: from ofwda1n02.rockwellcollins.com (HELO dtulimr01.rockwellcollins.com) ([205.175.227.14]) by da1vs04.rockwellcollins.com with ESMTP; 21 Nov 2018 22:26:23 -0600 X-Received: from largo.rockwellcollins.com (unknown [192.168.140.76]) by dtulimr01.rockwellcollins.com (Postfix) with ESMTP id BC431603CA; Wed, 21 Nov 2018 22:26:22 -0600 (CST) From: Matt Weber To: buildroot@buildroot.org Date: Wed, 21 Nov 2018 22:26:17 -0600 Message-Id: <1542860777-57909-7-git-send-email-matthew.weber@rockwellcollins.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1542860777-57909-1-git-send-email-matthew.weber@rockwellcollins.com> References: <1542860777-57909-1-git-send-email-matthew.weber@rockwellcollins.com> Subject: [Buildroot] [PATCH v4 7/7] package/llvm: disable libxml2 X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Romain Naour MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" libxml2 is needed during Windows builds to populate the COFF file manifest info If left enabled, a dependency would need to be added and paths set for libxml2 include and library paths. This isn't require at this point for Linux builds. Cc: Romain Naour Signed-off-by: Matthew Weber --- This change was discovered during testing of compiler-rt patchset. This patch could be independently applied. Romain, hopefully you are able to verify this resolves the path posioning you observed with the target build and libxml2. v4 - New patch --- package/llvm/llvm.mk | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/package/llvm/llvm.mk b/package/llvm/llvm.mk index 14f317d..ef54f10 100644 --- a/package/llvm/llvm.mk +++ b/package/llvm/llvm.mk @@ -123,6 +123,11 @@ HOST_LLVM_CONF_OPTS += -DLLVM_ENABLE_ZLIB=ON HOST_LLVM_DEPENDENCIES += host-zlib LLVM_CONF_OPTS += -DLLVM_ENABLE_ZLIB=OFF +# libxml2 can be disabled as it is used for LLVM Windows builds where COFF +# files include manifest info +HOST_LLVM_CONF_OPTS += -DLLVM_ENABLE_LIBXML2=OFF +LLVM_CONF_OPTS += -DLLVM_ENABLE_LIBXML2=OFF + # We don't use llvm for static only build, so enable PIC HOST_LLVM_CONF_OPTS += -DLLVM_ENABLE_PIC=ON LLVM_CONF_OPTS += -DLLVM_ENABLE_PIC=ON