From patchwork Wed Mar 18 15:50:40 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Andr=C3=A9_Erdmann?= X-Patchwork-Id: 451514 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from fraxinus.osuosl.org (fraxinus.osuosl.org [140.211.166.137]) by ozlabs.org (Postfix) with ESMTP id 1DA1D1400B6 for ; Thu, 19 Mar 2015 02:51:35 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=fail reason="verification failed; unprotected key" header.d=mailerd.de header.i=@mailerd.de header.b=cLHCprOO; dkim-adsp=none (unprotected policy); dkim-atps=neutral Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 33A02A373C; Wed, 18 Mar 2015 15:51:33 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m37UYDQYAUR9; Wed, 18 Mar 2015 15:51:31 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 27C4EA3723; Wed, 18 Mar 2015 15:51:19 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (hemlock.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 129CC1C2439 for ; Wed, 18 Mar 2015 15:51:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 0F8B79551C for ; Wed, 18 Mar 2015 15:51:10 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Rd3fh2wTm9Dh for ; Wed, 18 Mar 2015 15:51:08 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wi0-f180.google.com (mail-wi0-f180.google.com [209.85.212.180]) by hemlock.osuosl.org (Postfix) with ESMTPS id 3F5C79551D for ; Wed, 18 Mar 2015 15:51:08 +0000 (UTC) Received: by wifj2 with SMTP id j2so43570099wif.1 for ; Wed, 18 Mar 2015 08:51:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mailerd.de; s=mails; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; bh=0qIb2DiUR0NW/AF89AJvKOrZiKu29LxY0f6Oc3OHuGA=; b=cLHCprOOgm/fQHRbBDzxznJVXFIHlncx5dbLW+U4uLu+7FeAwNxE6FddZoV7QSGlmi CzRrTihrmnXVlEEoQwul3Z7S4iGsU8kXa8G+Jgtt9M6horDBQJnQaJZHvYCKNvW2qPcA YG1aQYpzd25+/fGQa86Msc5m2k5fKzMLubsu4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-type :content-transfer-encoding; bh=0qIb2DiUR0NW/AF89AJvKOrZiKu29LxY0f6Oc3OHuGA=; b=ceCmy6LhsJjINlS9F4ewutEdBbcQarJz7xjoqFyp6dUM+2QXHxaKUOyh9ir+PTvRrz s0QYKVFuYEYRRhSOwE/FIy+Kr7ry2g/B9rbjxxX/uaSYzz9V1BA82eLCNuyZ8djhaj98 wBEyP9WeZOLgFdGT/eWdgiA97BoTZqBrCCfHXrhQSBRAhTvX4uD07Z4yTHv72QciHPQm 3k0DOp+RMruJJ9aAbuOez17T9RGAd4oZqhjz4fRCZRAKD5ya321CpCgZzORm8SVDu6GQ mrK+eiL+oEmpD0k7srSV7irwuhvOU445TK7/TwUuLBotWcqVGK6fMibUFx2/1dt+TGga W4DQ== X-Gm-Message-State: ALoCoQmO2pOOxCTLjjknFF18RjbE2HEnk7p3INPRtojVwq7O4+les77iljrhwLx2CKGMgMMEpELU X-Received: by 10.180.74.230 with SMTP id x6mr8123333wiv.58.1426693866910; Wed, 18 Mar 2015 08:51:06 -0700 (PDT) Received: from localhost.localdomain (dslb-092-074-053-028.092.074.pools.vodafone-ip.de. [92.74.53.28]) by mx.google.com with ESMTPSA id u16sm25024794wjr.5.2015.03.18.08.51.05 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 18 Mar 2015 08:51:06 -0700 (PDT) From: =?UTF-8?q?Andr=C3=A9=20Erdmann?= To: buildroot@buildroot.org Date: Wed, 18 Mar 2015 16:50:40 +0100 Message-Id: <1426693843-28792-8-git-send-email-dywi@mailerd.de> X-Mailer: git-send-email 2.3.2 In-Reply-To: <1426693843-28792-1-git-send-email-dywi@mailerd.de> References: <1426693843-28792-1-git-send-email-dywi@mailerd.de> MIME-Version: 1.0 Cc: thomas.petazzoni@free-electrons.com Subject: [Buildroot] [PATCH v2 07/10] autobuild-run: move check_requirements() to SystemInfo X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" * build up the list of (additional) essential programs in main() * can check for optional progs in fixup_config() without having to list them in sysinfo * it'd be quite easy to dump the prog list to the logfile (for name, have_prog in sysinfo.progs.items(): ...) Signed-off-by: André Erdmann --- scripts/autobuild-run | 123 ++++++++++++++++++++++++++++++-------------------- 1 file changed, 73 insertions(+), 50 deletions(-) diff --git a/scripts/autobuild-run b/scripts/autobuild-run index fd294d4..409570d 100755 --- a/scripts/autobuild-run +++ b/scripts/autobuild-run @@ -47,8 +47,6 @@ # - Instead of excluding all configurations that have # BR2_PACKAGE_CLASSPATH=y, improve the script to detect whether the # necessary host machine requirements are there to build classpath. -# - Integrate method check-requirements with the SysInfo class, distinghuishing -# between required and optional dependencies. # - Extend required dependencies to subversion, mercurial, cpio, wget, python, # etc. # - Detect selection of multiple virtual package providers and don't consider it @@ -166,51 +164,72 @@ def check_version(): print("ERROR: script version too old, please upgrade.") sys.exit(1) -def has_prog(name, flags=os.X_OK, env=os.environ): - if not name or name[0] == os.sep: raise ValueError(name) - - prog_path = env.get("PATH", None) - # for windows compatibility, we'd need to take PATHEXT into account - - if prog_path: - for prog_dir in filter(None, prog_path.split(os.pathsep)): - # os.join() not necessary: non-empty prog_dir and name[0] != os.sep - prog = prog_dir + os.sep + name - if os.access(prog, flags): - return True - # -- - return False - -def check_requirements(upload=False): - needed_progs = ["make", "git", "gcc", "timeout"] - missing_requirements = False - - if upload: - needed_progs.append("curl") - - for prog in needed_progs: - if not has_prog(prog): - print("ERROR: your system lacks the '%s' program" % prog) - missing_requirements = True - - if missing_requirements: - sys.exit(1) - class SystemInfo: - def __init__(self): - devnull = open(os.devnull, "w") - # _grep_gcj :: str -> int - _grep_gcj = lambda prog: \ - subprocess.call("%s -version | grep gcj" % prog, shell=True, - stdout=devnull, stderr=devnull) - - self.has_bzr = has_prog("bzr") - - self.has_java = has_prog("java") and _grep_gcj("java") == 1 + DEFAULT_NEEDED_PROGS = ["make", "git", "gcc", "timeout"] + DEFAULT_OPTIONAL_PROGS = ["bzr", "java", "javac", "jar"] - self.has_javac = has_prog("javac") and _grep_gcj("javac") == 1 + def __init__(self): + self.needed_progs = list(self.__class__.DEFAULT_NEEDED_PROGS) + self.optional_progs = list(self.__class__.DEFAULT_OPTIONAL_PROGS) + self.progs = {} + + def find_prog(self, name, flags=os.X_OK, env=os.environ): + if not name or name[0] == os.sep: raise ValueError(name) + + prog_path = env.get("PATH", None) + # for windows compatibility, we'd need to take PATHEXT into account + + if prog_path: + for prog_dir in filter(None, prog_path.split(os.pathsep)): + # os.join() not necessary: non-empty prog_dir + # and name[0] != os.sep + prog = prog_dir + os.sep + name + if os.access(prog, flags): + return prog + # -- + return None - self.has_jar = has_prog("jar") + def has(self, prog): + """Checks whether a program is available. + Lazily evaluates missing entries. + + Returns: None if prog not found, else path to the program [evaluates to True] + """ + try: + return self.progs[prog] + except KeyError: + pass + + have_it = self.find_prog(prog) + # java[c] needs special care + if have_it and prog in ('java', 'javac'): + with open(os.devnull, "w") as devnull: + if subprocess.call("%s -version | grep gcj" % prog, shell=True, + stdout=devnull, stderr=devnull) != 1: + have_it = False + # -- + self.progs[prog] = have_it + return have_it + + def check_requirements(self): + """Checks program dependencies. + + Returns: True if all mandatory programs are present, else False. + """ + do_check_has_prog = self.has + + missing_requirements = False + for prog in self.needed_progs: + if not do_check_has_prog(prog): + print("ERROR: your system lacks the '%s' program" % prog) + missing_requirements = True + + # check optional programs here, + # else they'd get checked by each worker instance + for prog in self.optional_progs: + do_check_has_prog(prog) + + return not missing_requirements def get_toolchain_configs(): """Fetch and return the possible toolchain configurations @@ -338,14 +357,14 @@ def fixup_config(**kwargs): if "BR2_PACKAGE_MROUTED=y\n" in configlines and \ "BR2_TOOLCHAIN_USES_UCLIBC=y\n" in configlines: configlines.remove("BR2_PACKAGE_MROUTED=y\n") - if "BR2_NEEDS_HOST_JAVA=y\n" in configlines and not sysinfo.has_java: + if "BR2_NEEDS_HOST_JAVA=y\n" in configlines and not sysinfo.has("java"): return False - if "BR2_NEEDS_HOST_JAVAC=y\n" in configlines and not sysinfo.has_javac: + if "BR2_NEEDS_HOST_JAVAC=y\n" in configlines and not sysinfo.has("javac"): return False - if "BR2_NEEDS_HOST_JAR=y\n" in configlines and not sysinfo.has_jar: + if "BR2_NEEDS_HOST_JAR=y\n" in configlines and not sysinfo.has("jar"): return False # python-nfc needs bzr - if 'BR2_PACKAGE_PYTHON_NFC=y\n' in configlines and not sysinfo.has_bzr: + if 'BR2_PACKAGE_PYTHON_NFC=y\n' in configlines and not sysinfo.has("bzr"): return False # The ctng toolchain is affected by PR58854 if 'BR2_PACKAGE_LTTNG_TOOLS=y\n' in configlines and \ @@ -771,11 +790,15 @@ def main(): # explicitly on None. upload = (args['--http-login'] is not None) \ and (args['--http-password'] is not None) - check_requirements(upload) - if not upload: + if upload: + sysinfo.needed_progs.append("curl") + else: print("WARN: due to the lack of http login/password details, results will not be submitted") print("WARN: tarballs of results will be kept locally only") + if not sysinfo.check_requirements(): + sys.exit(1) + def sigterm_handler(signum, frame): """Kill all children"""