From patchwork Fri Jun 21 20:46:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1950970 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=CppBRy/V; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W5Tth4TCvz20Wb for ; Sat, 22 Jun 2024 06:47:08 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id E32C588571; Fri, 21 Jun 2024 22:46:45 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="CppBRy/V"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id BD7D98804F; Fri, 21 Jun 2024 22:46:44 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd36.google.com (mail-io1-xd36.google.com [IPv6:2607:f8b0:4864:20::d36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id A5D24884FF for ; Fri, 21 Jun 2024 22:46:41 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd36.google.com with SMTP id ca18e2360f4ac-7eba486df76so75725039f.0 for ; Fri, 21 Jun 2024 13:46:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719002800; x=1719607600; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=axCLXmh+685z5im5qNR5XYL9I9hRy3CKibhRPAAn6kI=; b=CppBRy/VFzPdEZgY738yPzLbn/eaV/eecKzSQCmz1uVXReDVjVM5+e1RvdXzgp6dNF qaUGvvYwC/zGU84Kr7zXZsnEAcPVNAKlJaLKGC/77XtGtvR04ARdwHVg3pDh1KGPu0c5 rhG28IDDHUgLwhoNOKLQGxso3cJpjViw0DSc0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719002800; x=1719607600; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=axCLXmh+685z5im5qNR5XYL9I9hRy3CKibhRPAAn6kI=; b=rUWkEVMFOYMoQ2sIkPWifm+ec+6tmZVLwkrxTcXas+32c3F1NAuHgppQ/n//p5DDCs v7GOtaaQW9W8CVMWtMPb1wIlDLph0m7fleixFmUkE+4bPVjr0Mxqn3Fc4ms8jdov0Qq/ G44HqsHVRKCnc0QrvXFvBvfmM2UBUlIEaZjM5UjaWKFs9XYPK107T62mh/C1s+N7j2pH bBRFD7bzrP5i/ko6uNoIAikOU5X/Zgta+4LcFLJGUu9boDTjYQ7d8P1phiMukk0iXIfW h1wHr6udDW7u+RsGtq6aIidOKuIwlLdbtxvF5mKxHYdqSfqe7qZoGocTglcA1wf4bTN6 6lPw== X-Gm-Message-State: AOJu0Yznl5ymPHOt3X2YkpGDMeLF3IPAwc8TPTkiRYOZk3ESAA4EbSMz nyvCC4hBk/32MBFuqm+j00t+Z2vT0qdOm3NWUhcmFsjLMC9hDOkRSxcfy/sIRDipnoaZbUiXqyf bNg== X-Google-Smtp-Source: AGHT+IHYF99e4r/O9hgBQEhk/RoHBotekAlwpX9k5CxTnoF2gB8veURsbLIXqqsqxEXxWB4vMTPajw== X-Received: by 2002:a05:6602:6b84:b0:7eb:eeba:5e83 with SMTP id ca18e2360f4ac-7f39abd7a16mr48431039f.9.1719002800266; Fri, 21 Jun 2024 13:46:40 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f39203ec51sm50698939f.53.2024.06.21.13.46.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jun 2024 13:46:40 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass , Alper Nebi Yasak , Christian Taedcke , Heinrich Schuchardt , Lukas Funke , Neha Malcom Francis , Peng Fan , Philippe Reynes , Stefan Herbrechtsmeier , Sughosh Ganu Subject: [PATCH 01/11] binman: efi: Correct entry docs Date: Fri, 21 Jun 2024 14:46:22 -0600 Message-Id: <20240621204632.2706813-2-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240621204632.2706813-1-sjg@chromium.org> References: <20240621204632.2706813-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Somehow the class documentation has got out of sync with the generated entries.rst file. Regenerating it causes errors, so correct these and regenerate the entries.rst file. Signed-off-by: Simon Glass Fixes: 809f28e7213 ("binman: capsule: Use dumped capsule header...") --- tools/binman/entries.rst | 58 ++++++++++++------------- tools/binman/etype/efi_capsule.py | 40 ++++++++--------- tools/binman/etype/efi_empty_capsule.py | 22 +++++----- 3 files changed, 61 insertions(+), 59 deletions(-) diff --git a/tools/binman/entries.rst b/tools/binman/entries.rst index 254afe76074..dc42a9cb97d 100644 --- a/tools/binman/entries.rst +++ b/tools/binman/entries.rst @@ -470,11 +470,11 @@ updating the EC on startup via software sync. .. _etype_efi_capsule: -Entry: capsule: Entry for generating EFI Capsule files ------------------------------------------------------- +Entry: efi-capsule: Generate EFI capsules +----------------------------------------- -The parameters needed for generation of the capsules can be provided -as properties in the entry. +The parameters needed for generation of the capsules can +be provided as properties in the entry. Properties / Entry arguments: - image-index: Unique number for identifying corresponding @@ -495,9 +495,9 @@ Properties / Entry arguments: file. Mandatory property for generating signed capsules. - oem-flags - OEM flags to be passed through capsule header. - Since this is a subclass of Entry_section, all properties of the parent - class also apply here. Except for the properties stated as mandatory, the - rest of the properties are optional. +Since this is a subclass of Entry_section, all properties of the parent +class also apply here. Except for the properties stated as mandatory, the +rest of the properties are optional. For more details on the description of the capsule format, and the capsule update functionality, refer Section 8.5 and Chapter 23 in the `UEFI @@ -510,17 +510,17 @@ provided as a subnode of the capsule entry. A typical capsule entry node would then look something like this:: capsule { - type = "efi-capsule"; - image-index = <0x1>; - /* Image GUID for testing capsule update */ - image-guid = SANDBOX_UBOOT_IMAGE_GUID; - hardware-instance = <0x0>; - private-key = "path/to/the/private/key"; - public-key-cert = "path/to/the/public-key-cert"; - oem-flags = <0x8000>; + type = "efi-capsule"; + image-index = <0x1>; + /* Image GUID for testing capsule update */ + image-guid = SANDBOX_UBOOT_IMAGE_GUID; + hardware-instance = <0x0>; + private-key = "path/to/the/private/key"; + public-key-cert = "path/to/the/public-key-cert"; + oem-flags = <0x8000>; - u-boot { - }; + u-boot { + }; }; In the above example, the capsule payload is the U-Boot image. The @@ -534,8 +534,8 @@ payload using the blob-ext subnode. .. _etype_efi_empty_capsule: -Entry: efi-empty-capsule: Entry for generating EFI Empty Capsule files ----------------------------------------------------------------------- +Entry: efi-empty-capsule: Generate EFI empty capsules +----------------------------------------------------- The parameters needed for generation of the empty capsules can be provided as properties in the entry. @@ -551,22 +551,22 @@ update functionality, refer Section 8.5 and Chapter 23 in the `UEFI specification`_. For more information on the empty capsule, refer the sections 2.3.2 and 2.3.3 in the `Dependable Boot specification`_. -A typical accept empty capsule entry node would then look something -like this:: +A typical accept empty capsule entry node would then look something like +this:: empty-capsule { - type = "efi-empty-capsule"; - /* GUID of the image being accepted */ - image-type-id = SANDBOX_UBOOT_IMAGE_GUID; - capsule-type = "accept"; + type = "efi-empty-capsule"; + /* GUID of image being accepted */ + image-type-id = SANDBOX_UBOOT_IMAGE_GUID; + capsule-type = "accept"; }; -A typical revert empty capsule entry node would then look something -like this:: +A typical revert empty capsule entry node would then look something like +this:: empty-capsule { - type = "efi-empty-capsule"; - capsule-type = "revert"; + type = "efi-empty-capsule"; + capsule-type = "revert"; }; The empty capsules do not have any input payload image. diff --git a/tools/binman/etype/efi_capsule.py b/tools/binman/etype/efi_capsule.py index e3203717822..751f654bf31 100644 --- a/tools/binman/etype/efi_capsule.py +++ b/tools/binman/etype/efi_capsule.py @@ -36,23 +36,23 @@ class Entry_efi_capsule(Entry_section): be provided as properties in the entry. Properties / Entry arguments: - - image-index: Unique number for identifying corresponding - payload image. Number between 1 and descriptor count, i.e. - the total number of firmware images that can be updated. Mandatory - property. - - image-guid: Image GUID which will be used for identifying the - updatable image on the board. Mandatory property. - - hardware-instance: Optional number for identifying unique - hardware instance of a device in the system. Default value of 0 - for images where value is not to be used. - - fw-version: Value of image version that can be put on the capsule - through the Firmware Management Protocol(FMP) header. - - monotonic-count: Count used when signing an image. - - private-key: Path to PEM formatted .key private key file. Mandatory - property for generating signed capsules. - - public-key-cert: Path to PEM formatted .crt public key certificate - file. Mandatory property for generating signed capsules. - - oem-flags - OEM flags to be passed through capsule header. + - image-index: Unique number for identifying corresponding + payload image. Number between 1 and descriptor count, i.e. + the total number of firmware images that can be updated. Mandatory + property. + - image-guid: Image GUID which will be used for identifying the + updatable image on the board. Mandatory property. + - hardware-instance: Optional number for identifying unique + hardware instance of a device in the system. Default value of 0 + for images where value is not to be used. + - fw-version: Value of image version that can be put on the capsule + through the Firmware Management Protocol(FMP) header. + - monotonic-count: Count used when signing an image. + - private-key: Path to PEM formatted .key private key file. Mandatory + property for generating signed capsules. + - public-key-cert: Path to PEM formatted .crt public key certificate + file. Mandatory property for generating signed capsules. + - oem-flags - OEM flags to be passed through capsule header. Since this is a subclass of Entry_section, all properties of the parent class also apply here. Except for the properties stated as mandatory, the @@ -66,9 +66,9 @@ class Entry_efi_capsule(Entry_section): properties in the entry. The payload to be used in the capsule is to be provided as a subnode of the capsule entry. - A typical capsule entry node would then look something like this + A typical capsule entry node would then look something like this:: - capsule { + capsule { type = "efi-capsule"; image-index = <0x1>; /* Image GUID for testing capsule update */ @@ -80,7 +80,7 @@ class Entry_efi_capsule(Entry_section): u-boot { }; - }; + }; In the above example, the capsule payload is the U-Boot image. The capsule entry would read the contents of the payload and put them diff --git a/tools/binman/etype/efi_empty_capsule.py b/tools/binman/etype/efi_empty_capsule.py index 064bf9a77f0..1d99fbfb3bb 100644 --- a/tools/binman/etype/efi_empty_capsule.py +++ b/tools/binman/etype/efi_empty_capsule.py @@ -19,31 +19,33 @@ class Entry_efi_empty_capsule(Entry_section): be provided as properties in the entry. Properties / Entry arguments: - - image-guid: Image GUID which will be used for identifying the - updatable image on the board. Mandatory for accept capsule. - - capsule-type - String to indicate type of capsule to generate. Valid - values are 'accept' and 'revert'. + - image-guid: Image GUID which will be used for identifying the + updatable image on the board. Mandatory for accept capsule. + - capsule-type - String to indicate type of capsule to generate. Valid + values are 'accept' and 'revert'. For more details on the description of the capsule format, and the capsule update functionality, refer Section 8.5 and Chapter 23 in the `UEFI specification`_. For more information on the empty capsule, refer the sections 2.3.2 and 2.3.3 in the `Dependable Boot specification`_. - A typical accept empty capsule entry node would then look something like this + A typical accept empty capsule entry node would then look something like + this:: - empty-capsule { + empty-capsule { type = "efi-empty-capsule"; /* GUID of image being accepted */ image-type-id = SANDBOX_UBOOT_IMAGE_GUID; capsule-type = "accept"; - }; + }; - A typical revert empty capsule entry node would then look something like this + A typical revert empty capsule entry node would then look something like + this:: - empty-capsule { + empty-capsule { type = "efi-empty-capsule"; capsule-type = "revert"; - }; + }; The empty capsules do not have any input payload image. From patchwork Fri Jun 21 20:46:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1950969 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=jn1ooAsf; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W5TtT3G2hz20Wb for ; Sat, 22 Jun 2024 06:46:57 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 853E3884FF; Fri, 21 Jun 2024 22:46:45 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="jn1ooAsf"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 9540988059; Fri, 21 Jun 2024 22:46:44 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.2 Received: from mail-io1-xd33.google.com (mail-io1-xd33.google.com [IPv6:2607:f8b0:4864:20::d33]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 7EE678845A for ; Fri, 21 Jun 2024 22:46:42 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd33.google.com with SMTP id ca18e2360f4ac-7eb895539e3so94105639f.2 for ; Fri, 21 Jun 2024 13:46:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719002801; x=1719607601; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BNrnuY9yrBYlnFhzQC8UMGqRbSpLi2JGWWqh3sxbDgE=; b=jn1ooAsfy3xGYHz2ENYlCxjPRKu/pubrTjrN50EbyPVlysGGCVB6NG1FPFiv+LW3DK IoLwuaw2gTKEMUr5KyKa5IP4OsxL3Xu/Tf88VCEYcqsx+wMqcXetdkr6ISoXx+xa/dpu D1Nrt0tKB8C9PJpXp80E/vPDuufsljZ00f+Yc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719002801; x=1719607601; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BNrnuY9yrBYlnFhzQC8UMGqRbSpLi2JGWWqh3sxbDgE=; b=e8YM87PL9MmMXDSvcNg8NqgWDNvBXso/MP5/RjZ6oKd0kNw0k/uaCqrt7FnOLbSva5 mk1pO4k+5We7E4DVK7z8bHdWQg5wcQUQbMU6CnpFynAouviX15ufVCBT2btIKRHhLYOY /vOy9bXt2hYp1OCAGrfloWILhT0Fr5M37awbOxt6pVEzMqrQVMiGqgfomYdsDyh0HcmE bJXccj5LSBYHV6YemNzjY8NL51w9ysyOa0nTwGrXZ9vx9/OmYvneuW9NyF55GiVBzA0Y /P78fRkw5ic+jkGM/5a8ZTwK1c3/5BY0bKDc0AaK2vjzbItD1sluYkTifsEnp2X/N6ZD WJ6w== X-Gm-Message-State: AOJu0YxPlUZpcLDZ1bCHjMWusb7N7pZLdOyLUMmRM9j0tpAd59mMou/A sJ45A2p6gT1ZsdtLHrwULUCXTTblKrreeehs7P1mOPt9vXbfak0t9p0M7JkZg0V7IG1sASkCLCV auw== X-Google-Smtp-Source: AGHT+IElFlRKmNHeIxXI4O74j575WWC0LLSCzDyAVlf66jHDwChDHMAAbuwtHr9IxkVW2H/S5y1vRQ== X-Received: by 2002:a05:6602:6c10:b0:7eb:7f2e:5b3a with SMTP id ca18e2360f4ac-7f13edb2453mr1153547639f.2.1719002801140; Fri, 21 Jun 2024 13:46:41 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f39203ec51sm50698939f.53.2024.06.21.13.46.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jun 2024 13:46:40 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass , Alper Nebi Yasak , Christian Taedcke , Heinrich Schuchardt , Lukas Funke , Marek Vasut , Neha Malcom Francis , Peng Fan , Philippe Reynes , Stefan Herbrechtsmeier , Sughosh Ganu , Vignesh Raghavendra Subject: [PATCH 02/11] binman: Regenerate nxp docs Date: Fri, 21 Jun 2024 14:46:23 -0600 Message-Id: <20240621204632.2706813-3-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240621204632.2706813-1-sjg@chromium.org> References: <20240621204632.2706813-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Regenerate the entries.rst file to include this recent addition. Note that more docs are needed here, to actually describe the entry type. Note also that the entry type needs Binman tests added. Signed-off-by: Simon Glass --- tools/binman/entries.rst | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/tools/binman/entries.rst b/tools/binman/entries.rst index dc42a9cb97d..1b9b868e33f 100644 --- a/tools/binman/entries.rst +++ b/tools/binman/entries.rst @@ -1521,6 +1521,28 @@ byte. +.. _etype_nxp_imx8mcst: + +Entry: nxp-imx8mcst: NXP i.MX8M CST .cfg file generator and cst invoker +----------------------------------------------------------------------- + +Properties / Entry arguments: + - nxp,loader-address - loader address (SPL text base) + + + +.. _etype_nxp_imx8mimage: + +Entry: nxp-imx8mimage: NXP i.MX8M imx8mimage .cfg file generator and mkimage invoker +------------------------------------------------------------------------------------ + +Properties / Entry arguments: + - nxp,boot-from - device to boot from (e.g. 'sd') + - nxp,loader-address - loader address (SPL text base) + - nxp,rom-version - BootROM version ('2' for i.MX8M Nano and Plus) + + + .. _etype_opensbi: Entry: opensbi: RISC-V OpenSBI fw_dynamic blob From patchwork Fri Jun 21 20:46:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1950971 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=ZUQV4lnH; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W5Tts5B93z20Wb for ; Sat, 22 Jun 2024 06:47:17 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 4D0BF88581; Fri, 21 Jun 2024 22:46:48 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="ZUQV4lnH"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 764908855C; Fri, 21 Jun 2024 22:46:46 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 541D18854E for ; Fri, 21 Jun 2024 22:46:43 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd30.google.com with SMTP id ca18e2360f4ac-7ebde93bf79so94264439f.1 for ; Fri, 21 Jun 2024 13:46:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719002802; x=1719607602; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JVYxCJh9dAyl2A1AAuXvhC+GANWxpmz1PU97AEow9BE=; b=ZUQV4lnHXfPI2MjXasBVZ7ZC71DVciMvNl+e8OxqMTmlwqKDepm8IpMD61FwiMOTR8 C0AroJur9Wx/2rIukfvkenUS4n6HDc3E2QYPJUL4r+/vkGWDl95adpLnDkxPt6aJCJeh y5KTDjwbyyh+D8107BGJHp7H0bRkD2wDLFP3k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719002802; x=1719607602; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JVYxCJh9dAyl2A1AAuXvhC+GANWxpmz1PU97AEow9BE=; b=OrVKY4uRZ5zFUkpSzmm3hZoK+6h193TpwqIh7Ka+VYis+S66jBeRhbBTzOvYEi5+oy osysYG+EhAq7TsXRJ1BV+3lM24IrasUZ3TL6o4rEcWG31WZrR4E3GOW0LQ5RmYBLdHxF C4eC9vtEyG39845anLN8dQJT+5fZQs+v55r8GO1HYgpGptiQ9eWn+RS4+mPeCLkaPZDD UJCT+aNYlvFGhLxdcB6lbXZNLYSaF8v65sgaa7MHF2/ZZdS7Ymgg8jru0wNMtkOkX6/y J6swqqKUjeNOBowZFkLIAVYkLtj9FnoHYvRfLFhOVobwfZ2I7Cd+6dB2YSqCSJSbOcQ8 fvGA== X-Gm-Message-State: AOJu0YyyzTlHJdb4ydRD+I6+n4SO5yiUs756+X5sWxesRG93rSfIVr3A 3YeVXR2P9PiblwHfd9A77DKgf75Z+RBtVq2hnve1JvsyBXuGMKPxFfrD80o5iKljqBQ3+o2KsMd MSA== X-Google-Smtp-Source: AGHT+IGAJKP8trYAd59YS+qB0inkFocS3RLft526QANIQ9og5uq4Zn/NYWjx5FnNWgiBiy9gqvuB5g== X-Received: by 2002:a05:6602:1347:b0:7eb:6d0a:613e with SMTP id ca18e2360f4ac-7f13ee2e374mr1052708139f.7.1719002801938; Fri, 21 Jun 2024 13:46:41 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f39203ec51sm50698939f.53.2024.06.21.13.46.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jun 2024 13:46:41 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass , Alper Nebi Yasak , Andrew Davis , Christian Taedcke , Lukas Funke , Manorit Chawdhry , Neha Malcom Francis , Peng Fan , Philippe Reynes , Stefan Herbrechtsmeier , Sughosh Ganu , Vignesh Raghavendra Subject: [PATCH 03/11] binman: ti: Regenerate entry docs Date: Fri, 21 Jun 2024 14:46:24 -0600 Message-Id: <20240621204632.2706813-4-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240621204632.2706813-1-sjg@chromium.org> References: <20240621204632.2706813-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Correct formatting errors in the documentation. Regenerate the entries.rst file to include this recent addition. Signed-off-by: Simon Glass --- tools/binman/entries.rst | 35 +++++++++++++++++++++++++ tools/binman/etype/ti_secure.py | 45 +++++++++++++++++---------------- 2 files changed, 58 insertions(+), 22 deletions(-) diff --git a/tools/binman/entries.rst b/tools/binman/entries.rst index 1b9b868e33f..bdda1ef2855 100644 --- a/tools/binman/entries.rst +++ b/tools/binman/entries.rst @@ -1951,6 +1951,12 @@ Properties / Entry arguments: - content: List of phandles to entries to sign - keyfile: Filename of file containing key to sign binary with - sha: Hash function to be used for signing + - auth-in-place: This is an integer field that contains two pieces + of information: + + - Lower Byte - Remains 0x02 as per our use case + ( 0x02: Move the authenticated binary back to the header ) + - Upper Byte - The Host ID of the core owning the firewall Output files: - input. - input file passed to openssl @@ -1959,6 +1965,35 @@ Output files: - cert. - output file generated by openssl (which is used as the entry contents) +Depending on auth-in-place information in the inputs, we read the +firewall nodes that describe the configurations of firewall that TIFS +will be doing after reading the certificate. + +The syntax of the firewall nodes are as such:: + + firewall-257-0 { + id = <257>; /* The ID of the firewall being configured */ + region = <0>; /* Region number to configure */ + + control = /* The control register */ + <(FWCTRL_EN | FWCTRL_LOCK | FWCTRL_BG | FWCTRL_CACHE)>; + + permissions = /* The permission registers */ + <((FWPRIVID_ALL << FWPRIVID_SHIFT) | + FWPERM_SECURE_PRIV_RWCD | + FWPERM_SECURE_USER_RWCD | + FWPERM_NON_SECURE_PRIV_RWCD | + FWPERM_NON_SECURE_USER_RWCD)>; + + /* More defines can be found in k3-security.h */ + + start_address = /* The Start Address of the firewall */ + <0x0 0x0>; + end_address = /* The End Address of the firewall */ + <0xff 0xffffffff>; + }; + + openssl signs the provided data, using the TI templated config file and writes the signature in this entry. This allows verification that the data is genuine. diff --git a/tools/binman/etype/ti_secure.py b/tools/binman/etype/ti_secure.py index 704dcf8a381..420ee263e4f 100644 --- a/tools/binman/etype/ti_secure.py +++ b/tools/binman/etype/ti_secure.py @@ -53,10 +53,11 @@ class Entry_ti_secure(Entry_x509_cert): - keyfile: Filename of file containing key to sign binary with - sha: Hash function to be used for signing - auth-in-place: This is an integer field that contains two pieces - of information - Lower Byte - Remains 0x02 as per our use case - ( 0x02: Move the authenticated binary back to the header ) - Upper Byte - The Host ID of the core owning the firewall + of information: + + - Lower Byte - Remains 0x02 as per our use case + ( 0x02: Move the authenticated binary back to the header ) + - Upper Byte - The Host ID of the core owning the firewall Output files: - input. - input file passed to openssl @@ -69,29 +70,29 @@ class Entry_ti_secure(Entry_x509_cert): firewall nodes that describe the configurations of firewall that TIFS will be doing after reading the certificate. - The syntax of the firewall nodes are as such: + The syntax of the firewall nodes are as such:: - firewall-257-0 { - id = <257>; /* The ID of the firewall being configured */ - region = <0>; /* Region number to configure */ + firewall-257-0 { + id = <257>; /* The ID of the firewall being configured */ + region = <0>; /* Region number to configure */ - control = /* The control register */ - <(FWCTRL_EN | FWCTRL_LOCK | FWCTRL_BG | FWCTRL_CACHE)>; + control = /* The control register */ + <(FWCTRL_EN | FWCTRL_LOCK | FWCTRL_BG | FWCTRL_CACHE)>; - permissions = /* The permission registers */ - <((FWPRIVID_ALL << FWPRIVID_SHIFT) | - FWPERM_SECURE_PRIV_RWCD | - FWPERM_SECURE_USER_RWCD | - FWPERM_NON_SECURE_PRIV_RWCD | - FWPERM_NON_SECURE_USER_RWCD)>; + permissions = /* The permission registers */ + <((FWPRIVID_ALL << FWPRIVID_SHIFT) | + FWPERM_SECURE_PRIV_RWCD | + FWPERM_SECURE_USER_RWCD | + FWPERM_NON_SECURE_PRIV_RWCD | + FWPERM_NON_SECURE_USER_RWCD)>; - /* More defines can be found in k3-security.h */ + /* More defines can be found in k3-security.h */ - start_address = /* The Start Address of the firewall */ - <0x0 0x0>; - end_address = /* The End Address of the firewall */ - <0xff 0xffffffff>; - }; + start_address = /* The Start Address of the firewall */ + <0x0 0x0>; + end_address = /* The End Address of the firewall */ + <0xff 0xffffffff>; + }; openssl signs the provided data, using the TI templated config file and From patchwork Fri Jun 21 20:46:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1950973 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=TZDMcxB7; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W5TvG0YDCz20Wb for ; Sat, 22 Jun 2024 06:47:38 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 1401F8859B; Fri, 21 Jun 2024 22:46:49 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="TZDMcxB7"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 660CE8804F; Fri, 21 Jun 2024 22:46:47 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.2 Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 2F89C8855E for ; Fri, 21 Jun 2024 22:46:44 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd2b.google.com with SMTP id ca18e2360f4ac-7eec09cc7f6so95407239f.0 for ; Fri, 21 Jun 2024 13:46:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719002803; x=1719607603; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FA+FlwdkRSMhQxY2MZH+AxkvYhCzQLUcdRiqc88p2+4=; b=TZDMcxB7fVliRhB1pGEex+efnh8ClP+bjF4exzXKUGMXbgb5yx9/Mnh47m0BDBTSwJ HFVF3bO5SHaP7Ln5nfxk5UbhSYOV4PJbNqEuLQ2yEy3SzlNH/czQbSMoNzRrWz5QtKZz c/5qc/2TGaCzkox/4fXMWtA0z4jpFzHC1jxYM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719002803; x=1719607603; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FA+FlwdkRSMhQxY2MZH+AxkvYhCzQLUcdRiqc88p2+4=; b=OpRZGOltNDetdjbWNCgbPM4Y9BO2UVNqEAgC+rNOkzkpOgIY1h65is9lgYOENgTskW aah0KzX+ed+RuOR2Dp2fsN89ytswxUr32cVKI/Yh+cLjIZb3U4+FayLzc5z1iGEaMo5E LN3RCXAF1ejDW7+VJ0iySZBu8i64aMzHUfEg522Mo3kuXwaR/0n0cn+nFaA+UIxenjln kcDSLXfht6K3mNQ/Q1ufYfgQa0xLK6/nhaJNMV4OcdozM8BXRoYa+iiDLn/IUHGtESgX 7s22HL2cGcq/slbziCODNpQI1gdLc0DwznIeHh/xO5yDwqgN6+x2VtSm4peCVVpGSmPs vLbg== X-Gm-Message-State: AOJu0YwN9V/lYRiWDfQDjDRNLiipyilBi1nVWXIAMWXRQ253VWiTPDKq sQVhKzhg60OeSJKjsj6H1wltmQ5/EKZ8fHMNHEqUBjyS5Fz6sJOmQ1kjVMYkJlQO7Rk5NzLGgXp 5EQ== X-Google-Smtp-Source: AGHT+IG9qBDK2b6k7SyNWheSvHlZUsRahUzgxzhjYMsPUXIoo2lhMTG174a6xxOaMXEt6GoSmHy98Q== X-Received: by 2002:a05:6602:6c0b:b0:7eb:846f:4280 with SMTP id ca18e2360f4ac-7f13ee87afbmr1009751239f.16.1719002802680; Fri, 21 Jun 2024 13:46:42 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f39203ec51sm50698939f.53.2024.06.21.13.46.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jun 2024 13:46:42 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass , Alper Nebi Yasak , Marek Vasut , Neha Malcom Francis , Peng Fan , Philippe Reynes , Stefan Herbrechtsmeier , Sughosh Ganu Subject: [PATCH 04/11] binman: Update the entrydocs header Date: Fri, 21 Jun 2024 14:46:25 -0600 Message-Id: <20240621204632.2706813-5-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240621204632.2706813-1-sjg@chromium.org> References: <20240621204632.2706813-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Reduce the length of the underline for this header, to match the heading itself. Signed-off-by: Simon Glass --- tools/binman/entry.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/binman/entry.py b/tools/binman/entry.py index 42e0b7b9145..2ed65800d22 100644 --- a/tools/binman/entry.py +++ b/tools/binman/entry.py @@ -812,7 +812,7 @@ class Entry(object): as missing """ print('''Binman Entry Documentation -=========================== +========================== This file describes the entry types supported by binman. These entry types can be placed in an image one by one to build up a final firmware image. It is From patchwork Fri Jun 21 20:46:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1950972 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=Q8pMKLe+; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W5Tv415mmz20Wb for ; Sat, 22 Jun 2024 06:47:28 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id A735F88563; Fri, 21 Jun 2024 22:46:48 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Q8pMKLe+"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 468108857A; Fri, 21 Jun 2024 22:46:47 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd2f.google.com (mail-io1-xd2f.google.com [IPv6:2607:f8b0:4864:20::d2f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 0BE298804F for ; Fri, 21 Jun 2024 22:46:45 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd2f.google.com with SMTP id ca18e2360f4ac-7ec0385de1fso89455639f.3 for ; Fri, 21 Jun 2024 13:46:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719002803; x=1719607603; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oksFfmSkHnmY059/50YjczK0o27dkuCx4uuoJ73RzBM=; b=Q8pMKLe+CrSOU28D+wFBG5q0dIFbIoZHpZaSL1zT8MPtus39+wrK/FAwXLHOx6dzd3 GV7h01FGSRM4bJnx5dsMkb2RcwGhOviPmEHQbHE7aqmsmjL3rPxB8RUUzTlKFdhDmcaN 9zb9tKx2bHWF8iGhgQTHGiyc0eJ+OOZrTdil0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719002803; x=1719607603; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oksFfmSkHnmY059/50YjczK0o27dkuCx4uuoJ73RzBM=; b=SmtrSVaZBVVjXJJsMOMze1RIAp9tYSTVswP2jRlHEjt+CFn3Vc1kfjtZY2UJgEBdSB PgV8NKfCpKimsep5Z8J2pYGvDp6BM103A8bqjCBxqwGxexOsfEmInYXm3RrIeJTsDajm NDRDLMDyfaUX3UY0iDEfooTwNZ6CYZSqjjr1rb4mxGtMN/QSODyZZxJKZP7n041yyMqn mO7fHXt9ThfIYgdoY2EXy42dgLyh8iAs0qg4wwtrC4TWnXOL3uvAwL61AHgcBB+5rz8B 2vSWcEU0e0srd/PDQvXWKWRnWayPPED4F2hvA0zpg8LkegtvAMRCCD2yp3HuVSx2hz7i fFsw== X-Gm-Message-State: AOJu0Yyrsf7yJgb1SHBYq01Y4YbSvoP5Ztwq93ULo6enskHFsQtZjGzA eERrs+kAyGYKi3Tc4RLdkZsnPlP5oXAnmkokyftE7UGEHTDsc6k7cXX4kY7I5EgGjGa84oB/LT+ 2hA== X-Google-Smtp-Source: AGHT+IEmogQ+r8HBb/hGQ3pdCGqlVYeZX3foxiA7MrW1DB4GA2TIUEcKdL0P89+dkjeZzUBAXan2sg== X-Received: by 2002:a05:6602:164e:b0:7eb:4d68:a56a with SMTP id ca18e2360f4ac-7f13eea5b41mr1007412439f.17.1719002803531; Fri, 21 Jun 2024 13:46:43 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f39203ec51sm50698939f.53.2024.06.21.13.46.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jun 2024 13:46:43 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass Subject: [PATCH 05/11] buildman: Make mrproper an argument to _reconfigure() Date: Fri, 21 Jun 2024 14:46:26 -0600 Message-Id: <20240621204632.2706813-6-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240621204632.2706813-1-sjg@chromium.org> References: <20240621204632.2706813-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Pass this in so the caller can change it independently of the member variable. Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index a8599c0bb2a..5d4426bf0d1 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -240,7 +240,7 @@ class BuilderThread(threading.Thread): return args, cwd, src_dir def _reconfigure(self, commit, brd, cwd, args, env, config_args, config_out, - cmd_list): + cmd_list, mrproper): """Reconfigure the build Args: @@ -251,11 +251,12 @@ class BuilderThread(threading.Thread): env (dict): Environment strings config_args (list of str): defconfig arg for this board cmd_list (list of str): List to add the commands to, for logging + mrproper (bool): True to run mrproper first Returns: CommandResult object """ - if self.mrproper: + if mrproper: result = self.make(commit, brd, 'mrproper', cwd, 'mrproper', *args, env=env) config_out.write(result.combined) @@ -419,7 +420,8 @@ class BuilderThread(threading.Thread): cmd_list = [] if do_config or adjust_cfg: result = self._reconfigure( - commit, brd, cwd, args, env, config_args, config_out, cmd_list) + commit, brd, cwd, args, env, config_args, config_out, cmd_list, + self.mrproper) do_config = False # No need to configure next time if adjust_cfg: cfgutil.adjust_cfg_file(cfg_file, adjust_cfg) From patchwork Fri Jun 21 20:46:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1950974 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=JI0FgkZ3; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W5TvS317nz20Wb for ; Sat, 22 Jun 2024 06:47:48 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 71E378855C; Fri, 21 Jun 2024 22:46:52 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="JI0FgkZ3"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id D74A68857F; Fri, 21 Jun 2024 22:46:47 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd34.google.com (mail-io1-xd34.google.com [IPv6:2607:f8b0:4864:20::d34]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id B080D88569 for ; Fri, 21 Jun 2024 22:46:45 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd34.google.com with SMTP id ca18e2360f4ac-7ec07f4170bso96547939f.0 for ; Fri, 21 Jun 2024 13:46:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719002804; x=1719607604; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wQWCHAppkaPnl6ePnNncon/+GhSpdXUItRx1/sMowC4=; b=JI0FgkZ3wVmuDemjzF4PKxlQdGSuKnH0mYNIl4e4Y6V7Hhej8ZtTJGoetPhP0TxWNI XgMi7peaJQ/G3UbU2O1J7N2MueqxsMCoEVfFCPQayDUoLByIWZBTnJRsPA21FFO/YJkj T5qNxL5Uo2+otke9M9MRw2AU2smciBtgY5BWg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719002804; x=1719607604; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wQWCHAppkaPnl6ePnNncon/+GhSpdXUItRx1/sMowC4=; b=FG/j0iu2TQmXA2gqNvBiqG4+k3PkmbC/hU4qmy+AJJ1QsoHVgaaphBOAaRO6k0y9c9 kNMWYmwG7O34pBX39E9K/1BWKWbW2I18wHPMJqjwCsFmdPwDMRNnBKaXDS4KSiK9h8kB i4jLVRNDZ7Znkaveee/tNVqTsI1/RlrCvsNs/ddO4CxMvD+QyWQzquMBcOftWOoIjdOf vZbxdDhdt29UbmtZUdo0OTf/5u7KQAmhBsZO7YeQyFpfbCbAbzjbtqL5Og83ybWDLaC3 l7l8YAwNjwl5qXegkh7P8WVg+8hDBFnxETgMLWgKjAWeQxk6vZJS+Ums8jKfIWD6uw3U SbMg== X-Gm-Message-State: AOJu0YyCYN0U6NgxVXduqO5dlZyv0VK3q0+wfndCatLRdxeBMXybz+RR Pr4cgL/o/F4weIBi7gycQzvU7IX4f74pQZ46zlEoeBm5+p/prgnXuwLhecztkI8mLdM/82b0ka+ Qng== X-Google-Smtp-Source: AGHT+IGiLD762Ic/mLPy6EFJJYHLNQpsIh2BfohlfaNsrAKDCwnv9fYFBd9DKR2yOKHf9lUAPXFOGw== X-Received: by 2002:a05:6602:1650:b0:7eb:7427:f878 with SMTP id ca18e2360f4ac-7f13ee8d0c8mr1126405139f.19.1719002804375; Fri, 21 Jun 2024 13:46:44 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f39203ec51sm50698939f.53.2024.06.21.13.46.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jun 2024 13:46:43 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass Subject: [PATCH 06/11] buildman: Make mrproper an argument to _config_and_build() Date: Fri, 21 Jun 2024 14:46:27 -0600 Message-Id: <20240621204632.2706813-7-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240621204632.2706813-1-sjg@chromium.org> References: <20240621204632.2706813-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Pass this in so the caller can change it independently of the member variable. Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index 5d4426bf0d1..ff63f9397e6 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -381,7 +381,7 @@ class BuilderThread(threading.Thread): commit = 'current' return commit - def _config_and_build(self, commit_upto, brd, work_dir, do_config, + def _config_and_build(self, commit_upto, brd, work_dir, do_config, mrproper, config_only, adjust_cfg, commit, out_dir, out_rel_dir, result): """Do the build, configuring first if necessary @@ -391,6 +391,7 @@ class BuilderThread(threading.Thread): brd (Board): Board to create arguments for work_dir (str): Directory to which the source will be checked out do_config (bool): True to run a make _defconfig on the source + mrproper (bool): True to run mrproper first config_only (bool): Only configure the source, do not build it adjust_cfg (list of str): See the cfgutil module and run_commit() commit (Commit): Commit only being built @@ -421,7 +422,7 @@ class BuilderThread(threading.Thread): if do_config or adjust_cfg: result = self._reconfigure( commit, brd, cwd, args, env, config_args, config_out, cmd_list, - self.mrproper) + mrproper) do_config = False # No need to configure next time if adjust_cfg: cfgutil.adjust_cfg_file(cfg_file, adjust_cfg) @@ -500,8 +501,9 @@ class BuilderThread(threading.Thread): if self.toolchain: commit = self._checkout(commit_upto, work_dir) result, do_config = self._config_and_build( - commit_upto, brd, work_dir, do_config, config_only, - adjust_cfg, commit, out_dir, out_rel_dir, result) + commit_upto, brd, work_dir, do_config, self.mrproper, + config_only, adjust_cfg, commit, out_dir, out_rel_dir, + result) result.already_done = False result.toolchain = self.toolchain From patchwork Fri Jun 21 20:46:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1950976 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=U4JLoc2f; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W5Tvs6PF3z20Wb for ; Sat, 22 Jun 2024 06:48:09 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 305CB8857D; Fri, 21 Jun 2024 22:46:53 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="U4JLoc2f"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 0D7F188548; Fri, 21 Jun 2024 22:46:50 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd31.google.com (mail-io1-xd31.google.com [IPv6:2607:f8b0:4864:20::d31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 7AC6D8854E for ; Fri, 21 Jun 2024 22:46:46 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd31.google.com with SMTP id ca18e2360f4ac-7eb699def37so72808139f.1 for ; Fri, 21 Jun 2024 13:46:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719002805; x=1719607605; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TilowMtmIbLEJX6Ew7pKKH3SkTzbW9M9g59F5fRsUic=; b=U4JLoc2forRgQLwPIObngBEoRRZ3xAbNeYyev0d7NdIP5X1FDJpRPTtsxgbTRpk11v qpCwKO9bJ7ndMPPDvySpJHGlA568lDw4ezPEzZyTxPg8TNC5aA+JN+CcoPA1buzaKyMt XYLFpSiHtU+eplUzi4xbJbQUv6/dscXfG9IgU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719002805; x=1719607605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TilowMtmIbLEJX6Ew7pKKH3SkTzbW9M9g59F5fRsUic=; b=IoEbOs03TBnvn/LH2qX5MEoZY0zj8BbjbtbzmbWqeOEFCA1DHNYTVgL3k0icfcDSIo yugLEJm7PF25a02iblwWDjl8DArjJorqWq2KgOp0o9QcKlWJXpCsLgryD7pAYProuK3k eP0TGZT95e5qmvDZkAmRbHv8CXI0Mr1ZrwMh8Fp8Qd9YQ9Pz30zpJsOr75LZ0iAikYA6 MPPtKzfg9dPLDe8ZBfSzIErk/p7bJGWfxSh7vOM/YnSLxxheus16XSDHtqjWClET28xA xOHhyNgDDd4FMVZZWPFy5A8o7sKDSdKgyC/ZjFfTK0mQEh6BqtNKeEaoGKd1yuuk13DA YXdg== X-Gm-Message-State: AOJu0YycL8nbEEjpyp4CDCKsKjJ3G+f/KxkKnV18YZfY91wAfOlDPLpw PmCyjwndN7WqyabeWWTdsTgzAbOjhFoQDigLuqHOJBTiRh7cX4b17xdMeEhwd2rYQ9P+oAWHtZW lXQ== X-Google-Smtp-Source: AGHT+IGivOP9ZQYU7+LnZ/LRqRBfEX8v7CrA1JVT3c+u5MMmFtiG9jaDfVfBUnW1c6A6ghNLrlY0ig== X-Received: by 2002:a05:6602:1653:b0:7eb:65ed:c1f2 with SMTP id ca18e2360f4ac-7f39aafec80mr56810139f.0.1719002805090; Fri, 21 Jun 2024 13:46:45 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f39203ec51sm50698939f.53.2024.06.21.13.46.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jun 2024 13:46:44 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass Subject: [PATCH 07/11] buildman: Make mrproper an argument to run_commit() Date: Fri, 21 Jun 2024 14:46:28 -0600 Message-Id: <20240621204632.2706813-8-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240621204632.2706813-1-sjg@chromium.org> References: <20240621204632.2706813-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Pass this in so the caller can change it independently of the member variable. Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index ff63f9397e6..0a7ff2e083e 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -448,9 +448,9 @@ class BuilderThread(threading.Thread): result.cmd_list = cmd_list return result, do_config - def run_commit(self, commit_upto, brd, work_dir, do_config, config_only, - force_build, force_build_failures, work_in_output, - adjust_cfg): + def run_commit(self, commit_upto, brd, work_dir, do_config, mrproper, + config_only, force_build, force_build_failures, + work_in_output, adjust_cfg): """Build a particular commit. If the build is already done, and we are not forcing a build, we skip @@ -461,6 +461,7 @@ class BuilderThread(threading.Thread): brd (Board): Board to build work_dir (str): Directory to which the source will be checked out do_config (bool): True to run a make _defconfig on the source + mrproper (bool): True to run mrproper first config_only (bool): Only configure the source, do not build it force_build (bool): Force a build even if one was previously done force_build_failures (bool): Force a bulid if the previous result @@ -501,7 +502,7 @@ class BuilderThread(threading.Thread): if self.toolchain: commit = self._checkout(commit_upto, work_dir) result, do_config = self._config_and_build( - commit_upto, brd, work_dir, do_config, self.mrproper, + commit_upto, brd, work_dir, do_config, mrproper, config_only, adjust_cfg, commit, out_dir, out_rel_dir, result) result.already_done = False @@ -692,7 +693,8 @@ class BuilderThread(threading.Thread): force_build = False for commit_upto in range(0, len(job.commits), job.step): result, request_config = self.run_commit(commit_upto, brd, - work_dir, do_config, self.builder.config_only, + work_dir, do_config, self.mrproper, + self.builder.config_only, force_build or self.builder.force_build, self.builder.force_build_failures, job.work_in_output, job.adjust_cfg) @@ -703,8 +705,8 @@ class BuilderThread(threading.Thread): # with a reconfig. if self.builder.force_config_on_failure: result, request_config = self.run_commit(commit_upto, - brd, work_dir, True, False, True, False, - job.work_in_output, job.adjust_cfg) + brd, work_dir, True, self.mrproper, False, True, + False, job.work_in_output, job.adjust_cfg) did_config = True if not self.builder.force_reconfig: do_config = request_config @@ -748,7 +750,7 @@ class BuilderThread(threading.Thread): else: # Just build the currently checked-out build result, request_config = self.run_commit(None, brd, work_dir, True, - self.builder.config_only, True, + self.mrproper, self.builder.config_only, True, self.builder.force_build_failures, job.work_in_output, job.adjust_cfg) result.commit_upto = 0 From patchwork Fri Jun 21 20:46:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1950975 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=nK+mUChb; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W5Tvg3kRGz20Wb for ; Sat, 22 Jun 2024 06:47:58 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id C6E2088569; Fri, 21 Jun 2024 22:46:52 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="nK+mUChb"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id E32DF88059; Fri, 21 Jun 2024 22:46:49 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd31.google.com (mail-io1-xd31.google.com [IPv6:2607:f8b0:4864:20::d31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 27D7B8855C for ; Fri, 21 Jun 2024 22:46:47 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd31.google.com with SMTP id ca18e2360f4ac-7ebc67365e3so91969739f.1 for ; Fri, 21 Jun 2024 13:46:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719002806; x=1719607606; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TJatQ67wLsUKufFx8+ZIAVVElx7A/F8+oG5u3y8kT7g=; b=nK+mUChbcN8Siaswur2Ju9TxoYF6P9RforgDxuZ5LlzeWVrxwxr0hAffOpI3SEnL0V 38+0ZmikPT48U7ecxzyUU42QGsrN5XYWEODZ6cNRBJti38vjauDN2fyNjX06gQcysD1c lIPBjpknYrl5okJWoNDmToAp8afWqTK6DC34Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719002806; x=1719607606; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TJatQ67wLsUKufFx8+ZIAVVElx7A/F8+oG5u3y8kT7g=; b=TDwcVjKDqVBQqApgOGtbXaSgPOy9NdCnFh7mmJzNjEn++Cl8MO7o29LFDQ6ScdWqbM T9uL5GrGjLdS1PPOr5xFEwlQMTWpf2Lgz5PcYto79r6KwqTdTv3n/SSqIQaKd6nTIRWb Lm8sP9KwvNdWnVDvUa5R9xSxKUE2f9qQU37Nm/FpkGxYvzfWGgDleF58VVaakCOVVZJK wSG2Fyezb/4s51cKwRnX22V0hIn7rqTz8SyFQK9DX2f5QDBFH7Tgw0Gka6g4UuPXf3HA wYCkIBbE0RXA/u+M0dcPcM2vPC+rztNI3Zsw159Q++Szp9o8FftZkY879WzSns/kj4t3 qEIQ== X-Gm-Message-State: AOJu0YzIoERxZig1GxnWpqItEPFKXLTL2GwcMI+jlvyAE4t0oWh0wi5F moB9pc2XHVPlXvsJiAYkzr9m+JsRsUYbsgMqLBl0tytq1ABijS7LzmxWMpsSQy2TDUyiHz9BCoz dew== X-Google-Smtp-Source: AGHT+IFT9C/VrzdtEeJ18w26UOwCK8FcXnZ+oBrnh66J7opEHGTEzHvBXpofE5ON5dOSHs4NwaVgHw== X-Received: by 2002:a05:6602:2dd4:b0:7eb:6b58:440e with SMTP id ca18e2360f4ac-7f13ee94480mr1120416639f.18.1719002805882; Fri, 21 Jun 2024 13:46:45 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f39203ec51sm50698939f.53.2024.06.21.13.46.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jun 2024 13:46:45 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass Subject: [PATCH 08/11] buildman: Avoid rebuilding when --mrproper is used Date: Fri, 21 Jun 2024 14:46:29 -0600 Message-Id: <20240621204632.2706813-9-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240621204632.2706813-1-sjg@chromium.org> References: <20240621204632.2706813-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean When this flag is enabled, 'make mrproper' is always used when reconfiguring, so there is no point in doing it again. Update this. Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index 0a7ff2e083e..c0b1067e3f7 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -700,7 +700,7 @@ class BuilderThread(threading.Thread): job.work_in_output, job.adjust_cfg) failed = result.return_code or result.stderr did_config = do_config - if failed and not do_config: + if failed and not do_config and not self.mrproper: # If our incremental build failed, try building again # with a reconfig. if self.builder.force_config_on_failure: From patchwork Fri Jun 21 20:46:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1950978 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=ZvdXR6Av; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W5TwG36n6z20Wb for ; Sat, 22 Jun 2024 06:48:30 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 066058858C; Fri, 21 Jun 2024 22:47:03 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="ZvdXR6Av"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 17B0E88405; Fri, 21 Jun 2024 22:46:52 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd44.google.com (mail-io1-xd44.google.com [IPv6:2607:f8b0:4864:20::d44]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id F1BD888576 for ; Fri, 21 Jun 2024 22:46:47 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd44.google.com with SMTP id ca18e2360f4ac-7eb895539e3so94107739f.2 for ; Fri, 21 Jun 2024 13:46:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719002806; x=1719607606; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=E+q4gqJa2swS7fW00RKTnduYoOd3HH3BvpJOyGpjEl4=; b=ZvdXR6Avgux+n5GQwtvT86II7RyiryJdxoR8O05B3M/KEkjBvnQD+fa00bx22sUsD7 9m0a08QX27Oq0kIY8v6fKychy4gMwMZHTHmoTr7ufEUZqOpC5oAFHmcmgWdPmucQeLsv CqnLVzbWODzc19eeOI2UBK8F9aXAuQHfujkEE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719002806; x=1719607606; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=E+q4gqJa2swS7fW00RKTnduYoOd3HH3BvpJOyGpjEl4=; b=a4Opq8JaIJteSJVsCibj4V0Uu0JcblrydNk2RPhbxRDXEFGq3Rq5ekyD4fd7eWj0f1 GbxIM+8SyptPQnW1j+KTc/95vcTdsU78364yKy82dXlYyW4xGjd756DxsGhuojPAg7zd DyTbF/j4GNlA8vWW28Uwa76lNs/hT6m/hC01DtmZdd8kMkK0U6mMZRB6FtjAy2U95OFG tFa6BMiCudWji+mixLGekp7y+yNvIQ9SxQ3831jZ5Id1b5aQ5+8JEoEV/hY3zp11LC36 FeduqVP6w3oWQhU7TVa+LjpjXA4SvyKIHCZmefIhNk0DHDDYJ7x3+iYwfT+Fjy7H2YMn ODfg== X-Gm-Message-State: AOJu0Yy7zUAEo/r5IYXNNOeY2XnCS93/kPJf4i7RaUQSXvAJCmSDyIuR pPj41ptRpW9ep2LNQ+pRdx+8NK2SgalZDUKRdVHQ9usvqE0o2XTeTmNkSi6qlWkhRJcGPtPNC3v UpUSH X-Google-Smtp-Source: AGHT+IGsovfAbOO8DT9/MNnO9k+6qdtlcinbjZxoIluWndM+s3UPTp6bs52A5GAuWMNFaGRuIJwhNw== X-Received: by 2002:a05:6602:14cc:b0:7eb:c7ff:26e3 with SMTP id ca18e2360f4ac-7f13ee7a920mr1012651939f.16.1719002806576; Fri, 21 Jun 2024 13:46:46 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f39203ec51sm50698939f.53.2024.06.21.13.46.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jun 2024 13:46:46 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass , Alexey Brodkin Subject: [PATCH 09/11] buildman: Add a flag to force mrproper on failure Date: Fri, 21 Jun 2024 14:46:30 -0600 Message-Id: <20240621204632.2706813-10-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240621204632.2706813-1-sjg@chromium.org> References: <20240621204632.2706813-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean When a file is removed by a commit (e.g. include/common.h yay!) it can cause incremental build failures since one of the dependency files from a previous build may mention the file. Add an option to run 'make mrproper' automatically when a build fails. This can be used to automatically resolve the problem, without always adding the large overhead of 'make mrproper' to every build. Signed-off-by: Simon Glass --- tools/buildman/builder.py | 18 ++++++++++-------- tools/buildman/builderthread.py | 6 ++++-- tools/buildman/buildman.rst | 3 ++- tools/buildman/cmdline.py | 4 +++- tools/buildman/control.py | 1 + 5 files changed, 20 insertions(+), 12 deletions(-) diff --git a/tools/buildman/builder.py b/tools/buildman/builder.py index f35175b4598..c4384f53e8d 100644 --- a/tools/buildman/builder.py +++ b/tools/buildman/builder.py @@ -256,14 +256,14 @@ class Builder: def __init__(self, toolchains, base_dir, git_dir, num_threads, num_jobs, gnu_make='make', checkout=True, show_unknown=True, step=1, no_subdirs=False, full_path=False, verbose_build=False, - mrproper=False, per_board_out_dir=False, - config_only=False, squash_config_y=False, - warnings_as_errors=False, work_in_output=False, - test_thread_exceptions=False, adjust_cfg=None, - allow_missing=False, no_lto=False, reproducible_builds=False, - force_build=False, force_build_failures=False, - force_reconfig=False, in_tree=False, - force_config_on_failure=False, make_func=None): + mrproper=False, fallback_mrproper=False, + per_board_out_dir=False, config_only=False, + squash_config_y=False, warnings_as_errors=False, + work_in_output=False, test_thread_exceptions=False, + adjust_cfg=None, allow_missing=False, no_lto=False, + reproducible_builds=False, force_build=False, + force_build_failures=False, force_reconfig=False, + in_tree=False, force_config_on_failure=False, make_func=None): """Create a new Builder object Args: @@ -283,6 +283,7 @@ class Builder: PATH verbose_build: Run build with V=1 and don't use 'make -s' mrproper: Always run 'make mrproper' when configuring + fallback_mrproper: Run 'make mrproper' and retry on build failure per_board_out_dir: Build in a separate persistent directory per board rather than a thread-specific directory config_only: Only configure each build, don't build it @@ -352,6 +353,7 @@ class Builder: self.force_reconfig = force_reconfig self.in_tree = in_tree self.force_config_on_failure = force_config_on_failure + self.fallback_mrproper = fallback_mrproper if not self.squash_config_y: self.config_filenames += EXTRA_CONFIG_FILENAMES diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index c0b1067e3f7..bbe2f6f0d24 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -705,8 +705,10 @@ class BuilderThread(threading.Thread): # with a reconfig. if self.builder.force_config_on_failure: result, request_config = self.run_commit(commit_upto, - brd, work_dir, True, self.mrproper, False, True, - False, job.work_in_output, job.adjust_cfg) + brd, work_dir, True, + self.mrproper or self.builder.fallback_mrproper, + False, True, False, job.work_in_output, + job.adjust_cfg) did_config = True if not self.builder.force_reconfig: do_config = request_config diff --git a/tools/buildman/buildman.rst b/tools/buildman/buildman.rst index aae2477b5c3..bd0482af5f7 100644 --- a/tools/buildman/buildman.rst +++ b/tools/buildman/buildman.rst @@ -995,7 +995,8 @@ By default, buildman doesn't execute 'make mrproper' prior to building the first commit for each board. This reduces the amount of work 'make' does, and hence speeds up the build. To force use of 'make mrproper', use -the -m flag. This flag will slow down any buildman invocation, since it increases the amount -of work done on any build. +of work done on any build. An alternative is to use the --fallback-mrproper +flag, which retries the build with 'make mrproper' only after a build failure. One possible application of buildman is as part of a continual edit, build, edit, build, ... cycle; repeatedly applying buildman to the same change or diff --git a/tools/buildman/cmdline.py b/tools/buildman/cmdline.py index 03211bd5aa5..8dc5a8787b5 100644 --- a/tools/buildman/cmdline.py +++ b/tools/buildman/cmdline.py @@ -90,7 +90,9 @@ def add_upto_m(parser): parser.add_argument('--list-tool-chains', action='store_true', default=False, help='List available tool chains (use -v to see probing detail)') parser.add_argument('-m', '--mrproper', action='store_true', - default=False, help="Run 'make mrproper before reconfiguring") + default=False, help="Run 'make mrproper' before reconfiguring") + parser.add_argument('--fallback-mrproper', action='store_true', + default=False, help="Run 'make mrproper' and retry on build failure") parser.add_argument( '-M', '--allow-missing', action='store_true', default=False, help='Tell binman to allow missing blobs and generate fake ones as needed') diff --git a/tools/buildman/control.py b/tools/buildman/control.py index 8f6850c5211..f2dd87814c3 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -656,6 +656,7 @@ def do_buildman(args, toolchains=None, make_func=None, brds=None, no_subdirs=args.no_subdirs, full_path=args.full_path, verbose_build=args.verbose_build, mrproper=args.mrproper, + fallback_mrproper=args.fallback_mrproper, per_board_out_dir=args.per_board_out_dir, config_only=args.config_only, squash_config_y=not args.preserve_config_y, From patchwork Fri Jun 21 20:46:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1950977 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=nyo9+FLA; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W5Tw35gdpz20Wb for ; Sat, 22 Jun 2024 06:48:19 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 9233388590; Fri, 21 Jun 2024 22:46:53 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="nyo9+FLA"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id CD84D88405; Fri, 21 Jun 2024 22:46:51 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd35.google.com (mail-io1-xd35.google.com [IPv6:2607:f8b0:4864:20::d35]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id A798B88591 for ; Fri, 21 Jun 2024 22:46:48 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd35.google.com with SMTP id ca18e2360f4ac-7eb9e7f3da2so101198339f.1 for ; Fri, 21 Jun 2024 13:46:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719002807; x=1719607607; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kSHg7qWjXsARtjBpbR4RtDAK6qA5RpzZmG7C39EaVxs=; b=nyo9+FLA2JUpYK74UYhOBHjxOr1szRneU0USz3PmlVOfLHOipROpkt77tCODSBfiNW 5mqB0zRZWwc0Pjq3mLthMT1kFKwG0DUcgA0/7xYPxT2Eak8YbbSZXaYPve6nIfdmVfx8 rpcrhqDvjwAla59nRWrVnUWRKIaZ1dJkeeAV8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719002807; x=1719607607; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kSHg7qWjXsARtjBpbR4RtDAK6qA5RpzZmG7C39EaVxs=; b=QkYbFEsG6hYd0UPEI2DrY0GMhXpf3lsrTE49UJqK975YLBUyXYKUjg7puMa/Ni/J1k zZF6WrML4goF/6M0A1z6Odr4L5h9CL668BX3xW1NiuwZFQcB81X8zRM3aNwN7ENqVxFL 7IQIJGijEE13R/AdQ5W4JcKSRlkZenBXFP2x9zSt1ESk6GM3CpEXTiQgOSaI/QrS1ztm AXlXctZDWebadgARNYVtEam0e9vVsPcrTzR5sGWmk5bZgkc6gQ1M+Vh90Tt+8183UqA1 1s9PhOADV1HIQpScCrUOt2T6a3RDWn9lbCjso0+anBbyd73Qh3MpjcwaRHvYPgMpyTNn gG1A== X-Gm-Message-State: AOJu0Yzts0YQW4VYdKI52PPEHOoHT46fhTZW/jAJQo5h7rB8LUi81kVv 8fJG8PvqQh+Ck9fZIK3uNNr+94rlnvj0cemkBVEFkT2CW91xa6+rmxjuf/K6/cahMzNn/zE898p ywA== X-Google-Smtp-Source: AGHT+IH8OIk9k8tiIOJMDw25TvWig3b+CYu0Y9Uoomge8qjUNhOr/5ydeUk3IdWL/FYnjuH1GwP/MA== X-Received: by 2002:a05:6602:6b0a:b0:7eb:832e:eecf with SMTP id ca18e2360f4ac-7f13ee7a8bemr1084427639f.17.1719002807337; Fri, 21 Jun 2024 13:46:47 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f39203ec51sm50698939f.53.2024.06.21.13.46.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jun 2024 13:46:46 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass Subject: [PATCH 10/11] buildman: Retry the build for current source Date: Fri, 21 Jun 2024 14:46:31 -0600 Message-Id: <20240621204632.2706813-11-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240621204632.2706813-1-sjg@chromium.org> References: <20240621204632.2706813-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Buildman retries a failed build when processing a branch, but does not do this when building current source. It is useful to do this retry in both cases, so add the logic for it. Signed-off-by: Simon Glass --- tools/buildman/builderthread.py | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index bbe2f6f0d24..55658487abf 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -755,6 +755,14 @@ class BuilderThread(threading.Thread): self.mrproper, self.builder.config_only, True, self.builder.force_build_failures, job.work_in_output, job.adjust_cfg) + failed = result.return_code or result.stderr + if failed and not self.mrproper: + result, request_config = self.run_commit(None, brd, work_dir, + True, self.builder.fallback_mrproper, + self.builder.config_only, True, + self.builder.force_build_failures, + job.work_in_output, job.adjust_cfg) + result.commit_upto = 0 self._write_result(result, job.keep_outputs, job.work_in_output) self._send_result(result) From patchwork Fri Jun 21 20:46:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1950979 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=AZ8hSdHs; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W5TwS0MM8z20Wb for ; Sat, 22 Jun 2024 06:48:40 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 6EA23885A9; Fri, 21 Jun 2024 22:47:03 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="AZ8hSdHs"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id A2B9288569; Fri, 21 Jun 2024 22:46:52 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd42.google.com (mail-io1-xd42.google.com [IPv6:2607:f8b0:4864:20::d42]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id AC31B8804F for ; Fri, 21 Jun 2024 22:46:49 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd42.google.com with SMTP id ca18e2360f4ac-7ec00e71a57so92531239f.3 for ; Fri, 21 Jun 2024 13:46:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719002808; x=1719607608; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eVShViM73zxJOwfrcyPwQc2E+SZEwYZjtlVJOQ7v5XY=; b=AZ8hSdHs20aLBOgk5ONUKlnXJKWGrMQV3M8wpQ3GaQIOgZtM76CpH7CsbX8gSeq8hQ NDDw6w8k4J6x8r9CONmNmXv0Hb1JAhc43S1Dkw3o/aABwS084eEHnAaf3/KvX8oGFgnr Q0hZHIO4OpC/1GzUbuOtH1i84Z90NBI93Kems= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719002808; x=1719607608; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eVShViM73zxJOwfrcyPwQc2E+SZEwYZjtlVJOQ7v5XY=; b=pT+cEojfs4FlE27g1JHinzfE1G7HjinwabR2VYDaqxXFQTLy26k4eIfaJ5582FL5ZA 97F9U82yJvzbFsPaTzc0iRq059upy3KKh3cblfICL+nZlC6yjXGxsgG+YDfrGjECgoD+ 9E4hot83XBzw/xvo+udFezN2WtO0cLoZg7Ukm//5K62lG7yp5NgLZQhBvDa/fMgGHOgM HFLgZhJIBfa8RD8l9q6Ymcm6dvLD7fnHqzphce3femS6RP3uMMI18p7cDos87xuyRKib 9ejQ9mqZjkFwSYIGWAiDoOAbVBl15EXtxXG5uQYBFr8HbXYuVtSfSj9r61VBNYhSgqOu uSkg== X-Gm-Message-State: AOJu0YzCMd0s5tSAXN8mIUFvcCfN9Ac7aOYncDmh79rwX/K3HkRuxQVY a4bNCTHrjkXVeKPXXLC4vKQCGZ2U9L90xjrUO+VjGNtn/UhHP8syad7JnwoxTwbhAhSzpVHiXFn ytxtr X-Google-Smtp-Source: AGHT+IEAc/D4U0YNH/4wB+a2s5jQjfBcrZIsq798c4xjQS29HRAHA+J85eCHfByy6qLNS+zhyX8jxA== X-Received: by 2002:a05:6602:158b:b0:7e1:8a93:48ef with SMTP id ca18e2360f4ac-7f13ee8ac9dmr1141421739f.21.1719002808083; Fri, 21 Jun 2024 13:46:48 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f39203ec51sm50698939f.53.2024.06.21.13.46.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jun 2024 13:46:47 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass , Alexey Brodkin , Heinrich Schuchardt Subject: [PATCH 11/11] buildman: Add a way to limit the number of buildmans Date: Fri, 21 Jun 2024 14:46:32 -0600 Message-Id: <20240621204632.2706813-12-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240621204632.2706813-1-sjg@chromium.org> References: <20240621204632.2706813-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Buildman uses all available CPUs by default, so running more than one or two concurrent processes is not normally useful. However in some CI cases we want to be able to run several jobs at once to save time. For example, in a lab situation we may want to run a test on 20 boards at a time, since only the build step actually takes much CPU. Add an option which allows such a limit. When buildman starts up, it waits until the number of running processes goes below the limit, then claims a spot in the list. The list is maintained with a temporary file. Note that the temp file is user-specific, since it is hard to create a locked temporary file which can be accessed by any user. In most cases, only one user is running jobs on a machine, so this should not matter. Signed-off-by: Simon Glass --- tools/buildman/buildman.rst | 5 ++ tools/buildman/cmdline.py | 2 + tools/buildman/control.py | 140 ++++++++++++++++++++++++++++++++- tools/buildman/pyproject.toml | 6 +- tools/buildman/test.py | 121 ++++++++++++++++++++++++++++ tools/u_boot_pylib/terminal.py | 7 +- 6 files changed, 277 insertions(+), 4 deletions(-) diff --git a/tools/buildman/buildman.rst b/tools/buildman/buildman.rst index bd0482af5f7..b8ff3bf1ab2 100644 --- a/tools/buildman/buildman.rst +++ b/tools/buildman/buildman.rst @@ -1286,6 +1286,11 @@ then buildman hangs. Failing to handle any eventuality is a bug in buildman and should be reported. But you can use -T0 to disable threading and hopefully figure out the root cause of the build failure. +For situations where buildman is invoked from multiple running processes, it is +sometimes useful to have buildman wait until the others have finished. Use the +--process-limit option for this: --process-limit 1 will allow only one buildman +to process jobs at a time. + Build summary ------------- diff --git a/tools/buildman/cmdline.py b/tools/buildman/cmdline.py index 8dc5a8787b5..544a391a464 100644 --- a/tools/buildman/cmdline.py +++ b/tools/buildman/cmdline.py @@ -129,6 +129,8 @@ def add_after_m(parser): default=False, help="Use an O= (output) directory per board rather than per thread") parser.add_argument('--print-arch', action='store_true', default=False, help="Print the architecture for a board (ARCH=)") + parser.add_argument('--process-limit', type=int, + default=0, help='Limit to number of buildmans running at once') parser.add_argument('-r', '--reproducible-builds', action='store_true', help='Set SOURCE_DATE_EPOCH=0 to suuport a reproducible build') parser.add_argument('-R', '--regen-board-list', type=str, diff --git a/tools/buildman/control.py b/tools/buildman/control.py index f2dd87814c3..464835c5be5 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -7,10 +7,13 @@ This holds the main control logic for buildman, when not running tests. """ +import getpass import multiprocessing import os import shutil import sys +import tempfile +import time from buildman import boards from buildman import bsettings @@ -21,10 +24,23 @@ from patman import gitutil from patman import patchstream from u_boot_pylib import command from u_boot_pylib import terminal -from u_boot_pylib.terminal import tprint +from u_boot_pylib import tools +from u_boot_pylib.terminal import print_clear, tprint TEST_BUILDER = None +# Space-separated list of buildman process IDs currently running jobs +RUNNING_FNAME = f'buildmanq.{getpass.getuser()}' + +# Lock file for access to RUNNING_FILE +LOCK_FNAME = f'{RUNNING_FNAME}.lock' + +# Wait time for access to lock (seconds) +LOCK_WAIT_S = 10 + +# Wait time to start running +RUN_WAIT_S = 300 + def get_plural(count): """Returns a plural 's' if count is not 1""" return 's' if count != 1 else '' @@ -578,6 +594,125 @@ def calc_adjust_cfg(adjust_cfg, reproducible_builds): return adjust_cfg +def read_procs(tmpdir=tempfile.gettempdir()): + """Read the list of running buildman processes + + If the list is corrupted, returns an empty list + + Args: + tmpdir (str): Temporary directory to use (for testing only) + """ + running_fname = os.path.join(tmpdir, RUNNING_FNAME) + procs = [] + if os.path.exists(running_fname): + items = tools.read_file(running_fname, binary=False).split() + try: + procs = [int(x) for x in items] + except ValueError: # Handle invalid format + pass + return procs + + +def check_pid(pid): + """Check for existence of a unix PID + + https://stackoverflow.com/questions/568271/how-to-check-if-there-exists-a-process-with-a-given-pid-in-python + + Args: + pid (int): PID to check + + Returns: + True if it exists, else False + """ + try: + os.kill(pid, 0) + except OSError: + return False + else: + return True + + +def write_procs(procs, tmpdir=tempfile.gettempdir()): + """Write the list of running buildman processes + + Args: + tmpdir (str): Temporary directory to use (for testing only) + """ + running_fname = os.path.join(tmpdir, RUNNING_FNAME) + tools.write_file(running_fname, ' '.join([str(p) for p in procs]), + binary=False) + + # Allow another user to access the file + os.chmod(running_fname, 0o666) + +def wait_for_process_limit(limit, tmpdir=tempfile.gettempdir(), + pid=os.getpid()): + """Wait until the number of buildman processes drops to the limit + + This uses FileLock to protect a 'running' file, which contains a list of + PIDs of running buildman processes. The number of PIDs in the file indicates + the number of running processes. + + When buildman starts up, it calls this function to wait until it is OK to + start the build. + + On exit, no attempt is made to remove the PID from the file, since other + buildman processes will notice that the PID is no-longer valid, and ignore + it. + + Two timeouts are provided: + LOCK_WAIT_S: length of time to wait for the lock; if this occurs, the + lock is busted / removed before trying again + RUN_WAIT_S: length of time to wait to be allowed to run; if this occurs, + the build starts, with the PID being added to the file. + + Args: + limit (int): Maximum number of buildman processes, including this one; + must be > 0 + tmpdir (str): Temporary directory to use (for testing only) + pid (int): Current process ID (for testing only) + """ + from filelock import Timeout, FileLock + + running_fname = os.path.join(tmpdir, RUNNING_FNAME) + lock_fname = os.path.join(tmpdir, LOCK_FNAME) + lock = FileLock(lock_fname) + + # Allow another user to access the file + col = terminal.Color() + tprint('Waiting for other buildman processes...', newline=False, + colour=col.RED) + + claimed = False + deadline = time.time() + RUN_WAIT_S + while True: + try: + with lock.acquire(timeout=LOCK_WAIT_S): + os.chmod(lock_fname, 0o666) + procs = read_procs(tmpdir) + + # Drop PIDs which are not running + procs = list(filter(check_pid, procs)) + + # If we haven't hit the limit, add ourself + if len(procs) < limit: + tprint('done...', newline=False) + claimed = True + if time.time() >= deadline: + tprint('timeout...', newline=False) + claimed = True + if claimed: + write_procs(procs + [pid], tmpdir) + break + + except Timeout: + tprint('failed to get lock: busting...', newline=False) + os.remove(lock_fname) + + time.sleep(1) + tprint('starting build', newline=False) + print_clear() + def do_buildman(args, toolchains=None, make_func=None, brds=None, clean_dir=False, test_thread_exceptions=False): """The main control code for buildman @@ -677,5 +812,8 @@ def do_buildman(args, toolchains=None, make_func=None, brds=None, TEST_BUILDER = builder + if args.process_limit: + wait_for_process_limit(args.process_limit) + return run_builder(builder, series.commits if series else None, brds.get_selected_dict(), args) diff --git a/tools/buildman/pyproject.toml b/tools/buildman/pyproject.toml index fe0f6421b53..68bfa45c3f4 100644 --- a/tools/buildman/pyproject.toml +++ b/tools/buildman/pyproject.toml @@ -8,7 +8,11 @@ version = "0.0.6" authors = [ { name="Simon Glass", email="sjg@chromium.org" }, ] -dependencies = ["u_boot_pylib >= 0.0.6", "patch-manager >= 0.0.6"] +dependencies = [ + "filelock >= 3.0.12", + "u_boot_pylib >= 0.0.6", + "patch-manager >= 0.0.6" +] description = "Buildman build tool for U-Boot" readme = "README.rst" requires-python = ">=3.7" diff --git a/tools/buildman/test.py b/tools/buildman/test.py index f92add7a7c5..d68395c2164 100644 --- a/tools/buildman/test.py +++ b/tools/buildman/test.py @@ -2,12 +2,14 @@ # Copyright (c) 2012 The Chromium OS Authors. # +from filelock import FileLock import os import shutil import sys import tempfile import time import unittest +from unittest.mock import patch from buildman import board from buildman import boards @@ -156,6 +158,11 @@ class TestBuild(unittest.TestCase): if not os.path.isdir(self.base_dir): os.mkdir(self.base_dir) + self.cur_time = 0 + self.valid_pids = [] + self.finish_time = None + self.finish_pid = None + def tearDown(self): shutil.rmtree(self.base_dir) @@ -747,6 +754,120 @@ class TestBuild(unittest.TestCase): self.assertEqual([ ['MARY="mary"', 'Missing expected line: CONFIG_MARY="mary"']], result) + def get_procs(self): + running_fname = os.path.join(self.base_dir, control.RUNNING_FNAME) + items = tools.read_file(running_fname, binary=False).split() + return [int(x) for x in items] + + def get_time(self): + return self.cur_time + + def inc_time(self, amount): + self.cur_time += amount + + # Handle a process exiting + if self.finish_time == self.cur_time: + self.valid_pids = [pid for pid in self.valid_pids + if pid != self.finish_pid] + + def kill(self, pid, signal): + if pid not in self.valid_pids: + raise OSError('Invalid PID') + + def test_process_limit(self): + """Test wait_for_process_limit() function""" + tmpdir = self.base_dir + + with (patch('time.time', side_effect=self.get_time), + patch('time.sleep', side_effect=self.inc_time), + patch('os.kill', side_effect=self.kill)): + # Grab the process. Since there is no other profcess, this should + # immediately succeed + control.wait_for_process_limit(1, tmpdir=tmpdir, pid=1) + lines = terminal.get_print_test_lines() + self.assertEqual(0, self.cur_time) + self.assertEqual('Waiting for other buildman processes...', + lines[0].text) + self.assertEqual(self._col.RED, lines[0].colour) + self.assertEqual(False, lines[0].newline) + self.assertEqual(True, lines[0].bright) + + self.assertEqual('done...', lines[1].text) + self.assertEqual(None, lines[1].colour) + self.assertEqual(False, lines[1].newline) + self.assertEqual(True, lines[1].bright) + + self.assertEqual('starting build', lines[2].text) + self.assertEqual([1], control.read_procs(tmpdir)) + self.assertEqual(None, lines[2].colour) + self.assertEqual(False, lines[2].newline) + self.assertEqual(True, lines[2].bright) + + # Try again, with a different PID...this should eventually timeout + # and start the build anyway + self.cur_time = 0 + self.valid_pids = [1] + control.wait_for_process_limit(1, tmpdir=tmpdir, pid=2) + lines = terminal.get_print_test_lines() + self.assertEqual('Waiting for other buildman processes...', + lines[0].text) + self.assertEqual('timeout...', lines[1].text) + self.assertEqual(None, lines[1].colour) + self.assertEqual(False, lines[1].newline) + self.assertEqual(True, lines[1].bright) + self.assertEqual('starting build', lines[2].text) + self.assertEqual([1, 2], control.read_procs(tmpdir)) + self.assertEqual(control.RUN_WAIT_S, self.cur_time) + + # Check lock-busting + self.cur_time = 0 + self.valid_pids = [1, 2] + lock_fname = os.path.join(tmpdir, control.LOCK_FNAME) + lock = FileLock(lock_fname) + lock.acquire(timeout=1) + control.wait_for_process_limit(1, tmpdir=tmpdir, pid=3) + lines = terminal.get_print_test_lines() + self.assertEqual('Waiting for other buildman processes...', + lines[0].text) + self.assertEqual('failed to get lock: busting...', lines[1].text) + self.assertEqual(None, lines[1].colour) + self.assertEqual(False, lines[1].newline) + self.assertEqual(True, lines[1].bright) + self.assertEqual('timeout...', lines[2].text) + self.assertEqual('starting build', lines[3].text) + self.assertEqual([1, 2, 3], control.read_procs(tmpdir)) + self.assertEqual(control.RUN_WAIT_S, self.cur_time) + lock.release() + + # Check handling of dead processes. Here we have PID 2 as a running + # process, even though the PID file contains 1, 2 and 3. So we can + # add one more PID, to make 2 and 4 + self.cur_time = 0 + self.valid_pids = [2] + control.wait_for_process_limit(2, tmpdir=tmpdir, pid=4) + lines = terminal.get_print_test_lines() + self.assertEqual('Waiting for other buildman processes...', + lines[0].text) + self.assertEqual('done...', lines[1].text) + self.assertEqual('starting build', lines[2].text) + self.assertEqual([2, 4], control.read_procs(tmpdir)) + self.assertEqual(0, self.cur_time) + + # Try again, with PID 2 quitting at time 50. This allows the new + # build to start + self.cur_time = 0 + self.valid_pids = [2, 4] + self.finish_pid = 2 + self.finish_time = 50 + control.wait_for_process_limit(2, tmpdir=tmpdir, pid=5) + lines = terminal.get_print_test_lines() + self.assertEqual('Waiting for other buildman processes...', + lines[0].text) + self.assertEqual('done...', lines[1].text) + self.assertEqual('starting build', lines[2].text) + self.assertEqual([4, 5], control.read_procs(tmpdir)) + self.assertEqual(self.finish_time, self.cur_time) + if __name__ == "__main__": unittest.main() diff --git a/tools/u_boot_pylib/terminal.py b/tools/u_boot_pylib/terminal.py index 40d79f8ac07..2cd5a54ab52 100644 --- a/tools/u_boot_pylib/terminal.py +++ b/tools/u_boot_pylib/terminal.py @@ -164,8 +164,11 @@ def print_clear(): global last_print_len if last_print_len: - print('\r%s\r' % (' '* last_print_len), end='', flush=True) - last_print_len = None + if print_test_mode: + print_test_list.append(PrintLine(None, None, None, None)) + else: + print('\r%s\r' % (' '* last_print_len), end='', flush=True) + last_print_len = None def set_print_test_mode(enable=True): """Go into test mode, where all printing is recorded"""