From patchwork Sun Jun 23 17:55:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1951280 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=Bf5lLX23; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W6f0y6LLLz20WR for ; Mon, 24 Jun 2024 03:56:34 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 9FF408858B; Sun, 23 Jun 2024 19:55:25 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Bf5lLX23"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 726F78857E; Sun, 23 Jun 2024 19:55:24 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 5A4F88855B for ; Sun, 23 Jun 2024 19:55:21 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd32.google.com with SMTP id ca18e2360f4ac-7ebdb5d9915so135559139f.1 for ; Sun, 23 Jun 2024 10:55:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719165320; x=1719770120; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1MNIMhO/Ndz6W2tKEOG699I123CEzO9+iNidV5hkSFM=; b=Bf5lLX23atHo6Vekdplg126pG2diIlAFFO6dWoAL+QPPGJ8J1lRVslXBxRRuV0TrBk zrSyD4f5/KTXzO8DNQxbJHh2dcmiFkWLtdayZKC/irb1ZsZpoFvnRNmId1xyFrt9lBLe jtAKC83zyKftAY8Lbp5+aLGADYUkqbAdvH3sQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719165320; x=1719770120; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1MNIMhO/Ndz6W2tKEOG699I123CEzO9+iNidV5hkSFM=; b=mNzLNboufdYQROY8XmoZrvxVJ0VnFtLSQ5+RqZ8kbN/9qi8o8qEYX9mRGDUEmyOoCG 52lv0oWSezUdiMMtz61WgPgcaY7ds11l+kwmrSMXR6ZKtWCUUkfucdr4zKSfdhmnyAoy n9cqZjtVnp6jSQ5aHuqIU1UJD8LfCxBAlh4JIoUX2jo2yBHBcvngU4Gmy2Nm5dZjs2Xx aokZWHC60CYlKXczOZOBVrWIkyV6Q6md2Yq0xJMTooioeaTmyP54b5Kd1v1fn4XqEq0/ 4mHkvpVuE9HgV/wv0vTCt3wqwGnXQme4K2tvCXycLXRyfLzf2jOx7uqbo4WErtUgY/Zh 5Wvw== X-Gm-Message-State: AOJu0Yxk24jPnC81sSlsy4zOm7NSWQzMySeoTZoRPOwiLFTrkpQf6w2F Lyk48eCmqlEXM/+0jkw37QELokCElgOzWBlDZWluRCS6fk+kFxiKlhEMi37APKmYjkCWr1LB8uj E2A== X-Google-Smtp-Source: AGHT+IFPVnz96UbHaSkI+UdTvdxJz3iqS/KYBcBm2sRtcwLXjlhIqTwBh4OPw3WCTV9bPnJYjEfh4g== X-Received: by 2002:a05:6602:2d8b:b0:7f3:a9d4:105f with SMTP id ca18e2360f4ac-7f3a9d412dfmr195850139f.5.1719165319699; Sun, 23 Jun 2024 10:55:19 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f391fc9b12sm134472239f.5.2024.06.23.10.55.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Jun 2024 10:55:19 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass , Alper Nebi Yasak , Christian Taedcke , Heinrich Schuchardt , Lukas Funke , Neha Malcom Francis , Peng Fan , Philippe Reynes , Stefan Herbrechtsmeier , Sughosh Ganu Subject: [PATCH v2 01/14] binman: efi: Correct entry docs Date: Sun, 23 Jun 2024 11:55:02 -0600 Message-Id: <20240623175515.1466908-2-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240623175515.1466908-1-sjg@chromium.org> References: <20240623175515.1466908-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Somehow the class documentation has got out of sync with the generated entries.rst file. Regenerating it causes errors, so correct these and regenerate the entries.rst file. Signed-off-by: Simon Glass Fixes: 809f28e7213 ("binman: capsule: Use dumped capsule header...") Signed-off-by: Simon Glass --- (no changes since v1) tools/binman/entries.rst | 58 ++++++++++++------------- tools/binman/etype/efi_capsule.py | 40 ++++++++--------- tools/binman/etype/efi_empty_capsule.py | 22 +++++----- 3 files changed, 61 insertions(+), 59 deletions(-) diff --git a/tools/binman/entries.rst b/tools/binman/entries.rst index 254afe76074..dc42a9cb97d 100644 --- a/tools/binman/entries.rst +++ b/tools/binman/entries.rst @@ -470,11 +470,11 @@ updating the EC on startup via software sync. .. _etype_efi_capsule: -Entry: capsule: Entry for generating EFI Capsule files ------------------------------------------------------- +Entry: efi-capsule: Generate EFI capsules +----------------------------------------- -The parameters needed for generation of the capsules can be provided -as properties in the entry. +The parameters needed for generation of the capsules can +be provided as properties in the entry. Properties / Entry arguments: - image-index: Unique number for identifying corresponding @@ -495,9 +495,9 @@ Properties / Entry arguments: file. Mandatory property for generating signed capsules. - oem-flags - OEM flags to be passed through capsule header. - Since this is a subclass of Entry_section, all properties of the parent - class also apply here. Except for the properties stated as mandatory, the - rest of the properties are optional. +Since this is a subclass of Entry_section, all properties of the parent +class also apply here. Except for the properties stated as mandatory, the +rest of the properties are optional. For more details on the description of the capsule format, and the capsule update functionality, refer Section 8.5 and Chapter 23 in the `UEFI @@ -510,17 +510,17 @@ provided as a subnode of the capsule entry. A typical capsule entry node would then look something like this:: capsule { - type = "efi-capsule"; - image-index = <0x1>; - /* Image GUID for testing capsule update */ - image-guid = SANDBOX_UBOOT_IMAGE_GUID; - hardware-instance = <0x0>; - private-key = "path/to/the/private/key"; - public-key-cert = "path/to/the/public-key-cert"; - oem-flags = <0x8000>; + type = "efi-capsule"; + image-index = <0x1>; + /* Image GUID for testing capsule update */ + image-guid = SANDBOX_UBOOT_IMAGE_GUID; + hardware-instance = <0x0>; + private-key = "path/to/the/private/key"; + public-key-cert = "path/to/the/public-key-cert"; + oem-flags = <0x8000>; - u-boot { - }; + u-boot { + }; }; In the above example, the capsule payload is the U-Boot image. The @@ -534,8 +534,8 @@ payload using the blob-ext subnode. .. _etype_efi_empty_capsule: -Entry: efi-empty-capsule: Entry for generating EFI Empty Capsule files ----------------------------------------------------------------------- +Entry: efi-empty-capsule: Generate EFI empty capsules +----------------------------------------------------- The parameters needed for generation of the empty capsules can be provided as properties in the entry. @@ -551,22 +551,22 @@ update functionality, refer Section 8.5 and Chapter 23 in the `UEFI specification`_. For more information on the empty capsule, refer the sections 2.3.2 and 2.3.3 in the `Dependable Boot specification`_. -A typical accept empty capsule entry node would then look something -like this:: +A typical accept empty capsule entry node would then look something like +this:: empty-capsule { - type = "efi-empty-capsule"; - /* GUID of the image being accepted */ - image-type-id = SANDBOX_UBOOT_IMAGE_GUID; - capsule-type = "accept"; + type = "efi-empty-capsule"; + /* GUID of image being accepted */ + image-type-id = SANDBOX_UBOOT_IMAGE_GUID; + capsule-type = "accept"; }; -A typical revert empty capsule entry node would then look something -like this:: +A typical revert empty capsule entry node would then look something like +this:: empty-capsule { - type = "efi-empty-capsule"; - capsule-type = "revert"; + type = "efi-empty-capsule"; + capsule-type = "revert"; }; The empty capsules do not have any input payload image. diff --git a/tools/binman/etype/efi_capsule.py b/tools/binman/etype/efi_capsule.py index e3203717822..751f654bf31 100644 --- a/tools/binman/etype/efi_capsule.py +++ b/tools/binman/etype/efi_capsule.py @@ -36,23 +36,23 @@ class Entry_efi_capsule(Entry_section): be provided as properties in the entry. Properties / Entry arguments: - - image-index: Unique number for identifying corresponding - payload image. Number between 1 and descriptor count, i.e. - the total number of firmware images that can be updated. Mandatory - property. - - image-guid: Image GUID which will be used for identifying the - updatable image on the board. Mandatory property. - - hardware-instance: Optional number for identifying unique - hardware instance of a device in the system. Default value of 0 - for images where value is not to be used. - - fw-version: Value of image version that can be put on the capsule - through the Firmware Management Protocol(FMP) header. - - monotonic-count: Count used when signing an image. - - private-key: Path to PEM formatted .key private key file. Mandatory - property for generating signed capsules. - - public-key-cert: Path to PEM formatted .crt public key certificate - file. Mandatory property for generating signed capsules. - - oem-flags - OEM flags to be passed through capsule header. + - image-index: Unique number for identifying corresponding + payload image. Number between 1 and descriptor count, i.e. + the total number of firmware images that can be updated. Mandatory + property. + - image-guid: Image GUID which will be used for identifying the + updatable image on the board. Mandatory property. + - hardware-instance: Optional number for identifying unique + hardware instance of a device in the system. Default value of 0 + for images where value is not to be used. + - fw-version: Value of image version that can be put on the capsule + through the Firmware Management Protocol(FMP) header. + - monotonic-count: Count used when signing an image. + - private-key: Path to PEM formatted .key private key file. Mandatory + property for generating signed capsules. + - public-key-cert: Path to PEM formatted .crt public key certificate + file. Mandatory property for generating signed capsules. + - oem-flags - OEM flags to be passed through capsule header. Since this is a subclass of Entry_section, all properties of the parent class also apply here. Except for the properties stated as mandatory, the @@ -66,9 +66,9 @@ class Entry_efi_capsule(Entry_section): properties in the entry. The payload to be used in the capsule is to be provided as a subnode of the capsule entry. - A typical capsule entry node would then look something like this + A typical capsule entry node would then look something like this:: - capsule { + capsule { type = "efi-capsule"; image-index = <0x1>; /* Image GUID for testing capsule update */ @@ -80,7 +80,7 @@ class Entry_efi_capsule(Entry_section): u-boot { }; - }; + }; In the above example, the capsule payload is the U-Boot image. The capsule entry would read the contents of the payload and put them diff --git a/tools/binman/etype/efi_empty_capsule.py b/tools/binman/etype/efi_empty_capsule.py index 064bf9a77f0..1d99fbfb3bb 100644 --- a/tools/binman/etype/efi_empty_capsule.py +++ b/tools/binman/etype/efi_empty_capsule.py @@ -19,31 +19,33 @@ class Entry_efi_empty_capsule(Entry_section): be provided as properties in the entry. Properties / Entry arguments: - - image-guid: Image GUID which will be used for identifying the - updatable image on the board. Mandatory for accept capsule. - - capsule-type - String to indicate type of capsule to generate. Valid - values are 'accept' and 'revert'. + - image-guid: Image GUID which will be used for identifying the + updatable image on the board. Mandatory for accept capsule. + - capsule-type - String to indicate type of capsule to generate. Valid + values are 'accept' and 'revert'. For more details on the description of the capsule format, and the capsule update functionality, refer Section 8.5 and Chapter 23 in the `UEFI specification`_. For more information on the empty capsule, refer the sections 2.3.2 and 2.3.3 in the `Dependable Boot specification`_. - A typical accept empty capsule entry node would then look something like this + A typical accept empty capsule entry node would then look something like + this:: - empty-capsule { + empty-capsule { type = "efi-empty-capsule"; /* GUID of image being accepted */ image-type-id = SANDBOX_UBOOT_IMAGE_GUID; capsule-type = "accept"; - }; + }; - A typical revert empty capsule entry node would then look something like this + A typical revert empty capsule entry node would then look something like + this:: - empty-capsule { + empty-capsule { type = "efi-empty-capsule"; capsule-type = "revert"; - }; + }; The empty capsules do not have any input payload image. From patchwork Sun Jun 23 17:55:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1951282 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=BycI1dTj; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W6f1B4wQ0z20WR for ; Mon, 24 Jun 2024 03:56:46 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 0CF52885A9; Sun, 23 Jun 2024 19:55:26 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="BycI1dTj"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id A1F9F88549; Sun, 23 Jun 2024 19:55:24 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id E66AA88576 for ; Sun, 23 Jun 2024 19:55:21 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd2b.google.com with SMTP id ca18e2360f4ac-7ec2d71059dso142114939f.2 for ; Sun, 23 Jun 2024 10:55:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719165320; x=1719770120; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7/NjtgmhdGNN51ppyRZf80LJV+wFQVnrrC1NW5Sw19c=; b=BycI1dTjT7tsofjdoWYpimCAbaWLYcW/P3itGKes5zq2Ww/Qf6Qdb0dDrKgDynxGZa 1Hz4etrgtU+CYIFbnTEEKJ2dEVb7NCP1gWSVnXzogs5mSNBnyTrJ0syjHoTgBYIa/jAO y1KmrQRNFxT1rQy8aPgo67WXLN7lp8aVzKB8g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719165320; x=1719770120; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7/NjtgmhdGNN51ppyRZf80LJV+wFQVnrrC1NW5Sw19c=; b=sVvawc9TFCYi/6hvT8AALoY5sw0V2sNAbc4dOjCLNNBiq+dEEn1nfF494UVOTheq4T L7/Ws8b6EwZPptBsFXcEU+GoUyoLLx/yU9kp/L0G7pdF40HZWm75t12VcYw4u5FdPMJO W6SHyYVF2n6TTyNQOf84zM3yqVzff0rb7h7Z5HD+ZkISWmcsoQmpe10tsGhD931tI9vx SpkicWjfEYgXmeTAsfQQSy+vgguvp68XG3uoM3+fAQghw93aquAzY3kepGiae5aDj/7l O5wpKmFW/h+G+RBzHSG2Ye3Y0HPhOSq6+gFTVk4seCi2HRxm7oqmSnzRPtkHlJ32ei6A beQg== X-Gm-Message-State: AOJu0YyjKK7GDIeh/Rk6dMf4EzEOLupQEW8SqRtnk3CJ8n1ouQVgkRFn qfgSF9xWoIzZmYMq/HJqMbM7Eh04rHs8seHGh7S1y4Os2kxDsqQ+UQ7rqYEEcRXkGbTZl36wVCU a0w== X-Google-Smtp-Source: AGHT+IHnKDWPICEjIw0aJRLiDBOCvELMABqjRnmUOAIJoAxWR2hCPoDYDBOfoMNXzUXdRFwXdJKnJg== X-Received: by 2002:a05:6602:6013:b0:7eb:db06:e067 with SMTP id ca18e2360f4ac-7f3a7532d73mr241284139f.9.1719165320461; Sun, 23 Jun 2024 10:55:20 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f391fc9b12sm134472239f.5.2024.06.23.10.55.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Jun 2024 10:55:20 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass , Alper Nebi Yasak , Christian Taedcke , Lukas Funke , Marek Vasut , Neha Malcom Francis , Peng Fan , Philippe Reynes , Stefan Herbrechtsmeier , Sughosh Ganu Subject: [PATCH v2 02/14] binman: Regenerate nxp docs Date: Sun, 23 Jun 2024 11:55:03 -0600 Message-Id: <20240623175515.1466908-3-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240623175515.1466908-1-sjg@chromium.org> References: <20240623175515.1466908-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Regenerate the entries.rst file to include this recent addition. Note that more docs are needed here, to actually describe the entry type. Note also that the entry type needs Binman tests added. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- (no changes since v1) tools/binman/entries.rst | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/tools/binman/entries.rst b/tools/binman/entries.rst index dc42a9cb97d..1b9b868e33f 100644 --- a/tools/binman/entries.rst +++ b/tools/binman/entries.rst @@ -1521,6 +1521,28 @@ byte. +.. _etype_nxp_imx8mcst: + +Entry: nxp-imx8mcst: NXP i.MX8M CST .cfg file generator and cst invoker +----------------------------------------------------------------------- + +Properties / Entry arguments: + - nxp,loader-address - loader address (SPL text base) + + + +.. _etype_nxp_imx8mimage: + +Entry: nxp-imx8mimage: NXP i.MX8M imx8mimage .cfg file generator and mkimage invoker +------------------------------------------------------------------------------------ + +Properties / Entry arguments: + - nxp,boot-from - device to boot from (e.g. 'sd') + - nxp,loader-address - loader address (SPL text base) + - nxp,rom-version - BootROM version ('2' for i.MX8M Nano and Plus) + + + .. _etype_opensbi: Entry: opensbi: RISC-V OpenSBI fw_dynamic blob From patchwork Sun Jun 23 17:55:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1951283 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=S2kCvljm; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W6f1P0dwRz20WR for ; Mon, 24 Jun 2024 03:56:57 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 833338854A; Sun, 23 Jun 2024 19:55:28 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="S2kCvljm"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 4930B882B7; Sun, 23 Jun 2024 19:55:26 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd34.google.com (mail-io1-xd34.google.com [IPv6:2607:f8b0:4864:20::d34]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id AB5E2882B7 for ; Sun, 23 Jun 2024 19:55:22 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd34.google.com with SMTP id ca18e2360f4ac-7eba486df76so106511939f.0 for ; Sun, 23 Jun 2024 10:55:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719165321; x=1719770121; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6HB2ErhIf6siF/0qMBG68GPp5zfuKT+xollSj3Pe6/g=; b=S2kCvljmfU84efjvfiXQ06D0ib/rDea7SnHMoDfjYmBwmbYGWnof8szMUeUUh12pR8 PdOuNr0arvzXznVtKVYdiCkQvB3r4i3R9PgPZL7Q5a2BHQawd8WJDQdVXlipCKk4JcZk 4zNlrBSMkX2bwM/3MYIddqGdWfTEGUT6WnD0U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719165321; x=1719770121; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6HB2ErhIf6siF/0qMBG68GPp5zfuKT+xollSj3Pe6/g=; b=NZJvbuZjRPPonSvY+isj/jDZeUtvXmFDHYa0fNq46uyEkFm2D/fPm876MvehZ8ltga IeDz+Op/M3gt4I2URuep3Rntnw3WCrIQdFESU+uAHtIs44GUHAD95EUfz4bI6ZGQ1Gwa O8tj0EkoQhUVn6zhKIThsDaJBaFMYMYGtR5KZ2UDgdFOv5Y2NtMkZn7uvay7wVtGdbMe rGWIlveQvL0dHpVsZ/hMpwR5Mjfk2L4OKempKrXl6DM43rjDCxP/BVgPDo+hhcerVC85 zIQhANJjaZUqbPRxPRjT+ZAJA9zCcDwxecfcval2U8lRviuw8VA0rVYNDhTdHWlIod83 AwJA== X-Gm-Message-State: AOJu0YxC97YfZDyuyxibScQ5Vxv7R6o3o/dobzdV4YB78dle7B3MMtIq 4x640oyPWzz+O0ULYM2wa6I2boh6wGT4ax8C0e78ByLOgS/74xyZUE4aRuRC744kPztIzZN6iqH rAw== X-Google-Smtp-Source: AGHT+IFQ2nEBQ9WTnUq166/qeOwj34fWPTwO5cySdNAAca1lNJL18kPcqP45mY8JWiRgrnRmREWphg== X-Received: by 2002:a05:6602:1604:b0:7eb:ff8b:e25b with SMTP id ca18e2360f4ac-7f3a0732675mr167366139f.1.1719165321305; Sun, 23 Jun 2024 10:55:21 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f391fc9b12sm134472239f.5.2024.06.23.10.55.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Jun 2024 10:55:21 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass , Alper Nebi Yasak , Christian Taedcke , Lukas Funke , Manorit Chawdhry , Neha Malcom Francis , Peng Fan , Philippe Reynes , Stefan Herbrechtsmeier , Sughosh Ganu , Vignesh Raghavendra Subject: [PATCH v2 03/14] binman: ti: Regenerate entry docs Date: Sun, 23 Jun 2024 11:55:04 -0600 Message-Id: <20240623175515.1466908-4-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240623175515.1466908-1-sjg@chromium.org> References: <20240623175515.1466908-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Correct formatting errors in the documentation. Regenerate the entries.rst file to include this recent addition. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- (no changes since v1) tools/binman/entries.rst | 35 +++++++++++++++++++++++++ tools/binman/etype/ti_secure.py | 45 +++++++++++++++++---------------- 2 files changed, 58 insertions(+), 22 deletions(-) diff --git a/tools/binman/entries.rst b/tools/binman/entries.rst index 1b9b868e33f..bdda1ef2855 100644 --- a/tools/binman/entries.rst +++ b/tools/binman/entries.rst @@ -1951,6 +1951,12 @@ Properties / Entry arguments: - content: List of phandles to entries to sign - keyfile: Filename of file containing key to sign binary with - sha: Hash function to be used for signing + - auth-in-place: This is an integer field that contains two pieces + of information: + + - Lower Byte - Remains 0x02 as per our use case + ( 0x02: Move the authenticated binary back to the header ) + - Upper Byte - The Host ID of the core owning the firewall Output files: - input. - input file passed to openssl @@ -1959,6 +1965,35 @@ Output files: - cert. - output file generated by openssl (which is used as the entry contents) +Depending on auth-in-place information in the inputs, we read the +firewall nodes that describe the configurations of firewall that TIFS +will be doing after reading the certificate. + +The syntax of the firewall nodes are as such:: + + firewall-257-0 { + id = <257>; /* The ID of the firewall being configured */ + region = <0>; /* Region number to configure */ + + control = /* The control register */ + <(FWCTRL_EN | FWCTRL_LOCK | FWCTRL_BG | FWCTRL_CACHE)>; + + permissions = /* The permission registers */ + <((FWPRIVID_ALL << FWPRIVID_SHIFT) | + FWPERM_SECURE_PRIV_RWCD | + FWPERM_SECURE_USER_RWCD | + FWPERM_NON_SECURE_PRIV_RWCD | + FWPERM_NON_SECURE_USER_RWCD)>; + + /* More defines can be found in k3-security.h */ + + start_address = /* The Start Address of the firewall */ + <0x0 0x0>; + end_address = /* The End Address of the firewall */ + <0xff 0xffffffff>; + }; + + openssl signs the provided data, using the TI templated config file and writes the signature in this entry. This allows verification that the data is genuine. diff --git a/tools/binman/etype/ti_secure.py b/tools/binman/etype/ti_secure.py index 704dcf8a381..420ee263e4f 100644 --- a/tools/binman/etype/ti_secure.py +++ b/tools/binman/etype/ti_secure.py @@ -53,10 +53,11 @@ class Entry_ti_secure(Entry_x509_cert): - keyfile: Filename of file containing key to sign binary with - sha: Hash function to be used for signing - auth-in-place: This is an integer field that contains two pieces - of information - Lower Byte - Remains 0x02 as per our use case - ( 0x02: Move the authenticated binary back to the header ) - Upper Byte - The Host ID of the core owning the firewall + of information: + + - Lower Byte - Remains 0x02 as per our use case + ( 0x02: Move the authenticated binary back to the header ) + - Upper Byte - The Host ID of the core owning the firewall Output files: - input. - input file passed to openssl @@ -69,29 +70,29 @@ class Entry_ti_secure(Entry_x509_cert): firewall nodes that describe the configurations of firewall that TIFS will be doing after reading the certificate. - The syntax of the firewall nodes are as such: + The syntax of the firewall nodes are as such:: - firewall-257-0 { - id = <257>; /* The ID of the firewall being configured */ - region = <0>; /* Region number to configure */ + firewall-257-0 { + id = <257>; /* The ID of the firewall being configured */ + region = <0>; /* Region number to configure */ - control = /* The control register */ - <(FWCTRL_EN | FWCTRL_LOCK | FWCTRL_BG | FWCTRL_CACHE)>; + control = /* The control register */ + <(FWCTRL_EN | FWCTRL_LOCK | FWCTRL_BG | FWCTRL_CACHE)>; - permissions = /* The permission registers */ - <((FWPRIVID_ALL << FWPRIVID_SHIFT) | - FWPERM_SECURE_PRIV_RWCD | - FWPERM_SECURE_USER_RWCD | - FWPERM_NON_SECURE_PRIV_RWCD | - FWPERM_NON_SECURE_USER_RWCD)>; + permissions = /* The permission registers */ + <((FWPRIVID_ALL << FWPRIVID_SHIFT) | + FWPERM_SECURE_PRIV_RWCD | + FWPERM_SECURE_USER_RWCD | + FWPERM_NON_SECURE_PRIV_RWCD | + FWPERM_NON_SECURE_USER_RWCD)>; - /* More defines can be found in k3-security.h */ + /* More defines can be found in k3-security.h */ - start_address = /* The Start Address of the firewall */ - <0x0 0x0>; - end_address = /* The End Address of the firewall */ - <0xff 0xffffffff>; - }; + start_address = /* The Start Address of the firewall */ + <0x0 0x0>; + end_address = /* The End Address of the firewall */ + <0xff 0xffffffff>; + }; openssl signs the provided data, using the TI templated config file and From patchwork Sun Jun 23 17:55:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1951284 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=cY2H4YSd; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W6f1f49Ykz20WR for ; Mon, 24 Jun 2024 03:57:10 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 088DD885B6; Sun, 23 Jun 2024 19:55:29 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="cY2H4YSd"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 57BB1885AE; Sun, 23 Jun 2024 19:55:26 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd2a.google.com (mail-io1-xd2a.google.com [IPv6:2607:f8b0:4864:20::d2a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 8A81E88433 for ; Sun, 23 Jun 2024 19:55:23 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd2a.google.com with SMTP id ca18e2360f4ac-7eeea388a8eso143729839f.2 for ; Sun, 23 Jun 2024 10:55:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719165322; x=1719770122; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=61teAF4z9QYPqE2dHTGBd5n7k4ZlCtrd5ypMBq94joc=; b=cY2H4YSdBoQJTTRhjbWORYRn6vLorfKL6dl0bA4NjTyE2ZDf7rQzxIV/GHzoEePXv7 LoNx++4bYrr46rrsnZYWIddW9iA2xPbpFKqH6O3uRVSF/ad+8LEy+15GzBCuXw3XjG5l 9IKP6yOs52b6CEHGab+JDfHE96LV7R3FU9z4M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719165322; x=1719770122; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=61teAF4z9QYPqE2dHTGBd5n7k4ZlCtrd5ypMBq94joc=; b=bI/8XnOncqTMHXoDDbux+Q+rJ9/5dhRHNIMYFuhLkzQ3e0IE1mfB1E/NLBuhpeCurW 2xBT8cyTv7/AAr1GrHnoZaMP8vrlEu76EngiA2fzdBnvzBwq6mdPT6Yts7ti3aox1qYA VfnAkmR0v0ogOHJTRGPIcIvIEyq41njdAZusP9g/bn4L++z0/b4yL3D9XUH1F1hyhotm JfEFHFmgnj934w+XsgLHNioqDFCrzveukdN3DJdAf0phGZf/9vYuWszzLvIQqO+CtkDF y9EcvyhIDOgbKlkqtq1/HJrGy5ccSNvIZQBfv64QX0BC0c9l7uj7CHioHi3MdMHEAm0a 2Qtw== X-Gm-Message-State: AOJu0Yy8b1HtDJGdo/Q7yjiOD8bzdmQ2qW937a61ZYPsmQ7PC2l32VEW 4FE8z/G53E6LfUpewkOIxhnFVOWnzcEZLAH9oMw/xsy089ZOQZFBr4r3rPbpxeKi3oza2o+CSeB 8Jw== X-Google-Smtp-Source: AGHT+IHTM/5TDzovqgtX2KfHIHN71JVNY6B/8nQo3CdLN0hCPdKthtYaeewp3/jVd4FHBz8TEeeb4w== X-Received: by 2002:a05:6602:6c04:b0:7eb:81d6:c914 with SMTP id ca18e2360f4ac-7f3a7471000mr331410939f.4.1719165322148; Sun, 23 Jun 2024 10:55:22 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f391fc9b12sm134472239f.5.2024.06.23.10.55.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Jun 2024 10:55:21 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass , Alper Nebi Yasak , Marek Vasut , Neha Malcom Francis , Peng Fan , Philippe Reynes , Stefan Herbrechtsmeier , Sughosh Ganu Subject: [PATCH v2 04/14] binman: Update the entrydocs header Date: Sun, 23 Jun 2024 11:55:05 -0600 Message-Id: <20240623175515.1466908-5-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240623175515.1466908-1-sjg@chromium.org> References: <20240623175515.1466908-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Reduce the length of the underline for this header, to match the heading itself. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- (no changes since v1) tools/binman/entry.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/binman/entry.py b/tools/binman/entry.py index 42e0b7b9145..2ed65800d22 100644 --- a/tools/binman/entry.py +++ b/tools/binman/entry.py @@ -812,7 +812,7 @@ class Entry(object): as missing """ print('''Binman Entry Documentation -=========================== +========================== This file describes the entry types supported by binman. These entry types can be placed in an image one by one to build up a final firmware image. It is From patchwork Sun Jun 23 17:55:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1951285 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=UBzewZ3L; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W6f1m2cBZz20WR for ; Mon, 24 Jun 2024 03:57:16 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 76373885C7; Sun, 23 Jun 2024 19:55:29 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="UBzewZ3L"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 02AFA882E2; Sun, 23 Jun 2024 19:55:27 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 Received: from mail-io1-xd2c.google.com (mail-io1-xd2c.google.com [IPv6:2607:f8b0:4864:20::d2c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id ABA398854A for ; Sun, 23 Jun 2024 19:55:24 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd2c.google.com with SMTP id ca18e2360f4ac-7eb01106015so140460239f.1 for ; Sun, 23 Jun 2024 10:55:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719165323; x=1719770123; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=reRO+PWnAbQoRezpZ6uNVJqdGMz4XgJ+cqxJGVBudhw=; b=UBzewZ3LQi8ZpKGW65QaYFboqQJQpyQQUi1eDNBweVrhXDsvD0q7LGdKFvSw4t5JVB w9hr7HBkD+LQtc5tKzivMrCeDSBFIvct1TV8b0GMDuevuo0Bcz8E0Xj/lapPquzf+HuW kno96ICDBaqdX4zxmDvS22RWy9q6LujhMBRk8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719165323; x=1719770123; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=reRO+PWnAbQoRezpZ6uNVJqdGMz4XgJ+cqxJGVBudhw=; b=OoHyRxM1ZTGgUW4RCeFkRHnOwV6yGVz7eogUOqd/b9ZiKMKm/ROklC3dJ0ENgJhwSy +6RZ5towqQepS4h6q8YsL1S0eD2M/plEXrMUMX/JFc8vhku4GMtg1Ihzlf7e/r2sMFtg A4gUQMsSzni47SJyU1RwLXjB3ZZ43PM1O2LMp02oFyD0+j8uUQ9+n9SiKvdZO3DPDqvw T/d9wHVyk/Z2RUAOLFWcseiV/Cv/ZXwxX1RoGqBa3jVrw7SdfewwRXBpWOsRF8qmDpOU JWzowNxKhSIL5Q/rypMeray9XfZzZgsWuAR+WuxoEkhBk7QBLorE3u2TgrsKOa5EPSAF BGkg== X-Gm-Message-State: AOJu0YyQjrqAlCxhNJZrYEMHrvMc0ZVn37jvtyjICGQ9jrFTVaA/nn4f pK4LJY9Z1zmzcr4acwG2vZTQIQ6pVB168pM4uyEqqxY3eCnbEALP0GePGAPguBLN4e+lLAnppfY 66w== X-Google-Smtp-Source: AGHT+IF2Ecz08inQurnUtTJTEWcMJQKhZ54vnw0r8mejInY4MzGj0pDY4a5gvp1q9RdxYT6Se/yE2A== X-Received: by 2002:a05:6602:158d:b0:7eb:8320:3662 with SMTP id ca18e2360f4ac-7f3a75fa6d3mr318294539f.19.1719165323207; Sun, 23 Jun 2024 10:55:23 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f391fc9b12sm134472239f.5.2024.06.23.10.55.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Jun 2024 10:55:22 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass , Alper Nebi Yasak , Dario Binacchi , Heinrich Schuchardt , Jonas Karlman , Lukas Funke , Marek Vasut , Massimo Pegorer , Neha Malcom Francis , Peng Fan , Philippe Reynes , Stefan Herbrechtsmeier , Sughosh Ganu Subject: [PATCH v2 05/14] binman: Support an assumed size for missing binaries Date: Sun, 23 Jun 2024 11:55:06 -0600 Message-Id: <20240623175515.1466908-6-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240623175515.1466908-1-sjg@chromium.org> References: <20240623175515.1466908-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Binman has a the useful feature of handling missing external blobs gracefully, including allowing them to be missing, deciding whether the resulting image is functional or not and faking blobs when this is necessary for particular tools (e.g. mkimage). This feature is widely used in CI. One drawback is that if U-Boot grows too large to fit along with the required blobs, then this is not discovered until someone does a 'real' build which includes the blobs. Add a 'assume-size' property to entries to allow Binman to reserve a given size for missing external blobs. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- (no changes since v1) tools/binman/binman.rst | 7 ++++++ tools/binman/entry.py | 1 + tools/binman/etype/blob.py | 7 +++++- tools/binman/ftest.py | 28 ++++++++++++++++++++++++ tools/binman/test/326_assume_size.dts | 16 ++++++++++++++ tools/binman/test/327_assume_size_ok.dts | 16 ++++++++++++++ 6 files changed, 74 insertions(+), 1 deletion(-) create mode 100644 tools/binman/test/326_assume_size.dts create mode 100644 tools/binman/test/327_assume_size_ok.dts diff --git a/tools/binman/binman.rst b/tools/binman/binman.rst index 230e055667f..872e9746c8c 100644 --- a/tools/binman/binman.rst +++ b/tools/binman/binman.rst @@ -711,6 +711,13 @@ missing-msg: information about what needs to be fixed. See missing-blob-help for the message for each tag. +assume-size: + Sets the assumed size of a blob entry if it is missing. This allows for a + check that the rest of the image fits into the available space, even when + the contents are not available. If the entry is missing, Binman will use + this assumed size for the entry size, including creating a fake file of that + size if requested. + no-expanded: By default binman substitutes entries with expanded versions if available, so that a `u-boot` entry type turns into `u-boot-expanded`, for example. The diff --git a/tools/binman/entry.py b/tools/binman/entry.py index 2ed65800d22..219d5dcecab 100644 --- a/tools/binman/entry.py +++ b/tools/binman/entry.py @@ -315,6 +315,7 @@ class Entry(object): self.overlap = fdt_util.GetBool(self._node, 'overlap') if self.overlap: self.required_props += ['offset', 'size'] + self.assume_size = fdt_util.GetInt(self._node, 'assume-size', 0) # This is only supported by blobs and sections at present self.compress = fdt_util.GetString(self._node, 'compress', 'none') diff --git a/tools/binman/etype/blob.py b/tools/binman/etype/blob.py index 064fae50365..041e1122953 100644 --- a/tools/binman/etype/blob.py +++ b/tools/binman/etype/blob.py @@ -48,11 +48,16 @@ class Entry_blob(Entry): self.external and (self.optional or self.section.GetAllowMissing())) # Allow the file to be missing if not self._pathname: + if not fake_size and self.assume_size: + fake_size = self.assume_size self._pathname, faked = self.check_fake_fname(self._filename, fake_size) self.missing = True if not faked: - self.SetContents(b'') + content_size = 0 + if self.assume_size: # Ensure we get test coverage on next line + content_size = self.assume_size + self.SetContents(tools.get_bytes(0, content_size)) return True self.ReadBlobContents() diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index 8a44bc051b3..bd0a10ff885 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -7460,5 +7460,33 @@ fdt fdtmap Extract the devicetree blob from the fdtmap with self.assertRaises(ValueError) as e: self._DoReadFile('323_capsule_accept_revert_missing.dts') + def test_assume_size(self): + """Test handling of the assume-size property for external blob""" + with self.assertRaises(ValueError) as e: + self._DoTestFile('326_assume_size.dts', allow_missing=True, + allow_fake_blobs=True) + self.assertIn("contents size 0xa (10) exceeds section size 0x9 (9)", + str(e.exception)) + + def test_assume_size_ok(self): + """Test handling of the assume-size where it fits OK""" + with test_util.capture_sys_output() as (stdout, stderr): + self._DoTestFile('327_assume_size_ok.dts', allow_missing=True, + allow_fake_blobs=True) + err = stderr.getvalue() + self.assertRegex( + err, + "Image '.*' has faked external blobs and is non-functional: .*") + + def test_assume_size_no_fake(self): + """Test handling of the assume-size where it fits OK""" + with test_util.capture_sys_output() as (stdout, stderr): + self._DoTestFile('327_assume_size_ok.dts', allow_missing=True) + err = stderr.getvalue() + self.assertRegex( + err, + "Image '.*' is missing external blobs and is non-functional: .*") + + if __name__ == "__main__": unittest.main() diff --git a/tools/binman/test/326_assume_size.dts b/tools/binman/test/326_assume_size.dts new file mode 100644 index 00000000000..4c5f8b418d8 --- /dev/null +++ b/tools/binman/test/326_assume_size.dts @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + size = <9>; + blob-ext { + filename = "assume_blob"; + assume-size = <10>; + }; + }; +}; diff --git a/tools/binman/test/327_assume_size_ok.dts b/tools/binman/test/327_assume_size_ok.dts new file mode 100644 index 00000000000..00ed726f872 --- /dev/null +++ b/tools/binman/test/327_assume_size_ok.dts @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + size = <10>; + blob-ext { + filename = "assume_blob"; + assume-size = <10>; + }; + }; +}; From patchwork Sun Jun 23 17:55:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1951286 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=nnohYxEY; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W6f1y00dvz20WR for ; Mon, 24 Jun 2024 03:57:25 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id CF2A988387; Sun, 23 Jun 2024 19:55:32 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="nnohYxEY"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 23F0988549; Sun, 23 Jun 2024 19:55:28 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd2e.google.com (mail-io1-xd2e.google.com [IPv6:2607:f8b0:4864:20::d2e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 5D0DD8855C for ; Sun, 23 Jun 2024 19:55:25 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd2e.google.com with SMTP id ca18e2360f4ac-7f38f5584ebso117232539f.3 for ; Sun, 23 Jun 2024 10:55:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719165324; x=1719770124; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=asc59Z1ZDtMFES+Na6cEttWuA//ajEC8Am+1ZXIXMXY=; b=nnohYxEY8Rym8CgQ39a0+5EsYVdki+eoi9Ak8ZzZh0MdfkJZjL9JBnkWhcQRibKCte aXi6Bl6zKhTO3e71qUetXivOAuEc6yNBMiD+slI2v0363HrLp7xG9WA0dPozOsroHRZA JNd1TLM3PPe7MT12jgidiY3nRh/iE/RIzMvuY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719165324; x=1719770124; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=asc59Z1ZDtMFES+Na6cEttWuA//ajEC8Am+1ZXIXMXY=; b=ZuDWamq8K7u5J9nbwJWBoM5Cbv/oi6CN+T1q1ptaPC1ccHilGEzJobEypp/elkkFjI hozQDiNR75DKv2BZLJ28OcMiT/7vP3hwXzNKwCKe0LbwAE4TLNPc5MLwfxPh3jezolbj fDXYNBs4roclmhVkPnZiNqJDV4rDEYE6WLN27LAsC7jbe+XoihJqsjSuq1dqgWn5cBSe eQSDT7c69pkU2Yjx8jFgcDnuP1LYkaqgtd5165HRk3HvscfgETxneprZ5GVWwAwtiEww k7p+arARyCqvUCTx7SoozX0rQGWphJ2idBBO6RrzqkwJwNEBg7KCKYIEIABJHqgUQBtl k4FA== X-Gm-Message-State: AOJu0YytPX8jTqkinIoicSUHJoHQiWW5CpB5p9Bpql4VZy67iJfI7w1t muG2Bk/v+7KxRg9vDBsKlEEkptLT2b08SpdoxPf2H7wkD61TLzn1v330euqgRhSDsactxLbmxq7 8Ww== X-Google-Smtp-Source: AGHT+IFuXQ2X6poqfrs6c9g0WdBsYODnh4TWtZEzS9qqlWV1LU2fwja70PK7BYWzkvV4OqPfDj+0+A== X-Received: by 2002:a05:6602:6c03:b0:7ea:fc3a:fff6 with SMTP id ca18e2360f4ac-7f3a4dc3607mr353921639f.6.1719165324004; Sun, 23 Jun 2024 10:55:24 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f391fc9b12sm134472239f.5.2024.06.23.10.55.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Jun 2024 10:55:23 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass , Alper Nebi Yasak , Neha Malcom Francis , Peng Fan , Philippe Reynes , Stefan Herbrechtsmeier Subject: [PATCH v2 06/14] binman: Make Intel ME default to position 0x1000 Date: Sun, 23 Jun 2024 11:55:07 -0600 Message-Id: <20240623175515.1466908-7-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240623175515.1466908-1-sjg@chromium.org> References: <20240623175515.1466908-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean This cannot ever go at offset 0 since the descriptor is there. Use a better offset for the ME, as used by link and coral, for example. This matters when we start using assumed sizes for missing blobs. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- (no changes since v1) tools/binman/etype/intel_descriptor.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/binman/etype/intel_descriptor.py b/tools/binman/etype/intel_descriptor.py index 7fe88a9ec1a..3ce967fe81a 100644 --- a/tools/binman/etype/intel_descriptor.py +++ b/tools/binman/etype/intel_descriptor.py @@ -59,7 +59,7 @@ class Entry_intel_descriptor(Entry_blob_ext): if self.missing: # Return zero offsets so that these entries get placed somewhere if self.HasSibling('intel-me'): - info['intel-me'] = [0, None] + info['intel-me'] = [0x1000, None] return info offset = self.data.find(FD_SIGNATURE) if offset == -1: From patchwork Sun Jun 23 17:55:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1951287 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=K9kPhlU/; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W6f273gdpz20WR for ; Mon, 24 Jun 2024 03:57:35 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 360E18850A; Sun, 23 Jun 2024 19:55:33 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="K9kPhlU/"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 34C618854A; Sun, 23 Jun 2024 19:55:28 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd29.google.com (mail-io1-xd29.google.com [IPv6:2607:f8b0:4864:20::d29]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 1D3D3885B0 for ; Sun, 23 Jun 2024 19:55:26 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd29.google.com with SMTP id ca18e2360f4ac-7f38da3800aso120621539f.0 for ; Sun, 23 Jun 2024 10:55:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719165324; x=1719770124; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=55AI1Zdq2PtMQvY/0hkcvJqBd9KlBMTMR5uFE0YcI/k=; b=K9kPhlU/arX7vBws8f1lS4X1yQEN/oY8SMRYsqRkbyTxmOSQV6EhRXDawR1QQ7E6bo H3iLtfcApPuIBkZBPWFlB1Qs0v4AhL8sly0tUPjmP3TRjU7iCBOrCLRzYDyD+qOXDvY2 HiOpDCBGbto10uUoT+fKrWt9ODuxjOTPRCsX8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719165324; x=1719770124; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=55AI1Zdq2PtMQvY/0hkcvJqBd9KlBMTMR5uFE0YcI/k=; b=m6ZTExBQkLLtqhuFMy8tf4l/kr/lRoSnxBp0OXeGShlfLMvuDMTOvMt+mgxW3HmtZh iUw801L+INuGJI2H9DL2i5ZqiKb+qFvKqS9yXoRy438LWfC0Mss9A2hoGIfss+EK2+M6 Y0Bt4eTP+BK9y4mECJej++SiWUzlAkDVyFUwsKLS3/ZZMCRCCq+XGvgh/D3jlIVXEg3w qhE7eKwgRlpdwVZ48zt2KDf7cAjObNEUEF2LnEgWxMS2QlbdUw4TpA7pbxAX+onEvzqz oqAH07Y0ue58MJe6CwpBHttIycUGYdQxN4/ioz2HPCUA6zS7Dbc6ySq2lo9DeBQxrGz0 TA6w== X-Gm-Message-State: AOJu0YzuEMEV1KrGyeFSqy29J7BjRYk436m438Jlfz47+Uvq18fkbjhh 3eKk0iuMmlvnNe1x4d/nmfjd6nGypA/L17U+faSluHIAf3sKMvDoZxngXsyyKiFJ4iIIWHoYGXj kcg== X-Google-Smtp-Source: AGHT+IFiPmtB504qAUNz4GhI1VojYHaBLyVfprpWc0kKnCqCioDLCKUjsMmjna/C34zAFXVkkM3nog== X-Received: by 2002:a05:6602:150d:b0:7f1:3e1a:db4c with SMTP id ca18e2360f4ac-7f3a78044e1mr406448939f.16.1719165324663; Sun, 23 Jun 2024 10:55:24 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f391fc9b12sm134472239f.5.2024.06.23.10.55.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Jun 2024 10:55:24 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass , Bin Meng Subject: [PATCH v2 07/14] x86: Set up some assumed sizes for binary blobs Date: Sun, 23 Jun 2024 11:55:08 -0600 Message-Id: <20240623175515.1466908-8-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240623175515.1466908-1-sjg@chromium.org> References: <20240623175515.1466908-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Add assumed sizes so that Binman can check that the U-Boot binaries do not grow too large. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- (no changes since v1) arch/x86/dts/u-boot.dtsi | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/x86/dts/u-boot.dtsi b/arch/x86/dts/u-boot.dtsi index e0de3318091..fdd28979e0b 100644 --- a/arch/x86/dts/u-boot.dtsi +++ b/arch/x86/dts/u-boot.dtsi @@ -24,9 +24,11 @@ #ifdef CONFIG_HAVE_INTEL_ME intel-descriptor { filename = CONFIG_FLASH_DESCRIPTOR_FILE; + assume-size = <0x1000>; }; intel-me { filename = CONFIG_INTEL_ME_FILE; + assume-size = <0x1ff000>; }; #endif #ifdef CONFIG_TPL @@ -87,6 +89,7 @@ #ifdef CONFIG_HAVE_MRC intel-mrc { offset = ; + assume-size = <0x2fc94>; }; #endif #ifdef CONFIG_FSP_VERSION1 @@ -98,6 +101,7 @@ #ifdef CONFIG_FSP_VERSION2 intel-descriptor { filename = CONFIG_FLASH_DESCRIPTOR_FILE; + assume-size = <4096>; }; intel-ifwi { filename = CONFIG_IFWI_INPUT_FILE; @@ -139,6 +143,7 @@ intel-vga { filename = CONFIG_VGA_BIOS_FILE; offset = ; + assume-size = <0x10000>; }; #endif #ifdef CONFIG_HAVE_VBT From patchwork Sun Jun 23 17:55:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1951288 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=g5gjoQhs; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W6f2J71xvz20WR for ; Mon, 24 Jun 2024 03:57:44 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 957B3885B5; Sun, 23 Jun 2024 19:55:33 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="g5gjoQhs"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id F3C60885B5; Sun, 23 Jun 2024 19:55:28 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd2a.google.com (mail-io1-xd2a.google.com [IPv6:2607:f8b0:4864:20::d2a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id D9E35882B7 for ; Sun, 23 Jun 2024 19:55:26 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd2a.google.com with SMTP id ca18e2360f4ac-7ec07f4170bso144585139f.0 for ; Sun, 23 Jun 2024 10:55:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719165325; x=1719770125; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oyLjbiw1FIti3TMf3PuFtVV65dBrTPn0NPhmAIvzCyQ=; b=g5gjoQhsKnrMjnXJG1A91pUeBdJdkZH1zKXX/wx6ecSxruvL650CkzJWxBkK1XNNfP WyTTquPn+0iFjVr+vaYtSpK6mDLEepahuM3D47+59RN9AfVeQpUxF2b5dMqRKJBq5Dx3 ASPMq8nsrcdJ4tCYebtWpSxMr+fGozHmKYQVY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719165325; x=1719770125; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oyLjbiw1FIti3TMf3PuFtVV65dBrTPn0NPhmAIvzCyQ=; b=t10QMga5Su+YZo3HQEJCqce9gy2DOlXi6wI7t5oYDo6QpWe7z+6su+lX6MEhnR8X6a ZFWgxnLKcr5AYCaANvRc7eL5Yjll/o9l+AUh6S4EyVcgbMT4PdZ5Y1Ky2U7iIg99Q3x2 6wBXQHrxVyrlzr6Ha0lFe0b56a7DBQhVVZ3M24mhVmDlJd2XkEruP86P3mpAEkuhyDMb rYH0W5G+QPWAJjpSK/kiVs3Ev8pNkonFkhgafVwPgoFRvPvbMrFfucBCOnPIOYxTs/P9 aaAGGR98+a4Wx5zMS8tgZKzsuA8hyQuvUyn7So+muYRtHBuzUT0wOWTDC9/8yxlP/mDD R3IA== X-Gm-Message-State: AOJu0YwY5CmAZSfzRPDvU6TwoTrGSNnvyoGTU5DvmUbI50hpV/9/TXOe FCg+Wh09VL0YQ45iBfn+cNMhyHYv1Lrf/dRei84dfyXTWGsb7YS95ccdDHNHFk5+eQgtikqTTmv bpw== X-Google-Smtp-Source: AGHT+IH+8Ayckqu8aODjrTNTfRWjaLOSp2gImiI3qRk5MP/YW9z1nxRh1zKiVAoOxbD83Sb47TzKQg== X-Received: by 2002:a05:6602:60c5:b0:7eb:7887:a496 with SMTP id ca18e2360f4ac-7f3a4db1cf6mr408978139f.4.1719165325490; Sun, 23 Jun 2024 10:55:25 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f391fc9b12sm134472239f.5.2024.06.23.10.55.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Jun 2024 10:55:25 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass Subject: [PATCH v2 08/14] buildman: Make mrproper an argument to _reconfigure() Date: Sun, 23 Jun 2024 11:55:09 -0600 Message-Id: <20240623175515.1466908-9-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240623175515.1466908-1-sjg@chromium.org> References: <20240623175515.1466908-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Pass this in so the caller can change it independently of the member variable. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- (no changes since v1) tools/buildman/builderthread.py | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index a8599c0bb2a..5d4426bf0d1 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -240,7 +240,7 @@ class BuilderThread(threading.Thread): return args, cwd, src_dir def _reconfigure(self, commit, brd, cwd, args, env, config_args, config_out, - cmd_list): + cmd_list, mrproper): """Reconfigure the build Args: @@ -251,11 +251,12 @@ class BuilderThread(threading.Thread): env (dict): Environment strings config_args (list of str): defconfig arg for this board cmd_list (list of str): List to add the commands to, for logging + mrproper (bool): True to run mrproper first Returns: CommandResult object """ - if self.mrproper: + if mrproper: result = self.make(commit, brd, 'mrproper', cwd, 'mrproper', *args, env=env) config_out.write(result.combined) @@ -419,7 +420,8 @@ class BuilderThread(threading.Thread): cmd_list = [] if do_config or adjust_cfg: result = self._reconfigure( - commit, brd, cwd, args, env, config_args, config_out, cmd_list) + commit, brd, cwd, args, env, config_args, config_out, cmd_list, + self.mrproper) do_config = False # No need to configure next time if adjust_cfg: cfgutil.adjust_cfg_file(cfg_file, adjust_cfg) From patchwork Sun Jun 23 17:55:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1951289 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=RXdy6TZk; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W6f2W10Gnz20WR for ; Mon, 24 Jun 2024 03:57:55 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 08AB3885D4; Sun, 23 Jun 2024 19:55:34 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="RXdy6TZk"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 0CAED88327; Sun, 23 Jun 2024 19:55:30 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id BCD8188387 for ; Sun, 23 Jun 2024 19:55:27 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd30.google.com with SMTP id ca18e2360f4ac-7f38f5584ebso117232839f.3 for ; Sun, 23 Jun 2024 10:55:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719165326; x=1719770126; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2+xaXs6sO3d9vTAI+00rdIgxFfQEj9CD7C75wf8pjpU=; b=RXdy6TZkx65+pObtVsvaNgyaZGLg6sBaP4AACCt0+qdms0D9MK5pkKRvZZ3yEL6A75 Rjn1YnVXnr2BllXTuOs2pYUeq0KWlmyFaEa0bhr9DpaJ3XovRUE9Px+cRC1Hy7Bq+OeV JOYjdPaSXN4dG8btf1cXfIssBtbKcwUj8yymE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719165326; x=1719770126; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2+xaXs6sO3d9vTAI+00rdIgxFfQEj9CD7C75wf8pjpU=; b=ioK79KKPlh6d5gmItvKu+9VsTbQE6ERAXKz7PnKendJ+LkEtHv5qKEotPiOblw9Zmm olSVVyyUisU+Qkq3d8O5ihjuEzyu3uj5niKTnnB9uqNHpPbaYOhtn4C0GisSc7xqMiPO rNY2Qy1RWGtMDvWvI+KAAY4QrFMBgEMsB2u3jrVjScI2PCwyytYtjwnkM8bcOxyeQPhm BRNgqkwStxnNGH53HjsqoyQ9fl4ROOpLPsDrTuafui1l+YMmRWyrZh0v63XR+AQRXUCP 9eIvMQpym8jPMuvXIqtKOrKIUyjAzOcgkdFAJZnkTrEmIYhfVTP780vjPDoMIhkGYQVi /GcQ== X-Gm-Message-State: AOJu0Yw6OyV//mXVTvE8WvLFdkhAY5wHzH2l0uw2xwwBdt3iD2v5gzAH OXuMREDrdWd5RYNOaEuKRoBLPwn8z6q0AZ4HixiOWoBpcF0rkWKBukqWQrv06bgeuog4JThGYLV tzw== X-Google-Smtp-Source: AGHT+IFGClowY2syo0Q3HlIVqMyH/j6vvC18Mbhr+DlitMfNBLLYeISmqZQgQpw9qYUATJEdBCt7og== X-Received: by 2002:a05:6602:1582:b0:7eb:7f2e:5b3a with SMTP id ca18e2360f4ac-7f3a4db2390mr373788439f.2.1719165326252; Sun, 23 Jun 2024 10:55:26 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f391fc9b12sm134472239f.5.2024.06.23.10.55.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Jun 2024 10:55:25 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass Subject: [PATCH v2 09/14] buildman: Make mrproper an argument to _config_and_build() Date: Sun, 23 Jun 2024 11:55:10 -0600 Message-Id: <20240623175515.1466908-10-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240623175515.1466908-1-sjg@chromium.org> References: <20240623175515.1466908-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Pass this in so the caller can change it independently of the member variable. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- (no changes since v1) tools/buildman/builderthread.py | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index 5d4426bf0d1..ff63f9397e6 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -381,7 +381,7 @@ class BuilderThread(threading.Thread): commit = 'current' return commit - def _config_and_build(self, commit_upto, brd, work_dir, do_config, + def _config_and_build(self, commit_upto, brd, work_dir, do_config, mrproper, config_only, adjust_cfg, commit, out_dir, out_rel_dir, result): """Do the build, configuring first if necessary @@ -391,6 +391,7 @@ class BuilderThread(threading.Thread): brd (Board): Board to create arguments for work_dir (str): Directory to which the source will be checked out do_config (bool): True to run a make _defconfig on the source + mrproper (bool): True to run mrproper first config_only (bool): Only configure the source, do not build it adjust_cfg (list of str): See the cfgutil module and run_commit() commit (Commit): Commit only being built @@ -421,7 +422,7 @@ class BuilderThread(threading.Thread): if do_config or adjust_cfg: result = self._reconfigure( commit, brd, cwd, args, env, config_args, config_out, cmd_list, - self.mrproper) + mrproper) do_config = False # No need to configure next time if adjust_cfg: cfgutil.adjust_cfg_file(cfg_file, adjust_cfg) @@ -500,8 +501,9 @@ class BuilderThread(threading.Thread): if self.toolchain: commit = self._checkout(commit_upto, work_dir) result, do_config = self._config_and_build( - commit_upto, brd, work_dir, do_config, config_only, - adjust_cfg, commit, out_dir, out_rel_dir, result) + commit_upto, brd, work_dir, do_config, self.mrproper, + config_only, adjust_cfg, commit, out_dir, out_rel_dir, + result) result.already_done = False result.toolchain = self.toolchain From patchwork Sun Jun 23 17:55:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1951290 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=VVIJrPNx; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W6f2j37c8z20WR for ; Mon, 24 Jun 2024 03:58:05 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 6BA89885E1; Sun, 23 Jun 2024 19:55:34 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="VVIJrPNx"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id A589988549; Sun, 23 Jun 2024 19:55:30 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd2d.google.com (mail-io1-xd2d.google.com [IPv6:2607:f8b0:4864:20::d2d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 5EB5988433 for ; Sun, 23 Jun 2024 19:55:28 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd2d.google.com with SMTP id ca18e2360f4ac-7f3b36cc47bso3594439f.0 for ; Sun, 23 Jun 2024 10:55:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719165327; x=1719770127; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=v/nMaNRMwNguxRS6FV41YnbcW7nKTBB7UT/MYiDOvB0=; b=VVIJrPNxOHIjI6aAr7D3fad5WQCcB60FknhE5CjWJko5IDfS6fAk2ucjrKhKWWKVDq Z4Zqt5rGaAo8d5s4cpehhbbEXHgT1iXyUBIN088SzTBLybhSMWARyCiDz3FjyV0b0nio 3Nd3ZwzK3uNocy0V9iJAV0u4j35f1jVCfu9dw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719165327; x=1719770127; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=v/nMaNRMwNguxRS6FV41YnbcW7nKTBB7UT/MYiDOvB0=; b=HD1ScEIaCXVKwkp3KOc7T0vJ5Upke8H+UBlxEDGYxE6znMNHl66DAtEBj12o7fFlln Ko0p4qNdz7qcm+AKBBGEM5E7pZ2NhC54Y1ekUCH4kRh9+hfEy+hX0TCytrXouGAowjVO 30C6VQwHoNF1W6U+U4RcpRlXesg8Hc8KBJQUFAgv3UaECHdojnfU0iFPVV3uOzFyCW7A iQCbOExwNP+e70DLXwK+5L2xGEaozM34f4be1ox0BsaMc/4T5kxHkxajuxKvNWg2fQTb yAvqX8hQLwajZxJH46iMyW83uwthYGbQm/Y7TaQ7q0e4cXnTU5ApPxaC/aSdVhVGL9H0 KPzg== X-Gm-Message-State: AOJu0Yxicar904G0IuIInlejm3k6G2KHeMbA/qOktD+Fj2gZsqDb8Uze eMRPhXpxW7TyxJ36U1cbx852y68RiVmR9S+v+XwVxd/U4yuCfekkkpy8G4iYFuNl20maKDMbhMX rjg== X-Google-Smtp-Source: AGHT+IF20Naonbyiv5bKlX8k7QSEiJWNA5vGDVrsYsYCy2AvtFpyU1WTO4JdeuEKlYSrl+6LkuXaFw== X-Received: by 2002:a05:6602:164e:b0:7eb:6d0a:613f with SMTP id ca18e2360f4ac-7f3a74c1b25mr296767339f.11.1719165327041; Sun, 23 Jun 2024 10:55:27 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f391fc9b12sm134472239f.5.2024.06.23.10.55.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Jun 2024 10:55:26 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass Subject: [PATCH v2 10/14] buildman: Make mrproper an argument to run_commit() Date: Sun, 23 Jun 2024 11:55:11 -0600 Message-Id: <20240623175515.1466908-11-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240623175515.1466908-1-sjg@chromium.org> References: <20240623175515.1466908-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Pass this in so the caller can change it independently of the member variable. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- (no changes since v1) tools/buildman/builderthread.py | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index ff63f9397e6..0a7ff2e083e 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -448,9 +448,9 @@ class BuilderThread(threading.Thread): result.cmd_list = cmd_list return result, do_config - def run_commit(self, commit_upto, brd, work_dir, do_config, config_only, - force_build, force_build_failures, work_in_output, - adjust_cfg): + def run_commit(self, commit_upto, brd, work_dir, do_config, mrproper, + config_only, force_build, force_build_failures, + work_in_output, adjust_cfg): """Build a particular commit. If the build is already done, and we are not forcing a build, we skip @@ -461,6 +461,7 @@ class BuilderThread(threading.Thread): brd (Board): Board to build work_dir (str): Directory to which the source will be checked out do_config (bool): True to run a make _defconfig on the source + mrproper (bool): True to run mrproper first config_only (bool): Only configure the source, do not build it force_build (bool): Force a build even if one was previously done force_build_failures (bool): Force a bulid if the previous result @@ -501,7 +502,7 @@ class BuilderThread(threading.Thread): if self.toolchain: commit = self._checkout(commit_upto, work_dir) result, do_config = self._config_and_build( - commit_upto, brd, work_dir, do_config, self.mrproper, + commit_upto, brd, work_dir, do_config, mrproper, config_only, adjust_cfg, commit, out_dir, out_rel_dir, result) result.already_done = False @@ -692,7 +693,8 @@ class BuilderThread(threading.Thread): force_build = False for commit_upto in range(0, len(job.commits), job.step): result, request_config = self.run_commit(commit_upto, brd, - work_dir, do_config, self.builder.config_only, + work_dir, do_config, self.mrproper, + self.builder.config_only, force_build or self.builder.force_build, self.builder.force_build_failures, job.work_in_output, job.adjust_cfg) @@ -703,8 +705,8 @@ class BuilderThread(threading.Thread): # with a reconfig. if self.builder.force_config_on_failure: result, request_config = self.run_commit(commit_upto, - brd, work_dir, True, False, True, False, - job.work_in_output, job.adjust_cfg) + brd, work_dir, True, self.mrproper, False, True, + False, job.work_in_output, job.adjust_cfg) did_config = True if not self.builder.force_reconfig: do_config = request_config @@ -748,7 +750,7 @@ class BuilderThread(threading.Thread): else: # Just build the currently checked-out build result, request_config = self.run_commit(None, brd, work_dir, True, - self.builder.config_only, True, + self.mrproper, self.builder.config_only, True, self.builder.force_build_failures, job.work_in_output, job.adjust_cfg) result.commit_upto = 0 From patchwork Sun Jun 23 17:55:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1951291 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=Gs3ntqzz; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W6f2v21Ccz20WR for ; Mon, 24 Jun 2024 03:58:15 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id D3BC3885E7; Sun, 23 Jun 2024 19:55:34 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Gs3ntqzz"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 4030188433; Sun, 23 Jun 2024 19:55:31 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd29.google.com (mail-io1-xd29.google.com [IPv6:2607:f8b0:4864:20::d29]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 0587D882E2 for ; Sun, 23 Jun 2024 19:55:29 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd29.google.com with SMTP id ca18e2360f4ac-7eb7c0f9784so135777239f.2 for ; Sun, 23 Jun 2024 10:55:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719165327; x=1719770127; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yEIyFClr1N2iZpbhFlJ046p2FxvzCXNHilvLmZrk4Mg=; b=Gs3ntqzz2MZqBHEZYTWl1gvg930BPFA1ohy1BOXmex6KvjgNnzpLrRhJsM3n6F4Xvs 52xMkCtqG8151mrl3I7WutMHhqsdmau4DzC9Lj5+Cac+VMYmSS4jbMO7JRQbI7hJ28Nn 8MhkW5k1lSlaM+4lUGIDZBVO93lmANm7nO558= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719165327; x=1719770127; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yEIyFClr1N2iZpbhFlJ046p2FxvzCXNHilvLmZrk4Mg=; b=mGYUUMaf4vNjA3WyJc5tBBqVGAHkSuhl887lZgEm5BRRyD3+bFImFArYg5jj49FZdg dVywnSEnhUsz8KWcWSgNTKPVPIcO0jeRCJtrQPxeV2c1o81/WbjVxXbTclPiytYOXMQi 6RVgpeKiGUceRHNa9hHAtDwbbSxTRcTz1jW7JvNJjm/3vhC9ijYRh8mcq8dVDamqDHGE isEsr0maY5rZ30sGYyqNUTjzMRu6jYP6Jkjoi6w7sNi4k1CcLApVihZoCaTlGMW4BfAM qDQUcBraup6WqDBiZjSqgURxi7pqLaO3OC46HwdMvcalFUBjS4guBocy+GS0P7JSE1dG SxiQ== X-Gm-Message-State: AOJu0YzKlQYt7GrRrktzQAcYh8TEFWAXZ/WVc3pu7jMpFIJyvCTvrtc9 iyBliVorMrCvSO6AnzdAvTdtTvSfnL4SPyhqvj6R/SPdU9+ZaXBu7wr1aqBz7/9Js5rMDvTNDqU M6Q== X-Google-Smtp-Source: AGHT+IE4gdPvCaJiN3MRB88nL6mwWRCHoqHwUn9ZjaYBuMmD69rWcMnYANojvLGfHWD/Evc/uOIlhw== X-Received: by 2002:a05:6602:13d4:b0:7eb:85fd:e29d with SMTP id ca18e2360f4ac-7f3a75b301cmr316233139f.16.1719165327665; Sun, 23 Jun 2024 10:55:27 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f391fc9b12sm134472239f.5.2024.06.23.10.55.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Jun 2024 10:55:27 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass Subject: [PATCH v2 11/14] buildman: Avoid rebuilding when --mrproper is used Date: Sun, 23 Jun 2024 11:55:12 -0600 Message-Id: <20240623175515.1466908-12-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240623175515.1466908-1-sjg@chromium.org> References: <20240623175515.1466908-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean When this flag is enabled, 'make mrproper' is always used when reconfiguring, so there is no point in doing it again. Update this. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- (no changes since v1) tools/buildman/builderthread.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index 0a7ff2e083e..c0b1067e3f7 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -700,7 +700,7 @@ class BuilderThread(threading.Thread): job.work_in_output, job.adjust_cfg) failed = result.return_code or result.stderr did_config = do_config - if failed and not do_config: + if failed and not do_config and not self.mrproper: # If our incremental build failed, try building again # with a reconfig. if self.builder.force_config_on_failure: From patchwork Sun Jun 23 17:55:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1951292 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=kp4ojowJ; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W6f344pBRz20WR for ; Mon, 24 Jun 2024 03:58:24 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 41DCA885F2; Sun, 23 Jun 2024 19:55:35 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="kp4ojowJ"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 1BD1388403; Sun, 23 Jun 2024 19:55:32 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd41.google.com (mail-io1-xd41.google.com [IPv6:2607:f8b0:4864:20::d41]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id CC09D885CE for ; Sun, 23 Jun 2024 19:55:29 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd41.google.com with SMTP id ca18e2360f4ac-7eb6ca2cd09so110020139f.2 for ; Sun, 23 Jun 2024 10:55:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719165328; x=1719770128; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GObNPrk/tCEopd9X1kpliNcES1sI2HkzYaqk/1pvvvQ=; b=kp4ojowJtiYmMbFuWVFuLJqNTC7Po72U/N7j1iCVsDhn9zEi5oCUvtdukCAP3dtNfu iBE7hnU+0Q4jy9O7Ds942pOYReDZblTf5AP0x0LZMa8IvFJPKtuGqhtAWzte1wGZ1uK3 Ny7svncZ+F/4apj15spthOPySn4/sbWo6Ollc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719165328; x=1719770128; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GObNPrk/tCEopd9X1kpliNcES1sI2HkzYaqk/1pvvvQ=; b=o+MC6EXZNuvDZ8mrXR+b4Yuqdu2A3eJk9/Xz+l76eR24NFTZvBqYyWYAwUUNMuX/Tk xghxG8x4NHpKY7tDA80TfRDVPHhd5gYMbPmfiJ/b1eMXvELer4brHeC0Ym/99kcnd312 hFBfoQRx6T5nHWNhXyWAZeo3FbBgRDZjucu7K/iliOz/kUdJ9/8xPtecjcIVRiqSUgxT CU5PFvdBLMvcCR0TQ16hstjZ04XrW880deSfOcVH+Uzh7BrNRDFtgTcHuMSiNSl/kxFP avdKmOFrzJ73mTijh5pan+7stVnH6p7Aptt9UYmh0uGIje5C3bknS+72T8R/RRNIgIO/ V9bA== X-Gm-Message-State: AOJu0Yw1HBF9CvnCyQ0+uLP6HSykohLGA5AK2d0hyD4o9ZM+sIePmn3j zccmjD6pXAtpELg8w+rdjaBX3F5SlutTqbobbmW73SoY0SGKjUFYccN2XlzAtXnFAVhBxfy6CBm 5+zud X-Google-Smtp-Source: AGHT+IGc6ZH4kAXD7Fw3E07ZH1DYbbFMyY8k8x1SAyLdLVCLrpJRhNbBivu2M68m8s95OOpceko7MQ== X-Received: by 2002:a05:6602:1606:b0:7eb:eb7e:93c3 with SMTP id ca18e2360f4ac-7f3a75ddb10mr269156739f.13.1719165328318; Sun, 23 Jun 2024 10:55:28 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f391fc9b12sm134472239f.5.2024.06.23.10.55.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Jun 2024 10:55:28 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass , Alexey Brodkin Subject: [PATCH v2 12/14] buildman: Add a flag to force mrproper on failure Date: Sun, 23 Jun 2024 11:55:13 -0600 Message-Id: <20240623175515.1466908-13-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240623175515.1466908-1-sjg@chromium.org> References: <20240623175515.1466908-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean When a file is removed by a commit (e.g. include/common.h yay!) it can cause incremental build failures since one of the dependency files from a previous build may mention the file. Add an option to run 'make mrproper' automatically when a build fails. This can be used to automatically resolve the problem, without always adding the large overhead of 'make mrproper' to every build. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- (no changes since v1) tools/buildman/builder.py | 18 ++++++++++-------- tools/buildman/builderthread.py | 6 ++++-- tools/buildman/buildman.rst | 3 ++- tools/buildman/cmdline.py | 4 +++- tools/buildman/control.py | 1 + 5 files changed, 20 insertions(+), 12 deletions(-) diff --git a/tools/buildman/builder.py b/tools/buildman/builder.py index f35175b4598..c4384f53e8d 100644 --- a/tools/buildman/builder.py +++ b/tools/buildman/builder.py @@ -256,14 +256,14 @@ class Builder: def __init__(self, toolchains, base_dir, git_dir, num_threads, num_jobs, gnu_make='make', checkout=True, show_unknown=True, step=1, no_subdirs=False, full_path=False, verbose_build=False, - mrproper=False, per_board_out_dir=False, - config_only=False, squash_config_y=False, - warnings_as_errors=False, work_in_output=False, - test_thread_exceptions=False, adjust_cfg=None, - allow_missing=False, no_lto=False, reproducible_builds=False, - force_build=False, force_build_failures=False, - force_reconfig=False, in_tree=False, - force_config_on_failure=False, make_func=None): + mrproper=False, fallback_mrproper=False, + per_board_out_dir=False, config_only=False, + squash_config_y=False, warnings_as_errors=False, + work_in_output=False, test_thread_exceptions=False, + adjust_cfg=None, allow_missing=False, no_lto=False, + reproducible_builds=False, force_build=False, + force_build_failures=False, force_reconfig=False, + in_tree=False, force_config_on_failure=False, make_func=None): """Create a new Builder object Args: @@ -283,6 +283,7 @@ class Builder: PATH verbose_build: Run build with V=1 and don't use 'make -s' mrproper: Always run 'make mrproper' when configuring + fallback_mrproper: Run 'make mrproper' and retry on build failure per_board_out_dir: Build in a separate persistent directory per board rather than a thread-specific directory config_only: Only configure each build, don't build it @@ -352,6 +353,7 @@ class Builder: self.force_reconfig = force_reconfig self.in_tree = in_tree self.force_config_on_failure = force_config_on_failure + self.fallback_mrproper = fallback_mrproper if not self.squash_config_y: self.config_filenames += EXTRA_CONFIG_FILENAMES diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index c0b1067e3f7..bbe2f6f0d24 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -705,8 +705,10 @@ class BuilderThread(threading.Thread): # with a reconfig. if self.builder.force_config_on_failure: result, request_config = self.run_commit(commit_upto, - brd, work_dir, True, self.mrproper, False, True, - False, job.work_in_output, job.adjust_cfg) + brd, work_dir, True, + self.mrproper or self.builder.fallback_mrproper, + False, True, False, job.work_in_output, + job.adjust_cfg) did_config = True if not self.builder.force_reconfig: do_config = request_config diff --git a/tools/buildman/buildman.rst b/tools/buildman/buildman.rst index aae2477b5c3..bd0482af5f7 100644 --- a/tools/buildman/buildman.rst +++ b/tools/buildman/buildman.rst @@ -995,7 +995,8 @@ By default, buildman doesn't execute 'make mrproper' prior to building the first commit for each board. This reduces the amount of work 'make' does, and hence speeds up the build. To force use of 'make mrproper', use -the -m flag. This flag will slow down any buildman invocation, since it increases the amount -of work done on any build. +of work done on any build. An alternative is to use the --fallback-mrproper +flag, which retries the build with 'make mrproper' only after a build failure. One possible application of buildman is as part of a continual edit, build, edit, build, ... cycle; repeatedly applying buildman to the same change or diff --git a/tools/buildman/cmdline.py b/tools/buildman/cmdline.py index 03211bd5aa5..8dc5a8787b5 100644 --- a/tools/buildman/cmdline.py +++ b/tools/buildman/cmdline.py @@ -90,7 +90,9 @@ def add_upto_m(parser): parser.add_argument('--list-tool-chains', action='store_true', default=False, help='List available tool chains (use -v to see probing detail)') parser.add_argument('-m', '--mrproper', action='store_true', - default=False, help="Run 'make mrproper before reconfiguring") + default=False, help="Run 'make mrproper' before reconfiguring") + parser.add_argument('--fallback-mrproper', action='store_true', + default=False, help="Run 'make mrproper' and retry on build failure") parser.add_argument( '-M', '--allow-missing', action='store_true', default=False, help='Tell binman to allow missing blobs and generate fake ones as needed') diff --git a/tools/buildman/control.py b/tools/buildman/control.py index 8f6850c5211..f2dd87814c3 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -656,6 +656,7 @@ def do_buildman(args, toolchains=None, make_func=None, brds=None, no_subdirs=args.no_subdirs, full_path=args.full_path, verbose_build=args.verbose_build, mrproper=args.mrproper, + fallback_mrproper=args.fallback_mrproper, per_board_out_dir=args.per_board_out_dir, config_only=args.config_only, squash_config_y=not args.preserve_config_y, From patchwork Sun Jun 23 17:55:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1951293 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=Sp8aWjbD; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W6f3P0Pdgz20WR for ; Mon, 24 Jun 2024 03:58:41 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 59074885FD; Sun, 23 Jun 2024 19:55:39 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Sp8aWjbD"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 168678843A; Sun, 23 Jun 2024 19:55:33 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 5FD438850A for ; Sun, 23 Jun 2024 19:55:30 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd2b.google.com with SMTP id ca18e2360f4ac-7eb7c8f114eso147914339f.1 for ; Sun, 23 Jun 2024 10:55:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719165329; x=1719770129; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Qk/LjuhGcEBLV525OvjxB9qqA75bigTVPm2AbxzlWq4=; b=Sp8aWjbDhfE2w61u8lEBfV+YVlQ1ciht4wWgyBnxTXDmkeKaQ0n/UGk+LVKx0QN5DY aOB7Z3hHB0J9W5V3Oqzp40hJjPJSpYqgrm56CgAF1F9fNb4w3Y2Uy6ftgFwtHJ2ICqYy DhD8jPjigWnlkEWeuqwcwTpcrsedu5Hapyn6w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719165329; x=1719770129; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Qk/LjuhGcEBLV525OvjxB9qqA75bigTVPm2AbxzlWq4=; b=rspgQ9MYHDsVEq5Hc7Th80WB65fpNeTWfpyNOnmMxudJnK9mhabBjskZ/TuJQaUT6A Ci6JgH9uQW/sqS1caj17jxZjNItSPEPjOKXUsALHkwROgeNYFY2A6JrgLGJMWEFnMmAx I3T1RbD/M/Fjqy+SMRkd32TsWcEubaWg22dNf8EvHAAj1bHcI746MoVLwNWpjC9nya1r lzrc4N52d4+5ao19VzvxDjTYMrTRYf7NTHBcVlYxRv8/NQzDJVgiJwQcZUV7tMv9DX6o RMLDUvSQrgEVPhMs7rpAMhL1ghGWNKLmJQrG5B46uVjqWPADjPhoEmfPtoed5yezCzE+ KEDg== X-Gm-Message-State: AOJu0YzLmZ0ZcgwY0au50dOpymn3v5uBuoHQkDNWOas10rlI96WpWYaZ vT7uqJRz/HAsBLjU1yeP07P0Pj508SvQk91VozEqqDHnXZ7PHyLhJc8l7kwuD50T2/+oeFYHEjT h6g== X-Google-Smtp-Source: AGHT+IFW/X8yBbt2O9oZVLdhkzLwGGF4oADeGWEmXdehjII8V9QU76B0WVDb2h05M/nimhm3WY2lEQ== X-Received: by 2002:a05:6602:14c2:b0:7eb:80dd:83c9 with SMTP id ca18e2360f4ac-7f3a13eb9f6mr452319739f.9.1719165329097; Sun, 23 Jun 2024 10:55:29 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f391fc9b12sm134472239f.5.2024.06.23.10.55.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Jun 2024 10:55:28 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass Subject: [PATCH v2 13/14] buildman: Retry the build for current source Date: Sun, 23 Jun 2024 11:55:14 -0600 Message-Id: <20240623175515.1466908-14-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240623175515.1466908-1-sjg@chromium.org> References: <20240623175515.1466908-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Buildman retries a failed build when processing a branch, but does not do this when building current source. It is useful to do this retry in both cases, so add the logic for it. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- (no changes since v1) tools/buildman/builderthread.py | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index bbe2f6f0d24..55658487abf 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -755,6 +755,14 @@ class BuilderThread(threading.Thread): self.mrproper, self.builder.config_only, True, self.builder.force_build_failures, job.work_in_output, job.adjust_cfg) + failed = result.return_code or result.stderr + if failed and not self.mrproper: + result, request_config = self.run_commit(None, brd, work_dir, + True, self.builder.fallback_mrproper, + self.builder.config_only, True, + self.builder.force_build_failures, + job.work_in_output, job.adjust_cfg) + result.commit_upto = 0 self._write_result(result, job.keep_outputs, job.work_in_output) self._send_result(result) From patchwork Sun Jun 23 17:55:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1951294 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=XoKfI5bt; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W6f3b61MFz20WR for ; Mon, 24 Jun 2024 03:58:51 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id B723788602; Sun, 23 Jun 2024 19:55:39 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="XoKfI5bt"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 9E0B2885E7; Sun, 23 Jun 2024 19:55:34 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-io1-xd44.google.com (mail-io1-xd44.google.com [IPv6:2607:f8b0:4864:20::d44]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 4E3EB882E2 for ; Sun, 23 Jun 2024 19:55:31 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd44.google.com with SMTP id ca18e2360f4ac-7eb6aedcabcso138500639f.0 for ; Sun, 23 Jun 2024 10:55:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1719165330; x=1719770130; darn=lists.denx.de; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=js1B3Fhwj3/sKBtfEbSwgo9Es5qzmAY0dew8DHdTdiw=; b=XoKfI5bt5Tj9ie+73TWNTu8qRfpxOGuUqHpq0KT/ZrpYAu1pbBaIRbNKyqcK1yMcZW XPukAaIPeL9XSpijXtRWMWgl5iMGM5toUEMl31yIRIQENuBSVEuFPoEcxG129GsYY/G/ WVoXQlYy1aby+wJmOXFhHKyf/A0SdgaR3gefs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719165330; x=1719770130; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=js1B3Fhwj3/sKBtfEbSwgo9Es5qzmAY0dew8DHdTdiw=; b=Dao490Mx5En7ANBsG4mn3xZq4sh6MpRCsSNYsKeXEZx6Q2Z4G9rL9tU2vCAmPGNoKt 2UJRcFoVz2gdKjIgzhfwjBGRGENVnoS5PcsxMAwYU+dhoJjP51MySgXRlT3W9N3rBYEM e5o4y9k9Wj3iRAnDIe88S7Nf7jTSYqPx5AUCitZU22F6tczIjBeRTutaystd/6BaYvUN 2JA0vunIhEyC6HZAiZjUd7s9+K+FDpC8VXfIVzoxhk/O7ta4KjsAeK4pZE85EK9IC3Ov oifD/hip+lwLb7HZqdvAqM1xy3UvimNkPPNJariG14lraRw3+/zN4FXym0OyFg7Tr/8U lrYQ== X-Gm-Message-State: AOJu0YySKP+cTmM4K12rTy6AysjtpB1r6y1hB1k0hRNwbc/Kjn58cJrK M+CRITOq662eF7+tqtnMzPHZ1CkZa0LtdiMmc/873spYA8gTrPoAplp3tR8XI1K7S2by1jUCxEo smypf X-Google-Smtp-Source: AGHT+IE1mZBM8RD5/ww4ta4yM6VMc727hJw9fPasXe3Kfd6EYPxIO4R9RiINHixIo0PZy9MY/F5XRg== X-Received: by 2002:a05:6602:3f81:b0:7eb:b257:65ec with SMTP id ca18e2360f4ac-7f3a75e3622mr221851239f.20.1719165329769; Sun, 23 Jun 2024 10:55:29 -0700 (PDT) Received: from chromium.org (c-73-14-173-85.hsd1.co.comcast.net. [73.14.173.85]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-7f391fc9b12sm134472239f.5.2024.06.23.10.55.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Jun 2024 10:55:29 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass , Alexey Brodkin , Heinrich Schuchardt Subject: [PATCH v2 14/14] buildman: Add a way to limit the number of buildmans Date: Sun, 23 Jun 2024 11:55:15 -0600 Message-Id: <20240623175515.1466908-15-sjg@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240623175515.1466908-1-sjg@chromium.org> References: <20240623175515.1466908-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Buildman uses all available CPUs by default, so running more than one or two concurrent processes is not normally useful. However in some CI cases we want to be able to run several jobs at once to save time. For example, in a lab situation we may want to run a test on 20 boards at a time, since only the build step actually takes much CPU. Add an option which allows such a limit. When buildman starts up, it waits until the number of running processes goes below the limit, then claims a spot in the list. The list is maintained with a temporary file. Note that the temp file is user-specific, since it is hard to create a locked temporary file which can be accessed by any user. In most cases, only one user is running jobs on a machine, so this should not matter. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: - Move the binman assume-size feature into this series tools/buildman/buildman.rst | 5 ++ tools/buildman/cmdline.py | 2 + tools/buildman/control.py | 140 ++++++++++++++++++++++++++++++++- tools/buildman/pyproject.toml | 6 +- tools/buildman/test.py | 121 ++++++++++++++++++++++++++++ tools/u_boot_pylib/terminal.py | 7 +- 6 files changed, 277 insertions(+), 4 deletions(-) diff --git a/tools/buildman/buildman.rst b/tools/buildman/buildman.rst index bd0482af5f7..b8ff3bf1ab2 100644 --- a/tools/buildman/buildman.rst +++ b/tools/buildman/buildman.rst @@ -1286,6 +1286,11 @@ then buildman hangs. Failing to handle any eventuality is a bug in buildman and should be reported. But you can use -T0 to disable threading and hopefully figure out the root cause of the build failure. +For situations where buildman is invoked from multiple running processes, it is +sometimes useful to have buildman wait until the others have finished. Use the +--process-limit option for this: --process-limit 1 will allow only one buildman +to process jobs at a time. + Build summary ------------- diff --git a/tools/buildman/cmdline.py b/tools/buildman/cmdline.py index 8dc5a8787b5..544a391a464 100644 --- a/tools/buildman/cmdline.py +++ b/tools/buildman/cmdline.py @@ -129,6 +129,8 @@ def add_after_m(parser): default=False, help="Use an O= (output) directory per board rather than per thread") parser.add_argument('--print-arch', action='store_true', default=False, help="Print the architecture for a board (ARCH=)") + parser.add_argument('--process-limit', type=int, + default=0, help='Limit to number of buildmans running at once') parser.add_argument('-r', '--reproducible-builds', action='store_true', help='Set SOURCE_DATE_EPOCH=0 to suuport a reproducible build') parser.add_argument('-R', '--regen-board-list', type=str, diff --git a/tools/buildman/control.py b/tools/buildman/control.py index f2dd87814c3..464835c5be5 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -7,10 +7,13 @@ This holds the main control logic for buildman, when not running tests. """ +import getpass import multiprocessing import os import shutil import sys +import tempfile +import time from buildman import boards from buildman import bsettings @@ -21,10 +24,23 @@ from patman import gitutil from patman import patchstream from u_boot_pylib import command from u_boot_pylib import terminal -from u_boot_pylib.terminal import tprint +from u_boot_pylib import tools +from u_boot_pylib.terminal import print_clear, tprint TEST_BUILDER = None +# Space-separated list of buildman process IDs currently running jobs +RUNNING_FNAME = f'buildmanq.{getpass.getuser()}' + +# Lock file for access to RUNNING_FILE +LOCK_FNAME = f'{RUNNING_FNAME}.lock' + +# Wait time for access to lock (seconds) +LOCK_WAIT_S = 10 + +# Wait time to start running +RUN_WAIT_S = 300 + def get_plural(count): """Returns a plural 's' if count is not 1""" return 's' if count != 1 else '' @@ -578,6 +594,125 @@ def calc_adjust_cfg(adjust_cfg, reproducible_builds): return adjust_cfg +def read_procs(tmpdir=tempfile.gettempdir()): + """Read the list of running buildman processes + + If the list is corrupted, returns an empty list + + Args: + tmpdir (str): Temporary directory to use (for testing only) + """ + running_fname = os.path.join(tmpdir, RUNNING_FNAME) + procs = [] + if os.path.exists(running_fname): + items = tools.read_file(running_fname, binary=False).split() + try: + procs = [int(x) for x in items] + except ValueError: # Handle invalid format + pass + return procs + + +def check_pid(pid): + """Check for existence of a unix PID + + https://stackoverflow.com/questions/568271/how-to-check-if-there-exists-a-process-with-a-given-pid-in-python + + Args: + pid (int): PID to check + + Returns: + True if it exists, else False + """ + try: + os.kill(pid, 0) + except OSError: + return False + else: + return True + + +def write_procs(procs, tmpdir=tempfile.gettempdir()): + """Write the list of running buildman processes + + Args: + tmpdir (str): Temporary directory to use (for testing only) + """ + running_fname = os.path.join(tmpdir, RUNNING_FNAME) + tools.write_file(running_fname, ' '.join([str(p) for p in procs]), + binary=False) + + # Allow another user to access the file + os.chmod(running_fname, 0o666) + +def wait_for_process_limit(limit, tmpdir=tempfile.gettempdir(), + pid=os.getpid()): + """Wait until the number of buildman processes drops to the limit + + This uses FileLock to protect a 'running' file, which contains a list of + PIDs of running buildman processes. The number of PIDs in the file indicates + the number of running processes. + + When buildman starts up, it calls this function to wait until it is OK to + start the build. + + On exit, no attempt is made to remove the PID from the file, since other + buildman processes will notice that the PID is no-longer valid, and ignore + it. + + Two timeouts are provided: + LOCK_WAIT_S: length of time to wait for the lock; if this occurs, the + lock is busted / removed before trying again + RUN_WAIT_S: length of time to wait to be allowed to run; if this occurs, + the build starts, with the PID being added to the file. + + Args: + limit (int): Maximum number of buildman processes, including this one; + must be > 0 + tmpdir (str): Temporary directory to use (for testing only) + pid (int): Current process ID (for testing only) + """ + from filelock import Timeout, FileLock + + running_fname = os.path.join(tmpdir, RUNNING_FNAME) + lock_fname = os.path.join(tmpdir, LOCK_FNAME) + lock = FileLock(lock_fname) + + # Allow another user to access the file + col = terminal.Color() + tprint('Waiting for other buildman processes...', newline=False, + colour=col.RED) + + claimed = False + deadline = time.time() + RUN_WAIT_S + while True: + try: + with lock.acquire(timeout=LOCK_WAIT_S): + os.chmod(lock_fname, 0o666) + procs = read_procs(tmpdir) + + # Drop PIDs which are not running + procs = list(filter(check_pid, procs)) + + # If we haven't hit the limit, add ourself + if len(procs) < limit: + tprint('done...', newline=False) + claimed = True + if time.time() >= deadline: + tprint('timeout...', newline=False) + claimed = True + if claimed: + write_procs(procs + [pid], tmpdir) + break + + except Timeout: + tprint('failed to get lock: busting...', newline=False) + os.remove(lock_fname) + + time.sleep(1) + tprint('starting build', newline=False) + print_clear() + def do_buildman(args, toolchains=None, make_func=None, brds=None, clean_dir=False, test_thread_exceptions=False): """The main control code for buildman @@ -677,5 +812,8 @@ def do_buildman(args, toolchains=None, make_func=None, brds=None, TEST_BUILDER = builder + if args.process_limit: + wait_for_process_limit(args.process_limit) + return run_builder(builder, series.commits if series else None, brds.get_selected_dict(), args) diff --git a/tools/buildman/pyproject.toml b/tools/buildman/pyproject.toml index fe0f6421b53..68bfa45c3f4 100644 --- a/tools/buildman/pyproject.toml +++ b/tools/buildman/pyproject.toml @@ -8,7 +8,11 @@ version = "0.0.6" authors = [ { name="Simon Glass", email="sjg@chromium.org" }, ] -dependencies = ["u_boot_pylib >= 0.0.6", "patch-manager >= 0.0.6"] +dependencies = [ + "filelock >= 3.0.12", + "u_boot_pylib >= 0.0.6", + "patch-manager >= 0.0.6" +] description = "Buildman build tool for U-Boot" readme = "README.rst" requires-python = ">=3.7" diff --git a/tools/buildman/test.py b/tools/buildman/test.py index f92add7a7c5..d68395c2164 100644 --- a/tools/buildman/test.py +++ b/tools/buildman/test.py @@ -2,12 +2,14 @@ # Copyright (c) 2012 The Chromium OS Authors. # +from filelock import FileLock import os import shutil import sys import tempfile import time import unittest +from unittest.mock import patch from buildman import board from buildman import boards @@ -156,6 +158,11 @@ class TestBuild(unittest.TestCase): if not os.path.isdir(self.base_dir): os.mkdir(self.base_dir) + self.cur_time = 0 + self.valid_pids = [] + self.finish_time = None + self.finish_pid = None + def tearDown(self): shutil.rmtree(self.base_dir) @@ -747,6 +754,120 @@ class TestBuild(unittest.TestCase): self.assertEqual([ ['MARY="mary"', 'Missing expected line: CONFIG_MARY="mary"']], result) + def get_procs(self): + running_fname = os.path.join(self.base_dir, control.RUNNING_FNAME) + items = tools.read_file(running_fname, binary=False).split() + return [int(x) for x in items] + + def get_time(self): + return self.cur_time + + def inc_time(self, amount): + self.cur_time += amount + + # Handle a process exiting + if self.finish_time == self.cur_time: + self.valid_pids = [pid for pid in self.valid_pids + if pid != self.finish_pid] + + def kill(self, pid, signal): + if pid not in self.valid_pids: + raise OSError('Invalid PID') + + def test_process_limit(self): + """Test wait_for_process_limit() function""" + tmpdir = self.base_dir + + with (patch('time.time', side_effect=self.get_time), + patch('time.sleep', side_effect=self.inc_time), + patch('os.kill', side_effect=self.kill)): + # Grab the process. Since there is no other profcess, this should + # immediately succeed + control.wait_for_process_limit(1, tmpdir=tmpdir, pid=1) + lines = terminal.get_print_test_lines() + self.assertEqual(0, self.cur_time) + self.assertEqual('Waiting for other buildman processes...', + lines[0].text) + self.assertEqual(self._col.RED, lines[0].colour) + self.assertEqual(False, lines[0].newline) + self.assertEqual(True, lines[0].bright) + + self.assertEqual('done...', lines[1].text) + self.assertEqual(None, lines[1].colour) + self.assertEqual(False, lines[1].newline) + self.assertEqual(True, lines[1].bright) + + self.assertEqual('starting build', lines[2].text) + self.assertEqual([1], control.read_procs(tmpdir)) + self.assertEqual(None, lines[2].colour) + self.assertEqual(False, lines[2].newline) + self.assertEqual(True, lines[2].bright) + + # Try again, with a different PID...this should eventually timeout + # and start the build anyway + self.cur_time = 0 + self.valid_pids = [1] + control.wait_for_process_limit(1, tmpdir=tmpdir, pid=2) + lines = terminal.get_print_test_lines() + self.assertEqual('Waiting for other buildman processes...', + lines[0].text) + self.assertEqual('timeout...', lines[1].text) + self.assertEqual(None, lines[1].colour) + self.assertEqual(False, lines[1].newline) + self.assertEqual(True, lines[1].bright) + self.assertEqual('starting build', lines[2].text) + self.assertEqual([1, 2], control.read_procs(tmpdir)) + self.assertEqual(control.RUN_WAIT_S, self.cur_time) + + # Check lock-busting + self.cur_time = 0 + self.valid_pids = [1, 2] + lock_fname = os.path.join(tmpdir, control.LOCK_FNAME) + lock = FileLock(lock_fname) + lock.acquire(timeout=1) + control.wait_for_process_limit(1, tmpdir=tmpdir, pid=3) + lines = terminal.get_print_test_lines() + self.assertEqual('Waiting for other buildman processes...', + lines[0].text) + self.assertEqual('failed to get lock: busting...', lines[1].text) + self.assertEqual(None, lines[1].colour) + self.assertEqual(False, lines[1].newline) + self.assertEqual(True, lines[1].bright) + self.assertEqual('timeout...', lines[2].text) + self.assertEqual('starting build', lines[3].text) + self.assertEqual([1, 2, 3], control.read_procs(tmpdir)) + self.assertEqual(control.RUN_WAIT_S, self.cur_time) + lock.release() + + # Check handling of dead processes. Here we have PID 2 as a running + # process, even though the PID file contains 1, 2 and 3. So we can + # add one more PID, to make 2 and 4 + self.cur_time = 0 + self.valid_pids = [2] + control.wait_for_process_limit(2, tmpdir=tmpdir, pid=4) + lines = terminal.get_print_test_lines() + self.assertEqual('Waiting for other buildman processes...', + lines[0].text) + self.assertEqual('done...', lines[1].text) + self.assertEqual('starting build', lines[2].text) + self.assertEqual([2, 4], control.read_procs(tmpdir)) + self.assertEqual(0, self.cur_time) + + # Try again, with PID 2 quitting at time 50. This allows the new + # build to start + self.cur_time = 0 + self.valid_pids = [2, 4] + self.finish_pid = 2 + self.finish_time = 50 + control.wait_for_process_limit(2, tmpdir=tmpdir, pid=5) + lines = terminal.get_print_test_lines() + self.assertEqual('Waiting for other buildman processes...', + lines[0].text) + self.assertEqual('done...', lines[1].text) + self.assertEqual('starting build', lines[2].text) + self.assertEqual([4, 5], control.read_procs(tmpdir)) + self.assertEqual(self.finish_time, self.cur_time) + if __name__ == "__main__": unittest.main() diff --git a/tools/u_boot_pylib/terminal.py b/tools/u_boot_pylib/terminal.py index 40d79f8ac07..2cd5a54ab52 100644 --- a/tools/u_boot_pylib/terminal.py +++ b/tools/u_boot_pylib/terminal.py @@ -164,8 +164,11 @@ def print_clear(): global last_print_len if last_print_len: - print('\r%s\r' % (' '* last_print_len), end='', flush=True) - last_print_len = None + if print_test_mode: + print_test_list.append(PrintLine(None, None, None, None)) + else: + print('\r%s\r' % (' '* last_print_len), end='', flush=True) + last_print_len = None def set_print_test_mode(enable=True): """Go into test mode, where all printing is recorded"""