From patchwork Fri Aug 19 14:25:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Herbrechtsmeier X-Patchwork-Id: 1668179 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-weidmueller-onmicrosoft-com header.b=4BR29gWQ; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4M8PKk225dz1yg7 for ; Sat, 20 Aug 2022 00:30:02 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 5861584BBD; Fri, 19 Aug 2022 16:27:09 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.b="4BR29gWQ"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id ADA1384BA3; Fri, 19 Aug 2022 16:26:25 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-eopbgr150088.outbound.protection.outlook.com [40.107.15.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 4120784B5D for ; Fri, 19 Aug 2022 16:26:04 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=stefan.herbrechtsmeier-oss@weidmueller.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MmXvbjpbfms2m50Vg0zswB+e5ldxle17R+vyM1bClx+ENN4sHgsUvAoGGoThg6he54FucMTZ9XzZ8s7DLs1Q4DFOG7AGPYlaD7mFO/S4UV+nn07Rn01Jeky+Qf4dsScQNY0lSPOsOuS/Jd2oXhrD7k9xhX+el+sFr6UlBL/yeDBJWo050Z1nwul7EE79yDDTQEsT/QGAMKWPyxBozZGVLhYcCGDXl7WsLvFUu8g9E7MbTMtOCo4J1fwB4YNLd9B+RPHYvhsenPP5+QCQT8zzraS6/9JX1hgJk6KuNAy8cQrnqf8i3CO/1rE239pQAVYFx+Lrop/nW6I2iQkm9f3JEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Hvw+ILN05X5gA8DW2Yk81hbIW7y+WtGxweOwtUdioVU=; b=brlnEWKq7UvbjXTIayeA8jrBoh6UsPvxeRrqzYqkASTHTwGoE3rsuINVT/THbRs6+sxXLDS4dIx1I3uYoJatn+KacZOJDP6uy3ZJZAIHHuw0eFNbWqGfDJq3qsqSUGdHOqNsteSjAynO8OuvqOz5/h2iQ60W3hHWbQwx2wZoysI14GzuCYHFWkDruB/eJx1l1b+qhiE4G7eK8LQiFffzDTpj8TPMHYrICZvJPWnZeHlnj8AeLfPK6b0xSw1if34QYD/OtxeoLRMhmSbpsM618XDlHfBaAGUp8FXrza4AyHTpgmOex6tDATKdjB0wJwLep4RFFLnHod9UERgdjyADvg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.onmicrosoft.com; s=selector1-weidmueller-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Hvw+ILN05X5gA8DW2Yk81hbIW7y+WtGxweOwtUdioVU=; b=4BR29gWQl7ZK8o3yYb0iENAAfzkKZzvJpzTc6AD1lsPOrURAwT18P/8xp2ZqWwO22yNaxeP/36kFubKrO0QYQssBfYscIt6BtNkGXKrFhI1CjA9wO2J00BXVst98mAl3L/uWdww652/joTue1uYdMDDtuoZag/urGZtQ8+NIheI= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) by PR3PR08MB5739.eurprd08.prod.outlook.com (2603:10a6:102:8e::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5546.18; Fri, 19 Aug 2022 14:26:03 +0000 Received: from PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44]) by PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44%3]) with mapi id 15.20.5546.018; Fri, 19 Aug 2022 14:26:03 +0000 From: Stefan Herbrechtsmeier To: u-boot@lists.denx.de Cc: Stefan Herbrechtsmeier , Alper Nebi Yasak , Simon Glass Subject: [PATCH v6 14/21] binman: Support missing compression tools Date: Fri, 19 Aug 2022 16:25:31 +0200 Message-Id: <20220819142538.24847-15-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220819142538.24847-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20220819142538.24847-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: AS4PR10CA0015.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:5dc::11) To PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 29678931-2a7a-47ac-60db-08da81eebefb X-MS-TrafficTypeDiagnostic: PR3PR08MB5739:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: d+Zfj1RK+Qf1bQ5iiDcf3U238jC+nUklNPisYCbEfJaJJxjmiKQ3XYaNGVU2A5Xz7cLJC6x5IDYqMCq5NXgHN/4GQhx8blTkozdklSJDR8v0qaDDtGVKGdD2q65/G4cTavODd14mnd6GagKukDDhHk2hDzliYaIksRjZByWuoUHWnXnN+G+rEKpcHBDabvYb5UTaAWH5sl5YQiW2f2gAhln2zrf+JsTGQvhEx5O4g9FTFwIARUa9SdQS/YXN3JXBU+Ktz2myOD7ipdlGydNBAqqHiyT+68DpXFmWzADOcgsAs79rWIe4uWN62gLGUmTnqhfGDv2jvl5wryeDeBEaxD+zdmNt099hozW0W/les098wCprZjDVmW8JqDk6eeq7OKRhBVBp4052p+WZ1vnGVQPyLaTyKNaEI8p7+5UlPP5bDnSAqXlnNfH2nOhv9RymOSoFhYUEFaXY/wJ6fy6Bui6sfqkL8g738w0TBGMFrfT8ISM7bRAcPfP6/TmM8HDVXGGGE6/6YKGlgHmOBAZ355asDpzAnypGk00v/tw9ysOf5WAe0rR0zqOs2HdxutgtQFo5qNZ7O5qIZIdHy6ZUCIuKSZTtOwDcRvboZLKPRv+olrPXOIjk4TsebL6Bqx3pAxxJo2ONSiOMCPPdD+uJMeDLSBBqM5CMePtxPzH8tLaiEhnzY6tuFOqSzlsX25wmxQoNB+W8PE12Jgz2qROqIM64foN5K13pTemcu33Xf29jGwScoKRZvGvPVAtlUhagwAMkO9IeY815NSmHxu7/0g== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAXPR08MB6969.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(396003)(366004)(376002)(136003)(346002)(39860400002)(4326008)(8676002)(66476007)(66946007)(66556008)(54906003)(316002)(6916009)(36756003)(5660300002)(8936002)(2906002)(478600001)(6486002)(41300700001)(6506007)(6666004)(6512007)(26005)(52116002)(2616005)(86362001)(1076003)(38350700002)(38100700002)(83380400001)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: mVBLAkqjb6a3CtTJPMOXtXjkHuGijNFXHtfGMn5wsMX1nc9dTwg5k2jpaISmkeF33JzJ5dnv8VPvKI9ZcHacIKioStGuUy0Z2NWXxCacaO2qGzyqMO2ZO9+LkdA9mT/f9pZa6yjHdd7SwteUhIE3glXVkTN/6qX4t4fEKWejGGik3Koip0N3+/4vqCFnmxTCG7jFreWZbFccBL9TmbZk+Wf2Flj+4hfN8lW86jAHXQCopgS7YRaMVeCO/Rzu4d4uj0fKo0yozK4MxTrMnaY08udDcUS/IAPmcW0MkmTFr/8LKmsD3xsRE20R18ZIHJCyFTK8L4l+FV3D3cAOj6N9Q1ERiJ7AAhREYBIa7juKjxgtPblOmnjLKme3SqBvMcUDZD56GfqiRT7/KuvvZj6Hta/jyhkzFMoGOaTyfZNxiLybF4yhpfi5Jyv8Vfdp7m8Q+TrurlGq4FuYWw8KlJ68pbyQGGLRFfLT9MKBFQQ3gJY9AZgwNyhSm7cxyC6ztajYmpwnBdXaxQXFXSEOuyfbJc4MAk3YGRCLPl4TXoo01gBGQCBAunyC8XXDPN5QkM7rVYOeSOI8kTZFu66tJeguR72aF2AyE2sx6H9tqytynhdvToSgByGdBYvkBG2QucrJs2F1g4yaoPHezXyj/P5nlff9m1NXZS7Oq10QQt6PM342zIVwRgwenhEt5M1ZyX9jztv0A+yHZpTPLZ6MDvunSE+l8PvlZVq54fKIxD2p1H+o5deM3UPRKzmSGkIBhpA5HeNCK4+xZ3dICPRA34dt8ySwXQHHt3bbakAy0Uc5F8E1wNUTOee1IqWUs89r/QT9mFu7kcnKZmogbJOt7f93ny3K/DfEglQxIVdVqnCcUECLekNSbH6TSFDlfRovZS+SSbi6F0sS2Fv0noPtHH6Ybuy0fnYfnaHjmGwAlQgPeBnL6hlmGh3Qoipjo00T34ztcDJeQDtNebe79/QiIW2TF5QeHS3sPbr5LEMXfINjWE65TZjihI4WPpA9jHM+rccYvBu+6ZPifGI1BGj+Tv4EA6X7GIDdStExEnmUGpTxpGQZEao7C4fKTyJzgcXeqHV584VGCvRb20iXMPaupiVfjaHpFxNNM2CChGyknUXDdyprT6RgE1RjaNb3bTdsqCAkaJruSSUDNz4wdceB2ePAi6zmwC4CwawO37gjwUdZGDUWIOW++Z8JhDeZXybMMcOdN+C+owKgCFGI6DxF2V2HcysiVS24+OMBbseEMyX4fVp/wioLm7tWYSZMoEFNHwwrhTpY4dm5rhyOMprB+W778gQQMIbsQmXailbGDObYWln5rUEQuFQoH54lSFtFQUv1xm+wJFTU+Y/9b5UB03p2rWaoubDf2vOlp5YsuLJ6bZCyKrWxr2jI7+dupZ6tzfzxjs4cBKPHsz94SVUdZnAzw80WOuHXlBT9BoqT+oJh/KvRbTvxEjrVf9bLRW02qb/K0573DxJNCpp9ipEWVnaZ4mnsbMRDzTBJG/1IIoWGc5oVlnA6JgU8pxIrkizxmM8VqhpgCC5CzdtkfQlupL2EED5OoytfhdX3nxpd/DfvE12fAgDrNjcy5mhtylvOzcTS6MxwWkGnyCQjwHjs+Ho8uw== X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: 29678931-2a7a-47ac-60db-08da81eebefb X-MS-Exchange-CrossTenant-AuthSource: PAXPR08MB6969.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Aug 2022 14:26:03.3628 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: oTBxESaeaXCzJkT6oBPFYo8PtLRBLt0QYZwBEuGgWPmZHmvQMH5bOL9gb3PRIHL/deBIK4RSbyrZu5vpIAhJqA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5739 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean From: Stefan Herbrechtsmeier Handle missing compression tools by returning empty data and record missing bintool. Signed-off-by: Stefan Herbrechtsmeier Reviewed-by: Simon Glass Reviewed-by: Simon Glass --- Changes in v6: - Set data with tools.get_bytes(0, 1024) instead returning Changes in v5: - Use record_missing_bintool function - Use tools.get_bytes function - Add support to DecompressData function - Reuse 85_compress_section.dts file - Remove 236_compress_dtb_missing_bintool.dts file Changes in v4: - Add missing 236_compress_dtb_missing_bintool.dts file Changes in v3: - Add commit to support missing compression tools tools/binman/entry.py | 14 +++++++++++--- tools/binman/entry_test.py | 9 +++++++++ tools/binman/ftest.py | 9 +++++++++ 3 files changed, 29 insertions(+), 3 deletions(-) diff --git a/tools/binman/entry.py b/tools/binman/entry.py index 5fa826aa71..48883bac3a 100644 --- a/tools/binman/entry.py +++ b/tools/binman/entry.py @@ -1084,7 +1084,11 @@ features to produce new behaviours. self.uncomp_data = indata if self.compress != 'none': self.uncomp_size = len(indata) - data = self.comp_bintool.compress(indata) + if self.comp_bintool.is_present(): + data = self.comp_bintool.compress(indata) + else: + self.record_missing_bintool(self.comp_bintool) + data = tools.get_bytes(0, 1024) else: data = indata return data @@ -1099,8 +1103,12 @@ features to produce new behaviours. Decompressed data """ if self.compress != 'none': - data = self.comp_bintool.decompress(indata) - self.uncomp_size = len(data) + if self.comp_bintool.is_present(): + data = self.comp_bintool.decompress(indata) + self.uncomp_size = len(data) + else: + self.record_missing_bintool(self.comp_bintool) + data = tools.get_bytes(0, 1024) else: data = indata self.uncomp_data = data diff --git a/tools/binman/entry_test.py b/tools/binman/entry_test.py index 1d60076be1..aa470c5816 100644 --- a/tools/binman/entry_test.py +++ b/tools/binman/entry_test.py @@ -105,6 +105,15 @@ class TestEntry(unittest.TestCase): self.assertTrue(isinstance(ent, Entry_blob)) self.assertEquals('missing', ent.etype) + def testDecompressData(self): + """Test the DecompressData() method of the base class""" + base = entry.Entry.Create(None, self.GetNode(), 'blob-dtb') + base.compress = 'lz4' + bintools = {} + base.comp_bintool = base.AddBintool(bintools, '_testing') + self.assertEquals(tools.get_bytes(0, 1024), base.CompressData(b'abc')) + self.assertEquals(tools.get_bytes(0, 1024), base.DecompressData(b'abc')) + if __name__ == "__main__": unittest.main() diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index 939b265d7c..ffcd7d2567 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -4422,6 +4422,15 @@ class TestFunctional(unittest.TestCase): } self.assertEqual(expected, props) + def testLz4Missing(self): + """Test that binman still produces an image if lz4 is missing""" + with test_util.capture_sys_output() as (_, stderr): + self._DoTestFile('185_compress_section.dts', + force_missing_bintools='lz4') + err = stderr.getvalue() + self.assertRegex(err, + "Image 'main-section'.*missing bintools.*: lz4") + def testCompressExtra(self): """Test compression of a section with no fixed size""" self._CheckLz4()