From patchwork Tue Aug 16 06:41:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stefan Herbrechtsmeier X-Patchwork-Id: 1666687 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-weidmueller-onmicrosoft-com header.b=Z/xQmiIB; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4M6M6q1tmtz1yft for ; Tue, 16 Aug 2022 16:43:31 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 5E0B8848F0; Tue, 16 Aug 2022 08:42:47 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=weidmueller.onmicrosoft.com header.i=@weidmueller.onmicrosoft.com header.b="Z/xQmiIB"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 6B752848EB; Tue, 16 Aug 2022 08:42:33 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50057.outbound.protection.outlook.com [40.107.5.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 4848D848A6 for ; Tue, 16 Aug 2022 08:42:27 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=weidmueller.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=stefan.herbrechtsmeier-oss@weidmueller.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lNlNcDcNO53tg7F5ZXo03c5M+glL/7UULwE5kNQWx9A6nXPyLTOj8R0PoFL2VgAzKtrNS6vNiMXrlNtwtXeonCy+otsoTd56JJg6ULLlFCyL6Ecis4F6EwX6kjUQ3ZY1y9Qh9GpoZFStung80TFmTtbRZPCsNDcLMoQKtT7kQDvKJ7gCdj0foXc0TqEhXoB19GMTfToLAcbAJFEB/j2E8Ag7PSOAdOOLUgSMWobcrmbsQBvoFvdETuMLAjeiV7AVEb4nzmMSt6HArEOxDnbgzvuWTRnovwyFqbBv81UMxqG9MIepeGmRbXvO1ZWgxSCluDHusUL2XEZGJSha1I1M8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/aDKAbshxZqz/YTDq6Q/OC+XD1ehoqXzGpt7T154zMk=; b=iP4MmB9uYwBgIPy6t6svTRfyXShIdIVXZLugZ9u9vDJPA9K68CYL2+0jq5vuA/sWHNa5fTiuVUHwKRSjJ5vhK3KntIj9jUb5Uqdtko+y7Om7JvGTyv3Ff8lSXSymC4cz/q5x/P0QQURN10pZzEeWjK1ofqQRpOc7rDKmgZlplVCuOE0ydYsYJ6Ho4ZWZVM9Wfc5FXCqMfcbL0tpymPNSocXkH1jZ1uJ132v18ZBCwApmqhLJRVVjsTos9CjCXrzjZkfmtxIQbMb0/5RK394XMbXCVlpjxwyGhFr19OFcuMMTKvWXiMBnwxlt6jKDxyctBsGGps4N7QmwHRfkJ/VvTw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=weidmueller.com; dmarc=pass action=none header.from=weidmueller.com; dkim=pass header.d=weidmueller.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weidmueller.onmicrosoft.com; s=selector1-weidmueller-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/aDKAbshxZqz/YTDq6Q/OC+XD1ehoqXzGpt7T154zMk=; b=Z/xQmiIBoHqec9qm+UpUkALvzjzWJiu+sTmrqCF8QlyL+MVl9Utd1vVaiCJLZMh1GXloNolcAKAcSqwR58xRjUJJbf4PD00DfdyNF3xEzZYZbKsWGGFdpwy7qoGmu7C/j63GLYQLrS8lEOwKLYmg50rgPWKZ6TEnDzmumL1r+0E= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=weidmueller.com; Received: from PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) by VE1PR08MB4685.eurprd08.prod.outlook.com (2603:10a6:802:ae::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.25; Tue, 16 Aug 2022 06:42:26 +0000 Received: from PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44]) by PAXPR08MB6969.eurprd08.prod.outlook.com ([fe80::f95b:2e64:9ee6:4d44%3]) with mapi id 15.20.5525.011; Tue, 16 Aug 2022 06:42:26 +0000 From: Stefan Herbrechtsmeier To: u-boot@lists.denx.de Cc: Stefan Herbrechtsmeier , Alper Nebi Yasak , Simon Glass Subject: [PATCH v3 05/13] binman: Simplify comp_util class Date: Tue, 16 Aug 2022 08:41:43 +0200 Message-Id: <20220816064151.9848-5-stefan.herbrechtsmeier-oss@weidmueller.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> References: <20220816064151.9848-1-stefan.herbrechtsmeier-oss@weidmueller.com> X-ClientProxiedBy: AS9PR06CA0744.eurprd06.prod.outlook.com (2603:10a6:20b:487::19) To PAXPR08MB6969.eurprd08.prod.outlook.com (2603:10a6:102:1d8::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ffe740bb-cfd3-46d1-0dfd-08da7f527b55 X-MS-TrafficTypeDiagnostic: VE1PR08MB4685:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lLspUvFpzQm64cZ6+dvJiPKn6h1MZW6co0McNghWXWMAxJ3i/VZfUnmjttXfwdOvcWL3wqLCFcsvMnKFIF/3sVL/V+wvyUUuKJz6nnEbaXVRNd9DAe87gXfihh6UcVJu82Y32ML5gAQbdT2gtbyNX6xRxYUBEYCJ0UvPGXAB7RXXxY+6Dk7897TuDaKXpimy1+GemWfUqDV8Ej6Kmsm3MzUeMGufs/MvxO6feROAyM/aezbL3DLG3aOQCzbYG/xENJuWcKBuaryM1EWo8Qn3tf4ytQGH0E089H8/Oafw/aUTQQaISCV0J/cLvi6K0GWSz711iHtKey2IGrhojTPiOXCf7Tp1nBWA8ZE1P6mMUrY0n+lRFv7HhyVLEi/i0H94HZWB/NnQQmCHuhXC+HgdichCAIW4FxzEjxA0DkBryzzJWKfxjBiOWes6gs2w4AxPRG7/W7JqXWBUi8kZhN8sP2AKoxRgUwQLjv9l10BO7sc6E2EIAz+tLjnyvYFDrGbCNscEq40VnhHDtKU/N1QwVUKj6a0KLsPv6gWXeMMm0uDSFaEbsfT3muSV0SS+gAVugyaB5JFhWAqjAgFzeSWoFGwOwSwuhwKY7670KKFhyuaK9hvqwO6Z4RAJBfjtCcb/uIh27mzXqzgd85r95OhiGDeMvB0c07kQJ3FTyi9hHtOikspBj8iZeC8CZcgKZ0CrqxfhzzJrL9yrDNWvLX9zsMTVk4d+I0kzZdZkK6B9NeBt0BFWinqjgIMQRzdJX5vU1AntQQprK7qGng6dxe7bwoFUWYqpHxsKoA4TVBOAtPU= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAXPR08MB6969.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(136003)(366004)(396003)(346002)(376002)(39860400002)(86362001)(38350700002)(38100700002)(36756003)(316002)(8936002)(41300700001)(4326008)(66946007)(5660300002)(1076003)(66476007)(478600001)(8676002)(66556008)(6486002)(6512007)(26005)(52116002)(83380400001)(6506007)(2906002)(6666004)(186003)(2616005)(6916009)(54906003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?hObMlWxhc5fFx0kiwd4RUuCHKFHv?= =?utf-8?q?PGx4sBMpK4LaF1Qt+JFmE6JbyOpZVup0FSXFVjcPznccoqzZBzZWGzTprqznOaRa0?= =?utf-8?q?GfHjyIC5z7qo/fkmh5RWvWzuTI23JIxi5umuX+Vv9oNtmIAvzbdwMQ7+BgxBv4QqR?= =?utf-8?q?UEUWxaRlbq4TqZglz/phmpljmj0y6AULUKP34sJNLnJLD8IHGK/Mwwa9+m/5xxfZY?= =?utf-8?q?9BCTz5ed3TG80hSQi+dJEAGrTSc2th8NLlLszL1aTTgEZwCPkiHk4reHWk/15hRxH?= =?utf-8?q?2uvuGsRDsjnYmyO76fg+rMD2w8AGGwD04n64Xt/1vdcVpey1J4//2/ILKFaAaaJ5C?= =?utf-8?q?1BsJseC5WRFaYORnbjdl6Ri1VPT/lzKuW0mjMx18sQeagrGQFjAIiO7tLH1+hQQaw?= =?utf-8?q?slWSTB9tdZyaFd33c7vmjVxJ7YvXESNPyv6SWD2Kk6TibDFnsaLcS3WrRZ+b/e+HU?= =?utf-8?q?T2apD0mwvGIkuxuzcn9LUeWVgvYKqiLXPGGA7a3WpyEup7XCka7N6V+axLRjuIXw+?= =?utf-8?q?jIT61UHYZfrYj++3IH3dmzha9dpm7gR2HAYpkkDGpc0v/7nNWchbZZbPlt48WsF48?= =?utf-8?q?boyJKEPxzjnYmfxDVfvE1DYSyDKCytuGiM/8h/gNTdVcPzodi+UBXwx0hanJqoxGK?= =?utf-8?q?CL7k4HRj9Ai4K5aC7e6jgBCN4Vtc2fmnOmqdTbDvMcrkNY2ESdBnyPUmM2OUDjmdx?= =?utf-8?q?vMNC/4zLwHjxH0dAePk2z6FJzn4iqU81JKce1gAmIGW3DqoQ5QOqp86+s0o4QzSmM?= =?utf-8?q?6TG1/90Ohc8HQX/jAhIktgjFjQoQn8Xih8ty0ktrEjAAuc7U6RTtcPw1UJUhgzKuR?= =?utf-8?q?phHRMN1lbb5Kk2bKqbAZHgLIEQE7+25T/OkOE/FC4dfhG8vrfE1Xs6NGBt5YfXxpb?= =?utf-8?q?qkLssUZ6iWQl+0LEzgE978cA7lt6zsK64zfI0ASCF4kBhGxsB9TN/3kvHLdMZWNJf?= =?utf-8?q?p4sipIPR/fwzvGpY6+IwCp98awIqg6DPL5ieXYvrQXwNXBtk0Gb7fyHrd1cG8P+cV?= =?utf-8?q?m1b5VID355PCw76xWjixWfZ7q6vmWv1bwg4HjRhFj99/qIGHnbkNNMi1EjFnOcKVN?= =?utf-8?q?0lag3k4OqziwnOJx144X9rAoOIippW9zUGzo0hzGlt5IwXphMtokW9xcBVLlX8WYU?= =?utf-8?q?ain6TSzQaTSyHmWkCxQEl+Vv7kntzeU/ZZDrcvtwymQX6Eyc7y2+XYDF0y2FYJU6/?= =?utf-8?q?5pExmdJ3YR4QIADZ8bVXO8pc6PcKLtR3lX8T/PucHNOfV5laNP0ky1W1k61N6g627?= =?utf-8?q?yVWvYm1/OBnAIWWvZKyee7AZSrg3lqHw6qoKc6UMT9l1bHL7NF+3cUWW8xs0oX0xr?= =?utf-8?q?Q9gHc54pRzukkc8lXJeYgjLN3+/WY8oZlVk/0c/Qu1XJUkU7k1k32pIJTGPQKWokN?= =?utf-8?q?1pCnteHyMbpU4lmYleMAE4OqOyMH/TyQdimEKh2Dc2csyfY9O5XLRgCeeUPuEuMA9?= =?utf-8?q?dg07p924uq9Bk+cdku7QqZ6nzGp/0HmWpaqzceehjtyCRW3TqW+bZbMgg2g8RbcYk?= =?utf-8?q?+OWKLSDkownf41oI6iE+YTZ2htdtBDynjA=3D=3D?= X-OriginatorOrg: weidmueller.com X-MS-Exchange-CrossTenant-Network-Message-Id: ffe740bb-cfd3-46d1-0dfd-08da7f527b55 X-MS-Exchange-CrossTenant-AuthSource: PAXPR08MB6969.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2022 06:42:26.0827 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e4289438-1c5f-4c95-a51a-ee553b8b18ec X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3aTrRGtZyvLWvbdibVpAbwn47Dwke46VbOE0tstIkwdg7eL9tLf42E8mTeqIL89p9hCjPYPLViu+Xexun6/MFg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4685 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean From: Stefan Herbrechtsmeier Simplify the comp_util class by remove duplicate code as preparation for additional compress algorithms. Signed-off-by: Stefan Herbrechtsmeier --- Changes in v3: - Rename COMPRESSIONS to ALGORITHMS - Rework existing comments - Add _get_tool_name function comment - Add _get_tool function comment tools/binman/cbfs_util_test.py | 2 +- tools/binman/comp_util.py | 101 +++++++++++++++++++++++---------- tools/binman/ftest.py | 2 +- 3 files changed, 72 insertions(+), 33 deletions(-) diff --git a/tools/binman/cbfs_util_test.py b/tools/binman/cbfs_util_test.py index f86b295149..44ebd04278 100755 --- a/tools/binman/cbfs_util_test.py +++ b/tools/binman/cbfs_util_test.py @@ -50,7 +50,7 @@ class TestCbfs(unittest.TestCase): cls.cbfstool = bintool.Bintool.create('cbfstool') cls.have_cbfstool = cls.cbfstool.is_present() - cls.have_lz4 = comp_util.HAVE_LZ4 + cls.have_lz4 = comp_util.is_present('lz4') @classmethod def tearDownClass(cls): diff --git a/tools/binman/comp_util.py b/tools/binman/comp_util.py index 269bbf7975..00761d44cc 100644 --- a/tools/binman/comp_util.py +++ b/tools/binman/comp_util.py @@ -1,69 +1,108 @@ # SPDX-License-Identifier: GPL-2.0+ # Copyright 2022 Google LLC +# Copyright (C) 2022 Weidmüller Interface GmbH & Co. KG +# Stefan Herbrechtsmeier # -"""Utilities to compress and decompress data""" +"""Utilities to compress and decompress data + +This supports the following compression algorithm: + none + lz4 + lzma + +Note that for lzma this uses an old version of the algorithm, not that +provided by xz. + +This requires the following tools: + lz4 + lzma_alone + +It also requires an output directory to be previously set up, by calling +PrepareOutputDir(). +""" import tempfile from binman import bintool from patman import tools -LZ4 = bintool.Bintool.create('lz4') -HAVE_LZ4 = LZ4.is_present() +# Supported compression algorithms +ALGORITHMS = ['lz4', 'lzma'] -LZMA_ALONE = bintool.Bintool.create('lzma_alone') -HAVE_LZMA_ALONE = LZMA_ALONE.is_present() +bintools = {} +def _get_tool_name(algo): + """Get the tool name of a compression algorithm -def compress(indata, algo): - """Compress some data using a given algorithm + Args: + algo (str): Algorithm to use - Note that for lzma this uses an old version of the algorithm, not that - provided by xz. + Returns: + str: Tool name + """ + names = {'lzma': 'lzma_alone'} + return names.get(algo, algo) + +def _get_tool(algo): + """Get the bintool object of a compression algorithm - This requires 'lz4' and 'lzma_alone' tools. It also requires an output - directory to be previously set up, by calling PrepareOutputDir(). + The function creates new bintool object on demand per compression algorithm + and save it in a global bintools dictionary. + + Args: + algo (str): Algorithm to use + + Returns: + A bintool object for the compression algorithm + """ + global bintools + name = _get_tool_name(algo) + tool = bintools.get(name) + if not tool: + tool = bintool.Bintool.create(name) + bintools[name] = tool + return tool + +def compress(indata, algo): + """Compress some data using a given algorithm Args: indata (bytes): Input data to compress - algo (str): Algorithm to use ('none', 'lz4' or 'lzma') + algo (str): Algorithm to use Returns: bytes: Compressed data """ if algo == 'none': return indata - if algo == 'lz4': - data = LZ4.compress(indata) - # cbfstool uses a very old version of lzma - elif algo == 'lzma': - data = LZMA_ALONE.compress(indata) - else: + if algo not in ALGORITHMS: raise ValueError("Unknown algorithm '%s'" % algo) + + tool = _get_tool(algo) + data = tool.compress(indata) + return data def decompress(indata, algo): """Decompress some data using a given algorithm - Note that for lzma this uses an old version of the algorithm, not that - provided by xz. - - This requires 'lz4' and 'lzma_alone' tools. It also requires an output - directory to be previously set up, by calling PrepareOutputDir(). - Args: indata (bytes): Input data to decompress - algo (str): Algorithm to use ('none', 'lz4' or 'lzma') + algo (str): Algorithm to use Returns: - (bytes) Compressed data + bytes: Decompressed data """ if algo == 'none': return indata - if algo == 'lz4': - data = LZ4.decompress(indata) - elif algo == 'lzma': - data = LZMA_ALONE.decompress(indata) - else: + if algo not in ALGORITHMS: raise ValueError("Unknown algorithm '%s'" % algo) + + tool = _get_tool(algo) + data = tool.decompress(indata) + return data + +def is_present(algo): + tool = _get_tool(algo) + return tool.is_present() diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py index 057b4e28b7..96c15cff77 100644 --- a/tools/binman/ftest.py +++ b/tools/binman/ftest.py @@ -212,7 +212,7 @@ class TestFunctional(unittest.TestCase): TestFunctional._MakeInputFile('tee.elf', tools.read_file(cls.ElfTestFile('elf_sections'))) - cls.have_lz4 = comp_util.HAVE_LZ4 + cls.have_lz4 = comp_util.is_present('lz4') @classmethod def tearDownClass(cls):