From patchwork Sun May 30 00:09:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Jo=C3=A3o_Marcos_Costa?= X-Patchwork-Id: 1485422 X-Patchwork-Delegate: trini@ti.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=Cabq85Lx; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FszMl38MWz9sWC for ; Sun, 30 May 2021 10:10:27 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id A717182B5B; Sun, 30 May 2021 02:10:10 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Cabq85Lx"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id B063082B1D; Sun, 30 May 2021 02:09:59 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,SPF_HELO_NONE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-qt1-x82d.google.com (mail-qt1-x82d.google.com [IPv6:2607:f8b0:4864:20::82d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id B9CC682B1D for ; Sun, 30 May 2021 02:09:54 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=jmcosta944@gmail.com Received: by mail-qt1-x82d.google.com with SMTP id h24so5485043qtm.12 for ; Sat, 29 May 2021 17:09:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pjbvxroSt3dRgJ2sfLyBKGmWoQtW6S/LZ3emNy2zaMM=; b=Cabq85LxS9ydr7L2JYqNxblvs1heNmtH/S9aBaJ6Ja8RsESZ6fcIQD6Qfy49S9e8WF Q8o6dSVi5ug3cLG5DITJ2hH0mv1ZaD5GMV/vmH+foNsBd0kkNyz+O4XHGwG1q24GJmh+ xoTENywiMzT0lW4ilSk80A2ZwayEIza6ZVghKNJbvPlpe7579UP5DJfoz2xzuCkE31yH CzeViXkcg2qXu01TU5CbC80w2QamYYRXjEB9n4EQ/WntoHa495PqQGyPy8vDeDU0EDlJ /QW2ydesFpPm110RibOD4O8LRIsldD29pM3gqzcdZbsbT1r98RE8aRUcaA3W1BxlvG9h 78TA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pjbvxroSt3dRgJ2sfLyBKGmWoQtW6S/LZ3emNy2zaMM=; b=WCwrnOgxEOkM36z0fsykUJ397Hd2rLeWYr2h4XWYSBtZhR5B26inUdCA6pZ0M8u5JG tPtfmU1unSnBKJr5sobaUfA1+SaxYlvsWYJOPcuao9SkvcSpNE804B7WxEzoOQsL/J74 KOrr7vziM1NB8k5xteNjApPWNUqaRj1hE8Lw3wraoa0SdB9WXkU5RYDMx7yWIumBsn8/ k9NMJLfGiZVMndtqHgn/YrzaDw9PF24ldTd2L/MLky/r04AwbPapXMMpRF4cFnuG84f8 Dxm/kMkfS4FO4v5hp40DL0S0Wef/fqOSMSnFORWQNGykPlPUMfCccEGv//oXJ5tSFmfT 9khA== X-Gm-Message-State: AOAM530x9Og9dzfVCESXwaKNOq0S8w6ke5lk1lIXDoEx7G2nvVh6iE6Y zRDVO2o8B01oTSTM4QyiwDeZg7INjzMpZP1y X-Google-Smtp-Source: ABdhPJzzNiDDEiT7IGVc9b4HZ0NzhujKTIrLnzjyALkn63dZo2olKkiT2PaGRWfroPeFn7+DKmsmDA== X-Received: by 2002:ac8:65d4:: with SMTP id t20mr9333513qto.15.1622333393228; Sat, 29 May 2021 17:09:53 -0700 (PDT) Received: from localhost.localdomain ([177.194.32.83]) by smtp.googlemail.com with ESMTPSA id u22sm1514300qtw.18.2021.05.29.17.09.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 29 May 2021 17:09:52 -0700 (PDT) From: Joao Marcos Costa To: u-boot@lists.denx.de Cc: thomas.petazzoni@bootlin.com, miquel.raynal@bootlin.com, richard.genoud@posteo.net, sjg@chromium.org, Joao Marcos Costa Subject: [PATCH v2 1/3] test/py: rewrite common tools for SquashFS tests Date: Sat, 29 May 2021 21:09:36 -0300 Message-Id: <20210530000938.48873-2-jmcosta944@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210530000938.48873-1-jmcosta944@gmail.com> References: <20210530000938.48873-1-jmcosta944@gmail.com> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.34 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.102.4 at phobos.denx.de X-Virus-Status: Clean Remove the previous OOP approach, which was confusing and incomplete. Add more test cases by making SquashFS images with various options, concerning file fragmentation and its compression. Add comments to properly document the code. Signed-off-by: Joao Marcos Costa --- .../test_fs/test_squashfs/sqfs_common.py | 237 +++++++++++++----- 1 file changed, 173 insertions(+), 64 deletions(-) diff --git a/test/py/tests/test_fs/test_squashfs/sqfs_common.py b/test/py/tests/test_fs/test_squashfs/sqfs_common.py index c96f92c1d8..5469b1c017 100644 --- a/test/py/tests/test_fs/test_squashfs/sqfs_common.py +++ b/test/py/tests/test_fs/test_squashfs/sqfs_common.py @@ -3,74 +3,183 @@ # Author: Joao Marcos Costa import os -import random -import string +import shutil import subprocess -def sqfs_get_random_letters(size): - letters = [] - for i in range(0, size): - letters.append(random.choice(string.ascii_letters)) +""" standard test images table: Each table item is a key:value pair +representing the output image name and its respective mksquashfs options. +This table should be modified only when adding support for new compression +algorithms. The 'default' case takes no options but the input and output +names, so it must be assigned with an empty string. +""" +STANDARD_TABLE = { + 'default' : '', + 'lzo_comp_frag' : '', + 'lzo_frag' : '', + 'lzo_no_frag' : '', + 'zstd_comp_frag' : '', + 'zstd_frag' : '', + 'zstd_no_frag' : '', + 'gzip_comp_frag' : '', + 'gzip_frag' : '', + 'gzip_no_frag' : '' +} - return ''.join(letters) +""" EXTRA_TABLE: Set this table's keys and values if you want to make squashfs +images with your own customized options. +""" +EXTRA_TABLE = {} -def sqfs_generate_file(path, size): - content = sqfs_get_random_letters(size) - file = open(path, "w") +# path to source directory used to make squashfs test images +SQFS_SRC_DIR = 'sqfs_src_dir' + +def get_opts_list(): + """ Combines fragmentation and compression options into a list of strings. + + opts_list's firts item is an empty string as STANDARD_TABLE's first item is + the 'default' case. + + Returns: + A list of strings whose items are formed by a compression and a + fragmentation option joined by a whitespace. + """ + # supported compression options only + comp_opts = ['-comp lzo', '-comp zstd', '-comp gzip'] + # file fragmentation options + frag_opts = ['-always-use-fragments', '-always-use-fragments -noF', '-no-fragments'] + + opts_list = [' '] + for comp_opt in comp_opts: + for frag_opt in frag_opts: + opts_list.append(' '.join([comp_opt, frag_opt])) + + return opts_list + +def init_standard_table(): + """ Initializes STANDARD_TABLE values. + + STANDARD_TABLE's keys are pre-defined, and init_standard_table() assigns + the right value for each one of them. + """ + opts_list = get_opts_list() + + for key, value in zip(STANDARD_TABLE.keys(), opts_list): + STANDARD_TABLE[key] = value + +def generate_file(file_name, file_size): + """ Generates a file filled with 'x'. + + Args: + file_name: the file's name. + file_size: the content's length and therefore the file size. + """ + content = 'x' * file_size + + file = open(file_name, 'w') file.write(content) file.close() -class Compression: - def __init__(self, name, files, sizes, block_size = 4096): - self.name = name - self.files = files - self.sizes = sizes - self.mksquashfs_opts = " -b " + str(block_size) + " -comp " + self.name - - def add_opt(self, opt): - self.mksquashfs_opts += " " + opt - - def gen_image(self, build_dir): - src = os.path.join(build_dir, "sqfs_src/") - os.mkdir(src) - for (f, s) in zip(self.files, self.sizes): - sqfs_generate_file(src + f, s) - - # the symbolic link always targets the first file - os.symlink(self.files[0], src + "sym") - - sqfs_img = os.path.join(build_dir, "sqfs-" + self.name) - i_o = src + " " + sqfs_img - opts = self.mksquashfs_opts - try: - subprocess.run(["mksquashfs " + i_o + opts], shell = True, check = True) - except: - print("mksquashfs error. Compression type: " + self.name) - raise RuntimeError - - def clean_source(self, build_dir): - src = os.path.join(build_dir, "sqfs_src/") - for f in self.files: - os.remove(src + f) - os.remove(src + "sym") - os.rmdir(src) - - def cleanup(self, build_dir): - self.clean_source(build_dir) - sqfs_img = os.path.join(build_dir, "sqfs-" + self.name) - os.remove(sqfs_img) - -files = ["blks_only", "blks_frag", "frag_only"] -sizes = [4096, 5100, 100] -gzip = Compression("gzip", files, sizes) -zstd = Compression("zstd", files, sizes) -lzo = Compression("lzo", files, sizes) - -# use fragment blocks for files larger than block_size -gzip.add_opt("-always-use-fragments") -zstd.add_opt("-always-use-fragments") - -# avoid fragments if lzo is used -lzo.add_opt("-no-fragments") - -comp_opts = [gzip, zstd, lzo] +def generate_sqfs_src_dir(build_dir): + """ Generates the source directory used to make the SquashFS images. + + The source directory is generated at build_dir, and it has the following + structure: + sqfs_src_dir/ + ├── empty-dir/ + ├── f1000 + ├── f4096 + ├── f5096 + ├── subdir/ + │   └── subdir-file + └── sym -> subdir + + 3 directories, 4 files + + The files in the root dir. are prefixed with an 'f' followed by its size. + + Args: + build_dir: u-boot's build-sandbox directory. + """ + + root = os.path.join(build_dir, SQFS_SRC_DIR) + # make root directory + os.makedirs(root) + + # 4096: minimum block size + file_name = 'f4096' + generate_file(os.path.join(root, file_name), 4096) + + # 5096: minimum block size + 1000 chars (fragment) + file_name = 'f5096' + generate_file(os.path.join(root, file_name), 5096) + + # 1000: less than minimum block size (fragment only) + file_name = 'f1000' + generate_file(os.path.join(root, file_name), 1000) + + # sub-directory with a single file inside + subdir_path = os.path.join(root, 'subdir') + os.makedirs(subdir_path) + generate_file(os.path.join(subdir_path, 'subdir-file'), 100) + + # symlink (target: sub-directory) + os.symlink('subdir', os.path.join(root, 'sym')) + + # empty directory + os.makedirs(os.path.join(root, 'empty-dir')) + +def mksquashfs(args): + """ Runs mksquashfs command. + + Args: + args: mksquashfs options (e.g.: compression and fragmentation). + """ + subprocess.run(['mksquashfs ' + args], shell=True, check=True, + stdout=subprocess.DEVNULL) + +def make_all_images(build_dir): + """ Makes the SquashFS images used in the test suite. + + The image names and respective mksquashfs options are defined in STANDARD_TABLE + and EXTRA_TABLE. The destination is defined by 'build_dir'. + + Args: + build_dir: u-boot's build-sandbox directory. + """ + + init_standard_table() + input_path = os.path.join(build_dir, SQFS_SRC_DIR) + + # make squashfs images according to STANDARD_TABLE + for out, opts in zip(STANDARD_TABLE.keys(), STANDARD_TABLE.values()): + output_path = os.path.join(build_dir, out) + mksquashfs(' '.join([input_path, output_path, opts])) + + # make squashfs images according to EXTRA_TABLE + for out, opts in zip(EXTRA_TABLE.keys(), EXTRA_TABLE.values()): + output_path = os.path.join(build_dir, out) + mksquashfs(' '.join([input_path, output_path, opts])) + +def clean_all_images(build_dir): + """ Deletes the SquashFS images at build_dir. + + Args: + build_dir: u-boot's build-sandbox directory. + """ + + for image_name in STANDARD_TABLE: + image_path = os.path.join(build_dir, image_name) + os.remove(image_path) + + for image_name in EXTRA_TABLE: + image_path = os.path.join(build_dir, image_name) + os.remove(image_path) + +def clean_sqfs_src_dir(build_dir): + """ Deletes the source directory at build_dir. + + Args: + build_dir: u-boot's build-sandbox directory. + """ + path = os.path.join(build_dir, SQFS_SRC_DIR) + shutil.rmtree(path) From patchwork Sun May 30 00:09:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Jo=C3=A3o_Marcos_Costa?= X-Patchwork-Id: 1485423 X-Patchwork-Delegate: trini@ti.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=sR/Q39J6; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FszMr6tnCz9sWC for ; Sun, 30 May 2021 10:10:32 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 4D22682D57; Sun, 30 May 2021 02:10:14 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="sR/Q39J6"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 06E3882D1C; Sun, 30 May 2021 02:10:05 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,SPF_HELO_NONE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-qv1-xf31.google.com (mail-qv1-xf31.google.com [IPv6:2607:f8b0:4864:20::f31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 3626D81C84 for ; Sun, 30 May 2021 02:09:57 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=jmcosta944@gmail.com Received: by mail-qv1-xf31.google.com with SMTP id u33so3823452qvf.9 for ; Sat, 29 May 2021 17:09:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1eZgcrTRmuljijRqSyIH1SbAmWfUwKu+j0j3uSeFE1A=; b=sR/Q39J6s9WIzCBAyangB047RRA/3HCcXo9pHLGBkw6OzKmL0zLzmdDVgJIC2++T9G rsMg8QuYGkZE16WHbNtJHUBYuuN8+veqHdr4dfG7XPTUQILsEpQVBKD97jqnX+qVUhyg CBB1jPrXXl64Yg+YL6FasITEJoAdmCz7F2NPl7tMfoQ5x9L28zb8lnUkYvMzDVVECQc7 hKAq2iXH4YZ/TG/JpyGq8saVuHUE7S/Tm3ZUCA1xM95zQXQVpikRVmSkcLZubqoMOODr AElLrctH5B2iwxShlalx4OQusi0pGAu7UYu7nBbsF0D3Ila4fkuKmQQK9qy6q0ndfmZE AdiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1eZgcrTRmuljijRqSyIH1SbAmWfUwKu+j0j3uSeFE1A=; b=fWANI7jbm5p6FLBpZwLVKM8AtDQsq4QITMPjiqHManQiFpz86Z0HgA3MqfuM4L8V1k SaNK/nu3YeXnAL/wSjwkJTQoDUb6E4cC3NmnyhCfJk4JBmFjhnmwlVQ6XWhspQUlsgh+ sVPbulPVIohyIuMzMjJN7EiaK5Y3nMkgmnolWlXWk8LzHIIV22RVIhnhBNy31mOabSkW aFsoZWJkZ+dVzD4uzmuIUU1hf7VOnuvfaJHWk6QvmrXyTr28DTpM0/Wp3hw7EH6SH4Jg kUKQjcyK6wwLc4FPLPfwOYED+KqVFM9hbg3Wszos2srEADm2lCsqHnkfpbwFJNDSUaeP CKlg== X-Gm-Message-State: AOAM531CVfAm8Kdsr1ytSz5kuKgiPYSFk4MqKDuEzpzoPke8GWhWOR0o ++otDm63BSj8ewJdaXh3taIrTDFwsPKN4isV X-Google-Smtp-Source: ABdhPJwDyrBnSy1d1EZLfsSOE+eVyJTWwfuN6VKplQdhwApeCyGUfDMR0V2paPDGqMX6BW1KIUxoLA== X-Received: by 2002:a0c:ee23:: with SMTP id l3mr10580208qvs.55.1622333395746; Sat, 29 May 2021 17:09:55 -0700 (PDT) Received: from localhost.localdomain ([177.194.32.83]) by smtp.googlemail.com with ESMTPSA id u22sm1514300qtw.18.2021.05.29.17.09.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 29 May 2021 17:09:55 -0700 (PDT) From: Joao Marcos Costa To: u-boot@lists.denx.de Cc: thomas.petazzoni@bootlin.com, miquel.raynal@bootlin.com, richard.genoud@posteo.net, sjg@chromium.org, Joao Marcos Costa Subject: [PATCH v2 2/3] test/py: rewrite sqfsload command test suite Date: Sat, 29 May 2021 21:09:37 -0300 Message-Id: <20210530000938.48873-3-jmcosta944@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210530000938.48873-1-jmcosta944@gmail.com> References: <20210530000938.48873-1-jmcosta944@gmail.com> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.34 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.102.4 at phobos.denx.de X-Virus-Status: Clean The previous strategy to know if a file was correctly loaded was to check for how many bytes were read and compare it against the file's original size. Since this is not a good solution, replace it by comparing the checksum of the loaded bytes against the original file's checksum. Add more test cases: files at a sub-directory and non-existent file. Signed-off-by: Joao Marcos Costa --- .../test_fs/test_squashfs/test_sqfs_load.py | 166 ++++++++++++++---- 1 file changed, 136 insertions(+), 30 deletions(-) diff --git a/test/py/tests/test_fs/test_squashfs/test_sqfs_load.py b/test/py/tests/test_fs/test_squashfs/test_sqfs_load.py index 9e90062384..ef4be1339c 100644 --- a/test/py/tests/test_fs/test_squashfs/test_sqfs_load.py +++ b/test/py/tests/test_fs/test_squashfs/test_sqfs_load.py @@ -3,8 +3,117 @@ # Author: Joao Marcos Costa import os +import subprocess import pytest -from sqfs_common import * + +from sqfs_common import SQFS_SRC_DIR, STANDARD_TABLE +from sqfs_common import generate_sqfs_src_dir, make_all_images +from sqfs_common import clean_sqfs_src_dir, clean_all_images + +@pytest.mark.requiredtool('md5sum') +def original_md5sum(path): + """ Runs md5sum command. + + Args: + path: path to original file. + Returns: + The original file's checksum as a string. + """ + + out = subprocess.run(['md5sum ' + path], shell=True, check=True, + capture_output=True, text=True) + checksum = out.stdout.split()[0] + + return checksum + +def uboot_md5sum(u_boot_console, address, count): + """ Runs U-Boot's md5sum command. + + Args: + u_boot_console: provides the means to interact with U-Boot's console. + address: address where the file was loaded (e.g.: $kernel_addr_r). + count: file's size. It was named 'count' to match md5sum's respective + argument name. + Returns: + The checksum of the file loaded with sqfsload as a string. + """ + + out = u_boot_console.run_command('md5sum {} {}'.format(address, count)) + checksum = out.split()[-1] + + return checksum + +def sqfs_load_files(u_boot_console, files, sizes, address): + """ Loads files and asserts their checksums. + + Args: + u_boot_console: provides the means to interact with U-Boot's console. + files: list of files to be loaded. + sizes: the sizes of each file. + address: the address where the files should be loaded. + """ + build_dir = u_boot_console.config.build_dir + for (file, size) in zip(files, sizes): + out = u_boot_console.run_command('sqfsload host 0 {} {}'.format(address, file)) + + # check if the right amount of bytes was read + assert size in out + + # compare original file's checksum against u-boot's + u_boot_checksum = uboot_md5sum(u_boot_console, address, hex(int(size))) + original_file_path = os.path.join(build_dir, SQFS_SRC_DIR + '/' + file) + original_checksum = original_md5sum(original_file_path) + assert u_boot_checksum == original_checksum + +def sqfs_load_files_at_root(u_boot_console): + """ Calls sqfs_load_files passing the files at the SquashFS image's root. + + Args: + u_boot_console: provides the means to interact with U-Boot's console. + """ + + files = ['f4096', 'f5096', 'f1000'] + sizes = ['4096', '5096', '1000'] + address = '$kernel_addr_r' + sqfs_load_files(u_boot_console, files, sizes, address) + +def sqfs_load_files_at_subdir(u_boot_console): + """ Calls sqfs_load_files passing the files at the SquashFS image's subdir. + + This test checks if the path resolution works, since the file is not at the + root directory. + + Args: + u_boot_console: provides the means to interact with U-Boot's console. + """ + files = ['subdir/subdir-file'] + sizes = ['100'] + address = '$kernel_addr_r' + sqfs_load_files(u_boot_console, files, sizes, address) + +def sqfs_load_non_existent_file(u_boot_console): + """ Calls sqfs_load_files passing an non-existent file to raise an error. + + This test checks if the SquashFS support won't crash if it doesn't find the + specified file. + + Args: + u_boot_console: provides the means to interact with U-Boot's console. + """ + address = '$kernel_addr_r' + file = 'non-existent' + out = u_boot_console.run_command('sqfsload host 0 {} {}'.format(address, file)) + assert 'Failed to load' in out + +def sqfs_run_all_load_tests(u_boot_console): + """ Runs all the previously defined test cases. + + Args: + u_boot_console: provides the means to interact with U-Boot's console. + """ + sqfs_load_files_at_root(u_boot_console) + sqfs_load_files_at_subdir(u_boot_console) + sqfs_load_non_existent_file(u_boot_console) @pytest.mark.boardspec('sandbox') @pytest.mark.buildconfigspec('cmd_fs_generic') @@ -12,35 +121,32 @@ from sqfs_common import * @pytest.mark.buildconfigspec('fs_squashfs') @pytest.mark.requiredtool('mksquashfs') def test_sqfs_load(u_boot_console): + """ Executes the sqfsload test suite. + + First, it generates the SquashFS images, then it runs the test cases and + finally cleans the workspace. If an exception is raised, the workspace is + cleaned before exiting. + + Args: + u_boot_console: provides the means to interact with U-Boot's console. + """ build_dir = u_boot_console.config.build_dir - command = "sqfsload host 0 $kernel_addr_r " - for opt in comp_opts: - # generate and load the squashfs image + # setup test environment + generate_sqfs_src_dir(build_dir) + make_all_images(build_dir) + + # run all tests for each image + for image in STANDARD_TABLE: try: - opt.gen_image(build_dir) - except RuntimeError: - opt.clean_source(build_dir) - # skip unsupported compression types - continue - - path = os.path.join(build_dir, "sqfs-" + opt.name) - output = u_boot_console.run_command("host bind 0 " + path) - - output = u_boot_console.run_command(command + "xxx") - assert "File not found." in output - - for (f, s) in zip(opt.files, opt.sizes): - try: - output = u_boot_console.run_command(command + f) - assert str(s) in output - except: - assert False - opt.cleanup(build_dir) - - # test symbolic link - output = u_boot_console.run_command(command + "sym") - assert str(opt.sizes[0]) in output - - # remove generated files - opt.cleanup(build_dir) + image_path = os.path.join(build_dir, image) + u_boot_console.run_command('host bind 0 {}'.format(image_path)) + sqfs_run_all_load_tests(u_boot_console) + except: + clean_all_images(build_dir) + clean_sqfs_src_dir(build_dir) + raise AssertionError + + # clean test environment + clean_all_images(build_dir) + clean_sqfs_src_dir(build_dir) From patchwork Sun May 30 00:09:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Jo=C3=A3o_Marcos_Costa?= X-Patchwork-Id: 1485424 X-Patchwork-Delegate: trini@ti.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=skx4W749; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FszN274mjz9sVv for ; Sun, 30 May 2021 10:10:42 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id E63AF82D6E; Sun, 30 May 2021 02:10:25 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="skx4W749"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id A2DE782B1D; Sun, 30 May 2021 02:10:09 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,SPF_HELO_NONE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-qk1-x733.google.com (mail-qk1-x733.google.com [IPv6:2607:f8b0:4864:20::733]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id F2A3C82B5B for ; Sun, 30 May 2021 02:09:59 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=jmcosta944@gmail.com Received: by mail-qk1-x733.google.com with SMTP id i67so7937572qkc.4 for ; Sat, 29 May 2021 17:09:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dSpt/Zr79RHcCqYdKD+Lr3mjiKlLIe3Ajafmdkb0PNQ=; b=skx4W749BeDCX4P2/TiZml/fZ6Q7Mx0Rfvm0daGc5DkliLLxlsjJN4ABP5dXkgqEXg ZGm0NX89KJZBPXF6aylEJQTAWHNkqEhKHXWb2XLr36i783DKa7P1hgri1XOGSYd22Abj iaq5L/KZezXZfsGOWJsBBBOC+/j7+rtuKLN9TWJa7ZFWoIv6ZbZOT8/QxHWpkP1ONwNS y4lDRrJtxllx+8ITSnDVk2/6stkSsQ0gVtRMW0qN6/1Lrnt8Cx3bU8EO9N2Mx301/6n0 c2O+ytuGGv7BElhtHhOyX8WgdfjBcky9uJZqloDVlpWS12p1GLrVrKAkw55+NDPrEioa pfMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dSpt/Zr79RHcCqYdKD+Lr3mjiKlLIe3Ajafmdkb0PNQ=; b=HVNdckmy0sXeTYUcDUtgnAYVNaLdM/z8PYH7d3RF9CkynqZvdEsVZk2p3CiN6A7Op1 u49RY4hkopOthaCl70LCJYCh57ahI9uJn3Oju0N3pPQukwC1nDk894xBbSMWCsquwufc 5xlIEZCYyS9GeO4COt1egoC8ycJ8yn3zpxbK0TjmTzw3eEmsF9YiD1V7bF1lmuv/53O4 YljAW/zEsyw/49ve5RzglvbI4b+moip+QnA92IiE7MsJ3ozUvXwhrhFmvszmRdLOc4Kl 2wBLwkp5VoFqPzXnSgaZOidakYiClETq4UT9e1l8AzZfiyHU+a11pxt1h3+TxnQfv1Vf 7YfA== X-Gm-Message-State: AOAM533z2KNox2QQD8OUU2Q4LSoODDq6tW4HIaGXeNCbYpHGS4ToXRM0 ZALgBcmlqQCofXqbgSwS/TnMKlzPvigwMK2V X-Google-Smtp-Source: ABdhPJw8VfrWqoVcUJz8YOhe7HkIzQ/LU9F1DMQAZuXK4IBWyhpaTux32BMHZiZLj/3xcSLpJ3Wf0g== X-Received: by 2002:a37:7844:: with SMTP id t65mr10433256qkc.429.1622333398402; Sat, 29 May 2021 17:09:58 -0700 (PDT) Received: from localhost.localdomain ([177.194.32.83]) by smtp.googlemail.com with ESMTPSA id u22sm1514300qtw.18.2021.05.29.17.09.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 29 May 2021 17:09:58 -0700 (PDT) From: Joao Marcos Costa To: u-boot@lists.denx.de Cc: thomas.petazzoni@bootlin.com, miquel.raynal@bootlin.com, richard.genoud@posteo.net, sjg@chromium.org, Joao Marcos Costa Subject: [PATCH v2 3/3] test/py: rewrite sqfsls command test suite Date: Sat, 29 May 2021 21:09:38 -0300 Message-Id: <20210530000938.48873-4-jmcosta944@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210530000938.48873-1-jmcosta944@gmail.com> References: <20210530000938.48873-1-jmcosta944@gmail.com> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.34 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.102.4 at phobos.denx.de X-Virus-Status: Clean Add more details to test cases by comparing each expected line with the command's output. Add new test cases: - sqfsls at an empty directory - sqfsls at a sub-directory Signed-off-by: Joao Marcos Costa --- .../test_fs/test_squashfs/test_sqfs_ls.py | 138 +++++++++++++++--- 1 file changed, 119 insertions(+), 19 deletions(-) diff --git a/test/py/tests/test_fs/test_squashfs/test_sqfs_ls.py b/test/py/tests/test_fs/test_squashfs/test_sqfs_ls.py index a0dca2e2fc..02d353f8ac 100644 --- a/test/py/tests/test_fs/test_squashfs/test_sqfs_ls.py +++ b/test/py/tests/test_fs/test_squashfs/test_sqfs_ls.py @@ -4,7 +4,100 @@ import os import pytest -from sqfs_common import * + +from sqfs_common import STANDARD_TABLE +from sqfs_common import generate_sqfs_src_dir, make_all_images +from sqfs_common import clean_sqfs_src_dir, clean_all_images + +def sqfs_ls_at_root(u_boot_console): + """ Runs sqfsls at image's root. + + This test checks if all the present files and directories were listed. Also, + it checks if passing the slash or not changes the output, which it shouldn't. + + Args: + u_boot_console: provides the means to interact with U-Boot's console. + """ + + no_slash = u_boot_console.run_command('sqfsls host 0') + slash = u_boot_console.run_command('sqfsls host 0 /') + assert no_slash == slash + + expected_lines = ['empty-dir/', '1000 f1000', '4096 f4096', '5096 f5096', + 'subdir/', ' sym', '4 file(s), 2 dir(s)'] + + output = u_boot_console.run_command('sqfsls host 0') + for line in expected_lines: + assert line in output + +def sqfs_ls_at_empty_dir(u_boot_console): + """ Runs sqfsls at an empty directory. + + This tests checks if sqfsls will print anything other than the 'Empty directory' + message. + + Args: + u_boot_console: provides the means to interact with U-Boot's console. + """ + assert u_boot_console.run_command('sqfsls host 0 empty-dir') == 'Empty directory.' + +def sqfs_ls_at_subdir(u_boot_console): + """ Runs sqfsls at the SquashFS image's subdir. + + This test checks if the path resolution works, since the directory is not the + root. + + Args: + u_boot_console: provides the means to interact with U-Boot's console. + """ + expected_lines = ['100 subdir-file', '1 file(s), 0 dir(s)'] + output = u_boot_console.run_command('sqfsls host 0 subdir') + for line in expected_lines: + assert line in output + +def sqfs_ls_at_symlink(u_boot_console): + """ Runs sqfsls at a SquashFS image's symbolic link. + + This test checks if the symbolic link's target resolution works. + + Args: + u_boot_console: provides the means to interact with U-Boot's console. + """ + # since sym -> subdir, the following outputs must be equal + output = u_boot_console.run_command('sqfsls host 0 sym') + output_subdir = u_boot_console.run_command('sqfsls host 0 subdir') + assert output == output_subdir + + expected_lines = ['100 subdir-file', '1 file(s), 0 dir(s)'] + for line in expected_lines: + assert line in output + +def sqfs_ls_at_non_existent_dir(u_boot_console): + """ Runs sqfsls at a file and at a non-existent directory. + + This test checks if the SquashFS support won't crash if it doesn't find the + specified directory or if it takes a file as an input instead of an actual + directory. In both cases, the output should be the same. + + Args: + u_boot_console: provides the means to interact with U-Boot's console. + """ + out_non_existent = u_boot_console.run_command('sqfsls host 0 fff') + out_not_dir = u_boot_console.run_command('sqfsls host 0 f1000') + assert out_non_existent == out_not_dir + assert '** Cannot find directory. **' in out_non_existent + +def sqfs_run_all_ls_tests(u_boot_console): + """ Runs all the previously defined test cases. + + Args: + u_boot_console: provides the means to interact with U-Boot's console. + """ + sqfs_ls_at_root(u_boot_console) + sqfs_ls_at_empty_dir(u_boot_console) + sqfs_ls_at_subdir(u_boot_console) + sqfs_ls_at_symlink(u_boot_console) + sqfs_ls_at_non_existent_dir(u_boot_console) @pytest.mark.boardspec('sandbox') @pytest.mark.buildconfigspec('cmd_fs_generic') @@ -12,25 +105,32 @@ from sqfs_common import * @pytest.mark.buildconfigspec('fs_squashfs') @pytest.mark.requiredtool('mksquashfs') def test_sqfs_ls(u_boot_console): + """ Executes the sqfsls test suite. + + First, it generates the SquashFS images, then it runs the test cases and + finally cleans the workspace. If an exception is raised, the workspace is + cleaned before exiting. + + Args: + u_boot_console: provides the means to interact with U-Boot's console. + """ build_dir = u_boot_console.config.build_dir - for opt in comp_opts: - try: - opt.gen_image(build_dir) - except RuntimeError: - opt.clean_source(build_dir) - # skip unsupported compression types - continue - path = os.path.join(build_dir, "sqfs-" + opt.name) - output = u_boot_console.run_command("host bind 0 " + path) + # setup test environment + generate_sqfs_src_dir(build_dir) + make_all_images(build_dir) + + # run all tests for each image + for image in STANDARD_TABLE: try: - # list files in root directory - output = u_boot_console.run_command("sqfsls host 0") - assert str(len(opt.files) + 1) + " file(s), 0 dir(s)" in output - assert " sym" in output - output = u_boot_console.run_command("sqfsls host 0 xxx") - assert "** Cannot find directory. **" in output + image_path = os.path.join(build_dir, image) + u_boot_console.run_command('host bind 0 {}'.format(image_path)) + sqfs_run_all_ls_tests(u_boot_console) except: - opt.cleanup(build_dir) - assert False - opt.cleanup(build_dir) + clean_all_images(build_dir) + clean_sqfs_src_dir(build_dir) + raise AssertionError + + # clean test environment + clean_all_images(build_dir) + clean_sqfs_src_dir(build_dir)