From patchwork Wed Feb 28 10:51:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?UmFmYcWCIE1pxYJlY2tp?= X-Patchwork-Id: 1905655 X-Patchwork-Delegate: zajec5@gmail.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=bombadil.20210309 header.b=ALlTN8ZF; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20230601 header.b=aypYhCLn; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.openwrt.org (client-ip=2607:7c80:54:3::133; helo=bombadil.infradead.org; envelope-from=openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org; receiver=patchwork.ozlabs.org) Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TlB5f5Mkqz23qX for ; Wed, 28 Feb 2024 21:52:52 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=Bs7pUzHqEXvs66eLCbiBgYgmWCTI4wqRAf/CwUBhg8U=; b=ALlTN8ZFShlA8d ccHQ4Go+vHEJP2ODXZVJ1Zi3LMqoLhvnOAYCN8BIgcCls3KA2PrUuHMu3oh1n4/jDJXmJxRDZSAeU eyy3xUPmu9cyfHFg0ktQ/5MXiziJEmzXhzeaO1iQYwC+NlxpKC1sBh4JiCgdCuDMmUmXo/dPsipd0 i0vpZHPBAaMYeq9JDx46ZO+YbQpUEUC4tCc0ElsVuxpwf1AvegsTnrqy16kcWlgbVXN+7rKujls8t RrNjW8JDFbqjWUGTJwqj8UaKq5S+UyC5WCi/xmkCJOG8r7oAEcGxIPf8ijCNNFSjYbOoQlA+37p7Z evGExWTiFvLJtwciGFVQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfHWp-00000008sk0-41LG; Wed, 28 Feb 2024 10:51:11 +0000 Received: from mail-ej1-x633.google.com ([2a00:1450:4864:20::633]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfHWm-00000008sj2-2w1Y for openwrt-devel@lists.openwrt.org; Wed, 28 Feb 2024 10:51:10 +0000 Received: by mail-ej1-x633.google.com with SMTP id a640c23a62f3a-a3d484a58f6so749237566b.3 for ; Wed, 28 Feb 2024 02:51:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709117467; x=1709722267; darn=lists.openwrt.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=p8xKcVOcDOfS/++lqcIP/RMMnv/pwSEUrrlKr0XuhbY=; b=aypYhCLnsvt86t2Wvd7PVjWMZ7iDPo1Df5bZx8uURdcb7pdpAy7XtvXon4GNjqYntj L31PTRZeHP19xCnIiUHgM8ThV0gsczF2LgwmKMlMGENdEsx5VWhB5QvCSSRw6u/Wxx15 Y/dW5vL/sObLkG6sV01Svn5Kybp0dicX+1KDTtk3f0s2tnv4U/AO1eBHwMy5zKc/g/L9 Eg7/0h07n80fQJM0K3Iofy/Bzqj1tM0ScEpi+lp/G6GMoDs4EAFTvfgppS/13OxUmmeh Jp74VVl4QLQ4a7be+KGBEv0aw3VdkYgSG9xPi8u70abUHbh/6SuFHOnYetgG4xWp0Y3I qzYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709117467; x=1709722267; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=p8xKcVOcDOfS/++lqcIP/RMMnv/pwSEUrrlKr0XuhbY=; b=Fjzx6ArKWPUsW/fd+TBXJsdy7RQiRl91iL0qWjL3U7OIYg2eiO6YNlNsDQgYtABzg2 2HkycCJNOiF5dlT1tuD8Zev8YR/o0aPe9vHRLYguTV2HjwFnbXpvI0QIuhpaSwvEM7uS boTqWg0K6FNCBNMhRhJl4eF/sJGJGdIU08iTjg41m/d/ky7XDoZCXRkiyrQ1f36U7rC4 AwGUVAXZrARJGr8J7Kk+A91jHP0Wqp/XLhD+VaLiRZkbokUMMez5Zel9pClCPwTDF2xT kGOufnnGUPNixHKqbWfo0sz6TMiD7a0ojTW7M/E+21urh8yXAXn4cytAg7jZd1BEA5zs AGLQ== X-Gm-Message-State: AOJu0YzOgdEpreN7MK7NM7DSft7ekJ410UVsfa0PCro3jsTKtMS5WoF8 +b4DLJTR5Bq1VenyuPRfnm4cYMGP9GhmVwOIL63N1e7Eh+Og+T4OrvCzEtkN X-Google-Smtp-Source: AGHT+IHgyHVajeC+K9AGO4yyM31Hk4wgXSEZUh0IDVU5E49uCpq76G9Ox7vje2xGfTLNSEsKKp3dRA== X-Received: by 2002:a17:906:e283:b0:a3f:2f67:4847 with SMTP id gg3-20020a170906e28300b00a3f2f674847mr7695382ejb.19.1709117466583; Wed, 28 Feb 2024 02:51:06 -0800 (PST) Received: from localhost.lan (031011218106.poznan.vectranet.pl. [31.11.218.106]) by smtp.gmail.com with ESMTPSA id a6-20020a1709063e8600b00a4138c3f054sm1702269ejj.32.2024.02.28.02.51.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Feb 2024 02:51:06 -0800 (PST) From: =?utf-8?b?UmFmYcWCIE1pxYJlY2tp?= To: openwrt-devel@lists.openwrt.org Cc: Jo-Philipp Wich , Luiz Angelo Daros de Luca , =?utf-8?b?UmFmYcWCIE1pxYJl?= =?utf-8?b?Y2tp?= Subject: [PATCH V3 1/3] base-files: sysupgrade: add tar.sh with helpers for building archives Date: Wed, 28 Feb 2024 11:51:00 +0100 Message-Id: <20240228105102.3399-1-zajec5@gmail.com> X-Mailer: git-send-email 2.35.3 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240228_025108_792899_CE457121 X-CRM114-Status: GOOD ( 11.80 ) X-Spam-Score: 0.0 (/) X-Spam-Report: =?unknown-8bit?q?Spam_detection_software=2C_running_on_the_sy?= =?unknown-8bit?q?stem_=22bombadil=2Einfradead=2Eorg=22=2C?= =?unknown-8bit?q?_has_NOT_identified_this_incoming_email_as_spam=2E__The_ori?= =?unknown-8bit?q?ginal?= =?unknown-8bit?q?_message_has_been_attached_to_this_so_you_can_view_it_or_la?= =?unknown-8bit?q?bel?= =?unknown-8bit?q?_similar_future_email=2E__If_you_have_any_questions=2C_see?= =?unknown-8bit?q?_the_administrator_of_that_system_for_details=2E?= =?unknown-8bit?q?_?= =?unknown-8bit?q?_Content_preview=3A__From=3A_Jo-Philipp_Wich_This_allows_bu?= =?unknown-8bit?q?ilding_uncompressed_tar?= =?unknown-8bit?q?_archives_from_shell_scripts_=28and_compressing_them_later_?= =?unknown-8bit?q?if_needed=29_Signed-off-by=3A?= =?unknown-8bit?q?_Rafa=C5=82_Mi=C5=82ecki_---_V2=3A_Simplify_dd_in_=5F=5Ftar?= =?unknown-8bit?q?=5Fprint=5Fpadding_=28I_still_think?= =?unknown-8bit?q?_helper_is_useful=29_Hardcode_0/0/_root/root_for_now_as_mos?= =?unknown-8bit?q?t_likely_it=27ll_be?= =?unknown-8bit?q?_enough_Simplify_name_validation_=28le_=5B=2E=2E=2E=5D_?= =?unknown-8bit?q?_?= =?unknown-8bit?q?_Content_analysis_details=3A___=280=2E0_points=2C_5=2E0_req?= =?unknown-8bit?q?uired=29?= =?unknown-8bit?q?_?= =?unknown-8bit?q?_pts_rule_name______________description?= =?unknown-8bit?q?_----_----------------------_------------------------------?= =?unknown-8bit?q?--------------------?= =?unknown-8bit?q?_-0=2E0_RCVD=5FIN=5FDNSWL=5FNONE_____RBL=3A_Sender_listed_a?= =?unknown-8bit?q?t_https=3A//www=2Ednswl=2Eorg/=2C_no?= =?unknown-8bit?q?_trust?= =?unknown-8bit?b?IFsyYTAwOjE0NTA6NDg2NDoyMDowOjA6MDo2MzMgbGlzdGVkIGluXQ==?= =?unknown-8bit?b?IFtsaXN0LmRuc3dsLm9yZ10=?= =?unknown-8bit?q?_-0=2E0_SPF=5FPASS_______________SPF=3A_sender_matches_SPF_?= =?unknown-8bit?q?record?= =?unknown-8bit?q?_0=2E0_SPF=5FHELO=5FNONE__________SPF=3A_HELO_does_not_publ?= =?unknown-8bit?q?ish_an_SPF_Record?= =?unknown-8bit?q?_-0=2E1_DKIM=5FVALID=5FAU__________Message_has_a_valid_DKIM?= =?unknown-8bit?q?_or_DK_signature_from_author=27s?= =?unknown-8bit?q?_domain?= =?unknown-8bit?q?_-0=2E1_DKIM=5FVALID_____________Message_has_at_least_one_v?= =?unknown-8bit?q?alid_DKIM_or_DK_signature?= =?unknown-8bit?q?_-0=2E1_DKIM=5FVALID=5FEF__________Message_has_a_valid_DKIM?= =?unknown-8bit?q?_or_DK_signature_from?= =?unknown-8bit?q?_envelope-from_domain?= =?unknown-8bit?q?_0=2E1_DKIM=5FSIGNED____________Message_has_a_DKIM_or_DK_si?= =?unknown-8bit?q?gnature=2C_not_necessarily_valid?= =?unknown-8bit?q?_0=2E0_FREEMAIL=5FFROM__________Sender_email_is_commonly_ab?= =?unknown-8bit?q?used_enduser_mail_provider?= =?unknown-8bit?q?_=5Bzajec5=28at=29gmail=2Ecom=5D?= =?unknown-8bit?q?_0=2E2_FREEMAIL=5FENVFROM=5FEND=5FDIGIT_Envelope-from_freem?= =?unknown-8bit?q?ail_username_ends_in?= =?unknown-8bit?q?_digit?= =?unknown-8bit?q?_=5Bzajec5=28at=29gmail=2Ecom=5D?= =?unknown-8bit?q?_-0=2E0_T=5FSCC=5FBODY=5FTEXT=5FLINE___No_description_avail?= =?unknown-8bit?q?able=2E?= X-BeenThere: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: OpenWrt Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "openwrt-devel" Errors-To: openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org From: Jo-Philipp Wich This allows building uncompressed tar archives from shell scripts (and compressing them later if needed) Signed-off-by: Rafał Miłecki Signed-off-by: Jo-Philipp Wich --- V2: Simplify dd in __tar_print_padding (I still think helper is useful) Hardcode 0/0/ root/root for now as most likely it'll be enough Simplify name validation (leasing slash) Reorder some variables V3: Fix dd in __tar_print_padding Rename functions Drop unused functions Document usage package/base-files/files/lib/upgrade/tar.sh | 71 +++++++++++++++++++++ 1 file changed, 71 insertions(+) create mode 100644 package/base-files/files/lib/upgrade/tar.sh diff --git a/package/base-files/files/lib/upgrade/tar.sh b/package/base-files/files/lib/upgrade/tar.sh new file mode 100644 index 0000000000..a9d1d559e6 --- /dev/null +++ b/package/base-files/files/lib/upgrade/tar.sh @@ -0,0 +1,71 @@ +# SPDX-License-Identifier: GPL-2.0-or-later OR MIT + +# Example usage: +# +# { +# tar_print_member "date.txt" "It's $(date +"%Y")" +# tar_print_trailer +# } > test.tar + +__tar_print_padding() { + dd if=/dev/zero bs=1 count=$1 2>/dev/null +} + +tar_print_member() { + local name="$1" + local content="$2" + local mtime="${3:-$(date +%s)}" + local mode=644 + local uid=0 + local gid=0 + local size=${#content} + local type=0 + local link="" + local username="root" + local groupname="root" + + # 100 byte of padding bytes, using 0x01 since the shell does not tolerate null bytes in strings + local pad=$'\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1' + + # validate name (strip leading slash if present) + name=${name#/} + + # truncate string header values to their maximum length + name=${name:0:100} + link=${link:0:100} + username=${username:0:32} + groupname=${groupname:0:32} + + # construct header part before checksum field + local header1="${name}${pad:0:$((100 - ${#name}))}" + header1="${header1}$(printf '%07d\1' $mode)" + header1="${header1}$(printf '%07o\1' $uid)" + header1="${header1}$(printf '%07o\1' $gid)" + header1="${header1}$(printf '%011o\1' $size)" + header1="${header1}$(printf '%011o\1' $mtime)" + + # construct header part after checksum field + local header2="$(printf '%d' $type)" + header2="${header2}${link}${pad:0:$((100 - ${#link}))}" + header2="${header2}ustar ${pad:0:1}" + header2="${header2}${username}${pad:0:$((32 - ${#username}))}" + header2="${header2}${groupname}${pad:0:$((32 - ${#groupname}))}" + + # calculate checksum over header fields + local checksum=0 + for byte in $(printf '%s%8s%s' "$header1" "" "$header2" | tr '\1' '\0' | hexdump -ve '1/1 "%u "'); do + checksum=$((checksum + byte)) + done + + # print member header, padded to 512 byte + printf '%s%06o\0 %s' "$header1" $checksum "$header2" | tr '\1' '\0' + __tar_print_padding 183 + + # print content data, padded to multiple of 512 byte + printf "%s" "$content" + __tar_print_padding $((512 - (size % 512))) +} + +tar_print_trailer() { + __tar_print_padding 1024 +}