From patchwork Tue Feb 13 21:31:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?UmFmYcWCIE1pxYJlY2tp?= X-Patchwork-Id: 1898522 X-Patchwork-Delegate: zajec5@gmail.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=bombadil.20210309 header.b=M0Y2P1Fe; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20230601 header.b=BwvLRqio; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.openwrt.org (client-ip=2607:7c80:54:3::133; helo=bombadil.infradead.org; envelope-from=openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org; receiver=patchwork.ozlabs.org) Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TZF1W1h3gz23j4 for ; Wed, 14 Feb 2024 08:33:15 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=n4hHQFyqbU9dYnsIqtpjnnRziP7GdqRPuqgxB2yZGaw=; b=M0Y2P1Fe7VLKbB ++6YCtdn7EIrgbcgKryL0AMS881Ejl9EecgJxK0VvmXth0dVtUBbnUvtiyLWBHbnRcFUQ9IxOmArx OTODOoh/Df3IsohOPg42ixHCoZzcI1DmBgiQEwhTVmlZOSZylTLufgByoOb9F9idt5fgcgTSG50E6 g2ymw1DjcYCS49E9sRLxfuac0FC+1mbUHLDhIb8TpCCviAcQFoO9nTVTU+WJtWwKlHpzaGDwzarTj 08tB8+JKsGzdxzDLgNq1cT5oHOvuWBIHFJAImB9GzpP7b7Xzn0E0XMLLcbNw8siwIbGGwH6AnJBKF M3Qj27zZQT27A0MYpZbA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1ra0NQ-0000000ApXx-3QMw; Tue, 13 Feb 2024 21:31:40 +0000 Received: from mail-ed1-x52c.google.com ([2a00:1450:4864:20::52c]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1ra0NO-0000000ApXN-1ADF for openwrt-devel@lists.openwrt.org; Tue, 13 Feb 2024 21:31:39 +0000 Received: by mail-ed1-x52c.google.com with SMTP id 4fb4d7f45d1cf-55c2cf644f3so6421985a12.1 for ; Tue, 13 Feb 2024 13:31:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707859896; x=1708464696; darn=lists.openwrt.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=p4f62PHRqbClCgbm63nIOqeXvJb6qbDcZ8IVTm3VvtI=; b=BwvLRqiooXEZrD+y1c1n5Nz3F9PxvEuU+ksolkWbUUkw2jIKx6KsIB8/Cui3SI45KS Ekd97+GAOn5sbhm/BvutJDdhopy9FzW7X1bgozuYv23NOmYcCiy9mnNt76EYcmKvKO9N /0V4ZWMUrzQ0cepYghb2mL9e6iCi3aAwRzpWBOxU7sLainRwR+HOjAbkvwJ78v63goRJ wS53fIpfLIx3Q4UbfCBi2pgSRYaTDRij1TA2GAiLv/A1DyCghwe+Kr7YaY5vEnKPIS8j UhTOhV32FO6TGLD948ucejNFn+8gUEcxe+2zfhzYFQRbf9MeQU1eFTT3K7pPQKhoYNYj /Tfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707859896; x=1708464696; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=p4f62PHRqbClCgbm63nIOqeXvJb6qbDcZ8IVTm3VvtI=; b=dTatlCSbZoKCjHHTW5VKKKus3KC67UBUTx2GfkcfKFUP130MtkCGL9f/aM0uDSuPnP xRurXlr0HADVAqAkFBlacBRe34pdig7WmMMbRaMmTpVTXnCbY6ZlKYjsQBhT+06tJucD bQosPBhfhH9AG5YMDn0kj2rYm3dQFpb8cYXMVDkgivx3MY9SGNIAjnnXx0moAKcdI/AX bxk6ulhuwj1/oo4qZ9oYOdVAPGceVhIGh40Mu4P2qWNcLoLPpX5YyhYrDR2/7+V85t80 lvxAJ/A671t42m/4ci0Pm3/EN9Pilvuni6sesOjiSWnTM8E/tLC0ty15AWncDRcXqDfj Z01g== X-Gm-Message-State: AOJu0Yyirl7WfF8Txm7lpBz/GTVwtwOC3AL8D8xfG1YBY8pFLT6zxzxc U8/edmIEm7sWDmqnCGtvoDhpohCzzHlCZxXkcwz2gYm2U610Eyr8w+oFD7zt X-Google-Smtp-Source: AGHT+IE9UPf5GQYFB/KnG+qoWUdJ56FATqX7Z5SCbHWnWwOn3gSASnI/nReHIA8x7zbAubAomfSUDg== X-Received: by 2002:a17:906:759:b0:a3c:12f1:2383 with SMTP id z25-20020a170906075900b00a3c12f12383mr371435ejb.57.1707859896221; Tue, 13 Feb 2024 13:31:36 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCXDm4EsqkbS9o0c4I5rHaw4y6cTYOzJoPJRMlbBIhPvuHcB0/TFqC7bfk3TbUeGNf+JZFo0rIzO13xeGXeveAIJ2M0wqb/MvTDXA1403q2en3w/moMf/rRsbF+7CwrwRAvTLCBYx++QWDyqoNFqfqJjJp9t6Mre/9F1CKmKoT9/YzZNFQceXJrxytZi9Fh1ELnG5J5/CZIH4g== Received: from localhost.lan (031011218106.poznan.vectranet.pl. [31.11.218.106]) by smtp.gmail.com with ESMTPSA id tj13-20020a170907c24d00b00a38a705121csm1649495ejc.47.2024.02.13.13.31.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 13:31:35 -0800 (PST) From: =?utf-8?b?UmFmYcWCIE1pxYJlY2tp?= To: openwrt-devel@lists.openwrt.org Cc: =?utf-8?b?UmFmYcWCIE1pxYJlY2tp?= , Luiz Angelo Daros de Luca , Christian Marangi , Jo-Philipp Wich , Jonas Gorski Subject: [PATCH RFC] base-files: sysupgrade: always setup overlay when creating backup Date: Tue, 13 Feb 2024 22:31:29 +0100 Message-Id: <20240213213129.18484-1-zajec5@gmail.com> X-Mailer: git-send-email 2.35.3 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240213_133138_371900_03347E3E X-CRM114-Status: GOOD ( 12.42 ) X-Spam-Score: 0.0 (/) X-Spam-Report: =?unknown-8bit?q?Spam_detection_software=2C_running_on_the_sy?= =?unknown-8bit?q?stem_=22bombadil=2Einfradead=2Eorg=22=2C?= =?unknown-8bit?q?_has_NOT_identified_this_incoming_email_as_spam=2E__The_ori?= =?unknown-8bit?q?ginal?= =?unknown-8bit?q?_message_has_been_attached_to_this_so_you_can_view_it_or_la?= =?unknown-8bit?q?bel?= =?unknown-8bit?q?_similar_future_email=2E__If_you_have_any_questions=2C_see?= =?unknown-8bit?q?_the_administrator_of_that_system_for_details=2E?= =?unknown-8bit?q?_?= =?unknown-8bit?q?_Content_preview=3A__From=3A_Rafa=C5=82_Mi=C5=82ecki_Settin?= =?unknown-8bit?q?g_overlay_while_creating_backup?= =?unknown-8bit?q?_allows_including_extra_files_in_archive_without_actually_w?= =?unknown-8bit?q?riting_them_to?= =?unknown-8bit?q?_flash=2E_Right_now_this_feature_is_limited_to_/etc/backup/?= =?unknown-8bit?q?_directory_and_is?= =?unknown-8bit?b?IHVzZWQgb25seSBmb3IgaSBbLi4uXSA=?= =?unknown-8bit?q?_?= =?unknown-8bit?q?_Content_analysis_details=3A___=280=2E0_points=2C_5=2E0_req?= =?unknown-8bit?q?uired=29?= =?unknown-8bit?q?_?= =?unknown-8bit?q?_pts_rule_name______________description?= =?unknown-8bit?q?_----_----------------------_------------------------------?= =?unknown-8bit?q?--------------------?= =?unknown-8bit?q?_0=2E0_SPF=5FHELO=5FNONE__________SPF=3A_HELO_does_not_publ?= =?unknown-8bit?q?ish_an_SPF_Record?= =?unknown-8bit?q?_-0=2E0_SPF=5FPASS_______________SPF=3A_sender_matches_SPF_?= =?unknown-8bit?q?record?= =?unknown-8bit?q?_-0=2E1_DKIM=5FVALID=5FAU__________Message_has_a_valid_DKIM?= =?unknown-8bit?q?_or_DK_signature_from_author=27s?= =?unknown-8bit?q?_domain?= =?unknown-8bit?q?_-0=2E1_DKIM=5FVALID_____________Message_has_at_least_one_v?= =?unknown-8bit?q?alid_DKIM_or_DK_signature?= =?unknown-8bit?q?_0=2E1_DKIM=5FSIGNED____________Message_has_a_DKIM_or_DK_si?= =?unknown-8bit?q?gnature=2C_not_necessarily_valid?= =?unknown-8bit?q?_-0=2E1_DKIM=5FVALID=5FEF__________Message_has_a_valid_DKIM?= =?unknown-8bit?q?_or_DK_signature_from?= =?unknown-8bit?q?_envelope-from_domain?= =?unknown-8bit?q?_0=2E2_FREEMAIL=5FENVFROM=5FEND=5FDIGIT_Envelope-from_freem?= =?unknown-8bit?q?ail_username_ends_in?= =?unknown-8bit?q?_digit?= =?unknown-8bit?q?_=5Bzajec5=28at=29gmail=2Ecom=5D?= =?unknown-8bit?q?_0=2E0_FREEMAIL=5FFROM__________Sender_email_is_commonly_ab?= =?unknown-8bit?q?used_enduser_mail_provider?= =?unknown-8bit?q?_=5Bzajec5=28at=29gmail=2Ecom=5D?= =?unknown-8bit?q?_-0=2E0_RCVD=5FIN=5FDNSWL=5FNONE_____RBL=3A_Sender_listed_a?= =?unknown-8bit?q?t_https=3A//www=2Ednswl=2Eorg/=2C_no?= =?unknown-8bit?q?_trust?= =?unknown-8bit?b?IFsyYTAwOjE0NTA6NDg2NDoyMDowOjA6MDo1MmMgbGlzdGVkIGluXQ==?= =?unknown-8bit?b?IFtsaXN0LmRuc3dsLm9yZ10=?= =?unknown-8bit?q?_-0=2E0_T=5FSCC=5FBODY=5FTEXT=5FLINE___No_description_avail?= =?unknown-8bit?q?able=2E?= X-BeenThere: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: OpenWrt Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "openwrt-devel" Errors-To: openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org From: Rafał Miłecki Setting overlay while creating backup allows including extra files in archive without actually writing them to flash. Right now this feature is limited to /etc/backup/ directory and is used only for including installed_packages.txt. Extend this solution to make it more generic: 1. Always mount overlay while creating backup 2. Overlay whole / to don't limit it to /etc/backup/ This allows including any additional files in backups and adding more sysupgrade features. Cc: Luiz Angelo Daros de Luca Cc: Christian Marangi Cc: Jo-Philipp Wich Cc: Jonas Gorski Signed-off-by: Rafał Miłecki --- This will allow me to include /etc/uci-defaults/ scripts in backups so we can e.g. have script disabling previously disabled services. package/base-files/files/sbin/sysupgrade | 37 ++++++++++++++---------- 1 file changed, 21 insertions(+), 16 deletions(-) diff --git a/package/base-files/files/sbin/sysupgrade b/package/base-files/files/sbin/sysupgrade index 1fcd44da2a..1e09f65e07 100755 --- a/package/base-files/files/sbin/sysupgrade +++ b/package/base-files/files/sbin/sysupgrade @@ -237,7 +237,8 @@ include /lib/upgrade create_backup_archive() { local conf_tar="$1" - local umount_etcbackup_dir=0 + local overlay + local dir [ "$(rootfs_type)" = "tmpfs" ] && { echo "Cannot save config while running from ramdisk." >&2 @@ -248,17 +249,20 @@ create_backup_archive() { run_hooks "$CONFFILES" $sysupgrade_init_conffiles ask_bool 0 "Edit config file list" && vi "$CONFFILES" + # Mount root directory with temporary overlay on top of it. + # This allows including extra (temporary) files in backup archive + # without messing actual rootfs. + overlay="$(mktemp -d -t overlay.XXXXXX)" + mkdir -p "$overlay/upper" "$overlay/work" + dir="$(mktemp -d -t backup.XXXXXX)" + if ! mount -t overlay overlay -o lowerdir=/,upperdir="$overlay/upper",workdir="$overlay/work" "$dir"; then + echo "Cannot mount backup overlay to $dir." >&2 + ask_bool 0 "Abort" && exit + fi + if [ "$SAVE_INSTALLED_PKGS" -eq 1 ]; then echo "${INSTALLED_PACKAGES}" >> "$CONFFILES" - mkdir -p "$ETCBACKUP_DIR" - # Avoid touching filesystem on each backup - RAMFS="$(mktemp -d -t sysupgrade.XXXXXX)" - mkdir -p "$RAMFS/upper" "$RAMFS/work" - mount -t overlay overlay -o lowerdir=$ETCBACKUP_DIR,upperdir=$RAMFS/upper,workdir=$RAMFS/work $ETCBACKUP_DIR && - umount_etcbackup_dir=1 || { - echo "Cannot mount '$ETCBACKUP_DIR' as tmpfs to avoid touching disk while saving the list of installed packages." >&2 - ask_bool 0 "Abort" && exit - } + mkdir -p "$dir/$ETCBACKUP_DIR" # Format: pkg-name{rom,overlay,unkown} # rom is used for pkgs in /rom, even if updated later @@ -266,22 +270,23 @@ create_backup_archive() { \( -exec test -f /rom/{} \; -exec echo {} rom \; \) -o \ \( -exec test -f /overlay/upper/{} \; -exec echo {} overlay \; \) -o \ \( -exec echo {} unknown \; \) \ - \) | sed -e 's,.*/,,;s/\.control /\t/' > ${INSTALLED_PACKAGES} + \) | sed -e 's,.*/,,;s/\.control /\t/' > "$dir/${INSTALLED_PACKAGES}" fi v "Saving config files..." + sed -i 's/^\///' "$CONFFILES" # Drop leading slashes [ "$VERBOSE" -gt 1 ] && TAR_V="v" || TAR_V="" - tar c${TAR_V}zf "$conf_tar" -T "$CONFFILES" 2>/dev/null + tar c${TAR_V}zf "$conf_tar" -C "${dir:-/}" -T "$CONFFILES" if [ "$?" -ne 0 ]; then echo "Failed to create the configuration backup." rm -f "$conf_tar" exit 1 fi - [ "$umount_etcbackup_dir" -eq 1 ] && { - umount "$ETCBACKUP_DIR" - rm -rf "$RAMFS" - } + umount "$dir" + rm -rf "$dir" + rm -rf "$overlay" + rm -f "$CONFFILES" }