Message ID | 20240213213129.18484-1-zajec5@gmail.com |
---|---|
State | RFC |
Delegated to: | Rafał Miłecki |
Headers | show |
Series | [RFC] base-files: sysupgrade: always setup overlay when creating backup | expand |
The sender domain has a DMARC Reject/Quarantine policy which disallows sending mailing list messages using the original "From" header. To mitigate this problem, the original message has been wrapped automatically by the mailing list software. On Tuesday, February 13th, 2024 at 13:31, Rafał Miłecki <zajec5@gmail.com> wrote: > From: Rafał Miłecki rafal@milecki.pl > > Setting overlay while creating backup allows including extra files in > archive without actually writing them to flash. Right now this feature > is limited to /etc/backup/ directory and is used only for including > installed_packages.txt. > > Extend this solution to make it more generic: > 1. Always mount overlay while creating backup > 2. Overlay whole / to don't limit it to /etc/backup/ > > This allows including any additional files in backups and adding more > sysupgrade features. ... > --- > This will allow me to include /etc/uci-defaults/ scripts in backups so > we can e.g. have script disabling previously disabled services. There is a pending PR[1] against the auc package that looks for /rom/etc/uci-defaults/99-asu-defaults and if found sends its contents in a request to the ASU build server for inclusion in uci-defaults. Am I correct in assuming that this proposal would allow an alternative mechanism for accomplishing the same thing, thus making the PR (and, in fact, the 'defaults' mechanism of the ASU build server) redundant? Also, what are the implications for uci-defaults on ext4? Do we need something that creates a first-boot-backup for sysupgrade to use later? [1] - https://github.com/openwrt/packages/pull/22144 Eric
> From: Rafał Miłecki <rafal@milecki.pl> > > Setting overlay while creating backup allows including extra files in > archive without actually writing them to flash. Right now this feature > is limited to /etc/backup/ directory and is used only for including > installed_packages.txt. > > Extend this solution to make it more generic: > 1. Always mount overlay while creating backup > 2. Overlay whole / to don't limit it to /etc/backup/ > > This allows including any additional files in backups and adding more > sysupgrade features. It is a nice feature. Thanks. > > Cc: Luiz Angelo Daros de Luca <luizluca@gmail.com> > Cc: Christian Marangi <ansuelsmth@gmail.com> > Cc: Jo-Philipp Wich <jo@mein.io> > Cc: Jonas Gorski <jonas.gorski@gmail.com> > Signed-off-by: Rafał Miłecki <rafal@milecki.pl> > --- > This will allow me to include /etc/uci-defaults/ scripts in backups so > we can e.g. have script disabling previously disabled services. > > package/base-files/files/sbin/sysupgrade | 37 ++++++++++++++---------- > 1 file changed, 21 insertions(+), 16 deletions(-) > > diff --git a/package/base-files/files/sbin/sysupgrade b/package/base-files/files/sbin/sysupgrade > index 1fcd44da2a..1e09f65e07 100755 > --- a/package/base-files/files/sbin/sysupgrade > +++ b/package/base-files/files/sbin/sysupgrade > @@ -237,7 +237,8 @@ include /lib/upgrade > create_backup_archive() { > local conf_tar="$1" > > - local umount_etcbackup_dir=0 > + local overlay > + local dir > > [ "$(rootfs_type)" = "tmpfs" ] && { > echo "Cannot save config while running from ramdisk." >&2 > @@ -248,17 +249,20 @@ create_backup_archive() { > run_hooks "$CONFFILES" $sysupgrade_init_conffiles > ask_bool 0 "Edit config file list" && vi "$CONFFILES" > > + # Mount root directory with temporary overlay on top of it. > + # This allows including extra (temporary) files in backup archive > + # without messing actual rootfs. > + overlay="$(mktemp -d -t overlay.XXXXXX)" > + mkdir -p "$overlay/upper" "$overlay/work" > + dir="$(mktemp -d -t backup.XXXXXX)" You could avoid two mktemp creating a single unique parent directory. It makes the cleanup simpler. > + if ! mount -t overlay overlay -o lowerdir=/,upperdir="$overlay/upper",workdir="$overlay/work" "$dir"; then > + echo "Cannot mount backup overlay to $dir." >&2 > + ask_bool 0 "Abort" && exit > + fi > + > if [ "$SAVE_INSTALLED_PKGS" -eq 1 ]; then > echo "${INSTALLED_PACKAGES}" >> "$CONFFILES" > - mkdir -p "$ETCBACKUP_DIR" > - # Avoid touching filesystem on each backup > - RAMFS="$(mktemp -d -t sysupgrade.XXXXXX)" > - mkdir -p "$RAMFS/upper" "$RAMFS/work" > - mount -t overlay overlay -o lowerdir=$ETCBACKUP_DIR,upperdir=$RAMFS/upper,workdir=$RAMFS/work $ETCBACKUP_DIR && > - umount_etcbackup_dir=1 || { > - echo "Cannot mount '$ETCBACKUP_DIR' as tmpfs to avoid touching disk while saving the list of installed packages." >&2 > - ask_bool 0 "Abort" && exit > - } > + mkdir -p "$dir/$ETCBACKUP_DIR" > > # Format: pkg-name<TAB>{rom,overlay,unkown} > # rom is used for pkgs in /rom, even if updated later > @@ -266,22 +270,23 @@ create_backup_archive() { > \( -exec test -f /rom/{} \; -exec echo {} rom \; \) -o \ > \( -exec test -f /overlay/upper/{} \; -exec echo {} overlay \; \) -o \ > \( -exec echo {} unknown \; \) \ > - \) | sed -e 's,.*/,,;s/\.control /\t/' > ${INSTALLED_PACKAGES} > + \) | sed -e 's,.*/,,;s/\.control /\t/' > "$dir/${INSTALLED_PACKAGES}" > fi > > v "Saving config files..." > + sed -i 's/^\///' "$CONFFILES" # Drop leading slashes > [ "$VERBOSE" -gt 1 ] && TAR_V="v" || TAR_V="" > - tar c${TAR_V}zf "$conf_tar" -T "$CONFFILES" 2>/dev/null > + tar c${TAR_V}zf "$conf_tar" -C "${dir:-/}" -T "$CONFFILES" These were already part of an PR that fixed a couple of sysupgrade issues: https://github.com/openwrt/openwrt/pull/11022 (since Oct 2022 but still relevant), including here. > if [ "$?" -ne 0 ]; then > echo "Failed to create the configuration backup." > rm -f "$conf_tar" > exit 1 Like this one. You need to unmount and delete everything before bailing out. Check the PR. Maybe we should merge it and rebase your change over it. Or you can merge it into this patch. > fi > > - [ "$umount_etcbackup_dir" -eq 1 ] && { > - umount "$ETCBACKUP_DIR" > - rm -rf "$RAMFS" > - } > + umount "$dir" > + rm -rf "$dir" > + rm -rf "$overlay" > + > rm -f "$CONFFILES" > } > > -- > 2.35.3 >
diff --git a/package/base-files/files/sbin/sysupgrade b/package/base-files/files/sbin/sysupgrade index 1fcd44da2a..1e09f65e07 100755 --- a/package/base-files/files/sbin/sysupgrade +++ b/package/base-files/files/sbin/sysupgrade @@ -237,7 +237,8 @@ include /lib/upgrade create_backup_archive() { local conf_tar="$1" - local umount_etcbackup_dir=0 + local overlay + local dir [ "$(rootfs_type)" = "tmpfs" ] && { echo "Cannot save config while running from ramdisk." >&2 @@ -248,17 +249,20 @@ create_backup_archive() { run_hooks "$CONFFILES" $sysupgrade_init_conffiles ask_bool 0 "Edit config file list" && vi "$CONFFILES" + # Mount root directory with temporary overlay on top of it. + # This allows including extra (temporary) files in backup archive + # without messing actual rootfs. + overlay="$(mktemp -d -t overlay.XXXXXX)" + mkdir -p "$overlay/upper" "$overlay/work" + dir="$(mktemp -d -t backup.XXXXXX)" + if ! mount -t overlay overlay -o lowerdir=/,upperdir="$overlay/upper",workdir="$overlay/work" "$dir"; then + echo "Cannot mount backup overlay to $dir." >&2 + ask_bool 0 "Abort" && exit + fi + if [ "$SAVE_INSTALLED_PKGS" -eq 1 ]; then echo "${INSTALLED_PACKAGES}" >> "$CONFFILES" - mkdir -p "$ETCBACKUP_DIR" - # Avoid touching filesystem on each backup - RAMFS="$(mktemp -d -t sysupgrade.XXXXXX)" - mkdir -p "$RAMFS/upper" "$RAMFS/work" - mount -t overlay overlay -o lowerdir=$ETCBACKUP_DIR,upperdir=$RAMFS/upper,workdir=$RAMFS/work $ETCBACKUP_DIR && - umount_etcbackup_dir=1 || { - echo "Cannot mount '$ETCBACKUP_DIR' as tmpfs to avoid touching disk while saving the list of installed packages." >&2 - ask_bool 0 "Abort" && exit - } + mkdir -p "$dir/$ETCBACKUP_DIR" # Format: pkg-name<TAB>{rom,overlay,unkown} # rom is used for pkgs in /rom, even if updated later @@ -266,22 +270,23 @@ create_backup_archive() { \( -exec test -f /rom/{} \; -exec echo {} rom \; \) -o \ \( -exec test -f /overlay/upper/{} \; -exec echo {} overlay \; \) -o \ \( -exec echo {} unknown \; \) \ - \) | sed -e 's,.*/,,;s/\.control /\t/' > ${INSTALLED_PACKAGES} + \) | sed -e 's,.*/,,;s/\.control /\t/' > "$dir/${INSTALLED_PACKAGES}" fi v "Saving config files..." + sed -i 's/^\///' "$CONFFILES" # Drop leading slashes [ "$VERBOSE" -gt 1 ] && TAR_V="v" || TAR_V="" - tar c${TAR_V}zf "$conf_tar" -T "$CONFFILES" 2>/dev/null + tar c${TAR_V}zf "$conf_tar" -C "${dir:-/}" -T "$CONFFILES" if [ "$?" -ne 0 ]; then echo "Failed to create the configuration backup." rm -f "$conf_tar" exit 1 fi - [ "$umount_etcbackup_dir" -eq 1 ] && { - umount "$ETCBACKUP_DIR" - rm -rf "$RAMFS" - } + umount "$dir" + rm -rf "$dir" + rm -rf "$overlay" + rm -f "$CONFFILES" }