| Message ID | 20210111220726.23069-1-devel-sven@geroedel.de |
|---|---|
| State | New |
| Headers | show |
| Series | [v2] base-files: restore status of system-services after sysupgrade | expand |
On 2021-01-11 23:07, Sven Roederer wrote: > Restore the status of the system-services after sysupgrade. > Create a file with the status of all known services and keep it during > upgrade. After upgrade run a uci-default script to restore every > single service. Doing the restore in an uci-default script is to late, because then you are already in the procd rcS helper "loop" and new added init links won't get called. Also, have you thought about a backup/restore of the configuration? The whole thing should/must work there as well. I would do the restore at the end of the pre-init stage. Martin
On 1/12/21 12:07 AM, Sven Roederer wrote: > Restore the status of the system-services after sysupgrade. > Create a file with the status of all known services and keep it during > upgrade. After upgrade run a uci-default script to restore every > single service. > The list of the service status will be stored in etc/backup folder also > and formated as: > /etc/init.d/<service> {enable|disable} > > When upgrading with an generic image all system services are enabled by > default which is usually not expected and can cause trouble. > The default behavior can be flipped with the "-s" option of sysupgrade. > > Signed-off-by: Sven Roederer <devel-sven@geroedel.de> > --- > > This v2 includes the feedback from the mailinglist: > * entangles this feature from "storing the packagelist" > * adds a option for disabling to sysupgrade > * makes it enabled by default > * includes the glue-script to restore the services after upgrade > > I was not able to test on a real system yet. > > package/base-files/Makefile | 2 +- > .../uci-defaults/14_restore-services-state | 3 ++ > package/base-files/files/sbin/sysupgrade | 38 ++++++++++++++----- > 3 files changed, 32 insertions(+), 11 deletions(-) > create mode 100644 package/base-files/files/etc/uci-defaults/14_restore-services-state > > diff --git a/package/base-files/Makefile b/package/base-files/Makefile > index f18f221129..5a9bacba13 100644 > --- a/package/base-files/Makefile > +++ b/package/base-files/Makefile > @@ -12,7 +12,7 @@ include $(INCLUDE_DIR)/version.mk > include $(INCLUDE_DIR)/feeds.mk > > PKG_NAME:=base-files > -PKG_RELEASE:=246 > +PKG_RELEASE:=247 > PKG_FLAGS:=nonshared > > PKG_FILE_DEPENDS:=$(PLATFORM_DIR)/ $(GENERIC_PLATFORM_DIR)/base-files/ > diff --git a/package/base-files/files/etc/uci-defaults/14_restore-services-state b/package/base-files/files/etc/uci-defaults/14_restore-services-state > new file mode 100644 > index 0000000000..d0085f3f80 > --- /dev/null > +++ b/package/base-files/files/etc/uci-defaults/14_restore-services-state > @@ -0,0 +1,3 @@ > +SERVICE_STATUS=/etc/backup/service_status.txt > + > +[ -e ${SERVICE_STATUS} ] && . ${SERVICE_STATUS} > diff --git a/package/base-files/files/sbin/sysupgrade b/package/base-files/files/sbin/sysupgrade > index 50f4e08e44..068ec27e3d 100755 > --- a/package/base-files/files/sbin/sysupgrade > +++ b/package/base-files/files/sbin/sysupgrade > @@ -15,6 +15,7 @@ export SAVE_OVERLAY_PATH= > export SAVE_PARTITIONS=1 > export SAVE_INSTALLED_PKGS=0 > export SKIP_UNCHANGED=1 > +export SAVE_SERVICE_STATUS=1 > export CONF_IMAGE= > export CONF_BACKUP_LIST=0 > export CONF_BACKUP= > @@ -35,6 +36,7 @@ while [ -n "$1" ]; do > -c) export SAVE_OVERLAY=1 SAVE_OVERLAY_PATH=/etc;; > -o) export SAVE_OVERLAY=1 SAVE_OVERLAY_PATH=/;; > -p) export SAVE_PARTITIONS=0;; > + -s) export SAVE_SERVICE_STATUS=0;; > -k) export SAVE_INSTALLED_PKGS=1;; > -U) export SKIP_UNCHANGED=0;; > -b|--create-backup) export CONF_BACKUP="$2" NEED_IMAGE=1; shift;; > @@ -57,13 +59,14 @@ export CONFFILES=/tmp/sysupgrade.conffiles > export CONF_TAR=/tmp/sysupgrade.tgz > export ETCBACKUP_DIR=/etc/backup > export INSTALLED_PACKAGES=${ETCBACKUP_DIR}/installed_packages.txt > +export SERVICE_STATUS=${ETCBACKUP_DIR}/service_status.txt > > IMAGE="$1" > > [ -z "$IMAGE" -a -z "$NEED_IMAGE" -a $CONF_BACKUP_LIST -eq 0 -o $HELP -gt 0 ] && { > cat <<EOF > Usage: $0 [<upgrade-option>...] <image file or URL> > - $0 [-q] [-i] [-c] [-U] [-o] [-k] <backup-command> <file> > + $0 [-q] [-i] [-c] [-U] [-o] [-s] [-k] <backup-command> <file> > > upgrade-option: > -f <config> restore configuration from .tar.gz (file or url) > @@ -75,6 +78,7 @@ upgrade-option: > (this is a modification of Freifunk Berlin to unset -u) > -n do not save configuration over reflash > -p do not attempt to restore the partition table after flash. > + -s do not attempt to restore the status of the services. > -k include in backup a list of current installed packages at > $INSTALLED_PACKAGES > -T | --test > @@ -231,8 +235,7 @@ do_save_conffiles() { > run_hooks "$CONFFILES" $sysupgrade_init_conffiles > ask_bool 0 "Edit config file list" && vi "$CONFFILES" > > - if [ "$SAVE_INSTALLED_PKGS" -eq 1 ]; then > - echo "${INSTALLED_PACKAGES}" >> "$CONFFILES" > + if [ "$SAVE_INSTALLED_PKGS" -eq 1 ] || [ "$SAVE_SERVICE_STATUS" -eq 1 ]; then > mkdir -p "$ETCBACKUP_DIR" > # Avoid touching filesystem on each backup > RAMFS="$(mktemp -d -t sysupgrade.XXXXXX)" > @@ -243,13 +246,28 @@ do_save_conffiles() { > ask_bool 0 "Abort" && exit > } > > - # Format: pkg-name<TAB>{rom,overlay,unkown} > - # rom is used for pkgs in /rom, even if updated later > - find /usr/lib/opkg/info -name "*.control" \( \ > - \( -exec test -f /rom/{} \; -exec echo {} rom \; \) -o \ > - \( -exec test -f /overlay/upper/{} \; -exec echo {} overlay \; \) -o \ > - \( -exec echo {} unknown \; \) \ > - \) | sed -e 's,.*/,,;s/\.control /\t/' > ${INSTALLED_PACKAGES} > + if [ "$SAVE_INSTALLED_PKGS" -eq 1 ]; then > + echo "${INSTALLED_PACKAGES}" >> "$CONFFILES" > + # Format: pkg-name<TAB>{rom,overlay,unkown} > + # rom is used for pkgs in /rom, even if updated later > + find /usr/lib/opkg/info -name "*.control" \( \ > + \( -exec test -f /rom/{} \; -exec echo {} rom \; \) -o \ > + \( -exec test -f /overlay/upper/{} \; -exec echo {} overlay \; \) -o \ > + \( -exec echo {} unknown \; \) \ > + \) | sed -e 's,.*/,,;s/\.control /\t/' > ${INSTALLED_PACKAGES} > + fi > + > + if [ "$SAVE_SERVICE_STATUS" -eq 1 ]; then > + echo "${SERVICE_STATUS}" >> "$CONFFILES" > + # Format: /etc/init.d/servicename {enable,disable} > + rm -f ${SERVICE_STATUS} > + for service in /etc/init.d/* ; do \ > + ${service} enabled && \ > + echo >> ${SERVICE_STATUS} "$service" "enable" || \ > + echo >> ${SERVICE_STATUS} "$service" "disable" \ > + ; \ > + done > + fi > fi > > v "Saving config files..." Hello, You don't have to run such complicated script. Just restore /etc/rc.d/ directory after upgrade.
> > @@ -231,8 +235,7 @@ do_save_conffiles() { > > run_hooks "$CONFFILES" $sysupgrade_init_conffiles > > ask_bool 0 "Edit config file list" && vi "$CONFFILES" > > > > - if [ "$SAVE_INSTALLED_PKGS" -eq 1 ]; then > > - echo "${INSTALLED_PACKAGES}" >> "$CONFFILES" > > + if [ "$SAVE_INSTALLED_PKGS" -eq 1 ] || [ "$SAVE_SERVICE_STATUS" > -eq > > +1 ]; then > > mkdir -p "$ETCBACKUP_DIR" > > # Avoid touching filesystem on each backup > > RAMFS="$(mktemp -d -t sysupgrade.XXXXXX)" > > @@ -243,13 +246,28 @@ do_save_conffiles() { > > ask_bool 0 "Abort" && exit > > } > > > > - # Format: pkg-name<TAB>{rom,overlay,unkown} > > - # rom is used for pkgs in /rom, even if updated later > > - find /usr/lib/opkg/info -name "*.control" \( \ > > - \( -exec test -f /rom/{} \; -exec echo {} rom \; \) -o \ > > - \( -exec test -f /overlay/upper/{} \; -exec echo {} > overlay \; \) -o \ > > - \( -exec echo {} unknown \; \) \ > > - \) | sed -e 's,.*/,,;s/\.control /\t/' > > ${INSTALLED_PACKAGES} > > + if [ "$SAVE_INSTALLED_PKGS" -eq 1 ]; then > > + echo "${INSTALLED_PACKAGES}" >> "$CONFFILES" > > + # Format: pkg-name<TAB>{rom,overlay,unkown} > > + # rom is used for pkgs in /rom, even if updated later > > + find /usr/lib/opkg/info -name "*.control" \( \ > > + \( -exec test -f /rom/{} \; -exec echo {} rom \; > \) -o \ > > + \( -exec test -f /overlay/upper/{} \; -exec > echo {} overlay \; \) -o \ > > + \( -exec echo {} unknown \; \) \ > > + \) | sed -e 's,.*/,,;s/\.control /\t/' > > ${INSTALLED_PACKAGES} > > + fi > > + > > + if [ "$SAVE_SERVICE_STATUS" -eq 1 ]; then > > + echo "${SERVICE_STATUS}" >> "$CONFFILES" > > + # Format: /etc/init.d/servicename {enable,disable} > > + rm -f ${SERVICE_STATUS} > > + for service in /etc/init.d/* ; do \ > > + ${service} enabled && \ > > + echo >> ${SERVICE_STATUS} > "$service" "enable" || \ > > + echo >> ${SERVICE_STATUS} > "$service" "disable" \ > > + ; \ > > + done > > + fi > > fi > > > > v "Saving config files..." > Hello, > You don't have to run such complicated script. Just restore /etc/rc.d/ > directory after upgrade. > Well, that depends on how we want to deal with new services. If I upgrade from 19.07.5 to master, I'd expect new services to be in their default initial state. Restoring "old" rc.d would have everything new disabled. That's counter-intuitive for me. Best Adrian
Martin Schiller kirjoitti 12.1.2021 klo 9.25: > On 2021-01-11 23:07, Sven Roederer wrote: >> Restore the status of the system-services after sysupgrade. >> Create a file with the status of all known services and keep it during >> upgrade. After upgrade run a uci-default script to restore every >> single service. > > Doing the restore in an uci-default script is to late, because then you > are already in the procd rcS helper "loop" and new added init links > won't get called. > > Also, have you thought about a backup/restore of the configuration? > The whole thing should/must work there as well. > > I would do the restore at the end of the pre-init stage. > > Martin My two cents to the discussion: * I assume that a small minority of users actually disable services, or at least disable several services, so defaulting to "lets by default always store status of all services" sounds like overkill. * Backuping always info that the dozens of standard services are enabled sounds unncessary. As services are installed by default as "enabled" to the image, it might make sense to store only info about "disabled" services and disable those when restoring the backup in sysupgrade. (Re-enabling the already enabled services is unnecessary.) That would also help to keep the amount of new backup info small. * make clear in sysupgrade script help texts and documentation that saving/restoring service status depends on "keep settings". If settings are not kept, the service status can not be copied in sysupugrade as there is no sysupgrade.tar.gz to be passed. (And naturally it would not even make sense to e.g. revert network settings to defaults but keep selected network services disabled. Easy to cause soft-bricks that way.)
Am Dienstag, 12. Januar 2021, 19:56:54 CET schrieb Hannu Nyman: > Martin Schiller kirjoitti 12.1.2021 klo 9.25: > > On 2021-01-11 23:07, Sven Roederer wrote: > >> Restore the status of the system-services after sysupgrade. > >> Create a file with the status of all known services and keep it during > >> upgrade. After upgrade run a uci-default script to restore every > >> single service. > > > > Doing the restore in an uci-default script is to late, because then you > > are already in the procd rcS helper "loop" and new added init links > > won't get called. > > > > Also, have you thought about a backup/restore of the configuration? > > The whole thing should/must work there as well. > > > > I would do the restore at the end of the pre-init stage. > > > > Martin > > My two cents to the discussion: > > * I assume that a small minority of users actually disable services, or at > least disable several services, so defaulting to "lets by default always > store status of all services" sounds like overkill. > > * Backuping always info that the dozens of standard services are enabled > sounds unncessary. As services are installed by default as "enabled" to the > image, it might make sense to store only info about "disabled" services and > disable those when restoring the backup in sysupgrade. (Re-enabling the > already enabled services is unnecessary.) That would also help to keep the > amount of new backup info small. > > * make clear in sysupgrade script help texts and documentation that > saving/restoring service status depends on "keep settings". If settings are > not kept, the service status can not be copied in sysupugrade as there is no > sysupgrade.tar.gz to be passed. (And naturally it would not even make sense > to e.g. revert network settings to defaults but keep selected network > services disabled. Easy to cause soft-bricks that way.) > > Hannu, as long as no service was disabled with the "DISABLED_SERVICES" option of the imagebuilder the explicit "enable" is not required. Indeed enabling a service that have been disabled during image creation is a bit edge-case, but I prefer here the "full featured" way. Not enabling such a service also breaks the user experience like the disable case. For sure I can add a explicit note, that the "-n" option will not keep the states. But "do not save configuration over reflash" imho implicitly means all image defaults are used.
Am Dienstag, 12. Januar 2021, 13:01:45 CET schrieb Adrian Schmutzler: > > Hello, > > You don't have to run such complicated script. Just restore /etc/rc.d/ > > directory after upgrade. > > Well, that depends on how we want to deal with new services. If I upgrade > from 19.07.5 to master, I'd expect new services to be in their default > initial state. Restoring "old" rc.d would have everything new disabled. > That's counter-intuitive for me. I'm also in favor of not touching new services, which have not been installed before the upgrade. Sven
On 2021-01-12 23:03, Sven Roederer wrote: > Am Dienstag, 12. Januar 2021, 19:56:54 CET schrieb Hannu Nyman: >> Martin Schiller kirjoitti 12.1.2021 klo 9.25: >> > On 2021-01-11 23:07, Sven Roederer wrote: >> >> Restore the status of the system-services after sysupgrade. >> >> Create a file with the status of all known services and keep it during >> >> upgrade. After upgrade run a uci-default script to restore every >> >> single service. >> > >> > Doing the restore in an uci-default script is to late, because then you >> > are already in the procd rcS helper "loop" and new added init links >> > won't get called. >> > >> > Also, have you thought about a backup/restore of the configuration? >> > The whole thing should/must work there as well. >> > >> > I would do the restore at the end of the pre-init stage. >> > >> > Martin >> >> My two cents to the discussion: >> >> * I assume that a small minority of users actually disable services, >> or at >> least disable several services, so defaulting to "lets by default >> always >> store status of all services" sounds like overkill. >> >> * Backuping always info that the dozens of standard services are >> enabled >> sounds unncessary. As services are installed by default as "enabled" >> to the >> image, it might make sense to store only info about "disabled" >> services and >> disable those when restoring the backup in sysupgrade. (Re-enabling >> the >> already enabled services is unnecessary.) That would also help to >> keep the >> amount of new backup info small. >> >> * make clear in sysupgrade script help texts and documentation that >> saving/restoring service status depends on "keep settings". If >> settings are >> not kept, the service status can not be copied in sysupugrade as there >> is no >> sysupgrade.tar.gz to be passed. (And naturally it would not even make >> sense >> to e.g. revert network settings to defaults but keep selected network >> services disabled. Easy to cause soft-bricks that way.) >> >> > Hannu, > > as long as no service was disabled with the "DISABLED_SERVICES" option > of the > imagebuilder the explicit "enable" is not required. > Indeed enabling a service that have been disabled during image creation > is a > bit edge-case, but I prefer here the "full featured" way. Not enabling > such a > service also breaks the user experience like the disable case. I totally agree. If you do this, you should cover all cases.
On 1/12/21 12:05, Sven Roederer wrote: > Am Dienstag, 12. Januar 2021, 13:01:45 CET schrieb Adrian Schmutzler: >>> Hello, >>> You don't have to run such complicated script. Just restore /etc/rc.d/ >>> directory after upgrade. >> Well, that depends on how we want to deal with new services. If I upgrade >> from 19.07.5 to master, I'd expect new services to be in their default >> initial state. Restoring "old" rc.d would have everything new disabled. >> That's counter-intuitive for me. > I'm also in favor of not touching new services, which have not been installed > before the upgrade. Any update on this? I'd be interested in this service since I often disable DHCP on testing devices.
diff --git a/package/base-files/Makefile b/package/base-files/Makefile index f18f221129..5a9bacba13 100644 --- a/package/base-files/Makefile +++ b/package/base-files/Makefile @@ -12,7 +12,7 @@ include $(INCLUDE_DIR)/version.mk include $(INCLUDE_DIR)/feeds.mk PKG_NAME:=base-files -PKG_RELEASE:=246 +PKG_RELEASE:=247 PKG_FLAGS:=nonshared PKG_FILE_DEPENDS:=$(PLATFORM_DIR)/ $(GENERIC_PLATFORM_DIR)/base-files/ diff --git a/package/base-files/files/etc/uci-defaults/14_restore-services-state b/package/base-files/files/etc/uci-defaults/14_restore-services-state new file mode 100644 index 0000000000..d0085f3f80 --- /dev/null +++ b/package/base-files/files/etc/uci-defaults/14_restore-services-state @@ -0,0 +1,3 @@ +SERVICE_STATUS=/etc/backup/service_status.txt + +[ -e ${SERVICE_STATUS} ] && . ${SERVICE_STATUS} diff --git a/package/base-files/files/sbin/sysupgrade b/package/base-files/files/sbin/sysupgrade index 50f4e08e44..068ec27e3d 100755 --- a/package/base-files/files/sbin/sysupgrade +++ b/package/base-files/files/sbin/sysupgrade @@ -15,6 +15,7 @@ export SAVE_OVERLAY_PATH= export SAVE_PARTITIONS=1 export SAVE_INSTALLED_PKGS=0 export SKIP_UNCHANGED=1 +export SAVE_SERVICE_STATUS=1 export CONF_IMAGE= export CONF_BACKUP_LIST=0 export CONF_BACKUP= @@ -35,6 +36,7 @@ while [ -n "$1" ]; do -c) export SAVE_OVERLAY=1 SAVE_OVERLAY_PATH=/etc;; -o) export SAVE_OVERLAY=1 SAVE_OVERLAY_PATH=/;; -p) export SAVE_PARTITIONS=0;; + -s) export SAVE_SERVICE_STATUS=0;; -k) export SAVE_INSTALLED_PKGS=1;; -U) export SKIP_UNCHANGED=0;; -b|--create-backup) export CONF_BACKUP="$2" NEED_IMAGE=1; shift;; @@ -57,13 +59,14 @@ export CONFFILES=/tmp/sysupgrade.conffiles export CONF_TAR=/tmp/sysupgrade.tgz export ETCBACKUP_DIR=/etc/backup export INSTALLED_PACKAGES=${ETCBACKUP_DIR}/installed_packages.txt +export SERVICE_STATUS=${ETCBACKUP_DIR}/service_status.txt IMAGE="$1" [ -z "$IMAGE" -a -z "$NEED_IMAGE" -a $CONF_BACKUP_LIST -eq 0 -o $HELP -gt 0 ] && { cat <<EOF Usage: $0 [<upgrade-option>...] <image file or URL> - $0 [-q] [-i] [-c] [-U] [-o] [-k] <backup-command> <file> + $0 [-q] [-i] [-c] [-U] [-o] [-s] [-k] <backup-command> <file> upgrade-option: -f <config> restore configuration from .tar.gz (file or url) @@ -75,6 +78,7 @@ upgrade-option: (this is a modification of Freifunk Berlin to unset -u) -n do not save configuration over reflash -p do not attempt to restore the partition table after flash. + -s do not attempt to restore the status of the services. -k include in backup a list of current installed packages at $INSTALLED_PACKAGES -T | --test @@ -231,8 +235,7 @@ do_save_conffiles() { run_hooks "$CONFFILES" $sysupgrade_init_conffiles ask_bool 0 "Edit config file list" && vi "$CONFFILES" - if [ "$SAVE_INSTALLED_PKGS" -eq 1 ]; then - echo "${INSTALLED_PACKAGES}" >> "$CONFFILES" + if [ "$SAVE_INSTALLED_PKGS" -eq 1 ] || [ "$SAVE_SERVICE_STATUS" -eq 1 ]; then mkdir -p "$ETCBACKUP_DIR" # Avoid touching filesystem on each backup RAMFS="$(mktemp -d -t sysupgrade.XXXXXX)" @@ -243,13 +246,28 @@ do_save_conffiles() { ask_bool 0 "Abort" && exit } - # Format: pkg-name<TAB>{rom,overlay,unkown} - # rom is used for pkgs in /rom, even if updated later - find /usr/lib/opkg/info -name "*.control" \( \ - \( -exec test -f /rom/{} \; -exec echo {} rom \; \) -o \ - \( -exec test -f /overlay/upper/{} \; -exec echo {} overlay \; \) -o \ - \( -exec echo {} unknown \; \) \ - \) | sed -e 's,.*/,,;s/\.control /\t/' > ${INSTALLED_PACKAGES} + if [ "$SAVE_INSTALLED_PKGS" -eq 1 ]; then + echo "${INSTALLED_PACKAGES}" >> "$CONFFILES" + # Format: pkg-name<TAB>{rom,overlay,unkown} + # rom is used for pkgs in /rom, even if updated later + find /usr/lib/opkg/info -name "*.control" \( \ + \( -exec test -f /rom/{} \; -exec echo {} rom \; \) -o \ + \( -exec test -f /overlay/upper/{} \; -exec echo {} overlay \; \) -o \ + \( -exec echo {} unknown \; \) \ + \) | sed -e 's,.*/,,;s/\.control /\t/' > ${INSTALLED_PACKAGES} + fi + + if [ "$SAVE_SERVICE_STATUS" -eq 1 ]; then + echo "${SERVICE_STATUS}" >> "$CONFFILES" + # Format: /etc/init.d/servicename {enable,disable} + rm -f ${SERVICE_STATUS} + for service in /etc/init.d/* ; do \ + ${service} enabled && \ + echo >> ${SERVICE_STATUS} "$service" "enable" || \ + echo >> ${SERVICE_STATUS} "$service" "disable" \ + ; \ + done + fi fi v "Saving config files..."
Restore the status of the system-services after sysupgrade. Create a file with the status of all known services and keep it during upgrade. After upgrade run a uci-default script to restore every single service. The list of the service status will be stored in etc/backup folder also and formated as: /etc/init.d/<service> {enable|disable} When upgrading with an generic image all system services are enabled by default which is usually not expected and can cause trouble. The default behavior can be flipped with the "-s" option of sysupgrade. Signed-off-by: Sven Roederer <devel-sven@geroedel.de> --- This v2 includes the feedback from the mailinglist: * entangles this feature from "storing the packagelist" * adds a option for disabling to sysupgrade * makes it enabled by default * includes the glue-script to restore the services after upgrade I was not able to test on a real system yet. package/base-files/Makefile | 2 +- .../uci-defaults/14_restore-services-state | 3 ++ package/base-files/files/sbin/sysupgrade | 38 ++++++++++++++----- 3 files changed, 32 insertions(+), 11 deletions(-) create mode 100644 package/base-files/files/etc/uci-defaults/14_restore-services-state