| Message ID | 1421895307-28550-1-git-send-email-mathieu@qca.qualcomm.com |
|---|---|
| State | Accepted |
| Headers | show |
On 22/01/2015 03:55, Mathieu Olivari wrote: > Currently, replacing a package available in openwrt.git requires > modifications in openwrt.git, or requires duplicating the package > in a feed but with a different name, which causes all kind of > problems related to dependencies (all packages selecting it would > have to be modified accordingly to select the new package). > > With this change, if a package with the same name is present both > in feeds/ and package/ folders, the one in feeds/ can override the > one in package/, both in the menuconfig and during the build, by > passing the "-f" option to "./scripts/feeds install" > > This mechanism is particularly useful for vendor tree, or in > general for application which needs to replace one particular > package which exists within openwrt.git by a custom/newer version. > > Signed-off-by: Mathieu Olivari <mathieu@qca.qualcomm.com> Hi Mathieu, this looks much better. i think we should also add some info in /etc/openwrt_release in the field DISTRIB_TAINTS='no-all' indicating that the build uses out of tree vendor packages. i will merge this patch as is, but could you put the openwrt_release patch onto your todo list and send it in the nearish future please John > --- v2: *fix an issue when trying to override packages which > include a dependency in the core packages. (ex: lua depends on > liblua, which exists in the core, which ended-up in the package not > being overridden properly). *address John's concerns about core > OpenWrt packages being overriden without notice and control. > Override is now disable by default, and "-f" needs to be passed to > "./scripts/feeds install" to allow it. --- include/scan.awk | 17 > +++++++++++++++++ include/scan.mk | 2 +- scripts/feeds | > 25 ++++++++++++++++++------- 3 files changed, 36 insertions(+), 8 > deletions(-) create mode 100644 include/scan.awk > > diff --git a/include/scan.awk b/include/scan.awk new file mode > 100644 index 0000000..39b2977 --- /dev/null +++ b/include/scan.awk > @@ -0,0 +1,17 @@ +BEGIN { FS="/" } +$1 ~ /^feeds/ { FEEDS[$NF]=$0 > } +$1 !~ /^feeds/ { PKGS[$NF]=$0 } +END { + # Filter-out OpenWrt > packages which have a feeds equivalent + for (pkg in PKGS) + if > (pkg in FEEDS) + delete PKGS[pkg] + n = asort(PKGS) + for (i=1; i > <= n; i++) { + print PKGS[i] + } + n = asort(FEEDS) + for (i=1; i > <= n; i++){ + print FEEDS[i] + } +} diff --git a/include/scan.mk > b/include/scan.mk index 0998333..138707d 100644 --- > a/include/scan.mk +++ b/include/scan.mk @@ -43,7 +43,7 @@ endef > > $(FILELIST): rm -f $(TMP_DIR)/info/.files-$(SCAN_TARGET)-* - $(call > FIND_L, $(SCAN_DIR)) $(SCAN_EXTRA) -mindepth 1 $(if > $(SCAN_DEPTH),-maxdepth $(SCAN_DEPTH)) -name Makefile | xargs grep > -HE 'call (Build/DefaultTargets|Build(Package|Target)|.+Package)' | > sed -e 's#^$(SCAN_DIR)/##' -e 's#/Makefile:.*##' | uniq > $@ + > $(call FIND_L, $(SCAN_DIR)) $(SCAN_EXTRA) -mindepth 1 $(if > $(SCAN_DEPTH),-maxdepth $(SCAN_DEPTH)) -name Makefile | xargs grep > -HE 'call (Build/DefaultTargets|Build(Package|Target)|.+Package)' | > sed -e 's#^$(SCAN_DIR)/##' -e 's#/Makefile:.*##' | uniq | awk -f > include/scan.awk > $@ > > $(TMP_DIR)/info/.files-$(SCAN_TARGET).mk: $(FILELIST) ( \ diff > --git a/scripts/feeds b/scripts/feeds index 31ad544..6c03375 > 100755 --- a/scripts/feeds +++ b/scripts/feeds @@ -358,6 +358,7 @@ > sub is_core_package($) { sub install_package { my $feed = shift; my > $name = shift; + my $force = shift; my $ret = 0; > > $feed = lookup_package($feed, $name); @@ -383,13 +384,22 @@ sub > install_package { $src or $src = $name; > > # previously installed packages set the runtime package - # newly > installed packages set the source package - $installed{$src} and > return 0; + # newly installed packages set the source package to 1 > + $installed{$src} and $installed{$src} == 1 and return 0; + + # > we'll trigger the override only with the 3 conditions below: + # - > override is allowed by command line (-f) + # - a package with the > same src exists in the core packages list + # - the package > previously installed is not from a feed + my $override = 1 if > ($force and is_core_package($src) and !$installed{$src}->{feed}); > > # check previously installed packages - $installed{$name} and > return 0; + $installed{$name} and !$override and return 0; > $installed{$src} = 1; - warn "Installing package '$src'\n"; + + > $override == 1 + and warn "Overriding package '$src'\n" + or warn > "Installing package '$src'\n"; > > $install_method{$type} or do { warn "Unknown installation method: > '$type'\n"; @@ -436,7 +446,7 @@ sub install { my $feed; my $ret = > 0; > > - getopts('ap:d:h', \%opts); + getopts('ap:d:fh', \%opts); > > if ($opts{h}) { usage(); @@ -462,7 +472,7 @@ sub install { my $p = > $feed_package->{$name}; next if $p->{vdepends}; if( $p->{name} ) { > - install_package($feed, $p->{name}) == 0 or $ret = 1; + > install_package($feed, $p->{name}, exists($opts{f})) == 0 or $ret = > 1; get_feed($f->[1]); } } @@ -470,7 +480,7 @@ sub install { } } > else { while ($name = shift @ARGV) { - install_package($feed, > $name) == 0 or $ret = 1; + install_package($feed, $name, > exists($opts{f})) == 0 or $ret = 1; } } > > @@ -638,6 +648,7 @@ Commands: -a : Install all packages > from all feeds or from the specified feed using the -p option. -p > <feedname>: Prefer this feed when installing packages. -d <y|m|n>: > Set default for newly installed packages. + -f : > Install will be forced even if the package exists in core OpenWrt > (override) > > search [options] <substring>: Search for a package Options: >
On 22/01/2015 03:55, Mathieu Olivari wrote: > Currently, replacing a package available in openwrt.git requires > modifications in openwrt.git, or requires duplicating the package in a > feed but with a different name, which causes all kind of problems > related to dependencies (all packages selecting it would have to be > modified accordingly to select the new package). > > With this change, if a package with the same name is present both in > feeds/ and package/ folders, the one in feeds/ can override the one > in package/, both in the menuconfig and during the build, by passing the > "-f" option to "./scripts/feeds install" > > This mechanism is particularly useful for vendor tree, or in general for > application which needs to replace one particular package which exists > within openwrt.git by a custom/newer version. > Hi, ./scripts/feeds install -p -a alljoyn Use of uninitialized value $override in numeric eq (==) at ./scripts/feeds line 400. Installing package 'alljoyn' i will revert the patch now and await a fixed version John > Signed-off-by: Mathieu Olivari <mathieu@qca.qualcomm.com> > --- > v2: > *fix an issue when trying to override packages which include a dependency in the > core packages. (ex: lua depends on liblua, which exists in the core, which > ended-up in the package not being overridden properly). > *address John's concerns about core OpenWrt packages being overriden without > notice and control. Override is now disable by default, and "-f" needs to be > passed to "./scripts/feeds install" to allow it. > --- > include/scan.awk | 17 +++++++++++++++++ > include/scan.mk | 2 +- > scripts/feeds | 25 ++++++++++++++++++------- > 3 files changed, 36 insertions(+), 8 deletions(-) > create mode 100644 include/scan.awk > > diff --git a/include/scan.awk b/include/scan.awk > new file mode 100644 > index 0000000..39b2977 > --- /dev/null > +++ b/include/scan.awk > @@ -0,0 +1,17 @@ > +BEGIN { FS="/" } > +$1 ~ /^feeds/ { FEEDS[$NF]=$0 } > +$1 !~ /^feeds/ { PKGS[$NF]=$0 } > +END { > + # Filter-out OpenWrt packages which have a feeds equivalent > + for (pkg in PKGS) > + if (pkg in FEEDS) > + delete PKGS[pkg] > + n = asort(PKGS) > + for (i=1; i <= n; i++) { > + print PKGS[i] > + } > + n = asort(FEEDS) > + for (i=1; i <= n; i++){ > + print FEEDS[i] > + } > +} > diff --git a/include/scan.mk b/include/scan.mk > index 0998333..138707d 100644 > --- a/include/scan.mk > +++ b/include/scan.mk > @@ -43,7 +43,7 @@ endef > > $(FILELIST): > rm -f $(TMP_DIR)/info/.files-$(SCAN_TARGET)-* > - $(call FIND_L, $(SCAN_DIR)) $(SCAN_EXTRA) -mindepth 1 $(if $(SCAN_DEPTH),-maxdepth $(SCAN_DEPTH)) -name Makefile | xargs grep -HE 'call (Build/DefaultTargets|Build(Package|Target)|.+Package)' | sed -e 's#^$(SCAN_DIR)/##' -e 's#/Makefile:.*##' | uniq > $@ > + $(call FIND_L, $(SCAN_DIR)) $(SCAN_EXTRA) -mindepth 1 $(if $(SCAN_DEPTH),-maxdepth $(SCAN_DEPTH)) -name Makefile | xargs grep -HE 'call (Build/DefaultTargets|Build(Package|Target)|.+Package)' | sed -e 's#^$(SCAN_DIR)/##' -e 's#/Makefile:.*##' | uniq | awk -f include/scan.awk > $@ > > $(TMP_DIR)/info/.files-$(SCAN_TARGET).mk: $(FILELIST) > ( \ > diff --git a/scripts/feeds b/scripts/feeds > index 31ad544..6c03375 100755 > --- a/scripts/feeds > +++ b/scripts/feeds > @@ -358,6 +358,7 @@ sub is_core_package($) { > sub install_package { > my $feed = shift; > my $name = shift; > + my $force = shift; > my $ret = 0; > > $feed = lookup_package($feed, $name); > @@ -383,13 +384,22 @@ sub install_package { > $src or $src = $name; > > # previously installed packages set the runtime package > - # newly installed packages set the source package > - $installed{$src} and return 0; > + # newly installed packages set the source package to 1 > + $installed{$src} and $installed{$src} == 1 and return 0; > + > + # we'll trigger the override only with the 3 conditions below: > + # - override is allowed by command line (-f) > + # - a package with the same src exists in the core packages list > + # - the package previously installed is not from a feed > + my $override = 1 if ($force and is_core_package($src) and !$installed{$src}->{feed}); > > # check previously installed packages > - $installed{$name} and return 0; > + $installed{$name} and !$override and return 0; > $installed{$src} = 1; > - warn "Installing package '$src'\n"; > + > + $override == 1 > + and warn "Overriding package '$src'\n" > + or warn "Installing package '$src'\n"; > > $install_method{$type} or do { > warn "Unknown installation method: '$type'\n"; > @@ -436,7 +446,7 @@ sub install { > my $feed; > my $ret = 0; > > - getopts('ap:d:h', \%opts); > + getopts('ap:d:fh', \%opts); > > if ($opts{h}) { > usage(); > @@ -462,7 +472,7 @@ sub install { > my $p = $feed_package->{$name}; > next if $p->{vdepends}; > if( $p->{name} ) { > - install_package($feed, $p->{name}) == 0 or $ret = 1; > + install_package($feed, $p->{name}, exists($opts{f})) == 0 or $ret = 1; > get_feed($f->[1]); > } > } > @@ -470,7 +480,7 @@ sub install { > } > } else { > while ($name = shift @ARGV) { > - install_package($feed, $name) == 0 or $ret = 1; > + install_package($feed, $name, exists($opts{f})) == 0 or $ret = 1; > } > } > > @@ -638,6 +648,7 @@ Commands: > -a : Install all packages from all feeds or from the specified feed using the -p option. > -p <feedname>: Prefer this feed when installing packages. > -d <y|m|n>: Set default for newly installed packages. > + -f : Install will be forced even if the package exists in core OpenWrt (override) > > search [options] <substring>: Search for a package > Options: >
diff --git a/include/scan.awk b/include/scan.awk new file mode 100644 index 0000000..39b2977 --- /dev/null +++ b/include/scan.awk @@ -0,0 +1,17 @@ +BEGIN { FS="/" } +$1 ~ /^feeds/ { FEEDS[$NF]=$0 } +$1 !~ /^feeds/ { PKGS[$NF]=$0 } +END { + # Filter-out OpenWrt packages which have a feeds equivalent + for (pkg in PKGS) + if (pkg in FEEDS) + delete PKGS[pkg] + n = asort(PKGS) + for (i=1; i <= n; i++) { + print PKGS[i] + } + n = asort(FEEDS) + for (i=1; i <= n; i++){ + print FEEDS[i] + } +} diff --git a/include/scan.mk b/include/scan.mk index 0998333..138707d 100644 --- a/include/scan.mk +++ b/include/scan.mk @@ -43,7 +43,7 @@ endef $(FILELIST): rm -f $(TMP_DIR)/info/.files-$(SCAN_TARGET)-* - $(call FIND_L, $(SCAN_DIR)) $(SCAN_EXTRA) -mindepth 1 $(if $(SCAN_DEPTH),-maxdepth $(SCAN_DEPTH)) -name Makefile | xargs grep -HE 'call (Build/DefaultTargets|Build(Package|Target)|.+Package)' | sed -e 's#^$(SCAN_DIR)/##' -e 's#/Makefile:.*##' | uniq > $@ + $(call FIND_L, $(SCAN_DIR)) $(SCAN_EXTRA) -mindepth 1 $(if $(SCAN_DEPTH),-maxdepth $(SCAN_DEPTH)) -name Makefile | xargs grep -HE 'call (Build/DefaultTargets|Build(Package|Target)|.+Package)' | sed -e 's#^$(SCAN_DIR)/##' -e 's#/Makefile:.*##' | uniq | awk -f include/scan.awk > $@ $(TMP_DIR)/info/.files-$(SCAN_TARGET).mk: $(FILELIST) ( \ diff --git a/scripts/feeds b/scripts/feeds index 31ad544..6c03375 100755 --- a/scripts/feeds +++ b/scripts/feeds @@ -358,6 +358,7 @@ sub is_core_package($) { sub install_package { my $feed = shift; my $name = shift; + my $force = shift; my $ret = 0; $feed = lookup_package($feed, $name); @@ -383,13 +384,22 @@ sub install_package { $src or $src = $name; # previously installed packages set the runtime package - # newly installed packages set the source package - $installed{$src} and return 0; + # newly installed packages set the source package to 1 + $installed{$src} and $installed{$src} == 1 and return 0; + + # we'll trigger the override only with the 3 conditions below: + # - override is allowed by command line (-f) + # - a package with the same src exists in the core packages list + # - the package previously installed is not from a feed + my $override = 1 if ($force and is_core_package($src) and !$installed{$src}->{feed}); # check previously installed packages - $installed{$name} and return 0; + $installed{$name} and !$override and return 0; $installed{$src} = 1; - warn "Installing package '$src'\n"; + + $override == 1 + and warn "Overriding package '$src'\n" + or warn "Installing package '$src'\n"; $install_method{$type} or do { warn "Unknown installation method: '$type'\n"; @@ -436,7 +446,7 @@ sub install { my $feed; my $ret = 0; - getopts('ap:d:h', \%opts); + getopts('ap:d:fh', \%opts); if ($opts{h}) { usage(); @@ -462,7 +472,7 @@ sub install { my $p = $feed_package->{$name}; next if $p->{vdepends}; if( $p->{name} ) { - install_package($feed, $p->{name}) == 0 or $ret = 1; + install_package($feed, $p->{name}, exists($opts{f})) == 0 or $ret = 1; get_feed($f->[1]); } } @@ -470,7 +480,7 @@ sub install { } } else { while ($name = shift @ARGV) { - install_package($feed, $name) == 0 or $ret = 1; + install_package($feed, $name, exists($opts{f})) == 0 or $ret = 1; } } @@ -638,6 +648,7 @@ Commands: -a : Install all packages from all feeds or from the specified feed using the -p option. -p <feedname>: Prefer this feed when installing packages. -d <y|m|n>: Set default for newly installed packages. + -f : Install will be forced even if the package exists in core OpenWrt (override) search [options] <substring>: Search for a package Options:
Currently, replacing a package available in openwrt.git requires modifications in openwrt.git, or requires duplicating the package in a feed but with a different name, which causes all kind of problems related to dependencies (all packages selecting it would have to be modified accordingly to select the new package). With this change, if a package with the same name is present both in feeds/ and package/ folders, the one in feeds/ can override the one in package/, both in the menuconfig and during the build, by passing the "-f" option to "./scripts/feeds install" This mechanism is particularly useful for vendor tree, or in general for application which needs to replace one particular package which exists within openwrt.git by a custom/newer version. Signed-off-by: Mathieu Olivari <mathieu@qca.qualcomm.com> --- v2: *fix an issue when trying to override packages which include a dependency in the core packages. (ex: lua depends on liblua, which exists in the core, which ended-up in the package not being overridden properly). *address John's concerns about core OpenWrt packages being overriden without notice and control. Override is now disable by default, and "-f" needs to be passed to "./scripts/feeds install" to allow it. --- include/scan.awk | 17 +++++++++++++++++ include/scan.mk | 2 +- scripts/feeds | 25 ++++++++++++++++++------- 3 files changed, 36 insertions(+), 8 deletions(-) create mode 100644 include/scan.awk