diff mbox series

[v2] scripts: create kernel configuration upgrade script

Message ID 414511fcb30b883565ae724aaf17141b104164ff.1707337693.git.ehem+openwrt@m5p.com
State Superseded
Delegated to: Petr Štetiar
Headers show
Series [v2] scripts: create kernel configuration upgrade script | expand

Commit Message

Elliott Mitchell Feb. 7, 2024, 1:16 a.m. UTC
Create a script for automating kernel version changes.  This
generates a pair of commits which cause history to remain attached to
all versioned configuration files.

Crucially this makes `git blame` work without needing
--find-copies-harder, which is too slow for routine use.  This also
updates *everything*, which greatly simplifies rebasing patches
which effect multiple devices.

Signed-off-by: Elliott Mitchell <ehem+openwrt@m5p.com>
---
v2:
Major tweaking.  No longer try to do `git merge --ff-only <hash>`,
but instead advise user to do so.

Add usage message.

Add statement about strategy.

Adjust commit messages.  Add advice to run `git bisect --skip` if
someone ends up on that commit.
---
 scripts/kernel_upgrade.pl | 261 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 261 insertions(+)
 create mode 100755 scripts/kernel_upgrade.pl

Comments

Elliott Mitchell Feb. 13, 2024, 9:54 p.m. UTC | #1
On Tue, Feb 06, 2024 at 05:16:41PM -0800, Elliott Mitchell wrote:
> Create a script for automating kernel version changes.  This
> generates a pair of commits which cause history to remain attached to
> all versioned configuration files.
> 
> Crucially this makes `git blame` work without needing
> --find-copies-harder, which is too slow for routine use.  This also
> updates *everything*, which greatly simplifies rebasing patches
> which effect multiple devices.

Seems I goofed and should have Cc'd Christian Marangi when sending this.

The script seems to work fine.  One remaining issue is whether any
adjustment to the commit messages on the generated commits is desired.
I'm unsure they need "Signed-off-by" since they're tool-generated.  I
also wonder whether the script should claim to be the author.

The one difference is this does all kernel configs in one pass.  This
seems superior as it makes rebasing proposed patches *much* simpler.
Specifically there is far less update work without continuously appearing
and disappearing configs.  This also means only a single commit per year
which troubles bisecting.
Rafał Miłecki Feb. 14, 2024, 9:03 a.m. UTC | #2
On 7.02.2024 02:16, Elliott Mitchell wrote:
> diff --git a/scripts/kernel_upgrade.pl b/scripts/kernel_upgrade.pl
> new file mode 100755
> index 0000000000..b9fe5882a3
> --- /dev/null
> +++ b/scripts/kernel_upgrade.pl
> @@ -0,0 +1,261 @@
> +#!/usr/bin/env perl
> +#########################################################################
> +# Copyright (C) 2024 Elliott Mitchell <ehem+openwrt@m5p.com>		#
> +#									#
> +# This is free software, licensed under the GNU General Public License	#
> +# v3.  See /LICENSE for more information.				#
> +#########################################################################

OpenWrt is GPL 2.0 licensed. This this is GPL 3.0 only.

It seems to be independent script (right now) but:
1. What if one day we make it share some common code?
2. What about using it to produce GPL 2.0 code?

This may be more or less obvious to some of us but should we have a
lawyer opinion to be sure?

Personally I'm against such mixing licenses in our tools (script).

Also: please use SPDX header to simplify licensing info.
Elliott Mitchell Feb. 14, 2024, 10:40 p.m. UTC | #3
On Wed, Feb 14, 2024 at 10:03:00AM +0100, Rafał Miłecki wrote:
> On 7.02.2024 02:16, Elliott Mitchell wrote:
> > diff --git a/scripts/kernel_upgrade.pl b/scripts/kernel_upgrade.pl
> > new file mode 100755
> > index 0000000000..b9fe5882a3
> > --- /dev/null
> > +++ b/scripts/kernel_upgrade.pl
> > @@ -0,0 +1,261 @@
> > +#!/usr/bin/env perl
> > +#########################################################################
> > +# Copyright (C) 2024 Elliott Mitchell <ehem+openwrt@m5p.com>		#
> > +#									#
> > +# This is free software, licensed under the GNU General Public License	#
> > +# v3.  See /LICENSE for more information.				#
> > +#########################################################################
> 
> OpenWrt is GPL 2.0 licensed. This this is GPL 3.0 only.
> 
> It seems to be independent script (right now) but:
> 1. What if one day we make it share some common code?

Then you have GPLv3.  Question is whether the common portion comes from
this script?  If yes, then we consider LGPL versus everything being
GPLv3.

This seems pretty unlikely as this is a rather specific task.
`git fast-import` isn't likely to be useful in very many places.  Also,
`git` isn't something I would expect to see on a 128MB RAM device.

> 2. What about using it to produce GPL 2.0 code?

This is very well-understood.  GPL applies to source code and the tools.
GIMP is routinely used to produce non-GPL images.  `git` is now used by
MS for handling their source code, most of it remains non-GPL and no
lawsuits have resulted.  GCC is routinely used to build non-GPL software.

Anything not directly linked in is completely uneffected.  In particular
while it does manipulate git's data, none of that is linked into the
script and therefore has no license effects.  This won't have any
particularly notable effects.
diff mbox series

Patch

diff --git a/scripts/kernel_upgrade.pl b/scripts/kernel_upgrade.pl
new file mode 100755
index 0000000000..b9fe5882a3
--- /dev/null
+++ b/scripts/kernel_upgrade.pl
@@ -0,0 +1,261 @@ 
+#!/usr/bin/env perl
+#########################################################################
+# Copyright (C) 2024 Elliott Mitchell <ehem+openwrt@m5p.com>		#
+#									#
+# This is free software, licensed under the GNU General Public License	#
+# v3.  See /LICENSE for more information.				#
+#########################################################################
+
+use warnings;
+use strict;
+
+use feature 'state';
+
+use Digest::SHA qw(sha1_hex);
+
+#
+# Credit to the person who knew about this workable solution:
+# <https://lists.openwrt.org/pipermail/openwrt-devel/2023-October/041672.html>
+#
+# Which originated from:
+# <https://devblogs.microsoft.com/oldnewthing/20190919-00/?p=102904>
+#
+# Problem is copying a file in git causes the new file to be created
+# without any history.  Files can move around without losing their
+# history, but that only leaves the history on the new location.
+#
+# As such this can be solved with two commits.  The first commit moves
+# files from their old name to their new name.  The second merges the
+# original commit with the rename commit.  The merge commit then has
+# files in both locations with the full history.
+#
+#
+# Note, git handles discarded data by garbage collection.  When doing
+# development on this script, beware this script is an excellent
+# garbage generator.  Frequent use of `git gc` and `git prune` may be
+# needed.
+#
+
+
+sub gethead()
+{
+	open(my $fd, '-|', 'git', 'rev-parse', 'HEAD');
+	$_=<$fd>;
+	chop;
+	return $_;
+}
+
+sub getlist($$)
+{
+	my ($target, $from)=@_;
+	my $ret=[];
+
+	local $/="\0";
+	open(my $fd, '-| :raw :bytes', 'git', 'ls-tree', '-trz', '--full-name',
+'--name-only', 'HEAD', '--', $target)||die("failed to read git tree");
+
+	while(<$fd>) {
+		chop($_);
+		push(@$ret, substr($_, 0, -length($from)))
+if(substr($_, -length($from)) eq $from);
+	}
+
+	@$ret=sort({length($b)-length($a)} @$ret);
+
+	return $ret;
+}
+
+# start of interface to git fast-import
+my $gitpid;
+my $gitfds=[undef, undef];
+
+sub startgit()
+{
+	my $child=[];
+	(pipe($child->[0], $gitfds->[0])&&pipe($gitfds->[1], $child->[1])) ||
+die("pipe() failed");
+	binmode($gitfds->[0]);
+	binmode($gitfds->[1]);
+
+	$gitpid=fork();
+	if($gitpid==0) {
+		close($gitfds->[0]);
+		close($gitfds->[1]);
+
+		open(STDIN, '<&', $child->[0]);
+		close($child->[0]);
+
+		open(STDOUT, '>&', $child->[1]);
+		close($child->[1]);
+
+		exec('git', 'fast-import', '--done');
+		die('exec() of git failed');
+	} elsif(!$gitpid) {
+		die('fork() failed');
+	}
+	close($child->[0]);
+	close($child->[1]);
+	$gitfds->[0]->autoflush(1);
+
+}
+
+sub gitsend
+{
+	return print({$gitfds->[0]} @_);
+}
+
+sub gitrecv()
+{
+	return $_=readline(${$gitfds}[1]);
+}
+
+sub gitls($$)
+{
+	my ($commit, $name)=@_;
+	local $/="\n";
+
+	$commit.=' ' if($commit);
+	gitsend("ls $commit$name\n");
+	gitrecv();
+
+	die('git ls failed') unless(/^([0-8]+)\s+[a-z]+\s+([0-9a-z]+)\s+.+$/);
+
+	return [$1, $2];
+}
+
+sub gitcommit($$$)
+{
+	my ($dest, $message, $mark)=@_;
+	local $/="\n";
+	local $|=1;
+	state $author=undef;
+	unless($author) {
+		$author=['', ''];
+		open(my $user, '-|', 'git', 'config', '--get', 'user.name');
+		while(<$user>) {
+			chomp;
+			$author->[0].=$_;
+		}
+		$author->[0]=[split(/,/, [getpwuid($<)]->[6])]->[0]
+unless($author->[0]);
+
+		open(my $email, '-|', 'git', 'config', '--get', 'user.email');
+		while(<$email>) {
+			chomp;
+			$author->[1].=$_;
+		}
+		$author->[1]='anonymous@example.com' unless($author->[1]);
+
+		$author=$author->[0].' <'.$author->[1].'>';
+	}
+
+	$_=sha1_hex(time());
+	gitsend("commit $_\n");
+	gitsend("mark $mark\n");
+	gitsend("committer $author ".time()." +0000\n");
+
+	$_=length($message);
+	gitsend("data $_\n");
+	gitsend($message);
+	gitsend("from $dest\n");
+}
+
+sub gitdone()
+{
+	local $/="\n";
+	gitsend("done\n");
+	close($gitfds->[0]);
+	$gitfds->[0]=undef;
+
+	0 while(waitpid($gitpid, 0)!=$gitpid);
+	close($gitfds->[1]);
+	$gitfds->[1]=undef;
+
+	print(STDERR <<~"__GIT_STATUS__") if($?);
+	WARNING: git returned error exit status: $?
+
+	This likely means `git gc` needs to be run, but the return codes of
+	`git fast-import` are undocumented.
+
+	__GIT_STATUS__
+}
+# end of interface to git fast-import
+
+
+die(<<"__USAGE__") if(@ARGV!=2);
+Usage: $0 <old-version> <new-version>
+
+Copies all kernel configuration files and patches from the old version
+to the new version.  Git history is preserved on the copies by using a
+move/merge strategy.  Must be run while somewhere inside the git
+repository directory, but it does not matter where.
+__USAGE__
+
+my ($from, $to)=@ARGV;
+
+
+my $target='target/linux';
+
+my $start=gethead();
+
+my $list=getlist($target, $from);
+
+die("no files matching \"$from\" found") unless(@$list);
+
+
+startgit();
+
+gitcommit($start, <<"__TMP__", ':1');
+kernel: add configs and patches for $to
+
+This is a special tool-generated commit.
+
+Copy configuration and patches from $from to $to.
+
+If you see this commit during a `git bisect` session, the recommended
+course of action is to run `git bisect --skip`.
+__TMP__
+
+foreach my $name (@$list) {
+	my $new=gitls($start, "$name$from");
+	gitsend("M $new->[0] $new->[1] $name$to\n");
+	gitsend("D $name$from\n");
+}
+gitsend("\n");
+
+
+gitcommit(':1', <<"__TMP__", ':2');
+kernel: finish update from $from to $to
+
+This is a special tool-generated commit.
+
+Merge the add commit into HEAD to create all files with full history.
+__TMP__
+
+gitsend("merge $start\n");
+
+foreach my $name (@$list) {
+	my $new=gitls($start, "$name$from");
+	gitsend("M $new->[0] $new->[1] $name$from\n");
+}
+gitsend("\n");
+
+
+gitsend("get-mark :2\n");
+my $result=gitrecv();
+chomp($result);
+
+gitdone();
+
+print(<<"__END__");
+Result is commit $result.
+
+Depending on the setup of your development environment, you now likely
+want to run one of two commands:
+
+	`git merge --ff-only $result`
+Or:
+	`git branch linux-$to $result`
+__END__
+
+exit(0);