diff mbox series

[U-Boot] scripts/setlocalversion: Improve -dirty check with git-status --no-optional-locks

Message ID 20200825112950.6993-1-rasmus.villemoes@prevas.dk
State Changes Requested
Delegated to: Tom Rini
Headers show
Series [U-Boot] scripts/setlocalversion: Improve -dirty check with git-status --no-optional-locks | expand

Commit Message

Rasmus Villemoes Aug. 25, 2020, 11:29 a.m. UTC
From: Brian Norris <briannorris@chromium.org>

[linux commit ff64dd4857303dd5550faed9fd598ac90f0f2238]

git-diff-index does not refresh the index for you, so using it for a
"-dirty" check can give misleading results. Commit 6147b1cf19651
("scripts/setlocalversion: git: Make -dirty check more robust") tried to
fix this by switching to git-status, but it overlooked the fact that
git-status also writes to the .git directory of the source tree, which
is definitely not kosher for an out-of-tree (O=) build. That is getting
reverted.

Fortunately, git-status now supports avoiding writing to the index via
the --no-optional-locks flag, as of git 2.14. It still calculates an
up-to-date index, but it avoids writing it out to the .git directory.

So, let's retry the solution from commit 6147b1cf19651 using this new
flag first, and if it fails, we assume this is an older version of git
and just use the old git-diff-index method.

It's hairy to get the 'grep -vq' (inverted matching) correct by stashing
the output of git-status (you have to be careful about the difference
betwen "empty stdin" and "blank line on stdin"), so just pipe the output
directly to grep and use a regex that's good enough for both the
git-status and git-diff-index version.

Cc: Christian Kujau <lists@nerdbynature.de>
Cc: Guenter Roeck <linux@roeck-us.net>
Suggested-by: Alexander Kapshuk <alexander.kapshuk@gmail.com>
Signed-off-by: Brian Norris <briannorris@chromium.org>
Tested-by: Genki Sky <sky@genki.is>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
---
This fixes real problems when building U-Boot via Yocto, since
Yocto creates hard-links to certain files in the source
repository, causing the "git diff-index" method to report -dirty,
even if no file contents are actually changed. See e.g.

https://lists.openembedded.org/g/openembedded-core/message/137702

 scripts/setlocalversion | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

Comments

Tom Rini Aug. 25, 2020, 12:56 p.m. UTC | #1
On Tue, Aug 25, 2020 at 01:29:50PM +0200, Rasmus Villemoes wrote:

> From: Brian Norris <briannorris@chromium.org>
> 
> [linux commit ff64dd4857303dd5550faed9fd598ac90f0f2238]
> 
> git-diff-index does not refresh the index for you, so using it for a
> "-dirty" check can give misleading results. Commit 6147b1cf19651
> ("scripts/setlocalversion: git: Make -dirty check more robust") tried to
> fix this by switching to git-status, but it overlooked the fact that
> git-status also writes to the .git directory of the source tree, which
> is definitely not kosher for an out-of-tree (O=) build. That is getting
> reverted.
> 
> Fortunately, git-status now supports avoiding writing to the index via
> the --no-optional-locks flag, as of git 2.14. It still calculates an
> up-to-date index, but it avoids writing it out to the .git directory.
> 
> So, let's retry the solution from commit 6147b1cf19651 using this new
> flag first, and if it fails, we assume this is an older version of git
> and just use the old git-diff-index method.
> 
> It's hairy to get the 'grep -vq' (inverted matching) correct by stashing
> the output of git-status (you have to be careful about the difference
> betwen "empty stdin" and "blank line on stdin"), so just pipe the output
> directly to grep and use a regex that's good enough for both the
> git-status and git-diff-index version.
> 
> Cc: Christian Kujau <lists@nerdbynature.de>
> Cc: Guenter Roeck <linux@roeck-us.net>
> Suggested-by: Alexander Kapshuk <alexander.kapshuk@gmail.com>
> Signed-off-by: Brian Norris <briannorris@chromium.org>
> Tested-by: Genki Sky <sky@genki.is>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
> ---
> This fixes real problems when building U-Boot via Yocto, since
> Yocto creates hard-links to certain files in the source
> repository, causing the "git diff-index" method to report -dirty,
> even if no file contents are actually changed. See e.g.
> 
> https://lists.openembedded.org/g/openembedded-core/message/137702
> 
>  scripts/setlocalversion | 12 ++++++++++--
>  1 file changed, 10 insertions(+), 2 deletions(-)

It looks like we have one local change to setlocalversion since our sync
from v3.16.  Can you please re-sync us to the v5.8 release?  Thanks!
Rasmus Villemoes Aug. 25, 2020, 2:53 p.m. UTC | #2
On 25/08/2020 14.56, Tom Rini wrote:
> On Tue, Aug 25, 2020 at 01:29:50PM +0200, Rasmus Villemoes wrote:
> 
>> From: Brian Norris <briannorris@chromium.org>
>>
>> [linux commit ff64dd4857303dd5550faed9fd598ac90f0f2238]
>>

>>  scripts/setlocalversion | 12 ++++++++++--
>>  1 file changed, 10 insertions(+), 2 deletions(-)
> 
> It looks like we have one local change to setlocalversion since our sync
> from v3.16.  Can you please re-sync us to the v5.8 release?  Thanks!
> 

Hmm, I suppose I could, but it's not really clear whether we'd still
need to apply that fix (81630a3b38c2, scripts: setlocalversion: safely
extract variables from auto.conf using awk), or if that has been
obsoleted by your a356e7a86b83 (spl: Kconfig: Escape '$(ARCH)' in
LDSCRIPT entries).

Rasmus
Tom Rini Aug. 25, 2020, 2:57 p.m. UTC | #3
On Tue, Aug 25, 2020 at 04:53:36PM +0200, Rasmus Villemoes wrote:
> On 25/08/2020 14.56, Tom Rini wrote:
> > On Tue, Aug 25, 2020 at 01:29:50PM +0200, Rasmus Villemoes wrote:
> > 
> >> From: Brian Norris <briannorris@chromium.org>
> >>
> >> [linux commit ff64dd4857303dd5550faed9fd598ac90f0f2238]
> >>
> 
> >>  scripts/setlocalversion | 12 ++++++++++--
> >>  1 file changed, 10 insertions(+), 2 deletions(-)
> > 
> > It looks like we have one local change to setlocalversion since our sync
> > from v3.16.  Can you please re-sync us to the v5.8 release?  Thanks!
> > 
> 
> Hmm, I suppose I could, but it's not really clear whether we'd still
> need to apply that fix (81630a3b38c2, scripts: setlocalversion: safely
> extract variables from auto.conf using awk), or if that has been
> obsoleted by your a356e7a86b83 (spl: Kconfig: Escape '$(ARCH)' in
> LDSCRIPT entries).

Ah,  it's entirely likely we could just use setlocalversion as-is.
diff mbox series

Patch

diff --git a/scripts/setlocalversion b/scripts/setlocalversion
index 8564bedd1a..55f8ace2ee 100755
--- a/scripts/setlocalversion
+++ b/scripts/setlocalversion
@@ -72,8 +72,16 @@  scm_version()
 			printf -- '-svn%s' "`git svn find-rev $head`"
 		fi
 
-		# Check for uncommitted changes
-		if git diff-index --name-only HEAD | grep -qv "^scripts/package"; then
+		# Check for uncommitted changes.
+		# First, with git-status, but --no-optional-locks is only
+		# supported in git >= 2.14, so fall back to git-diff-index if
+		# it fails. Note that git-diff-index does not refresh the
+		# index, so it may give misleading results. See
+		# git-update-index(1), git-diff-index(1), and git-status(1).
+		if {
+			git --no-optional-locks status -uno --porcelain 2>/dev/null ||
+			git diff-index --name-only HEAD
+		} | grep -qvE '^(.. )?scripts/package'; then
 			printf '%s' -dirty
 		fi