[v2,with,help] Re: run one test

Szabolcs Nagy <Szabolcs.Nagy@arm.com> writes:
> unfortunately there is currently no easy way to
> discover such make targets. (maybe there could be a
> make target that prints out usage information for
> the most commonly used make targets)

I added a help option, with mini-help.  I do NOT volunteer to write a
comprehensive one-page help doc for our Makefile, but invite everyone to
add their favorite targets willy-nilly after the initial commit :-)

The obvious next question is "how does the user know to run 'make
help'?"  but I tried to give them a hint there too.

> it can be convenient to rerun a set of tests too, e.g.
> by providing a list or pattern like 'math/test-double-*',
> which can be implemented in a similar way.

I think we can do that with wildcards and file matching, but I don't
know off the top of my head how to wrap the onetest code into that kind
of expansion.  Is there a compelling case for needing more than one but
less than a subdir of tests rerun?

> same for benchmarks.

I've not run the benchmarks myself, but... yeah, probably.

> for now the proposed patch looks good enough though.

Same patch, with help system, attached...

On 9/30/19 3:37 PM, DJ Delorie wrote:
> Szabolcs Nagy <Szabolcs.Nagy@arm.com> writes:
>> unfortunately there is currently no easy way to
>> discover such make targets. (maybe there could be a
>> make target that prints out usage information for
>> the most commonly used make targets)
> 
> I added a help option, with mini-help.  I do NOT volunteer to write a
> comprehensive one-page help doc for our Makefile, but invite everyone to
> add their favorite targets willy-nilly after the initial commit :-)

That's fine. I can add some notes on the targets I added for locales which
are useful to distro maintainers.

See my comments.

> The obvious next question is "how does the user know to run 'make
> help'?"  but I tried to give them a hint there too.
> 
>> it can be convenient to rerun a set of tests too, e.g.
>> by providing a list or pattern like 'math/test-double-*',
>> which can be implemented in a similar way.
> 
> I think we can do that with wildcards and file matching, but I don't
> know off the top of my head how to wrap the onetest code into that kind
> of expansion.  Is there a compelling case for needing more than one but
> less than a subdir of tests rerun?
> 
>> same for benchmarks.
> 
> I've not run the benchmarks myself, but... yeah, probably.
> 
>> for now the proposed patch looks good enough though.
> 
> Same patch, with help system, attached...
> 
> diff --git a/Makefile b/Makefile
> index 67ddd01bfe..0465609de0 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -26,8 +26,17 @@ include Makeconfig
>  
>  
>  # This is the default target; it makes everything except the tests.
> -.PHONY: all
> -all: lib others
> +.PHONY: all help minihelp
> +all: minihelp lib others
> +
> +help:
> +	@sed '0,/^help-starts-here$$/d' Makefile.help
> +
> +minihelp:
> +	@echo
> +	@echo type \"make help\" for help with common GLIBC makefile targets

s/GLIBC/glibc/g

> +	@echo
> +
>  
>  ifneq ($(AUTOCONF),no)
>  
> @@ -499,3 +508,11 @@ FORCE:
>  
>  iconvdata/% localedata/% po/%: FORCE
>  	$(MAKE) $(PARALLELMFLAGS) -C $(@D) $(@F)
> +
> +# Convenience target to rerun one test, from the top of the build tree
> +# Example: make onetest t=wcsmbs/test-wcsnlen
> +onetest :
> +	@-rm -f $(objpfx)$t.out
> +	$(MAKE) subdir=$(dir $t) -C $(dir $t) ..=../ $(objpfx)$t.out
> +	@cat $(objpfx)$t.test-result
> +	@cat $(objpfx)$t.out

Bikeshed. Suggest "onecheck" alias given that 'make test' is the same
as 'make check.'

> diff --git a/Makefile.help b/Makefile.help
> new file mode 100644
> index 0000000000..76b3418adb
> --- /dev/null
> +++ b/Makefile.help
> @@ -0,0 +1,41 @@
> +# Copyright (C) 2019 Free Software Foundation, Inc.
> +# This file is part of the GNU C Library.
> +
> +# The GNU C Library is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU Lesser General Public
> +# License as published by the Free Software Foundation; either
> +# version 2.1 of the License, or (at your option) any later version.
> +
> +# The GNU C Library is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +# Lesser General Public License for more details.
> +
> +# You should have received a copy of the GNU Lesser General Public
> +# License along with the GNU C Library; if not, see
> +# <https://www.gnu.org/licenses/>.

OK.

> +
> +This is the file that gets printed when the user runs "make help",
> +starting just after the "help-starts-here" line.
> +
> +help-starts-here
> +
> +all
> +	The usual default; builds everything but doesn't run the
> +	tests.
> +tests
> +	Runs the testsuite.  Also "make check" is a shortcut for this
> +
> +onetest

s/onetest/onecheck/g

Or add a onecheck alias.

> +	Runs one test.  Use like this:
> +		make onetest t=wcsmbs/test-wcsnlen
> +	Note that this will rebuild the test if needed, but will not
> +	rebuild what "make all" would have rebuilt.

OK.

> +
> +--
> +Other useful hints:
> +
> +$ rm $(objpfx)testroot.pristine/install.stamp
> +	Forces the testroot to be reinstalled the next time you run
> +	the testsuite (or just rm -rf testroot.pristine)
> +
> 

OK.

On Sep 30 2019, DJ Delorie <dj@redhat.com> wrote:

> @@ -499,3 +508,11 @@ FORCE:
>  
>  iconvdata/% localedata/% po/%: FORCE
>  	$(MAKE) $(PARALLELMFLAGS) -C $(@D) $(@F)
> +
> +# Convenience target to rerun one test, from the top of the build tree
> +# Example: make onetest t=wcsmbs/test-wcsnlen
> +onetest :
.PHONY: onetest

> +	@-rm -f $(objpfx)$t.out
> +	$(MAKE) subdir=$(dir $t) -C $(dir $t) ..=../ $(objpfx)$t.out
> +	@cat $(objpfx)$t.test-result
> +	@cat $(objpfx)$t.out

Or make the test name part of the target, so that you can use $*.

Andreas.

Latest iteration...

From b8c545e96f32420a1ebab0056e0a4b5dfe19cc1b Mon Sep 17 00:00:00 2001
From: DJ Delorie <dj@redhat.com>
Date: Mon, 30 Sep 2019 16:04:52 -0400
Subject: Add run-one-test convenience target and makefile help text

Adds "make test" for re-running just one test.  Also adds
"make help" for help with our Makefile targets, and adds a
mini-help when you just run "make".

diff --git a/Makefile b/Makefile
index 67ddd01bfe..1f3e088408 100644
--- a/Makefile
+++ b/Makefile
@@ -26,8 +26,17 @@ include Makeconfig
 
 
 # This is the default target; it makes everything except the tests.
-.PHONY: all
-all: lib others
+.PHONY: all help minihelp
+all: minihelp lib others
+
+help:
+	@sed '0,/^help-starts-here$$/d' Makefile.help
+
+minihelp:
+	@echo
+	@echo type \"make help\" for help with common glibc makefile targets
+	@echo
+
 
 ifneq ($(AUTOCONF),no)
 
@@ -499,3 +508,12 @@ FORCE:
 
 iconvdata/% localedata/% po/%: FORCE
 	$(MAKE) $(PARALLELMFLAGS) -C $(@D) $(@F)
+
+# Convenience target to rerun one test, from the top of the build tree
+# Example: make test t=wcsmbs/test-wcsnlen
+.PHONY: test
+test :
+	@-rm -f $(objpfx)$t.out
+	$(MAKE) subdir=$(dir $t) -C $(dir $t) ..=../ $(objpfx)$t.out
+	@cat $(objpfx)$t.test-result
+	@cat $(objpfx)$t.out
diff --git a/Makefile.help b/Makefile.help
new file mode 100644
index 0000000000..49815e9b9e
--- /dev/null
+++ b/Makefile.help
@@ -0,0 +1,42 @@
+# Copyright (C) 2019 Free Software Foundation, Inc.
+# This file is part of the GNU C Library.
+
+# The GNU C Library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+
+# The GNU C Library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Lesser General Public License for more details.
+
+# You should have received a copy of the GNU Lesser General Public
+# License along with the GNU C Library; if not, see
+# <https://www.gnu.org/licenses/>.
+
+This is the file that gets printed when the user runs "make help",
+starting just after the "help-starts-here" line.
+
+help-starts-here
+
+all
+	The usual default; builds everything but doesn't run the
+	tests.
+
+check (or tests)
+	Runs the standard set of tests.
+
+test
+	Runs one test.  Use like this:
+		make test t=wcsmbs/test-wcsnlen
+	Note that this will rebuild the test if needed, but will not
+	rebuild what "make all" would have rebuilt.
+
+--
+Other useful hints:
+
+$ rm $(objpfx)testroot.pristine/install.stamp
+	Forces the testroot to be reinstalled the next time you run
+	the testsuite (or just rm -rf testroot.pristine)
+

* DJ Delorie:

> +help:
> +	@sed '0,/^help-starts-here$$/d' Makefile.help

That doesn't look like something that would work in a build directory,
where Makefile.help will not be present in the current directory.

> +$ rm $(objpfx)testroot.pristine/install.stamp
> +	Forces the testroot to be reinstalled the next time you run
> +	the testsuite (or just rm -rf testroot.pristine)

It may make sense to put the actual path there, or otherwise avoid the
invalid shell syntax.

Thanks,
Florian

On 10/2/19 2:44 PM, DJ Delorie wrote:
> 
> Latest iteration...
> 

Reviewing other feedback. Looks good.

Szabolcs said it was good.

Florian and Andreas' comments are resolved or implemented in other ways.

My opinion is that we can play with this a bit and see what works.

This is my OK for master.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>

> From b8c545e96f32420a1ebab0056e0a4b5dfe19cc1b Mon Sep 17 00:00:00 2001
> From: DJ Delorie <dj@redhat.com>
> Date: Mon, 30 Sep 2019 16:04:52 -0400
> Subject: Add run-one-test convenience target and makefile help text
> 
> Adds "make test" for re-running just one test.  Also adds
> "make help" for help with our Makefile targets, and adds a
> mini-help when you just run "make".
> 
> diff --git a/Makefile b/Makefile
> index 67ddd01bfe..1f3e088408 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -26,8 +26,17 @@ include Makeconfig
>  
>  
>  # This is the default target; it makes everything except the tests.
> -.PHONY: all
> -all: lib others
> +.PHONY: all help minihelp
> +all: minihelp lib others
> +
> +help:
> +	@sed '0,/^help-starts-here$$/d' Makefile.help

OK. Resolved Florian's comment about $ojbdir.

> +
> +minihelp:
> +	@echo
> +	@echo type \"make help\" for help with common glibc makefile targets
> +	@echo
> +

OK.

>  
>  ifneq ($(AUTOCONF),no)
>  
> @@ -499,3 +508,12 @@ FORCE:
>  
>  iconvdata/% localedata/% po/%: FORCE
>  	$(MAKE) $(PARALLELMFLAGS) -C $(@D) $(@F)
> +
> +# Convenience target to rerun one test, from the top of the build tree
> +# Example: make test t=wcsmbs/test-wcsnlen
> +.PHONY: test
> +test :
> +	@-rm -f $(objpfx)$t.out
> +	$(MAKE) subdir=$(dir $t) -C $(dir $t) ..=../ $(objpfx)$t.out

OK, resolved differently from Andreas' suggestion, using t=TESTNAME.

> +	@cat $(objpfx)$t.test-result
> +	@cat $(objpfx)$t.out

OK.

> diff --git a/Makefile.help b/Makefile.help
> new file mode 100644
> index 0000000000..49815e9b9e
> --- /dev/null
> +++ b/Makefile.help
> @@ -0,0 +1,42 @@
> +# Copyright (C) 2019 Free Software Foundation, Inc.
> +# This file is part of the GNU C Library.
> +
> +# The GNU C Library is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU Lesser General Public
> +# License as published by the Free Software Foundation; either
> +# version 2.1 of the License, or (at your option) any later version.
> +
> +# The GNU C Library is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +# Lesser General Public License for more details.
> +
> +# You should have received a copy of the GNU Lesser General Public
> +# License along with the GNU C Library; if not, see
> +# <https://www.gnu.org/licenses/>.
> +
> +This is the file that gets printed when the user runs "make help",
> +starting just after the "help-starts-here" line.
> +
> +help-starts-here
> +
> +all
> +	The usual default; builds everything but doesn't run the
> +	tests.
> +
> +check (or tests)
> +	Runs the standard set of tests.
> +
> +test
> +	Runs one test.  Use like this:
> +		make test t=wcsmbs/test-wcsnlen
> +	Note that this will rebuild the test if needed, but will not
> +	rebuild what "make all" would have rebuilt.
> +
> +--
> +Other useful hints:
> +
> +$ rm $(objpfx)testroot.pristine/install.stamp
> +	Forces the testroot to be reinstalled the next time you run
> +	the testsuite (or just rm -rf testroot.pristine)

OK.

> +
>