| Message ID | ydd62u3p6m7.fsf@manam.CeBiTec.Uni-Bielefeld.DE |
|---|---|
| State | New |
| Headers | show |
On Wed, 5 Jan 2011, Dave Korn wrote: > Perhaps just build it in the makefile, as at present, and make the .la file > a dependency of the check target, so that it gets built before starting the > testrun? (Or conceivably if worried about it failing to build on some target Please no building of any files required by testsuites (other than site.exp) from makefiles; that breaks installed toolchain testing (where you should only need to create site.exp and run runtest with appropriate options).
On 05/01/2011 18:06, Rainer Orth wrote:
Hi Rainer,
> * The primary complication of the boehm-gc testsuite is the
> staticroottest testcase, which depends on a shared library.
> Currently, libtool is used to build that and I think that's the right
> decision rather than duplicating all of libtool's knowledge in
> DejaGnu.
>
> Unfortunately, DejaGnu knows nothing about libtool yet, and I'm still
> struggling with the right way to integrate it. For the moment (though
> I fear that wrong, over-complicated) I've chosen to add two additional
> keywords (ltassemble and ltlink) to match assemble and link, but for
> .la, .lo files instead. I'm open for suggestions for better ways to
> handle this, though.
Perhaps just build it in the makefile, as at present, and make the .la file
a dependency of the check target, so that it gets built before starting the
testrun? (Or conceivably if worried about it failing to build on some target
and that stopping the tests from running, build it using a recursive $(MAKE)
invocation, using the "-" prefix to ignore errors?)
cheers,
DaveK
I should probably have read further before hitting send... here's a couple more comments: On 05/01/2011 18:06, Rainer Orth wrote: > I don't really know what I'm asking for right now: I fear in it's > current state, the code could be too fragile for trunk and it would be > better to wait until 4.6 branches and then flesh out issues on mainline. I think this is unambiguously stage1 material. But that doesn't mean you can't post it now and get a deferred "OK once trunk is back in stage1". > However, all sorts of suggestions, encouragements (or discouragements :-) > are more than welcome. I do think it's an definite good idea, for all the reasons you suggested. cheers, DaveK
On 05/01/2011 20:25, Joseph S. Myers wrote: > On Wed, 5 Jan 2011, Dave Korn wrote: > >> Perhaps just build it in the makefile, as at present, and make the .la file >> a dependency of the check target, so that it gets built before starting the >> testrun? (Or conceivably if worried about it failing to build on some target > > Please no building of any files required by testsuites (other than > site.exp) from makefiles; that breaks installed toolchain testing (where > you should only need to create site.exp and run runtest with appropriate > options). Argh. But then again, boehm-gc doesn't get installed, so there is no possibility of testing the installed one anyway. So how about saying it's ok in those circumstances? cheers, DaveK
Hello Rainer, * Rainer Orth wrote on Wed, Jan 05, 2011 at 07:06:56PM CET: > I've long maintained that it would be very helpful to convert the > boehm-gc testsuite to DejaGnu: > > * It enables multilib testing, giving better test coverage. I don't understand. Why should the current tests not be run in multilib setting? Does toplevel not enter $target/$MULTIDIR/boehm-gc for check? > * I set DEJATOOL = boehm-gc in testsuite/Makefile.am. The default is > $PACKAGE, which is gc, but it seemed more intuitive to see boehm-gc in > mail-report.log, matching the toplevel directory. > > As usual, the site.exp target needs to be overridden to pass the > configure-determined THREADLIBS and EXTRA_TEST_LIBS down to the > testsuite. This is quite messy; it would be far better if automake > provided some facility for that. Can you open an Automake bug (send mail to bug-automake) with the features a better support from Automake should have? Thanks. I'm still very much a DejaGNU newbie. > * The primary complication of the boehm-gc testsuite is the > staticroottest testcase, which depends on a shared library. > Currently, libtool is used to build that and I think that's the right > decision rather than duplicating all of libtool's knowledge in > DejaGnu. > > Unfortunately, DejaGnu knows nothing about libtool yet, and I'm still > struggling with the right way to integrate it. For the moment (though > I fear that wrong, over-complicated) I've chosen to add two additional > keywords (ltassemble and ltlink) to match assemble and link, but for > .la, .lo files instead. I'm open for suggestions for better ways to > handle this, though. I'd be happy to try to help if there are libtool questions, but AFAICS the issue is mostly how to teach dejagnu how to use it? > --- /dev/null Thu Jan 01 00:00:00 1970 +0000 > +++ b/boehm-gc/testsuite/boehm-gc.lib/lib.exp Sat Jan 01 23:52:04 2011 +0100 > + # FIXME: Explain. Turn into parameter? > + set shopt "-version-info 1:2:0 -no-undefined -rpath /nowhere -shared-libgcc" You need to prefix -shared-libgcc with -Wc, to get it past libtool. When creating shared libraries, libtool drops most flags that it does not know about (and that it could do the wrong thing with if passed), so with -Wc, you're telling it that you know what you're doing. (And -shared-libgcc actually might not work correctly in all cases, which is what I mean with "it does not know about a flag"). > + set shopt "$shopt $gc_lib_conv" > + # Remove $bname.*o and .libs/$bname.*o. > + # FIXME: Might use libtool --mode=clean $bname.lo, but is this right in Missing 'rm -f' after --mode=clean. > + # cross-compilation scenarios? Why should it not be? > + # What is it? build, host? > + set to_delete [glob -nocomplain $bname.*o] > + set to_delete "$to_delete [glob -nocomplain .libs/$bname.*o]" > + eval remote_file build delete $to_delete > + > + return lib$bname.la > +} > + > +# FIXME: Comment. > +proc boehm-gc-lib-dg-runtest { testcases flags extra-flags } { > + global runtests > + > + foreach testcase $testcases { > + # If we're only testing specific files and this isn't one of them, skip it. > + if ![runtest_file_p $runtests $testcase] { > + return > + } > + > + # Library source corresponding to test file. > + regsub "test\.c$" $testcase "lib.c" lib > + verbose "lib: $lib" > + > + # Build support library. > + # FIXME: Handle via dg-add-shlib keyword? > + # If so, boehm-gc.lib becomes unnecessary. > + set shlib [boehm-gc-shlib $lib $flags ${extra-flags}] > + > + # Run testcase, linking with support library. > + dg-test $testcase $flags "${extra-flags} $shlib" > + > + # Remove $shlib, .libs/$shlib.*. > + # FIXME: Could the removal be done in a more general place? > + # Where is the rest of the removal done? Can't you use 'libtool --mode=clean' here as well? > + set to_delete $shlib > + set to_delete "$to_delete [glob -nocomplain .libs/[file rootname $shlib].*]" > + > + # FIXME: Doesn't delete .libs/libstaticrootslib.la and > + # .libs/libstaticrootslib.so.1, why? > + # Bug in remote.exp (standard_file): checks file exists and file > + # isfile first!? > + eval remote_file build delete $to_delete > + > + # Remove .libs/$execname. > + # FIXME: Should go into the framework somewhere, but dg-final > + # cannot be used since that is reset every time dg-test runs. > + remote_file build delete .libs/[file rootname [file tail $testcase]] > + } > +} > + > +dg-init > +boehm-gc-init > + > +global srcdir subdir > + > +# Gather list of tests, skipping library source files. > +set tests [lsort [glob -nocomplain $srcdir/$subdir/*.c]] > +set tests [prune $tests $srcdir/$subdir/*lib.c] > + > +boehm-gc-lib-dg-runtest $tests "-O2" "" > +dg-finish Cheers, Ralf
Hello Ralf, > * Rainer Orth wrote on Wed, Jan 05, 2011 at 07:06:56PM CET: >> I've long maintained that it would be very helpful to convert the >> boehm-gc testsuite to DejaGnu: >> >> * It enables multilib testing, giving better test coverage. > > I don't understand. Why should the current tests not be run in multilib > setting? Does toplevel not enter $target/$MULTIDIR/boehm-gc for check? while make check in a non-default multilib directory works (well, sort of: it doesn't look for the right libgcc_s.so.1), the check-TESTS target, which ultimately does this testing, isn't multilib-aware. Seems like an automake problem to me (unless there is some flag to change that; haven't looked). >> * I set DEJATOOL = boehm-gc in testsuite/Makefile.am. The default is >> $PACKAGE, which is gc, but it seemed more intuitive to see boehm-gc in >> mail-report.log, matching the toplevel directory. >> >> As usual, the site.exp target needs to be overridden to pass the >> configure-determined THREADLIBS and EXTRA_TEST_LIBS down to the >> testsuite. This is quite messy; it would be far better if automake >> provided some facility for that. > > Can you open an Automake bug (send mail to bug-automake) with the > features a better support from Automake should have? Thanks. > I'm still very much a DejaGNU newbie. Sure, will do. >> * The primary complication of the boehm-gc testsuite is the >> staticroottest testcase, which depends on a shared library. >> Currently, libtool is used to build that and I think that's the right >> decision rather than duplicating all of libtool's knowledge in >> DejaGnu. >> >> Unfortunately, DejaGnu knows nothing about libtool yet, and I'm still >> struggling with the right way to integrate it. For the moment (though >> I fear that wrong, over-complicated) I've chosen to add two additional >> keywords (ltassemble and ltlink) to match assemble and link, but for >> .la, .lo files instead. I'm open for suggestions for better ways to >> handle this, though. > > I'd be happy to try to help if there are libtool questions, but AFAICS > the issue is mostly how to teach dejagnu how to use it? I've now thought a bit more about that: the relevant DejaGnu procedures only have the input filename available to them, so cannot distinguish between compiling an input file into a regular or a libtool object. So I will need the ltassemble (compile .c etc. into .lo) type to go with assemble (.c -> .o). On the other hand, for --mode=link, I can see if the input file is a .lo or a .o and select the output filename accordingly, so my former ltlink can go. >> --- /dev/null Thu Jan 01 00:00:00 1970 +0000 >> +++ b/boehm-gc/testsuite/boehm-gc.lib/lib.exp Sat Jan 01 23:52:04 2011 +0100 > >> + # FIXME: Explain. Turn into parameter? >> + set shopt "-version-info 1:2:0 -no-undefined -rpath /nowhere -shared-libgcc" > > You need to prefix -shared-libgcc with -Wc, to get it past libtool. > When creating shared libraries, libtool drops most flags that it does > not know about (and that it could do the wrong thing with if passed), > so with -Wc, you're telling it that you know what you're doing. > (And -shared-libgcc actually might not work correctly in all cases, > which is what I mean with "it does not know about a flag"). Fixed, thanks. >> + set shopt "$shopt $gc_lib_conv" > >> + # Remove $bname.*o and .libs/$bname.*o. >> + # FIXME: Might use libtool --mode=clean $bname.lo, but is this right in > > Missing 'rm -f' after --mode=clean. Right, fixed. >> + # cross-compilation scenarios? > > Why should it not be? This may be just me not fully understanding the possible scenarios here: while a basic cross should work, what about a canadian cross (build != host != target)? The rm would have to be run on the host, but with libtool would run on the build machine. I have no idea if boehm-gc (or any other of the libtool libraries in GCC) work properly in such cases, though: the most I've tried so far is a basic cross to speed up reghunting for slow targets where possible. Maybe Joseph can shed some light here? >> + # Run testcase, linking with support library. >> + dg-test $testcase $flags "${extra-flags} $shlib" >> + >> + # Remove $shlib, .libs/$shlib.*. >> + # FIXME: Could the removal be done in a more general place? >> + # Where is the rest of the removal done? > > Can't you use 'libtool --mode=clean' here as well? Sure: if it works about, it will here, too. However, I think this would better be handled inside ${tool}-dg-test-1 than individually in each testsuite: that's what I meant by `more general place'. I've just done some cleanup work on the gnat.dg testsuite that gave me some ideas for this. In fact, I'm mostly done with a cleaned-up version of my patch that I'm almost happy with. I'll just have to work around a DejaGnu issue (cannot invoke dg-test inside a running dg-test call, which I have fixed in dg.exp itself for testing), then I'll post the result for review. Thanks for your feedback. Rainer
Dave Korn <dave.korn.cygwin@gmail.com> writes: > On 05/01/2011 20:25, Joseph S. Myers wrote: >> On Wed, 5 Jan 2011, Dave Korn wrote: >> >>> Perhaps just build it in the makefile, as at present, and make the .la file >>> a dependency of the check target, so that it gets built before starting the >>> testrun? (Or conceivably if worried about it failing to build on some target >> >> Please no building of any files required by testsuites (other than >> site.exp) from makefiles; that breaks installed toolchain testing (where >> you should only need to create site.exp and run runtest with appropriate >> options). > > Argh. But then again, boehm-gc doesn't get installed, so there is no > possibility of testing the installed one anyway. So how about saying it's ok > in those circumstances? I'm still with Joseph here: performing testsuite runs with a mixture of DejaGnu board facilities and external stuff is hard to understand and debug, so stay with one. Rainer
Dave Korn <dave.korn.cygwin@gmail.com> writes: > On 05/01/2011 18:06, Rainer Orth wrote: > >> I don't really know what I'm asking for right now: I fear in it's >> current state, the code could be too fragile for trunk and it would be >> better to wait until 4.6 branches and then flesh out issues on mainline. > > I think this is unambiguously stage1 material. But that doesn't mean you > can't post it now and get a deferred "OK once trunk is back in stage1". Ok. The trouble will probably be finding someone to review and approve that stuff, stage 1 or not. >> However, all sorts of suggestions, encouragements (or discouragements :-) >> are more than welcome. > > I do think it's an definite good idea, for all the reasons you suggested. Thanks. Looking at the current libgo testsuite makes me think so as well :-) Rainer
On 10 Jan 2011, at 10:43, Rainer Orth wrote: >>> However, all sorts of suggestions, encouragements (or >>> discouragements :-) >>> are more than welcome. >> >> I do think it's an definite good idea, for all the reasons you >> suggested. > > Thanks. Looking at the current libgo testsuite makes me think so as > well :-) +1 .. the failure of boehm-gc on ppc64-darwin9 was effectively hidden by this. thanks for looking at it. Iain
On Jan 5, 2011, at 12:44 PM, Dave Korn wrote:
> Perhaps just build it in the makefile,
Please, no. Pretend there is a configure flag to have the library installed, or a language is added that uses it, and installs it, installed testing should just work. `should just work' is an important property.
On Jan 5, 2011, at 12:46 PM, Dave Korn wrote: > I think this is unambiguously stage1 material. I think I'd tend to say stage1 as well. I'd like more bake time on it... > I do think it's an definite good idea As do I.
On Jan 10, 2011, at 2:40 AM, Rainer Orth wrote: > This may be just me not fully understanding the possible scenarios here: > while a basic cross should work, what about a canadian cross (build != > host != target)? The rm would have to be run on the host, but with > libtool would run on the build machine. I have no idea if boehm-gc (or > any other of the libtool libraries in GCC) work properly in such cases, Yes, historically they do. It is actually a nice feature of gcc. At times people blow things, but generally they are easy to fix, once you're in that environment and mindset.
On Jan 5, 2011, at 10:06 AM, Rainer Orth wrote: > I see lots of opportunities for code reuse here: most DejaGnu tools in > GCC started off as a copy of some other tool with search-and-replace > of the tool name and only a few (if any) local changes. This is a > total mess to understand and maintain and I hope to do something about > this in the future. :-) You now can, though, bear in mind, once you have a delegation neuron, and multiple inheritance neurons (ok, stop laughing at me)... the design of dejagnu is slightly cleaner than at first blush. If you've ever seen it wired up to telnet to a terminal server to connect to the serial console on a PC running dos, controled from a real unix machine, to test an entire toolchain (compiler, gdb and so on), you can start to gain some appreciation. Though, I admit, if you only ever do native testing, the thing is hard to read, convoluted and mysterious. If someone know of a good programming methodology to allow for the complexity, but hide it for most people, love to have a pointer... I'd love such a system. If you ever seen that, and the PC talking to a mips target, and seen it test 5 boards at once, it is sweet.
Mike Stump <mikestump@comcast.net> writes: > On Jan 5, 2011, at 12:46 PM, Dave Korn wrote: >> I think this is unambiguously stage1 material. > > I think I'd tend to say stage1 as well. I'd like more bake time on it... Absolutely. I know of two current issues: * one 64-bit failure on IRIX that I haven't analysed yet, so I cannot tell if this was caused by the DejaGnu testsuite (non-default multilibs aren't tested at all without DejaGnu, so this wouldn't have shown up before), and * failure to pass all required compilation flags (-pthread on Tru64 UNIX, which breaks testcase compilation). I do have a preliminary patch for the latter issue (mostly on the build side), but since my Tru64 UNIX V5.1 test system broke down recently and I'm still looking for a replacement. >> I do think it's an definite good idea > > As do I. Thanks. There are a few other testsuites that could be converted: libiberty and libgo immediately come to mind. Rainer
Mike Stump <mikestump@comcast.net> writes: > On Jan 5, 2011, at 10:06 AM, Rainer Orth wrote: >> I see lots of opportunities for code reuse here: most DejaGnu tools in >> GCC started off as a copy of some other tool with search-and-replace >> of the tool name and only a few (if any) local changes. This is a >> total mess to understand and maintain and I hope to do something about >> this in the future. > > :-) You now can, though, bear in mind, once you have a delegation neuron, and multiple inheritance neurons (ok, stop laughing at me)... the design of dejagnu is slightly cleaner than at first blush. My primary issue isn't so much with the design of DejaGnu (rather its lack of documentation), but with the current uses in GCC: duplicating the whole per-tool code for every tool with just a few often diverging changes isn't my idea of a maintainable code base. > If you've ever seen it wired up to telnet to a terminal server to connect to the serial console on a PC running dos, controled from a real unix machine, to test an entire toolchain (compiler, gdb and so on), you can start to gain some appreciation. Though, I admit, if you only ever do native testing, the thing is hard to read, convoluted and mysterious. If someone know of a good programming methodology to allow for the complexity, but hide it for most people, love to have a pointer... I'd love such a system. > > If you ever seen that, and the PC talking to a mips target, and seen it test 5 boards at once, it is sweet. No doubt about that :-) I haven't had much use for that myself so far, coming from a pure Unix background and no need to dive into embedded envionments. But certainly the generality helps tremendously, once you're in the right mindset. Rainer
On Feb 22, 2011, at 9:09 AM, Rainer Orth wrote: > My primary issue isn't so much with the design of DejaGnu (rather its > lack of documentation), but with the current uses in GCC: duplicating > the whole per-tool code for every tool with just a few often diverging > changes isn't my idea of a maintainable code base. Yeah, happens when someone wants to fix one testsuite, but isn't given free reign to modify others. The benefit, one change on one side can't hurt the other side. The downside, large scale replication. I'd support refactoring things... it would be a thankless job. One area in particular that I'd love to see improved are the loops like: foreach src [lsort [find $srcdir/$subdir *_main.c]] { # If we're only testing specific files and this isn't one of them, skip it. if ![runtest_file_p $runtests $src] then { continue } compat-execute $src $sid $compat_use_alt } in the .exp testcase files. Longer term, I'd like to modify our framework driver .exp file (those in lib) to do tests in parallel directly. I almost had it all wired up last weekend, but ran into two problems that made me sad, tcl sucks and the threads people that did the proc replacement function for quirting code into new threads, can't handle the full generality (proc ${tool}_init ...) of tcl. The other thing that made me sad was the shear replication of this loop in all the .exp files.
diff -r 71cfe7556b69 boehm-gc/Makefile.am --- a/boehm-gc/Makefile.am Sat Jan 01 23:04:07 2011 +0100 +++ b/boehm-gc/Makefile.am Sat Jan 01 23:52:04 2011 +0100 @@ -4,10 +4,10 @@ ## files that should be in the distribution are not mentioned in this ## Makefile.am. -AUTOMAKE_OPTIONS = cygnus subdir-objects +AUTOMAKE_OPTIONS = foreign subdir-objects ACLOCAL_AMFLAGS = -I .. -I ../config -SUBDIRS = include +SUBDIRS = include testsuite noinst_LTLIBRARIES = libgcjgc.la libgcjgc_convenience.la @@ -35,7 +35,7 @@ # Include THREADLIBS here to ensure that the correct versions of # linuxthread semaphore functions get linked: -libgcjgc_la_LIBADD = $(addobjs) $(THREADLIBS) $(UNWINDLIBS) +libgcjgc_la_LIBADD = $(addobjs) $(THREADLIBS) libgcjgc_la_DEPENDENCIES = $(addobjs) libgcjgc_la_LDFLAGS = $(extra_ldflags_libgc) -version-info 1:2:0 -rpath $(toolexeclibdir) libgcjgc_la_LINK = $(LINK) $(libgcjgc_la_LDFLAGS) @@ -48,43 +48,6 @@ AM_LDFLAGS = $(shell $(top_srcdir)/../libtool-ldflags $(LDFLAGS)) override CFLAGS := $(filter-out $(O0_CFLAGS), $(CFLAGS)) $(O0_CFLAGS) -test_ldadd = libgcjgc.la $(THREADLIBS) $(UNWINDLIBS) $(EXTRA_TEST_LIBS) - -check_PROGRAMS = gctest -gctest_SOURCES = tests/test.c -gctest_LDADD = $(test_ldadd) -gctest_LDFLAGS = -shared-libgcc -gctest_LINK = $(LINK) $(gctest_LDFLAGS) -TESTS_ENVIRONMENT = LD_LIBRARY_PATH=../../$(MULTIBUILDTOP)gcc -TESTS = gctest - -TESTS += leaktest$(EXEEXT) -check_PROGRAMS += leaktest -leaktest_SOURCES = tests/leak_test.c -leaktest_LDADD = $(test_ldadd) -leaktest_LDFLAGS = -shared-libgcc -leaktest_LINK = $(LINK) $(leaktest_LDFLAGS) - -TESTS += middletest$(EXEEXT) -check_PROGRAMS += middletest -middletest_SOURCES = tests/middle.c -middletest_LDADD = $(test_ldadd) -middletest_LDFLAGS = -shared-libgcc -middletest_LINK = $(LINK) $(middletest_LDFLAGS) - -TESTS += staticrootstest$(EXEEXT) -check_PROGRAMS += staticrootstest -staticrootstest_SOURCES = tests/staticrootstest.c -staticrootstest_LDADD = $(test_ldadd) libstaticrootslib.la -staticrootstest_LDFLAGS = -shared-libgcc -staticrootstest_LINK = $(LINK) $(staticrootstest_LDFLAGS) -check_LTLIBRARIES = libstaticrootslib.la -libstaticrootslib_la_SOURCES = tests/staticrootslib.c -libstaticrootslib_la_LIBADD = libgcjgc_convenience.la -libstaticrootslib_la_LDFLAGS = -version-info 1:2:0 -no-undefined \ - -rpath /nowhere -shared-libgcc -libstaticrootslib_la_DEPENDENCIES = libgcjgc_convenience.la - ## FIXME: we shouldn't have to do this, but automake forces us to. .s.lo: ## We use -Wp,-P to strip #line directives. Irix `as' chokes on @@ -116,7 +79,6 @@ "PICFLAG_FOR_TARGET=$(PICFLAG_FOR_TARGET)" \ "SHELL=$(SHELL)" \ "EXPECT=$(EXPECT)" \ - "RUNTEST=$(RUNTEST)" \ "RUNTESTFLAGS=$(RUNTESTFLAGS)" \ "exec_prefix=$(exec_prefix)" \ "infodir=$(infodir)" \ diff -r 71cfe7556b69 boehm-gc/configure.ac --- a/boehm-gc/configure.ac Sat Jan 01 23:04:07 2011 +0100 +++ b/boehm-gc/configure.ac Sat Jan 01 23:52:04 2011 +0100 @@ -1,4 +1,4 @@ -# Copyright (c) 1999, 2000, 2001, 2002, 2003, 2006, 2010 by Red Hat, Inc. +# Copyright (c) 1999, 2000, 2001, 2002, 2003, 2006, 2010, 2011 by Red Hat, Inc. # All rights reserved. # Copyright 2004 Nathanael Nerode # @@ -542,5 +542,5 @@ AC_CONFIG_HEADERS([include/gc_config.h include/gc_ext_config.h]) -AC_CONFIG_FILES(Makefile include/Makefile threads.mk) +AC_CONFIG_FILES(Makefile include/Makefile testsuite/Makefile threads.mk) AC_OUTPUT diff -r 71cfe7556b69 boehm-gc/testsuite/Makefile.am --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/boehm-gc/testsuite/Makefile.am Sat Jan 01 23:52:04 2011 +0100 @@ -0,0 +1,43 @@ +## Process this file with automake to produce Makefile.in. + +AUTOMAKE_OPTIONS = foreign dejagnu + +# Setup the testing framework, if you have one +EXPECT = `if [ -f $(top_builddir)/../expect/expect ] ; then \ + echo $(top_builddir)/../expect/expect ; \ + else echo expect ; fi` + +RUNTEST = `if [ -f $(top_srcdir)/../dejagnu/runtest ] ; then \ + echo $(top_srcdir)/../dejagnu/runtest ; \ + else echo runtest; fi` + +# Override default. +DEJATOOL = boehm-gc + +CLEANFILES = *.exe core* *.log *.sum + +# We need more things in site.exp, but automake completely controls the +# creation of that file; there's no way to append to it without messing up +# the dependancy chains. So we overrule automake. This rule is exactly +# what it would have generated, plus our own additions. +site.exp: Makefile + @echo 'Making a new site.exp file...' + @echo '## these variables are automatically generated by make ##' >site.tmp + @echo '# Do not edit here. If you wish to override these values' >>site.tmp + @echo '# edit the last section' >>site.tmp + @echo 'set srcdir $(srcdir)' >>site.tmp + @echo "set objdir `pwd`" >>site.tmp + @echo 'set build_alias "$(build_alias)"' >>site.tmp + @echo 'set build_triplet $(build_triplet)' >>site.tmp + @echo 'set host_alias "$(host_alias)"' >>site.tmp + @echo 'set host_triplet $(host_triplet)' >>site.tmp + @echo 'set target_alias "$(target_alias)"' >>site.tmp + @echo 'set target_triplet $(target_triplet)' >>site.tmp + @echo 'set threadlibs "$(THREADLIBS)"' >>site.tmp + @echo 'set extra_test_libs "$(EXTRA_TEST_LIBS)"' >>site.tmp + @echo '## All variables above are generated by configure. Do Not Edit ##' >>site.tmp + @test ! -f site.exp || \ + sed '1,/^## All variables above are.*##/ d' site.exp >> site.tmp + @-rm -f site.bak + @test ! -f site.exp || mv site.exp site.bak + @mv site.tmp site.exp diff -r 71cfe7556b69 boehm-gc/testsuite/boehm-gc.c/c.exp --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/boehm-gc/testsuite/boehm-gc.c/c.exp Sat Jan 01 23:52:04 2011 +0100 @@ -0,0 +1,27 @@ +# Copyright (C) 2011 Free Software Foundation, Inc. + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; see the file COPYING3. If not see +# <http://www.gnu.org/licenses/>. + +dg-init +boehm-gc-init + +global srcdir subdir + +dg-runtest [lsort [glob -nocomplain $srcdir/$subdir/*.c]] "-O2" "" +dg-finish + +# Local Variables: +# tcl-indent-level:4 +# End: diff -r 71cfe7556b69 boehm-gc/testsuite/boehm-gc.c/trace_test.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/boehm-gc/testsuite/boehm-gc.c/trace_test.c Sat Jan 01 23:52:04 2011 +0100 @@ -0,0 +1,32 @@ +/* { dg-skip-if "requires --enable-gc-debug" *-*-* } */ + +#include <stdio.h> +#define GC_DEBUG +#include "gc.h" + +struct treenode { + struct treenode *x; + struct treenode *y; +} * root[10]; + +struct treenode * mktree(int i) { + struct treenode * r = GC_MALLOC(sizeof(struct treenode)); + if (0 == i) return 0; + if (1 == i) r = GC_MALLOC_ATOMIC(sizeof(struct treenode)); + r -> x = mktree(i-1); + r -> y = mktree(i-1); + return r; +} + +int main() +{ + int i; + for (i = 0; i < 10; ++i) { + root[i] = mktree(12); + } + GC_generate_random_backtrace(); + GC_generate_random_backtrace(); + GC_generate_random_backtrace(); + GC_generate_random_backtrace(); + return 0; +} diff -r 71cfe7556b69 boehm-gc/testsuite/boehm-gc.lib/lib.exp --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/boehm-gc/testsuite/boehm-gc.lib/lib.exp Sat Jan 01 23:52:04 2011 +0100 @@ -0,0 +1,115 @@ +# Copyright (C) 2011 Free Software Foundation, Inc. + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; see the file COPYING3. If not see +# <http://www.gnu.org/licenses/>. + +# FIXME: Comment. +# Better name? boehm-gc-build-shlib? +proc boehm-gc-shlib { lib flags extra-flags } { + global subdir + global dg-do-what-default + global gc_lib_conv + + # Determine basename. + # FIXME: Use dg-trim-dirname $srcdir $lib? + set nshort "$subdir/[file tail $lib]" + set bname "[file rootname [file tail $nshort]]" + verbose "bname: $bname" + + set saved-dg-do-what ${dg-do-what-default} + + # FIXME: Maybe do this in one step? + # Doesn't work right now? libtool issue, it seems. + set dg-do-what-default ltassemble + dg-test -keep-output $lib $flags ${extra-flags} + + # FIXME: Explain. Turn into parameter? + set shopt "-version-info 1:2:0 -no-undefined -rpath /nowhere -shared-libgcc" + set shopt "$shopt $gc_lib_conv" + + # FIXME: Message seems strange: e.g. + # PASS: staticrootslib.lo -O2 (test for excess errors) + # FIXME: How to avoid knowledge on output filenames? + set dg-do-what-default ltlink + dg-test -keep-output $bname.lo $flags "${extra-flags} $shopt" + + set dg-do-what-default ${saved-dg-do-what} + + # Remove $bname.*o and .libs/$bname.*o. + # FIXME: Might use libtool --mode=clean $bname.lo, but is this right in + # cross-compilation scenarios? + # What is it? build, host? + set to_delete [glob -nocomplain $bname.*o] + set to_delete "$to_delete [glob -nocomplain .libs/$bname.*o]" + eval remote_file build delete $to_delete + + return lib$bname.la +} + +# FIXME: Comment. +proc boehm-gc-lib-dg-runtest { testcases flags extra-flags } { + global runtests + + foreach testcase $testcases { + # If we're only testing specific files and this isn't one of them, skip it. + if ![runtest_file_p $runtests $testcase] { + return + } + + # Library source corresponding to test file. + regsub "test\.c$" $testcase "lib.c" lib + verbose "lib: $lib" + + # Build support library. + # FIXME: Handle via dg-add-shlib keyword? + # If so, boehm-gc.lib becomes unnecessary. + set shlib [boehm-gc-shlib $lib $flags ${extra-flags}] + + # Run testcase, linking with support library. + dg-test $testcase $flags "${extra-flags} $shlib" + + # Remove $shlib, .libs/$shlib.*. + # FIXME: Could the removal be done in a more general place? + # Where is the rest of the removal done? + set to_delete $shlib + set to_delete "$to_delete [glob -nocomplain .libs/[file rootname $shlib].*]" + + # FIXME: Doesn't delete .libs/libstaticrootslib.la and + # .libs/libstaticrootslib.so.1, why? + # Bug in remote.exp (standard_file): checks file exists and file + # isfile first!? + eval remote_file build delete $to_delete + + # Remove .libs/$execname. + # FIXME: Should go into the framework somewhere, but dg-final + # cannot be used since that is reset every time dg-test runs. + remote_file build delete .libs/[file rootname [file tail $testcase]] + } +} + +dg-init +boehm-gc-init + +global srcdir subdir + +# Gather list of tests, skipping library source files. +set tests [lsort [glob -nocomplain $srcdir/$subdir/*.c]] +set tests [prune $tests $srcdir/$subdir/*lib.c] + +boehm-gc-lib-dg-runtest $tests "-O2" "" +dg-finish + +# Local Variables: +# tcl-indent-level:4 +# End: diff -r 71cfe7556b69 boehm-gc/testsuite/config/default.exp --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/boehm-gc/testsuite/config/default.exp Sat Jan 01 23:52:04 2011 +0100 @@ -0,0 +1,1 @@ +load_lib "standard.exp" diff -r 71cfe7556b69 boehm-gc/testsuite/lib/boehm-gc.exp --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/boehm-gc/testsuite/lib/boehm-gc.exp Sat Jan 01 23:52:04 2011 +0100 @@ -0,0 +1,235 @@ +# Copyright (C) 2011 Free Software Foundation, Inc. + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; see the file COPYING3. If not see +# <http://www.gnu.org/licenses/>. + +proc load_gcc_lib { filename } { + global srcdir + load_file $srcdir/../../gcc/testsuite/lib/$filename +} + +load_lib dg.exp +load_lib libgloss.exp +load_gcc_lib target-libpath.exp +load_gcc_lib wrapper.exp +# Required by target-supports-dg.exp. +load_gcc_lib target-supports.exp +# For dg-skip-if. +load_gcc_lib target-supports-dg.exp + +set dg-do-what-default run + +# Define boehm-gc callbacks for dg.exp. + +# FIXME: The next two are Independent of boehm-gc; move to some library. +proc ${tool}-dg-test-1 { target_compile prog do_what extra_tool_flags } { + + # Set up the compiler flags, based on what we're going to do. + + set options [list] + switch $do_what { + "compile" { + set compile_type "assembly" + set output_file "[file rootname [file tail $prog]].s" + } + "assemble" { + set compile_type "object" + set output_file "[file rootname [file tail $prog]].o" + } + # FIXME: Really need different do_what's for libtool? + "ltassemble" { + set compile_type "object" + set output_file "[file rootname [file tail $prog]].lo" + } + "link" { + set compile_type "executable" + set output_file "[file rootname [file tail $prog]]" + } + # FIXME: See above. + "ltlink" { + set compile_type "executable" + set output_file "lib[file rootname [file tail $prog]].la" + } + "run" { + set compile_type "executable" + # FIXME: "./" is to cope with "." not being in $PATH. + # Should this be handled elsewhere? + # YES. + set output_file "./[file rootname [file tail $prog]]" + # This is the only place where we care if an executable was + # created or not. If it was, dg.exp will try to run it. + remote_file build delete $output_file + } + default { + perror "$do_what: not a valid dg-do keyword" + return "" + } + } + + if { $extra_tool_flags != "" } { + lappend options "additional_flags=$extra_tool_flags" + } + + set comp_output [$target_compile "$prog" "$output_file" "$compile_type" $options]; + + return [list $comp_output $output_file] +} + +proc ${tool}-dg-test { prog do_what extra_tool_flags } { + global tool + + return [${tool}-dg-test-1 ${tool}_target_compile $prog $do_what $extra_tool_flags] +} + +proc ${tool}-init { args } { + global gluefile wrap_flags + global srcdir + global blddirgc + global TOOL_EXECUTABLE + global GCC_UNDER_TEST + global objdir + global gc_include + global gc_lib + global gc_lib_conv + global tool + global tool_root_dir + global ld_library_path + + set blddirgc [lookfor_file [get_multilibs] ${tool}] + verbose "blddirgc: $blddirgc" + + if ![info exists GCC_UNDER_TEST] then { + if [info exists TOOL_EXECUTABLE] { + set GCC_UNDER_TEST $TOOL_EXECUTABLE + } else { + set GCC_UNDER_TEST "[find_gcc]" + } + } + + set gccdir [lookfor_file $tool_root_dir gcc/libgcc.a] + if {$gccdir != ""} { + set gccdir [file dirname $gccdir] + } + verbose "gccdir: $gccdir" + + set ld_library_path "." + append ld_library_path ":${gccdir}" + + set compiler "${gccdir}/xgcc" + if { [is_remote host] == 0 && [which $compiler] != 0 } { + foreach i "[exec $compiler --print-multi-lib]" { + set mldir "" + regexp -- "\[a-z0-9=_/\.-\]*;" $i mldir + set mldir [string trimright $mldir "\;@"] + if { "$mldir" == "." } { + continue + } + if { [llength [glob -nocomplain ${gccdir}/${mldir}/libgcc_s*.so.*]] >= 1 } { + append ld_library_path ":${gccdir}/${mldir}" + } + } + } + # Add the library path for boehm-gc. + append ld_library_path ":${blddirgc}/.libs" + + # Add the library path for boehm-gc.lib libtool libraries. + # FIXME: Do this here instead of in driver? + append ld_library_path ":.libs" + + # FIXME: ld_library_path is doubled!? + # Starts at 'Setting LD_LIBRARY_PATH to ...'; from config/unix.exp + # (unix_load). + # Seems that lib/target-libpath.exp (set_ld_library_path_env_vars sets + # it first and unix_load doubles it. + verbose "ld_library_path: $ld_library_path" + + # Point to the headers in boehm-gc. + set gc_include "${blddirgc}/include" + verbose "gc_include: $gc_include" + + set gc_lib "${blddirgc}/libgcjgc.la" + verbose "gc_lib: $gc_lib" + + set gc_lib_conv "${blddirgc}/libgcjgc_convenience.la" + verbose "gc_lib_conv: $gc_lib_conv" + + set_ld_library_path_env_vars + ${tool}_maybe_build_wrapper "${objdir}/testglue.o" +} + +proc ${tool}_finish { } { + # FIXME: Remove .libs? +} + +proc ${tool}_exit { } { + global gluefile; + + if [info exists gluefile] { + file_on_build delete $gluefile; + unset gluefile; + } +} + +proc ${tool}_target_compile { source dest type options } { + global gluefile wrap_flags; + global srcdir + global TOOL_OPTIONS + global GCC_UNDER_TEST + global gc_include + global gc_lib + global gc_lib_conv + global threadlibs + global extra_test_libs + + if { [target_info needs_status_wrapper]!="" && [info exists gluefile] } { + lappend options "libs=${gluefile}" + lappend options "ldflags=$wrap_flags" + } + + # TOOL_OPTIONS must come first, so that it doesn't override testcase + # specific options. + if [info exists TOOL_OPTIONS] { + lappend options [concat "additional_flags=$TOOL_OPTIONS" $options]; + } + + # Map type to libtool mode. + switch $type { + "object" { + set mode "compile" + } + "executable" { + set mode "link" + } + default { + perror "$type: unhandled mode" + return "" + } + } + + # We have to run silently to avoid DejaGnu lossage. + lappend options "compiler=../libtool --silent --tag=CC --mode=$mode \ + $GCC_UNDER_TEST" + + lappend options "additional_flags=-I${gc_include} -I${srcdir}/../include" + + lappend options "libs= -shared-libgcc" + lappend options "libs= ${gc_lib} ${threadlibs} ${extra_test_libs}" + + verbose "options: $options" + return [target_compile $source $dest $type $options] +} + +# Local Variables: +# tcl-indent-level:4 +# End:
I've long maintained that it would be very helpful to convert the boehm-gc testsuite to DejaGnu: * It enables multilib testing, giving better test coverage. * Unlike the present situation, failures are not easily overlooked because the results immediately show up in mail-report.log instead of being hidden in a multi-megabyte make check output. I've many times found that e.g. libjava failures were ultimately cause by problems in boehm-gc. * All DejaGnu features (like timeouts to avoid hanging the whole make check run when gctest doesn't complete, target-specific xfails etc.) suddenly become available. Apart from that, I'd thought that boehm-gc would make a good case study for developing a new DejaGnu tool/testsuite, but that proved wrong since I had to use libtool to do all the compiling and linking ;-) Anyway, here's the result, with several points worth mentioning: * Makefile.am used AUTOMAKE_OPTIONS = cygnus, although the current Makefile.in was generated with automake --foreign. Since I could see no reason to stay with cygnus here, I've moved to foreign like most (all?) other Makefile.am's. UNWINDLIBS was present, but never set anywhere, so I removed it. One must not include RUNTEST=$(RUNTEST) in the toplevel Makefile's AM_MAKEFLAGS: it isn't set anywhere and breaks make check by passing down that empty value. * I set DEJATOOL = boehm-gc in testsuite/Makefile.am. The default is $PACKAGE, which is gc, but it seemed more intuitive to see boehm-gc in mail-report.log, matching the toplevel directory. As usual, the site.exp target needs to be overridden to pass the configure-determined THREADLIBS and EXTRA_TEST_LIBS down to the testsuite. This is quite messy; it would be far better if automake provided some facility for that. * I've moved all the testcases from tests to subdirectories to testsuite/boehm-gc.* where DejaGnu expects them. Unlike the current Makefile.am, where the names of the test sources and executables sometimes differ, I've kept the source file names as is, with the exception of test.c which I've renamed to gctest.c since that test name is quite well-known, I think. Three tests weren't run before: test_cpp.cc I haven't bothered yet to add the testsuite goo for a single C++ test. It can certainly be done if need be, but I'd rather to concentrate on my next project, ACATS to DejaGnu conversion, instead. thread_leak_test.c No apparent reason, just works. trace_test.c Uses GC_generate_random_backtrace which is only available when configuring with --enable-gc-debug, so I'm skipping this everywhere. * The meat of the testsuite (a new tool in DejaGnu parlance) lives in testsuite/lib/boehm-gc.exp. I've started with the libffi equivalent since it was the smallest and further tried to strip it down. I see lots of opportunities for code reuse here: most DejaGnu tools in GCC started off as a copy of some other tool with search-and-replace of the tool name and only a few (if any) local changes. This is a total mess to understand and maintain and I hope to do something about this in the future. I've tried to identify possibly reusable code and liberally scattered lots of FIXME comments all over. The primary problem right now is to get a highlevel picture of the control flow in a tool. Unfortunately, the DejaGnu documentation is somewhat lacking here to put it mildly. (To be honest, dg.exp documentation in DejaGnu is completely absent. Janis added some documentation to sourcebuild.texi, but more needs to be done. Hopefully I'll be able to contribute here.) * The primary complication of the boehm-gc testsuite is the staticroottest testcase, which depends on a shared library. Currently, libtool is used to build that and I think that's the right decision rather than duplicating all of libtool's knowledge in DejaGnu. Unfortunately, DejaGnu knows nothing about libtool yet, and I'm still struggling with the right way to integrate it. For the moment (though I fear that wrong, over-complicated) I've chosen to add two additional keywords (ltassemble and ltlink) to match assemble and link, but for .la, .lo files instead. I'm open for suggestions for better ways to handle this, though. As I said, there are many FIXMEs embbed below, and I'd be grateful for suggestions. Anyway, the code below works fine for me, as can be seen in this excerpt from i386-pc-solaris2.11 mail-report.log: === boehm-gc tests === Running target unix FAIL: boehm-gc.c/gctest.c -O2 execution test === boehm-gc Summary for unix === # of expected passes 11 # of unexpected failures 1 # of unsupported tests 1 Running target unix/-m64 === boehm-gc Summary for unix/-m64 === # of expected passes 12 # of unsupported tests 1 === boehm-gc Summary === # of expected passes 23 # of unexpected failures 1 # of unsupported tests 2 This nicely shows the 32-bit-only gctest execution failure which I mentioned earlier: this only happens with GNU ld, Sun ld is fine. Without DejaGnu, such a failure would only be noticed if one happened to look at the make check output or were specificially looking for boehm-gc issues for some reason. I don't really know what I'm asking for right now: I fear in it's current state, the code could be too fragile for trunk and it would be better to wait until 4.6 branches and then flesh out issues on mainline. However, all sorts of suggestions, encouragements (or discouragements :-) are more than welcome. Rainer 2010-12-31 Rainer Orth <ro@CeBiTec.Uni-Bielefeld.DE> PR boehm-gc/11412 * Makefile.am (AUTOMAKE_OPTIONS): Use foreign instead of cygnus. (SUBDIRS): Add testsuite. (libgcjgc_la_LIBADD): Remove $(UNWINDLIBS). Remove TESTS related variables. (AM_MAKEFLAGS): Don't pass RUNTEST. * configure.ac: Add testsuite/Makefile to AC_CONFIG_FILES. * Makefile.in: Regenerate. * configure: Regenerate. * testsuite/Makefile.am: New file. * testsuite/Makefile.in: New file. * testsuite/lib/boehm-gc.exp: New file. * testsuite/config/default.exp: New file. * testsuite/boehm-gc.c/c.exp: New file. * testsuite/boehm-gc.lib/lib.exp: New file. * tests/test.c: Move ... * testsuite/boehm-gc.c/gctest.c: ... here. * tests/leak_test.c, tests/middle.c, tests/thread_leak_test.c, tests/trace_test.c: Move ... * testsuite/boehm-gc.c: ... here. * testsuite/boehm-gc.c/trace_test.c: Skip everywhere. * tests/staticrootslib.c, tests/staticrootstest.c: Move ... * testsuite/boehm-gc.lib: ... here. * tests/test_cpp.cc: Move ... * testsuite/boehm-gc.c++: ... here.