[build,testsuite,v3] Increase gcc, g++, gfortran and libstdc++-v3 testsuite parallelism

[Please keep me on the Cc:, I'm not subscribed to libstdc++.  Thanks.]

make check times on slow machines with many cores (like the
UltraSPARC-T1/T2) is currently limited by too little parallelism in the
testsuite.  E.g., on a Sun Fire T5220 with a single UltraSPARC-T2 CPU
(1.2 GHz, 8 cores with 8 strands/hw threads each), you can easily run
bootstrap with make -j128 and the build completes in about 2 1/2 hours.
A subsequent make -j128 -k check (both 32 and 64-bit) takes 6 hours,
while even on my laptop (Core2 Duo T7500, 2.2 GHz, 2 cores), build and
test times are roughly identical.  This is not only a SPARC problem,
though: with 8-socket 8-core Nehalem EX machines coming, we have 64
cores per machine, doubling that with HT, so gcc testing won't exploit
the available parallelism.

As a first step to improve things, I've increased parallelism in the
testsuites with the largest number of tests (using wc -l <testsuite>.sum
as an estimate):

    2361 gcc/testsuite/ada/acats/acats.sum				 1
   49546 gcc/testsuite/g++/g++.sum					20
  145926 gcc/testsuite/gcc/gcc.sum					60
   74594 gcc/testsuite/gfortran/gfortran.sum				30
    1673 gcc/testsuite/gnat/gnat.sum					 1
    4079 gcc/testsuite/objc/objc.sum					 2
    5287 i386-pc-solaris2.11/libffi/testsuite/libffi.sum		 2
    5020 i386-pc-solaris2.11/libgomp/testsuite/libgomp.sum		 2
    5196 i386-pc-solaris2.11/libjava/testsuite/libjava.sum		 2
   14841 i386-pc-solaris2.11/libstdc++-v3/testsuite/libstdc++.sum	 6
  308523 total

With 300+ ktests total and a goal of spreading them about 128 jobs, we
get 2410 tests for each, and I've used that number as a guideline.  The
third column in the table above gives the number of jobs necessary to
achieve that goal.  One could go even further here considering there are
machines like the Sun Fire T5440 with 4 US-T2 CPUs, yielding 256 cores
and thus 512 jobs, but while single-cpu T1/T2 machines are readily
available, the latter are probably too expensive and I fear that further
increasing parallelism right now might add too much overhead on small
machines.

While using the test counts as estimates worked reasonably well in most
cases, it broke down in two cases: 

* The libstdc++ tests take much more time per test, so I had to increase
  parallelism considerably beyond the first estimate.

* The gcc.c-torture/compile/limits-fnargs.c test case alone takes 16+
  minutes to compile, the test is run 8 times with different
  optimization levels and the number is doubled again by testing both
  the 32 and 64-bit multilibs, so this single test case alone takes more
  than 4 hours to complete.  This is way over the top, so I've halved
  the number of function arguments.  This way, it still takes 3+
  minutes, which is bad, but manageable.

There are still some testsuites that run excessively long (last column
is runtime in seconds):

g++23.sep             6665
libstdc++13.sep       6920
libstdc++12.sep       7191
libjava               7994
gcc16.sep             8212
libstdc++15.sep       8466
libgomp               8615

I've so far refrained from parallelizing the libjava and libgomp
testsuites runs, and some of the libstdc++ test names use very long
common prefixes.  Still, the new state is far better than the current
one, so we might incrementally improve things if libstdc++ or the serial
testsuite runs begin to further dominate the make check times.

The changes below achive this and still try to keep the patterns
manageable.  Here are the make -j<2 * ncpu> -k check times for two tests
(T5220 as above, X7350 is a Sun Fire X4450 with 4 x 2.93 GHz Xeon X7350):

CPU		mainline	parallel

X7350	-j32	 63 min		 56 min
US-T2   -j128	358 min         143 min  

As you can see, on faster CPUs the change is in the noise (or the
parallel test is actually faster), while on the T2 there's a factor of 2.5
speedup.

mail-report.log is identical between both runs.

There's one partially related change included: even before this change,
there could be a strange double summary line in mail-report.log:

                === libstdc++ Summary ===

# of expected passes            13934
# of unexpected failures        8
# of unexpected successes       2
# of expected failures          118
# of unsupported tests          732

                === libstdc++ Summary ===

I could trace this to the fact that currently no libstdc++ ABI baselines
exist for Solaris 2 (being worked on), thus runtest abi.exp produces

[...]
                === libstdc++ Summary for unix ===

Running target unix/-m64
Running /vol/gcc/src/hg/trunk/local/libstdc++-v3/testsuite/libstdc++-abi/abi.exp ...

                === libstdc++ Summary for unix/-m64 ===

                === libstdc++ Summary ===

This situation is likely to occur more often in the future when runtest
is run with a .exp file that doesn't match any tests on a given platform
(like i386.exp on SPARC).  The fix is trivial: only emit the summary
line if any of the test counts is non-zero.

Ok for mainline and the 4.4 and 4.5 branches after some soaktime?

Thanks.
	Rainer

2010-03-07  Rainer Orth  <ro@CeBiTec.Uni-Bielefeld.DE>

	contrib:
	* dg-extract-results.sh (total.awk): Only print combined summary
	for non-zero total.

	gcc:
	* Makefile.in (check_gcc_parallelize): Increase parallelism.
	(check_p_numbers): Extend to 70.

	gcc/cp:
	* Make-lang.in (check_g++_parallelize): Increase parallelism.

	gcc/fortran:
	* Make-lang.in (check_gfortran_parallelize): Increase parallelism.

	gcc/testsuite:
	* gcc.c-torture/compile/limits-fnargs.c (PAR4HALF): Define.
	(func): Replace PAR4 by PAR4HALF.
	(ARG4HALF): Define.
	(caller): Call func with ARG4HALF.

	libstdc++-v3:
	* testsuite/Makefile.am (check_DEJAGNU_normal_targets): Extend to 17.
	Increase parallelism.
	* testsuite/Makefile.in: Regenerate.

[build,testsuite,v3] Increase gcc, g++, gfortran and libstdc++-v3 testsuite parallelism

Commit Message

Comments

Patch