Message ID | alpine.DEB.2.20.13.1607181905370.1419@idea |
---|---|
State | New |
Headers | show |
On Mon, Jul 18, 2016 at 08:39:34PM -0400, Patrick Palka wrote: > One thing that was not clear to me is whether the object file paths > stored in a thin archive are relative or absolute paths. If they are > absolute paths then that would be a problem due to how the build system > moves build directories in between stages (gcc/ -> prev-gcc/ etc). But > it looks like the object file paths are relative to the location of the > archive which is compatible. It's simple. The paths stored in the archive are the paths supplied on the ar command line (*). Supply relative, you'll get relative in the archive and files will be opened relative to the archive directory. *) Well, not quite. Relative paths are adjusted to be relative to the archive directory.
On Tue, Jul 19, 2016 at 2:39 AM, Patrick Palka <patrick@parcs.ath.cx> wrote: > On Mon, 18 Jul 2016, Segher Boessenkool wrote: > >> On Mon, Jul 18, 2016 at 06:35:11AM -0500, Segher Boessenkool wrote: >> > Or, if using GNU ar, you can even use -S, if that helps (after testing >> > for it in configure, of course). >> >> I meant -T. Some day I will learn how to type, promise! > > According to the documentation of GNU ar, > > "gnu ar can optionally create a thin archive, which contains a symbol > index and references to the original copies of the member files of the > archive. This is useful for building libraries for use within a local > build tree, where the relocatable objects are expected to remain > available, and copying the contents of each object would only waste time > and space." > > Since the objects which libbackend.a is composed of remain available > throughout the build process I think it should be safe to make > libbackend.a a thin archive. > > So here's a patch which builds libbackend.a as a thin archive if the > toolchain supports it. The time it takes to rebuild a > --disable-bootstrap tree after touching a single source file is now 7.5s > instead of 35+s -- a much better speedup than when simply eliding the > call to ranlib since the archive is now 1-5MB in size instead of 450MB. > > Instead of changing AR_FLAGS, only the invocation of ar on libbackend.a > is changed because that is by far the largest archive (by a factor of > 20x) and it seems less risky this way. > > One thing that was not clear to me is whether the object file paths > stored in a thin archive are relative or absolute paths. If they are > absolute paths then that would be a problem due to how the build system > moves build directories in between stages (gcc/ -> prev-gcc/ etc). But > it looks like the object file paths are relative to the location of the > archive which is compatible. > > Bootstrapped on x86_64-pc-linux-gnu. Thoughts? I like it. Improving re-build time in my dev tree is very much welcome, and yes, libbackend build time is a big part of it usually (plus of course cc1 link time). Richard. > -- >8 -- > > Subject: [PATCH] Build libbackend.a as a thin archive if possible > > gcc/ChangeLog: > > * configure.ac (thin_archive_support): New variable. AC_SUBST it. > * configure: Regenerate. > * Makefile.in (THIN_ARCHIVE_SUPPORT): New variable. > (USE_THIN_ARCHIVES): New variable. > (libbackend.a): If USE_THIN_ARCHIVES then pass T to ar to build > this archive as a thin archive. > --- > gcc/Makefile.in | 17 +++++++++++++++++ > gcc/configure | 20 ++++++++++++++++++-- > gcc/configure.ac | 13 +++++++++++++ > 3 files changed, 48 insertions(+), 2 deletions(-) > > diff --git a/gcc/Makefile.in b/gcc/Makefile.in > index 0786fa3..15a879b 100644 > --- a/gcc/Makefile.in > +++ b/gcc/Makefile.in > @@ -275,6 +275,17 @@ else > LLINKER = $(LINKER) > endif > > +THIN_ARCHIVE_SUPPORT = @thin_archive_support@ > + > +USE_THIN_ARCHIVES = no > +ifeq ($(THIN_ARCHIVE_SUPPORT),yes) > +ifeq ($(AR_FLAGS),rc) > +ifeq ($(RANLIB_FLAGS),) > +USE_THIN_ARCHIVES = yes > +endif > +endif > +endif > + > # ------------------------------------------- > # Programs which operate on the build machine > # ------------------------------------------- > @@ -1882,8 +1893,14 @@ compilations: $(BACKEND) > # This archive is strictly for the host. > libbackend.a: $(OBJS) > -rm -rf libbackend.a > + @# Build libbackend.a as a thin archive if possible, as doing so > + @# significantly reduces build times. > +ifeq ($(USE_THIN_ARCHIVES),yes) > + $(AR) $(AR_FLAGS)T libbackend.a $(OBJS) > +else > $(AR) $(AR_FLAGS) libbackend.a $(OBJS) > -$(RANLIB) $(RANLIB_FLAGS) libbackend.a > +endif > > libcommon-target.a: $(OBJS-libcommon-target) > -rm -rf libcommon-target.a > diff --git a/gcc/configure b/gcc/configure > index ed44472..81c81b3 100755 > --- a/gcc/configure > +++ b/gcc/configure > @@ -679,6 +679,7 @@ zlibinc > zlibdir > HOST_LIBS > enable_default_ssp > +thin_archive_support > libgcc_visibility > gcc_cv_readelf > gcc_cv_objdump > @@ -18475,7 +18476,7 @@ else > lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 > lt_status=$lt_dlunknown > cat > conftest.$ac_ext <<_LT_EOF > -#line 18478 "configure" > +#line 18479 "configure" > #include "confdefs.h" > > #if HAVE_DLFCN_H > @@ -18581,7 +18582,7 @@ else > lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 > lt_status=$lt_dlunknown > cat > conftest.$ac_ext <<_LT_EOF > -#line 18584 "configure" > +#line 18585 "configure" > #include "confdefs.h" > > #if HAVE_DLFCN_H > @@ -27846,6 +27847,21 @@ $as_echo "#define HAVE_AS_LINE_ZERO 1" >>confdefs.h > > fi > > +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking support for thin archives" >&5 > +$as_echo_n "checking support for thin archives... " >&6; } > +thin_archive_support=no > +echo 'int main (void) { return 0; }' > conftest.c > +if ($AR --version | sed 1q | grep "GNU ar" \ > + && $CC $CFLAGS -c conftest.c \ > + && $AR rcT conftest.a conftest.o \ > + && $CC -o conftest conftest.a) >/dev/null 2>&1; then > + thin_archive_support=yes > +fi > +rm -f conftest.c conftest.o conftest.a conftest > +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $thin_archive_support" >&5 > +$as_echo "$thin_archive_support" >&6; } > + > + > { $as_echo "$as_me:${as_lineno-$LINENO}: checking linker PT_GNU_EH_FRAME support" >&5 > $as_echo_n "checking linker PT_GNU_EH_FRAME support... " >&6; } > gcc_cv_ld_eh_frame_hdr=no > diff --git a/gcc/configure.ac b/gcc/configure.ac > index 086d0fc..63052ba 100644 > --- a/gcc/configure.ac > +++ b/gcc/configure.ac > @@ -4899,6 +4899,19 @@ if test "x$gcc_cv_as_line_zero" = xyes; then > [Define if the assembler won't complain about a line such as # 0 "" 2.]) > fi > > +AC_MSG_CHECKING(support for thin archives) > +thin_archive_support=no > +echo 'int main (void) { return 0; }' > conftest.c > +if ($AR --version | sed 1q | grep "GNU ar" \ > + && $CC $CFLAGS -c conftest.c \ > + && $AR rcT conftest.a conftest.o \ > + && $CC -o conftest conftest.a) >/dev/null 2>&1; then > + thin_archive_support=yes > +fi > +rm -f conftest.c conftest.o conftest.a conftest > +AC_MSG_RESULT($thin_archive_support) > +AC_SUBST(thin_archive_support) > + > AC_MSG_CHECKING(linker PT_GNU_EH_FRAME support) > gcc_cv_ld_eh_frame_hdr=no > if test $in_tree_ld = yes ; then > -- > 2.9.2.321.g3e7e728 >
On Tue, Jul 19, 2016 at 1:20 AM, Richard Biener <richard.guenther@gmail.com> wrote: > On Tue, Jul 19, 2016 at 2:39 AM, Patrick Palka <patrick@parcs.ath.cx> wrote: >> On Mon, 18 Jul 2016, Segher Boessenkool wrote: >> >>> On Mon, Jul 18, 2016 at 06:35:11AM -0500, Segher Boessenkool wrote: >>> > Or, if using GNU ar, you can even use -S, if that helps (after testing >>> > for it in configure, of course). >>> >>> I meant -T. Some day I will learn how to type, promise! >> >> According to the documentation of GNU ar, >> >> "gnu ar can optionally create a thin archive, which contains a symbol >> index and references to the original copies of the member files of the >> archive. This is useful for building libraries for use within a local >> build tree, where the relocatable objects are expected to remain >> available, and copying the contents of each object would only waste time >> and space." >> >> Since the objects which libbackend.a is composed of remain available >> throughout the build process I think it should be safe to make >> libbackend.a a thin archive. >> >> So here's a patch which builds libbackend.a as a thin archive if the >> toolchain supports it. The time it takes to rebuild a >> --disable-bootstrap tree after touching a single source file is now 7.5s >> instead of 35+s -- a much better speedup than when simply eliding the >> call to ranlib since the archive is now 1-5MB in size instead of 450MB. >> >> Instead of changing AR_FLAGS, only the invocation of ar on libbackend.a >> is changed because that is by far the largest archive (by a factor of >> 20x) and it seems less risky this way. >> >> One thing that was not clear to me is whether the object file paths >> stored in a thin archive are relative or absolute paths. If they are >> absolute paths then that would be a problem due to how the build system >> moves build directories in between stages (gcc/ -> prev-gcc/ etc). But >> it looks like the object file paths are relative to the location of the >> archive which is compatible. >> >> Bootstrapped on x86_64-pc-linux-gnu. Thoughts? > > I like it. Improving re-build time in my dev tree is very much > welcome, and yes, > libbackend build time is a big part of it usually (plus of course cc1 > link time). I like it too because a lot is spent of my builds are spent creating libbackend and linking. I usually go and grab coffee during that time due to the disk usage and the kernel likes to grind to a halt during the build at this point. Thanks, Andrew > > Richard. > >> -- >8 -- >> >> Subject: [PATCH] Build libbackend.a as a thin archive if possible >> >> gcc/ChangeLog: >> >> * configure.ac (thin_archive_support): New variable. AC_SUBST it. >> * configure: Regenerate. >> * Makefile.in (THIN_ARCHIVE_SUPPORT): New variable. >> (USE_THIN_ARCHIVES): New variable. >> (libbackend.a): If USE_THIN_ARCHIVES then pass T to ar to build >> this archive as a thin archive. >> --- >> gcc/Makefile.in | 17 +++++++++++++++++ >> gcc/configure | 20 ++++++++++++++++++-- >> gcc/configure.ac | 13 +++++++++++++ >> 3 files changed, 48 insertions(+), 2 deletions(-) >> >> diff --git a/gcc/Makefile.in b/gcc/Makefile.in >> index 0786fa3..15a879b 100644 >> --- a/gcc/Makefile.in >> +++ b/gcc/Makefile.in >> @@ -275,6 +275,17 @@ else >> LLINKER = $(LINKER) >> endif >> >> +THIN_ARCHIVE_SUPPORT = @thin_archive_support@ >> + >> +USE_THIN_ARCHIVES = no >> +ifeq ($(THIN_ARCHIVE_SUPPORT),yes) >> +ifeq ($(AR_FLAGS),rc) >> +ifeq ($(RANLIB_FLAGS),) >> +USE_THIN_ARCHIVES = yes >> +endif >> +endif >> +endif >> + >> # ------------------------------------------- >> # Programs which operate on the build machine >> # ------------------------------------------- >> @@ -1882,8 +1893,14 @@ compilations: $(BACKEND) >> # This archive is strictly for the host. >> libbackend.a: $(OBJS) >> -rm -rf libbackend.a >> + @# Build libbackend.a as a thin archive if possible, as doing so >> + @# significantly reduces build times. >> +ifeq ($(USE_THIN_ARCHIVES),yes) >> + $(AR) $(AR_FLAGS)T libbackend.a $(OBJS) >> +else >> $(AR) $(AR_FLAGS) libbackend.a $(OBJS) >> -$(RANLIB) $(RANLIB_FLAGS) libbackend.a >> +endif >> >> libcommon-target.a: $(OBJS-libcommon-target) >> -rm -rf libcommon-target.a >> diff --git a/gcc/configure b/gcc/configure >> index ed44472..81c81b3 100755 >> --- a/gcc/configure >> +++ b/gcc/configure >> @@ -679,6 +679,7 @@ zlibinc >> zlibdir >> HOST_LIBS >> enable_default_ssp >> +thin_archive_support >> libgcc_visibility >> gcc_cv_readelf >> gcc_cv_objdump >> @@ -18475,7 +18476,7 @@ else >> lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 >> lt_status=$lt_dlunknown >> cat > conftest.$ac_ext <<_LT_EOF >> -#line 18478 "configure" >> +#line 18479 "configure" >> #include "confdefs.h" >> >> #if HAVE_DLFCN_H >> @@ -18581,7 +18582,7 @@ else >> lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 >> lt_status=$lt_dlunknown >> cat > conftest.$ac_ext <<_LT_EOF >> -#line 18584 "configure" >> +#line 18585 "configure" >> #include "confdefs.h" >> >> #if HAVE_DLFCN_H >> @@ -27846,6 +27847,21 @@ $as_echo "#define HAVE_AS_LINE_ZERO 1" >>confdefs.h >> >> fi >> >> +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking support for thin archives" >&5 >> +$as_echo_n "checking support for thin archives... " >&6; } >> +thin_archive_support=no >> +echo 'int main (void) { return 0; }' > conftest.c >> +if ($AR --version | sed 1q | grep "GNU ar" \ >> + && $CC $CFLAGS -c conftest.c \ >> + && $AR rcT conftest.a conftest.o \ >> + && $CC -o conftest conftest.a) >/dev/null 2>&1; then >> + thin_archive_support=yes >> +fi >> +rm -f conftest.c conftest.o conftest.a conftest >> +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $thin_archive_support" >&5 >> +$as_echo "$thin_archive_support" >&6; } >> + >> + >> { $as_echo "$as_me:${as_lineno-$LINENO}: checking linker PT_GNU_EH_FRAME support" >&5 >> $as_echo_n "checking linker PT_GNU_EH_FRAME support... " >&6; } >> gcc_cv_ld_eh_frame_hdr=no >> diff --git a/gcc/configure.ac b/gcc/configure.ac >> index 086d0fc..63052ba 100644 >> --- a/gcc/configure.ac >> +++ b/gcc/configure.ac >> @@ -4899,6 +4899,19 @@ if test "x$gcc_cv_as_line_zero" = xyes; then >> [Define if the assembler won't complain about a line such as # 0 "" 2.]) >> fi >> >> +AC_MSG_CHECKING(support for thin archives) >> +thin_archive_support=no >> +echo 'int main (void) { return 0; }' > conftest.c >> +if ($AR --version | sed 1q | grep "GNU ar" \ >> + && $CC $CFLAGS -c conftest.c \ >> + && $AR rcT conftest.a conftest.o \ >> + && $CC -o conftest conftest.a) >/dev/null 2>&1; then >> + thin_archive_support=yes >> +fi >> +rm -f conftest.c conftest.o conftest.a conftest >> +AC_MSG_RESULT($thin_archive_support) >> +AC_SUBST(thin_archive_support) >> + >> AC_MSG_CHECKING(linker PT_GNU_EH_FRAME support) >> gcc_cv_ld_eh_frame_hdr=no >> if test $in_tree_ld = yes ; then >> -- >> 2.9.2.321.g3e7e728 >>
On 07/19/2016 10:20 AM, Richard Biener wrote: > I like it. Improving re-build time in my dev tree is very much > welcome, and yes, > libbackend build time is a big part of it usually (plus of course cc1 > link time). Since that wasn't an entirely explicit ack, I'll add mine. Thank you for doing this. Bernd
diff --git a/gcc/Makefile.in b/gcc/Makefile.in index 0786fa3..15a879b 100644 --- a/gcc/Makefile.in +++ b/gcc/Makefile.in @@ -275,6 +275,17 @@ else LLINKER = $(LINKER) endif +THIN_ARCHIVE_SUPPORT = @thin_archive_support@ + +USE_THIN_ARCHIVES = no +ifeq ($(THIN_ARCHIVE_SUPPORT),yes) +ifeq ($(AR_FLAGS),rc) +ifeq ($(RANLIB_FLAGS),) +USE_THIN_ARCHIVES = yes +endif +endif +endif + # ------------------------------------------- # Programs which operate on the build machine # ------------------------------------------- @@ -1882,8 +1893,14 @@ compilations: $(BACKEND) # This archive is strictly for the host. libbackend.a: $(OBJS) -rm -rf libbackend.a + @# Build libbackend.a as a thin archive if possible, as doing so + @# significantly reduces build times. +ifeq ($(USE_THIN_ARCHIVES),yes) + $(AR) $(AR_FLAGS)T libbackend.a $(OBJS) +else $(AR) $(AR_FLAGS) libbackend.a $(OBJS) -$(RANLIB) $(RANLIB_FLAGS) libbackend.a +endif libcommon-target.a: $(OBJS-libcommon-target) -rm -rf libcommon-target.a diff --git a/gcc/configure b/gcc/configure index ed44472..81c81b3 100755 --- a/gcc/configure +++ b/gcc/configure @@ -679,6 +679,7 @@ zlibinc zlibdir HOST_LIBS enable_default_ssp +thin_archive_support libgcc_visibility gcc_cv_readelf gcc_cv_objdump @@ -18475,7 +18476,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 18478 "configure" +#line 18479 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -18581,7 +18582,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 18584 "configure" +#line 18585 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -27846,6 +27847,21 @@ $as_echo "#define HAVE_AS_LINE_ZERO 1" >>confdefs.h fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking support for thin archives" >&5 +$as_echo_n "checking support for thin archives... " >&6; } +thin_archive_support=no +echo 'int main (void) { return 0; }' > conftest.c +if ($AR --version | sed 1q | grep "GNU ar" \ + && $CC $CFLAGS -c conftest.c \ + && $AR rcT conftest.a conftest.o \ + && $CC -o conftest conftest.a) >/dev/null 2>&1; then + thin_archive_support=yes +fi +rm -f conftest.c conftest.o conftest.a conftest +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $thin_archive_support" >&5 +$as_echo "$thin_archive_support" >&6; } + + { $as_echo "$as_me:${as_lineno-$LINENO}: checking linker PT_GNU_EH_FRAME support" >&5 $as_echo_n "checking linker PT_GNU_EH_FRAME support... " >&6; } gcc_cv_ld_eh_frame_hdr=no diff --git a/gcc/configure.ac b/gcc/configure.ac index 086d0fc..63052ba 100644 --- a/gcc/configure.ac +++ b/gcc/configure.ac @@ -4899,6 +4899,19 @@ if test "x$gcc_cv_as_line_zero" = xyes; then [Define if the assembler won't complain about a line such as # 0 "" 2.]) fi +AC_MSG_CHECKING(support for thin archives) +thin_archive_support=no +echo 'int main (void) { return 0; }' > conftest.c +if ($AR --version | sed 1q | grep "GNU ar" \ + && $CC $CFLAGS -c conftest.c \ + && $AR rcT conftest.a conftest.o \ + && $CC -o conftest conftest.a) >/dev/null 2>&1; then + thin_archive_support=yes +fi +rm -f conftest.c conftest.o conftest.a conftest +AC_MSG_RESULT($thin_archive_support) +AC_SUBST(thin_archive_support) + AC_MSG_CHECKING(linker PT_GNU_EH_FRAME support) gcc_cv_ld_eh_frame_hdr=no if test $in_tree_ld = yes ; then