Message ID | 1486899181-8210-1-git-send-email-casantos@datacom.ind.br |
---|---|
State | Superseded |
Headers | show |
Carlos, All, On 2017-02-12 09:33 -0200, Carlos Santos spake thusly: > Ensure that the GCC_COLORS environment variable is empty when calling > ccache. When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to > GCC but this flag requires GCC v4.9 or later. Older versions complain > about the unrecognized command line option. > > That behavior is hard-coded in ccache; we would need a patch to either > make it optional or disable it at all. Notice also that the problem is > not detectable in the autobuilders because they redirect the output to > files/pipes and ccache adds the offending flag only when stderr is a > terminal. > > Signed-off-by: Carlos Santos <casantos@datacom.ind.br> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Should go to master. Regards, Yann E. MORIN. > --- > Changes v1->v2 > Fix indentation, use positive logic and improve the commit message, > acording to comments from Thomas Petazzoni and Yann E. MORIN. > > Signed-off-by: Carlos Santos <casantos@datacom.ind.br> > --- > package/Makefile.in | 12 ++++++++++++ > 1 file changed, 12 insertions(+) > > diff --git a/package/Makefile.in b/package/Makefile.in > index 4a3eb26..e5e595a 100644 > --- a/package/Makefile.in > +++ b/package/Makefile.in > @@ -236,6 +236,13 @@ export PERL5LIB=$(HOST_DIR)/usr/lib/perl > > TARGET_MAKE_ENV = PATH=$(BR_PATH) > > +# When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but > +# this flag requires GCC v4.9 or later. Older versions fail, complaining > +# about unrecognized command line option. > +ifeq ($(BR2_TOOLCHAIN_GCC_AT_LEAST_4_9),) > +TARGET_MAKE_ENV += GCC_COLORS="" > +endif > + > TARGET_CONFIGURE_OPTS = \ > $(TARGET_MAKE_ENV) \ > AR="$(TARGET_AR)" \ > @@ -285,6 +292,11 @@ HOST_MAKE_ENV = \ > PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \ > PKG_CONFIG_LIBDIR="$(HOST_DIR)/usr/lib/pkgconfig:$(HOST_DIR)/usr/share/pkgconfig" > > +# Prevent ccache from passing '-fdiagnostics-color' (see above). > +ifeq ($(BR2_HOST_GCC_AT_LEAST_4_9),) > +HOST_MAKE_ENV += GCC_COLORS="" > +endif > + > HOST_CONFIGURE_OPTS = \ > $(HOST_MAKE_ENV) \ > AR="$(HOSTAR)" \ > -- > 2.7.4 >
Carlos, All, On 2017-02-12 09:33 -0200, Carlos Santos spake thusly: > Ensure that the GCC_COLORS environment variable is empty when calling > ccache. When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to > GCC but this flag requires GCC v4.9 or later. Older versions complain > about the unrecognized command line option. So, while fixing my own patch about the issue [0], I noticed that I was not even able to make it fail in the first place... I'm using this defconfig, which uses a gcc-4.7: BR2_x86_i686=y BR2_CCACHE=y BR2_TOOLCHAIN_EXTERNAL=y and Thomas reported he was also unable to reproduce the failure with: BR2_sh=y BR2_sh4a=y BR2_CCACHE=y BR2_TOOLCHAIN_EXTERNAL=y BR2_INIT_NONE=y BR2_SYSTEM_BIN_SH_NONE=y # BR2_PACKAGE_BUSYBOX is not set BR2_PACKAGE_DEVMEM2=y # BR2_TARGET_ROOTFS_TAR is not set which also uses gcc-4.7. Can you provide a reliable way to reproduce the issue, please? Regards, Yann E. MORIN. > That behavior is hard-coded in ccache; we would need a patch to either > make it optional or disable it at all. Notice also that the problem is > not detectable in the autobuilders because they redirect the output to > files/pipes and ccache adds the offending flag only when stderr is a > terminal. > > Signed-off-by: Carlos Santos <casantos@datacom.ind.br> > --- > Changes v1->v2 > Fix indentation, use positive logic and improve the commit message, > acording to comments from Thomas Petazzoni and Yann E. MORIN. > > Signed-off-by: Carlos Santos <casantos@datacom.ind.br> > --- > package/Makefile.in | 12 ++++++++++++ > 1 file changed, 12 insertions(+) > > diff --git a/package/Makefile.in b/package/Makefile.in > index 4a3eb26..e5e595a 100644 > --- a/package/Makefile.in > +++ b/package/Makefile.in > @@ -236,6 +236,13 @@ export PERL5LIB=$(HOST_DIR)/usr/lib/perl > > TARGET_MAKE_ENV = PATH=$(BR_PATH) > > +# When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but > +# this flag requires GCC v4.9 or later. Older versions fail, complaining > +# about unrecognized command line option. > +ifeq ($(BR2_TOOLCHAIN_GCC_AT_LEAST_4_9),) > +TARGET_MAKE_ENV += GCC_COLORS="" > +endif > + > TARGET_CONFIGURE_OPTS = \ > $(TARGET_MAKE_ENV) \ > AR="$(TARGET_AR)" \ > @@ -285,6 +292,11 @@ HOST_MAKE_ENV = \ > PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \ > PKG_CONFIG_LIBDIR="$(HOST_DIR)/usr/lib/pkgconfig:$(HOST_DIR)/usr/share/pkgconfig" > > +# Prevent ccache from passing '-fdiagnostics-color' (see above). > +ifeq ($(BR2_HOST_GCC_AT_LEAST_4_9),) > +HOST_MAKE_ENV += GCC_COLORS="" > +endif > + > HOST_CONFIGURE_OPTS = \ > $(HOST_MAKE_ENV) \ > AR="$(HOSTAR)" \ > -- > 2.7.4 >
> From: "Yann E. MORIN" <yann.morin.1998@free.fr> > To: "Carlos Santos" <casantos@datacom.ind.br> > Cc: buildroot@buildroot.org, "Thomas Petazzoni" <thomas.petazzoni@free-electrons.com>, arnout@mind.be, > gustavo@zacarias.com.ar > Sent: Sunday, February 12, 2017 12:14:25 PM > Subject: Re: [PATCH v2] infra/ccache: avoid color diagnostics with GCC older than v4.9 > Carlos, All, > > On 2017-02-12 09:33 -0200, Carlos Santos spake thusly: >> Ensure that the GCC_COLORS environment variable is empty when calling >> ccache. When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to >> GCC but this flag requires GCC v4.9 or later. Older versions complain >> about the unrecognized command line option. > > So, while fixing my own patch about the issue [0], I noticed that I was > not even able to make it fail in the first place... > > I'm using this defconfig, which uses a gcc-4.7: > > BR2_x86_i686=y > BR2_CCACHE=y > BR2_TOOLCHAIN_EXTERNAL=y > > and Thomas reported he was also unable to reproduce the failure with: > > BR2_sh=y > BR2_sh4a=y > BR2_CCACHE=y > BR2_TOOLCHAIN_EXTERNAL=y > BR2_INIT_NONE=y > BR2_SYSTEM_BIN_SH_NONE=y > # BR2_PACKAGE_BUSYBOX is not set > BR2_PACKAGE_DEVMEM2=y > # BR2_TARGET_ROOTFS_TAR is not set > > which also uses gcc-4.7. > > Can you provide a reliable way to reproduce the issue, please? What does "echo $GCC_COLORS" say? And remember that you must not redirect the output to a file or pipe. Carlos Santos (Casantos) - DATACOM, P&D “Something must be done. This is something. Therefore we must do it.” (from the BBC series “Yes, Prime Minister”, “Power to the people”)
Carlos, All, On 2017-02-12 12:55 -0200, Carlos Santos spake thusly: > > From: "Yann E. MORIN" <yann.morin.1998@free.fr> > > To: "Carlos Santos" <casantos@datacom.ind.br> > > Cc: buildroot@buildroot.org, "Thomas Petazzoni" <thomas.petazzoni@free-electrons.com>, arnout@mind.be, > > gustavo@zacarias.com.ar > > Sent: Sunday, February 12, 2017 12:14:25 PM > > Subject: Re: [PATCH v2] infra/ccache: avoid color diagnostics with GCC older than v4.9 > > > Carlos, All, > > > > On 2017-02-12 09:33 -0200, Carlos Santos spake thusly: > >> Ensure that the GCC_COLORS environment variable is empty when calling > >> ccache. When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to > >> GCC but this flag requires GCC v4.9 or later. Older versions complain > >> about the unrecognized command line option. > > > > So, while fixing my own patch about the issue [0], I noticed that I was > > not even able to make it fail in the first place... > > > > I'm using this defconfig, which uses a gcc-4.7: > > > > BR2_x86_i686=y > > BR2_CCACHE=y > > BR2_TOOLCHAIN_EXTERNAL=y > > > > and Thomas reported he was also unable to reproduce the failure with: > > > > BR2_sh=y > > BR2_sh4a=y > > BR2_CCACHE=y > > BR2_TOOLCHAIN_EXTERNAL=y > > BR2_INIT_NONE=y > > BR2_SYSTEM_BIN_SH_NONE=y > > # BR2_PACKAGE_BUSYBOX is not set > > BR2_PACKAGE_DEVMEM2=y > > # BR2_TARGET_ROOTFS_TAR is not set > > > > which also uses gcc-4.7. > > > > Can you provide a reliable way to reproduce the issue, please? > > What does "echo $GCC_COLORS" say? And remember that you must not > redirect the output to a file or pipe. The GCC_COLORS are the default as dopcumented in the man page for gcc-5. $ cat defconfig BR2_x86_i686=y BR2_CCACHE=y BR2_TOOLCHAIN_EXTERNAL=y BR2_PACKAGE_LIBCEC=y $ export GCC_COLORS='error=01;31:warning=01;35:note=01;36:caret=01;32:locus=01:quote=01' $ make BR2_DEFCONFIG=$(pwd)/defconfig defconfig $ make busybox libcec And the build does not fail... :-/ So I did a few other tentative: $ rm -rf ~/.buildroot-ccache/*/ $ echo log_file = /home/ymorin/meh >> ~/.buildroot-ccache/ccache.conf $ touch meh.c $ ./host/usr/bin/i686-pc-linux-gnu-gcc -c -o meh.o meh.c $ ~/meh [2017-02-12T18:19:31.365731 7668 ] === CCACHE 3.3.3 STARTED ========================================= [2017-02-12T18:19:31.365731 7668 ] Config: (environment) base_dir = /home/ymorin/dev/buildroot/O [2017-02-12T18:19:31.365731 7668 ] Config: (default) cache_dir = /home/ymorin/.buildroot-ccache [2017-02-12T18:19:31.365731 7668 ] Config: (default) cache_dir_levels = 2 [2017-02-12T18:19:31.365731 7668 ] Config: (default) compiler = [2017-02-12T18:19:31.365731 7668 ] Config: (default) compiler_check = mtime [2017-02-12T18:19:31.365731 7668 ] Config: (default) compression = false [2017-02-12T18:19:31.365731 7668 ] Config: (default) compression_level = 6 [2017-02-12T18:19:31.365731 7668 ] Config: (default) cpp_extension = [2017-02-12T18:19:31.365731 7668 ] Config: (default) direct_mode = true [2017-02-12T18:19:31.365731 7668 ] Config: (default) disable = false [2017-02-12T18:19:31.365731 7668 ] Config: (default) extra_files_to_hash = [2017-02-12T18:19:31.365731 7668 ] Config: (default) hard_link = false [2017-02-12T18:19:31.365731 7668 ] Config: (default) hash_dir = true [2017-02-12T18:19:31.365731 7668 ] Config: (default) ignore_headers_in_manifest = [2017-02-12T18:19:31.365731 7668 ] Config: (default) keep_comments_cpp = false [2017-02-12T18:19:31.365731 7668 ] Config: (default) limit_multiple = 0.8 [2017-02-12T18:19:31.365731 7668 ] Config: (/home/ymorin/.buildroot-ccache/ccache.conf) log_file = /home/ymorin/meh [2017-02-12T18:19:31.365731 7668 ] Config: (default) max_files = 0 [2017-02-12T18:19:31.365731 7668 ] Config: (/home/ymorin/.buildroot-ccache/ccache.conf) max_size = 5.0G [2017-02-12T18:19:31.365731 7668 ] Config: (default) path = [2017-02-12T18:19:31.365731 7668 ] Config: (default) prefix_command = [2017-02-12T18:19:31.365731 7668 ] Config: (default) prefix_command_cpp = [2017-02-12T18:19:31.365731 7668 ] Config: (default) read_only = false [2017-02-12T18:19:31.365731 7668 ] Config: (default) read_only_direct = false [2017-02-12T18:19:31.365731 7668 ] Config: (default) recache = false [2017-02-12T18:19:31.365731 7668 ] Config: (default) run_second_cpp = true [2017-02-12T18:19:31.365731 7668 ] Config: (default) sloppiness = [2017-02-12T18:19:31.365731 7668 ] Config: (default) stats = true [2017-02-12T18:19:31.365731 7668 ] Config: (default) temporary_dir = [2017-02-12T18:19:31.365731 7668 ] Config: (default) umask = [2017-02-12T18:19:31.365731 7668 ] Config: (default) unify = false [2017-02-12T18:19:31.365999 7668 ] Command line: /home/ymorin/dev/buildroot/O/host/usr/bin/ccache /home/ymorin/dev/buildroot/O/host/opt/ext-toolchain/bin/i686-pc-linux-gnu-gcc --sysroot /home/ymorin/dev/buildroot/O/host/usr/i686-buildroot-linux-gnu/sysroot -march=i686 -c -o meh.o meh.c [2017-02-12T18:19:31.366080 7668 ] Hostname: gourin [2017-02-12T18:19:31.366157 7668 ] Working directory: /home/ymorin/dev/buildroot/O [2017-02-12T18:19:31.366280 7668 ] Automatically enabling colors [2017-02-12T18:19:31.366312 7668 ] Source file: meh.c [2017-02-12T18:19:31.366331 7668 ] Object file: meh.o [2017-02-12T18:19:31.366367 7668 ] Trying direct lookup [2017-02-12T18:19:31.366441 7668 ] Looking for object file hash in /home/ymorin/.buildroot-ccache/e/2/095355fd431a7dfa30fcd2c18aa6c7-359.manifest [2017-02-12T18:19:31.366468 7668 ] No such manifest file [2017-02-12T18:19:31.366486 7668 ] Did not find object file hash in manifest [2017-02-12T18:19:31.366695 7668 ] Running preprocessor [2017-02-12T18:19:31.366720 7668 ] Executing /home/ymorin/dev/buildroot/O/host/opt/ext-toolchain/bin/i686-pc-linux-gnu-gcc --sysroot host/usr/i686-buildroot-linux-gnu/sysroot -march=i686 -fdiagnostics-color -E meh.c [2017-02-12T18:19:31.368786 7668 ] Preprocessor gave exit status 1 [2017-02-12T18:19:31.368834 7668 ] Failed; falling back to running the real compiler [2017-02-12T18:19:31.368847 7668 ] Executing /home/ymorin/dev/buildroot/O/host/opt/ext-toolchain/bin/i686-pc-linux-gnu-gcc --sysroot /home/ymorin/dev/buildroot/O/host/usr/i686-buildroot-linux-gnu/sysroot -march=i686 -c -o meh.o meh.c [2017-02-12T18:19:31.368943 7668 ] lockfile_acquire: symlink /home/ymorin/.buildroot-ccache/c/stats.lock: No such file or directory [2017-02-12T18:19:31.369031 7668 ] Acquired lock /home/ymorin/.buildroot-ccache/c/stats.lock [2017-02-12T18:19:31.369147 7668 ] Releasing lock /home/ymorin/.buildroot-ccache/c/stats.lock [2017-02-12T18:19:31.369163 7668 ] Unlink /home/ymorin/.buildroot-ccache/c/stats.lock [2017-02-12T18:19:31.369189 7668 ] Result: preprocessor error Hope you can make something out of this. BTW, I never trusted ccache; it only ever brought me more headaches on which I spent more time than it ever saved me... Regards, Yann E. MORIN.
Hello, On Sun, 12 Feb 2017 12:55:37 -0200 (BRST), Carlos Santos wrote: > > Can you provide a reliable way to reproduce the issue, please? > > What does "echo $GCC_COLORS" say? thomas@skate:~$ echo $GCC_COLORS thomas@skate:~$ So the issue only arise when GCC_COLORS is defined non-empty in the user's environment? > And remember that you must not redirect the output to a file or pipe. I just ran "make" without redirecting the output to a file or pipe. Thomas
On 2017-02-12 18:22 +0100, Thomas Petazzoni spake thusly: > Hello, > > On Sun, 12 Feb 2017 12:55:37 -0200 (BRST), Carlos Santos wrote: > > > > Can you provide a reliable way to reproduce the issue, please? > > > > What does "echo $GCC_COLORS" say? > > thomas@skate:~$ echo $GCC_COLORS > > thomas@skate:~$ > > So the issue only arise when GCC_COLORS is defined non-empty in the > user's environment? Well, from the point of view of ccache, any value whould have triggered the issue, because ccache only ever checks if the variables exists in the environment. But see my other reply: even with a valid value, I could not trigger the bug either... > > And remember that you must not redirect the output to a file or pipe. > I just ran "make" without redirecting the output to a file or pipe. Same as I did. Regards, Yann E. MORIN.
> From: "Yann E. MORIN" <yann.morin.1998@free.fr> > To: "Carlos Santos" <casantos@datacom.ind.br> > Cc: buildroot@buildroot.org, "Thomas Petazzoni" <thomas.petazzoni@free-electrons.com>, arnout@mind.be, > gustavo@zacarias.com.ar > Sent: Sunday, February 12, 2017 12:14:25 PM > Subject: Re: [PATCH v2] infra/ccache: avoid color diagnostics with GCC older than v4.9 > Carlos, All, [...] > > Can you provide a reliable way to reproduce the issue, please? > > Regards, > Yann E. MORIN. Sorry, I forgot so send this yesterday. Here is a reliable reproducer: $ cat defconfig BR2_powerpc=y BR2_powerpc_8548=y BR2_CCACHE=y BR2_TOOLCHAIN_EXTERNAL=y BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/powerpc-ctng_e500v2-linux-gnuspe.tar.xz" BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX="powerpc-ctng_e500v2-linux-gnuspe" BR2_TOOLCHAIN_EXTERNAL_GCC_4_7=y BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_12=y BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y BR2_TOOLCHAIN_EXTERNAL_CXX=y BR2_PACKAGE_OPENSSL=y $ $ echo $GCC_COLORS error=01;31:warning=01;35:note=01;36:caret=01;32:locus=01:quote=01 $ $ make V=1 BR2_JLEVEL=1 openssl-dirclean openssl [...] /work/ccache-colors/host/usr/bin/powerpc-ctng_e500v2-linux-gnuspe-gcc -I. -I.. -I../include -fPIC -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DB_ENDIAN -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -mabi=spe -mfloat-gprs=double -Wa,-me500x2 -Os -Wall -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DVPAES_ASM -c -o ppccpuid.o ppccpuid.s powerpc-ctng_e500v2-linux-gnuspe-gcc: error: unrecognized command line option '-fdiagnostics-color' <builtin>: recipe for target 'ppccpuid.o' failed make[3]: *** [ppccpuid.o] Error 1 Makefile:287: recipe for target 'build_crypto' failed make[2]: *** [build_crypto] Error 1 package/pkg-generic.mk:216: recipe for target '/work/ccache-colors/build/openssl-1.0.2k/.stamp_built' failed make[1]: *** [/work/ccache-colors/build/openssl-1.0.2k/.stamp_built] Error 2 Makefile:16: recipe for target '_all' failed make: *** [_all] Error 2 Carlos Santos (Casantos) - DATACOM, P&D “Something must be done. This is something. Therefore we must do it.” (from the BBC series “Yes, Prime Minister”, “Power to the people”)
Carlos, All, On 2017-02-13 20:41 -0200, Carlos Santos spake thusly: > Sorry, I forgot so send this yesterday. Here is a reliable reproducer: > > $ cat defconfig > BR2_powerpc=y > BR2_powerpc_8548=y > BR2_CCACHE=y > BR2_TOOLCHAIN_EXTERNAL=y > BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y > BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/powerpc-ctng_e500v2-linux-gnuspe.tar.xz" > BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX="powerpc-ctng_e500v2-linux-gnuspe" > BR2_TOOLCHAIN_EXTERNAL_GCC_4_7=y > BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_12=y > BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y > BR2_TOOLCHAIN_EXTERNAL_CXX=y > BR2_PACKAGE_OPENSSL=y > $ > $ echo $GCC_COLORS > error=01;31:warning=01;35:note=01;36:caret=01;32:locus=01:quote=01 > $ > $ make V=1 BR2_JLEVEL=1 openssl-dirclean openssl Indeed, I could reproduce. Thanks. So, I think Arnout is right in saying we should just globally unexport GCC_COLORS and be done with that. Regards, Yann E. MORIN.
diff --git a/package/Makefile.in b/package/Makefile.in index 4a3eb26..e5e595a 100644 --- a/package/Makefile.in +++ b/package/Makefile.in @@ -236,6 +236,13 @@ export PERL5LIB=$(HOST_DIR)/usr/lib/perl TARGET_MAKE_ENV = PATH=$(BR_PATH) +# When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but +# this flag requires GCC v4.9 or later. Older versions fail, complaining +# about unrecognized command line option. +ifeq ($(BR2_TOOLCHAIN_GCC_AT_LEAST_4_9),) +TARGET_MAKE_ENV += GCC_COLORS="" +endif + TARGET_CONFIGURE_OPTS = \ $(TARGET_MAKE_ENV) \ AR="$(TARGET_AR)" \ @@ -285,6 +292,11 @@ HOST_MAKE_ENV = \ PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \ PKG_CONFIG_LIBDIR="$(HOST_DIR)/usr/lib/pkgconfig:$(HOST_DIR)/usr/share/pkgconfig" +# Prevent ccache from passing '-fdiagnostics-color' (see above). +ifeq ($(BR2_HOST_GCC_AT_LEAST_4_9),) +HOST_MAKE_ENV += GCC_COLORS="" +endif + HOST_CONFIGURE_OPTS = \ $(HOST_MAKE_ENV) \ AR="$(HOSTAR)" \