| Message ID | ZpFHnjCA1apPlsa2@tucnak |
|---|---|
| State | New |
| Headers | show |
| Series | varasm: Add support for emitting binary data with the new gas .base64 directive | expand |
On Fri, 12 Jul 2024, Jakub Jelinek wrote: > Hi! > > Nick has implemented a new .base64 directive in gas (to be shipped in > the upcoming binutils 2.43; big thanks for that). > See https://sourceware.org/bugzilla/show_bug.cgi?id=31964 > > The following patch adjusts default_elf_asm_output_ascii (i.e. > ASM_OUTPUT_ASCII elfos.h implementation) to use it if it detects binary > data and gas supports it. > > Without this patch, we emit stuff like: > .string "\177ELF\002\001\001\003" > .string "" > .string "" > .string "" > .string "" > .string "" > .string "" > .string "" > .string "\002" > .string ">" > ... > .string "\324\001\236 0FS\202\002E\n0@\203\004\005&\202\021\337)\021\203C\020A\300\220I\004\t\b\206(\234\0132l\004b\300\bK\006\220$0\303\020P$\233\211\002D\f" > etc., with this patch more compact > .base64 "f0VMRgIBAQMAAAAAAAAAAAIAPgABAAAAABf3AAAAAABAAAAAAAAAAACneB0AAAAAAAAAAEAAOAAOAEAALAArAAYAAAAEAAAAQAAAAAAAAABAAEAAAAAAAEAAQAAAAAAAEAMAAAAAAAAQAwAAAAAAAAgAAAAAAAAAAwAAAAQAAABQAwAAAAAAAFADQAAAAAAAUANAAAAAAAAcAAAAAAAAABwAAAAAAAAAAQAAAAAAAAABAAAABAAAAAAAAAAAAAAAAABAAAAAAAAAAEAAAAAAADBwOQAAAAAAMHA5AAAAAAAAEAAAAAAAAAEAAAAFAAAAAIA5AAAAAAAAgHkAAAAA" > .base64 "AACAeQAAAAAAxSSgAgAAAADFJKACAAAAAAAQAAAAAAAAAQAAAAQAAAAAsNkCAAAAAACwGQMAAAAAALAZAwAAAADMtc0AAAAAAMy1zQAAAAAAABAAAAAAAAABAAAABgAAAGhmpwMAAAAAaHbnAwAAAABoducDAAAAAOAMAQAAAAAA4MEeAAAAAAAAEAAAAAAAAAIAAAAGAAAAkH2nAwAAAACQjecDAAAAAJCN5wMAAAAAQAIAAAAAAABAAgAAAAAAAAgAAAAAAAAABAAAAAQAAABwAwAAAAAAAHADQAAAAAAAcANAAAAAAABAAAAAAAAAAEAAAAAAAAAACAAAAAAA" > .base64 "AAAEAAAABAAAALADAAAAAAAAsANAAAAAAACwA0AAAAAAACAAAAAAAAAAIAAAAAAAAAAEAAAAAAAAAAcAAAAEAAAAaGanAwAAAABoducDAAAAAGh25wMAAAAAAAAAAAAAAAAQAAAAAAAAAAgAAAAAAAAAU+V0ZAQAAABwAwAAAAAAAHADQAAAAAAAcANAAAAAAABAAAAAAAAAAEAAAAAAAAAACAAAAAAAAABQ5XRkBAAAAAw/WAMAAAAADD+YAwAAAAAMP5gDAAAAAPy7CgAAAAAA/LsKAAAAAAAEAAAAAAAAAFHldGQGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" > .base64 "AAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAUuV0ZAQAAABoZqcDAAAAAGh25wMAAAAAaHbnAwAAAACYGQAAAAAAAJgZAAAAAAAAAQAAAAAAAAAvbGliNjQvbGQtbGludXgteDg2LTY0LnNvLjIAAAAAAAQAAAAwAAAABQAAAEdOVQACgADABAAAAAEAAAAAAAAAAQABwAQAAAAJAAAAAAAAAAIAAcAEAAAAAwAAAAAAAAAEAAAAEAAAAAEAAABHTlUAAAAAAAMAAAACAAAAAAAAAAOAAACsqAAAgS0AAOJWAAAjNwAAXjAAAAAAAAAAAAAAF1gAAHsxAABBBwAA" > .base64 "G0kAALGmAACwoAAAAAAAAAAAAACQhAAAAAAAAOw1AACNYgAAAAAAAFQoAAAAAAAAx3UAALZAAAAAAAAAiIUAALGeAABBlAAAWEsAAPmRAACmOgAAAAAAADh3AAAAAAAAlCAAAAAAAABymgAAaosAAMIjAAAKMQAAMkIAADU0AAAAAAAA5ZwAAAAAAAAAAAAAAAAAAFIdAAAIGQAAAAAAAMFbAAAoTQAAGDcAAIRgAAA6HgAAlxwAAAAAAADOlgAAAAAAAEhPAAARiwAAMGgAAOVtAADMFgAAAAAAAAAAAACrjgAAYl4AACZVAAA/HgAAAAAAAAAAAABqPwAAAAAA" > The patch attempts to juggle between readability and compactness, so > if it detects some hunk of the initializer that would be shorter to be > emitted as .string/.ascii directive, it does so, but if it previously > used .base64 directive it switches mode only if there is a 16+ char > ASCII-ish string. > > On my #embed testcase from yesterday > unsigned char a[] = { > #embed "cc1plus" > }; > without this patch it emits 2.4GB of assembly, while with this > patch 649M. > Compile times (trunk, so yes,rtl,extra checking) are: > time ./xgcc -B ./ -S -std=c23 -O2 embed-11.c > > real 0m13.647s > user 0m7.157s > sys 0m2.597s > time ./xgcc -B ./ -c -std=c23 -O2 embed-11.c > > real 0m28.649s > user 0m26.653s > sys 0m1.958s > without the patch and > time ./xgcc -B ./ -S -std=c23 -O2 embed-11.c > > real 0m4.283s > user 0m2.288s > sys 0m0.859s > time ./xgcc -B ./ -c -std=c23 -O2 embed-11.c > > real 0m6.888s > user 0m5.876s > sys 0m1.002s > with the patch, so that feels like significant improvement. > The resulting embed-11.o is identical between the two ways of expressing > the mostly binary data in the assembly. But note that there are portions > like: > .base64 "nAAAAAAAAAAvZRcAIgAOAFAzMwEAAAAABgAAAAAAAACEQBgAEgAOAFBHcAIAAAAA7AAAAAAAAAAAX19nbXB6X2dldF9zaQBtcGZyX3NldF9zaV8yZXhwAG1wZnJfY29zaABtcGZyX3RhbmgAbXBmcl9zZXRfbmFuAG1wZnJfc3ViAG1wZnJfdGFuAG1wZnJfc3RydG9mcgBfX2dtcHpfc3ViX3VpAF9fZ21wX2dldF9tZW1vcnlfZnVuY3Rpb25zAF9fZ21wel9zZXRfdWkAbXBmcl9wb3cAX19nbXB6X3N1YgBfX2dtcHpfZml0c19zbG9uZ19wAG1wZnJfYXRh" > .base64 "bjIAX19nbXB6X2RpdmV4YWN0AG1wZnJfc2V0X2VtaW4AX19nbXB6X3NldABfX2dtcHpfbXVsAG1wZnJfY2xlYXIAbXBmcl9sb2cAbXBmcl9hdGFuaABfX2dtcHpfc3dhcABtcGZyX2FzaW5oAG1wZnJfYXNpbgBtcGZyX2NsZWFycwBfX2dtcHpfbXVsXzJleHAAX19nbXB6X2FkZG11bABtcGZyX3NpbmgAX19nbXB6X2FkZF91aQBfX2dtcHFfY2xlYXIAX19nbW9uX3N0YXJ0X18AbXBmcl9hY29zAG1wZnJfc2V0X2VtYXgAbXBmcl9jb3MAbXBmcl9zaW4A" > .string "__gmpz_ui_pow_ui" > .string "mpfr_get_str" > .string "mpfr_acosh" > .string "mpfr_sub_ui" > .string "__gmpq_set_ui" > .string "mpfr_set_inf" > ... > .string "GLIBC_2.14" > .string "GLIBC_2.11" > .base64 "AAABAAIAAQADAAMAAwADAAMAAwAEAAUABgADAAEAAQADAAMABwABAAEAAwADAAMAAwAIAAEAAwADAAEAAwABAAMAAwABAAMAAQADAAMAAwADAAMAAwADAAYAAwADAAEAAQAIAAMAAwADAAMAAwABAAMAAQADAAMAAQABAAEAAwAIAAEAAwADAAEAAwABAAMAAQADAAEABgADAAMAAQAHAAMAAwADAAMAAwABAAMAAQABAAMAAwADAAkAAQABAAEAAwAKAAEAAwADAAMAAQABAAMAAwALAAEAAwADAAEAAQADAAMAAwABAAMAAwABAAEAAwADAAMABwABAAMAAwAB" > .base64 "AAEAAwADAAEAAwABAAMAAQADAAMAAwADAAEAAQABAAEAAwADAAMAAQABAAEAAQABAAEAAQADAAMAAwADAAMAAQABAAwAAwADAA0AAwADAAMAAwADAAEAAQADAAMAAQABAAMAAwADAAEAAwADAAEAAwAIAAMAAwADAAMABgABAA4ACwAGAAEAAQADAAEAAQADAAEAAwABAAMAAwABAAEAAwABAAMAAwABAAEAAwADAAMAAwABAAMAAQABAAEAAQABAAMADwABAAMAAQADAAMAAwABAAEAAQAIAAEADAADAAMAAQABAAMAAwADAAEAAQABAAEAAQADAAEAAwADAAEA" > .base64 "AwABAAMAAQADAAMAAQABAAEAAwADAAMAAwADAAMAAQADAAMACAAQAA8AAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQA=" > so it isn't all just totally unreadable stuff. > > Tested on top of the #embed patchset (just to get the above > data) and then with full bootstrap/regtest on top of vanilla trunk, > on x86_64-linux against trunk binutils prepended into $PATH (HAVE_GAS_BASE64 > is true) and i686-linux (against older binutils, HAVE_GAS_BASE64 not > defined), ok for trunk? OK. Do we want to have a command-line switch controlling the use of .base64? It might be the 16+ char limit is too large for the readability purpose and/or with wide-strings, etc. .base64 is unwanted to some people at least when debugging? (does gas support sth like .wstring or UTF-8/16/32 input?) I don't have a good suggestion for the flag name, a quick scan for assembler related options doesn't reveal a nice "pattern"; -f[no-]base64-in-asm? Thanks, Richard. > 2024-07-12 Jakub Jelinek <jakub@redhat.com> > > * configure.ac (HAVE_GAS_BASE64): New check. > * config/elfos.h (BASE64_ASM_OP): Define if HAVE_GAS_BASE64 is > defined. > * varasm.cc (assemble_string): Bump maximum from 2000 to 16384 if > BASE64_ASM_OP is defined. > (default_elf_asm_output_limited_string): Emit opening '"' together > with STRING_ASM_OP. > (default_elf_asm_output_ascii): Use BASE64_ASM_OP if defined and > beneficial. Remove UB when last_null is NULL. > * configure: Regenerate. > * config.in: Regenerate. > > --- gcc/configure.ac.jj 2024-07-01 11:28:22.435241431 +0200 > +++ gcc/configure.ac 2024-07-12 10:28:38.038804308 +0200 > @@ -3054,6 +3054,11 @@ case "${target}" in > ;; > esac > > +gcc_GAS_CHECK_FEATURE([.base64], gcc_cv_as_base64,, > + [ .section .rodata > + .base64 "Tm9uIHB1ZG9yIGVzdCBuaWwgc2NpcmUsIHB1ZG9yIG5pbCBkaXNjZXJlIHZlbGxlLgo="],, > +[AC_DEFINE(HAVE_GAS_BASE64, 1, [Define if your assembler supports .base64.])]) > + > # gnu_indirect_function type is an extension proposed at > # http://groups.google/com/group/generic-abi/files. It allows dynamic runtime > # selection of function implementation > --- gcc/config/elfos.h.jj 2024-01-03 11:51:43.966558606 +0100 > +++ gcc/config/elfos.h 2024-07-12 10:40:59.721200959 +0200 > @@ -444,6 +444,10 @@ see the files COPYING3 and COPYING.RUNTI > > #define STRING_ASM_OP "\t.string\t" > > +#ifdef HAVE_GAS_BASE64 > +#define BASE64_ASM_OP "\t.base64\t" > +#endif > + > /* The routine used to output NUL terminated strings. We use a special > version of this for most svr4 targets because doing so makes the > generated assembly code more compact (and thus faster to assemble) > --- gcc/varasm.cc.jj 2024-07-08 11:00:54.723980651 +0200 > +++ gcc/varasm.cc 2024-07-12 13:28:39.934619478 +0200 > @@ -2101,7 +2101,19 @@ void > assemble_string (const char *p, int size) > { > int pos = 0; > +#if defined(BASE64_ASM_OP) \ > + && BITS_PER_UNIT == 8 \ > + && CHAR_BIT == 8 \ > + && 'A' == 65 \ > + && 'a' == 97 \ > + && '0' == 48 \ > + && '+' == 43 \ > + && '/' == 47 \ > + && '=' == 61 > + int maximum = 16384; > +#else > int maximum = 2000; > +#endif > > /* If the string is very long, split it up. */ > > @@ -8469,8 +8481,7 @@ default_elf_asm_output_limited_string (F > int escape; > unsigned char c; > > - fputs (STRING_ASM_OP, f); > - putc ('"', f); > + fputs (STRING_ASM_OP "\"", f); > while (*s != '\0') > { > c = *s; > @@ -8504,9 +8515,11 @@ default_elf_asm_output_ascii (FILE *f, c > { > const char *limit = s + len; > const char *last_null = NULL; > + const char *last_base64 = s; > unsigned bytes_in_chunk = 0; > unsigned char c; > int escape; > + bool prev_base64 = false; > > for (; s < limit; s++) > { > @@ -8519,7 +8532,7 @@ default_elf_asm_output_ascii (FILE *f, c > bytes_in_chunk = 0; > } > > - if (s > last_null) > + if ((uintptr_t) s > (uintptr_t) last_null) > { > for (p = s; p < limit && *p != '\0'; p++) > continue; > @@ -8528,6 +8541,104 @@ default_elf_asm_output_ascii (FILE *f, c > else > p = last_null; > > +#if defined(BASE64_ASM_OP) \ > + && BITS_PER_UNIT == 8 \ > + && CHAR_BIT == 8 \ > + && 'A' == 65 \ > + && 'a' == 97 \ > + && '0' == 48 \ > + && '+' == 43 \ > + && '/' == 47 \ > + && '=' == 61 > + if (s >= last_base64) > + { > + unsigned cnt = 0; > + const char *t; > + for (t = s; t < limit && (t - s) < (long) ELF_STRING_LIMIT - 1; t++) > + { > + if (t == p && t != s) > + { > + if (cnt <= (t - s + 1 + 2) / 3 * 4 > + && (!prev_base64 || (t - s) >= 16) > + && ((t - s) > 1 || cnt <= 2)) > + { > + last_base64 = p; > + goto no_base64; > + } > + } > + c = *t; > + escape = ELF_ASCII_ESCAPES[c]; > + switch (escape) > + { > + case 0: > + ++cnt; > + break; > + case 1: > + if (c == 0) > + cnt += 2 + strlen (STRING_ASM_OP) + 1; > + else > + cnt += 4; > + break; > + default: > + cnt += 2; > + break; > + } > + } > + if (cnt > (t - s + 2) / 3 * 4 && (t - s) >= 3) > + { > + if (bytes_in_chunk > 0) > + { > + putc ('\"', f); > + putc ('\n', f); > + bytes_in_chunk = 0; > + } > + > + unsigned char buf[(ELF_STRING_LIMIT + 2) / 3 * 4 + 3]; > + unsigned j = 0; > + static const char base64_enc[] = > + "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz" > + "0123456789+/"; > + > + fputs (BASE64_ASM_OP "\"", f); > + while (s < t) > + { > + unsigned char a = *s; > + unsigned char b = 0, c = 0; > + if (s < t - 1) > + b = s[1]; > + if (s < t - 2) > + c = s[2]; > + unsigned long v = ((((unsigned long) a) << 16) > + | (((unsigned long) b) << 8) > + | c); > + buf[j++] = base64_enc[(v >> 18) & 63]; > + buf[j++] = base64_enc[(v >> 12) & 63]; > + buf[j++] = base64_enc[(v >> 6) & 63]; > + buf[j++] = base64_enc[v & 63]; > + if (s >= t - 2) > + { > + buf[j - 1] = '='; > + if (s >= t - 1) > + buf[j - 2] = '='; > + break; > + } > + s += 3; > + } > + memcpy (buf + j, "\"\n", 3); > + fputs ((const char *) buf, f); > + s = t - 1; > + prev_base64 = true; > + continue; > + } > + last_base64 = t; > + no_base64: > + prev_base64 = false; > + } > +#else > + (void) last_base64; > + (void) prev_base64; > +#endif > + > if (p < limit && (p - s) <= (long) ELF_STRING_LIMIT) > { > if (bytes_in_chunk > 0) > --- gcc/configure.jj 2024-07-01 11:28:22.432241469 +0200 > +++ gcc/configure 2024-07-12 10:28:44.843716198 +0200 > @@ -25859,6 +25859,39 @@ case "${target}" in > ;; > esac > > +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking assembler for .base64" >&5 > +$as_echo_n "checking assembler for .base64... " >&6; } > +if ${gcc_cv_as_base64+:} false; then : > + $as_echo_n "(cached) " >&6 > +else > + gcc_cv_as_base64=no > + if test x$gcc_cv_as != x; then > + $as_echo ' .section .rodata > + .base64 "Tm9uIHB1ZG9yIGVzdCBuaWwgc2NpcmUsIHB1ZG9yIG5pbCBkaXNjZXJlIHZlbGxlLgo="' > conftest.s > + if { ac_try='$gcc_cv_as $gcc_cv_as_flags -o conftest.o conftest.s >&5' > + { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_try\""; } >&5 > + (eval $ac_try) 2>&5 > + ac_status=$? > + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 > + test $ac_status = 0; }; } > + then > + gcc_cv_as_base64=yes > + else > + echo "configure: failed program was" >&5 > + cat conftest.s >&5 > + fi > + rm -f conftest.o conftest.s > + fi > +fi > +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $gcc_cv_as_base64" >&5 > +$as_echo "$gcc_cv_as_base64" >&6; } > +if test $gcc_cv_as_base64 = yes; then > + > +$as_echo "#define HAVE_GAS_BASE64 1" >>confdefs.h > + > +fi > + > + > # gnu_indirect_function type is an extension proposed at > # http://groups.google/com/group/generic-abi/files. It allows dynamic runtime > # selection of function implementation > --- gcc/config.in.jj 2024-06-17 18:50:45.791187189 +0200 > +++ gcc/config.in 2024-07-12 10:28:41.994753122 +0200 > @@ -1431,6 +1431,12 @@ > #endif > > > +/* Define if your assembler supports .base64. */ > +#ifndef USED_FOR_TARGET > +#undef HAVE_GAS_BASE64 > +#endif > + > + > /* Define 0/1 if your assembler supports CFI directives. */ > #undef HAVE_GAS_CFI_DIRECTIVE > > > Jakub > >
On Mon, Jul 15, 2024 at 09:16:29AM +0200, Richard Biener wrote: > > Nick has implemented a new .base64 directive in gas (to be shipped in > > the upcoming binutils 2.43; big thanks for that). > > See https://sourceware.org/bugzilla/show_bug.cgi?id=31964 > > > > The following patch adjusts default_elf_asm_output_ascii (i.e. > > ASM_OUTPUT_ASCII elfos.h implementation) to use it if it detects binary > > data and gas supports it. > > > > Without this patch, we emit stuff like: > > .string "\177ELF\002\001\001\003" > > .string "" > > .string "" > > .string "" > > .string "" > > .string "" > > .string "" > > .string "" > > .string "\002" > > .string ">" > > ... > > .string "\324\001\236 0FS\202\002E\n0@\203\004\005&\202\021\337)\021\203C\020A\300\220I\004\t\b\206(\234\0132l\004b\300\bK\006\220$0\303\020P$\233\211\002D\f" > > etc., with this patch more compact > > .base64 "f0VMRgIBAQMAAAAAAAAAAAIAPgABAAAAABf3AAAAAABAAAAAAAAAAACneB0AAAAAAAAAAEAAOAAOAEAALAArAAYAAAAEAAAAQAAAAAAAAABAAEAAAAAAAEAAQAAAAAAAEAMAAAAAAAAQAwAAAAAAAAgAAAAAAAAAAwAAAAQAAABQAwAAAAAAAFADQAAAAAAAUANAAAAAAAAcAAAAAAAAABwAAAAAAAAAAQAAAAAAAAABAAAABAAAAAAAAAAAAAAAAABAAAAAAAAAAEAAAAAAADBwOQAAAAAAMHA5AAAAAAAAEAAAAAAAAAEAAAAFAAAAAIA5AAAAAAAAgHkAAAAA" > > .base64 "AACAeQAAAAAAxSSgAgAAAADFJKACAAAAAAAQAAAAAAAAAQAAAAQAAAAAsNkCAAAAAACwGQMAAAAAALAZAwAAAADMtc0AAAAAAMy1zQAAAAAAABAAAAAAAAABAAAABgAAAGhmpwMAAAAAaHbnAwAAAABoducDAAAAAOAMAQAAAAAA4MEeAAAAAAAAEAAAAAAAAAIAAAAGAAAAkH2nAwAAAACQjecDAAAAAJCN5wMAAAAAQAIAAAAAAABAAgAAAAAAAAgAAAAAAAAABAAAAAQAAABwAwAAAAAAAHADQAAAAAAAcANAAAAAAABAAAAAAAAAAEAAAAAAAAAACAAAAAAA" > > .base64 "AAAEAAAABAAAALADAAAAAAAAsANAAAAAAACwA0AAAAAAACAAAAAAAAAAIAAAAAAAAAAEAAAAAAAAAAcAAAAEAAAAaGanAwAAAABoducDAAAAAGh25wMAAAAAAAAAAAAAAAAQAAAAAAAAAAgAAAAAAAAAU+V0ZAQAAABwAwAAAAAAAHADQAAAAAAAcANAAAAAAABAAAAAAAAAAEAAAAAAAAAACAAAAAAAAABQ5XRkBAAAAAw/WAMAAAAADD+YAwAAAAAMP5gDAAAAAPy7CgAAAAAA/LsKAAAAAAAEAAAAAAAAAFHldGQGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" > > .base64 "AAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAUuV0ZAQAAABoZqcDAAAAAGh25wMAAAAAaHbnAwAAAACYGQAAAAAAAJgZAAAAAAAAAQAAAAAAAAAvbGliNjQvbGQtbGludXgteDg2LTY0LnNvLjIAAAAAAAQAAAAwAAAABQAAAEdOVQACgADABAAAAAEAAAAAAAAAAQABwAQAAAAJAAAAAAAAAAIAAcAEAAAAAwAAAAAAAAAEAAAAEAAAAAEAAABHTlUAAAAAAAMAAAACAAAAAAAAAAOAAACsqAAAgS0AAOJWAAAjNwAAXjAAAAAAAAAAAAAAF1gAAHsxAABBBwAA" > > .base64 "G0kAALGmAACwoAAAAAAAAAAAAACQhAAAAAAAAOw1AACNYgAAAAAAAFQoAAAAAAAAx3UAALZAAAAAAAAAiIUAALGeAABBlAAAWEsAAPmRAACmOgAAAAAAADh3AAAAAAAAlCAAAAAAAABymgAAaosAAMIjAAAKMQAAMkIAADU0AAAAAAAA5ZwAAAAAAAAAAAAAAAAAAFIdAAAIGQAAAAAAAMFbAAAoTQAAGDcAAIRgAAA6HgAAlxwAAAAAAADOlgAAAAAAAEhPAAARiwAAMGgAAOVtAADMFgAAAAAAAAAAAACrjgAAYl4AACZVAAA/HgAAAAAAAAAAAABqPwAAAAAA" > > The patch attempts to juggle between readability and compactness, so > > if it detects some hunk of the initializer that would be shorter to be > > emitted as .string/.ascii directive, it does so, but if it previously > > used .base64 directive it switches mode only if there is a 16+ char > > ASCII-ish string. > > > > On my #embed testcase from yesterday > > unsigned char a[] = { > > #embed "cc1plus" > > }; > > without this patch it emits 2.4GB of assembly, while with this > > patch 649M. > > Compile times (trunk, so yes,rtl,extra checking) are: > > time ./xgcc -B ./ -S -std=c23 -O2 embed-11.c > > > > real 0m13.647s > > user 0m7.157s > > sys 0m2.597s > > time ./xgcc -B ./ -c -std=c23 -O2 embed-11.c > > > > real 0m28.649s > > user 0m26.653s > > sys 0m1.958s > > without the patch and > > time ./xgcc -B ./ -S -std=c23 -O2 embed-11.c > > > > real 0m4.283s > > user 0m2.288s > > sys 0m0.859s > > time ./xgcc -B ./ -c -std=c23 -O2 embed-11.c > > > > real 0m6.888s > > user 0m5.876s > > sys 0m1.002s > > with the patch, so that feels like significant improvement. > > The resulting embed-11.o is identical between the two ways of expressing > > the mostly binary data in the assembly. But note that there are portions > > like: > > .base64 "nAAAAAAAAAAvZRcAIgAOAFAzMwEAAAAABgAAAAAAAACEQBgAEgAOAFBHcAIAAAAA7AAAAAAAAAAAX19nbXB6X2dldF9zaQBtcGZyX3NldF9zaV8yZXhwAG1wZnJfY29zaABtcGZyX3RhbmgAbXBmcl9zZXRfbmFuAG1wZnJfc3ViAG1wZnJfdGFuAG1wZnJfc3RydG9mcgBfX2dtcHpfc3ViX3VpAF9fZ21wX2dldF9tZW1vcnlfZnVuY3Rpb25zAF9fZ21wel9zZXRfdWkAbXBmcl9wb3cAX19nbXB6X3N1YgBfX2dtcHpfZml0c19zbG9uZ19wAG1wZnJfYXRh" > > .base64 "bjIAX19nbXB6X2RpdmV4YWN0AG1wZnJfc2V0X2VtaW4AX19nbXB6X3NldABfX2dtcHpfbXVsAG1wZnJfY2xlYXIAbXBmcl9sb2cAbXBmcl9hdGFuaABfX2dtcHpfc3dhcABtcGZyX2FzaW5oAG1wZnJfYXNpbgBtcGZyX2NsZWFycwBfX2dtcHpfbXVsXzJleHAAX19nbXB6X2FkZG11bABtcGZyX3NpbmgAX19nbXB6X2FkZF91aQBfX2dtcHFfY2xlYXIAX19nbW9uX3N0YXJ0X18AbXBmcl9hY29zAG1wZnJfc2V0X2VtYXgAbXBmcl9jb3MAbXBmcl9zaW4A" > > .string "__gmpz_ui_pow_ui" > > .string "mpfr_get_str" > > .string "mpfr_acosh" > > .string "mpfr_sub_ui" > > .string "__gmpq_set_ui" > > .string "mpfr_set_inf" > > ... > > .string "GLIBC_2.14" > > .string "GLIBC_2.11" > > .base64 "AAABAAIAAQADAAMAAwADAAMAAwAEAAUABgADAAEAAQADAAMABwABAAEAAwADAAMAAwAIAAEAAwADAAEAAwABAAMAAwABAAMAAQADAAMAAwADAAMAAwADAAYAAwADAAEAAQAIAAMAAwADAAMAAwABAAMAAQADAAMAAQABAAEAAwAIAAEAAwADAAEAAwABAAMAAQADAAEABgADAAMAAQAHAAMAAwADAAMAAwABAAMAAQABAAMAAwADAAkAAQABAAEAAwAKAAEAAwADAAMAAQABAAMAAwALAAEAAwADAAEAAQADAAMAAwABAAMAAwABAAEAAwADAAMABwABAAMAAwAB" > > .base64 "AAEAAwADAAEAAwABAAMAAQADAAMAAwADAAEAAQABAAEAAwADAAMAAQABAAEAAQABAAEAAQADAAMAAwADAAMAAQABAAwAAwADAA0AAwADAAMAAwADAAEAAQADAAMAAQABAAMAAwADAAEAAwADAAEAAwAIAAMAAwADAAMABgABAA4ACwAGAAEAAQADAAEAAQADAAEAAwABAAMAAwABAAEAAwABAAMAAwABAAEAAwADAAMAAwABAAMAAQABAAEAAQABAAMADwABAAMAAQADAAMAAwABAAEAAQAIAAEADAADAAMAAQABAAMAAwADAAEAAQABAAEAAQADAAEAAwADAAEA" > > .base64 "AwABAAMAAQADAAMAAQABAAEAAwADAAMAAwADAAMAAQADAAMACAAQAA8AAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQA=" > > so it isn't all just totally unreadable stuff. > > > > Tested on top of the #embed patchset (just to get the above > > data) and then with full bootstrap/regtest on top of vanilla trunk, > > on x86_64-linux against trunk binutils prepended into $PATH (HAVE_GAS_BASE64 > > is true) and i686-linux (against older binutils, HAVE_GAS_BASE64 not > > defined), ok for trunk? > > OK. > > Do we want to have a command-line switch controlling the use of > .base64? It might be the 16+ char limit is too large for the > readability purpose and/or with wide-strings, etc. .base64 is > unwanted to some people at least when debugging? (does gas > support sth like .wstring or UTF-8/16/32 input?) I don't have > a good suggestion for the flag name, a quick scan for assembler > related options doesn't reveal a nice "pattern"; -f[no-]base64-in-asm? There is no directive to emit wide etc. strings, so UCS-2/UTF-16/UTF-32 are definitely barely readable even now. const char8_t a[] = u8"Žluťoučký"; const char16_t b[] = u"Žluťoučký"; const char32_t c[] = U"Žluťoučký"; The first one is .string "\305\275lu\305\245ou\304\215k\303\275" so one can with a magnifying glass find the ASCII chars in there, the second one .string "}\001l" .string "u" .string "e\001o" .string "u" .string "\r\001k" .string "\375" .string "" .string "" on little endian and third .string "}\001" .string "" .string "l" .string "" .string "" .string "u" .string "" .string "" .string "e\001" .string "" .string "o" .string "" .string "" .string "u" .string "" .string "" .string "\r\001" .string "" .string "k" .string "" .string "" .string "\375" .string "" .string "" .string "" .string "" .string "" .string "" I think that is simply binary rubbish. The way the patch is written is that if it sees a limited string (i.e. zero terminated) string and it is in itself shorter than base64 encoding, then it emits it as such, so .string "foobar" should be emitted like that, or even .string "barbaz\305\275foobarbar" but once if is not cheaper, it chooses base64 in the ~ 256 char chunks including the '\0's too. The 16 char limit is there just to avoid .base64 "somethingveryloooooooooooooooooooooooooooooooong" .string "I" .base64 "somethingverylongagain" which did happen but only with short readable strings that appear at the ~ 256 char boundaries, others in the middle of base64 were still emitted unreadably. And it happens only after a previous .base64. I think best way to read partially binary data is objdump or similar, but if people insist otherwise, -fno-base64-data-asm is possible. Jakub
On Mon, 15 Jul 2024, Jakub Jelinek wrote: > On Mon, Jul 15, 2024 at 09:16:29AM +0200, Richard Biener wrote: > > > Nick has implemented a new .base64 directive in gas (to be shipped in > > > the upcoming binutils 2.43; big thanks for that). > > > See https://sourceware.org/bugzilla/show_bug.cgi?id=31964 > > > > > > The following patch adjusts default_elf_asm_output_ascii (i.e. > > > ASM_OUTPUT_ASCII elfos.h implementation) to use it if it detects binary > > > data and gas supports it. > > > > > > Without this patch, we emit stuff like: > > > .string "\177ELF\002\001\001\003" > > > .string "" > > > .string "" > > > .string "" > > > .string "" > > > .string "" > > > .string "" > > > .string "" > > > .string "\002" > > > .string ">" > > > ... > > > .string "\324\001\236 0FS\202\002E\n0@\203\004\005&\202\021\337)\021\203C\020A\300\220I\004\t\b\206(\234\0132l\004b\300\bK\006\220$0\303\020P$\233\211\002D\f" > > > etc., with this patch more compact > > > .base64 "f0VMRgIBAQMAAAAAAAAAAAIAPgABAAAAABf3AAAAAABAAAAAAAAAAACneB0AAAAAAAAAAEAAOAAOAEAALAArAAYAAAAEAAAAQAAAAAAAAABAAEAAAAAAAEAAQAAAAAAAEAMAAAAAAAAQAwAAAAAAAAgAAAAAAAAAAwAAAAQAAABQAwAAAAAAAFADQAAAAAAAUANAAAAAAAAcAAAAAAAAABwAAAAAAAAAAQAAAAAAAAABAAAABAAAAAAAAAAAAAAAAABAAAAAAAAAAEAAAAAAADBwOQAAAAAAMHA5AAAAAAAAEAAAAAAAAAEAAAAFAAAAAIA5AAAAAAAAgHkAAAAA" > > > .base64 "AACAeQAAAAAAxSSgAgAAAADFJKACAAAAAAAQAAAAAAAAAQAAAAQAAAAAsNkCAAAAAACwGQMAAAAAALAZAwAAAADMtc0AAAAAAMy1zQAAAAAAABAAAAAAAAABAAAABgAAAGhmpwMAAAAAaHbnAwAAAABoducDAAAAAOAMAQAAAAAA4MEeAAAAAAAAEAAAAAAAAAIAAAAGAAAAkH2nAwAAAACQjecDAAAAAJCN5wMAAAAAQAIAAAAAAABAAgAAAAAAAAgAAAAAAAAABAAAAAQAAABwAwAAAAAAAHADQAAAAAAAcANAAAAAAABAAAAAAAAAAEAAAAAAAAAACAAAAAAA" > > > .base64 "AAAEAAAABAAAALADAAAAAAAAsANAAAAAAACwA0AAAAAAACAAAAAAAAAAIAAAAAAAAAAEAAAAAAAAAAcAAAAEAAAAaGanAwAAAABoducDAAAAAGh25wMAAAAAAAAAAAAAAAAQAAAAAAAAAAgAAAAAAAAAU+V0ZAQAAABwAwAAAAAAAHADQAAAAAAAcANAAAAAAABAAAAAAAAAAEAAAAAAAAAACAAAAAAAAABQ5XRkBAAAAAw/WAMAAAAADD+YAwAAAAAMP5gDAAAAAPy7CgAAAAAA/LsKAAAAAAAEAAAAAAAAAFHldGQGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" > > > .base64 "AAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAUuV0ZAQAAABoZqcDAAAAAGh25wMAAAAAaHbnAwAAAACYGQAAAAAAAJgZAAAAAAAAAQAAAAAAAAAvbGliNjQvbGQtbGludXgteDg2LTY0LnNvLjIAAAAAAAQAAAAwAAAABQAAAEdOVQACgADABAAAAAEAAAAAAAAAAQABwAQAAAAJAAAAAAAAAAIAAcAEAAAAAwAAAAAAAAAEAAAAEAAAAAEAAABHTlUAAAAAAAMAAAACAAAAAAAAAAOAAACsqAAAgS0AAOJWAAAjNwAAXjAAAAAAAAAAAAAAF1gAAHsxAABBBwAA" > > > .base64 "G0kAALGmAACwoAAAAAAAAAAAAACQhAAAAAAAAOw1AACNYgAAAAAAAFQoAAAAAAAAx3UAALZAAAAAAAAAiIUAALGeAABBlAAAWEsAAPmRAACmOgAAAAAAADh3AAAAAAAAlCAAAAAAAABymgAAaosAAMIjAAAKMQAAMkIAADU0AAAAAAAA5ZwAAAAAAAAAAAAAAAAAAFIdAAAIGQAAAAAAAMFbAAAoTQAAGDcAAIRgAAA6HgAAlxwAAAAAAADOlgAAAAAAAEhPAAARiwAAMGgAAOVtAADMFgAAAAAAAAAAAACrjgAAYl4AACZVAAA/HgAAAAAAAAAAAABqPwAAAAAA" > > > The patch attempts to juggle between readability and compactness, so > > > if it detects some hunk of the initializer that would be shorter to be > > > emitted as .string/.ascii directive, it does so, but if it previously > > > used .base64 directive it switches mode only if there is a 16+ char > > > ASCII-ish string. > > > > > > On my #embed testcase from yesterday > > > unsigned char a[] = { > > > #embed "cc1plus" > > > }; > > > without this patch it emits 2.4GB of assembly, while with this > > > patch 649M. > > > Compile times (trunk, so yes,rtl,extra checking) are: > > > time ./xgcc -B ./ -S -std=c23 -O2 embed-11.c > > > > > > real 0m13.647s > > > user 0m7.157s > > > sys 0m2.597s > > > time ./xgcc -B ./ -c -std=c23 -O2 embed-11.c > > > > > > real 0m28.649s > > > user 0m26.653s > > > sys 0m1.958s > > > without the patch and > > > time ./xgcc -B ./ -S -std=c23 -O2 embed-11.c > > > > > > real 0m4.283s > > > user 0m2.288s > > > sys 0m0.859s > > > time ./xgcc -B ./ -c -std=c23 -O2 embed-11.c > > > > > > real 0m6.888s > > > user 0m5.876s > > > sys 0m1.002s > > > with the patch, so that feels like significant improvement. > > > The resulting embed-11.o is identical between the two ways of expressing > > > the mostly binary data in the assembly. But note that there are portions > > > like: > > > .base64 "nAAAAAAAAAAvZRcAIgAOAFAzMwEAAAAABgAAAAAAAACEQBgAEgAOAFBHcAIAAAAA7AAAAAAAAAAAX19nbXB6X2dldF9zaQBtcGZyX3NldF9zaV8yZXhwAG1wZnJfY29zaABtcGZyX3RhbmgAbXBmcl9zZXRfbmFuAG1wZnJfc3ViAG1wZnJfdGFuAG1wZnJfc3RydG9mcgBfX2dtcHpfc3ViX3VpAF9fZ21wX2dldF9tZW1vcnlfZnVuY3Rpb25zAF9fZ21wel9zZXRfdWkAbXBmcl9wb3cAX19nbXB6X3N1YgBfX2dtcHpfZml0c19zbG9uZ19wAG1wZnJfYXRh" > > > .base64 "bjIAX19nbXB6X2RpdmV4YWN0AG1wZnJfc2V0X2VtaW4AX19nbXB6X3NldABfX2dtcHpfbXVsAG1wZnJfY2xlYXIAbXBmcl9sb2cAbXBmcl9hdGFuaABfX2dtcHpfc3dhcABtcGZyX2FzaW5oAG1wZnJfYXNpbgBtcGZyX2NsZWFycwBfX2dtcHpfbXVsXzJleHAAX19nbXB6X2FkZG11bABtcGZyX3NpbmgAX19nbXB6X2FkZF91aQBfX2dtcHFfY2xlYXIAX19nbW9uX3N0YXJ0X18AbXBmcl9hY29zAG1wZnJfc2V0X2VtYXgAbXBmcl9jb3MAbXBmcl9zaW4A" > > > .string "__gmpz_ui_pow_ui" > > > .string "mpfr_get_str" > > > .string "mpfr_acosh" > > > .string "mpfr_sub_ui" > > > .string "__gmpq_set_ui" > > > .string "mpfr_set_inf" > > > ... > > > .string "GLIBC_2.14" > > > .string "GLIBC_2.11" > > > .base64 "AAABAAIAAQADAAMAAwADAAMAAwAEAAUABgADAAEAAQADAAMABwABAAEAAwADAAMAAwAIAAEAAwADAAEAAwABAAMAAwABAAMAAQADAAMAAwADAAMAAwADAAYAAwADAAEAAQAIAAMAAwADAAMAAwABAAMAAQADAAMAAQABAAEAAwAIAAEAAwADAAEAAwABAAMAAQADAAEABgADAAMAAQAHAAMAAwADAAMAAwABAAMAAQABAAMAAwADAAkAAQABAAEAAwAKAAEAAwADAAMAAQABAAMAAwALAAEAAwADAAEAAQADAAMAAwABAAMAAwABAAEAAwADAAMABwABAAMAAwAB" > > > .base64 "AAEAAwADAAEAAwABAAMAAQADAAMAAwADAAEAAQABAAEAAwADAAMAAQABAAEAAQABAAEAAQADAAMAAwADAAMAAQABAAwAAwADAA0AAwADAAMAAwADAAEAAQADAAMAAQABAAMAAwADAAEAAwADAAEAAwAIAAMAAwADAAMABgABAA4ACwAGAAEAAQADAAEAAQADAAEAAwABAAMAAwABAAEAAwABAAMAAwABAAEAAwADAAMAAwABAAMAAQABAAEAAQABAAMADwABAAMAAQADAAMAAwABAAEAAQAIAAEADAADAAMAAQABAAMAAwADAAEAAQABAAEAAQADAAEAAwADAAEA" > > > .base64 "AwABAAMAAQADAAMAAQABAAEAAwADAAMAAwADAAMAAQADAAMACAAQAA8AAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQA=" > > > so it isn't all just totally unreadable stuff. > > > > > > Tested on top of the #embed patchset (just to get the above > > > data) and then with full bootstrap/regtest on top of vanilla trunk, > > > on x86_64-linux against trunk binutils prepended into $PATH (HAVE_GAS_BASE64 > > > is true) and i686-linux (against older binutils, HAVE_GAS_BASE64 not > > > defined), ok for trunk? > > > > OK. > > > > Do we want to have a command-line switch controlling the use of > > .base64? It might be the 16+ char limit is too large for the > > readability purpose and/or with wide-strings, etc. .base64 is > > unwanted to some people at least when debugging? (does gas > > support sth like .wstring or UTF-8/16/32 input?) I don't have > > a good suggestion for the flag name, a quick scan for assembler > > related options doesn't reveal a nice "pattern"; -f[no-]base64-in-asm? > > There is no directive to emit wide etc. strings, so UCS-2/UTF-16/UTF-32 > are definitely barely readable even now. > const char8_t a[] = u8"Žluťoučký"; > const char16_t b[] = u"Žluťoučký"; > const char32_t c[] = U"Žluťoučký"; > The first one is > .string "\305\275lu\305\245ou\304\215k\303\275" > so one can with a magnifying glass find the ASCII chars in there, > the second one > .string "}\001l" > .string "u" > .string "e\001o" > .string "u" > .string "\r\001k" > .string "\375" > .string "" > .string "" > on little endian and third > .string "}\001" > .string "" > .string "l" > .string "" > .string "" > .string "u" > .string "" > .string "" > .string "e\001" > .string "" > .string "o" > .string "" > .string "" > .string "u" > .string "" > .string "" > .string "\r\001" > .string "" > .string "k" > .string "" > .string "" > .string "\375" > .string "" > .string "" > .string "" > .string "" > .string "" > .string "" > I think that is simply binary rubbish. OK, so the "fix" for this would be to have .w8string .w16string and .w32string (or similar), or even allow .string U"Žluťoučký" directly. It's unrelated to .base64 which I agree doesn't make the above worse. > The way the patch is written is that if it sees a limited string (i.e. zero > terminated) string and it is in itself shorter than base64 encoding, then it > emits it as such, so > .string "foobar" > should be emitted like that, or even > .string "barbaz\305\275foobarbar" > but once if is not cheaper, it chooses base64 in the ~ 256 char chunks > including the '\0's too. The 16 char limit is there just to avoid > .base64 "somethingveryloooooooooooooooooooooooooooooooong" > .string "I" > .base64 "somethingverylongagain" > which did happen but only with short readable strings that appear at the > ~ 256 char boundaries, others in the middle of base64 were still emitted > unreadably. And it happens only after a previous .base64. So the overhead is at most '\t.string ""' which is 11 chars, you save the \0. If you make the limit 8 chars does the size get any worse when encoding cc1plus? How many extra strings get visible that way and how many of those are gibberish? > I think best way to read partially binary data is objdump or similar, > but if people insist otherwise, -fno-base64-data-asm is possible. We'll see if it pops up as a request. I do find it quite nice to have the actual strings better visually separated from binary data which is worse in the current way with only .string Richard.
On Mon, Jul 15, 2024 at 09:52:18AM +0200, Richard Biener wrote: > > .string "k" > > .string "" > > .string "" > > .string "\375" > > .string "" > > .string "" > > .string "" > > .string "" > > .string "" > > .string "" > > I think that is simply binary rubbish. > > OK, so the "fix" for this would be to have .w8string .w16string and > .w32string (or similar), or even allow .string U"Žluťoučký" directly. Maybe, but we'd also need to know that we are actually dealing with sensible readable data and not binary stuff, we just don't differentiate between that, even without the #embed stuff we have STRING_CST which can be binary data (e.g. the LTO sections, or user provided unsigned char data[] = { 0x83, 0x35, 0x9a, ... }; or it can be readable stuff. And whether it was parsed as a string literal or array of constants doesn't tell. And for UTF-16/UTF-32 the endianity also matters. > So the overhead is at most '\t.string ""' which is 11 chars, you > save the \0. If you make the limit 8 chars does the size get any I think that is the most important, typically in binary data one has tons of binary zeros and 11 chars for that is simply too much. Perhaps we could save some size for that even without .base64 directive by emitting .zero 10 instead of .string "" .string "" .string "" .string "" .string "" .string "" .string "" .string "" .string "" .string "" Though, it isn't just that, the 4 characters per all >= 0x7f bytes and most of 0x01..0x1f bytes (exceptions are the 2 character \b, \t, \n, \f, \r) is significant too, base64 is just 4./3 characters per byte. > worse when encoding cc1plus? How many extra strings get visible > that way and how many of those are gibberish? With either setting, it is about where exactly in the ~ 256 byte boundaries ASCII chars appear or don't, if it is in the middle of those chunks, it won't be readable anyway. And I didn't want to make the decisions too expensive. So the intent is just differentiate between clearly binary data vs. clearly readable data, the rest is unclear how it will be emitted. > > I think best way to read partially binary data is objdump or similar, > > but if people insist otherwise, -fno-base64-data-asm is possible. > > We'll see if it pops up as a request. I do find it quite nice to > have the actual strings better visually separated from binary data > which is worse in the current way with only .string Ok. Jakub
On Mon, 15 Jul 2024, Jakub Jelinek wrote: > On Mon, Jul 15, 2024 at 09:52:18AM +0200, Richard Biener wrote: > > > .string "k" > > > .string "" > > > .string "" > > > .string "\375" > > > .string "" > > > .string "" > > > .string "" > > > .string "" > > > .string "" > > > .string "" > > > I think that is simply binary rubbish. > > > > OK, so the "fix" for this would be to have .w8string .w16string and > > .w32string (or similar), or even allow .string U"Žluťoučký" directly. > > Maybe, but we'd also need to know that we are actually dealing with sensible > readable data and not binary stuff, we just don't differentiate between > that, even without the #embed stuff we have STRING_CST which can be binary data > (e.g. the LTO sections, or user provided unsigned char data[] = { 0x83, > 0x35, 0x9a, ... }; > or it can be readable stuff. And whether it was parsed as a string literal > or array of constants doesn't tell. > And for UTF-16/UTF-32 the endianity also matters. I think varasm could know from the type of the variable that's initialized. We have TYPE_STRING_FLAG and we could possibly have a flag on the STRING_CST itself to tell whether it was lexed from a string literal or not. Of course it's somewhat pointless to "optimize" assembler output for readability. Richard.
--- gcc/configure.ac.jj 2024-07-01 11:28:22.435241431 +0200 +++ gcc/configure.ac 2024-07-12 10:28:38.038804308 +0200 @@ -3054,6 +3054,11 @@ case "${target}" in ;; esac +gcc_GAS_CHECK_FEATURE([.base64], gcc_cv_as_base64,, + [ .section .rodata + .base64 "Tm9uIHB1ZG9yIGVzdCBuaWwgc2NpcmUsIHB1ZG9yIG5pbCBkaXNjZXJlIHZlbGxlLgo="],, +[AC_DEFINE(HAVE_GAS_BASE64, 1, [Define if your assembler supports .base64.])]) + # gnu_indirect_function type is an extension proposed at # http://groups.google/com/group/generic-abi/files. It allows dynamic runtime # selection of function implementation --- gcc/config/elfos.h.jj 2024-01-03 11:51:43.966558606 +0100 +++ gcc/config/elfos.h 2024-07-12 10:40:59.721200959 +0200 @@ -444,6 +444,10 @@ see the files COPYING3 and COPYING.RUNTI #define STRING_ASM_OP "\t.string\t" +#ifdef HAVE_GAS_BASE64 +#define BASE64_ASM_OP "\t.base64\t" +#endif + /* The routine used to output NUL terminated strings. We use a special version of this for most svr4 targets because doing so makes the generated assembly code more compact (and thus faster to assemble) --- gcc/varasm.cc.jj 2024-07-08 11:00:54.723980651 +0200 +++ gcc/varasm.cc 2024-07-12 13:28:39.934619478 +0200 @@ -2101,7 +2101,19 @@ void assemble_string (const char *p, int size) { int pos = 0; +#if defined(BASE64_ASM_OP) \ + && BITS_PER_UNIT == 8 \ + && CHAR_BIT == 8 \ + && 'A' == 65 \ + && 'a' == 97 \ + && '0' == 48 \ + && '+' == 43 \ + && '/' == 47 \ + && '=' == 61 + int maximum = 16384; +#else int maximum = 2000; +#endif /* If the string is very long, split it up. */ @@ -8469,8 +8481,7 @@ default_elf_asm_output_limited_string (F int escape; unsigned char c; - fputs (STRING_ASM_OP, f); - putc ('"', f); + fputs (STRING_ASM_OP "\"", f); while (*s != '\0') { c = *s; @@ -8504,9 +8515,11 @@ default_elf_asm_output_ascii (FILE *f, c { const char *limit = s + len; const char *last_null = NULL; + const char *last_base64 = s; unsigned bytes_in_chunk = 0; unsigned char c; int escape; + bool prev_base64 = false; for (; s < limit; s++) { @@ -8519,7 +8532,7 @@ default_elf_asm_output_ascii (FILE *f, c bytes_in_chunk = 0; } - if (s > last_null) + if ((uintptr_t) s > (uintptr_t) last_null) { for (p = s; p < limit && *p != '\0'; p++) continue; @@ -8528,6 +8541,104 @@ default_elf_asm_output_ascii (FILE *f, c else p = last_null; +#if defined(BASE64_ASM_OP) \ + && BITS_PER_UNIT == 8 \ + && CHAR_BIT == 8 \ + && 'A' == 65 \ + && 'a' == 97 \ + && '0' == 48 \ + && '+' == 43 \ + && '/' == 47 \ + && '=' == 61 + if (s >= last_base64) + { + unsigned cnt = 0; + const char *t; + for (t = s; t < limit && (t - s) < (long) ELF_STRING_LIMIT - 1; t++) + { + if (t == p && t != s) + { + if (cnt <= (t - s + 1 + 2) / 3 * 4 + && (!prev_base64 || (t - s) >= 16) + && ((t - s) > 1 || cnt <= 2)) + { + last_base64 = p; + goto no_base64; + } + } + c = *t; + escape = ELF_ASCII_ESCAPES[c]; + switch (escape) + { + case 0: + ++cnt; + break; + case 1: + if (c == 0) + cnt += 2 + strlen (STRING_ASM_OP) + 1; + else + cnt += 4; + break; + default: + cnt += 2; + break; + } + } + if (cnt > (t - s + 2) / 3 * 4 && (t - s) >= 3) + { + if (bytes_in_chunk > 0) + { + putc ('\"', f); + putc ('\n', f); + bytes_in_chunk = 0; + } + + unsigned char buf[(ELF_STRING_LIMIT + 2) / 3 * 4 + 3]; + unsigned j = 0; + static const char base64_enc[] = + "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz" + "0123456789+/"; + + fputs (BASE64_ASM_OP "\"", f); + while (s < t) + { + unsigned char a = *s; + unsigned char b = 0, c = 0; + if (s < t - 1) + b = s[1]; + if (s < t - 2) + c = s[2]; + unsigned long v = ((((unsigned long) a) << 16) + | (((unsigned long) b) << 8) + | c); + buf[j++] = base64_enc[(v >> 18) & 63]; + buf[j++] = base64_enc[(v >> 12) & 63]; + buf[j++] = base64_enc[(v >> 6) & 63]; + buf[j++] = base64_enc[v & 63]; + if (s >= t - 2) + { + buf[j - 1] = '='; + if (s >= t - 1) + buf[j - 2] = '='; + break; + } + s += 3; + } + memcpy (buf + j, "\"\n", 3); + fputs ((const char *) buf, f); + s = t - 1; + prev_base64 = true; + continue; + } + last_base64 = t; + no_base64: + prev_base64 = false; + } +#else + (void) last_base64; + (void) prev_base64; +#endif + if (p < limit && (p - s) <= (long) ELF_STRING_LIMIT) { if (bytes_in_chunk > 0) --- gcc/configure.jj 2024-07-01 11:28:22.432241469 +0200 +++ gcc/configure 2024-07-12 10:28:44.843716198 +0200 @@ -25859,6 +25859,39 @@ case "${target}" in ;; esac +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking assembler for .base64" >&5 +$as_echo_n "checking assembler for .base64... " >&6; } +if ${gcc_cv_as_base64+:} false; then : + $as_echo_n "(cached) " >&6 +else + gcc_cv_as_base64=no + if test x$gcc_cv_as != x; then + $as_echo ' .section .rodata + .base64 "Tm9uIHB1ZG9yIGVzdCBuaWwgc2NpcmUsIHB1ZG9yIG5pbCBkaXNjZXJlIHZlbGxlLgo="' > conftest.s + if { ac_try='$gcc_cv_as $gcc_cv_as_flags -o conftest.o conftest.s >&5' + { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_try\""; } >&5 + (eval $ac_try) 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; } + then + gcc_cv_as_base64=yes + else + echo "configure: failed program was" >&5 + cat conftest.s >&5 + fi + rm -f conftest.o conftest.s + fi +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $gcc_cv_as_base64" >&5 +$as_echo "$gcc_cv_as_base64" >&6; } +if test $gcc_cv_as_base64 = yes; then + +$as_echo "#define HAVE_GAS_BASE64 1" >>confdefs.h + +fi + + # gnu_indirect_function type is an extension proposed at # http://groups.google/com/group/generic-abi/files. It allows dynamic runtime # selection of function implementation --- gcc/config.in.jj 2024-06-17 18:50:45.791187189 +0200 +++ gcc/config.in 2024-07-12 10:28:41.994753122 +0200 @@ -1431,6 +1431,12 @@ #endif +/* Define if your assembler supports .base64. */ +#ifndef USED_FOR_TARGET +#undef HAVE_GAS_BASE64 +#endif + + /* Define 0/1 if your assembler supports CFI directives. */ #undef HAVE_GAS_CFI_DIRECTIVE
Hi! Nick has implemented a new .base64 directive in gas (to be shipped in the upcoming binutils 2.43; big thanks for that). See https://sourceware.org/bugzilla/show_bug.cgi?id=31964 The following patch adjusts default_elf_asm_output_ascii (i.e. ASM_OUTPUT_ASCII elfos.h implementation) to use it if it detects binary data and gas supports it. Without this patch, we emit stuff like: .string "\177ELF\002\001\001\003" .string "" .string "" .string "" .string "" .string "" .string "" .string "" .string "\002" .string ">" ... .string "\324\001\236 0FS\202\002E\n0@\203\004\005&\202\021\337)\021\203C\020A\300\220I\004\t\b\206(\234\0132l\004b\300\bK\006\220$0\303\020P$\233\211\002D\f" etc., with this patch more compact .base64 "f0VMRgIBAQMAAAAAAAAAAAIAPgABAAAAABf3AAAAAABAAAAAAAAAAACneB0AAAAAAAAAAEAAOAAOAEAALAArAAYAAAAEAAAAQAAAAAAAAABAAEAAAAAAAEAAQAAAAAAAEAMAAAAAAAAQAwAAAAAAAAgAAAAAAAAAAwAAAAQAAABQAwAAAAAAAFADQAAAAAAAUANAAAAAAAAcAAAAAAAAABwAAAAAAAAAAQAAAAAAAAABAAAABAAAAAAAAAAAAAAAAABAAAAAAAAAAEAAAAAAADBwOQAAAAAAMHA5AAAAAAAAEAAAAAAAAAEAAAAFAAAAAIA5AAAAAAAAgHkAAAAA" .base64 "AACAeQAAAAAAxSSgAgAAAADFJKACAAAAAAAQAAAAAAAAAQAAAAQAAAAAsNkCAAAAAACwGQMAAAAAALAZAwAAAADMtc0AAAAAAMy1zQAAAAAAABAAAAAAAAABAAAABgAAAGhmpwMAAAAAaHbnAwAAAABoducDAAAAAOAMAQAAAAAA4MEeAAAAAAAAEAAAAAAAAAIAAAAGAAAAkH2nAwAAAACQjecDAAAAAJCN5wMAAAAAQAIAAAAAAABAAgAAAAAAAAgAAAAAAAAABAAAAAQAAABwAwAAAAAAAHADQAAAAAAAcANAAAAAAABAAAAAAAAAAEAAAAAAAAAACAAAAAAA" .base64 "AAAEAAAABAAAALADAAAAAAAAsANAAAAAAACwA0AAAAAAACAAAAAAAAAAIAAAAAAAAAAEAAAAAAAAAAcAAAAEAAAAaGanAwAAAABoducDAAAAAGh25wMAAAAAAAAAAAAAAAAQAAAAAAAAAAgAAAAAAAAAU+V0ZAQAAABwAwAAAAAAAHADQAAAAAAAcANAAAAAAABAAAAAAAAAAEAAAAAAAAAACAAAAAAAAABQ5XRkBAAAAAw/WAMAAAAADD+YAwAAAAAMP5gDAAAAAPy7CgAAAAAA/LsKAAAAAAAEAAAAAAAAAFHldGQGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" .base64 "AAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAUuV0ZAQAAABoZqcDAAAAAGh25wMAAAAAaHbnAwAAAACYGQAAAAAAAJgZAAAAAAAAAQAAAAAAAAAvbGliNjQvbGQtbGludXgteDg2LTY0LnNvLjIAAAAAAAQAAAAwAAAABQAAAEdOVQACgADABAAAAAEAAAAAAAAAAQABwAQAAAAJAAAAAAAAAAIAAcAEAAAAAwAAAAAAAAAEAAAAEAAAAAEAAABHTlUAAAAAAAMAAAACAAAAAAAAAAOAAACsqAAAgS0AAOJWAAAjNwAAXjAAAAAAAAAAAAAAF1gAAHsxAABBBwAA" .base64 "G0kAALGmAACwoAAAAAAAAAAAAACQhAAAAAAAAOw1AACNYgAAAAAAAFQoAAAAAAAAx3UAALZAAAAAAAAAiIUAALGeAABBlAAAWEsAAPmRAACmOgAAAAAAADh3AAAAAAAAlCAAAAAAAABymgAAaosAAMIjAAAKMQAAMkIAADU0AAAAAAAA5ZwAAAAAAAAAAAAAAAAAAFIdAAAIGQAAAAAAAMFbAAAoTQAAGDcAAIRgAAA6HgAAlxwAAAAAAADOlgAAAAAAAEhPAAARiwAAMGgAAOVtAADMFgAAAAAAAAAAAACrjgAAYl4AACZVAAA/HgAAAAAAAAAAAABqPwAAAAAA" The patch attempts to juggle between readability and compactness, so if it detects some hunk of the initializer that would be shorter to be emitted as .string/.ascii directive, it does so, but if it previously used .base64 directive it switches mode only if there is a 16+ char ASCII-ish string. On my #embed testcase from yesterday unsigned char a[] = { #embed "cc1plus" }; without this patch it emits 2.4GB of assembly, while with this patch 649M. Compile times (trunk, so yes,rtl,extra checking) are: time ./xgcc -B ./ -S -std=c23 -O2 embed-11.c real 0m13.647s user 0m7.157s sys 0m2.597s time ./xgcc -B ./ -c -std=c23 -O2 embed-11.c real 0m28.649s user 0m26.653s sys 0m1.958s without the patch and time ./xgcc -B ./ -S -std=c23 -O2 embed-11.c real 0m4.283s user 0m2.288s sys 0m0.859s time ./xgcc -B ./ -c -std=c23 -O2 embed-11.c real 0m6.888s user 0m5.876s sys 0m1.002s with the patch, so that feels like significant improvement. The resulting embed-11.o is identical between the two ways of expressing the mostly binary data in the assembly. But note that there are portions like: .base64 "nAAAAAAAAAAvZRcAIgAOAFAzMwEAAAAABgAAAAAAAACEQBgAEgAOAFBHcAIAAAAA7AAAAAAAAAAAX19nbXB6X2dldF9zaQBtcGZyX3NldF9zaV8yZXhwAG1wZnJfY29zaABtcGZyX3RhbmgAbXBmcl9zZXRfbmFuAG1wZnJfc3ViAG1wZnJfdGFuAG1wZnJfc3RydG9mcgBfX2dtcHpfc3ViX3VpAF9fZ21wX2dldF9tZW1vcnlfZnVuY3Rpb25zAF9fZ21wel9zZXRfdWkAbXBmcl9wb3cAX19nbXB6X3N1YgBfX2dtcHpfZml0c19zbG9uZ19wAG1wZnJfYXRh" .base64 "bjIAX19nbXB6X2RpdmV4YWN0AG1wZnJfc2V0X2VtaW4AX19nbXB6X3NldABfX2dtcHpfbXVsAG1wZnJfY2xlYXIAbXBmcl9sb2cAbXBmcl9hdGFuaABfX2dtcHpfc3dhcABtcGZyX2FzaW5oAG1wZnJfYXNpbgBtcGZyX2NsZWFycwBfX2dtcHpfbXVsXzJleHAAX19nbXB6X2FkZG11bABtcGZyX3NpbmgAX19nbXB6X2FkZF91aQBfX2dtcHFfY2xlYXIAX19nbW9uX3N0YXJ0X18AbXBmcl9hY29zAG1wZnJfc2V0X2VtYXgAbXBmcl9jb3MAbXBmcl9zaW4A" .string "__gmpz_ui_pow_ui" .string "mpfr_get_str" .string "mpfr_acosh" .string "mpfr_sub_ui" .string "__gmpq_set_ui" .string "mpfr_set_inf" ... .string "GLIBC_2.14" .string "GLIBC_2.11" .base64 "AAABAAIAAQADAAMAAwADAAMAAwAEAAUABgADAAEAAQADAAMABwABAAEAAwADAAMAAwAIAAEAAwADAAEAAwABAAMAAwABAAMAAQADAAMAAwADAAMAAwADAAYAAwADAAEAAQAIAAMAAwADAAMAAwABAAMAAQADAAMAAQABAAEAAwAIAAEAAwADAAEAAwABAAMAAQADAAEABgADAAMAAQAHAAMAAwADAAMAAwABAAMAAQABAAMAAwADAAkAAQABAAEAAwAKAAEAAwADAAMAAQABAAMAAwALAAEAAwADAAEAAQADAAMAAwABAAMAAwABAAEAAwADAAMABwABAAMAAwAB" .base64 "AAEAAwADAAEAAwABAAMAAQADAAMAAwADAAEAAQABAAEAAwADAAMAAQABAAEAAQABAAEAAQADAAMAAwADAAMAAQABAAwAAwADAA0AAwADAAMAAwADAAEAAQADAAMAAQABAAMAAwADAAEAAwADAAEAAwAIAAMAAwADAAMABgABAA4ACwAGAAEAAQADAAEAAQADAAEAAwABAAMAAwABAAEAAwABAAMAAwABAAEAAwADAAMAAwABAAMAAQABAAEAAQABAAMADwABAAMAAQADAAMAAwABAAEAAQAIAAEADAADAAMAAQABAAMAAwADAAEAAQABAAEAAQADAAEAAwADAAEA" .base64 "AwABAAMAAQADAAMAAQABAAEAAwADAAMAAwADAAMAAQADAAMACAAQAA8AAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQABAAEAAQA=" so it isn't all just totally unreadable stuff. Tested on top of the #embed patchset (just to get the above data) and then with full bootstrap/regtest on top of vanilla trunk, on x86_64-linux against trunk binutils prepended into $PATH (HAVE_GAS_BASE64 is true) and i686-linux (against older binutils, HAVE_GAS_BASE64 not defined), ok for trunk? 2024-07-12 Jakub Jelinek <jakub@redhat.com> * configure.ac (HAVE_GAS_BASE64): New check. * config/elfos.h (BASE64_ASM_OP): Define if HAVE_GAS_BASE64 is defined. * varasm.cc (assemble_string): Bump maximum from 2000 to 16384 if BASE64_ASM_OP is defined. (default_elf_asm_output_limited_string): Emit opening '"' together with STRING_ASM_OP. (default_elf_asm_output_ascii): Use BASE64_ASM_OP if defined and beneficial. Remove UB when last_null is NULL. * configure: Regenerate. * config.in: Regenerate. Jakub