| Message ID | alpine.DEB.2.00.1806181810570.20622@tp.orcam.me.uk |
|---|---|
| State | Superseded |
| Headers | show |
| Series | Accept absolute symbols whose value is zero [BZ #23307] | expand |
* Maciej W. Rozycki: > Define a new ABSOLUTE ABI for static linker's use with EI_ABIVERSION > where correct absolute (SHN_ABS) symbol run-time load semantics is > required. This way it can be ensured at static link time that a program > or DSO will not suffer from previous semantics where absolute symbols > were relocated by the base address, or symbols whose `st_value' is zero > silently ignored leading to a confusing "undefined symbol" error message > at load time, and instead "ELF file ABI version invalid" is printed with > old dynamic loaders, making it clear that there is an ABI version > incompatibility. Is this really necessary? It essentially precludes backporting the fixes.
Hi Florian, > > Define a new ABSOLUTE ABI for static linker's use with EI_ABIVERSION > > where correct absolute (SHN_ABS) symbol run-time load semantics is > > required. This way it can be ensured at static link time that a program > > or DSO will not suffer from previous semantics where absolute symbols > > were relocated by the base address, or symbols whose `st_value' is zero > > silently ignored leading to a confusing "undefined symbol" error message > > at load time, and instead "ELF file ABI version invalid" is printed with > > old dynamic loaders, making it clear that there is an ABI version > > incompatibility. > > Is this really necessary? It essentially precludes backporting the > fixes. Thank you for your input. Of course bumping up the ABI version is not itself required for the change to work, and all existing working binaries will continue to. Technically it does not preclude backporting either, although of course it is limited by the previous ABI bump, which is (in the reverse order by version/date): * glibc 2.21 for MIPS targets, by commit 12e6ee86c4f2 ("Add support for MIPS O32 FPXX and .MIPS.abiflags"), * glibc 2.12 for PowerPC and SPARC targets, by commit 12e6ee86c4f2 ("A few more archs have IFUNC support."), * glibc 2.12 for the remaining targets, by commit 92ad15a8f1d3 ("Implement handling of libc ABI in ELF header."), when the feature was introduced. I don't believe we backport *that* far in upstream branches, however I'll be happy to be corrected. Otherwise whether to backport or not to would be a matter of policy. And policies are there to make life easier and not more difficult. So what does our current actually policy say? Well, at: <https://sourceware.org/glibc/wiki/Release/#General_policy> I read this: "Patch backports to stable branches are discussed on libc-stable, and any patch on master that doesn't change ABI or API is immediately suitable for backporting to a stable branch." To me it means that a patch which does change an ABI or an API, while not immediately, can still be suitable for backporting, however consensus has to be reached. So what are the pros and the cons of having an ABI defined for absolute symbol handling? The pros I can identify are: 1. It matches reality. Previously absolute symbols were effectively completely not supported. Not just failing to work in some cases or consistently working incorrectly, while still being recognised. No, they were not recognised at all and treated as ordinary symbols that express memory addresses and, if applicable, are relocated by the base address at load time. Yes, from the ELF gABI's point of view this is a conformance bug fix, however from GNU C library's point of view this is a new feature. 2. It is reported by running `libc.so', e.g. with the patch applied on a MIPS system you get information like this: GNU C Library (GNU libc) development release version 2.27.9000. Copyright (C) 2018 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Compiled by GNU CC version 8.0.1 20180410 (experimental). libc ABIs: MIPS_PLT UNIQUE MIPS_O32_FP64 ABSOLUTE For bug reporting instructions, please see: <http://www.gnu.org/software/libc/bugs.html>. where you can readily determine that the feature is supported, which is suitable for inclusion in release notes of binary software packages that have been compiled to require that feature for correct operation. 3. It replaces an obscure "undefined symbol" error message with an informative "ELF file ABI version invalid" one for absolute symbols whose value is zero. Such symbols will be produced by the static linker in MIPS ELF binaries for undefined weak symbols whose scope is local (and therefore have to be wired to zero) and which are referred to via the GOT once the upcoming fix for binutils PR ld/21375 has been applied. The reason for that solution is that the MIPS psABI makes the GOT implicitly relocated and consequently undefined weak symbol references cannot use a local GOT entry, because all such entries are relocated by the base address at load time. Consequently such references have to be made to a global GOT entry associated with a global absolute symbol whose value is zero and remains such at load time. The usual approach with targets that have the GOT explicitly relocated (all the non-MIPS targets I believe) is to have no dynamic R_*_RELATIVE relocation attached to local GOT entries associated with undefined weak symbols whose scope is local, which prevents them from being relocated at load time. NB the fix for binutils PR ld/21375 will strive to avoid using that global absolute symbol where possible by code relaxation applied to the usual instructions used for GOT references, but GOT relocations are generic and if used with an instruction the linker cannot interpret it will have to stay there and refer to a global GOT entry associated with such a symbol. (Granted, the message we print for an attempt to load an ELF file that requests an ABI version higher than the dynamic loader supports is also but obscure, however at least it is indicative and we can improve it, whereas the message printed for a symbol lookup that has failed because a symbol definition has been ignored is completely useless to whoever has encountered it.) 4. It prevents software relying on non-zero absolute symbols from malfunctioning, perhaps in a way that may be difficult to identify. I have no plans at the moment to arrange for the static linker to recognise the general case of dynamic absolute symbols being produced and to adjust the libc ABI requested accordingly. Using dynamic symbols in such a way would I think be a means to determine the presence of features or to examine configuration settings among a set of DSOs whose configuration may vary between builds. 5. This is similar in spirit a change to the dynamic loader to the earlier addition of the STB_GNU_UNIQUE ELF gABI extenstion. Likewise we could have no ABI version defined for STB_GNU_UNIQUE and rely on an obscure error path or the lack of, wherever an old dynamic loader handles modules using the feature. The cons I can identify are: 1. It may too unimportant a feature to deserve an ABI bump. The only user of the feature at the moment is the upcoming fix for binutils PR ld/21375, which addresses an obscure corner case of the MIPS psABI. So its use it quite limited. Overall I think the maximum libc ABI version supported is not a property of the ABI that should prevent the change introducing it from being backported: 1. Bumping it up does not affect, in any way, binary software whose ABI version requested is lower than or equal to the current maximum ABI version. 2. It does not export itself any extra feature available to binary software; that is already done by the absolute symbol handling change itself. It only announces the presence of the change, which would otherwise be hidden. 3. It does affect newly-built software which will request that ABI version from the dynamic loader. However that software, in the absence of a way to request a new ABI version, would not work anyway with an old dynamic loader. Therefore I believe there should be no policy issue with backporting the change. Although I think that consensus is still required on whether the ABI update is important enough to deserve a new version. Comments, thoughts, questions? Maciej
On 06/28/2018 12:07 AM, Maciej W. Rozycki wrote: >> Is this really necessary? It essentially precludes backporting the >> fixes. > Thank you for your input. Of course bumping up the ABI version is not > itself required for the change to work, and all existing working binaries > will continue to. > > Technically it does not preclude backporting either, although of course > it is limited by the previous ABI bump, which is (in the reverse order by > version/date): Maybe I misunderstood what you were trying to do here. Will the EI_ABIVERSION in created binaries increase if the programmer simply upgrades binutils? Or will this happen only when particular features are used? Thanks, Florian
On Wed, 27 Jun 2018, Florian Weimer wrote: > > > Is this really necessary? It essentially precludes backporting the > > > fixes. > > Thank you for your input. Of course bumping up the ABI version is not > > itself required for the change to work, and all existing working binaries > > will continue to. > > > > Technically it does not preclude backporting either, although of course > > it is limited by the previous ABI bump, which is (in the reverse order by > > version/date): > > Maybe I misunderstood what you were trying to do here. > > Will the EI_ABIVERSION in created binaries increase if the programmer simply > upgrades binutils? Or will this happen only when particular features are > used? Only when this particular feature is required, like with the older ABI versions defined so far. I've double-checked with binutils and the MIPS target is the only one actively switching EI_ABIVERSION; all the remaining targets appear to have the value fixed and I am not going to change that. This is how the relevant part of the linker update in question looks like: Index: binutils/bfd/elfxx-mips.c =================================================================== --- binutils.orig/bfd/elfxx-mips.c 2018-06-28 00:44:35.780943617 +0100 +++ binutils/bfd/elfxx-mips.c 2018-06-28 00:44:35.913406279 +0100 @@ -16305,13 +16449,14 @@ enum MIPS_LIBC_ABI_MIPS_PLT, MIPS_LIBC_ABI_UNIQUE, MIPS_LIBC_ABI_MIPS_O32_FP64, + MIPS_LIBC_ABI_ABSOLUTE, MIPS_LIBC_ABI_MAX }; void _bfd_mips_post_process_headers (bfd *abfd, struct bfd_link_info *link_info) { - struct mips_elf_link_hash_table *htab; + struct mips_elf_link_hash_table *htab = NULL; Elf_Internal_Ehdr *i_ehdrp; i_ehdrp = elf_elfheader (abfd); @@ -16319,15 +16464,19 @@ _bfd_mips_post_process_headers (bfd *abf { htab = mips_elf_hash_table (link_info); BFD_ASSERT (htab != NULL); - - if (htab->use_plts_and_copy_relocs && !htab->is_vxworks) - i_ehdrp->e_ident[EI_ABIVERSION] = MIPS_LIBC_ABI_MIPS_PLT; } + if (htab != NULL && htab->use_plts_and_copy_relocs && !htab->is_vxworks) + i_ehdrp->e_ident[EI_ABIVERSION] = MIPS_LIBC_ABI_MIPS_PLT; + if (mips_elf_tdata (abfd)->abiflags.fp_abi == Val_GNU_MIPS_ABI_FP_64 || mips_elf_tdata (abfd)->abiflags.fp_abi == Val_GNU_MIPS_ABI_FP_64A) i_ehdrp->e_ident[EI_ABIVERSION] = MIPS_LIBC_ABI_MIPS_O32_FP64; + /* Mark that we need support for absolute symbols in the dynamic loader. */ + if (htab != NULL && htab->use_absolute_zero) + i_ehdrp->e_ident[EI_ABIVERSION] = MIPS_LIBC_ABI_ABSOLUTE; + _bfd_elf_post_process_headers (abfd, link_info); } and then `htab->use_absolute_zero' will be TRUE iff a GOT relocation has been calculated to actually refer to a specially-created absolute symbol used to address the issue covered by binutils PR ld/21375. The flag will be FALSE if code has been relaxed to avoid referring the GOT (by using immediate zero instead), in which case no absolute symbol will have been specially created. Does this explanation clear your concern? Maciej
On 06/28/2018 03:49 PM, Maciej W. Rozycki wrote: > + /* Mark that we need support for absolute symbols in the dynamic loader. */ > + if (htab != NULL && htab->use_absolute_zero) > + i_ehdrp->e_ident[EI_ABIVERSION] = MIPS_LIBC_ABI_ABSOLUTE; > + > _bfd_elf_post_process_headers (abfd, link_info); > } > > and then `htab->use_absolute_zero' will be TRUE iff a GOT relocation has > been calculated to actually refer to a specially-created absolute symbol > used to address the issue covered by binutils PR ld/21375. > > The flag will be FALSE if code has been relaxed to avoid referring the > GOT (by using immediate zero instead), in which case no absolute symbol > will have been specially created. > > Does this explanation clear your concern? Yes, I think this is okay then. We cannot express this change at the RPM dependency level, but I'm not sure how active the Fedora MIPS port is these days, so it probably does not matter. Thanks, Florian
On Thu, 28 Jun 2018, Florian Weimer wrote: > On 06/28/2018 03:49 PM, Maciej W. Rozycki wrote: > > + /* Mark that we need support for absolute symbols in the dynamic loader. > > */ > > + if (htab != NULL && htab->use_absolute_zero) > > + i_ehdrp->e_ident[EI_ABIVERSION] = MIPS_LIBC_ABI_ABSOLUTE; > > + > > _bfd_elf_post_process_headers (abfd, link_info); > > } > > and then `htab->use_absolute_zero' will be TRUE iff a GOT relocation has > > been calculated to actually refer to a specially-created absolute symbol > > used to address the issue covered by binutils PR ld/21375. > > > > The flag will be FALSE if code has been relaxed to avoid referring the > > GOT (by using immediate zero instead), in which case no absolute symbol > > will have been specially created. > > > > Does this explanation clear your concern? > > Yes, I think this is okay then. Great! > We cannot express this change at the RPM dependency level, but I'm not sure > how active the Fedora MIPS port is these days, so it probably does not matter. The case of MIPS_LIBC_ABI_ABSOLUTE is supposed to be exceedingly rare. However I decided to handle it, rather than having the linker fail, be it with an assertion failure or gracefully. I think RPM could be easily expanded to include EI_ABIVERSION with its existing dependency mechanism. The EI_ABIVERSION field is reported by `readelf -h', so the requirement could be easily extracted from binaries just like symbol versioning information is. The only complication is the package supplying the dynamic loader, where the provision of the maximum value of EI_ABIVERSION supported would I think have to be entered manually in the package spec. Decades ago I modified the RPM dependency extraction scripts (`linux.req' and `linux.prov') to use `readelf' rather than a combination of `objdump' and `ldd' (although I left the latter as a fallback to keep supporting a.out dependencies), so that dependencies were right for cross-compiled packages. I have never submitted that work back (as I recall it was not exactly easy to get through), however experience gained with that makes me think expanding the mechanism to include EI_ABIVERSION should be straightforward to implement. Perhaps it would be worth doing anyway, just in case? NB I have no idea what RPM uses these days for dependency tracking and whether it handles cross-compilation right or not. Work I did back then remains available from `ftp.linux-mips.org', in the form of RPM 3.0.6 RPM packages; the GNU GPL applies. Maciej
On Thu, 28 Jun 2018, Maciej W. Rozycki wrote: > > > + /* Mark that we need support for absolute symbols in the dynamic loader. > > > */ > > > + if (htab != NULL && htab->use_absolute_zero) > > > + i_ehdrp->e_ident[EI_ABIVERSION] = MIPS_LIBC_ABI_ABSOLUTE; > > > + > > > _bfd_elf_post_process_headers (abfd, link_info); > > > } > > > and then `htab->use_absolute_zero' will be TRUE iff a GOT relocation has > > > been calculated to actually refer to a specially-created absolute symbol > > > used to address the issue covered by binutils PR ld/21375. > > > > > > The flag will be FALSE if code has been relaxed to avoid referring the > > > GOT (by using immediate zero instead), in which case no absolute symbol > > > will have been specially created. > > > > > > Does this explanation clear your concern? > > > > Yes, I think this is okay then. > > Great! Any further input? Have we reached consensus? If so, then I think a NEWS entry will be due, such as: * The maximum libc ABI supported has been increased to reflect absolute symbol support. This can be used by the static linker in the ELF file header's EI_ABIVERSION field to indicate such support is required. which I will include in the actual commit. Maciej
On 06/29/2018 12:29 PM, Maciej W. Rozycki wrote: > On Thu, 28 Jun 2018, Maciej W. Rozycki wrote: > >>>> + /* Mark that we need support for absolute symbols in the dynamic loader. >>>> */ >>>> + if (htab != NULL && htab->use_absolute_zero) >>>> + i_ehdrp->e_ident[EI_ABIVERSION] = MIPS_LIBC_ABI_ABSOLUTE; >>>> + >>>> _bfd_elf_post_process_headers (abfd, link_info); >>>> } >>>> and then `htab->use_absolute_zero' will be TRUE iff a GOT relocation has >>>> been calculated to actually refer to a specially-created absolute symbol >>>> used to address the issue covered by binutils PR ld/21375. >>>> >>>> The flag will be FALSE if code has been relaxed to avoid referring the >>>> GOT (by using immediate zero instead), in which case no absolute symbol >>>> will have been specially created. >>>> >>>> Does this explanation clear your concern? >>> >>> Yes, I think this is okay then. >> >> Great! > > Any further input? Have we reached consensus? > > If so, then I think a NEWS entry will be due, such as: > > * The maximum libc ABI supported has been increased to reflect absolute > symbol support. This can be used by the static linker in the ELF file > header's EI_ABIVERSION field to indicate such support is required. Suggest: * The GNU C Library now has correct support for ABSOLUTE symbols (SHN_ABS-relative symbols). Previously such ABSOLUTE symbols were relocated inconsistently or wrongly in some cases. The GNU linker has been enhanced to fix this issue, but it must communicate these newer semantics to the dynamic loader by setting the ELF file's identification (EI_ABIVERSION field) to indicate such support is required. > which I will include in the actual commit. This looks good to me. I don't see an easy way around this problem. I think bumping EI_ABIVERSION is the best cost/value solution there.
Index: glibc/libc-abis =================================================================== --- glibc.orig/libc-abis 2015-05-20 23:10:30.000000000 +0100 +++ glibc/libc-abis 2018-06-16 20:03:55.899525111 +0100 @@ -46,3 +46,5 @@ IFUNC powerpc64-*-linux* IFUNC powerpc-*-linux* IFUNC sparc64-*-linux* IFUNC sparc-*-linux* +# Absolute (SHN_ABS) symbols working correctly. +ABSOLUTE Index: glibc/sysdeps/unix/sysv/linux/mips/libc-abis =================================================================== --- glibc.orig/sysdeps/unix/sysv/linux/mips/libc-abis 2015-05-20 23:10:47.000000000 +0100 +++ glibc/sysdeps/unix/sysv/linux/mips/libc-abis 2018-06-16 20:04:29.888042455 +0100 @@ -14,3 +14,5 @@ UNIQUE # # MIPS O32 FP64 MIPS_O32_FP64 mips*-*-linux* +# Absolute (SHN_ABS) symbols working correctly. +ABSOLUTE