Message ID | fd879f60-3f0b-48d1-bfa1-6d337768207e@linux.ibm.com (mailing list archive) |
---|---|
State | Handled Elsewhere, archived |
Headers | show |
Series | uapi/auxvec: Define AT_HWCAP3 and AT_HWCAP4 aux vector, entries | expand |
Context | Check | Description |
---|---|---|
snowpatch_ozlabs/github-powerpc_clang | success | Successfully ran 6 jobs. |
snowpatch_ozlabs/github-powerpc_kernel_qemu | success | Successfully ran 23 jobs. |
snowpatch_ozlabs/github-powerpc_sparse | success | Successfully ran 4 jobs. |
On 26/09/23 19:02, Peter Bergner wrote: > The powerpc toolchain keeps a copy of the HWCAP bit masks in our TCB for fast > access by our __builtin_cpu_supports built-in function. The TCB space for > the HWCAP entries - which are created in pairs - is an ABI extension, so > waiting to create the space for HWCAP3 and HWCAP4 until we need them is > problematical, given distro unwillingness to apply ABI modifying patches > to distro point releases. Define AT_HWCAP3 and AT_HWCAP4 in the generic > uapi header so they can be used in GLIBC to reserve space in the powerpc > TCB for their future use. This is different than previously exported auxv, where each AT_* constant would have a auxv entry. On glibc it would require changing _dl_parse_auxv to iterate over the auxv_values to find AT_HWCAP3/AT_HWCAP4 (not ideal, but doable). Wouldn't be better to always export it on fs/binfmt_elf.c, along with all the machinery to setup it (ELF_HWCAP3, etc), along with proper documentation? > > I scanned both the Linux and GLIBC source codes looking for unused AT_* > values and 29 and 30 did not seem to be used, so they are what I went > with. If anyone sees a problem with using those specific values, I'm > amenable to using other values, just let me know what would be better. > > Peter > > > Signed-off-by: Peter Bergner <bergner@linux.ibm.com> > --- > include/uapi/linux/auxvec.h | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/include/uapi/linux/auxvec.h b/include/uapi/linux/auxvec.h > index 6991c4b8ab18..cc61cb9b3e9a 100644 > --- a/include/uapi/linux/auxvec.h > +++ b/include/uapi/linux/auxvec.h > @@ -32,6 +32,8 @@ > #define AT_HWCAP2 26 /* extension of AT_HWCAP */ > #define AT_RSEQ_FEATURE_SIZE 27 /* rseq supported feature size */ > #define AT_RSEQ_ALIGN 28 /* rseq allocation alignment */ > +#define AT_HWCAP3 29 /* extension of AT_HWCAP */ > +#define AT_HWCAP4 30 /* extension of AT_HWCAP */ > > #define AT_EXECFN 31 /* filename of program */ >
Hi Adhemerval, sorry for the delay in replying, I was a little under the weather last week. On 9/27/23 11:03 AM, Adhemerval Zanella Netto wrote: > On 26/09/23 19:02, Peter Bergner wrote: >> The powerpc toolchain keeps a copy of the HWCAP bit masks in our TCB for fast >> access by our __builtin_cpu_supports built-in function. The TCB space for >> the HWCAP entries - which are created in pairs - is an ABI extension, so >> waiting to create the space for HWCAP3 and HWCAP4 until we need them is >> problematical, given distro unwillingness to apply ABI modifying patches >> to distro point releases. Define AT_HWCAP3 and AT_HWCAP4 in the generic >> uapi header so they can be used in GLIBC to reserve space in the powerpc >> TCB for their future use. > > This is different than previously exported auxv, where each AT_* constant > would have a auxv entry. On glibc it would require changing _dl_parse_auxv > to iterate over the auxv_values to find AT_HWCAP3/AT_HWCAP4 (not ideal, > but doable). When you say different, do you mean because all AUXVs exported by the kernel *will* have an AT_HWCAP and AT_HWCAP2 entry and AT_HWCAP3/AT_HWCAP4 won't? I don't think that's a problem for either kernel or glibc side of things. I'm not even sure there is a guarantee that every AT_* value *must* be present in the exported AUXV. The new AT_HWCAP3/AT_HWCAP4 defines are less than AT_MINSIGSTKSZ, so they don't affect the size of _dl_parse_auxv's auxv_values array size and the AT_HWCAP3 and AT_HWCAP4 entries in auxv_values[] are already initialized to zero today. Additionally, the loop in _dl_parse_auxv already parses the entire AUXV, so there is no extra work for it to do, unless and until AT_HWCAP3 and AT_HWCAP4 start being exported by the kernel. Really, the only extra work _dl_parse_auxv would need to do, is add two new stores: GLRO(dl_hwcap3) = auxv_values[AT_HWCAP3]; GLRO(dl_hwcap4) = auxv_values[AT_HWCAP4]; > Wouldn't be better to always export it on fs/binfmt_elf.c, along with all > the machinery to setup it (ELF_HWCAP3, etc), along with proper documentation? You mean modify the kernel now to export AT_HWCAP3 and AT_HWCAP4 as zero masks? Is that really necessary since we don't need or have any features defined in them yet? GLIBC's _dl_parse_auxv doesn't really need them to be exported either, since in the absence of the entries, it will just end up using zero masks for dl_hwcap3 and dl_hwcap4, so everything is copacetic even without any kernel changes. As I mentioned, our real problem is the lead time for getting changes that affect the user ABI into a distro release, and ppc's copy/cache of the HWCAP masks is an ABI change. If we wait to add this support until when we actually have a need for HWCAP3, then we won't have any support until the next major distro release. However, if we can add this support now, which I don't think is an onerous change on glibc's part, then we can start using it immediately when Linux starts exporting them. Peter
On 02/10/23 18:19, Peter Bergner wrote: > Hi Adhemerval, sorry for the delay in replying, I was a little under the > weather last week. > > > On 9/27/23 11:03 AM, Adhemerval Zanella Netto wrote: >> On 26/09/23 19:02, Peter Bergner wrote: >>> The powerpc toolchain keeps a copy of the HWCAP bit masks in our TCB for fast >>> access by our __builtin_cpu_supports built-in function. The TCB space for >>> the HWCAP entries - which are created in pairs - is an ABI extension, so >>> waiting to create the space for HWCAP3 and HWCAP4 until we need them is >>> problematical, given distro unwillingness to apply ABI modifying patches >>> to distro point releases. Define AT_HWCAP3 and AT_HWCAP4 in the generic >>> uapi header so they can be used in GLIBC to reserve space in the powerpc >>> TCB for their future use. >> >> This is different than previously exported auxv, where each AT_* constant >> would have a auxv entry. On glibc it would require changing _dl_parse_auxv >> to iterate over the auxv_values to find AT_HWCAP3/AT_HWCAP4 (not ideal, >> but doable). > > When you say different, do you mean because all AUXVs exported by the kernel > *will* have an AT_HWCAP and AT_HWCAP2 entry and AT_HWCAP3/AT_HWCAP4 won't? > I don't think that's a problem for either kernel or glibc side of things. > I'm not even sure there is a guarantee that every AT_* value *must* be > present in the exported AUXV. > > The new AT_HWCAP3/AT_HWCAP4 defines are less than AT_MINSIGSTKSZ, so they > don't affect the size of _dl_parse_auxv's auxv_values array size and the > AT_HWCAP3 and AT_HWCAP4 entries in auxv_values[] are already initialized > to zero today. Additionally, the loop in _dl_parse_auxv already parses > the entire AUXV, so there is no extra work for it to do, unless and until > AT_HWCAP3 and AT_HWCAP4 start being exported by the kernel. Really, the > only extra work _dl_parse_auxv would need to do, is add two new stores: > > GLRO(dl_hwcap3) = auxv_values[AT_HWCAP3]; > GLRO(dl_hwcap4) = auxv_values[AT_HWCAP4]; > Indeed you are right, I wrong assumed that the AT_HWCAP3/AT_HWCAP4 would be higher than AT_MINSIGSTKSZ. > > >> Wouldn't be better to always export it on fs/binfmt_elf.c, along with all >> the machinery to setup it (ELF_HWCAP3, etc), along with proper documentation? > > You mean modify the kernel now to export AT_HWCAP3 and AT_HWCAP4 as zero > masks? Is that really necessary since we don't need or have any features > defined in them yet? GLIBC's _dl_parse_auxv doesn't really need them to > be exported either, since in the absence of the entries, it will just end > up using zero masks for dl_hwcap3 and dl_hwcap4, so everything is copacetic > even without any kernel changes. > > As I mentioned, our real problem is the lead time for getting changes that > affect the user ABI into a distro release, and ppc's copy/cache of the HWCAP > masks is an ABI change. If we wait to add this support until when we > actually have a need for HWCAP3, then we won't have any support until > the next major distro release. However, if we can add this support now, > which I don't think is an onerous change on glibc's part, then we can > start using it immediately when Linux starts exporting them. What it is not clear to me is what kind of ABI boundary you are trying to preemptively add support here. The TCB ABI for __builtin_cpu_supports is userland only, so if your intention is just to allow gcc to work on older glibcs, it should be a matter to just reserve the space on tcbhead_t. Once kernel actually provides AT_HWCAP3/AT_HWCAP4, backporting should be straightforward. If your intention is to also add support on glibc, it makes more sense to already reserve it. For __builtin_cpu_supports it should work, although for glibc itself some backporting would be required (to correctly showing the bits with LD_SHOW_AUXV).
On 10/3/23 9:08 AM, Adhemerval Zanella Netto wrote: > What it is not clear to me is what kind of ABI boundary you are trying to > preemptively add support here. The TCB ABI for __builtin_cpu_supports is > userland only, so if your intention is just to allow gcc to work on older > glibcs, it should be a matter to just reserve the space on tcbhead_t. Yes, extending tcbhead_t to contain the slots for hwcap3 and hwcap4 are the ABI extensions we are interested in, and not something that can be backported into a distro point release. Yes, we don't strictly need the AT_HWCAP3 and AT_HWCAP4 kernel defines to reserve (and clear) that space in glibc, but.... > If your intention is to also add support on glibc, it makes more sense to > already reserve it. For __builtin_cpu_supports it should work, although > for glibc itself some backporting would be required (to correctly showing > the bits with LD_SHOW_AUXV). Our intention is to also add the glibc support too once we have the AT_HWCAP3 and AT_HWCAP4 kernel macros defined. 1) Once the defines are there, adding the support should be pretty straight forward, so why wait? And 2) part of the glibc and compiler support introduces a new symbol that is exported by glibc and referenced by the compilers to ensure the compilers *never* access the hwcap* fields in the TCB unless the glibc supports them. See the symbol __parse_hwcap_and_convert_at_platform used for HWCAP/HWCAP2. We'll need a similar one for HWCAP3/HWCAP4 and I'm doubtful whether the distros will allow the backport of a patch that introduces a new exported symbol from glibc in a distro point release. Peter
On 03/10/23 19:12, Peter Bergner wrote: > On 10/3/23 9:08 AM, Adhemerval Zanella Netto wrote: >> What it is not clear to me is what kind of ABI boundary you are trying to >> preemptively add support here. The TCB ABI for __builtin_cpu_supports is >> userland only, so if your intention is just to allow gcc to work on older >> glibcs, it should be a matter to just reserve the space on tcbhead_t. > > Yes, extending tcbhead_t to contain the slots for hwcap3 and hwcap4 are the > ABI extensions we are interested in, and not something that can be backported > into a distro point release. Yes, we don't strictly need the AT_HWCAP3 and > AT_HWCAP4 kernel defines to reserve (and clear) that space in glibc, but.... > > > >> If your intention is to also add support on glibc, it makes more sense to >> already reserve it. For __builtin_cpu_supports it should work, although >> for glibc itself some backporting would be required (to correctly showing >> the bits with LD_SHOW_AUXV). > > Our intention is to also add the glibc support too once we have the > AT_HWCAP3 and AT_HWCAP4 kernel macros defined. 1) Once the defines are > there, adding the support should be pretty straight forward, so why wait? > And 2) part of the glibc and compiler support introduces a new symbol > that is exported by glibc and referenced by the compilers to ensure the > compilers *never* access the hwcap* fields in the TCB unless the glibc > supports them. See the symbol __parse_hwcap_and_convert_at_platform used > for HWCAP/HWCAP2. We'll need a similar one for HWCAP3/HWCAP4 and I'm > doubtful whether the distros will allow the backport of a patch that > introduces a new exported symbol from glibc in a distro point release. Alright, I makes more sense it now. And indeed backporting a __parse_hwcap for HWCAP3/HWCAP4 will be frown upon.
CCing linux-kernel for more exposure. PING. I'm waiting on a reply from anyone on the kernel side of things to see whether they have an issue with reserving values for AT_HWCAP3 and AT_HWCAP4. I'll note reviews from the GLIBC camp did not have an issue with the below patch. Thanks. Peter On 9/26/23 5:02 PM, Peter Bergner wrote: > The powerpc toolchain keeps a copy of the HWCAP bit masks in our TCB for fast > access by our __builtin_cpu_supports built-in function. The TCB space for > the HWCAP entries - which are created in pairs - is an ABI extension, so > waiting to create the space for HWCAP3 and HWCAP4 until we need them is > problematical, given distro unwillingness to apply ABI modifying patches > to distro point releases. Define AT_HWCAP3 and AT_HWCAP4 in the generic > uapi header so they can be used in GLIBC to reserve space in the powerpc > TCB for their future use. > > I scanned both the Linux and GLIBC source codes looking for unused AT_* > values and 29 and 30 did not seem to be used, so they are what I went > with. If anyone sees a problem with using those specific values, I'm > amenable to using other values, just let me know what would be better. > > Peter > > > Signed-off-by: Peter Bergner <bergner@linux.ibm.com> > --- > include/uapi/linux/auxvec.h | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/include/uapi/linux/auxvec.h b/include/uapi/linux/auxvec.h > index 6991c4b8ab18..cc61cb9b3e9a 100644 > --- a/include/uapi/linux/auxvec.h > +++ b/include/uapi/linux/auxvec.h > @@ -32,6 +32,8 @@ > #define AT_HWCAP2 26 /* extension of AT_HWCAP */ > #define AT_RSEQ_FEATURE_SIZE 27 /* rseq supported feature size */ > #define AT_RSEQ_ALIGN 28 /* rseq allocation alignment */ > +#define AT_HWCAP3 29 /* extension of AT_HWCAP */ > +#define AT_HWCAP4 30 /* extension of AT_HWCAP */ > > #define AT_EXECFN 31 /* filename of program */ >
The 10/17/2023 18:14, Peter Bergner wrote: > CCing linux-kernel for more exposure. > > PING. I'm waiting on a reply from anyone on the kernel side of things > to see whether they have an issue with reserving values for AT_HWCAP3 > and AT_HWCAP4. > > I'll note reviews from the GLIBC camp did not have an issue with the below patch. fwiw, aarch64 is quickly filling up AT_HWCAP2 so this will be useful for arm64 too eventually, but we are not in a hurry. > > +#define AT_HWCAP3 29 /* extension of AT_HWCAP */ > > +#define AT_HWCAP4 30 /* extension of AT_HWCAP */
diff --git a/include/uapi/linux/auxvec.h b/include/uapi/linux/auxvec.h index 6991c4b8ab18..cc61cb9b3e9a 100644 --- a/include/uapi/linux/auxvec.h +++ b/include/uapi/linux/auxvec.h @@ -32,6 +32,8 @@ #define AT_HWCAP2 26 /* extension of AT_HWCAP */ #define AT_RSEQ_FEATURE_SIZE 27 /* rseq supported feature size */ #define AT_RSEQ_ALIGN 28 /* rseq allocation alignment */ +#define AT_HWCAP3 29 /* extension of AT_HWCAP */ +#define AT_HWCAP4 30 /* extension of AT_HWCAP */ #define AT_EXECFN 31 /* filename of program */
The powerpc toolchain keeps a copy of the HWCAP bit masks in our TCB for fast access by our __builtin_cpu_supports built-in function. The TCB space for the HWCAP entries - which are created in pairs - is an ABI extension, so waiting to create the space for HWCAP3 and HWCAP4 until we need them is problematical, given distro unwillingness to apply ABI modifying patches to distro point releases. Define AT_HWCAP3 and AT_HWCAP4 in the generic uapi header so they can be used in GLIBC to reserve space in the powerpc TCB for their future use. I scanned both the Linux and GLIBC source codes looking for unused AT_* values and 29 and 30 did not seem to be used, so they are what I went with. If anyone sees a problem with using those specific values, I'm amenable to using other values, just let me know what would be better. Peter Signed-off-by: Peter Bergner <bergner@linux.ibm.com> --- include/uapi/linux/auxvec.h | 2 ++ 1 file changed, 2 insertions(+)