Message ID | 20220615174129.620476-2-goldstein.w.n@gmail.com |
---|---|
State | New |
Headers | show |
Series | [v4,1/2] x86: Cleanup bounds checking in large memcpy case | expand |
On Wed, Jun 15, 2022 at 10:41 AM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > The lower-bound (131072) and upper-bound (SIZE_MAX / 16) are assumed > by memmove-vec-unaligned-erms. > > The lower-bound is needed because memmove-vec-unaligned-erms unrolls > the loop aggressively in the L(large_memset_4x) case. > > The upper-bound is needed because memmove-vec-unaligned-erms > right-shifts the value of `x86_non_temporal_threshold` by > LOG_4X_MEMCPY_THRESH (4) which without a bound may overflow. > > The lack of lower-bound can be a correctness issue. The lack of > upper-bound cannot. > --- > manual/tunables.texi | 2 +- > sysdeps/x86/dl-cacheinfo.h | 6 +++++- > 2 files changed, 6 insertions(+), 2 deletions(-) > > diff --git a/manual/tunables.texi b/manual/tunables.texi > index 1482412078..49daf3eb4a 100644 > --- a/manual/tunables.texi > +++ b/manual/tunables.texi > @@ -47,7 +47,7 @@ glibc.malloc.mxfast: 0x0 (min: 0x0, max: 0xffffffffffffffff) > glibc.elision.skip_lock_busy: 3 (min: -2147483648, max: 2147483647) > glibc.malloc.top_pad: 0x0 (min: 0x0, max: 0xffffffffffffffff) > glibc.cpu.x86_rep_stosb_threshold: 0x800 (min: 0x1, max: 0xffffffffffffffff) > -glibc.cpu.x86_non_temporal_threshold: 0xc0000 (min: 0x0, max: 0xffffffffffffffff) > +glibc.cpu.x86_non_temporal_threshold: 0xc0000 (min: 0x0, max: 0x0fffffffffffffff) > glibc.cpu.x86_shstk: > glibc.cpu.hwcap_mask: 0x6 (min: 0x0, max: 0xffffffffffffffff) > glibc.malloc.mmap_max: 0 (min: -2147483648, max: 2147483647) > diff --git a/sysdeps/x86/dl-cacheinfo.h b/sysdeps/x86/dl-cacheinfo.h > index cc3b840f9c..f94ff2df43 100644 > --- a/sysdeps/x86/dl-cacheinfo.h > +++ b/sysdeps/x86/dl-cacheinfo.h > @@ -931,8 +931,12 @@ dl_init_cacheinfo (struct cpu_features *cpu_features) > > TUNABLE_SET_WITH_BOUNDS (x86_data_cache_size, data, 0, SIZE_MAX); > TUNABLE_SET_WITH_BOUNDS (x86_shared_cache_size, shared, 0, SIZE_MAX); > + /* SIZE_MAX >> 4 because memmove-vec-unaligned-erms right-shifts the value of > + 'x86_non_temporal_threshold' by `LOG_4X_MEMCPY_THRESH` (4) and it is best > + if that operation cannot overflow. Not the '>> 4' also reflect the bound > + in the manual. */ > TUNABLE_SET_WITH_BOUNDS (x86_non_temporal_threshold, non_temporal_threshold, > - 0, SIZE_MAX); > + 0, SIZE_MAX >> 4); You didn't change the lower bound. > TUNABLE_SET_WITH_BOUNDS (x86_rep_movsb_threshold, rep_movsb_threshold, > minimum_rep_movsb_threshold, SIZE_MAX); > TUNABLE_SET_WITH_BOUNDS (x86_rep_stosb_threshold, rep_stosb_threshold, 1, > -- > 2.34.1 >
On Wed, Jun 15, 2022 at 11:22 AM H.J. Lu <hjl.tools@gmail.com> wrote: > > On Wed, Jun 15, 2022 at 10:41 AM Noah Goldstein <goldstein.w.n@gmail.com> wrote: > > > > The lower-bound (131072) and upper-bound (SIZE_MAX / 16) are assumed > > by memmove-vec-unaligned-erms. > > > > The lower-bound is needed because memmove-vec-unaligned-erms unrolls > > the loop aggressively in the L(large_memset_4x) case. > > > > The upper-bound is needed because memmove-vec-unaligned-erms > > right-shifts the value of `x86_non_temporal_threshold` by > > LOG_4X_MEMCPY_THRESH (4) which without a bound may overflow. > > > > The lack of lower-bound can be a correctness issue. The lack of > > upper-bound cannot. > > --- > > manual/tunables.texi | 2 +- > > sysdeps/x86/dl-cacheinfo.h | 6 +++++- > > 2 files changed, 6 insertions(+), 2 deletions(-) > > > > diff --git a/manual/tunables.texi b/manual/tunables.texi > > index 1482412078..49daf3eb4a 100644 > > --- a/manual/tunables.texi > > +++ b/manual/tunables.texi > > @@ -47,7 +47,7 @@ glibc.malloc.mxfast: 0x0 (min: 0x0, max: 0xffffffffffffffff) > > glibc.elision.skip_lock_busy: 3 (min: -2147483648, max: 2147483647) > > glibc.malloc.top_pad: 0x0 (min: 0x0, max: 0xffffffffffffffff) > > glibc.cpu.x86_rep_stosb_threshold: 0x800 (min: 0x1, max: 0xffffffffffffffff) > > -glibc.cpu.x86_non_temporal_threshold: 0xc0000 (min: 0x0, max: 0xffffffffffffffff) > > +glibc.cpu.x86_non_temporal_threshold: 0xc0000 (min: 0x0, max: 0x0fffffffffffffff) > > glibc.cpu.x86_shstk: > > glibc.cpu.hwcap_mask: 0x6 (min: 0x0, max: 0xffffffffffffffff) > > glibc.malloc.mmap_max: 0 (min: -2147483648, max: 2147483647) > > diff --git a/sysdeps/x86/dl-cacheinfo.h b/sysdeps/x86/dl-cacheinfo.h > > index cc3b840f9c..f94ff2df43 100644 > > --- a/sysdeps/x86/dl-cacheinfo.h > > +++ b/sysdeps/x86/dl-cacheinfo.h > > @@ -931,8 +931,12 @@ dl_init_cacheinfo (struct cpu_features *cpu_features) > > > > TUNABLE_SET_WITH_BOUNDS (x86_data_cache_size, data, 0, SIZE_MAX); > > TUNABLE_SET_WITH_BOUNDS (x86_shared_cache_size, shared, 0, SIZE_MAX); > > + /* SIZE_MAX >> 4 because memmove-vec-unaligned-erms right-shifts the value of > > + 'x86_non_temporal_threshold' by `LOG_4X_MEMCPY_THRESH` (4) and it is best > > + if that operation cannot overflow. Not the '>> 4' also reflect the bound > > + in the manual. */ > > TUNABLE_SET_WITH_BOUNDS (x86_non_temporal_threshold, non_temporal_threshold, > > - 0, SIZE_MAX); > > + 0, SIZE_MAX >> 4); > > You didn't change the lower bound. Fixed in V5. > > > TUNABLE_SET_WITH_BOUNDS (x86_rep_movsb_threshold, rep_movsb_threshold, > > minimum_rep_movsb_threshold, SIZE_MAX); > > TUNABLE_SET_WITH_BOUNDS (x86_rep_stosb_threshold, rep_stosb_threshold, 1, > > -- > > 2.34.1 > > > > > -- > H.J.
diff --git a/manual/tunables.texi b/manual/tunables.texi index 1482412078..49daf3eb4a 100644 --- a/manual/tunables.texi +++ b/manual/tunables.texi @@ -47,7 +47,7 @@ glibc.malloc.mxfast: 0x0 (min: 0x0, max: 0xffffffffffffffff) glibc.elision.skip_lock_busy: 3 (min: -2147483648, max: 2147483647) glibc.malloc.top_pad: 0x0 (min: 0x0, max: 0xffffffffffffffff) glibc.cpu.x86_rep_stosb_threshold: 0x800 (min: 0x1, max: 0xffffffffffffffff) -glibc.cpu.x86_non_temporal_threshold: 0xc0000 (min: 0x0, max: 0xffffffffffffffff) +glibc.cpu.x86_non_temporal_threshold: 0xc0000 (min: 0x0, max: 0x0fffffffffffffff) glibc.cpu.x86_shstk: glibc.cpu.hwcap_mask: 0x6 (min: 0x0, max: 0xffffffffffffffff) glibc.malloc.mmap_max: 0 (min: -2147483648, max: 2147483647) diff --git a/sysdeps/x86/dl-cacheinfo.h b/sysdeps/x86/dl-cacheinfo.h index cc3b840f9c..f94ff2df43 100644 --- a/sysdeps/x86/dl-cacheinfo.h +++ b/sysdeps/x86/dl-cacheinfo.h @@ -931,8 +931,12 @@ dl_init_cacheinfo (struct cpu_features *cpu_features) TUNABLE_SET_WITH_BOUNDS (x86_data_cache_size, data, 0, SIZE_MAX); TUNABLE_SET_WITH_BOUNDS (x86_shared_cache_size, shared, 0, SIZE_MAX); + /* SIZE_MAX >> 4 because memmove-vec-unaligned-erms right-shifts the value of + 'x86_non_temporal_threshold' by `LOG_4X_MEMCPY_THRESH` (4) and it is best + if that operation cannot overflow. Not the '>> 4' also reflect the bound + in the manual. */ TUNABLE_SET_WITH_BOUNDS (x86_non_temporal_threshold, non_temporal_threshold, - 0, SIZE_MAX); + 0, SIZE_MAX >> 4); TUNABLE_SET_WITH_BOUNDS (x86_rep_movsb_threshold, rep_movsb_threshold, minimum_rep_movsb_threshold, SIZE_MAX); TUNABLE_SET_WITH_BOUNDS (x86_rep_stosb_threshold, rep_stosb_threshold, 1,