| Message ID | 20240201122216.2634007-7-aleksander.lobakin@intel.com |
|---|---|
| State | Handled Elsewhere |
| Headers | show |
| Series | ice: add PFCP filter support | expand |
On Thu, Feb 01, 2024 at 01:22:01PM +0100, Alexander Lobakin wrote: > Since commit b03fc1173c0c ("bitops: let optimize out non-atomic bitops > on compile-time constants"), the compilers are able to expand inline > bitmap operations to compile-time initializers when possible. > However, during the round of replacement if-__set-else-__clear with > __assign_bit() as per Andy's advice, bloat-o-meter showed +1024 bytes > difference in object code size for one module (even one function), > where the pattern: > > DECLARE_BITMAP(foo) = { }; // on the stack, zeroed > > if (a) > __set_bit(const_bit_num, foo); > if (b) > __set_bit(another_const_bit_num, foo); > ... > > is heavily used, although there should be no difference: the bitmap is > zeroed, so the second half of __assign_bit() should be compiled-out as > a no-op. > I either missed the fact that __assign_bit() has bitmap pointer marked > as `volatile` (as we usually do for bitops) or was hoping that the > compilers would at least try to look past the `volatile` for > __always_inline functions. Anyhow, due to that attribute, the compilers > were always compiling the whole expression and no mentioned compile-time > optimizations were working. > > Convert __assign_bit() to a macro since it's a very simple if-else and > all of the checks are performed inside __set_bit() and __clear_bit(), > thus that wrapper has to be as transparent as possible. After that > change, despite it showing only -20 bytes change for vmlinux (due to > that it's still relatively unpopular), no drastic code size changes > happen when replacing if-set-else-clear for onstack bitmaps with > __assign_bit(), meaning the compiler now expands them to the actual > operations will all the expected optimizations. > > Atomic assign_bit() is less affected due to its nature, but let's > convert it to a macro as well to keep the code consistent and not > leave a place for possible suboptimal codegen. Moreover, with certain > kernel configuration it actually gives some saves (x86): > > do_ip_setsockopt 4154 4099 -55 > > Suggested-by: Yury Norov <yury.norov@gmail.com> # assign_bit(), too > Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com> > Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com> > Signed-off-by: Alexander Lobakin <aleksander.lobakin@intel.com> Acked-by: Yury Norov <yury.norov@gmail.com>
diff --git a/include/linux/bitops.h b/include/linux/bitops.h index e0cd09eb91cd..b25dc8742124 100644 --- a/include/linux/bitops.h +++ b/include/linux/bitops.h @@ -275,23 +275,11 @@ static inline unsigned long fns(unsigned long word, unsigned int n) * @addr: the address to start counting from * @value: the value to assign */ -static __always_inline void assign_bit(long nr, volatile unsigned long *addr, - bool value) -{ - if (value) - set_bit(nr, addr); - else - clear_bit(nr, addr); -} +#define assign_bit(nr, addr, value) \ + ((value) ? set_bit((nr), (addr)) : clear_bit((nr), (addr))) -static __always_inline void __assign_bit(long nr, volatile unsigned long *addr, - bool value) -{ - if (value) - __set_bit(nr, addr); - else - __clear_bit(nr, addr); -} +#define __assign_bit(nr, addr, value) \ + ((value) ? __set_bit((nr), (addr)) : __clear_bit((nr), (addr))) /** * __ptr_set_bit - Set bit in a pointer's value
Since commit b03fc1173c0c ("bitops: let optimize out non-atomic bitops on compile-time constants"), the compilers are able to expand inline bitmap operations to compile-time initializers when possible. However, during the round of replacement if-__set-else-__clear with __assign_bit() as per Andy's advice, bloat-o-meter showed +1024 bytes difference in object code size for one module (even one function), where the pattern: DECLARE_BITMAP(foo) = { }; // on the stack, zeroed if (a) __set_bit(const_bit_num, foo); if (b) __set_bit(another_const_bit_num, foo); ... is heavily used, although there should be no difference: the bitmap is zeroed, so the second half of __assign_bit() should be compiled-out as a no-op. I either missed the fact that __assign_bit() has bitmap pointer marked as `volatile` (as we usually do for bitops) or was hoping that the compilers would at least try to look past the `volatile` for __always_inline functions. Anyhow, due to that attribute, the compilers were always compiling the whole expression and no mentioned compile-time optimizations were working. Convert __assign_bit() to a macro since it's a very simple if-else and all of the checks are performed inside __set_bit() and __clear_bit(), thus that wrapper has to be as transparent as possible. After that change, despite it showing only -20 bytes change for vmlinux (due to that it's still relatively unpopular), no drastic code size changes happen when replacing if-set-else-clear for onstack bitmaps with __assign_bit(), meaning the compiler now expands them to the actual operations will all the expected optimizations. Atomic assign_bit() is less affected due to its nature, but let's convert it to a macro as well to keep the code consistent and not leave a place for possible suboptimal codegen. Moreover, with certain kernel configuration it actually gives some saves (x86): do_ip_setsockopt 4154 4099 -55 Suggested-by: Yury Norov <yury.norov@gmail.com> # assign_bit(), too Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com> Signed-off-by: Alexander Lobakin <aleksander.lobakin@intel.com> --- include/linux/bitops.h | 20 ++++---------------- 1 file changed, 4 insertions(+), 16 deletions(-)