Correct range checking in mallopt/mxfast/tcache [BZ #25194]

From 2da566d7d7c956658d1d6009875ceab85b3d190b Mon Sep 17 00:00:00 2001
From: DJ Delorie <dj@redhat.com>
Date: Tue, 3 Dec 2019 17:44:36 -0500
Subject: Correct range checking in mallopt/mxfast/tcache [BZ #25194]

do_set_tcache_max, do_set_mxfast:
Fix two instances of comparing "size_t < 0"
Both cases have upper limit, so the "negative value" case
is already handled via (undefined) overflow semantics.

mallopt:
pass return value of do_set_mxfast to user.

DJ Delorie wrote:

> Subject: Correct range checking in mallopt/mxfast/tcache [BZ #25194]
>
> do_set_tcache_max, do_set_mxfast:
> Fix two instances of comparing "size_t < 0"
> Both cases have upper limit, so the "negative value" case
> is already handled via (undefined) overflow semantics.

nit: unsigned overflow is defined in C to wrap around, so we don't
have to worry about undefined behavior here.

This also changes the return value from these helpers.  Forgive my
ignorance: where do they get called?  I assumed it would be
__libc_mallopt, but I don't see any calls from there.

Thanks,
Jonathan

DJ Delorie wrote:
> Jonathan Nieder <jrnieder@gmail.com> writes:

>> nit: unsigned overflow is defined in C to wrap around, so we don't
>> have to worry about undefined behavior here.
>
> Including implied overflow by type cast conversions?

Yes, C99 section 6.3.1.3 describes signed-to-unsigned conversions.
Thanks for asking, though, since that wasn't the case I was thinking
of before.

Walking through the caller, we're talking about TUNABLE_SET, which does

  __tunable_set_val (TUNABLE_ENUM_NAME (__top, __ns, __id),		      \
			& (__type) {__val});				      \

The only caller for TUNABLE_SET I see is do_tunable_update_val, which
bounds-checks its input.

>> This also changes the return value from these helpers.  Forgive my
>> ignorance: where do they get called?  I assumed it would be
>> __libc_mallopt, but I don't see any calls from there.
>
> The only two places they could be called from is mallopt() and the
> tunables macros in arena.c.  The tunables code ignores any return value.

Ah --- thanks for the pointer.  (Not about this patch: a comment
mentioning the arena.c tunables macros would be handy for the unwary
reader.)

So the only intended callers are set_tcache_max and set_mxfast (today)
and __libc_mallopt (in the future)?  Makes sense.

Thanks,
Jonathan

On 12/3/19 6:09 PM, DJ Delorie wrote:
> 
> From 2da566d7d7c956658d1d6009875ceab85b3d190b Mon Sep 17 00:00:00 2001
> From: DJ Delorie <dj@redhat.com>
> Date: Tue, 3 Dec 2019 17:44:36 -0500
> Subject: Correct range checking in mallopt/mxfast/tcache [BZ #25194]
> 
> do_set_tcache_max, do_set_mxfast:
> Fix two instances of comparing "size_t < 0"
> Both cases have upper limit, so the "negative value" case
> is already handled via (undefined) overflow semantics.
> 
> mallopt:
> pass return value of do_set_mxfast to user.
> 
> diff --git a/malloc/malloc.c b/malloc/malloc.c
> index 70cc35a473..ed16a72dbd 100644
> --- a/malloc/malloc.c
> +++ b/malloc/malloc.c
> @@ -5086,13 +5086,14 @@ do_set_arena_max (size_t value)
>  static __always_inline int
>  do_set_tcache_max (size_t value)
>  {
> -  if (value >= 0 && value <= MAX_TCACHE_SIZE)
> +  if (value <= MAX_TCACHE_SIZE)

Does this allow mallopt to accept an invalid large negative 
value but after wrapping be treated as less than MAX_TCACHE_SIZE
and thus be accepted instead of rejected?

mallopt takes an int value, and I think we should be checking
that input for validity before casting it.

>      {
>        LIBC_PROBE (memory_tunable_tcache_max_bytes, 2, value, mp_.tcache_max_bytes);
>        mp_.tcache_max_bytes = value;
>        mp_.tcache_bins = csize2tidx (request2size(value)) + 1;
> +      return 1;
>      }
> -  return 1;
> +  return 0;
>  }
>  
>  static __always_inline int
> @@ -5102,8 +5103,9 @@ do_set_tcache_count (size_t value)
>      {
>        LIBC_PROBE (memory_tunable_tcache_count, 2, value, mp_.tcache_count);
>        mp_.tcache_count = value;
> +      return 1;
>      }
> -  return 1;
> +  return 0;
>  }
>  
>  static __always_inline int
> @@ -5119,7 +5121,7 @@ static inline int
>  __always_inline
>  do_set_mxfast (size_t value)
>  {
> -  if (value >= 0 && value <= MAX_FAST_SIZE)
> +  if (value <= MAX_FAST_SIZE)
>      {
>        LIBC_PROBE (memory_mallopt_mxfast, 2, value, get_max_fast ());
>        set_max_fast (value);
> @@ -5147,7 +5149,7 @@ __libc_mallopt (int param_number, int value)
>    switch (param_number)
>      {
>      case M_MXFAST:
> -      do_set_mxfast (value);
> +      res = do_set_mxfast (value);
>        break;
>  
>      case M_TRIM_THRESHOLD:
>

DJ Delorie wrote:
> Jonathan Nieder <jrnieder@gmail.com> writes:

>> So the only intended callers are set_tcache_max and set_mxfast (today)
>> and __libc_mallopt (in the future)?  Makes sense.
>
> Well, mallopt already calls do_set_mxfast, which is where the original
> bug came from.
>
>     case M_MXFAST:
>       res = do_set_mxfast (value);
>       break;

Ah.  You can ignore most of what I wrote except the bit about C99
section 6.3.1.3, then.

This patch feels like three different logical patches squashed
together:

 1. Removing the redundant comparison against zero, which is a no-op
    and thus a nice cleanup.

 2. Updating the return value from do_set_tcache_max and
    do_set_tcache_count, which no caller observes.  This makes it
    more consistent with do_set_mmap_threshold, so it seems like a good
    change (though it also makes me wonder if the tunable interface
    could change to allow these to return void).

 3. Propagating the return value from do_set_mxfast in mallopt.

All three are good changes, so LGTM.

It also makes me wonder: should we propagate the return value from the
other do_set_* calls in mallopt?  Even if they always succeed, this
seems less error-prone in case they're changed later to not always
succeed.

Thanks,
Jonathan