[AArch64] update libm-test-ulps

On 07/04/15 17:55, Joseph Myers wrote:
> See what I said in 
> <https://sourceware.org/ml/libc-alpha/2015-03/msg00756.html>.  You need to 
> regenerate *from scratch* every so often (say every release cycle) and the 
> presence of *_tonearest entries indicates that the last such from-scratch 
> regeneration was a long time ago.

i removed the old libm-test-ulps and ran make regen-ulps again

is it ok?

Changelog:

2015-04-08  Szabolcs Nagy  <szabolcs.nagy@arm.com>

	* sysdeps/aarch64/libm-test-ulps: Update.

On Wed, 8 Apr 2015, Szabolcs Nagy wrote:

> On 07/04/15 17:55, Joseph Myers wrote:
> > See what I said in 
> > <https://sourceware.org/ml/libc-alpha/2015-03/msg00756.html>.  You need to 
> > regenerate *from scratch* every so often (say every release cycle) and the 
> > presence of *_tonearest entries indicates that the last such from-scratch 
> > regeneration was a long time ago.
> 
> i removed the old libm-test-ulps and ran make regen-ulps again
> 
> is it ok?

libm-test-ulps regeneration is considered obvious.  (The libm-test.inc 
code now ensures that ulps don't get added for functions that should have 
exactly determined results, and that, if added, such ulps are ignored when 
libm-test-ulps is read.  So, given that code

#if TEST_COND_ldbl_128ibm
  /* The documented accuracy of IBM long double division is 3ulp (see
     libgcc/config/rs6000/ibm-ldouble-format), so do not require
     better accuracy for libm functions that are exactly defined for
     other formats.  */
  max_valid_error = exact ? 3 : 14;
#else
  max_valid_error = exact ? 0 : 9;
#endif

there's no real possibility of a bad libm-test-ulps update as long as you 
use "make regen-ulps" rather than trying to update it manually.)

On 08/04/15 16:53, Joseph Myers wrote:
> On Wed, 8 Apr 2015, Szabolcs Nagy wrote:
> 
>> On 07/04/15 17:55, Joseph Myers wrote:
>>> See what I said in 
>>> <https://sourceware.org/ml/libc-alpha/2015-03/msg00756.html>.  You need to 
>>> regenerate *from scratch* every so often (say every release cycle) and the 
>>> presence of *_tonearest entries indicates that the last such from-scratch 
>>> regeneration was a long time ago.
>>
>> i removed the old libm-test-ulps and ran make regen-ulps again
>>
>> is it ok?
> 
> libm-test-ulps regeneration is considered obvious.  (The libm-test.inc 

if i manually have to remove files from the source repository
then it's not obvious..

> code now ensures that ulps don't get added for functions that should have 
> exactly determined results, and that, if added, such ulps are ignored when 
> libm-test-ulps is read.  So, given that code
> 
> #if TEST_COND_ldbl_128ibm
>   /* The documented accuracy of IBM long double division is 3ulp (see
>      libgcc/config/rs6000/ibm-ldouble-format), so do not require
>      better accuracy for libm functions that are exactly defined for
>      other formats.  */
>   max_valid_error = exact ? 3 : 14;
> #else
>   max_valid_error = exact ? 0 : 9;

is more than 9 ulp error considered a bug?
(ie. shall i send bug reports about it?)

On Wed, 8 Apr 2015, Szabolcs Nagy wrote:

> > libm-test-ulps regeneration is considered obvious.  (The libm-test.inc 
> 
> if i manually have to remove files from the source repository
> then it's not obvious..

obvious = commit without review.  Truncating / removing libm-test-ulps 
before regeneration is a documented procedure to use at least once per 
release cycle.

> > #if TEST_COND_ldbl_128ibm
> >   /* The documented accuracy of IBM long double division is 3ulp (see
> >      libgcc/config/rs6000/ibm-ldouble-format), so do not require
> >      better accuracy for libm functions that are exactly defined for
> >      other formats.  */
> >   max_valid_error = exact ? 3 : 14;
> > #else
> >   max_valid_error = exact ? 0 : 9;
> 
> is more than 9 ulp error considered a bug?
> (ie. shall i send bug reports about it?)

Yes, such bugs should be reported to Bugzilla, if they don't duplicate 
existing bugs (we have existing bug reports for Bessel functions, cpow, 
lgamma for negative arguments and atan/atan2 in non-default rounding 
modes, for example, and if the essential implementation issue is the same 
it's not really useful to have extra bug reports just because there are 
multiple implementations with the same bug).  Similarly, non-duplicate 
bugs about any inaccuracy in functions with fully-determined results are 
useful.

I encourage people to look for bugs with mpcheck on non-x86_64 
architectures (as far as I know the tests described at 
<http://www.loria.fr/~zimmerma/glibc/> are for x86_64 only), though I 
don't think results for powerpc would be that useful (because too many 
issues would arise with bugs in the underlying libgcc arithmetic for 
ldbl-128ibm).  Especially, tests for ldbl-128 would be useful, and it's 
quite likely there are issues in function implementations for 32-bit x86.  
Testing with other tools for libm testing is also useful - mpcheck only 
covers a limited subset of libm functions, and even within those it covers 
I've found issues it missed such as bug 18196.

On 08/04/15 17:41, Joseph Myers wrote:
> On Wed, 8 Apr 2015, Szabolcs Nagy wrote:
>> is more than 9 ulp error considered a bug?
>> (ie. shall i send bug reports about it?)
> 
> Yes, such bugs should be reported to Bugzilla, if they don't duplicate 
> existing bugs (we have existing bug reports for Bessel functions, cpow, 
> lgamma for negative arguments and atan/atan2 in non-default rounding 
> modes, for example, and if the essential implementation issue is the same 
> it's not really useful to have extra bug reports just because there are 
> multiple implementations with the same bug).  Similarly, non-duplicate 
> bugs about any inaccuracy in functions with fully-determined results are 
> useful.
> 

i don't see the bessel function bug report on bugzilla (but y0 is hard
to search for) i assume the large ulp errors near zeros are known then.

i don't see lgammal bug report either, my test case is lgammal(-2.5)
which gives -10.465 ulp error on x86_64.

i don't have ldbl-128 tests yet, i plan to do that, but it is significant
amount of work (mpfr could be more helpful by providing functions to
easily get IEEE semantics).

i ran libc-test against glibc now (we use this for musl libc)
and found the listed issues
http://repo.or.cz/w/libc-test.git

api:
BZ #16747 aio.h missing required restrict in lio_listio
BZ #18228 fcntl.h missing O_ constants
BZ #18229 limits.h missing _XOPEN constants
BZ #18230 netinet/in.h IN6_IS_ADDR_* macros are broken
BZ #18231 sys/ipc.h wrong ipc_perm mode member type
BZ #18232 sys/sem.h wrong semid_ds sem_nsems member type
BZ #18233 sys/socket.h wrong msghdr and cmsghdr types
BZ #18234 sys/stat.h missing st_atim, etc in linux generic stat
BZ #18235 utmpx.h utmpx struct ut_tv member type
BZ #18236 wchar.h missing va_list type

functions:
BZ #18237 dn_expand(empty, emptyend, output, 1) fails
BZ #18238 QoI fmin/fmax on signed zeros
BZ #18239 fnmatch "[[:a]" does not fail
BZ #18240 hcreate((size_t)-1) should fail
BZ #16637 inet_pton accepts invalid ipv6 address
BZ #18241 fseek does not work on memstreams
BZ #18242 powf(0,-INFINITY) does not raise divbyzero
BZ #18243 cancellation point bugs: sem_wait, sem_timedwait, shm_open
BZ #18244 remquol(0, INFINITY, &q) returns nan
BZ #18245 sqrtl(-1.1L) raises the inexact flag
BZ #18246 sscanf("0x12", "%2d", &x) should fail
BZ #18247 strtof is not correctly rounded
BZ #18248 omitted underflow in math functions

some unclear cases others might be interested in:

ERE /(a)\1/ matches "aa" not "a1" in glibc, but ERE should not
support backrefs. (however \1 is ub in the spec so anything goes)

i think regcomp should fail on illegal sequence in a pattern
including /\xff/ in UTF-8 locale

%a printf format behaviour on glibc is ugly, but i don't see exact
spec for it (subnormals and . alignment are not what i expect)

there are some corner cases for fnmatch where i'm not yet sure
what's the right behaviour: "[\\]" and "\\" patterns

shm_open does not work with static linking because __shm_directory
depends on dynamic linking (fails at link time), sem_open likewise
(except it segfaults at runtime)

rounding functions (ceil,trunc,floor,round) don't have specified
inexact behaviour, but i think the next C floating-point standard
will be more strict, on x86_64 i see inconsistent raise of inexact.

BZ #13154 was closed WONTFIX but i think strtol("0",&endptr,99)
should set endptr for quality of implementation: the failure of
strtol is usually checked by comparing endptr to the input, which
is UB if it's not set.

> I encourage people to look for bugs with mpcheck on non-x86_64 
> architectures (as far as I know the tests described at 
> <http://www.loria.fr/~zimmerma/glibc/> are for x86_64 only), though I 
> don't think results for powerpc would be that useful (because too many 
> issues would arise with bugs in the underlying libgcc arithmetic for 
> ldbl-128ibm).  Especially, tests for ldbl-128 would be useful, and it's 
> quite likely there are issues in function implementations for 32-bit x86.  
> Testing with other tools for libm testing is also useful - mpcheck only 
> covers a limited subset of libm functions, and even within those it covers 
> I've found issues it missed such as bug 18196.

i think i have full coverage in libc-test, but the code needs to be
reorganised a bit to make it easy to compare mpfr results with libc
results on random inputs.

On Fri, 10 Apr 2015, Szabolcs Nagy wrote:

> i don't see the bessel function bug report on bugzilla (but y0 is hard
> to search for) i assume the large ulp errors near zeros are known then.

See bugs 2555, 2567, 2577, 14469, 14470, 14471, 14472, 16492, 16507, 
16559, at least.  The basic issue is much the same everywhere (at least 
where inaccuracy near zeros is concerned).

> i don't see lgammal bug report either, my test case is lgammal(-2.5)
> which gives -10.465 ulp error on x86_64.

See bugs 2542, 2543, 2558 (gamma = lgamma - and since the underlying issue 
is the same in all implementations, extra bugs for long double versions 
aren't really that useful; I think only the completely separate ia64 
implementation might be accurate near zeros).

> api:
> BZ #16747 aio.h missing required restrict in lio_listio
> BZ #18228 fcntl.h missing O_ constants
> BZ #18229 limits.h missing _XOPEN constants
> BZ #18230 netinet/in.h IN6_IS_ADDR_* macros are broken
> BZ #18231 sys/ipc.h wrong ipc_perm mode member type
> BZ #18232 sys/sem.h wrong semid_ds sem_nsems member type
> BZ #18233 sys/socket.h wrong msghdr and cmsghdr types
> BZ #18234 sys/stat.h missing st_atim, etc in linux generic stat
> BZ #18235 utmpx.h utmpx struct ut_tv member type
> BZ #18236 wchar.h missing va_list type

I presume many of these correspond to (XFAILed) failures of conform/ tests 
(some such failures indicate actual API bugs, some indicate bugs in the 
tests - anyway, where there are actual API bugs, having them in Bugzilla 
is useful, since they'd need to be filed there as part of a fix if not 
filed before).

> shm_open does not work with static linking because __shm_directory
> depends on dynamic linking (fails at link time), sem_open likewise
> (except it segfaults at runtime)

Why does it depend on dynamic linking?  That looks rather like a bug to 
me.  Functions should work for static linking unless there are strong 
reasons it can't be supported.

> rounding functions (ceil,trunc,floor,round) don't have specified
> inexact behaviour, but i think the next C floating-point standard
> will be more strict, on x86_64 i see inconsistent raise of inexact.

See bug 15479.

I hope other people will help review / fix more of these bugs (we still 
need more people adopting the role of experts in particular glibc areas 
and systematically reviewing and fixing open bugs and looking for related 
issues in the code as they do so ...).

* Joseph Myers <joseph@codesourcery.com> [2015-04-10 17:42:23 +0000]:
> On Fri, 10 Apr 2015, Szabolcs Nagy wrote:
> 
> > shm_open does not work with static linking because __shm_directory
> > depends on dynamic linking (fails at link time), sem_open likewise
> > (except it segfaults at runtime)
> 
> Why does it depend on dynamic linking?  That looks rather like a bug to 
> me.  Functions should work for static linking unless there are strong 
> reasons it can't be supported.
> 

ok i was wrong (probably)

shm_open is in librt which references __shm_directory in libpthread
and i got the link error because the test system used -lrt -lpthread
in the wrong order

then __shm_directory segfaults because it uses __libc_once which
uses weakref __pthread_once for whatever reason

so the way to statically link something with shm_open is

cc foo.o -lrt -Wl,--whole-archive -lpthread -Wl,--no-whole-archive

but i find this solution unsatisfying

Szabolcs Nagy <nsz@port70.net> writes:

> so the way to statically link something with shm_open is
>
> cc foo.o -lrt -Wl,--whole-archive -lpthread -Wl,--no-whole-archive
>
> but i find this solution unsatisfying

This is BZ #5784.

Andreas.