From patchwork Mon Aug 14 15:16:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "H.J. Lu" X-Patchwork-Id: 1821048 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=Poz7yNLa; dkim-atps=neutral Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RPdKg4hphz1yf2 for ; Tue, 15 Aug 2023 01:16:55 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 6D9D53858C33 for ; Mon, 14 Aug 2023 15:16:53 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 6D9D53858C33 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1692026213; bh=0psoIZALJl1/7sG8ZYmNxcfLkRCSdORrxI7GS43ikQY=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=Poz7yNLaOuuPqRJH86ilo0R4n9H5GWIOBS0JSE7sXicpqao8ay56xOgZCRl90BKuG uPoCsHmh0TTHeNXQtgwJ+XRu2E7/a5Buom9hTGfSqvGYt09ARjwcOmBEpijrYhz014 qBGy8U1KetnwgHhIPbJWsTnVg5JWNERCs8moPNFw= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pg1-x52b.google.com (mail-pg1-x52b.google.com [IPv6:2607:f8b0:4864:20::52b]) by sourceware.org (Postfix) with ESMTPS id 2ACCE3858C20 for ; Mon, 14 Aug 2023 15:16:38 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 2ACCE3858C20 Received: by mail-pg1-x52b.google.com with SMTP id 41be03b00d2f7-564b8ea94c1so2609368a12.1 for ; Mon, 14 Aug 2023 08:16:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692026197; x=1692630997; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=0psoIZALJl1/7sG8ZYmNxcfLkRCSdORrxI7GS43ikQY=; b=k51Vhr+4Jk90WWqQKqYbhFjmoPBh0FiBrDWMOFsRfcW4WkNOpty2DMeicrYWgyRVza k9W/8LudLW3r+J52Iqp/SIFG5V7vZBnDnPm3+aYdIgKEO+2AdWL5PRyOZH1lfm13Hwfi nWNfLuKzkvmS/zFgZ05ynjwjh7iIEQz/IdoIP7I5gdY1zg8ZoVCFS7XDctxseKDj8FnG /+FV2xsVrzpCT1n2/zek6bKZS0J/yg7zbaDLdLEE16wZwaq8dcSyQrJhmuNFcmJ1wNS+ 9FehcddTidPC8yRbxmn3tK39pkXDGo7xwGoCyC6+rrMTxg6snhv7tfeikJXvkvWZWpsd Vh2g== X-Gm-Message-State: AOJu0Ywm7DT1zuDRICtLgMcZxnx2UMq9tO+zkUTDkKbvdyicWUy89w13 zW0noPTO0lm6LB02RA+IOGXDcDO5c4g= X-Google-Smtp-Source: AGHT+IGBImxsZGHEYWDWDla/6dUpx3n1AEr4ds6AdyelWkPOE0s0PI8aIGYCGW08yr9t6Wgfg/nsCg== X-Received: by 2002:a17:90b:3c4:b0:268:10a3:cea8 with SMTP id go4-20020a17090b03c400b0026810a3cea8mr7016852pjb.9.1692026196716; Mon, 14 Aug 2023 08:16:36 -0700 (PDT) Received: from gnu-cfl-3.localdomain ([172.59.161.42]) by smtp.gmail.com with ESMTPSA id m22-20020a17090a159600b002684b837d88sm8847889pja.14.2023.08.14.08.16.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Aug 2023 08:16:35 -0700 (PDT) Received: from gnu-cfl-3.. (localhost [IPv6:::1]) by gnu-cfl-3.localdomain (Postfix) with ESMTP id 78962740166 for ; Mon, 14 Aug 2023 08:16:34 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH] x86_64: Add sinh with FMA Date: Mon, 14 Aug 2023 08:16:34 -0700 Message-ID: <20230814151634.2860117-1-hjl.tools@gmail.com> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-Spam-Status: No, score=-3024.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, KAM_SHORT, KAM_STOCKGEN, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: "H.J. Lu via Libc-alpha" From: "H.J. Lu" Reply-To: "H.J. Lu" Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" On Skylake, it improves sinh bench performance by: Before After Improvement max 64.06 60.416 7% min 10.874 9.674 11% mean 14.0546 12.2018 13% NB: Add extern long double __expm1l (long double); extern long double __expm1f128 (long double); for __typeof (__expm1l) and __typeof (__expm1f128) when __expm1 is defined since __expm1 may be expanded in their declarations which causes the build failure. --- sysdeps/ieee754/dbl-64/e_sinh.c | 7 +++++ sysdeps/x86_64/fpu/multiarch/Makefile | 2 ++ sysdeps/x86_64/fpu/multiarch/e_sinh-fma.c | 12 +++++++++ sysdeps/x86_64/fpu/multiarch/e_sinh.c | 32 +++++++++++++++++++++++ 4 files changed, 53 insertions(+) create mode 100644 sysdeps/x86_64/fpu/multiarch/e_sinh-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/e_sinh.c diff --git a/sysdeps/ieee754/dbl-64/e_sinh.c b/sysdeps/ieee754/dbl-64/e_sinh.c index b4b5857ddd..08611c94df 100644 --- a/sysdeps/ieee754/dbl-64/e_sinh.c +++ b/sysdeps/ieee754/dbl-64/e_sinh.c @@ -41,6 +41,11 @@ static char rcsid[] = "$NetBSD: e_sinh.c,v 1.7 1995/05/10 20:46:13 jtc Exp $"; static const double one = 1.0, shuge = 1.0e307; +#ifndef SECTION +# define SECTION +#endif + +SECTION double __ieee754_sinh (double x) { @@ -90,4 +95,6 @@ __ieee754_sinh (double x) /* |x| > overflowthresold, sinh(x) overflow */ return math_narrow_eval (x * shuge); } +#ifndef __ieee754_sinh libm_alias_finite (__ieee754_sinh, __sinh) +#endif diff --git a/sysdeps/x86_64/fpu/multiarch/Makefile b/sysdeps/x86_64/fpu/multiarch/Makefile index add339a876..15c44fc343 100644 --- a/sysdeps/x86_64/fpu/multiarch/Makefile +++ b/sysdeps/x86_64/fpu/multiarch/Makefile @@ -36,6 +36,7 @@ libm-sysdep_routines += \ e_log-fma \ e_log2-fma \ e_pow-fma \ + e_sinh-fma \ s_atan-fma \ s_expm1-fma \ s_sin-fma \ @@ -49,6 +50,7 @@ CFLAGS-e_exp-fma.c = -mfma -mavx2 CFLAGS-e_log-fma.c = -mfma -mavx2 CFLAGS-e_log2-fma.c = -mfma -mavx2 CFLAGS-e_pow-fma.c = -mfma -mavx2 +CFLAGS-e_sinh-fma.c = -mfma -mavx2 -march=x86-64-v3 CFLAGS-s_atan-fma.c = -mfma -mavx2 CFLAGS-s_expm1-fma.c = -mfma -mavx2 CFLAGS-s_sin-fma.c = -mfma -mavx2 diff --git a/sysdeps/x86_64/fpu/multiarch/e_sinh-fma.c b/sysdeps/x86_64/fpu/multiarch/e_sinh-fma.c new file mode 100644 index 0000000000..e0e1e39a7a --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/e_sinh-fma.c @@ -0,0 +1,12 @@ +#define __ieee754_sinh __ieee754_sinh_fma +#define __ieee754_exp __ieee754_exp_fma +#define __expm1 __expm1_fma + +/* NB: __expm1 may be expanded to __expm1_fma in the following + prototypes. */ +extern long double __expm1l (long double); +extern long double __expm1f128 (long double); + +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/e_sinh.c b/sysdeps/x86_64/fpu/multiarch/e_sinh.c new file mode 100644 index 0000000000..4c42adeb21 --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/e_sinh.c @@ -0,0 +1,32 @@ +/* Multiple versions of sinh. + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include + +extern double __redirect_ieee754_sinh (double); + +#define SYMBOL_NAME ieee754_sinh +#include "ifunc-fma.h" + +libc_ifunc_redirected (__redirect_ieee754_sinh, __ieee754_sinh, + IFUNC_SELECTOR ()); +libm_alias_finite (__ieee754_sinh, __sinh) + +#define __ieee754_sinh __ieee754_sinh_sse2 +#include