From patchwork Tue Aug 8 14:07:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Ramsay X-Patchwork-Id: 1818627 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=8.43.85.97; helo=server2.sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=BSQUbM7s; dkim-atps=neutral Received: from server2.sourceware.org (ip-8-43-85-97.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RKw6Y2Cg6z1yYC for ; Wed, 9 Aug 2023 00:09:25 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 43D713870935 for ; Tue, 8 Aug 2023 14:09:23 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 43D713870935 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1691503763; bh=VzHoHWDm0A7AUp+Nt6zELd9XjtRjERBqyut+uujCl4I=; h=To:CC:Subject:Date:List-Id:List-Unsubscribe:List-Archive: List-Post:List-Help:List-Subscribe:From:Reply-To:From; b=BSQUbM7sx37mnWlAiUvCI9VOUWJCV3Utq4jni/7/PK1XIv5GSldYYIs3HuLRzORi9 TB3pYTPsB1u/F5cRlh8WkqMxFjsR7M73OTH2Ir745AnTztOsHWQNLII4XiLHbK02GR wc/1n71nhFDD21geE/qUUHcPEWHFvyvrMA/3BDqw= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2068.outbound.protection.outlook.com [40.107.20.68]) by sourceware.org (Postfix) with ESMTPS id B366C385735A for ; Tue, 8 Aug 2023 14:07:26 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org B366C385735A Received: from AM5PR1001CA0031.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:206:2::44) by AS4PR08MB7712.eurprd08.prod.outlook.com (2603:10a6:20b:513::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.26; Tue, 8 Aug 2023 14:07:23 +0000 Received: from AM7EUR03FT045.eop-EUR03.prod.protection.outlook.com (2603:10a6:206:2:cafe::69) by AM5PR1001CA0031.outlook.office365.com (2603:10a6:206:2::44) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.27 via Frontend Transport; Tue, 8 Aug 2023 14:07:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM7EUR03FT045.mail.protection.outlook.com (100.127.140.150) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6678.17 via Frontend Transport; Tue, 8 Aug 2023 14:07:22 +0000 Received: ("Tessian outbound 997ae1cc9f47:v145"); Tue, 08 Aug 2023 14:07:22 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: adeecdb9bc0df8fa X-CR-MTA-TID: 64aa7808 Received: from 3ed4d360348d.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 72A1DDB5-52CC-4C26-9B14-5B9299D3E10B.1; Tue, 08 Aug 2023 14:07:15 +0000 Received: from EUR04-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 3ed4d360348d.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 08 Aug 2023 14:07:15 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XmF07ffXw59MnGQHNp1aRTCFWwaLH4mLg5Jxrlk9H0viTEaDp4NC30HDrowOoqnu1HvahdUtr9f20PsN0bnRQdBJS3k4sWXCePmGquz3mfNYi62Lff9dFs7hYRg0hDKk03Q5OqtgkYjMoNj5pvsdGUG8QYtJOqsxnRlFVGoCJKqjJqigM+FWHPDv9zL5HDHoOzZ0O+GQd/zKPgX2iQG3mEjjZdIujUKuap/GWCWeiXE2lDxFYdTgKlXEqredHhJkmrd7PHR4buoHSg+zKwjtNSk3bNnaif3J2mYYQKIkH0vYkY2H4l8oZA0C3JUz7R43lcCSjZBDp0Z262sexFMg7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VzHoHWDm0A7AUp+Nt6zELd9XjtRjERBqyut+uujCl4I=; b=aMD3kSyyPLrKxttfUQK950bWpUK7K3hC8TMiH4xu47X4A24c/IXeERMEiyXbCNhKzeipkIEQyT+rHkjEbee94BHs3/d8fSV5Gl0hMTL0Vwj5V7z/pePf9lkQgPw4OA4TeC3hLonqCadvUifJ7/WspYC3SjN24+eXkX/kLS34lURxSU6xJU14nD5mKJlEKxZUAZj/MMBpL/XHlawxdvTe32Z2E3qG9RppbPIl9xhVLqIn4eVniHO96An2HRFZolLMcfpNI2OCC7jH/uhQCOnGeNL3Ulr514D6bZXD/4GkpkhyM2xBw3GEeh6mR91ldp7/QoEAmcrQIb41+NUMboeW7Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none Received: from AS4P190CA0031.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:5d1::6) by AS8PR08MB7886.eurprd08.prod.outlook.com (2603:10a6:20b:52a::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.27; Tue, 8 Aug 2023 14:07:12 +0000 Received: from AM7EUR03FT060.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:5d1:cafe::a4) by AS4P190CA0031.outlook.office365.com (2603:10a6:20b:5d1::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.27 via Frontend Transport; Tue, 8 Aug 2023 14:07:12 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (40.67.248.234) by AM7EUR03FT060.mail.protection.outlook.com (100.127.140.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6678.16 via Frontend Transport; Tue, 8 Aug 2023 14:07:11 +0000 Received: from AZ-NEU-EX02.Emea.Arm.com (10.251.26.5) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Tue, 8 Aug 2023 14:07:11 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX02.Emea.Arm.com (10.251.26.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Tue, 8 Aug 2023 14:07:10 +0000 Received: from vcn-man-apps.manchester.arm.com (10.32.108.22) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.27 via Frontend Transport; Tue, 8 Aug 2023 14:07:10 +0000 To: CC: Joe Ramsay Subject: [PATCH 1/5 v2] aarch64: Add vector implementations of tan routines Date: Tue, 8 Aug 2023 15:07:05 +0100 Message-ID: <20230808140709.47733-1-Joe.Ramsay@arm.com> X-Mailer: git-send-email 2.27.0 MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: AM7EUR03FT060:EE_|AS8PR08MB7886:EE_|AM7EUR03FT045:EE_|AS4PR08MB7712:EE_ X-MS-Office365-Filtering-Correlation-Id: bd835f87-7604-4aae-77dc-08db9818c96c x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: ZszxLL9TLRwxJBAdM1g/mFCpsaUsJwYgVLwgZkScDk5keGFTptiKeK/QXIpv348b2op0w7aKZ7NOxnTDUH12vIR1VXwXB6/Ki4c9WE8N3IJFTzQq+vViyOz5Gks/siK0rOQ+8paFGi/VTrLUDTjd18YSbjJ2RsI2DWU4z+3rPvqb+qx5bWk+Pc9Xn2HpmjT+PnJ0BQ3KAbbCJwZeV/EFZYX4RH4CU5NAYN5wNg77/yW0ohUVxI691Zld7oAtiGldCWwlUs24AiI+M8c8kdYZWSP2gEW9C96BBbSpGxOdQDFWaRUiOLp30GJy9f2sAoHyClrZnhIsWS3tAiFne+OJEbgEsdlzmF8EcpFTWHs3Sr0Js7V2N30/9cTg1HjMGzneOZuWPxShKXkLF1nM1/MbePTbqM1YKKpnTjD2tfHSNCkqrb41nqWXrcHrqxay5g21rQ1mzhsR34FflbBYMBf6iO09WF7TrlN2wEV5teB9i6BxnyL7YJu3cxuEyn848vs7W9yeAsFU0AUpJDADEHgjSjaxfK5kG5gzSuTR3GRL2lbVxHKzmItVPmsGwN4vWqNjAarBSip4zQ0B6oXECMxbJ7zaTn4rlMteMIsp910K0DqH8nP1+POCvWGzP38foiYAeWOsQsDlPmwjpFfpUP1ciYuY+ODrk7NiGH/hQtgYutKPZNMGgUpeTpAvHjYogntAEuwaH0uaYd+Bh9ixZpDKJ67MsT8H4uoRcLOr6NJa88w8FtgOBIdMukM+XiTTOtTuqKP5Y3PDxzt58Mkw6yzVodYnB1h+JbhIks+KHt4JGhzKgo4j3VHgG4HsQgR1AQNs X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(396003)(376002)(346002)(39860400002)(136003)(451199021)(82310400008)(1800799003)(186006)(40470700004)(46966006)(36840700001)(426003)(8676002)(8936002)(5660300002)(4326008)(6916009)(41300700001)(316002)(47076005)(83380400001)(40480700001)(86362001)(40460700003)(30864003)(36860700001)(2906002)(6666004)(7696005)(2616005)(1076003)(26005)(36756003)(336012)(70586007)(70206006)(356005)(478600001)(81166007)(82740400003)(2004002)(36900700001)(579004)(559001)(473944003); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB7886 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM7EUR03FT045.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 01f06a44-fba6-4bf1-3e63-08db9818c2f0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kPsb8go/VH/egYFV4IF8dHyMoKxXSAr/Yr7hR5l9PBRTK0caFRkQHvgzOT2SA1PID9/KOVYlqWBDFf6bYrpDJhEErr4azKV5sBwssfVOkKQDC1fEtKizYuBbsIOAUITSAKf58xvlTwYNUruYrXaE2LvfGWKunNVc0oX9kJ0B0BRrrtEGEcbnqM2Htuk3QIEgRG1uJcRZ8Eehu5Zunal4rc0O88dRiIg7MPPBtlnVRJTo0on+/k1Q9qmshHPv/7dCIc8iPf8QZjQuubS/+KfV0HEjVcV+bixWihfQy9eIGDpQm7XKIf2WmOKBo+l0tTcv3iIsUjBt94uJlqBs2RRHd2Ns+VP5Kv8sNl0X+4bBKZSkoWiax1XaUzonfOpsy2y1Vpn2z9f5Rx5UkhL+LMRlfWwdYDWK2jBMgjbnfPRKUyHpL3u1mlZMMSdmYvcBITBipKjQeW2ezVme7+XAqHQMYdE2QsD1WhbkMjO6pvTsOczG56/fHLFpU9rYBgD7zotLEhlPDz0ykH+XqPmKiz/C0TK1iUxLt25QVD2vt4uiOKJE5+04svHdnSCrdkvJ4lXHO8h5ptFDDkN6evhgEzDuZsOeV4uUEE8WQFM6shmdIPcnbJs/zWQVnbWzQyLyzAj1ktuDCEFg+KmacA2P/jNM60ZYbHVJK77sYKf0Slcf4zDZDGvpe+JWiBd/+vkUjT5NHjUZ7/xPtt4W+C6Lkg7iMTzgJkVc1uXz8MNNO4w0tTIrGG9j6oHkdRzxEPm72MhDFD+19gYgrdkdKMgX7ThO+A== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230028)(4636009)(396003)(39860400002)(376002)(136003)(346002)(82310400008)(451199021)(186006)(1800799003)(46966006)(40470700004)(36840700001)(26005)(40480700001)(1076003)(70586007)(40460700003)(30864003)(36756003)(41300700001)(86362001)(5660300002)(4326008)(2906002)(6916009)(70206006)(316002)(8936002)(8676002)(6666004)(7696005)(47076005)(478600001)(82740400003)(81166007)(2616005)(36860700001)(426003)(83380400001)(336012)(2004002)(579004)(559001)(473944003); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Aug 2023 14:07:22.8242 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bd835f87-7604-4aae-77dc-08db9818c96c X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM7EUR03FT045.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS4PR08MB7712 X-Spam-Status: No, score=-12.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Joe Ramsay via Libc-alpha From: Joe Ramsay Reply-To: Joe Ramsay Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" This includes some utility headers for evaluating polynomials using various schemes. --- Changes to v1: Fix check-abi Thanks, Joe sysdeps/aarch64/fpu/Makefile | 3 +- sysdeps/aarch64/fpu/Versions | 6 + sysdeps/aarch64/fpu/bits/math-vector.h | 4 + sysdeps/aarch64/fpu/poly_advsimd_f32.h | 36 ++ sysdeps/aarch64/fpu/poly_advsimd_f64.h | 36 ++ sysdeps/aarch64/fpu/poly_generic.h | 285 ++++++++++++++++ sysdeps/aarch64/fpu/poly_sve_f32.h | 38 +++ sysdeps/aarch64/fpu/poly_sve_f64.h | 38 +++ sysdeps/aarch64/fpu/poly_sve_generic.h | 313 ++++++++++++++++++ sysdeps/aarch64/fpu/tan_advsimd.c | 123 +++++++ sysdeps/aarch64/fpu/tan_sve.c | 106 ++++++ sysdeps/aarch64/fpu/tanf_advsimd.c | 134 ++++++++ sysdeps/aarch64/fpu/tanf_sve.c | 123 +++++++ .../fpu/test-double-advsimd-wrappers.c | 1 + .../aarch64/fpu/test-double-sve-wrappers.c | 1 + .../aarch64/fpu/test-float-advsimd-wrappers.c | 1 + sysdeps/aarch64/fpu/test-float-sve-wrappers.c | 1 + sysdeps/aarch64/libm-test-ulps | 8 + .../unix/sysv/linux/aarch64/libmvec.abilist | 4 + 19 files changed, 1260 insertions(+), 1 deletion(-) create mode 100644 sysdeps/aarch64/fpu/poly_advsimd_f32.h create mode 100644 sysdeps/aarch64/fpu/poly_advsimd_f64.h create mode 100644 sysdeps/aarch64/fpu/poly_generic.h create mode 100644 sysdeps/aarch64/fpu/poly_sve_f32.h create mode 100644 sysdeps/aarch64/fpu/poly_sve_f64.h create mode 100644 sysdeps/aarch64/fpu/poly_sve_generic.h create mode 100644 sysdeps/aarch64/fpu/tan_advsimd.c create mode 100644 sysdeps/aarch64/fpu/tan_sve.c create mode 100644 sysdeps/aarch64/fpu/tanf_advsimd.c create mode 100644 sysdeps/aarch64/fpu/tanf_sve.c diff --git a/sysdeps/aarch64/fpu/Makefile b/sysdeps/aarch64/fpu/Makefile index 04aa2e37ca..a1bbc9bcaa 100644 --- a/sysdeps/aarch64/fpu/Makefile +++ b/sysdeps/aarch64/fpu/Makefile @@ -1,7 +1,8 @@ libmvec-supported-funcs = cos \ exp \ log \ - sin + sin \ + tan float-advsimd-funcs = $(libmvec-supported-funcs) double-advsimd-funcs = $(libmvec-supported-funcs) diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions index c85c0f3efb..f0ca0940a9 100644 --- a/sysdeps/aarch64/fpu/Versions +++ b/sysdeps/aarch64/fpu/Versions @@ -17,4 +17,10 @@ libmvec { _ZGVsMxv_sin; _ZGVsMxv_sinf; } + GLIBC_2.39 { + _ZGVnN4v_tanf; + _ZGVnN2v_tan; + _ZGVsMxv_tanf; + _ZGVsMxv_tan; + } } diff --git a/sysdeps/aarch64/fpu/bits/math-vector.h b/sysdeps/aarch64/fpu/bits/math-vector.h index 7c200599c1..6193213147 100644 --- a/sysdeps/aarch64/fpu/bits/math-vector.h +++ b/sysdeps/aarch64/fpu/bits/math-vector.h @@ -53,11 +53,13 @@ __vpcs __f32x4_t _ZGVnN4v_cosf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_expf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_logf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_sinf (__f32x4_t); +__vpcs __f32x4_t _ZGVnN4v_tanf (__f32x4_t); __vpcs __f64x2_t _ZGVnN2v_cos (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_sin (__f64x2_t); +__vpcs __f64x2_t _ZGVnN2v_tan (__f64x2_t); # undef __ADVSIMD_VEC_MATH_SUPPORTED #endif /* __ADVSIMD_VEC_MATH_SUPPORTED */ @@ -68,11 +70,13 @@ __sv_f32_t _ZGVsMxv_cosf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_expf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_logf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_sinf (__sv_f32_t, __sv_bool_t); +__sv_f32_t _ZGVsMxv_tanf (__sv_f32_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_cos (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_sin (__sv_f64_t, __sv_bool_t); +__sv_f64_t _ZGVsMxv_tan (__sv_f64_t, __sv_bool_t); # undef __SVE_VEC_MATH_SUPPORTED #endif /* __SVE_VEC_MATH_SUPPORTED */ diff --git a/sysdeps/aarch64/fpu/poly_advsimd_f32.h b/sysdeps/aarch64/fpu/poly_advsimd_f32.h new file mode 100644 index 0000000000..9e2ad9ad94 --- /dev/null +++ b/sysdeps/aarch64/fpu/poly_advsimd_f32.h @@ -0,0 +1,36 @@ +/* Helpers for evaluating polynomials on single-precision AdvSIMD input, using + various schemes. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef AARCH64_FPU_POLY_ADVSIMD_F32_H +#define AARCH64_FPU_POLY_ADVSIMD_F32_H + +#include + +/* Wrap AdvSIMD f32 helpers: evaluation of some scheme/order has form: + v_[scheme]_[order]_f32. */ +#define VTYPE float32x4_t +#define FMA(x, y, z) vfmaq_f32 (z, x, y) +#define VWRAP(f) v_##f##_f32 +#include "poly_generic.h" +#undef VWRAP +#undef FMA +#undef VTYPE + +#endif diff --git a/sysdeps/aarch64/fpu/poly_advsimd_f64.h b/sysdeps/aarch64/fpu/poly_advsimd_f64.h new file mode 100644 index 0000000000..955cfc08ce --- /dev/null +++ b/sysdeps/aarch64/fpu/poly_advsimd_f64.h @@ -0,0 +1,36 @@ +/* Helpers for evaluating polynomials on double-precision AdvSIMD input, using + various schemes. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef AARCH64_FPU_POLY_ADVSIMD_F64_H +#define AARCH64_FPU_POLY_ADVSIMD_F64_H + +#include + +/* Wrap AdvSIMD f64 helpers: evaluation of some scheme/order has form: + v_[scheme]_[order]_f64. */ +#define VTYPE float64x2_t +#define FMA(x, y, z) vfmaq_f64 (z, x, y) +#define VWRAP(f) v_##f##_f64 +#include "poly_generic.h" +#undef VWRAP +#undef FMA +#undef VTYPE + +#endif diff --git a/sysdeps/aarch64/fpu/poly_generic.h b/sysdeps/aarch64/fpu/poly_generic.h new file mode 100644 index 0000000000..84f042182b --- /dev/null +++ b/sysdeps/aarch64/fpu/poly_generic.h @@ -0,0 +1,285 @@ +/* Generic helpers for evaluating polynomials with various schemes. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + + +#ifndef VTYPE +# error Cannot use poly_generic without defining VTYPE +#endif +#ifndef VWRAP +# error Cannot use poly_generic without defining VWRAP +#endif +#ifndef FMA +# error Cannot use poly_generic without defining FMA +#endif + +static inline VTYPE VWRAP (pairwise_poly_3) (VTYPE x, VTYPE x2, + const VTYPE *poly) +{ + /* At order 3, Estrin and Pairwise Horner are identical. */ + VTYPE p01 = FMA (poly[1], x, poly[0]); + VTYPE p23 = FMA (poly[3], x, poly[2]); + return FMA (p23, x2, p01); +} + +static inline VTYPE VWRAP (estrin_4) (VTYPE x, VTYPE x2, VTYPE x4, + const VTYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (x, x2, poly); + return FMA (poly[4], x4, p03); +} +static inline VTYPE VWRAP (estrin_5) (VTYPE x, VTYPE x2, VTYPE x4, + const VTYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (x, x2, poly); + VTYPE p45 = FMA (poly[5], x, poly[4]); + return FMA (p45, x4, p03); +} +static inline VTYPE VWRAP (estrin_6) (VTYPE x, VTYPE x2, VTYPE x4, + const VTYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (x, x2, poly); + VTYPE p45 = FMA (poly[5], x, poly[4]); + VTYPE p46 = FMA (poly[6], x2, p45); + return FMA (p46, x4, p03); +} +static inline VTYPE VWRAP (estrin_7) (VTYPE x, VTYPE x2, VTYPE x4, + const VTYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (x, x2, poly); + VTYPE p47 = VWRAP (pairwise_poly_3) (x, x2, poly + 4); + return FMA (p47, x4, p03); +} +static inline VTYPE VWRAP (estrin_8) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + return FMA (poly[8], x8, VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_9) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + VTYPE p89 = FMA (poly[9], x, poly[8]); + return FMA (p89, x8, VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_10) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + VTYPE p89 = FMA (poly[9], x, poly[8]); + VTYPE p8_10 = FMA (poly[10], x2, p89); + return FMA (p8_10, x8, VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_11) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + VTYPE p8_11 = VWRAP (pairwise_poly_3) (x, x2, poly + 8); + return FMA (p8_11, x8, VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_12) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + return FMA (VWRAP (estrin_4) (x, x2, x4, poly + 8), x8, + VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_13) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + return FMA (VWRAP (estrin_5) (x, x2, x4, poly + 8), x8, + VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_14) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + return FMA (VWRAP (estrin_6) (x, x2, x4, poly + 8), x8, + VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_15) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + return FMA (VWRAP (estrin_7) (x, x2, x4, poly + 8), x8, + VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_16) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + VTYPE x16, const VTYPE *poly) +{ + return FMA (poly[16], x16, VWRAP (estrin_15) (x, x2, x4, x8, poly)); +} +static inline VTYPE VWRAP (estrin_17) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + VTYPE x16, const VTYPE *poly) +{ + VTYPE p16_17 = FMA (poly[17], x, poly[16]); + return FMA (p16_17, x16, VWRAP (estrin_15) (x, x2, x4, x8, poly)); +} +static inline VTYPE VWRAP (estrin_18) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + VTYPE x16, const VTYPE *poly) +{ + VTYPE p16_17 = FMA (poly[17], x, poly[16]); + VTYPE p16_18 = FMA (poly[18], x2, p16_17); + return FMA (p16_18, x16, VWRAP (estrin_15) (x, x2, x4, x8, poly)); +} +static inline VTYPE VWRAP (estrin_19) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + VTYPE x16, const VTYPE *poly) +{ + VTYPE p16_19 = VWRAP (pairwise_poly_3) (x, x2, poly + 16); + return FMA (p16_19, x16, VWRAP (estrin_15) (x, x2, x4, x8, poly)); +} + +static inline VTYPE VWRAP (horner_3) (VTYPE x, const VTYPE *poly) +{ + VTYPE p = FMA (poly[3], x, poly[2]); + p = FMA (x, p, poly[1]); + p = FMA (x, p, poly[0]); + return p; +} +static inline VTYPE VWRAP (horner_4) (VTYPE x, const VTYPE *poly) +{ + VTYPE p = FMA (poly[4], x, poly[3]); + p = FMA (x, p, poly[2]); + p = FMA (x, p, poly[1]); + p = FMA (x, p, poly[0]); + return p; +} +static inline VTYPE VWRAP (horner_5) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_4) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_6) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_5) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_7) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_6) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_8) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_7) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_9) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_8) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_10) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_9) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_11) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_10) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_12) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_11) (x, poly + 1), poly[0]); +} + +static inline VTYPE VWRAP (pw_horner_4) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p01 = FMA (poly[1], x, poly[0]); + VTYPE p23 = FMA (poly[3], x, poly[2]); + VTYPE p; + p = FMA (x2, poly[4], p23); + p = FMA (x2, p, p01); + return p; +} +static inline VTYPE VWRAP (pw_horner_5) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p01 = FMA (poly[1], x, poly[0]); + VTYPE p23 = FMA (poly[3], x, poly[2]); + VTYPE p45 = FMA (poly[5], x, poly[4]); + VTYPE p; + p = FMA (x2, p45, p23); + p = FMA (x2, p, p01); + return p; +} +static inline VTYPE VWRAP (pw_horner_6) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p26 = VWRAP (pw_horner_4) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p26, p01); +} +static inline VTYPE VWRAP (pw_horner_7) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p27 = VWRAP (pw_horner_5) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p27, p01); +} +static inline VTYPE VWRAP (pw_horner_8) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p28 = VWRAP (pw_horner_6) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p28, p01); +} +static inline VTYPE VWRAP (pw_horner_9) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p29 = VWRAP (pw_horner_7) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p29, p01); +} +static inline VTYPE VWRAP (pw_horner_10) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_10 = VWRAP (pw_horner_8) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_10, p01); +} +static inline VTYPE VWRAP (pw_horner_11) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_11 = VWRAP (pw_horner_9) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_11, p01); +} +static inline VTYPE VWRAP (pw_horner_12) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_12 = VWRAP (pw_horner_10) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_12, p01); +} +static inline VTYPE VWRAP (pw_horner_13) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_13 = VWRAP (pw_horner_11) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_13, p01); +} +static inline VTYPE VWRAP (pw_horner_14) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_14 = VWRAP (pw_horner_12) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_14, p01); +} +static inline VTYPE VWRAP (pw_horner_15) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_15 = VWRAP (pw_horner_13) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_15, p01); +} +static inline VTYPE VWRAP (pw_horner_16) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_16 = VWRAP (pw_horner_14) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_16, p01); +} +static inline VTYPE VWRAP (pw_horner_17) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_17 = VWRAP (pw_horner_15) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_17, p01); +} +static inline VTYPE VWRAP (pw_horner_18) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_18 = VWRAP (pw_horner_16) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_18, p01); +} diff --git a/sysdeps/aarch64/fpu/poly_sve_f32.h b/sysdeps/aarch64/fpu/poly_sve_f32.h new file mode 100644 index 0000000000..dcf2fab8dd --- /dev/null +++ b/sysdeps/aarch64/fpu/poly_sve_f32.h @@ -0,0 +1,38 @@ +/* Helpers for evaluating polynomials on single-precision SVE input, using + various schemes. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef AARCH64_FPU_POLY_SVE_F32_H +#define AARCH64_FPU_POLY_SVE_F32_H + +#include + +/* Wrap SVE f32 helpers: evaluation of some scheme/order has form: + sv_[scheme]_[order]_f32_x. */ +#define VTYPE svfloat32_t +#define STYPE float +#define VWRAP(f) sv_##f##_f32_x +#define DUP svdup_n_f32 +#include "poly_sve_generic.h" +#undef DUP +#undef VWRAP +#undef STYPE +#undef VTYPE + +#endif diff --git a/sysdeps/aarch64/fpu/poly_sve_f64.h b/sysdeps/aarch64/fpu/poly_sve_f64.h new file mode 100644 index 0000000000..97a0b76637 --- /dev/null +++ b/sysdeps/aarch64/fpu/poly_sve_f64.h @@ -0,0 +1,38 @@ +/* Helpers for evaluating polynomials on double-precision SVE input, using + various schemes. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef AARCH64_FPU_POLY_SVE_F64_H +#define AARCH64_FPU_POLY_SVE_F64_H + +#include + +/* Wrap SVE f64 helpers: evaluation of some scheme/order has form: + sv_[scheme]_[order]_f64_x. */ +#define VTYPE svfloat64_t +#define STYPE double +#define VWRAP(f) sv_##f##_f64_x +#define DUP svdup_n_f64 +#include "poly_sve_generic.h" +#undef DUP +#undef VWRAP +#undef STYPE +#undef VTYPE + +#endif diff --git a/sysdeps/aarch64/fpu/poly_sve_generic.h b/sysdeps/aarch64/fpu/poly_sve_generic.h new file mode 100644 index 0000000000..0ecf5ce45b --- /dev/null +++ b/sysdeps/aarch64/fpu/poly_sve_generic.h @@ -0,0 +1,313 @@ +/* Helpers for evaluating polynomials with various schemes - specific to SVE + but precision-agnostic. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef VTYPE +# error Cannot use poly_generic without defining VTYPE +#endif +#ifndef STYPE +# error Cannot use poly_generic without defining STYPE +#endif +#ifndef VWRAP +# error Cannot use poly_generic without defining VWRAP +#endif +#ifndef DUP +# error Cannot use poly_generic without defining DUP +#endif + +static inline VTYPE VWRAP (pairwise_poly_3) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + /* At order 3, Estrin and Pairwise Horner are identical. */ + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + VTYPE p23 = svmla_x (pg, DUP (poly[2]), x, poly[3]); + return svmla_x (pg, p01, p23, x2); +} + +static inline VTYPE VWRAP (estrin_4) (svbool_t pg, VTYPE x, VTYPE x2, VTYPE x4, + const STYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (pg, x, x2, poly); + return svmla_x (pg, p03, x4, poly[4]); +} +static inline VTYPE VWRAP (estrin_5) (svbool_t pg, VTYPE x, VTYPE x2, VTYPE x4, + const STYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (pg, x, x2, poly); + VTYPE p45 = svmla_x (pg, DUP (poly[4]), x, poly[5]); + return svmla_x (pg, p03, p45, x4); +} +static inline VTYPE VWRAP (estrin_6) (svbool_t pg, VTYPE x, VTYPE x2, VTYPE x4, + const STYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (pg, x, x2, poly); + VTYPE p45 = svmla_x (pg, DUP (poly[4]), x, poly[5]); + VTYPE p46 = svmla_x (pg, p45, x, poly[6]); + return svmla_x (pg, p03, p46, x4); +} +static inline VTYPE VWRAP (estrin_7) (svbool_t pg, VTYPE x, VTYPE x2, VTYPE x4, + const STYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (pg, x, x2, poly); + VTYPE p47 = VWRAP (pairwise_poly_3) (pg, x, x2, poly + 4); + return svmla_x (pg, p03, p47, x4); +} +static inline VTYPE VWRAP (estrin_8) (svbool_t pg, VTYPE x, VTYPE x2, VTYPE x4, + VTYPE x8, const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), x8, poly[8]); +} +static inline VTYPE VWRAP (estrin_9) (svbool_t pg, VTYPE x, VTYPE x2, VTYPE x4, + VTYPE x8, const STYPE *poly) +{ + VTYPE p89 = svmla_x (pg, DUP (poly[8]), x, poly[9]); + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), p89, x8); +} +static inline VTYPE VWRAP (estrin_10) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, const STYPE *poly) +{ + VTYPE p89 = svmla_x (pg, DUP (poly[8]), x, poly[9]); + VTYPE p8_10 = svmla_x (pg, p89, x2, poly[10]); + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), p8_10, x8); +} +static inline VTYPE VWRAP (estrin_11) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, const STYPE *poly) +{ + VTYPE p8_11 = VWRAP (pairwise_poly_3) (pg, x, x2, poly + 8); + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), p8_11, x8); +} +static inline VTYPE VWRAP (estrin_12) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), + VWRAP (estrin_4) (pg, x, x2, x4, poly + 8), x8); +} +static inline VTYPE VWRAP (estrin_13) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), + VWRAP (estrin_5) (pg, x, x2, x4, poly + 8), x8); +} +static inline VTYPE VWRAP (estrin_14) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), + VWRAP (estrin_6) (pg, x, x2, x4, poly + 8), x8); +} +static inline VTYPE VWRAP (estrin_15) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), + VWRAP (estrin_7) (pg, x, x2, x4, poly + 8), x8); +} +static inline VTYPE VWRAP (estrin_16) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, VTYPE x16, + const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_15) (pg, x, x2, x4, x8, poly), x16, + poly[16]); +} +static inline VTYPE VWRAP (estrin_17) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, VTYPE x16, + const STYPE *poly) +{ + VTYPE p16_17 = svmla_x (pg, DUP (poly[16]), x, poly[17]); + return svmla_x (pg, VWRAP (estrin_15) (pg, x, x2, x4, x8, poly), p16_17, + x16); +} +static inline VTYPE VWRAP (estrin_18) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, VTYPE x16, + const STYPE *poly) +{ + VTYPE p16_17 = svmla_x (pg, DUP (poly[16]), x, poly[17]); + VTYPE p16_18 = svmla_x (pg, p16_17, x2, poly[18]); + return svmla_x (pg, VWRAP (estrin_15) (pg, x, x2, x4, x8, poly), p16_18, + x16); +} +static inline VTYPE VWRAP (estrin_19) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, VTYPE x16, + const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_15) (pg, x, x2, x4, x8, poly), + VWRAP (pairwise_poly_3) (pg, x, x2, poly + 16), x16); +} + +static inline VTYPE VWRAP (horner_3) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + VTYPE p = svmla_x (pg, DUP (poly[2]), x, poly[3]); + p = svmad_x (pg, x, p, poly[1]); + p = svmad_x (pg, x, p, poly[0]); + return p; +} +static inline VTYPE VWRAP (horner_4) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + VTYPE p = svmla_x (pg, DUP (poly[3]), x, poly[4]); + p = svmad_x (pg, x, p, poly[2]); + p = svmad_x (pg, x, p, poly[1]); + p = svmad_x (pg, x, p, poly[0]); + return p; +} +static inline VTYPE VWRAP (horner_5) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, VWRAP (horner_4) (pg, x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_6) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, VWRAP (horner_5) (pg, x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_7) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, VWRAP (horner_6) (pg, x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_8) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, VWRAP (horner_7) (pg, x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_9) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, VWRAP (horner_8) (pg, x, poly + 1), poly[0]); +} +static inline VTYPE +sv_horner_10_f32_x (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, VWRAP (horner_9) (pg, x, poly + 1), poly[0]); +} +static inline VTYPE +sv_horner_11_f32_x (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, sv_horner_10_f32_x (pg, x, poly + 1), poly[0]); +} +static inline VTYPE +sv_horner_12_f32_x (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, sv_horner_11_f32_x (pg, x, poly + 1), poly[0]); +} + +static inline VTYPE VWRAP (pw_horner_4) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + VTYPE p23 = svmla_x (pg, DUP (poly[2]), x, poly[3]); + VTYPE p; + p = svmla_x (pg, p23, x2, poly[4]); + p = svmla_x (pg, p01, x2, p); + return p; +} +static inline VTYPE VWRAP (pw_horner_5) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + VTYPE p23 = svmla_x (pg, DUP (poly[2]), x, poly[3]); + VTYPE p45 = svmla_x (pg, DUP (poly[4]), x, poly[5]); + VTYPE p; + p = svmla_x (pg, p23, x2, p45); + p = svmla_x (pg, p01, x2, p); + return p; +} +static inline VTYPE VWRAP (pw_horner_6) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p26 = VWRAP (pw_horner_4) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p26); +} +static inline VTYPE VWRAP (pw_horner_7) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p27 = VWRAP (pw_horner_5) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p27); +} +static inline VTYPE VWRAP (pw_horner_8) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p28 = VWRAP (pw_horner_6) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p28); +} +static inline VTYPE VWRAP (pw_horner_9) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p29 = VWRAP (pw_horner_7) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p29); +} +static inline VTYPE VWRAP (pw_horner_10) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_10 = VWRAP (pw_horner_8) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_10); +} +static inline VTYPE VWRAP (pw_horner_11) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_11 = VWRAP (pw_horner_9) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_11); +} +static inline VTYPE VWRAP (pw_horner_12) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_12 = VWRAP (pw_horner_10) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_12); +} +static inline VTYPE VWRAP (pw_horner_13) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_13 = VWRAP (pw_horner_11) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_13); +} +static inline VTYPE VWRAP (pw_horner_14) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_14 = VWRAP (pw_horner_12) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_14); +} +static inline VTYPE VWRAP (pw_horner_15) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_15 = VWRAP (pw_horner_13) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_15); +} +static inline VTYPE VWRAP (pw_horner_16) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_16 = VWRAP (pw_horner_14) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_16); +} +static inline VTYPE VWRAP (pw_horner_17) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_17 = VWRAP (pw_horner_15) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_17); +} +static inline VTYPE VWRAP (pw_horner_18) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_18 = VWRAP (pw_horner_16) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_18); +} diff --git a/sysdeps/aarch64/fpu/tan_advsimd.c b/sysdeps/aarch64/fpu/tan_advsimd.c new file mode 100644 index 0000000000..936a0569c8 --- /dev/null +++ b/sysdeps/aarch64/fpu/tan_advsimd.c @@ -0,0 +1,123 @@ +/* Double-precision vector (Advanced SIMD) tan function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f64.h" + +static const struct data +{ + float64x2_t poly[9]; + float64x2_t half_pi_hi, half_pi_lo, two_over_pi, shift; +#if !WANT_SIMD_EXCEPT + float64x2_t range_val; +#endif +} data = { + /* Coefficients generated using FPMinimax. */ + .poly = { V2 (0x1.5555555555556p-2), V2 (0x1.1111111110a63p-3), + V2 (0x1.ba1ba1bb46414p-5), V2 (0x1.664f47e5b5445p-6), + V2 (0x1.226e5e5ecdfa3p-7), V2 (0x1.d6c7ddbf87047p-9), + V2 (0x1.7ea75d05b583ep-10), V2 (0x1.289f22964a03cp-11), + V2 (0x1.4e4fd14147622p-12) }, + .half_pi_hi = V2 (0x1.921fb54442d18p0), + .half_pi_lo = V2 (0x1.1a62633145c07p-54), + .two_over_pi = V2 (0x1.45f306dc9c883p-1), + .shift = V2 (0x1.8p52), +#if !WANT_SIMD_EXCEPT + .range_val = V2 (0x1p23), +#endif +}; + +#define RangeVal 0x4160000000000000 /* asuint64(0x1p23). */ +#define TinyBound 0x3e50000000000000 /* asuint64(2^-26). */ +#define Thresh 0x310000000000000 /* RangeVal - TinyBound. */ + +/* Special cases (fall back to scalar calls). */ +static float64x2_t VPCS_ATTR NOINLINE +special_case (float64x2_t x) +{ + return v_call_f64 (tan, x, x, v_u64 (-1)); +} + +/* Vector approximation for double-precision tan. + Maximum measured error is 3.48 ULP: + __v_tan(0x1.4457047ef78d8p+20) got -0x1.f6ccd8ecf7dedp+37 + want -0x1.f6ccd8ecf7deap+37. */ +float64x2_t VPCS_ATTR V_NAME_D1 (tan) (float64x2_t x) +{ + const struct data *dat = ptr_barrier (&data); + /* Our argument reduction cannot calculate q with sufficient accuracy for very + large inputs. Fall back to scalar routine for all lanes if any are too + large, or Inf/NaN. If fenv exceptions are expected, also fall back for tiny + input to avoid underflow. */ +#if WANT_SIMD_EXCEPT + uint64x2_t iax = vreinterpretq_u64_f64 (vabsq_f64 (x)); + /* iax - tiny_bound > range_val - tiny_bound. */ + uint64x2_t special + = vcgtq_u64 (vsubq_u64 (iax, v_u64 (TinyBound)), v_u64 (Thresh)); + if (__glibc_unlikely (v_any_u64 (special))) + return special_case (x); +#endif + + /* q = nearest integer to 2 * x / pi. */ + float64x2_t q + = vsubq_f64 (vfmaq_f64 (dat->shift, x, dat->two_over_pi), dat->shift); + int64x2_t qi = vcvtq_s64_f64 (q); + + /* Use q to reduce x to r in [-pi/4, pi/4], by: + r = x - q * pi/2, in extended precision. */ + float64x2_t r = x; + r = vfmsq_f64 (r, q, dat->half_pi_hi); + r = vfmsq_f64 (r, q, dat->half_pi_lo); + /* Further reduce r to [-pi/8, pi/8], to be reconstructed using double angle + formula. */ + r = vmulq_n_f64 (r, 0.5); + + /* Approximate tan(r) using order 8 polynomial. + tan(x) is odd, so polynomial has the form: + tan(x) ~= x + C0 * x^3 + C1 * x^5 + C3 * x^7 + ... + Hence we first approximate P(r) = C1 + C2 * r^2 + C3 * r^4 + ... + Then compute the approximation by: + tan(r) ~= r + r^3 * (C0 + r^2 * P(r)). */ + float64x2_t r2 = vmulq_f64 (r, r), r4 = vmulq_f64 (r2, r2), + r8 = vmulq_f64 (r4, r4); + /* Offset coefficients to evaluate from C1 onwards. */ + float64x2_t p = v_estrin_7_f64 (r2, r4, r8, dat->poly + 1); + p = vfmaq_f64 (dat->poly[0], p, r2); + p = vfmaq_f64 (r, r2, vmulq_f64 (p, r)); + + /* Recombination uses double-angle formula: + tan(2x) = 2 * tan(x) / (1 - (tan(x))^2) + and reciprocity around pi/2: + tan(x) = 1 / (tan(pi/2 - x)) + to assemble result using change-of-sign and conditional selection of + numerator/denominator, dependent on odd/even-ness of q (hence quadrant). */ + float64x2_t n = vfmaq_f64 (v_f64 (-1), p, p); + float64x2_t d = vaddq_f64 (p, p); + + uint64x2_t no_recip = vtstq_u64 (vreinterpretq_u64_s64 (qi), v_u64 (1)); + +#if !WANT_SIMD_EXCEPT + uint64x2_t special = vceqzq_u64 (vcaleq_f64 (x, dat->range_val)); + if (__glibc_unlikely (v_any_u64 (special))) + return special_case (x); +#endif + + return vdivq_f64 (vbslq_f64 (no_recip, n, vnegq_f64 (d)), + vbslq_f64 (no_recip, d, n)); +} diff --git a/sysdeps/aarch64/fpu/tan_sve.c b/sysdeps/aarch64/fpu/tan_sve.c new file mode 100644 index 0000000000..e19d8890e5 --- /dev/null +++ b/sysdeps/aarch64/fpu/tan_sve.c @@ -0,0 +1,106 @@ +/* Double-precision vector (SVE) tan function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f64.h" + +static const struct data +{ + double poly[9]; + double half_pi_hi, half_pi_lo, inv_half_pi, range_val, shift; +} data = { + /* Polynomial generated with FPMinimax. */ + .poly = { 0x1.5555555555556p-2, 0x1.1111111110a63p-3, 0x1.ba1ba1bb46414p-5, + 0x1.664f47e5b5445p-6, 0x1.226e5e5ecdfa3p-7, 0x1.d6c7ddbf87047p-9, + 0x1.7ea75d05b583ep-10, 0x1.289f22964a03cp-11, + 0x1.4e4fd14147622p-12, }, + .half_pi_hi = 0x1.921fb54442d18p0, + .half_pi_lo = 0x1.1a62633145c07p-54, + .inv_half_pi = 0x1.45f306dc9c883p-1, + .range_val = 0x1p23, + .shift = 0x1.8p52, +}; + +static svfloat64_t NOINLINE +special_case (svfloat64_t x) +{ + return sv_call_f64 (tan, x, x, svptrue_b64 ()); +} + +/* Vector approximation for double-precision tan. + Maximum measured error is 3.48 ULP: + _ZGVsMxv_tan(0x1.4457047ef78d8p+20) got -0x1.f6ccd8ecf7dedp+37 + want -0x1.f6ccd8ecf7deap+37. */ +svfloat64_t SV_NAME_D1 (tan) (svfloat64_t x, svbool_t pg) +{ + const struct data *dat = ptr_barrier (&data); + + /* Invert condition to catch NaNs and Infs as well as large values. */ + svbool_t special = svnot_z (pg, svaclt (pg, x, dat->range_val)); + + /* Fallback for all lanes if any are out of bounds. */ + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x); + + /* q = nearest integer to 2 * x / pi. */ + svfloat64_t shift = sv_f64 (dat->shift); + svfloat64_t q = svmla_x (pg, shift, x, dat->inv_half_pi); + q = svsub_x (pg, q, shift); + svint64_t qi = svcvt_s64_x (pg, q); + + /* Use q to reduce x to r in [-pi/4, pi/4], by: + r = x - q * pi/2, in extended precision. */ + svfloat64_t r = x; + svfloat64_t half_pi = svld1rq (svptrue_b64 (), &dat->half_pi_hi); + r = svmls_lane (r, q, half_pi, 0); + r = svmls_lane (r, q, half_pi, 1); + /* Further reduce r to [-pi/8, pi/8], to be reconstructed using double angle + formula. */ + r = svmul_x (pg, r, 0.5); + + /* Approximate tan(r) using order 8 polynomial. + tan(x) is odd, so polynomial has the form: + tan(x) ~= x + C0 * x^3 + C1 * x^5 + C3 * x^7 + ... + Hence we first approximate P(r) = C1 + C2 * r^2 + C3 * r^4 + ... + Then compute the approximation by: + tan(r) ~= r + r^3 * (C0 + r^2 * P(r)). */ + svfloat64_t r2 = svmul_x (pg, r, r); + svfloat64_t r4 = svmul_x (pg, r2, r2); + svfloat64_t r8 = svmul_x (pg, r4, r4); + /* Use offset version coeff array by 1 to evaluate from C1 onwards. */ + svfloat64_t p = sv_estrin_7_f64_x (pg, r2, r4, r8, dat->poly + 1); + p = svmad_x (pg, p, r2, dat->poly[0]); + p = svmla_x (pg, r, r2, svmul_x (pg, p, r)); + + /* Recombination uses double-angle formula: + tan(2x) = 2 * tan(x) / (1 - (tan(x))^2) + and reciprocity around pi/2: + tan(x) = 1 / (tan(pi/2 - x)) + to assemble result using change-of-sign and conditional selection of + numerator/denominator dependent on odd/even-ness of q (hence quadrant). */ + svbool_t use_recip + = svcmpeq (pg, svand_x (pg, svreinterpret_u64 (qi), 1), 0); + + svfloat64_t n = svmad_x (pg, p, p, -1); + svfloat64_t d = svmul_x (pg, p, 2); + svfloat64_t swap = n; + n = svneg_m (n, use_recip, d); + d = svsel (use_recip, swap, d); + return svdiv_x (pg, n, d); +} diff --git a/sysdeps/aarch64/fpu/tanf_advsimd.c b/sysdeps/aarch64/fpu/tanf_advsimd.c new file mode 100644 index 0000000000..b9a055bba0 --- /dev/null +++ b/sysdeps/aarch64/fpu/tanf_advsimd.c @@ -0,0 +1,134 @@ +/* Single-precision vector (Advanced SIMD) tan function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f32.h" + +static const struct data +{ + float32x4_t poly[6]; + float32x4_t neg_half_pi_1, neg_half_pi_2, neg_half_pi_3, two_over_pi, shift; +#if !WANT_SIMD_EXCEPT + float32x4_t range_val; +#endif +} data = { + /* Coefficients generated using FPMinimax. */ + .poly = { V4 (0x1.55555p-2f), V4 (0x1.11166p-3f), V4 (0x1.b88a78p-5f), + V4 (0x1.7b5756p-6f), V4 (0x1.4ef4cep-8f), V4 (0x1.0e1e74p-7f) }, + .neg_half_pi_1 = V4 (-0x1.921fb6p+0f), + .neg_half_pi_2 = V4 (0x1.777a5cp-25f), + .neg_half_pi_3 = V4 (0x1.ee59dap-50f), + .two_over_pi = V4 (0x1.45f306p-1f), + .shift = V4 (0x1.8p+23f), +#if !WANT_SIMD_EXCEPT + .range_val = V4 (0x1p15f), +#endif +}; + +#define RangeVal v_u32 (0x47000000) /* asuint32(0x1p15f). */ +#define TinyBound v_u32 (0x30000000) /* asuint32 (0x1p-31f). */ +#define Thresh v_u32 (0x16000000) /* asuint32(RangeVal) - TinyBound. */ + +/* Special cases (fall back to scalar calls). */ +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t x, float32x4_t y, uint32x4_t cmp) +{ + return v_call_f32 (tanf, x, y, cmp); +} + +/* Use a full Estrin scheme to evaluate polynomial. */ +static inline float32x4_t +eval_poly (float32x4_t z, const struct data *d) +{ + float32x4_t z2 = vmulq_f32 (z, z); +#if WANT_SIMD_EXCEPT + /* Tiny z (<= 0x1p-31) will underflow when calculating z^4. If fp exceptions + are to be triggered correctly, sidestep this by fixing such lanes to 0. */ + uint32x4_t will_uflow + = vcleq_u32 (vreinterpretq_u32_f32 (vabsq_f32 (z)), TinyBound); + if (__glibc_unlikely (v_any_u32 (will_uflow))) + z2 = vbslq_f32 (will_uflow, v_f32 (0), z2); +#endif + float32x4_t z4 = vmulq_f32 (z2, z2); + return v_estrin_5_f32 (z, z2, z4, d->poly); +} + +/* Fast implementation of AdvSIMD tanf. + Maximum error is 3.45 ULP: + __v_tanf(-0x1.e5f0cap+13) got 0x1.ff9856p-1 + want 0x1.ff9850p-1. */ +float32x4_t VPCS_ATTR V_NAME_F1 (tan) (float32x4_t x) +{ + const struct data *d = ptr_barrier (&data); + float32x4_t special_arg = x; + + /* iax >= RangeVal means x, if not inf or NaN, is too large to perform fast + regression. */ +#if WANT_SIMD_EXCEPT + uint32x4_t iax = vreinterpretq_u32_f32 (vabsq_f32 (x)); + /* If fp exceptions are to be triggered correctly, also special-case tiny + input, as this will load to overflow later. Fix any special lanes to 1 to + prevent any exceptions being triggered. */ + uint32x4_t special = vcgeq_u32 (vsubq_u32 (iax, TinyBound), Thresh); + if (__glibc_unlikely (v_any_u32 (special))) + x = vbslq_f32 (special, v_f32 (1.0f), x); +#else + /* Otherwise, special-case large and special values. */ + uint32x4_t special = vcageq_f32 (x, d->range_val); +#endif + + /* n = rint(x/(pi/2)). */ + float32x4_t q = vfmaq_f32 (d->shift, d->two_over_pi, x); + float32x4_t n = vsubq_f32 (q, d->shift); + /* Determine if x lives in an interval, where |tan(x)| grows to infinity. */ + uint32x4_t pred_alt = vtstq_u32 (vreinterpretq_u32_f32 (q), v_u32 (1)); + + /* r = x - n * (pi/2) (range reduction into -pi./4 .. pi/4). */ + float32x4_t r; + r = vfmaq_f32 (x, d->neg_half_pi_1, n); + r = vfmaq_f32 (r, d->neg_half_pi_2, n); + r = vfmaq_f32 (r, d->neg_half_pi_3, n); + + /* If x lives in an interval, where |tan(x)| + - is finite, then use a polynomial approximation of the form + tan(r) ~ r + r^3 * P(r^2) = r + r * r^2 * P(r^2). + - grows to infinity then use symmetries of tangent and the identity + tan(r) = cotan(pi/2 - r) to express tan(x) as 1/tan(-r). Finally, use + the same polynomial approximation of tan as above. */ + + /* Invert sign of r if odd quadrant. */ + float32x4_t z = vmulq_f32 (r, vbslq_f32 (pred_alt, v_f32 (-1), v_f32 (1))); + + /* Evaluate polynomial approximation of tangent on [-pi/4, pi/4]. */ + float32x4_t z2 = vmulq_f32 (r, r); + float32x4_t p = eval_poly (z2, d); + float32x4_t y = vfmaq_f32 (z, vmulq_f32 (z, z2), p); + + /* Compute reciprocal and apply if required. */ + float32x4_t inv_y = vdivq_f32 (v_f32 (1.0f), y); + y = vbslq_f32 (pred_alt, inv_y, y); + + /* Fast reduction does not handle the x = -0.0 case well, + therefore it is fixed here. */ + y = vbslq_f32 (vceqzq_f32 (x), x, y); + + if (__glibc_unlikely (v_any_u32 (special))) + return special_case (special_arg, y, special); + return y; +} diff --git a/sysdeps/aarch64/fpu/tanf_sve.c b/sysdeps/aarch64/fpu/tanf_sve.c new file mode 100644 index 0000000000..48343ae197 --- /dev/null +++ b/sysdeps/aarch64/fpu/tanf_sve.c @@ -0,0 +1,123 @@ +/* Single-precision vector (SVE) tan function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" + +static const struct data +{ + float pio2_1, pio2_2, pio2_3, invpio2; + float c1, c3, c5; + float c0, c2, c4, range_val, shift; +} data = { + /* Coefficients generated using: + poly = fpminimax((tan(sqrt(x))-sqrt(x))/x^(3/2), + deg, + [|single ...|], + [a*a;b*b]); + optimize relative error + final prec : 23 bits + deg : 5 + a : 0x1p-126 ^ 2 + b : ((pi) / 0x1p2) ^ 2 + dirty rel error: 0x1.f7c2e4p-25 + dirty abs error: 0x1.f7c2ecp-25. */ + .c0 = 0x1.55555p-2, .c1 = 0x1.11166p-3, + .c2 = 0x1.b88a78p-5, .c3 = 0x1.7b5756p-6, + .c4 = 0x1.4ef4cep-8, .c5 = 0x1.0e1e74p-7, + + .pio2_1 = 0x1.921fb6p+0f, .pio2_2 = -0x1.777a5cp-25f, + .pio2_3 = -0x1.ee59dap-50f, .invpio2 = 0x1.45f306p-1f, + .range_val = 0x1p15f, .shift = 0x1.8p+23f +}; + +static svfloat32_t NOINLINE +special_case (svfloat32_t x, svfloat32_t y, svbool_t cmp) +{ + return sv_call_f32 (tanf, x, y, cmp); +} + +/* Fast implementation of SVE tanf. + Maximum error is 3.45 ULP: + SV_NAME_F1 (tan)(-0x1.e5f0cap+13) got 0x1.ff9856p-1 + want 0x1.ff9850p-1. */ +svfloat32_t SV_NAME_F1 (tan) (svfloat32_t x, const svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + + /* Determine whether input is too large to perform fast regression. */ + svbool_t cmp = svacge (pg, x, d->range_val); + svbool_t peqz = svcmpeq (pg, x, 0.0); + + svfloat32_t odd_coeffs = svld1rq (svptrue_b32 (), &d->c1); + svfloat32_t pi_vals = svld1rq (svptrue_b32 (), &d->pio2_1); + + /* n = rint(x/(pi/2)). */ + svfloat32_t q = svmla_lane (sv_f32 (d->shift), x, pi_vals, 3); + svfloat32_t n = svsub_x (pg, q, d->shift); + /* n is already a signed integer, simply convert it. */ + svint32_t in = svcvt_s32_x (pg, n); + /* Determine if x lives in an interval, where |tan(x)| grows to infinity. */ + svint32_t alt = svand_x (pg, in, 1); + svbool_t pred_alt = svcmpne (pg, alt, 0); + + /* r = x - n * (pi/2) (range reduction into 0 .. pi/4). */ + svfloat32_t r; + r = svmls_lane (x, n, pi_vals, 0); + r = svmls_lane (r, n, pi_vals, 1); + r = svmls_lane (r, n, pi_vals, 2); + + /* If x lives in an interval, where |tan(x)| + - is finite, then use a polynomial approximation of the form + tan(r) ~ r + r^3 * P(r^2) = r + r * r^2 * P(r^2). + - grows to infinity then use symmetries of tangent and the identity + tan(r) = cotan(pi/2 - r) to express tan(x) as 1/tan(-r). Finally, use + the same polynomial approximation of tan as above. */ + + /* Perform additional reduction if required. */ + svfloat32_t z = svneg_m (r, pred_alt, r); + + /* Evaluate polynomial approximation of tangent on [-pi/4, pi/4], + using Estrin on z^2. */ + svfloat32_t z2 = svmul_x (pg, z, z); + svfloat32_t p01 = svmla_lane (sv_f32 (d->c0), z2, odd_coeffs, 0); + svfloat32_t p23 = svmla_lane (sv_f32 (d->c2), z2, odd_coeffs, 1); + svfloat32_t p45 = svmla_lane (sv_f32 (d->c4), z2, odd_coeffs, 2); + + svfloat32_t z4 = svmul_x (pg, z2, z2); + svfloat32_t p = svmla_x (pg, p01, z4, p23); + + svfloat32_t z8 = svmul_x (pg, z4, z4); + p = svmla_x (pg, p, z8, p45); + + svfloat32_t y = svmla_x (pg, z, p, svmul_x (pg, z, z2)); + + /* Transform result back, if necessary. */ + svfloat32_t inv_y = svdivr_x (pg, y, 1.0f); + y = svsel (pred_alt, inv_y, y); + + /* Fast reduction does not handle the x = -0.0 case well, + therefore it is fixed here. */ + y = svsel (peqz, x, y); + + /* No need to pass pg to specialcase here since cmp is a strict subset, + guaranteed by the cmpge above. */ + if (__glibc_unlikely (svptest_any (pg, cmp))) + return special_case (x, y, cmp); + return y; +} diff --git a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c index 3b6b1e343d..f76726e324 100644 --- a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c @@ -27,3 +27,4 @@ VPCS_VECTOR_WRAPPER (cos_advsimd, _ZGVnN2v_cos) VPCS_VECTOR_WRAPPER (exp_advsimd, _ZGVnN2v_exp) VPCS_VECTOR_WRAPPER (log_advsimd, _ZGVnN2v_log) VPCS_VECTOR_WRAPPER (sin_advsimd, _ZGVnN2v_sin) +VPCS_VECTOR_WRAPPER (tan_advsimd, _ZGVnN2v_tan) diff --git a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c index d7ac47ca22..5a9e5b552a 100644 --- a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c @@ -36,3 +36,4 @@ SVE_VECTOR_WRAPPER (cos_sve, _ZGVsMxv_cos) SVE_VECTOR_WRAPPER (exp_sve, _ZGVsMxv_exp) SVE_VECTOR_WRAPPER (log_sve, _ZGVsMxv_log) SVE_VECTOR_WRAPPER (sin_sve, _ZGVsMxv_sin) +SVE_VECTOR_WRAPPER (tan_sve, _ZGVsMxv_tan) diff --git a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c index d4a9ac7154..66c6227087 100644 --- a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c @@ -27,3 +27,4 @@ VPCS_VECTOR_WRAPPER (cosf_advsimd, _ZGVnN4v_cosf) VPCS_VECTOR_WRAPPER (expf_advsimd, _ZGVnN4v_expf) VPCS_VECTOR_WRAPPER (logf_advsimd, _ZGVnN4v_logf) VPCS_VECTOR_WRAPPER (sinf_advsimd, _ZGVnN4v_sinf) +VPCS_VECTOR_WRAPPER (tanf_advsimd, _ZGVnN4v_tanf) diff --git a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c index d44033eab0..3cbf8b48b7 100644 --- a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c @@ -36,3 +36,4 @@ SVE_VECTOR_WRAPPER (cosf_sve, _ZGVsMxv_cosf) SVE_VECTOR_WRAPPER (expf_sve, _ZGVsMxv_expf) SVE_VECTOR_WRAPPER (logf_sve, _ZGVsMxv_logf) SVE_VECTOR_WRAPPER (sinf_sve, _ZGVsMxv_sinf) +SVE_VECTOR_WRAPPER (tanf_sve, _ZGVsMxv_tanf) diff --git a/sysdeps/aarch64/libm-test-ulps b/sysdeps/aarch64/libm-test-ulps index 6b25ed43e0..ec84381c75 100644 --- a/sysdeps/aarch64/libm-test-ulps +++ b/sysdeps/aarch64/libm-test-ulps @@ -1340,11 +1340,19 @@ Function: "tan": float: 1 ldouble: 1 +Function: "tan_advsimd": +double: 2 +float: 2 + Function: "tan_downward": double: 1 float: 2 ldouble: 1 +Function: "tan_sve": +double: 2 +float: 2 + Function: "tan_towardzero": double: 1 float: 1 diff --git a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist index ae46ef8c34..0ec668582e 100644 --- a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist +++ b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist @@ -14,3 +14,7 @@ GLIBC_2.38 _ZGVsMxv_log F GLIBC_2.38 _ZGVsMxv_logf F GLIBC_2.38 _ZGVsMxv_sin F GLIBC_2.38 _ZGVsMxv_sinf F +GLIBC_2.39 _ZGVnN2v_tan F +GLIBC_2.39 _ZGVnN4v_tanf F +GLIBC_2.39 _ZGVsMxv_tan F +GLIBC_2.39 _ZGVsMxv_tanf F From patchwork Tue Aug 8 14:07:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Ramsay X-Patchwork-Id: 1818622 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=8.43.85.97; helo=server2.sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=I34qMW/B; dkim-atps=neutral Received: from server2.sourceware.org (ip-8-43-85-97.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RKw4h4j4Vz1yYC for ; Wed, 9 Aug 2023 00:07:48 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 94A2A385702B for ; Tue, 8 Aug 2023 14:07:46 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 94A2A385702B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1691503666; bh=uRRd6AHSs++BrIpPZE3tbC88VUkG7Ni7rqVGn5Y8ot4=; h=To:CC:Subject:Date:In-Reply-To:References:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=I34qMW/BwmTsvG6D82f5LmQB8LDNqTfscv4F+77yl0pvY3WPdd0CvN1EHT/Ef9BJb WdYXqyhztRhPfYaNArai8VA0dSFHSpPk8jPEHq4xoXabWVpAoeAo4KhFklnx5EDX+I ACGG0MwAkSetMVN8/m5Qukoxme+u4y63tRTmXDFE= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR03-AM7-obe.outbound.protection.outlook.com (mail-am7eur03on2057.outbound.protection.outlook.com [40.107.105.57]) by sourceware.org (Postfix) with ESMTPS id 2C5153857341 for ; Tue, 8 Aug 2023 14:07:27 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 2C5153857341 Received: from DB9PR02CA0017.eurprd02.prod.outlook.com (2603:10a6:10:1d9::22) by VI1PR08MB10276.eurprd08.prod.outlook.com (2603:10a6:800:1bd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.47; Tue, 8 Aug 2023 14:07:22 +0000 Received: from DBAEUR03FT039.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:1d9:cafe::5) by DB9PR02CA0017.outlook.office365.com (2603:10a6:10:1d9::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.27 via Frontend Transport; Tue, 8 Aug 2023 14:07:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT039.mail.protection.outlook.com (100.127.142.225) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.45 via Frontend Transport; Tue, 8 Aug 2023 14:07:22 +0000 Received: ("Tessian outbound 997ae1cc9f47:v145"); Tue, 08 Aug 2023 14:07:22 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 0a678fc69dade2cf X-CR-MTA-TID: 64aa7808 Received: from 2997d9e8eb41.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id E1AEF5E5-FFC6-4B68-ACBD-309E05F9227D.1; Tue, 08 Aug 2023 14:07:15 +0000 Received: from EUR05-DB8-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 2997d9e8eb41.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 08 Aug 2023 14:07:15 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=g/XuScSyk0F6Y+chfrIeIHXqvWNXa/2ihU3W7YBmIqlYDkzHDGvLEcXZDBDxpyvCDaUEXD2sTVkr9cHuihBXCMeJ1IOa8UmEKnIcQdUv8qImWmg0UigvPQNLjRFqpuDs7Ig1NsB3ybR50akibZVahtslyVt256lCue45IkowxpqjjAX/B/bVouflHjY8AuAsd8V6lokf9BA6Qek/TWH75CZSMcyNhEMJkOOJcXNg7k8OWriETi0uxYw8CD0G/6xv0oAMZdlnfLk6qazV5whi4cZzqrvGtp8g/9OhTeo5vPbuCCrq0BeqNzCeoOTF26iBfKvEpAKb3259VEuLK497BQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uRRd6AHSs++BrIpPZE3tbC88VUkG7Ni7rqVGn5Y8ot4=; b=a7L/vAl3ZIfpDAWBDgQET7HnOYEVebtz3uo1TT2kDNDjaaYTWiji6wCwvZhG4yGsSPtKCN53W9whkpUtJrbvaedNWH3Sjd339+l7dKO73IY4kbxQZSatuxs8i1LH1IhUAAskxsITtlC2AcluuE3Q2jCv1ZMFjRptKN19yAt/IKfdXNJseZscZuI9cMY7LmuxcF1XYXPxpwzrM30T9mTmDVs56UzYP5daxy3znHoC/0GIF1n4toDfkhO/gLI+mGHqigP3Ars1ifRMfmBYLgtgRbDiwNQRtj4QEcNmK+sXoB5cG5vm8JvTN1sqeG0qcpzLMwLCeYgHhBV3vR/XRhGM8Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none Received: from AS4P189CA0037.EURP189.PROD.OUTLOOK.COM (2603:10a6:20b:5dd::8) by AS8PR08MB8298.eurprd08.prod.outlook.com (2603:10a6:20b:536::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.27; Tue, 8 Aug 2023 14:07:13 +0000 Received: from AM7EUR03FT061.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:5dd:cafe::61) by AS4P189CA0037.outlook.office365.com (2603:10a6:20b:5dd::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.27 via Frontend Transport; Tue, 8 Aug 2023 14:07:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (40.67.248.234) by AM7EUR03FT061.mail.protection.outlook.com (100.127.140.72) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6678.17 via Frontend Transport; Tue, 8 Aug 2023 14:07:13 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX03.Arm.com (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Tue, 8 Aug 2023 14:07:10 +0000 Received: from vcn-man-apps.manchester.arm.com (10.32.108.22) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.27 via Frontend Transport; Tue, 8 Aug 2023 14:07:10 +0000 To: CC: Joe Ramsay Subject: [PATCH 2/5 v2] aarch64: Add vector implementations of exp2 routines Date: Tue, 8 Aug 2023 15:07:06 +0100 Message-ID: <20230808140709.47733-2-Joe.Ramsay@arm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230808140709.47733-1-Joe.Ramsay@arm.com> References: <20230808140709.47733-1-Joe.Ramsay@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: AM7EUR03FT061:EE_|AS8PR08MB8298:EE_|DBAEUR03FT039:EE_|VI1PR08MB10276:EE_ X-MS-Office365-Filtering-Correlation-Id: d83c1871-5a78-4c8f-b17f-08db9818c93c x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: gtyPhxJjpvrE3WKT0sPp2ounxnGTGiIvqejQi3gkK+XqxJC55jhhuhaewn7RHLTsDU9dw5ETlr0cCjhn4j+EnvHd4WahQkh0vMD0Z/K6o2YPU4my025EyNrfJMnL0UTiynKjLaNMQvA+wnohx5LSuzdMednQdnbo4ds4NMGw85qjXDn4UtRCEGKDBnPyn/yP6yOrjsOmu1N3jZ8uphGOyibJlX942kjvL/uwmBZ9xGCgbibQ/WGE8NW/PIZr3cmXpZIjAGRz7tb/jKkWSNKbKK7/BWjmZcoiZTfCMrBmceYhaJfGyyq4c0UkvRD0Saz7Jelr8rbC56iDB4t7oz4uxtLIXBsNafrn1a/UC77FPk3wZIeCGF7ewE6TI414wrGNQ7JSpT+4e6tyY8uc9xhFaHPKrk9be9sO+lP3WEkOsnJ4Y99vYXa37IDl50qHaBmGegk7ziUWXLiLOUwG5sN9P6RyGGsqwfaKWuNdd7kaI1NV5aw5MqrC0ZUO3E7Y2n9DusfQxt02a4/e6luvRF3C8m6ZsauaHxAFMln+SQxlHHAgzSAQnNU+cp5sYYDS1+HAg+HaFIOU7D3w9Ub0Sf3Zvj9JrsrRrql1EUPp2pIbeE/yEYlh+3G9U3Rro9Sln3nBj0hn5H+eDCXNqWjxM6pvXuPECHotXW0Irl7PiKeEta2nLZBz8OXAl42lnyZWzIaxz0ndhYZPSE6x8glK3e22Nqz5u6RzetUZSKLl+HDuK0D4/q3xhCnweYFD94XWzNAhMHcn4l6oxIx2/jzRnjD/1w== X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(396003)(39860400002)(376002)(346002)(136003)(451199021)(82310400008)(186006)(1800799003)(40470700004)(36840700001)(46966006)(426003)(8676002)(8936002)(5660300002)(6916009)(4326008)(41300700001)(316002)(47076005)(86362001)(83380400001)(40480700001)(40460700003)(36860700001)(2906002)(30864003)(6666004)(7696005)(2616005)(1076003)(26005)(36756003)(336012)(70206006)(70586007)(356005)(478600001)(81166007)(82740400003)(2004002)(36900700001); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB8298 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT039.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 160bf13e-c977-47e7-8e95-08db9818c3d3 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: gb2z6IK/fxt3BKEaBV39TqWsXV7jeBuQ69uEsNJA5XVvdKtprFslhnChT5Hdkq3/j66DTksyhepZ7PmUcjKYTIs3QnKAKVVSX3Y5+Y1fda6dJLJCqkW0imQeWeCYc1pnqVboxdU2j4eEON3/oAH8ZTuggGmAaM/l+Dy9gqMdNjIJmBiAWDf5SUFRkjIltNZ+RMkliJ5vjeIjK1ZC6RIhCL7k4Mf4UcHJQg6+na9jrellLgtUCqelHDh6VyczId4gq3XHnSoEkzQmSOjKlvba5VmVkzqwm+HRdMVYM6QJpaCkYN91Kh3ajXJT9090HbDFa3WWlJsAVakGSCGxMkt1ewoDNX74hUilKJPf1xQai2jLTlDB8ObVt2UcEeBKobgyTvkVs+F9a15lr2BDexvokDviIBLO4FgRd50EYJm1KeCdExY+zcoVpn1iC0LAzTEzcbXFzFU87IdGccwTIcjF51wJFStM/yQMAn7EnH8jbsaQGIG1TuaBCXqLXHhfTxqxbncrfdruP3YTaZNIyxD5I+yemqCvif9O7jZqz5eeaQnnVbx3Q+UuDwvcylttblP1fddQ6oLMc7c8qnrhzGz3HsYi8hJkNZgzZcCfvFJ4luOTEFBU0WHHVj3x0XJQ7k5q9qOpL6HrHeho8MyPces3iW8aGSOjoV1OBd0T4zJF2r6KmSeN+g/67bvLt9mrER/domICq5iYX00JRydp/47Ql7CbGIedyGEQmfgOBvMJ2sEUF9tdStIn1AdCQ7snHjyL X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230028)(4636009)(396003)(136003)(346002)(39860400002)(376002)(82310400008)(451199021)(186006)(1800799003)(36840700001)(40470700004)(46966006)(426003)(83380400001)(36860700001)(40480700001)(316002)(70586007)(41300700001)(70206006)(1076003)(8936002)(26005)(8676002)(6666004)(7696005)(6916009)(5660300002)(478600001)(47076005)(86362001)(2616005)(40460700003)(82740400003)(81166007)(336012)(36756003)(2906002)(30864003)(4326008)(2004002); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Aug 2023 14:07:22.5735 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d83c1871-5a78-4c8f-b17f-08db9818c93c X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT039.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB10276 X-Spam-Status: No, score=-12.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Joe Ramsay via Libc-alpha From: Joe Ramsay Reply-To: Joe Ramsay Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" Some routines reuse table from v_exp_data.c --- Changes to v1: Fix check-abi Thanks, Joe sysdeps/aarch64/fpu/Makefile | 1 + sysdeps/aarch64/fpu/Versions | 4 + sysdeps/aarch64/fpu/bits/math-vector.h | 4 + sysdeps/aarch64/fpu/exp2_advsimd.c | 128 ++++++++++++++++++ sysdeps/aarch64/fpu/exp2_sve.c | 111 +++++++++++++++ sysdeps/aarch64/fpu/exp2f_advsimd.c | 124 +++++++++++++++++ sysdeps/aarch64/fpu/exp2f_sve.c | 75 ++++++++++ .../fpu/test-double-advsimd-wrappers.c | 1 + .../aarch64/fpu/test-double-sve-wrappers.c | 1 + .../aarch64/fpu/test-float-advsimd-wrappers.c | 1 + sysdeps/aarch64/fpu/test-float-sve-wrappers.c | 1 + sysdeps/aarch64/libm-test-ulps | 8 ++ .../unix/sysv/linux/aarch64/libmvec.abilist | 4 + 13 files changed, 463 insertions(+) create mode 100644 sysdeps/aarch64/fpu/exp2_advsimd.c create mode 100644 sysdeps/aarch64/fpu/exp2_sve.c create mode 100644 sysdeps/aarch64/fpu/exp2f_advsimd.c create mode 100644 sysdeps/aarch64/fpu/exp2f_sve.c diff --git a/sysdeps/aarch64/fpu/Makefile b/sysdeps/aarch64/fpu/Makefile index a1bbc9bcaa..9c7c768301 100644 --- a/sysdeps/aarch64/fpu/Makefile +++ b/sysdeps/aarch64/fpu/Makefile @@ -1,5 +1,6 @@ libmvec-supported-funcs = cos \ exp \ + exp2 \ log \ sin \ tan diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions index f0ca0940a9..05de4325d5 100644 --- a/sysdeps/aarch64/fpu/Versions +++ b/sysdeps/aarch64/fpu/Versions @@ -18,6 +18,10 @@ libmvec { _ZGVsMxv_sinf; } GLIBC_2.39 { + _ZGVnN4v_exp2f; + _ZGVnN2v_exp2; + _ZGVsMxv_exp2f; + _ZGVsMxv_exp2; _ZGVnN4v_tanf; _ZGVnN2v_tan; _ZGVsMxv_tanf; diff --git a/sysdeps/aarch64/fpu/bits/math-vector.h b/sysdeps/aarch64/fpu/bits/math-vector.h index 6193213147..50921b22e5 100644 --- a/sysdeps/aarch64/fpu/bits/math-vector.h +++ b/sysdeps/aarch64/fpu/bits/math-vector.h @@ -51,12 +51,14 @@ typedef __SVBool_t __sv_bool_t; __vpcs __f32x4_t _ZGVnN4v_cosf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_expf (__f32x4_t); +__vpcs __f32x4_t _ZGVnN4v_exp2f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_logf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_sinf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_tanf (__f32x4_t); __vpcs __f64x2_t _ZGVnN2v_cos (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp (__f64x2_t); +__vpcs __f64x2_t _ZGVnN2v_exp2 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_sin (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_tan (__f64x2_t); @@ -68,12 +70,14 @@ __vpcs __f64x2_t _ZGVnN2v_tan (__f64x2_t); __sv_f32_t _ZGVsMxv_cosf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_expf (__sv_f32_t, __sv_bool_t); +__sv_f32_t _ZGVsMxv_exp2f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_logf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_sinf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_tanf (__sv_f32_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_cos (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp (__sv_f64_t, __sv_bool_t); +__sv_f64_t _ZGVsMxv_exp2 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_sin (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_tan (__sv_f64_t, __sv_bool_t); diff --git a/sysdeps/aarch64/fpu/exp2_advsimd.c b/sysdeps/aarch64/fpu/exp2_advsimd.c new file mode 100644 index 0000000000..8e26033cb9 --- /dev/null +++ b/sysdeps/aarch64/fpu/exp2_advsimd.c @@ -0,0 +1,128 @@ +/* Double-precision vector (AdvSIMD) exp2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f64.h" + +#define N (1 << V_EXP_TABLE_BITS) +#define IndexMask (N - 1) +#define BigBound 1022.0 +#define UOFlowBound 1280.0 + +static const struct data +{ + float64x2_t poly[4]; + float64x2_t shift, scale_big_bound, scale_uoflow_bound; +} data = { + /* Coefficients are computed using Remez algorithm with + minimisation of the absolute error. */ + .poly = { V2 (0x1.62e42fefa3686p-1), V2 (0x1.ebfbdff82c241p-3), + V2 (0x1.c6b09b16de99ap-5), V2 (0x1.3b2abf5571ad8p-7) }, + .shift = V2 (0x1.8p52 / N), + .scale_big_bound = V2 (BigBound), + .scale_uoflow_bound = V2 (UOFlowBound), +}; + +static inline uint64x2_t +lookup_sbits (uint64x2_t i) +{ + return (uint64x2_t){ __v_exp_data[i[0] & IndexMask], + __v_exp_data[i[1] & IndexMask] }; +} + +#if WANT_SIMD_EXCEPT + +# define TinyBound 0x2000000000000000 /* asuint64(0x1p-511). */ +# define Thres 0x2080000000000000 /* asuint64(512.0) - TinyBound. */ + +/* Call scalar exp2 as a fallback. */ +static float64x2_t VPCS_ATTR NOINLINE +special_case (float64x2_t x) +{ + return v_call_f64 (exp2, x, x, v_u64 (0xffffffffffffffff)); +} + +#else + +# define SpecialOffset 0x6000000000000000 /* 0x1p513. */ +/* SpecialBias1 + SpecialBias1 = asuint(1.0). */ +# define SpecialBias1 0x7000000000000000 /* 0x1p769. */ +# define SpecialBias2 0x3010000000000000 /* 0x1p-254. */ + +static float64x2_t VPCS_ATTR +special_case (float64x2_t s, float64x2_t y, float64x2_t n, + const struct data *d) +{ + /* 2^(n/N) may overflow, break it up into s1*s2. */ + uint64x2_t b = vandq_u64 (vclezq_f64 (n), v_u64 (SpecialOffset)); + float64x2_t s1 = vreinterpretq_f64_u64 (vsubq_u64 (v_u64 (SpecialBias1), b)); + float64x2_t s2 = vreinterpretq_f64_u64 ( + vaddq_u64 (vsubq_u64 (vreinterpretq_u64_f64 (s), v_u64 (SpecialBias2)), b)); + uint64x2_t cmp = vcagtq_f64 (n, d->scale_uoflow_bound); + float64x2_t r1 = vmulq_f64 (s1, s1); + float64x2_t r0 = vmulq_f64 (vfmaq_f64 (s2, s2, y), s1); + return vbslq_f64 (cmp, r1, r0); +} + +#endif + +/* Fast vector implementation of exp2. + Maximum measured error is 1.65 ulp. + _ZGVnN2v_exp2(-0x1.4c264ab5b559bp-6) got 0x1.f8db0d4df721fp-1 + want 0x1.f8db0d4df721dp-1. */ +VPCS_ATTR +float64x2_t V_NAME_D1 (exp2) (float64x2_t x) +{ + const struct data *d = ptr_barrier (&data); + uint64x2_t cmp; +#if WANT_SIMD_EXCEPT + uint64x2_t ia = vreinterpretq_u64_f64 (vabsq_f64 (x)); + cmp = vcgeq_u64 (vsubq_u64 (ia, v_u64 (TinyBound)), v_u64 (Thres)); + /* If any special case (inf, nan, small and large x) is detected, + fall back to scalar for all lanes. */ + if (__glibc_unlikely (v_any_u64 (cmp))) + return special_case (x); +#else + cmp = vcagtq_f64 (x, d->scale_big_bound); +#endif + + /* n = round(x/N). */ + float64x2_t z = vaddq_f64 (d->shift, x); + uint64x2_t u = vreinterpretq_u64_f64 (z); + float64x2_t n = vsubq_f64 (z, d->shift); + + /* r = x - n/N. */ + float64x2_t r = vsubq_f64 (x, n); + + /* s = 2^(n/N). */ + uint64x2_t e = vshlq_n_u64 (u, 52 - V_EXP_TABLE_BITS); + u = lookup_sbits (u); + float64x2_t s = vreinterpretq_f64_u64 (vaddq_u64 (u, e)); + + /* y ~ exp2(r) - 1. */ + float64x2_t r2 = vmulq_f64 (r, r); + float64x2_t y = v_pairwise_poly_3_f64 (r, r2, d->poly); + y = vmulq_f64 (r, y); + +#if !WANT_SIMD_EXCEPT + if (__glibc_unlikely (v_any_u64 (cmp))) + return special_case (s, y, n, d); +#endif + return vfmaq_f64 (s, s, y); +} diff --git a/sysdeps/aarch64/fpu/exp2_sve.c b/sysdeps/aarch64/fpu/exp2_sve.c new file mode 100644 index 0000000000..c9314a9fee --- /dev/null +++ b/sysdeps/aarch64/fpu/exp2_sve.c @@ -0,0 +1,111 @@ +/* Double-precision vector (SVE) exp2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f64.h" + +#define N (1 << V_EXP_TABLE_BITS) + +#define BigBound 1022 +#define UOFlowBound 1280 + +static const struct data +{ + double poly[4]; + double shift, big_bound, uoflow_bound; +} data = { + /* Coefficients are computed using Remez algorithm with + minimisation of the absolute error. */ + .poly = { 0x1.62e42fefa3686p-1, 0x1.ebfbdff82c241p-3, 0x1.c6b09b16de99ap-5, + 0x1.3b2abf5571ad8p-7 }, + .shift = 0x1.8p52 / N, + .uoflow_bound = UOFlowBound, + .big_bound = BigBound, +}; + +#define SpecialOffset 0x6000000000000000 /* 0x1p513. */ +/* SpecialBias1 + SpecialBias1 = asuint(1.0). */ +#define SpecialBias1 0x7000000000000000 /* 0x1p769. */ +#define SpecialBias2 0x3010000000000000 /* 0x1p-254. */ + +/* Update of both special and non-special cases, if any special case is + detected. */ +static inline svfloat64_t +special_case (svbool_t pg, svfloat64_t s, svfloat64_t y, svfloat64_t n, + const struct data *d) +{ + /* s=2^n may overflow, break it up into s=s1*s2, + such that exp = s + s*y can be computed as s1*(s2+s2*y) + and s1*s1 overflows only if n>0. */ + + /* If n<=0 then set b to 0x6, 0 otherwise. */ + svbool_t p_sign = svcmple (pg, n, 0.0); /* n <= 0. */ + svuint64_t b = svdup_u64_z (p_sign, SpecialOffset); + + /* Set s1 to generate overflow depending on sign of exponent n. */ + svfloat64_t s1 = svreinterpret_f64 (svsubr_x (pg, b, SpecialBias1)); + /* Offset s to avoid overflow in final result if n is below threshold. */ + svfloat64_t s2 = svreinterpret_f64 ( + svadd_x (pg, svsub_x (pg, svreinterpret_u64 (s), SpecialBias2), b)); + + /* |n| > 1280 => 2^(n) overflows. */ + svbool_t p_cmp = svacgt (pg, n, d->uoflow_bound); + + svfloat64_t r1 = svmul_x (pg, s1, s1); + svfloat64_t r2 = svmla_x (pg, s2, s2, y); + svfloat64_t r0 = svmul_x (pg, r2, s1); + + return svsel (p_cmp, r1, r0); +} + +/* Fast vector implementation of exp2. + Maximum measured error is 1.65 ulp. + _ZGVsMxv_exp2(-0x1.4c264ab5b559bp-6) got 0x1.f8db0d4df721fp-1 + want 0x1.f8db0d4df721dp-1. */ +svfloat64_t SV_NAME_D1 (exp2) (svfloat64_t x, svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + svbool_t no_big_scale = svacle (pg, x, d->big_bound); + svbool_t special = svnot_z (pg, no_big_scale); + + /* Reduce x to k/N + r, where k is integer and r in [-1/2N, 1/2N]. */ + svfloat64_t shift = sv_f64 (d->shift); + svfloat64_t kd = svadd_x (pg, x, shift); + svuint64_t ki = svreinterpret_u64 (kd); + /* kd = k/N. */ + kd = svsub_x (pg, kd, shift); + svfloat64_t r = svsub_x (pg, x, kd); + + /* scale ~= 2^(k/N). */ + svuint64_t idx = svand_x (pg, ki, N - 1); + svuint64_t sbits = svld1_gather_index (pg, __v_exp_data, idx); + /* This is only a valid scale when -1023*N < k < 1024*N. */ + svuint64_t top = svlsl_x (pg, ki, 52 - V_EXP_TABLE_BITS); + svfloat64_t scale = svreinterpret_f64 (svadd_x (pg, sbits, top)); + + /* Approximate exp2(r) using polynomial. */ + svfloat64_t r2 = svmul_x (pg, r, r); + svfloat64_t p = sv_pairwise_poly_3_f64_x (pg, r, r2, d->poly); + svfloat64_t y = svmul_x (pg, r, p); + + /* Assemble exp2(x) = exp2(r) * scale. */ + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (pg, scale, y, kd, d); + return svmla_x (pg, scale, scale, y); +} diff --git a/sysdeps/aarch64/fpu/exp2f_advsimd.c b/sysdeps/aarch64/fpu/exp2f_advsimd.c new file mode 100644 index 0000000000..70b3ab66c1 --- /dev/null +++ b/sysdeps/aarch64/fpu/exp2f_advsimd.c @@ -0,0 +1,124 @@ +/* Single-precision vector (AdvSIMD) exp2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" + +static const struct data +{ + float32x4_t poly[5]; + uint32x4_t exponent_bias; +#if !WANT_SIMD_EXCEPT + float32x4_t special_bound, scale_thresh; +#endif +} data = { + /* maxerr: 1.962 ulp. */ + .poly = { V4 (0x1.59977ap-10f), V4 (0x1.3ce9e4p-7f), V4 (0x1.c6bd32p-5f), + V4 (0x1.ebf9bcp-3f), V4 (0x1.62e422p-1f) }, + .exponent_bias = V4 (0x3f800000), +#if !WANT_SIMD_EXCEPT + .special_bound = V4 (126.0f), + .scale_thresh = V4 (192.0f), +#endif +}; + +#define C(i) d->poly[i] + +#if WANT_SIMD_EXCEPT + +# define TinyBound v_u32 (0x20000000) /* asuint (0x1p-63). */ +# define BigBound v_u32 (0x42800000) /* asuint (0x1p6). */ +# define SpecialBound v_u32 (0x22800000) /* BigBound - TinyBound. */ + +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t x, float32x4_t y, uint32x4_t cmp) +{ + /* If fenv exceptions are to be triggered correctly, fall back to the scalar + routine for special lanes. */ + return v_call_f32 (exp2f, x, y, cmp); +} + +#else + +# define SpecialOffset v_u32 (0x82000000) +# define SpecialBias v_u32 (0x7f000000) + +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t poly, float32x4_t n, uint32x4_t e, uint32x4_t cmp1, + float32x4_t scale, const struct data *d) +{ + /* 2^n may overflow, break it up into s1*s2. */ + uint32x4_t b = vandq_u32 (vclezq_f32 (n), SpecialOffset); + float32x4_t s1 = vreinterpretq_f32_u32 (vaddq_u32 (b, SpecialBias)); + float32x4_t s2 = vreinterpretq_f32_u32 (vsubq_u32 (e, b)); + uint32x4_t cmp2 = vcagtq_f32 (n, d->scale_thresh); + float32x4_t r2 = vmulq_f32 (s1, s1); + float32x4_t r1 = vmulq_f32 (vfmaq_f32 (s2, poly, s2), s1); + /* Similar to r1 but avoids double rounding in the subnormal range. */ + float32x4_t r0 = vfmaq_f32 (scale, poly, scale); + float32x4_t r = vbslq_f32 (cmp1, r1, r0); + return vbslq_f32 (cmp2, r2, r); +} + +#endif + +float32x4_t VPCS_ATTR V_NAME_F1 (exp2) (float32x4_t x) +{ + const struct data *d = ptr_barrier (&data); + float32x4_t n, r, r2, scale, p, q, poly; + uint32x4_t cmp, e; + +#if WANT_SIMD_EXCEPT + /* asuint(|x|) - TinyBound >= BigBound - TinyBound. */ + uint32x4_t ia = vreinterpretq_u32_f32 (vabsq_f32 (x)); + cmp = vcgeq_u32 (vsubq_u32 (ia, TinyBound), SpecialBound); + float32x4_t xm = x; + /* If any lanes are special, mask them with 1 and retain a copy of x to allow + special_case to fix special lanes later. This is only necessary if fenv + exceptions are to be triggered correctly. */ + if (__glibc_unlikely (v_any_u32 (cmp))) + x = vbslq_f32 (cmp, v_f32 (1), x); +#endif + + /* exp2(x) = 2^n (1 + poly(r)), with 1 + poly(r) in [1/sqrt(2),sqrt(2)] + x = n + r, with r in [-1/2, 1/2]. */ + n = vrndaq_f32 (x); + r = vsubq_f32 (x, n); + e = vshlq_n_u32 (vreinterpretq_u32_s32 (vcvtaq_s32_f32 (x)), 23); + scale = vreinterpretq_f32_u32 (vaddq_u32 (e, d->exponent_bias)); + +#if !WANT_SIMD_EXCEPT + cmp = vcagtq_f32 (n, d->special_bound); +#endif + + r2 = vmulq_f32 (r, r); + p = vfmaq_f32 (C (1), C (0), r); + q = vfmaq_f32 (C (3), C (2), r); + q = vfmaq_f32 (q, p, r2); + p = vmulq_f32 (C (4), r); + poly = vfmaq_f32 (p, q, r2); + + if (__glibc_unlikely (v_any_u32 (cmp))) +#if WANT_SIMD_EXCEPT + return special_case (xm, vfmaq_f32 (scale, poly, scale), cmp); +#else + return special_case (poly, n, e, cmp, scale, d); +#endif + + return vfmaq_f32 (scale, poly, scale); +} diff --git a/sysdeps/aarch64/fpu/exp2f_sve.c b/sysdeps/aarch64/fpu/exp2f_sve.c new file mode 100644 index 0000000000..88d10daefc --- /dev/null +++ b/sysdeps/aarch64/fpu/exp2f_sve.c @@ -0,0 +1,75 @@ +/* Single-precision vector (SVE) exp2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f32.h" + +static const struct data +{ + float poly[5]; + float shift, thres; +} data = { + /* Coefficients copied from the polynomial in AdvSIMD variant, reversed for + compatibility with polynomial helpers. */ + .poly = { 0x1.62e422p-1f, 0x1.ebf9bcp-3f, 0x1.c6bd32p-5f, 0x1.3ce9e4p-7f, + 0x1.59977ap-10f }, + /* 1.5*2^17 + 127. */ + .shift = 0x1.903f8p17f, + /* Roughly 87.3. For x < -Thres, the result is subnormal and not handled + correctly by FEXPA. */ + .thres = 0x1.5d5e2ap+6f, +}; + +static svfloat32_t NOINLINE +special_case (svfloat32_t x, svfloat32_t y, svbool_t special) +{ + return sv_call_f32 (exp2f, x, y, special); +} + +/* Single-precision SVE exp2f routine. Implements the same algorithm + as AdvSIMD exp2f. + Worst case error is 1.04 ULPs. + SV_NAME_F1 (exp2)(0x1.943b9p-1) got 0x1.ba7eb2p+0 + want 0x1.ba7ebp+0. */ +svfloat32_t SV_NAME_F1 (exp2) (svfloat32_t x, const svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + /* exp2(x) = 2^n (1 + poly(r)), with 1 + poly(r) in [1/sqrt(2),sqrt(2)] + x = n + r, with r in [-1/2, 1/2]. */ + svfloat32_t shift = sv_f32 (d->shift); + svfloat32_t z = svadd_x (pg, x, shift); + svfloat32_t n = svsub_x (pg, z, shift); + svfloat32_t r = svsub_x (pg, x, n); + + svbool_t special = svacgt (pg, x, d->thres); + svfloat32_t scale = svexpa (svreinterpret_u32 (z)); + + /* Polynomial evaluation: poly(r) ~ exp2(r)-1. + Evaluate polynomial use hybrid scheme - offset ESTRIN by 1 for + coefficients 1 to 4, and apply most significant coefficient directly. */ + svfloat32_t r2 = svmul_x (pg, r, r); + svfloat32_t p14 = sv_pairwise_poly_3_f32_x (pg, r, r2, d->poly + 1); + svfloat32_t p0 = svmul_x (pg, r, d->poly[0]); + svfloat32_t poly = svmla_x (pg, p0, r2, p14); + + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x, svmla_x (pg, scale, scale, poly), special); + + return svmla_x (pg, scale, scale, poly); +} diff --git a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c index f76726e324..b2b36fd847 100644 --- a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c @@ -25,6 +25,7 @@ VPCS_VECTOR_WRAPPER (cos_advsimd, _ZGVnN2v_cos) VPCS_VECTOR_WRAPPER (exp_advsimd, _ZGVnN2v_exp) +VPCS_VECTOR_WRAPPER (exp2_advsimd, _ZGVnN2v_exp2) VPCS_VECTOR_WRAPPER (log_advsimd, _ZGVnN2v_log) VPCS_VECTOR_WRAPPER (sin_advsimd, _ZGVnN2v_sin) VPCS_VECTOR_WRAPPER (tan_advsimd, _ZGVnN2v_tan) diff --git a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c index 5a9e5b552a..88b76ed678 100644 --- a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c @@ -34,6 +34,7 @@ SVE_VECTOR_WRAPPER (cos_sve, _ZGVsMxv_cos) SVE_VECTOR_WRAPPER (exp_sve, _ZGVsMxv_exp) +SVE_VECTOR_WRAPPER (exp2_sve, _ZGVsMxv_exp2) SVE_VECTOR_WRAPPER (log_sve, _ZGVsMxv_log) SVE_VECTOR_WRAPPER (sin_sve, _ZGVsMxv_sin) SVE_VECTOR_WRAPPER (tan_sve, _ZGVsMxv_tan) diff --git a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c index 66c6227087..02ab609b5a 100644 --- a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c @@ -25,6 +25,7 @@ VPCS_VECTOR_WRAPPER (cosf_advsimd, _ZGVnN4v_cosf) VPCS_VECTOR_WRAPPER (expf_advsimd, _ZGVnN4v_expf) +VPCS_VECTOR_WRAPPER (exp2f_advsimd, _ZGVnN4v_exp2f) VPCS_VECTOR_WRAPPER (logf_advsimd, _ZGVnN4v_logf) VPCS_VECTOR_WRAPPER (sinf_advsimd, _ZGVnN4v_sinf) VPCS_VECTOR_WRAPPER (tanf_advsimd, _ZGVnN4v_tanf) diff --git a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c index 3cbf8b48b7..fa41ce09d8 100644 --- a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c @@ -34,6 +34,7 @@ SVE_VECTOR_WRAPPER (cosf_sve, _ZGVsMxv_cosf) SVE_VECTOR_WRAPPER (expf_sve, _ZGVsMxv_expf) +SVE_VECTOR_WRAPPER (exp2f_sve, _ZGVsMxv_exp2f) SVE_VECTOR_WRAPPER (logf_sve, _ZGVsMxv_logf) SVE_VECTOR_WRAPPER (sinf_sve, _ZGVsMxv_sinf) SVE_VECTOR_WRAPPER (tanf_sve, _ZGVsMxv_tanf) diff --git a/sysdeps/aarch64/libm-test-ulps b/sysdeps/aarch64/libm-test-ulps index ec84381c75..a1e5651c87 100644 --- a/sysdeps/aarch64/libm-test-ulps +++ b/sysdeps/aarch64/libm-test-ulps @@ -990,11 +990,19 @@ double: 1 float: 1 ldouble: 1 +Function: "exp2_advsimd": +double: 1 +float: 1 + Function: "exp2_downward": double: 1 float: 1 ldouble: 1 +Function: "exp2_sve": +double: 1 +float: 1 + Function: "exp2_towardzero": double: 1 float: 1 diff --git a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist index 0ec668582e..6046c3d046 100644 --- a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist +++ b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist @@ -14,7 +14,11 @@ GLIBC_2.38 _ZGVsMxv_log F GLIBC_2.38 _ZGVsMxv_logf F GLIBC_2.38 _ZGVsMxv_sin F GLIBC_2.38 _ZGVsMxv_sinf F +GLIBC_2.39 _ZGVnN2v_exp2 F GLIBC_2.39 _ZGVnN2v_tan F +GLIBC_2.39 _ZGVnN4v_exp2f F GLIBC_2.39 _ZGVnN4v_tanf F +GLIBC_2.39 _ZGVsMxv_exp2 F +GLIBC_2.39 _ZGVsMxv_exp2f F GLIBC_2.39 _ZGVsMxv_tan F GLIBC_2.39 _ZGVsMxv_tanf F From patchwork Tue Aug 8 14:07:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Ramsay X-Patchwork-Id: 1818625 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=8.43.85.97; helo=server2.sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=mFzudz+L; dkim-atps=neutral Received: from server2.sourceware.org (ip-8-43-85-97.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RKw5f0WYSz1yYC for ; Wed, 9 Aug 2023 00:08:38 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 0CA193851171 for ; Tue, 8 Aug 2023 14:08:36 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 0CA193851171 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1691503716; bh=Ai5zclvqZ/RG5WEPn7aD8VwNvwB2XOH0xJNlTrVCBLc=; h=To:CC:Subject:Date:In-Reply-To:References:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=mFzudz+LnQ1ct8whzsDLh35bl5z3xwBoGxGZY5g28l2uCv3iFVJ4ka45Ro3Ldmgkg AHGuXUaDq8EjQsyi2g3/TW/j7yV2k7aAGRf2bK9sAxHz7sm3pgoWsnneEKI0TeWzrQ wIKFZ96fU1+F3LVcsUyfIbtHeMdmWC0yanjC4PYw= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2069.outbound.protection.outlook.com [40.107.21.69]) by sourceware.org (Postfix) with ESMTPS id 0A4813857342 for ; Tue, 8 Aug 2023 14:07:27 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 0A4813857342 Received: from DUZPR01CA0325.eurprd01.prod.exchangelabs.com (2603:10a6:10:4ba::22) by DU0PR08MB9680.eurprd08.prod.outlook.com (2603:10a6:10:444::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.26; Tue, 8 Aug 2023 14:07:23 +0000 Received: from DBAEUR03FT038.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:4ba:cafe::25) by DUZPR01CA0325.outlook.office365.com (2603:10a6:10:4ba::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.27 via Frontend Transport; Tue, 8 Aug 2023 14:07:23 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT038.mail.protection.outlook.com (100.127.143.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.45 via Frontend Transport; Tue, 8 Aug 2023 14:07:23 +0000 Received: ("Tessian outbound f9124736ff4f:v145"); Tue, 08 Aug 2023 14:07:23 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 54725a60067872b5 X-CR-MTA-TID: 64aa7808 Received: from 9dd7a23dd3f9.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 4C17AB44-7D0A-4274-AB3E-AEC9C2A6ECB0.1; Tue, 08 Aug 2023 14:07:16 +0000 Received: from EUR05-DB8-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 9dd7a23dd3f9.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 08 Aug 2023 14:07:16 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Vg6iZo+rthA9M4E9Ecxe8a1XD/2t3ahIK8KBpGn3uotyRJxYKSao/VILzs/9+7dRZqX+Du5x6qZQ8Cl4esqKfmxGMIWTqrquimW6fv/SijSG7ClbDVngwmc67epkVykFA0VEFV7eDh3T2+VvB+Gy5nOWdlDObMD3zPVGQl1myjEe7vTdkZhKwx8iwGXIGR6pkfEswthaKSNy52EKG+j7ZKHf0nXBBPMzxbA+L/jLLId3O00p5+hFGQmg+kSlXeHeTTJfNKKg6EyE48Kp2lDVq7h4njPEWtW73Y75p4JWfZlWxjIcUL2fWIrMDWUH125SUeQIBZfOo67X3ffAiSForQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ai5zclvqZ/RG5WEPn7aD8VwNvwB2XOH0xJNlTrVCBLc=; b=Dhy2Us6dVayEzfITSRq5OwYQw69pmLJ67ZQdmBGSN9ADziJ3ntxY9MR29xGR/DSkKe759255rkonOK2a9no6V4RE/y9xV88baIT9u75ymnc+xT13Wz0p3ZwwS4R13jRkgAGwNcZ8DflVaF9DQkYlLcMbB1Pj/He8m5/e47dBaofHjveZ3512pcBVrRk5QecgcB3OpuNzpZDwPI2qQJo55iwEGVEi38SEQ7BuCVXbwGmthioVpcEB+DFp1wAp1B8EVoQvrpj0v4depkCmYLfTGeSXYXi7YsWDflDtlm5ggYL435jpzNo/Ygd0G67p2Yfd7Mu6JnUNQJD7vfJxCNvmuA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none Received: from AS4P189CA0035.EURP189.PROD.OUTLOOK.COM (2603:10a6:20b:5dd::14) by AS8PR08MB9885.eurprd08.prod.outlook.com (2603:10a6:20b:5b0::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.27; Tue, 8 Aug 2023 14:07:14 +0000 Received: from AM7EUR03FT061.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:5dd:cafe::5c) by AS4P189CA0035.outlook.office365.com (2603:10a6:20b:5dd::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.25 via Frontend Transport; Tue, 8 Aug 2023 14:07:14 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (40.67.248.234) by AM7EUR03FT061.mail.protection.outlook.com (100.127.140.72) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6678.17 via Frontend Transport; Tue, 8 Aug 2023 14:07:14 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX03.Arm.com (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Tue, 8 Aug 2023 14:07:11 +0000 Received: from vcn-man-apps.manchester.arm.com (10.32.108.22) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.27 via Frontend Transport; Tue, 8 Aug 2023 14:07:11 +0000 To: CC: Joe Ramsay Subject: [PATCH 3/5 v2] aarch64: Add vector implementations of log2 routines Date: Tue, 8 Aug 2023 15:07:07 +0100 Message-ID: <20230808140709.47733-3-Joe.Ramsay@arm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230808140709.47733-1-Joe.Ramsay@arm.com> References: <20230808140709.47733-1-Joe.Ramsay@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: AM7EUR03FT061:EE_|AS8PR08MB9885:EE_|DBAEUR03FT038:EE_|DU0PR08MB9680:EE_ X-MS-Office365-Filtering-Correlation-Id: c7260996-0191-4d22-b680-08db9818c9c4 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: WwbOtTtpfVHcBGJfuqOWOCHu4/7poSVt/z14Eo6On0aWKFDZP5YGbbYUio1Eez16ai/hkGHV61Pzm+7aVP4pgIfbPMDI/RiCsblocnxlRFg4na8nLHEx9gZDsB7gYHOQyOJzLvGrYEICoDjEei4KpPYCsXDSft14NnlVIrMriOJnCf7fnZ977b3Drm5y++c9pUBulqYXIY2vPhlpR6n8UvGhE/T0qSBygUoEebV7/BT+AG4lgBZWdXj1fFX1NVK3iMCWosoZCqRzfu4ZSS4m9AEDp1KMFqzEQmDxIxm4vNICZ+hd0fyVBzHXwN/2xqiyJAzEJseHxpSqavPenFRxCEb6shdwbIHDy2wfSXs30ZamJCTgCcZXPMPyXLWWRvxEwHlq9UkskJI/r13kALBdJ+sDHRE9HbVJWArS0qmcqYI66xbBJuHc9Fqy2pDH1fn6SdrQVvPjtb+Mn3oEqHxFksztrqqnUTN2Yg19N1EwRUCb3lEd6csXmKUmcXsJm+Ut7wUxEqcsDpCF4Bth1MOjgzBwMqP+Ecb+pQekttkvljKJKbrhwATySxAsbHmMP4ZHGs06CgPV6zP4ltIVXUzjsvtI6IaqEBJI3qfa9uguGgdxz7ClDlAEPLfuZc92TX9Sh5hqKTpuat4CiEY1vZONjx8+i4fFduOCequkg53YDQ/WiBpMI2Maf9LdekRMdJMKKuDcGsdYcU/4D63MrDScZoOM9mOViNDF2ZavdrIO18Ti+Dcvngs6xRloLHV+GvAuxTsZW5oeicmZpgFtgXEaqt8jcQDUGSwkdRj2zG7V9LHrw59q2vxeAfk3NWYteBF0 X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(39860400002)(376002)(396003)(346002)(136003)(82310400008)(451199021)(1800799003)(186006)(46966006)(36840700001)(86362001)(40480700001)(41300700001)(336012)(478600001)(8676002)(1076003)(26005)(8936002)(7696005)(6666004)(426003)(2616005)(5660300002)(47076005)(2906002)(83380400001)(30864003)(36756003)(36860700001)(316002)(356005)(70206006)(70586007)(6916009)(81166007)(82740400003)(4326008)(2004002)(36900700001)(473944003); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB9885 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT038.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 8434a7fc-2b62-45d4-e8a3-08db9818c451 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: p6/+c9mhJS+UQGmrVeleG1HhMMoCIegK1Qd/Bnwn4qmpLkxxM8aKqtc/klY+4DuwyLhnTH15yBY3uOkUUy8Nn9Bwu771qe6YcnKzBcvp8etN7nhYz4Vt1uJ2pNHIiI/NWyGP2A+2mZHCT+n1VdkOv0yheY/xoI8vBYM8Sx8QZZif+VIkhrJbJu/8nzHmSOijcUvYn0pKrI7sr4WNaOLNlXs9eWO5Vti2l2emtLZsLVOS74SgYyyqw/MOwkYR6HTyxJ2AJU7HcX2nkSExDM4yoeKxxrmtbnlb1uEnzVnqketK5cNDdAqagYsGv9612dzfxaV6bmHglsxiB45GGg/B/oyQUcDVjRbrzTlyqkuGuhrERwcb1eMBokASQvDrtQG5Xzfq6U2BHHSIox1ac4KCYVuPeGib1UJlorWV5xTsMzZhZeZ+i4e36ipbEWyJng6IvanuB4n43c0J0Nn/3Ch0snm9pZ3945YzdDbW4DP4pHiZx80Up00bZHYSQa5JR92AuLwQTeImJdGA6FXh5/PDANaGIP1WJGTTxUY5QJjTU5EQJYoiBMmDHf4QU9iQdmqJGNciQ7B/xrYwr3re4uYzDCMMx3uSh3ybqyniyh/HO1dpiU2xFqYfIJtXjc8fMUlxhfJJDUickqUxsE0fk0R5RDcBiQH9T/ijWsZ1DuNXYWDisJBZo42ki6wOeXQRlGlINjwGISYshnFmfjTq4nB2v5906TsPwRCkr2z03mSji1HyBuDYgn6SGswVbLrkCQh6jVSQm7vsQ0ZjJTfwCYe94A== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230028)(4636009)(396003)(376002)(39860400002)(346002)(136003)(82310400008)(451199021)(186006)(1800799003)(46966006)(36840700001)(40470700004)(86362001)(40480700001)(41300700001)(336012)(478600001)(40460700003)(1076003)(8676002)(8936002)(26005)(7696005)(6666004)(2616005)(426003)(5660300002)(47076005)(36756003)(2906002)(30864003)(83380400001)(36860700001)(316002)(70206006)(6916009)(70586007)(82740400003)(81166007)(4326008)(2004002)(473944003); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Aug 2023 14:07:23.4806 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c7260996-0191-4d22-b680-08db9818c9c4 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT038.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB9680 X-Spam-Status: No, score=-12.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Joe Ramsay via Libc-alpha From: Joe Ramsay Reply-To: Joe Ramsay Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" A table is also added, which is shared between AdvSIMD and SVE log2. --- Changes to v1: Fix check-abi Thanks, Joe sysdeps/aarch64/fpu/Makefile | 4 +- sysdeps/aarch64/fpu/Versions | 4 + sysdeps/aarch64/fpu/bits/math-vector.h | 4 + sysdeps/aarch64/fpu/log2_advsimd.c | 113 ++++++++++++ sysdeps/aarch64/fpu/log2_sve.c | 71 ++++++++ sysdeps/aarch64/fpu/log2f_advsimd.c | 77 ++++++++ sysdeps/aarch64/fpu/log2f_sve.c | 87 +++++++++ .../fpu/test-double-advsimd-wrappers.c | 1 + .../aarch64/fpu/test-double-sve-wrappers.c | 1 + .../aarch64/fpu/test-float-advsimd-wrappers.c | 1 + sysdeps/aarch64/fpu/test-float-sve-wrappers.c | 1 + sysdeps/aarch64/fpu/v_log2_data.c | 165 ++++++++++++++++++ sysdeps/aarch64/fpu/vecmath_config.h | 10 ++ sysdeps/aarch64/libm-test-ulps | 8 + .../unix/sysv/linux/aarch64/libmvec.abilist | 4 + 15 files changed, 550 insertions(+), 1 deletion(-) create mode 100644 sysdeps/aarch64/fpu/log2_advsimd.c create mode 100644 sysdeps/aarch64/fpu/log2_sve.c create mode 100644 sysdeps/aarch64/fpu/log2f_advsimd.c create mode 100644 sysdeps/aarch64/fpu/log2f_sve.c create mode 100644 sysdeps/aarch64/fpu/v_log2_data.c diff --git a/sysdeps/aarch64/fpu/Makefile b/sysdeps/aarch64/fpu/Makefile index 9c7c768301..c3f204ff0d 100644 --- a/sysdeps/aarch64/fpu/Makefile +++ b/sysdeps/aarch64/fpu/Makefile @@ -2,6 +2,7 @@ libmvec-supported-funcs = cos \ exp \ exp2 \ log \ + log2 \ sin \ tan @@ -16,7 +17,8 @@ libmvec-support = $(addsuffix f_advsimd,$(float-advsimd-funcs)) \ $(addsuffix f_sve,$(float-sve-funcs)) \ $(addsuffix _sve,$(double-sve-funcs)) \ v_log_data \ - v_exp_data + v_exp_data \ + v_log2_data endif sve-cflags = -march=armv8-a+sve diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions index 05de4325d5..ffe62a6f65 100644 --- a/sysdeps/aarch64/fpu/Versions +++ b/sysdeps/aarch64/fpu/Versions @@ -22,6 +22,10 @@ libmvec { _ZGVnN2v_exp2; _ZGVsMxv_exp2f; _ZGVsMxv_exp2; + _ZGVnN4v_log2f; + _ZGVnN2v_log2; + _ZGVsMxv_log2f; + _ZGVsMxv_log2; _ZGVnN4v_tanf; _ZGVnN2v_tan; _ZGVsMxv_tanf; diff --git a/sysdeps/aarch64/fpu/bits/math-vector.h b/sysdeps/aarch64/fpu/bits/math-vector.h index 50921b22e5..92f214b194 100644 --- a/sysdeps/aarch64/fpu/bits/math-vector.h +++ b/sysdeps/aarch64/fpu/bits/math-vector.h @@ -53,6 +53,7 @@ __vpcs __f32x4_t _ZGVnN4v_cosf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_expf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_exp2f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_logf (__f32x4_t); +__vpcs __f32x4_t _ZGVnN4v_log2f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_sinf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_tanf (__f32x4_t); @@ -60,6 +61,7 @@ __vpcs __f64x2_t _ZGVnN2v_cos (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp2 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log (__f64x2_t); +__vpcs __f64x2_t _ZGVnN2v_log2 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_sin (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_tan (__f64x2_t); @@ -72,6 +74,7 @@ __sv_f32_t _ZGVsMxv_cosf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_expf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_exp2f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_logf (__sv_f32_t, __sv_bool_t); +__sv_f32_t _ZGVsMxv_log2f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_sinf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_tanf (__sv_f32_t, __sv_bool_t); @@ -79,6 +82,7 @@ __sv_f64_t _ZGVsMxv_cos (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp2 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log (__sv_f64_t, __sv_bool_t); +__sv_f64_t _ZGVsMxv_log2 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_sin (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_tan (__sv_f64_t, __sv_bool_t); diff --git a/sysdeps/aarch64/fpu/log2_advsimd.c b/sysdeps/aarch64/fpu/log2_advsimd.c new file mode 100644 index 0000000000..7617ac57e0 --- /dev/null +++ b/sysdeps/aarch64/fpu/log2_advsimd.c @@ -0,0 +1,113 @@ +/* Double-precision vector (AdvSIMD) exp2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" + +#define V_LOG2_TABLE_BITS 7 +#define N (1 << V_LOG2_TABLE_BITS) + +static const struct data +{ + float64x2_t poly[5]; + float64x2_t invln2; + uint64x2_t min_norm, special_bound, sign_exp_mask; +} data = { + /* Each coefficient was generated to approximate log(r) for |r| < 0x1.fp-9 + and N = 128, then scaled by log2(e) in extended precision and rounded back + to double precision. */ + .poly = { V2 (-0x1.71547652b83p-1), V2 (0x1.ec709dc340953p-2), + V2 (-0x1.71547651c8f35p-2), V2 (0x1.2777ebe12dda5p-2), + V2 (-0x1.ec738d616fe26p-3) }, + .invln2 = V2 (0x1.71547652b82fep0), + .min_norm = V2 (0x0010000000000000), /* asuint64(0x1p-1022). */ + .special_bound = V2 (0x7fe0000000000000), /* asuint64(inf) - min_norm. */ + .sign_exp_mask = V2 (0xfff0000000000000), +}; + +#define Off v_u64 (0x3fe6900900000000) +#define IndexMask v_u64 (N - 1) + +#define T(s, i) __v_log2_data.s[i] +#define C(i) d->poly[i] + +struct entry +{ + float64x2_t invc; + float64x2_t log2c; +}; + +static inline struct entry +lookup (uint64x2_t i) +{ + struct entry e; + e.invc[0] = T (invc, i[0]); + e.log2c[0] = T (log2c, i[0]); + e.invc[1] = T (invc, i[1]); + e.log2c[1] = T (log2c, i[1]); + return e; +} + +static float64x2_t VPCS_ATTR NOINLINE +special_case (float64x2_t x, float64x2_t y, uint64x2_t special) +{ + return v_call_f64 (log2, x, y, special); +} + +/* Double-precision vector log2 routine. Implements the same algorithm as + vector log10, with coefficients and table entries scaled in extended + precision. The maximum observed error is 2.58 ULP: + _ZGVnN2v_log2(0x1.0b556b093869bp+0) got 0x1.fffb34198d9dap-5 + want 0x1.fffb34198d9ddp-5. */ +float64x2_t VPCS_ATTR V_NAME_D1 (log2) (float64x2_t x) +{ + const struct data *d = ptr_barrier (&data); + uint64x2_t ix = vreinterpretq_u64_f64 (x); + uint64x2_t special + = vcgeq_u64 (vsubq_u64 (ix, d->min_norm), d->special_bound); + + /* x = 2^k z; where z is in range [Off,2*Off) and exact. + The range is split into N subintervals. + The ith subinterval contains z and c is near its center. */ + uint64x2_t tmp = vsubq_u64 (ix, Off); + uint64x2_t i + = vandq_u64 (vshrq_n_u64 (tmp, 52 - V_LOG2_TABLE_BITS), IndexMask); + int64x2_t k = vshrq_n_s64 (vreinterpretq_s64_u64 (tmp), 52); + uint64x2_t iz = vsubq_u64 (ix, vandq_u64 (tmp, d->sign_exp_mask)); + float64x2_t z = vreinterpretq_f64_u64 (iz); + + struct entry e = lookup (i); + + /* log2(x) = log1p(z/c-1)/log(2) + log2(c) + k. */ + + float64x2_t r = vfmaq_f64 (v_f64 (-1.0), z, e.invc); + float64x2_t kd = vcvtq_f64_s64 (k); + float64x2_t w = vfmaq_f64 (e.log2c, r, d->invln2); + + float64x2_t r2 = vmulq_f64 (r, r); + float64x2_t p_23 = vfmaq_f64 (C (2), C (3), r); + float64x2_t p_01 = vfmaq_f64 (C (0), C (1), r); + float64x2_t y; + y = vfmaq_f64 (p_23, C (4), r2); + y = vfmaq_f64 (p_01, r2, y); + y = vfmaq_f64 (vaddq_f64 (kd, w), r2, y); + + if (__glibc_unlikely (v_any_u64 (special))) + return special_case (x, y, special); + return y; +} diff --git a/sysdeps/aarch64/fpu/log2_sve.c b/sysdeps/aarch64/fpu/log2_sve.c new file mode 100644 index 0000000000..6645e0369b --- /dev/null +++ b/sysdeps/aarch64/fpu/log2_sve.c @@ -0,0 +1,71 @@ +/* Double-precision vector (SVE) log2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f64.h" + +#define N (1 << V_LOG2_TABLE_BITS) +#define Off 0x3fe6900900000000 +#define Max (0x7ff0000000000000) +#define Min (0x0010000000000000) +#define Thresh (0x7fe0000000000000) /* Max - Min. */ + +static svfloat64_t NOINLINE +special_case (svfloat64_t x, svfloat64_t y, const svbool_t cmp) +{ + return sv_call_f64 (log2, x, y, cmp); +} + +/* Double-precision SVE log2 routine. + Implements the same algorithm as AdvSIMD log10, with coefficients and table + entries scaled in extended precision. + The maximum observed error is 2.58 ULP: + SV_NAME_D1 (log2)(0x1.0b556b093869bp+0) got 0x1.fffb34198d9dap-5 + want 0x1.fffb34198d9ddp-5. */ +svfloat64_t SV_NAME_D1 (log2) (svfloat64_t x, const svbool_t pg) +{ + svuint64_t ix = svreinterpret_u64 (x); + svbool_t special = svcmpge (pg, svsub_x (pg, ix, Min), Thresh); + + /* x = 2^k z; where z is in range [Off,2*Off) and exact. + The range is split into N subintervals. + The ith subinterval contains z and c is near its center. */ + svuint64_t tmp = svsub_x (pg, ix, Off); + svuint64_t i + = sv_mod_n_u64_x (pg, svlsr_x (pg, tmp, 52 - V_LOG2_TABLE_BITS), N); + svfloat64_t k = svcvt_f64_x (pg, svasr_x (pg, svreinterpret_s64 (tmp), 52)); + svfloat64_t z = svreinterpret_f64 ( + svsub_x (pg, ix, svand_x (pg, tmp, 0xfffULL << 52))); + + svfloat64_t invc = svld1_gather_index (pg, __v_log2_data.invc, i); + svfloat64_t log2c = svld1_gather_index (pg, __v_log2_data.log2c, i); + + /* log2(x) = log1p(z/c-1)/log(2) + log2(c) + k. */ + + svfloat64_t r = svmad_x (pg, invc, z, -1.0); + svfloat64_t w = svmla_x (pg, log2c, r, __v_log2_data.invln2); + + svfloat64_t r2 = svmul_x (pg, r, r); + svfloat64_t y = sv_pw_horner_4_f64_x (pg, r, r2, __v_log2_data.poly); + y = svmla_x (pg, svadd_x (pg, k, w), r2, y); + + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x, y, special); + return y; +} diff --git a/sysdeps/aarch64/fpu/log2f_advsimd.c b/sysdeps/aarch64/fpu/log2f_advsimd.c new file mode 100644 index 0000000000..085c18489f --- /dev/null +++ b/sysdeps/aarch64/fpu/log2f_advsimd.c @@ -0,0 +1,77 @@ +/* Single-precision vector (AdvSIMD) exp2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f32.h" + +static const struct data +{ + float32x4_t poly[9]; + float32x4_t ln2; + uint32x4_t min_norm, special_bound, off, mantissa_mask; +} data = { + /* Coefficients generated using Remez algorithm approximate + log2(1+r)/r for r in [ -1/3, 1/3 ]. + rel error: 0x1.c4c4b0cp-26. */ + .poly = { V4 (0x1.715476p0f), /* (float)(1 / ln(2)). */ + V4 (-0x1.715458p-1f), V4 (0x1.ec701cp-2f), V4 (-0x1.7171a4p-2f), + V4 (0x1.27a0b8p-2f), V4 (-0x1.e5143ep-3f), V4 (0x1.9d8ecap-3f), + V4 (-0x1.c675bp-3f), V4 (0x1.9e495p-3f) }, + .ln2 = V4 (0x1.62e43p-1f), + .min_norm = V4 (0x00800000), + .special_bound = V4 (0x7f000000), /* asuint32(inf) - min_norm. */ + .off = V4 (0x3f2aaaab), /* 0.666667. */ + .mantissa_mask = V4 (0x007fffff), +}; + +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t x, float32x4_t y, uint32x4_t cmp) +{ + /* Fall back to scalar code. */ + return v_call_f32 (log2f, x, y, cmp); +} + +/* Fast implementation for single precision AdvSIMD log2, + relies on same argument reduction as AdvSIMD logf. + Maximum error: 2.48 ULPs + _ZGVnN4v_log2f(0x1.558174p+0) got 0x1.a9be84p-2 + want 0x1.a9be8p-2. */ +float32x4_t VPCS_ATTR V_NAME_F1 (log2) (float32x4_t x) +{ + const struct data *d = ptr_barrier (&data); + uint32x4_t u = vreinterpretq_u32_f32 (x); + uint32x4_t special + = vcgeq_u32 (vsubq_u32 (u, d->min_norm), d->special_bound); + + /* x = 2^n * (1+r), where 2/3 < 1+r < 4/3. */ + u = vsubq_u32 (u, d->off); + float32x4_t n = vcvtq_f32_s32 ( + vshrq_n_s32 (vreinterpretq_s32_u32 (u), 23)); /* signextend. */ + u = vaddq_u32 (vandq_u32 (u, d->mantissa_mask), d->off); + float32x4_t r = vsubq_f32 (vreinterpretq_f32_u32 (u), v_f32 (1.0f)); + + /* y = log2(1+r) + n. */ + float32x4_t r2 = vmulq_f32 (r, r); + float32x4_t p = v_pw_horner_8_f32 (r, r2, d->poly); + float32x4_t y = vfmaq_f32 (n, p, r); + + if (__glibc_unlikely (v_any_u32 (special))) + return special_case (x, y, special); + return y; +} diff --git a/sysdeps/aarch64/fpu/log2f_sve.c b/sysdeps/aarch64/fpu/log2f_sve.c new file mode 100644 index 0000000000..1efdd00ab1 --- /dev/null +++ b/sysdeps/aarch64/fpu/log2f_sve.c @@ -0,0 +1,87 @@ +/* Single-precision vector (SVE) log2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" + +static const struct data +{ + float poly_02468[5]; + float poly_1357[4]; +} data = { + .poly_1357 = { + /* Coefficients copied from the AdvSIMD routine, then rearranged so that coeffs + 1, 3, 5 and 7 can be loaded as a single quad-word, hence used with _lane + variant of MLA intrinsic. */ + -0x1.715458p-1f, -0x1.7171a4p-2f, -0x1.e5143ep-3f, -0x1.c675bp-3f + }, + .poly_02468 = { 0x1.715476p0f, 0x1.ec701cp-2f, 0x1.27a0b8p-2f, + 0x1.9d8ecap-3f, 0x1.9e495p-3f }, +}; + +#define Min (0x00800000) +#define Max (0x7f800000) +#define Thres (0x7f000000) /* Max - Min. */ +#define MantissaMask (0x007fffff) +#define Off (0x3f2aaaab) /* 0.666667. */ + +static svfloat32_t NOINLINE +special_case (svfloat32_t x, svfloat32_t y, svbool_t cmp) +{ + return sv_call_f32 (log2f, x, y, cmp); +} + +/* Optimised implementation of SVE log2f, using the same algorithm + and polynomial as AdvSIMD log2f. + Maximum error is 2.48 ULPs: + SV_NAME_F1 (log2)(0x1.558174p+0) got 0x1.a9be84p-2 + want 0x1.a9be8p-2. */ +svfloat32_t SV_NAME_F1 (log2) (svfloat32_t x, const svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + + svuint32_t u = svreinterpret_u32 (x); + svbool_t special = svcmpge (pg, svsub_x (pg, u, Min), Thres); + + /* x = 2^n * (1+r), where 2/3 < 1+r < 4/3. */ + u = svsub_x (pg, u, Off); + svfloat32_t n = svcvt_f32_x ( + pg, svasr_x (pg, svreinterpret_s32 (u), 23)); /* Sign-extend. */ + u = svand_x (pg, u, MantissaMask); + u = svadd_x (pg, u, Off); + svfloat32_t r = svsub_x (pg, svreinterpret_f32 (u), 1.0f); + + /* y = log2(1+r) + n. */ + svfloat32_t r2 = svmul_x (pg, r, r); + + /* Evaluate polynomial using pairwise Horner scheme. */ + svfloat32_t p_1357 = svld1rq (pg, &d->poly_1357[0]); + svfloat32_t q_01 = svmla_lane (sv_f32 (d->poly_02468[0]), r, p_1357, 0); + svfloat32_t q_23 = svmla_lane (sv_f32 (d->poly_02468[1]), r, p_1357, 1); + svfloat32_t q_45 = svmla_lane (sv_f32 (d->poly_02468[2]), r, p_1357, 2); + svfloat32_t q_67 = svmla_lane (sv_f32 (d->poly_02468[3]), r, p_1357, 3); + svfloat32_t y = svmla_x (pg, q_67, r2, sv_f32 (d->poly_02468[4])); + y = svmla_x (pg, q_45, r2, y); + y = svmla_x (pg, q_23, r2, y); + y = svmla_x (pg, q_01, r2, y); + y = svmla_x (pg, n, r, y); + + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x, y, special); + return y; +} diff --git a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c index b2b36fd847..d30dcd6f95 100644 --- a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c @@ -27,5 +27,6 @@ VPCS_VECTOR_WRAPPER (cos_advsimd, _ZGVnN2v_cos) VPCS_VECTOR_WRAPPER (exp_advsimd, _ZGVnN2v_exp) VPCS_VECTOR_WRAPPER (exp2_advsimd, _ZGVnN2v_exp2) VPCS_VECTOR_WRAPPER (log_advsimd, _ZGVnN2v_log) +VPCS_VECTOR_WRAPPER (log2_advsimd, _ZGVnN2v_log2) VPCS_VECTOR_WRAPPER (sin_advsimd, _ZGVnN2v_sin) VPCS_VECTOR_WRAPPER (tan_advsimd, _ZGVnN2v_tan) diff --git a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c index 88b76ed678..22a8479100 100644 --- a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c @@ -36,5 +36,6 @@ SVE_VECTOR_WRAPPER (cos_sve, _ZGVsMxv_cos) SVE_VECTOR_WRAPPER (exp_sve, _ZGVsMxv_exp) SVE_VECTOR_WRAPPER (exp2_sve, _ZGVsMxv_exp2) SVE_VECTOR_WRAPPER (log_sve, _ZGVsMxv_log) +SVE_VECTOR_WRAPPER (log2_sve, _ZGVsMxv_log2) SVE_VECTOR_WRAPPER (sin_sve, _ZGVsMxv_sin) SVE_VECTOR_WRAPPER (tan_sve, _ZGVsMxv_tan) diff --git a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c index 02ab609b5a..e8f7f47c67 100644 --- a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c @@ -27,5 +27,6 @@ VPCS_VECTOR_WRAPPER (cosf_advsimd, _ZGVnN4v_cosf) VPCS_VECTOR_WRAPPER (expf_advsimd, _ZGVnN4v_expf) VPCS_VECTOR_WRAPPER (exp2f_advsimd, _ZGVnN4v_exp2f) VPCS_VECTOR_WRAPPER (logf_advsimd, _ZGVnN4v_logf) +VPCS_VECTOR_WRAPPER (log2f_advsimd, _ZGVnN4v_log2f) VPCS_VECTOR_WRAPPER (sinf_advsimd, _ZGVnN4v_sinf) VPCS_VECTOR_WRAPPER (tanf_advsimd, _ZGVnN4v_tanf) diff --git a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c index fa41ce09d8..f5e9584265 100644 --- a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c @@ -36,5 +36,6 @@ SVE_VECTOR_WRAPPER (cosf_sve, _ZGVsMxv_cosf) SVE_VECTOR_WRAPPER (expf_sve, _ZGVsMxv_expf) SVE_VECTOR_WRAPPER (exp2f_sve, _ZGVsMxv_exp2f) SVE_VECTOR_WRAPPER (logf_sve, _ZGVsMxv_logf) +SVE_VECTOR_WRAPPER (log2f_sve, _ZGVsMxv_log2f) SVE_VECTOR_WRAPPER (sinf_sve, _ZGVsMxv_sinf) SVE_VECTOR_WRAPPER (tanf_sve, _ZGVsMxv_tanf) diff --git a/sysdeps/aarch64/fpu/v_log2_data.c b/sysdeps/aarch64/fpu/v_log2_data.c new file mode 100644 index 0000000000..2b62c59fe1 --- /dev/null +++ b/sysdeps/aarch64/fpu/v_log2_data.c @@ -0,0 +1,165 @@ +/* Coefficients and table entries for vector log2 + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "vecmath_config.h" + +#define N (1 << V_LOG2_TABLE_BITS) + +const struct v_log2_data __v_log2_data = { + + /* Each coefficient was generated to approximate log(r) for |r| < 0x1.fp-9 + and N = 128, then scaled by log2(e) in extended precision and rounded back + to double precision. */ + .poly = { -0x1.71547652b83p-1, 0x1.ec709dc340953p-2, -0x1.71547651c8f35p-2, + 0x1.2777ebe12dda5p-2, -0x1.ec738d616fe26p-3 }, + + .invln2 = 0x1.71547652b82fep0, + + /* Derived from tables in v_log_data.c in a similar way as v_log10_data.c. + This means invc is unchanged and log2c was calculated by scaling log(c) by + log2(e) in extended precision and rounding back to double precision. */ + .invc = { 0x1.6a133d0dec120p+0, 0x1.6815f2f3e42edp+0, + 0x1.661e39be1ac9ep+0, 0x1.642bfa30ac371p+0, + 0x1.623f1d916f323p+0, 0x1.60578da220f65p+0, + 0x1.5e75349dea571p+0, 0x1.5c97fd387a75ap+0, + 0x1.5abfd2981f200p+0, 0x1.58eca051dc99cp+0, + 0x1.571e526d9df12p+0, 0x1.5554d555b3fcbp+0, + 0x1.539015e2a20cdp+0, 0x1.51d0014ee0164p+0, + 0x1.50148538cd9eep+0, 0x1.4e5d8f9f698a1p+0, + 0x1.4cab0edca66bep+0, 0x1.4afcf1a9db874p+0, + 0x1.495327136e16fp+0, 0x1.47ad9e84af28fp+0, + 0x1.460c47b39ae15p+0, 0x1.446f12b278001p+0, + 0x1.42d5efdd720ecp+0, 0x1.4140cfe001a0fp+0, + 0x1.3fafa3b421f69p+0, 0x1.3e225c9c8ece5p+0, + 0x1.3c98ec29a211ap+0, 0x1.3b13442a413fep+0, + 0x1.399156baa3c54p+0, 0x1.38131639b4cdbp+0, + 0x1.36987540fbf53p+0, 0x1.352166b648f61p+0, + 0x1.33adddb3eb575p+0, 0x1.323dcd99fc1d3p+0, + 0x1.30d129fefc7d2p+0, 0x1.2f67e6b72fe7dp+0, + 0x1.2e01f7cf8b187p+0, 0x1.2c9f518ddc86ep+0, + 0x1.2b3fe86e5f413p+0, 0x1.29e3b1211b25cp+0, + 0x1.288aa08b373cfp+0, 0x1.2734abcaa8467p+0, + 0x1.25e1c82459b81p+0, 0x1.2491eb1ad59c5p+0, + 0x1.23450a54048b5p+0, 0x1.21fb1bb09e578p+0, + 0x1.20b415346d8f7p+0, 0x1.1f6fed179a1acp+0, + 0x1.1e2e99b93c7b3p+0, 0x1.1cf011a7a882ap+0, + 0x1.1bb44b97dba5ap+0, 0x1.1a7b3e66cdd4fp+0, + 0x1.1944e11dc56cdp+0, 0x1.18112aebb1a6ep+0, + 0x1.16e013231b7e9p+0, 0x1.15b1913f156cfp+0, + 0x1.14859cdedde13p+0, 0x1.135c2dc68cfa4p+0, + 0x1.12353bdb01684p+0, 0x1.1110bf25b85b4p+0, + 0x1.0feeafd2f8577p+0, 0x1.0ecf062c51c3bp+0, + 0x1.0db1baa076c8bp+0, 0x1.0c96c5bb3048ep+0, + 0x1.0b7e20263e070p+0, 0x1.0a67c2acd0ce3p+0, + 0x1.0953a6391e982p+0, 0x1.0841c3caea380p+0, + 0x1.07321489b13eap+0, 0x1.062491aee9904p+0, + 0x1.05193497a7cc5p+0, 0x1.040ff6b5f5e9fp+0, + 0x1.0308d19aa6127p+0, 0x1.0203beedb0c67p+0, + 0x1.010037d38bcc2p+0, 1.0, + 0x1.fc06d493cca10p-1, 0x1.f81e6ac3b918fp-1, + 0x1.f44546ef18996p-1, 0x1.f07b10382c84bp-1, + 0x1.ecbf7070e59d4p-1, 0x1.e91213f715939p-1, + 0x1.e572a9a75f7b7p-1, 0x1.e1e0e2c530207p-1, + 0x1.de5c72d8a8be3p-1, 0x1.dae50fa5658ccp-1, + 0x1.d77a71145a2dap-1, 0x1.d41c51166623ep-1, + 0x1.d0ca6ba0bb29fp-1, 0x1.cd847e8e59681p-1, + 0x1.ca4a499693e00p-1, 0x1.c71b8e399e821p-1, + 0x1.c3f80faf19077p-1, 0x1.c0df92dc2b0ecp-1, + 0x1.bdd1de3cbb542p-1, 0x1.baceb9e1007a3p-1, + 0x1.b7d5ef543e55ep-1, 0x1.b4e749977d953p-1, + 0x1.b20295155478ep-1, 0x1.af279f8e82be2p-1, + 0x1.ac5638197fdf3p-1, 0x1.a98e2f102e087p-1, + 0x1.a6cf5606d05c1p-1, 0x1.a4197fc04d746p-1, + 0x1.a16c80293dc01p-1, 0x1.9ec82c4dc5bc9p-1, + 0x1.9c2c5a491f534p-1, 0x1.9998e1480b618p-1, + 0x1.970d9977c6c2dp-1, 0x1.948a5c023d212p-1, + 0x1.920f0303d6809p-1, 0x1.8f9b698a98b45p-1, + 0x1.8d2f6b81726f6p-1, 0x1.8acae5bb55badp-1, + 0x1.886db5d9275b8p-1, 0x1.8617ba567c13cp-1, + 0x1.83c8d27487800p-1, 0x1.8180de3c5dbe7p-1, + 0x1.7f3fbe71cdb71p-1, 0x1.7d055498071c1p-1, + 0x1.7ad182e54f65ap-1, 0x1.78a42c3c90125p-1, + 0x1.767d342f76944p-1, 0x1.745c7ef26b00ap-1, + 0x1.7241f15769d0fp-1, 0x1.702d70d396e41p-1, + 0x1.6e1ee3700cd11p-1, 0x1.6c162fc9cbe02p-1 }, + .log2c = { -0x1.00130d57f5fadp-1, -0x1.f802661bd725ep-2, + -0x1.efea1c6f73a5bp-2, -0x1.e7dd1dcd06f05p-2, + -0x1.dfdb4ae024809p-2, -0x1.d7e484d101958p-2, + -0x1.cff8ad452f6ep-2, -0x1.c817a666c997fp-2, + -0x1.c04152d640419p-2, -0x1.b87595a3f64b2p-2, + -0x1.b0b4526c44d07p-2, -0x1.a8fd6d1a90f5ep-2, + -0x1.a150ca2559fc6p-2, -0x1.99ae4e62cca29p-2, + -0x1.9215df1a1e842p-2, -0x1.8a8761fe1f0d9p-2, + -0x1.8302bd1cc9a54p-2, -0x1.7b87d6fb437f6p-2, + -0x1.741696673a86dp-2, -0x1.6caee2b3c6fe4p-2, + -0x1.6550a3666c27ap-2, -0x1.5dfbc08de02a4p-2, + -0x1.56b022766c84ap-2, -0x1.4f6db1c955536p-2, + -0x1.4834579063054p-2, -0x1.4103fd2249a76p-2, + -0x1.39dc8c3fe6dabp-2, -0x1.32bdeed4b5c8fp-2, + -0x1.2ba80f41e20ddp-2, -0x1.249ad8332f4a7p-2, + -0x1.1d96347e7f3ebp-2, -0x1.169a0f7d6604ap-2, + -0x1.0fa654a221909p-2, -0x1.08baefcf8251ap-2, + -0x1.01d7cd14deecdp-2, -0x1.f5f9b1ad55495p-3, + -0x1.e853ff76a77afp-3, -0x1.dabe5d624cba1p-3, + -0x1.cd38a5cef4822p-3, -0x1.bfc2b38d315f9p-3, + -0x1.b25c61f5edd0fp-3, -0x1.a5058d18e9cacp-3, + -0x1.97be1113e47a3p-3, -0x1.8a85cafdf5e27p-3, + -0x1.7d5c97e8fc45bp-3, -0x1.704255d6486e4p-3, + -0x1.6336e2cedd7bfp-3, -0x1.563a1d9b0cc6ap-3, + -0x1.494be541aaa6fp-3, -0x1.3c6c1964dd0f2p-3, + -0x1.2f9a99f19a243p-3, -0x1.22d747344446p-3, + -0x1.1622020d4f7f5p-3, -0x1.097aabb3553f3p-3, + -0x1.f9c24b48014c5p-4, -0x1.e0aaa3bdc858ap-4, + -0x1.c7ae257c952d6p-4, -0x1.aecc960a03e58p-4, + -0x1.9605bb724d541p-4, -0x1.7d595ca7147cep-4, + -0x1.64c74165002d9p-4, -0x1.4c4f31c86d344p-4, + -0x1.33f0f70388258p-4, -0x1.1bac5abb3037dp-4, + -0x1.0381272495f21p-4, -0x1.d6de4eba2de2ap-5, + -0x1.a6ec4e8156898p-5, -0x1.772be542e3e1bp-5, + -0x1.479cadcde852dp-5, -0x1.183e4265faa5p-5, + -0x1.d2207fdaa1b85p-6, -0x1.742486cb4a6a2p-6, + -0x1.1687d77cfc299p-6, -0x1.7293623a6b5dep-7, + -0x1.70ec80ec8f25dp-8, 0.0, + 0x1.704c1ca6b6bc9p-7, 0x1.6eac8ba664beap-6, + 0x1.11e67d040772dp-5, 0x1.6bc665e2105dep-5, + 0x1.c4f8a9772bf1dp-5, 0x1.0ebff10fbb951p-4, + 0x1.3aaf4d7805d11p-4, 0x1.664ba81a4d717p-4, + 0x1.9196387da6de4p-4, 0x1.bc902f2b7796p-4, + 0x1.e73ab5f584f28p-4, 0x1.08cb78510d232p-3, + 0x1.1dd2fe2f0dcb5p-3, 0x1.32b4784400df4p-3, + 0x1.47706f3d49942p-3, 0x1.5c0768ee4a4dcp-3, + 0x1.7079e86fc7c6dp-3, 0x1.84c86e1183467p-3, + 0x1.98f377a34b499p-3, 0x1.acfb803bc924bp-3, + 0x1.c0e10098b025fp-3, 0x1.d4a46efe103efp-3, + 0x1.e8463f45b8d0bp-3, 0x1.fbc6e3228997fp-3, + 0x1.079364f2e5aa8p-2, 0x1.1133306010a63p-2, + 0x1.1ac309631bd17p-2, 0x1.24432485370c1p-2, + 0x1.2db3b5449132fp-2, 0x1.3714ee1d7a32p-2, + 0x1.406700ab52c94p-2, 0x1.49aa1d87522b2p-2, + 0x1.52de746d7ecb2p-2, 0x1.5c0434336b343p-2, + 0x1.651b8ad6c90d1p-2, 0x1.6e24a56ab5831p-2, + 0x1.771fb04ec29b1p-2, 0x1.800cd6f19c25ep-2, + 0x1.88ec441df11dfp-2, 0x1.91be21b7c93f5p-2, + 0x1.9a8298f8c7454p-2, 0x1.a339d255c04ddp-2, + 0x1.abe3f59f43db7p-2, 0x1.b48129deca9efp-2, + 0x1.bd119575364c1p-2, 0x1.c5955e23ebcbcp-2, + 0x1.ce0ca8f4e1557p-2, 0x1.d6779a5a75774p-2, + 0x1.ded6563550d27p-2, 0x1.e728ffafd840ep-2, + 0x1.ef6fb96c8d739p-2, 0x1.f7aaa57907219p-2 } +}; diff --git a/sysdeps/aarch64/fpu/vecmath_config.h b/sysdeps/aarch64/fpu/vecmath_config.h index e7d30b477f..82a68fd35d 100644 --- a/sysdeps/aarch64/fpu/vecmath_config.h +++ b/sysdeps/aarch64/fpu/vecmath_config.h @@ -48,4 +48,14 @@ extern const struct v_log_data #define V_EXP_TABLE_BITS 7 extern const uint64_t __v_exp_data[1 << V_EXP_TABLE_BITS] attribute_hidden; + +#define V_LOG2_TABLE_BITS 7 +extern const struct v_log2_data +{ + double poly[5]; + double invln2; + double invc[1 << V_LOG2_TABLE_BITS]; + double log2c[1 << V_LOG2_TABLE_BITS]; +} __v_log2_data attribute_hidden; + #endif diff --git a/sysdeps/aarch64/libm-test-ulps b/sysdeps/aarch64/libm-test-ulps index a1e5651c87..7a5af571e2 100644 --- a/sysdeps/aarch64/libm-test-ulps +++ b/sysdeps/aarch64/libm-test-ulps @@ -1220,11 +1220,19 @@ double: 1 float: 1 ldouble: 3 +Function: "log2_advsimd": +double: 1 +float: 2 + Function: "log2_downward": double: 3 float: 3 ldouble: 3 +Function: "log2_sve": +double: 1 +float: 2 + Function: "log2_towardzero": double: 2 float: 2 diff --git a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist index 6046c3d046..657edab7ae 100644 --- a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist +++ b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist @@ -15,10 +15,14 @@ GLIBC_2.38 _ZGVsMxv_logf F GLIBC_2.38 _ZGVsMxv_sin F GLIBC_2.38 _ZGVsMxv_sinf F GLIBC_2.39 _ZGVnN2v_exp2 F +GLIBC_2.39 _ZGVnN2v_log2 F GLIBC_2.39 _ZGVnN2v_tan F GLIBC_2.39 _ZGVnN4v_exp2f F +GLIBC_2.39 _ZGVnN4v_log2f F GLIBC_2.39 _ZGVnN4v_tanf F GLIBC_2.39 _ZGVsMxv_exp2 F GLIBC_2.39 _ZGVsMxv_exp2f F +GLIBC_2.39 _ZGVsMxv_log2 F +GLIBC_2.39 _ZGVsMxv_log2f F GLIBC_2.39 _ZGVsMxv_tan F GLIBC_2.39 _ZGVsMxv_tanf F From patchwork Tue Aug 8 14:07:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Ramsay X-Patchwork-Id: 1818626 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=fjHOVIHA; dkim-atps=neutral Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RKw6443Szz1yYC for ; Wed, 9 Aug 2023 00:09:00 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id C98B338708B0 for ; Tue, 8 Aug 2023 14:08:57 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org C98B338708B0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1691503737; bh=42GNK/N+OBxV2LnL9hFp3nUoobjjPg9CvSjWdyd7SXw=; h=To:CC:Subject:Date:In-Reply-To:References:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=fjHOVIHAitqUVGBsp9PXC8v2mgjuKTPpsQkfF/eZ/pViDDtPL7lN+kgODqW8Rt0qa tLW7/YRX8VVozalbDzD5lu5oqbJStDZL8cGAsrybMuu4n8aMA1IC8zkmGxzJWnApwf ud+jK3kc/NAG8iBzEaZPwiA37ZFxpSpY3Z+A6YiA= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03on2050.outbound.protection.outlook.com [40.107.104.50]) by sourceware.org (Postfix) with ESMTPS id 58DE93858D20 for ; Tue, 8 Aug 2023 14:07:30 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 58DE93858D20 Received: from AM5PR1001CA0035.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:206:2::48) by PAXPR08MB6685.eurprd08.prod.outlook.com (2603:10a6:102:133::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.26; Tue, 8 Aug 2023 14:07:25 +0000 Received: from AM7EUR03FT045.eop-EUR03.prod.protection.outlook.com (2603:10a6:206:2:cafe::96) by AM5PR1001CA0035.outlook.office365.com (2603:10a6:206:2::48) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.27 via Frontend Transport; Tue, 8 Aug 2023 14:07:25 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM7EUR03FT045.mail.protection.outlook.com (100.127.140.150) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6678.17 via Frontend Transport; Tue, 8 Aug 2023 14:07:25 +0000 Received: ("Tessian outbound 997ae1cc9f47:v145"); Tue, 08 Aug 2023 14:07:25 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 6cc73ee0d74e831d X-CR-MTA-TID: 64aa7808 Received: from 340ca0f60534.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 03B608B9-151C-47FE-93EC-B6C065AEE753.1; Tue, 08 Aug 2023 14:07:18 +0000 Received: from EUR05-AM6-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 340ca0f60534.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 08 Aug 2023 14:07:18 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aEw9KQrFQOtsLV0Y4lp8Vld2CMz5z3MoroMrQYG4/q2REwTXNiQA2rkSo2CF4fIfuYGiZV3zfDtyA78o7E2TIaJNJMsFOAyJyj3y9+9h1yNMhrT1xgsVFya8dxO4CwxhQUw6oOptNPJJXUxPrAcANInwNqBFt2ZOYuVBSZBawGdY6EeqNPbuSGIm6OkBw5au/FDDjaEj71t2mkl1lTbrvKW3gWrfg6V9mGV41b/EFyDvcS4moxYEnws94IbYFgMeVAtHiKIJbZnWneTMu6Jg6xWEUtcWxHot5D0+iiiwvMtjJBSAEy/OXMwozwbx0AEjW3XgbOqnAHduymBWmgoeRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=42GNK/N+OBxV2LnL9hFp3nUoobjjPg9CvSjWdyd7SXw=; b=FHZqklUyZrIC3rKS5/+CZtdfsZ+nayygetbXkehlQOBMI0ZBtJ2CNcBzOKclG5WxzUW73+SoboWghLuH5KXWxLNcCcL6LKvJN21+sXpUD+Yyb7Blfr9uveiztaMIqjd337cn/OBukoIcJ1acLwSGNWL/E3OM0zejzq/CHzK55SB7gAcmDsVOcQtQPv5nTMBnJEkV6whruJvpCEmPxPsFwwvM5HlmT/DTL0k4loPuuWrsnEPCt8CqxnMzx9IYDtTV92Ue/KhGSqZO1fJlvdEk6mpjWboYW0AUe/hj01pLuX6KpoW7fFXUm0jYHl4WBp1KH/vLpWuSVcXq/V9L/Ck9SA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none Received: from AM0PR03CA0002.eurprd03.prod.outlook.com (2603:10a6:208:14::15) by AS8PR08MB9790.eurprd08.prod.outlook.com (2603:10a6:20b:615::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.26; Tue, 8 Aug 2023 14:07:14 +0000 Received: from AM7EUR03FT044.eop-EUR03.prod.protection.outlook.com (2603:10a6:208:14:cafe::69) by AM0PR03CA0002.outlook.office365.com (2603:10a6:208:14::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.27 via Frontend Transport; Tue, 8 Aug 2023 14:07:14 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (40.67.248.234) by AM7EUR03FT044.mail.protection.outlook.com (100.127.140.169) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6678.17 via Frontend Transport; Tue, 8 Aug 2023 14:07:14 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX03.Arm.com (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Tue, 8 Aug 2023 14:07:11 +0000 Received: from vcn-man-apps.manchester.arm.com (10.32.108.22) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.27 via Frontend Transport; Tue, 8 Aug 2023 14:07:11 +0000 To: CC: Joe Ramsay Subject: [PATCH 4/5 v2] aarch64: Add vector implementations of log10 routines Date: Tue, 8 Aug 2023 15:07:08 +0100 Message-ID: <20230808140709.47733-4-Joe.Ramsay@arm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230808140709.47733-1-Joe.Ramsay@arm.com> References: <20230808140709.47733-1-Joe.Ramsay@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: AM7EUR03FT044:EE_|AS8PR08MB9790:EE_|AM7EUR03FT045:EE_|PAXPR08MB6685:EE_ X-MS-Office365-Filtering-Correlation-Id: c989cb5e-8e37-4886-6e2f-08db9818cb31 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: bS1wmN3B8RHGLA/81Hirwyf0m4hr3z3iXG1n1o5A/uNK0v+qI/FnzmcFBCrjPzVupdfLZ+3/zlhHR466hhtp7LSdbuS7fDAPk9wVwSZU4GtalzSIKvjf9VdZqRNiomOVpMn76twB409gRQZnBUo4nTrktbJ+5cjTtoBZcbFWVcoDRDyVUOobAbBlOl+D3XmttVJkHos0EiCQ16j/Jaf2u9Y4yI42j9dwNC/XkwT7umLHPA25dff6fRLcBw7zh64im6m5fPEh4ARAlQT4GqWPc4RF0CD+GIQshVnLnEH9o0pr3eQj3rzFA12qbyFvpgrmf0oAvztMd1hyr7AbwGyR5R0q9HYIsqq0OjohRplatDqPWDU7RBPng04RSm2RgzeKVO8Iwu4S8LFYlwv6TM/FGAVxFMfAuSnanWyTG1VtClqBFiwoOjoiaovu9uNiVrmGXyNvxNAjMxH3Zxbfdf9zWnrAYcsxHRo8mLpI17vuIDj3GyBzIIv6Kzrwm8Z/LYeRwTL+DiEVAnUQCNjcE7NzuJQiKGuK8t0jveJEnGs0oVNoOLiaIPkGQNLyei/cDOhpXmeVrnwtMK8aDkjsct0xgniTH3/xbAUTOt5oA/pyeRRL9OMJqavB2LkgH5OET2F52MIQo+g0Nr+jxlN7CymlSJ2HzKBqM7VFOCfji2CvuGxiPi7XVfgjbfPRvfsMe9LacmWhtINgRH2HiqGnY0UuG/TjGRhI+oRjl3ze2BQk1j6wOJCkRwJ/VsUP/Wsl4T2HYTgdzYHGABbrYw1o7Qor6g== X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(376002)(39860400002)(136003)(346002)(396003)(1800799003)(186006)(451199021)(82310400008)(46966006)(36840700001)(2616005)(47076005)(82740400003)(81166007)(478600001)(356005)(336012)(36860700001)(83380400001)(426003)(40480700001)(26005)(1076003)(41300700001)(70586007)(316002)(8676002)(8936002)(70206006)(4326008)(30864003)(2906002)(6916009)(7696005)(6666004)(5660300002)(86362001)(36756003)(2004002)(36900700001)(473944003); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB9790 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM7EUR03FT045.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: e1f2f4eb-eca1-4728-bb13-08db9818c486 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: tTiDMiHefrZ7EqTDUnPEGP00WKnaYyipnqql++rKAlEGZRepHW+m3Fq9IeNHwMz3T58+Jsd9fdZLFqqV1NkBTLCrodD5dHJYUFs1d7p4UmIdOttP/X4dS94MMH2SFGCLEOfJLMVDMrbLZ3HdyQqG6888Scr2ajgsoCxzWLtatfnydGpMpmS3LKrjQ53LS+9+0tmjvMn1/mHpCywhXDiJra4LFVJH2SbS7KJt5s2aflSBIF50CUyztXdsL4tx9HR0mXPN3Yj6M893UemVCBZgnz4xjhyCem4cGmOnUXPqYAgh9YlW4Db+TL8pvonpCEx9KSLo1uJTJhqgYZKhDkEUp0077QhCaqY0OhlSPQuTwBNr7Z1kWO+jZhToS4pxGqu/HmGZDK6Ur20ruepwj1QeUgZgx1lsgQeYTwHFXIByY0swHzQC9+OtJMW5HZTlTExWEkwPtPlxKUWStt2xBQZeKo910k/d+10R6jx1sttWKjEhBEgXiQvYFGo88iBUcvbrUC+nF3lTMaaEFf5DNW6E0F1P2IGwCJF+ia5hw03AFnqZ4huZVCrxWMyN6RG9HbNM/Sl57kc4H7fxg+/cHe4o+jmxfxPPnftJ8x+dSTFGszcrwFFrWfG2kHXcSALlNoqe3cWnkEGDnH+UuNxBeaebcBEqfHqzki9wBIDkVhv+RZkiHsR9rvxxca1aFHxhJkiieYCurLfIY4Tu77yRai7aTzDsNwkav54m0ghkgicIZn+HCTxhzPdJ1IgJhB3CxcwGfmD7QjNsXet6zC4Jc1NgWw== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230028)(4636009)(396003)(376002)(39860400002)(346002)(136003)(82310400008)(451199021)(186006)(1800799003)(46966006)(36840700001)(40470700004)(86362001)(40480700001)(41300700001)(336012)(478600001)(40460700003)(1076003)(8676002)(8936002)(26005)(7696005)(6666004)(2616005)(426003)(5660300002)(47076005)(36756003)(2906002)(30864003)(83380400001)(36860700001)(316002)(70206006)(6916009)(70586007)(82740400003)(81166007)(4326008)(2004002)(473944003); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Aug 2023 14:07:25.7927 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c989cb5e-8e37-4886-6e2f-08db9818cb31 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM7EUR03FT045.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6685 X-Spam-Status: No, score=-12.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_DMARC_NONE, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NONE, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Joe Ramsay via Libc-alpha From: Joe Ramsay Reply-To: Joe Ramsay Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" A table is also added, which is shared between AdvSIMD and SVE log10. --- Changes to v1: Fix check-abi Thanks, Joe sysdeps/aarch64/fpu/Makefile | 4 +- sysdeps/aarch64/fpu/Versions | 4 + sysdeps/aarch64/fpu/bits/math-vector.h | 4 + sysdeps/aarch64/fpu/log10_advsimd.c | 117 ++++++++++++ sysdeps/aarch64/fpu/log10_sve.c | 77 ++++++++ sysdeps/aarch64/fpu/log10f_advsimd.c | 79 ++++++++ sysdeps/aarch64/fpu/log10f_sve.c | 94 ++++++++++ .../fpu/test-double-advsimd-wrappers.c | 1 + .../aarch64/fpu/test-double-sve-wrappers.c | 1 + .../aarch64/fpu/test-float-advsimd-wrappers.c | 1 + sysdeps/aarch64/fpu/test-float-sve-wrappers.c | 1 + sysdeps/aarch64/fpu/v_log10_data.c | 175 ++++++++++++++++++ sysdeps/aarch64/fpu/vecmath_config.h | 9 + sysdeps/aarch64/libm-test-ulps | 8 + .../unix/sysv/linux/aarch64/libmvec.abilist | 4 + 15 files changed, 578 insertions(+), 1 deletion(-) create mode 100644 sysdeps/aarch64/fpu/log10_advsimd.c create mode 100644 sysdeps/aarch64/fpu/log10_sve.c create mode 100644 sysdeps/aarch64/fpu/log10f_advsimd.c create mode 100644 sysdeps/aarch64/fpu/log10f_sve.c create mode 100644 sysdeps/aarch64/fpu/v_log10_data.c diff --git a/sysdeps/aarch64/fpu/Makefile b/sysdeps/aarch64/fpu/Makefile index c3f204ff0d..0a047a150d 100644 --- a/sysdeps/aarch64/fpu/Makefile +++ b/sysdeps/aarch64/fpu/Makefile @@ -2,6 +2,7 @@ libmvec-supported-funcs = cos \ exp \ exp2 \ log \ + log10 \ log2 \ sin \ tan @@ -18,7 +19,8 @@ libmvec-support = $(addsuffix f_advsimd,$(float-advsimd-funcs)) \ $(addsuffix _sve,$(double-sve-funcs)) \ v_log_data \ v_exp_data \ - v_log2_data + v_log2_data \ + v_log10_data endif sve-cflags = -march=armv8-a+sve diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions index ffe62a6f65..358efed5ee 100644 --- a/sysdeps/aarch64/fpu/Versions +++ b/sysdeps/aarch64/fpu/Versions @@ -22,6 +22,10 @@ libmvec { _ZGVnN2v_exp2; _ZGVsMxv_exp2f; _ZGVsMxv_exp2; + _ZGVnN4v_log10f; + _ZGVnN2v_log10; + _ZGVsMxv_log10f; + _ZGVsMxv_log10; _ZGVnN4v_log2f; _ZGVnN2v_log2; _ZGVsMxv_log2f; diff --git a/sysdeps/aarch64/fpu/bits/math-vector.h b/sysdeps/aarch64/fpu/bits/math-vector.h index 92f214b194..59f2efa6d7 100644 --- a/sysdeps/aarch64/fpu/bits/math-vector.h +++ b/sysdeps/aarch64/fpu/bits/math-vector.h @@ -53,6 +53,7 @@ __vpcs __f32x4_t _ZGVnN4v_cosf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_expf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_exp2f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_logf (__f32x4_t); +__vpcs __f32x4_t _ZGVnN4v_log10f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_log2f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_sinf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_tanf (__f32x4_t); @@ -61,6 +62,7 @@ __vpcs __f64x2_t _ZGVnN2v_cos (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp2 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log (__f64x2_t); +__vpcs __f64x2_t _ZGVnN2v_log10 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log2 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_sin (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_tan (__f64x2_t); @@ -74,6 +76,7 @@ __sv_f32_t _ZGVsMxv_cosf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_expf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_exp2f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_logf (__sv_f32_t, __sv_bool_t); +__sv_f32_t _ZGVsMxv_log10f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_log2f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_sinf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_tanf (__sv_f32_t, __sv_bool_t); @@ -82,6 +85,7 @@ __sv_f64_t _ZGVsMxv_cos (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp2 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log (__sv_f64_t, __sv_bool_t); +__sv_f64_t _ZGVsMxv_log10 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log2 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_sin (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_tan (__sv_f64_t, __sv_bool_t); diff --git a/sysdeps/aarch64/fpu/log10_advsimd.c b/sysdeps/aarch64/fpu/log10_advsimd.c new file mode 100644 index 0000000000..f518b55b60 --- /dev/null +++ b/sysdeps/aarch64/fpu/log10_advsimd.c @@ -0,0 +1,117 @@ +/* Double-precision vector (AdvSIMD) log10 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f64.h" + +#define N (1 << V_LOG10_TABLE_BITS) + +static const struct data +{ + float64x2_t poly[5]; + float64x2_t invln10, log10_2, ln2; + uint64x2_t min_norm, special_bound, sign_exp_mask; +} data = { + /* Computed from log coefficients divided by log(10) then rounded to double + precision. */ + .poly = { V2 (-0x1.bcb7b1526e506p-3), V2 (0x1.287a7636be1d1p-3), + V2 (-0x1.bcb7b158af938p-4), V2 (0x1.63c78734e6d07p-4), + V2 (-0x1.287461742fee4p-4) }, + .ln2 = V2 (0x1.62e42fefa39efp-1), + .invln10 = V2 (0x1.bcb7b1526e50ep-2), + .log10_2 = V2 (0x1.34413509f79ffp-2), + .min_norm = V2 (0x0010000000000000), /* asuint64(0x1p-1022). */ + .special_bound = V2 (0x7fe0000000000000), /* asuint64(inf) - min_norm. */ + .sign_exp_mask = V2 (0xfff0000000000000), +}; + +#define Off v_u64 (0x3fe6900900000000) +#define IndexMask v_u64 (N - 1) + +#define T(s, i) __v_log10_data.s[i] + +struct entry +{ + float64x2_t invc; + float64x2_t log10c; +}; + +static inline struct entry +lookup (uint64x2_t i) +{ + struct entry e; + e.invc[0] = T (invc, i[0]); + e.log10c[0] = T (log10c, i[0]); + e.invc[1] = T (invc, i[1]); + e.log10c[1] = T (log10c, i[1]); + return e; +} + +static float64x2_t VPCS_ATTR NOINLINE +special_case (float64x2_t x, float64x2_t y, uint64x2_t special) +{ + return v_call_f64 (log10, x, y, special); +} + +/* Fast implementation of double-precision vector log10 + is a slight modification of double-precision vector log. + Max ULP error: < 2.5 ulp (nearest rounding.) + Maximum measured at 2.46 ulp for x in [0.96, 0.97] + _ZGVnN2v_log10(0x1.13192407fcb46p+0) got 0x1.fff6be3cae4bbp-6 + want 0x1.fff6be3cae4b9p-6. */ +float64x2_t VPCS_ATTR V_NAME_D1 (log10) (float64x2_t x) +{ + const struct data *d = ptr_barrier (&data); + uint64x2_t ix = vreinterpretq_u64_f64 (x); + uint64x2_t special + = vcgeq_u64 (vsubq_u64 (ix, d->min_norm), d->special_bound); + + /* x = 2^k z; where z is in range [OFF,2*OFF) and exact. + The range is split into N subintervals. + The ith subinterval contains z and c is near its center. */ + uint64x2_t tmp = vsubq_u64 (ix, Off); + uint64x2_t i + = vandq_u64 (vshrq_n_u64 (tmp, 52 - V_LOG10_TABLE_BITS), IndexMask); + int64x2_t k = vshrq_n_s64 (vreinterpretq_s64_u64 (tmp), 52); + uint64x2_t iz = vsubq_u64 (ix, vandq_u64 (tmp, d->sign_exp_mask)); + float64x2_t z = vreinterpretq_f64_u64 (iz); + + struct entry e = lookup (i); + + /* log10(x) = log1p(z/c-1)/log(10) + log10(c) + k*log10(2). */ + float64x2_t r = vfmaq_f64 (v_f64 (-1.0), z, e.invc); + float64x2_t kd = vcvtq_f64_s64 (k); + + /* hi = r / log(10) + log10(c) + k*log10(2). + Constants in v_log10_data.c are computed (in extended precision) as + e.log10c := e.logc * ivln10. */ + float64x2_t w = vfmaq_f64 (e.log10c, r, d->invln10); + + /* y = log10(1+r) + n * log10(2). */ + float64x2_t hi = vfmaq_f64 (w, kd, d->log10_2); + + /* y = r2*(A0 + r*A1 + r2*(A2 + r*A3 + r2*A4)) + hi. */ + float64x2_t r2 = vmulq_f64 (r, r); + float64x2_t y = v_pw_horner_4_f64 (r, r2, d->poly); + y = vfmaq_f64 (hi, r2, y); + + if (__glibc_unlikely (v_any_u64 (special))) + return special_case (x, y, special); + return y; +} diff --git a/sysdeps/aarch64/fpu/log10_sve.c b/sysdeps/aarch64/fpu/log10_sve.c new file mode 100644 index 0000000000..4522879f99 --- /dev/null +++ b/sysdeps/aarch64/fpu/log10_sve.c @@ -0,0 +1,77 @@ +/* Double-precision vector (SVE) log10 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f64.h" + +#define Min 0x0010000000000000 +#define Max 0x7ff0000000000000 +#define Thres 0x7fe0000000000000 /* Max - Min. */ +#define Off 0x3fe6900900000000 +#define N (1 << V_LOG10_TABLE_BITS) + +#define T(s, i) __v_log10_data.s[i] + +static svfloat64_t NOINLINE +special_case (svfloat64_t x, svfloat64_t y, svbool_t special) +{ + return sv_call_f64 (log10, x, y, special); +} + +/* SVE log10 algorithm. + Maximum measured error is 2.46 ulps. + SV_NAME_D1 (log10)(0x1.131956cd4b627p+0) got 0x1.fffbdf6eaa669p-6 + want 0x1.fffbdf6eaa667p-6. */ +svfloat64_t SV_NAME_D1 (log10) (svfloat64_t x, const svbool_t pg) +{ + svuint64_t ix = svreinterpret_u64 (x); + svbool_t special = svcmpge (pg, svsub_x (pg, ix, Min), Thres); + + /* x = 2^k z; where z is in range [Off,2*Off) and exact. + The range is split into N subintervals. + The ith subinterval contains z and c is near its center. */ + svuint64_t tmp = svsub_x (pg, ix, Off); + svuint64_t i + = sv_mod_n_u64_x (pg, svlsr_x (pg, tmp, 52 - V_LOG10_TABLE_BITS), N); + svfloat64_t k = svcvt_f64_x (pg, svasr_x (pg, svreinterpret_s64 (tmp), 52)); + svfloat64_t z = svreinterpret_f64 ( + svsub_x (pg, ix, svand_x (pg, tmp, 0xfffULL << 52))); + + /* log(x) = k*log(2) + log(c) + log(z/c). */ + svfloat64_t invc = svld1_gather_index (pg, &T (invc, 0), i); + svfloat64_t logc = svld1_gather_index (pg, &T (log10c, 0), i); + + /* We approximate log(z/c) with a polynomial P(x) ~= log(x + 1): + r = z/c - 1 (we look up precomputed 1/c) + log(z/c) ~= P(r). */ + svfloat64_t r = svmad_x (pg, invc, z, -1.0); + + /* hi = log(c) + k*log(2). */ + svfloat64_t w = svmla_x (pg, logc, r, __v_log10_data.invln10); + svfloat64_t hi = svmla_x (pg, w, k, __v_log10_data.log10_2); + + /* y = r2*(A0 + r*A1 + r2*(A2 + r*A3 + r2*A4)) + hi. */ + svfloat64_t r2 = svmul_x (pg, r, r); + svfloat64_t y = sv_pw_horner_4_f64_x (pg, r, r2, __v_log10_data.poly); + y = svmla_x (pg, hi, r2, y); + + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x, y, special); + return y; +} diff --git a/sysdeps/aarch64/fpu/log10f_advsimd.c b/sysdeps/aarch64/fpu/log10f_advsimd.c new file mode 100644 index 0000000000..371495b541 --- /dev/null +++ b/sysdeps/aarch64/fpu/log10f_advsimd.c @@ -0,0 +1,79 @@ +/* Single-precision vector (AdvSIMD) log10 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f32.h" + +static const struct data +{ + float32x4_t poly[8]; + float32x4_t inv_ln10, ln2; + uint32x4_t min_norm, special_bound, off, mantissa_mask; +} data = { + /* Use order 9 for log10(1+x), i.e. order 8 for log10(1+x)/x, with x in + [-1/3, 1/3] (offset=2/3). Max. relative error: 0x1.068ee468p-25. */ + .poly = { V4 (-0x1.bcb79cp-3f), V4 (0x1.2879c8p-3f), V4 (-0x1.bcd472p-4f), + V4 (0x1.6408f8p-4f), V4 (-0x1.246f8p-4f), V4 (0x1.f0e514p-5f), + V4 (-0x1.0fc92cp-4f), V4 (0x1.f5f76ap-5f) }, + .ln2 = V4 (0x1.62e43p-1f), + .inv_ln10 = V4 (0x1.bcb7b2p-2f), + .min_norm = V4 (0x00800000), + .special_bound = V4 (0x7f000000), /* asuint32(inf) - min_norm. */ + .off = V4 (0x3f2aaaab), /* 0.666667. */ + .mantissa_mask = V4 (0x007fffff), +}; + +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t x, float32x4_t y, uint32x4_t cmp) +{ + /* Fall back to scalar code. */ + return v_call_f32 (log10f, x, y, cmp); +} + +/* Fast implementation of AdvSIMD log10f, + uses a similar approach as AdvSIMD logf with the same offset (i.e., 2/3) and + an order 9 polynomial. + Maximum error: 3.305ulps (nearest rounding.) + _ZGVnN4v_log10f(0x1.555c16p+0) got 0x1.ffe2fap-4 + want 0x1.ffe2f4p-4. */ +float32x4_t VPCS_ATTR V_NAME_F1 (log10) (float32x4_t x) +{ + const struct data *d = ptr_barrier (&data); + uint32x4_t u = vreinterpretq_u32_f32 (x); + uint32x4_t special + = vcgeq_u32 (vsubq_u32 (u, d->min_norm), d->special_bound); + + /* x = 2^n * (1+r), where 2/3 < 1+r < 4/3. */ + u = vsubq_u32 (u, d->off); + float32x4_t n = vcvtq_f32_s32 ( + vshrq_n_s32 (vreinterpretq_s32_u32 (u), 23)); /* signextend. */ + u = vaddq_u32 (vandq_u32 (u, d->mantissa_mask), d->off); + float32x4_t r = vsubq_f32 (vreinterpretq_f32_u32 (u), v_f32 (1.0f)); + + /* y = log10(1+r) + n * log10(2). */ + float32x4_t r2 = vmulq_f32 (r, r); + float32x4_t poly = v_pw_horner_7_f32 (r, r2, d->poly); + /* y = Log10(2) * n + poly * InvLn(10). */ + float32x4_t y = vfmaq_f32 (r, d->ln2, n); + y = vfmaq_f32 (vmulq_f32 (y, d->inv_ln10), poly, r2); + + if (__glibc_unlikely (v_any_u32 (special))) + return special_case (x, y, special); + return y; +} diff --git a/sysdeps/aarch64/fpu/log10f_sve.c b/sysdeps/aarch64/fpu/log10f_sve.c new file mode 100644 index 0000000000..22c63bb2fa --- /dev/null +++ b/sysdeps/aarch64/fpu/log10f_sve.c @@ -0,0 +1,94 @@ +/* Single-precision vector (SVE) log10 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" + +static const struct data +{ + float poly_0246[4]; + float poly_1357[4]; + float ln2, inv_ln10; +} data = { + .poly_1357 = { + /* Coefficients copied from the AdvSIMD routine, then rearranged so that coeffs + 1, 3, 5 and 7 can be loaded as a single quad-word, hence used with _lane + variant of MLA intrinsic. */ + 0x1.2879c8p-3f, 0x1.6408f8p-4f, 0x1.f0e514p-5f, 0x1.f5f76ap-5f + }, + .poly_0246 = { -0x1.bcb79cp-3f, -0x1.bcd472p-4f, -0x1.246f8p-4f, + -0x1.0fc92cp-4f }, + .ln2 = 0x1.62e43p-1f, + .inv_ln10 = 0x1.bcb7b2p-2f, +}; + +#define Min 0x00800000 +#define Max 0x7f800000 +#define Thres 0x7f000000 /* Max - Min. */ +#define Offset 0x3f2aaaab /* 0.666667. */ +#define MantissaMask 0x007fffff + +static svfloat32_t NOINLINE +special_case (svfloat32_t x, svfloat32_t y, svbool_t special) +{ + return sv_call_f32 (log10f, x, y, special); +} + +/* Optimised implementation of SVE log10f using the same algorithm and + polynomial as AdvSIMD log10f. + Maximum error is 3.31ulps: + SV_NAME_F1 (log10)(0x1.555c16p+0) got 0x1.ffe2fap-4 + want 0x1.ffe2f4p-4. */ +svfloat32_t SV_NAME_F1 (log10) (svfloat32_t x, const svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + svuint32_t ix = svreinterpret_u32 (x); + svbool_t special = svcmpge (pg, svsub_x (pg, ix, Min), Thres); + + /* x = 2^n * (1+r), where 2/3 < 1+r < 4/3. */ + ix = svsub_x (pg, ix, Offset); + svfloat32_t n = svcvt_f32_x ( + pg, svasr_x (pg, svreinterpret_s32 (ix), 23)); /* signextend. */ + ix = svand_x (pg, ix, MantissaMask); + ix = svadd_x (pg, ix, Offset); + svfloat32_t r = svsub_x (pg, svreinterpret_f32 (ix), 1.0f); + + /* y = log10(1+r) + n*log10(2) + log10(1+r) ~ r * InvLn(10) + P(r) + where P(r) is a polynomial. Use order 9 for log10(1+x), i.e. order 8 for + log10(1+x)/x, with x in [-1/3, 1/3] (offset=2/3). */ + svfloat32_t r2 = svmul_x (pg, r, r); + svfloat32_t r4 = svmul_x (pg, r2, r2); + svfloat32_t p_1357 = svld1rq (pg, &d->poly_1357[0]); + svfloat32_t q_01 = svmla_lane (sv_f32 (d->poly_0246[0]), r, p_1357, 0); + svfloat32_t q_23 = svmla_lane (sv_f32 (d->poly_0246[1]), r, p_1357, 1); + svfloat32_t q_45 = svmla_lane (sv_f32 (d->poly_0246[2]), r, p_1357, 2); + svfloat32_t q_67 = svmla_lane (sv_f32 (d->poly_0246[3]), r, p_1357, 3); + svfloat32_t q_47 = svmla_x (pg, q_45, r2, q_67); + svfloat32_t q_03 = svmla_x (pg, q_01, r2, q_23); + svfloat32_t y = svmla_x (pg, q_03, r4, q_47); + + /* Using hi = Log10(2)*n + r*InvLn(10) is faster but less accurate. */ + svfloat32_t hi = svmla_x (pg, r, n, d->ln2); + hi = svmul_x (pg, hi, d->inv_ln10); + y = svmla_x (pg, hi, r2, y); + + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x, y, special); + return y; +} diff --git a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c index d30dcd6f95..8d05498ec9 100644 --- a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c @@ -27,6 +27,7 @@ VPCS_VECTOR_WRAPPER (cos_advsimd, _ZGVnN2v_cos) VPCS_VECTOR_WRAPPER (exp_advsimd, _ZGVnN2v_exp) VPCS_VECTOR_WRAPPER (exp2_advsimd, _ZGVnN2v_exp2) VPCS_VECTOR_WRAPPER (log_advsimd, _ZGVnN2v_log) +VPCS_VECTOR_WRAPPER (log10_advsimd, _ZGVnN2v_log10) VPCS_VECTOR_WRAPPER (log2_advsimd, _ZGVnN2v_log2) VPCS_VECTOR_WRAPPER (sin_advsimd, _ZGVnN2v_sin) VPCS_VECTOR_WRAPPER (tan_advsimd, _ZGVnN2v_tan) diff --git a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c index 22a8479100..b65bc6f1e6 100644 --- a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c @@ -36,6 +36,7 @@ SVE_VECTOR_WRAPPER (cos_sve, _ZGVsMxv_cos) SVE_VECTOR_WRAPPER (exp_sve, _ZGVsMxv_exp) SVE_VECTOR_WRAPPER (exp2_sve, _ZGVsMxv_exp2) SVE_VECTOR_WRAPPER (log_sve, _ZGVsMxv_log) +SVE_VECTOR_WRAPPER (log10_sve, _ZGVsMxv_log10) SVE_VECTOR_WRAPPER (log2_sve, _ZGVsMxv_log2) SVE_VECTOR_WRAPPER (sin_sve, _ZGVsMxv_sin) SVE_VECTOR_WRAPPER (tan_sve, _ZGVsMxv_tan) diff --git a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c index e8f7f47c67..6ced0d4488 100644 --- a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c @@ -27,6 +27,7 @@ VPCS_VECTOR_WRAPPER (cosf_advsimd, _ZGVnN4v_cosf) VPCS_VECTOR_WRAPPER (expf_advsimd, _ZGVnN4v_expf) VPCS_VECTOR_WRAPPER (exp2f_advsimd, _ZGVnN4v_exp2f) VPCS_VECTOR_WRAPPER (logf_advsimd, _ZGVnN4v_logf) +VPCS_VECTOR_WRAPPER (log10f_advsimd, _ZGVnN4v_log10f) VPCS_VECTOR_WRAPPER (log2f_advsimd, _ZGVnN4v_log2f) VPCS_VECTOR_WRAPPER (sinf_advsimd, _ZGVnN4v_sinf) VPCS_VECTOR_WRAPPER (tanf_advsimd, _ZGVnN4v_tanf) diff --git a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c index f5e9584265..2ed8d0659a 100644 --- a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c @@ -36,6 +36,7 @@ SVE_VECTOR_WRAPPER (cosf_sve, _ZGVsMxv_cosf) SVE_VECTOR_WRAPPER (expf_sve, _ZGVsMxv_expf) SVE_VECTOR_WRAPPER (exp2f_sve, _ZGVsMxv_exp2f) SVE_VECTOR_WRAPPER (logf_sve, _ZGVsMxv_logf) +SVE_VECTOR_WRAPPER (log10f_sve, _ZGVsMxv_log10f) SVE_VECTOR_WRAPPER (log2f_sve, _ZGVsMxv_log2f) SVE_VECTOR_WRAPPER (sinf_sve, _ZGVsMxv_sinf) SVE_VECTOR_WRAPPER (tanf_sve, _ZGVsMxv_tanf) diff --git a/sysdeps/aarch64/fpu/v_log10_data.c b/sysdeps/aarch64/fpu/v_log10_data.c new file mode 100644 index 0000000000..b0b07bafe7 --- /dev/null +++ b/sysdeps/aarch64/fpu/v_log10_data.c @@ -0,0 +1,175 @@ +/* Lookup table for double-precision log10(x) vector function. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "vecmath_config.h" + +const struct v_log10_data __v_log10_data = { + /* Computed from log's coefficients div by log(10) then rounded to double + precision. */ + .poly = { -0x1.bcb7b1526e506p-3, 0x1.287a7636be1d1p-3, -0x1.bcb7b158af938p-4, + 0x1.63c78734e6d07p-4, -0x1.287461742fee4p-4 }, + .invln10 = 0x1.bcb7b1526e50ep-2, + .log10_2 = 0x1.34413509f79ffp-2, + /* Algorithm: + + x = 2^k z + log10(x) = k log10(2) + log10(c) + poly(z/c - 1) / log(10) + + where z is in [a;2a) which is split into N subintervals (a=0x1.69009p-1, + N=128) and log(c) and 1/c for the ith subinterval comes from lookup + tables: + + invc[i] = 1/c + log10c[i] = (double)log10(c) + + where c is near the center of the subinterval and is chosen by trying + several floating point invc candidates around 1/center and selecting one + for which the error in (double)log(c) is minimized (< 0x1p-74), except the + subinterval that contains 1 and the previous one got tweaked to avoid + cancellation. NB: invc should be optimized to minimize error in + (double)log10(c) instead. */ + .invc = { 0x1.6a133d0dec120p+0, 0x1.6815f2f3e42edp+0, + 0x1.661e39be1ac9ep+0, 0x1.642bfa30ac371p+0, + 0x1.623f1d916f323p+0, 0x1.60578da220f65p+0, + 0x1.5e75349dea571p+0, 0x1.5c97fd387a75ap+0, + 0x1.5abfd2981f200p+0, 0x1.58eca051dc99cp+0, + 0x1.571e526d9df12p+0, 0x1.5554d555b3fcbp+0, + 0x1.539015e2a20cdp+0, 0x1.51d0014ee0164p+0, + 0x1.50148538cd9eep+0, 0x1.4e5d8f9f698a1p+0, + 0x1.4cab0edca66bep+0, 0x1.4afcf1a9db874p+0, + 0x1.495327136e16fp+0, 0x1.47ad9e84af28fp+0, + 0x1.460c47b39ae15p+0, 0x1.446f12b278001p+0, + 0x1.42d5efdd720ecp+0, 0x1.4140cfe001a0fp+0, + 0x1.3fafa3b421f69p+0, 0x1.3e225c9c8ece5p+0, + 0x1.3c98ec29a211ap+0, 0x1.3b13442a413fep+0, + 0x1.399156baa3c54p+0, 0x1.38131639b4cdbp+0, + 0x1.36987540fbf53p+0, 0x1.352166b648f61p+0, + 0x1.33adddb3eb575p+0, 0x1.323dcd99fc1d3p+0, + 0x1.30d129fefc7d2p+0, 0x1.2f67e6b72fe7dp+0, + 0x1.2e01f7cf8b187p+0, 0x1.2c9f518ddc86ep+0, + 0x1.2b3fe86e5f413p+0, 0x1.29e3b1211b25cp+0, + 0x1.288aa08b373cfp+0, 0x1.2734abcaa8467p+0, + 0x1.25e1c82459b81p+0, 0x1.2491eb1ad59c5p+0, + 0x1.23450a54048b5p+0, 0x1.21fb1bb09e578p+0, + 0x1.20b415346d8f7p+0, 0x1.1f6fed179a1acp+0, + 0x1.1e2e99b93c7b3p+0, 0x1.1cf011a7a882ap+0, + 0x1.1bb44b97dba5ap+0, 0x1.1a7b3e66cdd4fp+0, + 0x1.1944e11dc56cdp+0, 0x1.18112aebb1a6ep+0, + 0x1.16e013231b7e9p+0, 0x1.15b1913f156cfp+0, + 0x1.14859cdedde13p+0, 0x1.135c2dc68cfa4p+0, + 0x1.12353bdb01684p+0, 0x1.1110bf25b85b4p+0, + 0x1.0feeafd2f8577p+0, 0x1.0ecf062c51c3bp+0, + 0x1.0db1baa076c8bp+0, 0x1.0c96c5bb3048ep+0, + 0x1.0b7e20263e070p+0, 0x1.0a67c2acd0ce3p+0, + 0x1.0953a6391e982p+0, 0x1.0841c3caea380p+0, + 0x1.07321489b13eap+0, 0x1.062491aee9904p+0, + 0x1.05193497a7cc5p+0, 0x1.040ff6b5f5e9fp+0, + 0x1.0308d19aa6127p+0, 0x1.0203beedb0c67p+0, + 0x1.010037d38bcc2p+0, 1.0, + 0x1.fc06d493cca10p-1, 0x1.f81e6ac3b918fp-1, + 0x1.f44546ef18996p-1, 0x1.f07b10382c84bp-1, + 0x1.ecbf7070e59d4p-1, 0x1.e91213f715939p-1, + 0x1.e572a9a75f7b7p-1, 0x1.e1e0e2c530207p-1, + 0x1.de5c72d8a8be3p-1, 0x1.dae50fa5658ccp-1, + 0x1.d77a71145a2dap-1, 0x1.d41c51166623ep-1, + 0x1.d0ca6ba0bb29fp-1, 0x1.cd847e8e59681p-1, + 0x1.ca4a499693e00p-1, 0x1.c71b8e399e821p-1, + 0x1.c3f80faf19077p-1, 0x1.c0df92dc2b0ecp-1, + 0x1.bdd1de3cbb542p-1, 0x1.baceb9e1007a3p-1, + 0x1.b7d5ef543e55ep-1, 0x1.b4e749977d953p-1, + 0x1.b20295155478ep-1, 0x1.af279f8e82be2p-1, + 0x1.ac5638197fdf3p-1, 0x1.a98e2f102e087p-1, + 0x1.a6cf5606d05c1p-1, 0x1.a4197fc04d746p-1, + 0x1.a16c80293dc01p-1, 0x1.9ec82c4dc5bc9p-1, + 0x1.9c2c5a491f534p-1, 0x1.9998e1480b618p-1, + 0x1.970d9977c6c2dp-1, 0x1.948a5c023d212p-1, + 0x1.920f0303d6809p-1, 0x1.8f9b698a98b45p-1, + 0x1.8d2f6b81726f6p-1, 0x1.8acae5bb55badp-1, + 0x1.886db5d9275b8p-1, 0x1.8617ba567c13cp-1, + 0x1.83c8d27487800p-1, 0x1.8180de3c5dbe7p-1, + 0x1.7f3fbe71cdb71p-1, 0x1.7d055498071c1p-1, + 0x1.7ad182e54f65ap-1, 0x1.78a42c3c90125p-1, + 0x1.767d342f76944p-1, 0x1.745c7ef26b00ap-1, + 0x1.7241f15769d0fp-1, 0x1.702d70d396e41p-1, + 0x1.6e1ee3700cd11p-1, 0x1.6c162fc9cbe02p-1 }, + .log10c = { -0x1.345825f221684p-3, -0x1.2f71a1f0c554ep-3, + -0x1.2a91fdb30b1f4p-3, -0x1.25b9260981a04p-3, + -0x1.20e7081762193p-3, -0x1.1c1b914aeefacp-3, + -0x1.1756af5de404dp-3, -0x1.12985059c90bfp-3, + -0x1.0de0628f63df4p-3, -0x1.092ed492e08eep-3, + -0x1.0483954caf1dfp-3, -0x1.ffbd27a9adbcp-4, + -0x1.f67f7f2e3d1ap-4, -0x1.ed4e1071ceebep-4, + -0x1.e428bb47413c4p-4, -0x1.db0f6003028d6p-4, + -0x1.d201df6749831p-4, -0x1.c9001ac5c9672p-4, + -0x1.c009f3c78c79p-4, -0x1.b71f4cb642e53p-4, + -0x1.ae400818526b2p-4, -0x1.a56c091954f87p-4, + -0x1.9ca3332f096eep-4, -0x1.93e56a3f23e55p-4, + -0x1.8b3292a3903bp-4, -0x1.828a9112d9618p-4, + -0x1.79ed4ac35f5acp-4, -0x1.715aa51ed28c4p-4, + -0x1.68d2861c999e9p-4, -0x1.6054d40ded21p-4, + -0x1.57e17576bc9a2p-4, -0x1.4f7851798bb0bp-4, + -0x1.47194f5690ae3p-4, -0x1.3ec456d58ec47p-4, + -0x1.36794ff3e5f55p-4, -0x1.2e382315725e4p-4, + -0x1.2600b8ed82e91p-4, -0x1.1dd2fa85efc12p-4, + -0x1.15aed136e3961p-4, -0x1.0d94269d1a30dp-4, + -0x1.0582e4a7659f5p-4, -0x1.faf5eb655742dp-5, + -0x1.eaf888487e8eep-5, -0x1.db0d75ef25a82p-5, + -0x1.cb348a49e6431p-5, -0x1.bb6d9c69acdd8p-5, + -0x1.abb88368aa7ap-5, -0x1.9c1517476af14p-5, + -0x1.8c833051bfa4dp-5, -0x1.7d02a78e7fb31p-5, + -0x1.6d93565e97c5fp-5, -0x1.5e351695db0c5p-5, + -0x1.4ee7c2ba67adcp-5, -0x1.3fab35ba16c01p-5, + -0x1.307f4ad854bc9p-5, -0x1.2163ddf4f988cp-5, + -0x1.1258cb5d19e22p-5, -0x1.035defdba3188p-5, + -0x1.e8e651191bce4p-6, -0x1.cb30a62be444cp-6, + -0x1.ad9a9b3043823p-6, -0x1.9023ecda1ccdep-6, + -0x1.72cc592bd82dp-6, -0x1.55939eb1f9c6ep-6, + -0x1.38797ca6cc5ap-6, -0x1.1b7db35c2c072p-6, + -0x1.fd400812ee9a2p-7, -0x1.c3c05fb4620f1p-7, + -0x1.8a7bf3c40e2e3p-7, -0x1.517249c15a75cp-7, + -0x1.18a2ea5330c91p-7, -0x1.c01abc8cdc4e2p-8, + -0x1.4f6261750dec9p-8, -0x1.be37b6612afa7p-9, + -0x1.bc3a8398ac26p-10, 0.0, + 0x1.bb796219f30a5p-9, 0x1.b984fdcba61cep-8, + 0x1.49cf12adf8e8cp-7, 0x1.b6075b5217083p-7, + 0x1.10b7466fc30ddp-6, 0x1.4603e4db6a3a1p-6, + 0x1.7aeb10e99e105p-6, 0x1.af6e49b0f0e36p-6, + 0x1.e38f064f41179p-6, 0x1.0ba75abbb7623p-5, + 0x1.25575ee2dba86p-5, 0x1.3ed83f477f946p-5, + 0x1.582aa79af60efp-5, 0x1.714f400fa83aep-5, + 0x1.8a46ad3901cb9p-5, 0x1.a311903b6b87p-5, + 0x1.bbb086f216911p-5, 0x1.d4242bdda648ep-5, + 0x1.ec6d167c2af1p-5, 0x1.0245ed8221426p-4, + 0x1.0e40856c74f64p-4, 0x1.1a269a31120fep-4, + 0x1.25f8718fc076cp-4, 0x1.31b64ffc95bfp-4, + 0x1.3d60787ca5063p-4, 0x1.48f72ccd187fdp-4, + 0x1.547aad6602f1cp-4, 0x1.5feb3989d3acbp-4, + 0x1.6b490f3978c79p-4, 0x1.76946b3f5e703p-4, + 0x1.81cd895717c83p-4, 0x1.8cf4a4055c30ep-4, + 0x1.9809f4c48c0ebp-4, 0x1.a30db3f9899efp-4, + 0x1.ae001905458fcp-4, 0x1.b8e15a2e3a2cdp-4, + 0x1.c3b1ace2b0996p-4, 0x1.ce71456edfa62p-4, + 0x1.d9205759882c4p-4, 0x1.e3bf1513af0dfp-4, + 0x1.ee4db0412c414p-4, 0x1.f8cc5998de3a5p-4, + 0x1.019da085eaeb1p-3, 0x1.06cd4acdb4e3dp-3, + 0x1.0bf542bef813fp-3, 0x1.11159f14da262p-3, + 0x1.162e761c10d1cp-3, 0x1.1b3fddc60d43ep-3, + 0x1.2049ebac86aa6p-3, 0x1.254cb4fb7836ap-3, + 0x1.2a484e8d0d252p-3, 0x1.2f3ccce1c860bp-3 } +}; diff --git a/sysdeps/aarch64/fpu/vecmath_config.h b/sysdeps/aarch64/fpu/vecmath_config.h index 82a68fd35d..4eb7b587b3 100644 --- a/sysdeps/aarch64/fpu/vecmath_config.h +++ b/sysdeps/aarch64/fpu/vecmath_config.h @@ -58,4 +58,13 @@ extern const struct v_log2_data double log2c[1 << V_LOG2_TABLE_BITS]; } __v_log2_data attribute_hidden; +#define V_LOG10_TABLE_BITS 7 +extern const struct v_log10_data +{ + double poly[5]; + double invln10, log10_2; + double invc[1 << V_LOG10_TABLE_BITS]; + double log10c[1 << V_LOG10_TABLE_BITS]; +} __v_log10_data attribute_hidden; + #endif diff --git a/sysdeps/aarch64/libm-test-ulps b/sysdeps/aarch64/libm-test-ulps index 7a5af571e2..6641c7fa0b 100644 --- a/sysdeps/aarch64/libm-test-ulps +++ b/sysdeps/aarch64/libm-test-ulps @@ -1180,11 +1180,19 @@ double: 2 float: 2 ldouble: 2 +Function: "log10_advsimd": +double: 1 +float: 2 + Function: "log10_downward": double: 2 float: 3 ldouble: 1 +Function: "log10_sve": +double: 1 +float: 2 + Function: "log10_towardzero": double: 2 float: 2 diff --git a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist index 657edab7ae..f8776a6bea 100644 --- a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist +++ b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist @@ -15,13 +15,17 @@ GLIBC_2.38 _ZGVsMxv_logf F GLIBC_2.38 _ZGVsMxv_sin F GLIBC_2.38 _ZGVsMxv_sinf F GLIBC_2.39 _ZGVnN2v_exp2 F +GLIBC_2.39 _ZGVnN2v_log10 F GLIBC_2.39 _ZGVnN2v_log2 F GLIBC_2.39 _ZGVnN2v_tan F GLIBC_2.39 _ZGVnN4v_exp2f F +GLIBC_2.39 _ZGVnN4v_log10f F GLIBC_2.39 _ZGVnN4v_log2f F GLIBC_2.39 _ZGVnN4v_tanf F GLIBC_2.39 _ZGVsMxv_exp2 F GLIBC_2.39 _ZGVsMxv_exp2f F +GLIBC_2.39 _ZGVsMxv_log10 F +GLIBC_2.39 _ZGVsMxv_log10f F GLIBC_2.39 _ZGVsMxv_log2 F GLIBC_2.39 _ZGVsMxv_log2f F GLIBC_2.39 _ZGVsMxv_tan F From patchwork Tue Aug 8 14:07:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Ramsay X-Patchwork-Id: 1818623 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=bWNBnkqZ; dkim-atps=neutral Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RKw5C04tjz1yYC for ; Wed, 9 Aug 2023 00:08:14 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id E5059385772D for ; Tue, 8 Aug 2023 14:08:12 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org E5059385772D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1691503692; bh=OlcuKflGoYuLpSH19lRq5V0aiYylhsgYHtvjLNhjJ+w=; h=To:CC:Subject:Date:In-Reply-To:References:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=bWNBnkqZ5Ou5qqMozXfPWVvLmbjxtOTmDm9YGoAoi4dkb/azd9u/gROMSi5m9mqCD nSi0XUeb030recj/o5qMwbANYvPfyaqb8khJhdJMd9b1dXQTPdRkcCgrpfV3wtUrOX WH9UvhryLxHIIUooTNiuFFTG+Z3d+cWr/+VcoGfY= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2041.outbound.protection.outlook.com [40.107.21.41]) by sourceware.org (Postfix) with ESMTPS id 7D2C538582BC for ; Tue, 8 Aug 2023 14:07:29 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 7D2C538582BC Received: from AM0PR03CA0010.eurprd03.prod.outlook.com (2603:10a6:208:14::23) by DB9PR08MB7723.eurprd08.prod.outlook.com (2603:10a6:10:392::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.26; Tue, 8 Aug 2023 14:07:24 +0000 Received: from AM7EUR03FT044.eop-EUR03.prod.protection.outlook.com (2603:10a6:208:14:cafe::cb) by AM0PR03CA0010.outlook.office365.com (2603:10a6:208:14::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.27 via Frontend Transport; Tue, 8 Aug 2023 14:07:24 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM7EUR03FT044.mail.protection.outlook.com (100.127.140.169) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6678.17 via Frontend Transport; Tue, 8 Aug 2023 14:07:24 +0000 Received: ("Tessian outbound f9124736ff4f:v145"); Tue, 08 Aug 2023 14:07:24 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 4bb0718b5cfd9b11 X-CR-MTA-TID: 64aa7808 Received: from 014931aacbc7.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 60A16690-91F7-40C9-8145-453D89288C2C.1; Tue, 08 Aug 2023 14:07:18 +0000 Received: from EUR05-AM6-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 014931aacbc7.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 08 Aug 2023 14:07:18 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gSotQKCanqCf8KZVFIsS5ImIntizPfaocrg2t5zd6384s6PMHhjEDKMY4gNlfcwXclvRNOApW9Z775izcgivXG2XXPG/brL3Nf8F0zZIBeGusf37YFP3l32WRhoR+CpzavM/ktspB4tlTwf4Mrg/do8j2kro/paxFqLUgsT5DoOXrlGs127bQ+pG1gd8OQpJLJp3re82Kf2WiU04+NiHdKdxl8oVSDTWBNCMgknwXtlxZqSD/imlT84c+aCldkJbxsnX+iPKyaAEdmyfJb28xm9eQ1bG29OeTJP/hr5wuArwmShtZ53nJjG6VztY9/XS8U2pxtEV0uDc15ND64xF9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OlcuKflGoYuLpSH19lRq5V0aiYylhsgYHtvjLNhjJ+w=; b=NQ18Nkvk6Pgpf6wkZhPlh9dPSo60cGDUMnYI4QoUhtz4tKprY7UozCG24lIFK7mEQ+amoXZnRQar78qQNQ8zSINlghAQprOSpkONA3sS38TWKNhCM/fLBjwF7ovN7zxkmzU0pdHwJ0+Y4iuJHIT7EaD+KVLnSmOCUxDcCwyUWDTmvncdauoP3byjaLQUmAP7EE0Vr9lcsVRpJnDYQa0i+OK4naaq/IdeVdpVr9egKufDhMbGnwHQy2HJ+iUiaFkOJEcsrwqWiOJYmbxpLkpEjBaAB/GuzXMUh0omZJCZCrozX4wodym/jjJ28U5H7/CFRaTS3TcUnnA+oBu2GDAHMg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none Received: from AS4P189CA0035.EURP189.PROD.OUTLOOK.COM (2603:10a6:20b:5dd::14) by PAVPR08MB9506.eurprd08.prod.outlook.com (2603:10a6:102:314::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.25; Tue, 8 Aug 2023 14:07:15 +0000 Received: from AM7EUR03FT061.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:5dd:cafe::20) by AS4P189CA0035.outlook.office365.com (2603:10a6:20b:5dd::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.25 via Frontend Transport; Tue, 8 Aug 2023 14:07:15 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (40.67.248.234) by AM7EUR03FT061.mail.protection.outlook.com (100.127.140.72) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6678.17 via Frontend Transport; Tue, 8 Aug 2023 14:07:15 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX03.Arm.com (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Tue, 8 Aug 2023 14:07:11 +0000 Received: from vcn-man-apps.manchester.arm.com (10.32.108.22) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.27 via Frontend Transport; Tue, 8 Aug 2023 14:07:11 +0000 To: CC: Joe Ramsay Subject: [PATCH 5/5 v2] aarch64: Add vector implementations of exp10 routines Date: Tue, 8 Aug 2023 15:07:09 +0100 Message-ID: <20230808140709.47733-5-Joe.Ramsay@arm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230808140709.47733-1-Joe.Ramsay@arm.com> References: <20230808140709.47733-1-Joe.Ramsay@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: AM7EUR03FT061:EE_|PAVPR08MB9506:EE_|AM7EUR03FT044:EE_|DB9PR08MB7723:EE_ X-MS-Office365-Filtering-Correlation-Id: 19b74933-bef3-4b4d-1e2d-08db9818ca7a x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 9rB5FzGcadfqywpw15L5lA4Tr2lw+krbyC3pfpUGgQZ4KbFVAKx35nWItgk0fX4F79fkeqbO8lWfS4mLbbnc7zLtVt56yJkKrxyYmn4+QrI1cKOILKAwyV5fJXEeLCfRK9XWwpEmxxBJnaTukD2sH8jDImCAw675Q0qtBCQfAiIq6nVnFFIfcMDKW82I5L+fDZ88EEOt1C9TCzZNcoprUplOWB/2DrC4xpz4JKYXxf7Nk2/quTIXtRAtNeoCbqSoOyaTQx9DsnuNfrKG+PP+VLkOcuXll4O7MvpdN3dk0NRQevms2STax0LdVsE4UZX5yPANElUnPWhYWX/yEpFpNQM7XrcU105IJ/Hk5NP87w5hM4MyW5SsOtAAIMPo/SkqF4hBxTpWLI2emE8jm7VppLeIMovSktTAW7nw+ZmTK4ftPCixdiZSQve0mg8ald6cZqTEOgyLmibl1mTBwMBJPksdUEMcFuUxH7TkDBeH3RtFC5cj2nL18jMOxTeFhhp2s4Y9r1Wig84JpbWLyXjoxrskb/zCDDnV5A51YeiqeCYxdf1LKUA5io/MpeMhAzuHIEpQ6CINE13cM7dCZj+tEcJpmcT5fKFQyzqd36gzf/l5xsDnbKkrs8FU5kGwrd6dslISTVUcZg+E9ux7SlL2w9jHDX1XfyIdKLiysk9A6QF+o4UE5zi4VwvU9SvCaQjVI6tdS5QU0Ifr59B0RT7Z09yyk0HJ4jCDz9TmkHszJuleXcFy/6coCsvr89jFJbQL7//c/Y0u1yweJHskM5wxqw== X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(39860400002)(396003)(376002)(346002)(136003)(186006)(1800799003)(451199021)(82310400008)(46966006)(36840700001)(47076005)(83380400001)(2906002)(36756003)(426003)(30864003)(5660300002)(2616005)(36860700001)(7696005)(70206006)(70586007)(6916009)(356005)(4326008)(82740400003)(316002)(81166007)(40480700001)(41300700001)(86362001)(8676002)(26005)(8936002)(1076003)(336012)(478600001)(2004002)(36900700001)(473944003); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAVPR08MB9506 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM7EUR03FT044.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 6a81f02c-650f-4365-cbee-08db9818c4d9 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3ciwLARgBO8vElZLVxwGjWkwHMAYA62Zf3Z34iWujEY2aq/PZtAeAk0Bkxxo9J9ETD9Ml8IxVzTFt5gzTT1FuYamt/xB9l7zmdqYru4cI8lLiUzB0dbrHEOMFLdcSpPHCC6F2nYAuLKTbKRlbDP/l/+tlIdxEckp5fa2ltF4Haa8ErfwdnWJ/CYPtVmVQfc6k8Z2cIveqS8c6PO7e/Q/5qz6b0RHpNWE1VwTslRVAFAxLkT8h2M5QElf8WmwHJHjJkGJW2reJaza//kiYwecFu/8y0cZszEd6mxiHxTt2ZKkSZas8lE0QdP3qPLe/RkLtbZvx/V+pFB8Jl++RX0JcJD8wiF7/CnGtON/AJUvbSvwoMT51MheTOLlqEe2pFPMqDHBrVK9jlm8KUSug20MRVbAjFVaB1dJhr706Rn+S/lmAXQADhw7uynp7zhCZKUYXCdipwFbT0UW91xuMV3rLIP8tKZNJ62poaytuW8VGBihPMI1IoPiWYkpQdmp7sUi10w6ewlu3loErDwQVGhwU6Up8jV9I9BEAVDQ3yV8t9QhB88EOGlgnd9uHeRwFWym7dajxkDL4H+NY5CB9ZH6GmyoDaURQzMYhxDNBWvNOWc/NQ4b0I6ji9Ay/OEpIBEmjVVzzCpXqOCdtd2vBDw4sCqlC/PnTMjq2sn3XsFHmtSIE9bJNeYw48SHn2r3pOaXi3J4p66AvBHHXlLxfwvt3sqD4IOhgoghT/VFKel0E+DD56bluV8sSQ9TgEFR9MppLrIFEMkpV1gBjkfSE1nP1Q== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230028)(4636009)(346002)(396003)(376002)(136003)(39860400002)(451199021)(82310400008)(186006)(1800799003)(36840700001)(46966006)(40470700004)(82740400003)(478600001)(40480700001)(81166007)(426003)(2616005)(40460700003)(86362001)(1076003)(26005)(36756003)(7696005)(336012)(6916009)(47076005)(70206006)(70586007)(2906002)(4326008)(36860700001)(5660300002)(8676002)(8936002)(316002)(30864003)(41300700001)(83380400001)(2004002)(473944003); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Aug 2023 14:07:24.5913 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 19b74933-bef3-4b4d-1e2d-08db9818ca7a X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM7EUR03FT044.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB7723 X-Spam-Status: No, score=-12.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Joe Ramsay via Libc-alpha From: Joe Ramsay Reply-To: Joe Ramsay Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" Double-precision routines either reuse the exp table (AdvSIMD) or use SVE FEXPA intruction. --- Changes to v1: Fix check-abi Thanks, Joe sysdeps/aarch64/fpu/Makefile | 1 + sysdeps/aarch64/fpu/Versions | 4 + sysdeps/aarch64/fpu/bits/math-vector.h | 4 + sysdeps/aarch64/fpu/exp10_advsimd.c | 145 ++++++++++++++++++ sysdeps/aarch64/fpu/exp10_sve.c | 127 +++++++++++++++ sysdeps/aarch64/fpu/exp10f_advsimd.c | 140 +++++++++++++++++ sysdeps/aarch64/fpu/exp10f_sve.c | 91 +++++++++++ .../fpu/test-double-advsimd-wrappers.c | 1 + .../aarch64/fpu/test-double-sve-wrappers.c | 1 + .../aarch64/fpu/test-float-advsimd-wrappers.c | 1 + sysdeps/aarch64/fpu/test-float-sve-wrappers.c | 1 + sysdeps/aarch64/libm-test-ulps | 8 + .../unix/sysv/linux/aarch64/libmvec.abilist | 4 + 13 files changed, 528 insertions(+) create mode 100644 sysdeps/aarch64/fpu/exp10_advsimd.c create mode 100644 sysdeps/aarch64/fpu/exp10_sve.c create mode 100644 sysdeps/aarch64/fpu/exp10f_advsimd.c create mode 100644 sysdeps/aarch64/fpu/exp10f_sve.c diff --git a/sysdeps/aarch64/fpu/Makefile b/sysdeps/aarch64/fpu/Makefile index 0a047a150d..1f1ac2a2b8 100644 --- a/sysdeps/aarch64/fpu/Makefile +++ b/sysdeps/aarch64/fpu/Makefile @@ -1,5 +1,6 @@ libmvec-supported-funcs = cos \ exp \ + exp10 \ exp2 \ log \ log10 \ diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions index 358efed5ee..eb5ad50017 100644 --- a/sysdeps/aarch64/fpu/Versions +++ b/sysdeps/aarch64/fpu/Versions @@ -18,6 +18,10 @@ libmvec { _ZGVsMxv_sinf; } GLIBC_2.39 { + _ZGVnN4v_exp10f; + _ZGVnN2v_exp10; + _ZGVsMxv_exp10f; + _ZGVsMxv_exp10; _ZGVnN4v_exp2f; _ZGVnN2v_exp2; _ZGVsMxv_exp2f; diff --git a/sysdeps/aarch64/fpu/bits/math-vector.h b/sysdeps/aarch64/fpu/bits/math-vector.h index 59f2efa6d7..06587ffa91 100644 --- a/sysdeps/aarch64/fpu/bits/math-vector.h +++ b/sysdeps/aarch64/fpu/bits/math-vector.h @@ -51,6 +51,7 @@ typedef __SVBool_t __sv_bool_t; __vpcs __f32x4_t _ZGVnN4v_cosf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_expf (__f32x4_t); +__vpcs __f32x4_t _ZGVnN4v_exp10f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_exp2f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_logf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_log10f (__f32x4_t); @@ -60,6 +61,7 @@ __vpcs __f32x4_t _ZGVnN4v_tanf (__f32x4_t); __vpcs __f64x2_t _ZGVnN2v_cos (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp (__f64x2_t); +__vpcs __f64x2_t _ZGVnN2v_exp10 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp2 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log10 (__f64x2_t); @@ -74,6 +76,7 @@ __vpcs __f64x2_t _ZGVnN2v_tan (__f64x2_t); __sv_f32_t _ZGVsMxv_cosf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_expf (__sv_f32_t, __sv_bool_t); +__sv_f32_t _ZGVsMxv_exp10f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_exp2f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_logf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_log10f (__sv_f32_t, __sv_bool_t); @@ -83,6 +86,7 @@ __sv_f32_t _ZGVsMxv_tanf (__sv_f32_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_cos (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp (__sv_f64_t, __sv_bool_t); +__sv_f64_t _ZGVsMxv_exp10 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp2 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log10 (__sv_f64_t, __sv_bool_t); diff --git a/sysdeps/aarch64/fpu/exp10_advsimd.c b/sysdeps/aarch64/fpu/exp10_advsimd.c new file mode 100644 index 0000000000..c584e2a270 --- /dev/null +++ b/sysdeps/aarch64/fpu/exp10_advsimd.c @@ -0,0 +1,145 @@ +/* Double-precision vector (AdvSIMD) exp10 function. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" + +/* Value of |x| above which scale overflows without special treatment. */ +#define SpecialBound 306.0 /* floor (log10 (2^1023)) - 1. */ +/* Value of n above which scale overflows even with special treatment. */ +#define ScaleBound 163840.0 /* 1280.0 * N. */ + +const static struct data +{ + float64x2_t poly[4]; + float64x2_t log10_2, log2_10_hi, log2_10_lo, shift; +#if !WANT_SIMD_EXCEPT + float64x2_t special_bound, scale_thresh; +#endif +} data = { + /* Coefficients generated using Remez algorithm. + rel error: 0x1.5ddf8f28p-54 + abs error: 0x1.5ed266c8p-54 in [ -log10(2)/256, log10(2)/256 ] + maxerr: 1.14432 +0.5 ulp. */ + .poly = { V2 (0x1.26bb1bbb5524p1), V2 (0x1.53524c73cecdap1), + V2 (0x1.047060efb781cp1), V2 (0x1.2bd76040f0d16p0) }, + .log10_2 = V2 (0x1.a934f0979a371p8), /* N/log2(10). */ + .log2_10_hi = V2 (0x1.34413509f79ffp-9), /* log2(10)/N. */ + .log2_10_lo = V2 (-0x1.9dc1da994fd21p-66), + .shift = V2 (0x1.8p+52), +#if !WANT_SIMD_EXCEPT + .scale_thresh = V2 (ScaleBound), + .special_bound = V2 (SpecialBound), +#endif +}; + +#define N (1 << V_EXP_TABLE_BITS) +#define IndexMask v_u64 (N - 1) + +#if WANT_SIMD_EXCEPT + +# define TinyBound v_u64 (0x2000000000000000) /* asuint64 (0x1p-511). */ +# define BigBound v_u64 (0x4070000000000000) /* asuint64 (0x1p8). */ +# define Thres v_u64 (0x2070000000000000) /* BigBound - TinyBound. */ + +static inline float64x2_t VPCS_ATTR +special_case (float64x2_t x, float64x2_t y, uint64x2_t cmp) +{ + /* If fenv exceptions are to be triggered correctly, fall back to the scalar + routine for special lanes. */ + return v_call_f64 (exp10, x, y, cmp); +} + +#else + +# define SpecialOffset v_u64 (0x6000000000000000) /* 0x1p513. */ +/* SpecialBias1 + SpecialBias1 = asuint(1.0). */ +# define SpecialBias1 v_u64 (0x7000000000000000) /* 0x1p769. */ +# define SpecialBias2 v_u64 (0x3010000000000000) /* 0x1p-254. */ + +static float64x2_t VPCS_ATTR NOINLINE +special_case (float64x2_t s, float64x2_t y, float64x2_t n, + const struct data *d) +{ + /* 2^(n/N) may overflow, break it up into s1*s2. */ + uint64x2_t b = vandq_u64 (vcltzq_f64 (n), SpecialOffset); + float64x2_t s1 = vreinterpretq_f64_u64 (vsubq_u64 (SpecialBias1, b)); + float64x2_t s2 = vreinterpretq_f64_u64 ( + vaddq_u64 (vsubq_u64 (vreinterpretq_u64_f64 (s), SpecialBias2), b)); + uint64x2_t cmp = vcagtq_f64 (n, d->scale_thresh); + float64x2_t r1 = vmulq_f64 (s1, s1); + float64x2_t r0 = vmulq_f64 (vfmaq_f64 (s2, y, s2), s1); + return vbslq_f64 (cmp, r1, r0); +} + +#endif + +/* Fast vector implementation of exp10. + Maximum measured error is 1.64 ulp. + _ZGVnN2v_exp10(0x1.ccd1c9d82cc8cp+0) got 0x1.f8dab6d7fed0cp+5 + want 0x1.f8dab6d7fed0ap+5. */ +float64x2_t VPCS_ATTR V_NAME_D1 (exp10) (float64x2_t x) +{ + const struct data *d = ptr_barrier (&data); + uint64x2_t cmp; +#if WANT_SIMD_EXCEPT + /* If any lanes are special, mask them with 1 and retain a copy of x to allow + special_case to fix special lanes later. This is only necessary if fenv + exceptions are to be triggered correctly. */ + float64x2_t xm = x; + uint64x2_t iax = vreinterpretq_u64_f64 (vabsq_f64 (x)); + cmp = vcgeq_u64 (vsubq_u64 (iax, TinyBound), Thres); + if (__glibc_unlikely (v_any_u64 (cmp))) + x = vbslq_f64 (cmp, v_f64 (1), x); +#else + cmp = vcageq_f64 (x, d->special_bound); +#endif + + /* n = round(x/(log10(2)/N)). */ + float64x2_t z = vfmaq_f64 (d->shift, x, d->log10_2); + uint64x2_t u = vreinterpretq_u64_f64 (z); + float64x2_t n = vsubq_f64 (z, d->shift); + + /* r = x - n*log10(2)/N. */ + float64x2_t r = x; + r = vfmsq_f64 (r, d->log2_10_hi, n); + r = vfmsq_f64 (r, d->log2_10_lo, n); + + uint64x2_t e = vshlq_n_u64 (u, 52 - V_EXP_TABLE_BITS); + uint64x2_t i = vandq_u64 (u, IndexMask); + + /* y = exp10(r) - 1 ~= C0 r + C1 r^2 + C2 r^3 + C3 r^4. */ + float64x2_t r2 = vmulq_f64 (r, r); + float64x2_t p = vfmaq_f64 (d->poly[0], r, d->poly[1]); + float64x2_t y = vfmaq_f64 (d->poly[2], r, d->poly[3]); + p = vfmaq_f64 (p, y, r2); + y = vmulq_f64 (r, p); + + /* s = 2^(n/N). */ + u = v_lookup_u64 (__v_exp_data, i); + float64x2_t s = vreinterpretq_f64_u64 (vaddq_u64 (u, e)); + + if (__glibc_unlikely (v_any_u64 (cmp))) +#if WANT_SIMD_EXCEPT + return special_case (xm, vfmaq_f64 (s, y, s), cmp); +#else + return special_case (s, y, n, d); +#endif + + return vfmaq_f64 (s, y, s); +} diff --git a/sysdeps/aarch64/fpu/exp10_sve.c b/sysdeps/aarch64/fpu/exp10_sve.c new file mode 100644 index 0000000000..a8cef7b692 --- /dev/null +++ b/sysdeps/aarch64/fpu/exp10_sve.c @@ -0,0 +1,127 @@ +/* Double-precision vector (SVE) exp10 function. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f64.h" + +#define SpecialBound 307.0 /* floor (log10 (2^1023)). */ + +static const struct data +{ + double poly[5]; + double shift, log10_2, log2_10_hi, log2_10_lo, scale_thres, special_bound; +} data = { + /* Coefficients generated using Remez algorithm. + rel error: 0x1.9fcb9b3p-60 + abs error: 0x1.a20d9598p-60 in [ -log10(2)/128, log10(2)/128 ] + max ulp err 0.52 +0.5. */ + .poly = { 0x1.26bb1bbb55516p1, 0x1.53524c73cd32ap1, 0x1.0470591daeafbp1, + 0x1.2bd77b1361ef6p0, 0x1.142b5d54e9621p-1 }, + /* 1.5*2^46+1023. This value is further explained below. */ + .shift = 0x1.800000000ffc0p+46, + .log10_2 = 0x1.a934f0979a371p1, /* 1/log2(10). */ + .log2_10_hi = 0x1.34413509f79ffp-2, /* log2(10). */ + .log2_10_lo = -0x1.9dc1da994fd21p-59, + .scale_thres = 1280.0, + .special_bound = SpecialBound, +}; + +#define SpecialOffset 0x6000000000000000 /* 0x1p513. */ +/* SpecialBias1 + SpecialBias1 = asuint(1.0). */ +#define SpecialBias1 0x7000000000000000 /* 0x1p769. */ +#define SpecialBias2 0x3010000000000000 /* 0x1p-254. */ + +/* Update of both special and non-special cases, if any special case is + detected. */ +static inline svfloat64_t +special_case (svbool_t pg, svfloat64_t s, svfloat64_t y, svfloat64_t n, + const struct data *d) +{ + /* s=2^n may overflow, break it up into s=s1*s2, + such that exp = s + s*y can be computed as s1*(s2+s2*y) + and s1*s1 overflows only if n>0. */ + + /* If n<=0 then set b to 0x6, 0 otherwise. */ + svbool_t p_sign = svcmple (pg, n, 0.0); /* n <= 0. */ + svuint64_t b = svdup_u64_z (p_sign, SpecialOffset); + + /* Set s1 to generate overflow depending on sign of exponent n. */ + svfloat64_t s1 = svreinterpret_f64 (svsubr_x (pg, b, SpecialBias1)); + /* Offset s to avoid overflow in final result if n is below threshold. */ + svfloat64_t s2 = svreinterpret_f64 ( + svadd_x (pg, svsub_x (pg, svreinterpret_u64 (s), SpecialBias2), b)); + + /* |n| > 1280 => 2^(n) overflows. */ + svbool_t p_cmp = svacgt (pg, n, d->scale_thres); + + svfloat64_t r1 = svmul_x (pg, s1, s1); + svfloat64_t r2 = svmla_x (pg, s2, s2, y); + svfloat64_t r0 = svmul_x (pg, r2, s1); + + return svsel (p_cmp, r1, r0); +} + +/* Fast vector implementation of exp10 using FEXPA instruction. + Maximum measured error is 1.02 ulp. + SV_NAME_D1 (exp10)(-0x1.2862fec805e58p+2) got 0x1.885a89551d782p-16 + want 0x1.885a89551d781p-16. */ +svfloat64_t SV_NAME_D1 (exp10) (svfloat64_t x, svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + svbool_t no_big_scale = svacle (pg, x, d->special_bound); + svbool_t special = svnot_z (pg, no_big_scale); + + /* n = round(x/(log10(2)/N)). */ + svfloat64_t shift = sv_f64 (d->shift); + svfloat64_t z = svmla_x (pg, shift, x, d->log10_2); + svfloat64_t n = svsub_x (pg, z, shift); + + /* r = x - n*log10(2)/N. */ + svfloat64_t log2_10 = svld1rq (svptrue_b64 (), &d->log2_10_hi); + svfloat64_t r = x; + r = svmls_lane (r, n, log2_10, 0); + r = svmls_lane (r, n, log2_10, 1); + + /* scale = 2^(n/N), computed using FEXPA. FEXPA does not propagate NaNs, so + for consistent NaN handling we have to manually propagate them. This + comes at significant performance cost. */ + svuint64_t u = svreinterpret_u64 (z); + svfloat64_t scale = svexpa (u); + + /* Approximate exp10(r) using polynomial. */ + svfloat64_t r2 = svmul_x (pg, r, r); + svfloat64_t y = svmla_x (pg, svmul_x (pg, r, d->poly[0]), r2, + sv_pairwise_poly_3_f64_x (pg, r, r2, d->poly + 1)); + + /* Assemble result as exp10(x) = 2^n * exp10(r). If |x| > SpecialBound + multiplication may overflow, so use special case routine. */ + if (__glibc_unlikely (svptest_any (pg, special))) + { + /* FEXPA zeroes the sign bit, however the sign is meaningful to the + special case function so needs to be copied. + e = sign bit of u << 46. */ + svuint64_t e = svand_x (pg, svlsl_x (pg, u, 46), 0x8000000000000000); + /* Copy sign to scale. */ + scale = svreinterpret_f64 (svadd_x (pg, e, svreinterpret_u64 (scale))); + return special_case (pg, scale, y, n, d); + } + + /* No special case. */ + return svmla_x (pg, scale, scale, y); +} diff --git a/sysdeps/aarch64/fpu/exp10f_advsimd.c b/sysdeps/aarch64/fpu/exp10f_advsimd.c new file mode 100644 index 0000000000..9e754c46fa --- /dev/null +++ b/sysdeps/aarch64/fpu/exp10f_advsimd.c @@ -0,0 +1,140 @@ +/* Single-precision vector (AdvSIMD) exp10 function. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f32.h" + +#define ScaleBound 192.0f + +static const struct data +{ + float32x4_t poly[5]; + float32x4_t shift, log10_2, log2_10_hi, log2_10_lo; +#if !WANT_SIMD_EXCEPT + float32x4_t scale_thresh; +#endif +} data = { + /* Coefficients generated using Remez algorithm with minimisation of relative + error. + rel error: 0x1.89dafa3p-24 + abs error: 0x1.167d55p-23 in [-log10(2)/2, log10(2)/2] + maxerr: 1.85943 +0.5 ulp. */ + .poly = { V4 (0x1.26bb16p+1f), V4 (0x1.5350d2p+1f), V4 (0x1.04744ap+1f), + V4 (0x1.2d8176p+0f), V4 (0x1.12b41ap-1f) }, + .shift = V4 (0x1.8p23f), + .log10_2 = V4 (0x1.a934fp+1), + .log2_10_hi = V4 (0x1.344136p-2), + .log2_10_lo = V4 (-0x1.ec10cp-27), +#if !WANT_SIMD_EXCEPT + .scale_thresh = V4 (ScaleBound) +#endif +}; + +#define ExponentBias v_u32 (0x3f800000) + +#if WANT_SIMD_EXCEPT + +# define SpecialBound 38.0f /* rint(log10(2^127)). */ +# define TinyBound v_u32 (0x20000000) /* asuint (0x1p-63). */ +# define BigBound v_u32 (0x42180000) /* asuint (SpecialBound). */ +# define Thres v_u32 (0x22180000) /* BigBound - TinyBound. */ + +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t x, float32x4_t y, uint32x4_t cmp) +{ + /* If fenv exceptions are to be triggered correctly, fall back to the scalar + routine to special lanes. */ + return v_call_f32 (exp10f, x, y, cmp); +} + +#else + +# define SpecialBound 126.0f /* rint (log2 (2^127 / (1 + sqrt (2)))). */ +# define SpecialOffset v_u32 (0x82000000) +# define SpecialBias v_u32 (0x7f000000) + +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t poly, float32x4_t n, uint32x4_t e, uint32x4_t cmp1, + float32x4_t scale, const struct data *d) +{ + /* 2^n may overflow, break it up into s1*s2. */ + uint32x4_t b = vandq_u32 (vclezq_f32 (n), SpecialOffset); + float32x4_t s1 = vreinterpretq_f32_u32 (vaddq_u32 (b, SpecialBias)); + float32x4_t s2 = vreinterpretq_f32_u32 (vsubq_u32 (e, b)); + uint32x4_t cmp2 = vcagtq_f32 (n, d->scale_thresh); + float32x4_t r2 = vmulq_f32 (s1, s1); + float32x4_t r1 = vmulq_f32 (vfmaq_f32 (s2, poly, s2), s1); + /* Similar to r1 but avoids double rounding in the subnormal range. */ + float32x4_t r0 = vfmaq_f32 (scale, poly, scale); + float32x4_t r = vbslq_f32 (cmp1, r1, r0); + return vbslq_f32 (cmp2, r2, r); +} + +#endif + +/* Fast vector implementation of single-precision exp10. + Algorithm is accurate to 2.36 ULP. + _ZGVnN4v_exp10f(0x1.be2b36p+1) got 0x1.7e79c4p+11 + want 0x1.7e79cp+11. */ +float32x4_t VPCS_ATTR V_NAME_F1 (exp10) (float32x4_t x) +{ + const struct data *d = ptr_barrier (&data); +#if WANT_SIMD_EXCEPT + /* asuint(x) - TinyBound >= BigBound - TinyBound. */ + uint32x4_t cmp = vcgeq_u32 ( + vsubq_u32 (vandq_u32 (vreinterpretq_u32_f32 (x), v_u32 (0x7fffffff)), + TinyBound), + Thres); + float32x4_t xm = x; + /* If any lanes are special, mask them with 1 and retain a copy of x to allow + special case handler to fix special lanes later. This is only necessary if + fenv exceptions are to be triggered correctly. */ + if (__glibc_unlikely (v_any_u32 (cmp))) + x = vbslq_f32 (cmp, v_f32 (1), x); +#endif + + /* exp10(x) = 2^n * 10^r = 2^n * (1 + poly (r)), + with poly(r) in [1/sqrt(2), sqrt(2)] and + x = r + n * log10 (2), with r in [-log10(2)/2, log10(2)/2]. */ + float32x4_t z = vfmaq_f32 (d->shift, x, d->log10_2); + float32x4_t n = vsubq_f32 (z, d->shift); + float32x4_t r = vfmsq_f32 (x, n, d->log2_10_hi); + r = vfmsq_f32 (r, n, d->log2_10_lo); + uint32x4_t e = vshlq_n_u32 (vreinterpretq_u32_f32 (z), 23); + + float32x4_t scale = vreinterpretq_f32_u32 (vaddq_u32 (e, ExponentBias)); + +#if !WANT_SIMD_EXCEPT + uint32x4_t cmp = vcagtq_f32 (n, v_f32 (SpecialBound)); +#endif + + float32x4_t r2 = vmulq_f32 (r, r); + float32x4_t poly + = vfmaq_f32 (vmulq_f32 (r, d->poly[0]), + v_pairwise_poly_3_f32 (r, r2, d->poly + 1), r2); + + if (__glibc_unlikely (v_any_u32 (cmp))) +#if WANT_SIMD_EXCEPT + return special_case (xm, vfmaq_f32 (scale, poly, scale), cmp); +#else + return special_case (poly, n, e, cmp, scale, d); +#endif + + return vfmaq_f32 (scale, poly, scale); +} diff --git a/sysdeps/aarch64/fpu/exp10f_sve.c b/sysdeps/aarch64/fpu/exp10f_sve.c new file mode 100644 index 0000000000..3df3246e05 --- /dev/null +++ b/sysdeps/aarch64/fpu/exp10f_sve.c @@ -0,0 +1,91 @@ +/* Single-precision vector (SVE) exp10 function. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f32.h" + +/* For x < -SpecialBound, the result is subnormal and not handled correctly by + FEXPA. */ +#define SpecialBound 37.9 + +static const struct data +{ + float poly[5]; + float shift, log10_2, log2_10_hi, log2_10_lo, special_bound; +} data = { + /* Coefficients generated using Remez algorithm with minimisation of relative + error. + rel error: 0x1.89dafa3p-24 + abs error: 0x1.167d55p-23 in [-log10(2)/2, log10(2)/2] + maxerr: 0.52 +0.5 ulp. */ + .poly = { 0x1.26bb16p+1f, 0x1.5350d2p+1f, 0x1.04744ap+1f, 0x1.2d8176p+0f, + 0x1.12b41ap-1f }, + /* 1.5*2^17 + 127, a shift value suitable for FEXPA. */ + .shift = 0x1.903f8p17f, + .log10_2 = 0x1.a934fp+1, + .log2_10_hi = 0x1.344136p-2, + .log2_10_lo = -0x1.ec10cp-27, + .special_bound = SpecialBound, +}; + +static svfloat32_t NOINLINE +special_case (svfloat32_t x, svfloat32_t y, svbool_t special) +{ + return sv_call_f32 (exp10f, x, y, special); +} + +/* Single-precision SVE exp10f routine. Implements the same algorithm + as AdvSIMD exp10f. + Worst case error is 1.02 ULPs. + _ZGVsMxv_exp10f(-0x1.040488p-4) got 0x1.ba5f9ep-1 + want 0x1.ba5f9cp-1. */ +svfloat32_t SV_NAME_F1 (exp10) (svfloat32_t x, const svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + /* exp10(x) = 2^(n/N) * 10^r = 2^n * (1 + poly (r)), + with poly(r) in [1/sqrt(2), sqrt(2)] and + x = r + n * log10(2) / N, with r in [-log10(2)/2N, log10(2)/2N]. */ + + /* Load some constants in quad-word chunks to minimise memory access (last + lane is wasted). */ + svfloat32_t log10_2_and_inv = svld1rq (svptrue_b32 (), &d->log10_2); + + /* n = round(x/(log10(2)/N)). */ + svfloat32_t shift = sv_f32 (d->shift); + svfloat32_t z = svmla_lane (shift, x, log10_2_and_inv, 0); + svfloat32_t n = svsub_x (pg, z, shift); + + /* r = x - n*log10(2)/N. */ + svfloat32_t r = svmls_lane (x, n, log10_2_and_inv, 1); + r = svmls_lane (r, n, log10_2_and_inv, 2); + + svbool_t special = svacgt (pg, x, d->special_bound); + svfloat32_t scale = svexpa (svreinterpret_u32 (z)); + + /* Polynomial evaluation: poly(r) ~ exp10(r)-1. */ + svfloat32_t r2 = svmul_x (pg, r, r); + svfloat32_t poly + = svmla_x (pg, svmul_x (pg, r, d->poly[0]), + sv_pairwise_poly_3_f32_x (pg, r, r2, d->poly + 1), r2); + + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x, svmla_x (pg, scale, scale, poly), special); + + return svmla_x (pg, scale, scale, poly); +} diff --git a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c index 8d05498ec9..26d5ecf66f 100644 --- a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c @@ -25,6 +25,7 @@ VPCS_VECTOR_WRAPPER (cos_advsimd, _ZGVnN2v_cos) VPCS_VECTOR_WRAPPER (exp_advsimd, _ZGVnN2v_exp) +VPCS_VECTOR_WRAPPER (exp10_advsimd, _ZGVnN2v_exp10) VPCS_VECTOR_WRAPPER (exp2_advsimd, _ZGVnN2v_exp2) VPCS_VECTOR_WRAPPER (log_advsimd, _ZGVnN2v_log) VPCS_VECTOR_WRAPPER (log10_advsimd, _ZGVnN2v_log10) diff --git a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c index b65bc6f1e6..86efd60779 100644 --- a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c @@ -34,6 +34,7 @@ SVE_VECTOR_WRAPPER (cos_sve, _ZGVsMxv_cos) SVE_VECTOR_WRAPPER (exp_sve, _ZGVsMxv_exp) +SVE_VECTOR_WRAPPER (exp10_sve, _ZGVsMxv_exp10) SVE_VECTOR_WRAPPER (exp2_sve, _ZGVsMxv_exp2) SVE_VECTOR_WRAPPER (log_sve, _ZGVsMxv_log) SVE_VECTOR_WRAPPER (log10_sve, _ZGVsMxv_log10) diff --git a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c index 6ced0d4488..8f7ebea1ac 100644 --- a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c @@ -25,6 +25,7 @@ VPCS_VECTOR_WRAPPER (cosf_advsimd, _ZGVnN4v_cosf) VPCS_VECTOR_WRAPPER (expf_advsimd, _ZGVnN4v_expf) +VPCS_VECTOR_WRAPPER (exp10f_advsimd, _ZGVnN4v_exp10f) VPCS_VECTOR_WRAPPER (exp2f_advsimd, _ZGVnN4v_exp2f) VPCS_VECTOR_WRAPPER (logf_advsimd, _ZGVnN4v_logf) VPCS_VECTOR_WRAPPER (log10f_advsimd, _ZGVnN4v_log10f) diff --git a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c index 2ed8d0659a..885e58ac39 100644 --- a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c @@ -34,6 +34,7 @@ SVE_VECTOR_WRAPPER (cosf_sve, _ZGVsMxv_cosf) SVE_VECTOR_WRAPPER (expf_sve, _ZGVsMxv_expf) +SVE_VECTOR_WRAPPER (exp10f_sve, _ZGVsMxv_exp10f) SVE_VECTOR_WRAPPER (exp2f_sve, _ZGVsMxv_exp2f) SVE_VECTOR_WRAPPER (logf_sve, _ZGVsMxv_logf) SVE_VECTOR_WRAPPER (log10f_sve, _ZGVsMxv_log10f) diff --git a/sysdeps/aarch64/libm-test-ulps b/sysdeps/aarch64/libm-test-ulps index 6641c7fa0b..d117209c06 100644 --- a/sysdeps/aarch64/libm-test-ulps +++ b/sysdeps/aarch64/libm-test-ulps @@ -970,11 +970,19 @@ double: 2 float: 1 ldouble: 2 +Function: "exp10_advsimd": +double: 1 +float: 2 + Function: "exp10_downward": double: 2 float: 1 ldouble: 3 +Function: "exp10_sve": +double: 1 +float: 1 + Function: "exp10_towardzero": double: 2 float: 1 diff --git a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist index f8776a6bea..cad774521a 100644 --- a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist +++ b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist @@ -14,14 +14,18 @@ GLIBC_2.38 _ZGVsMxv_log F GLIBC_2.38 _ZGVsMxv_logf F GLIBC_2.38 _ZGVsMxv_sin F GLIBC_2.38 _ZGVsMxv_sinf F +GLIBC_2.39 _ZGVnN2v_exp10 F GLIBC_2.39 _ZGVnN2v_exp2 F GLIBC_2.39 _ZGVnN2v_log10 F GLIBC_2.39 _ZGVnN2v_log2 F GLIBC_2.39 _ZGVnN2v_tan F +GLIBC_2.39 _ZGVnN4v_exp10f F GLIBC_2.39 _ZGVnN4v_exp2f F GLIBC_2.39 _ZGVnN4v_log10f F GLIBC_2.39 _ZGVnN4v_log2f F GLIBC_2.39 _ZGVnN4v_tanf F +GLIBC_2.39 _ZGVsMxv_exp10 F +GLIBC_2.39 _ZGVsMxv_exp10f F GLIBC_2.39 _ZGVsMxv_exp2 F GLIBC_2.39 _ZGVsMxv_exp2f F GLIBC_2.39 _ZGVsMxv_log10 F