From patchwork Wed Mar 13 14:31:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wilco Dijkstra X-Patchwork-Id: 1911739 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=armh.onmicrosoft.com header.i=@armh.onmicrosoft.com header.a=rsa-sha256 header.s=selector2-armh-onmicrosoft-com header.b=sUaSbgE7; dkim=pass (1024-bit key) header.d=armh.onmicrosoft.com header.i=@armh.onmicrosoft.com header.a=rsa-sha256 header.s=selector2-armh-onmicrosoft-com header.b=sUaSbgE7; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TvtHz4P6Cz1ydl for ; Thu, 14 Mar 2024 01:31:59 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 61A1F3857BA8 for ; Wed, 13 Mar 2024 14:31:57 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2076.outbound.protection.outlook.com [40.107.20.76]) by sourceware.org (Postfix) with ESMTPS id 637443858C3A for ; Wed, 13 Mar 2024 14:31:39 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 637443858C3A Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 637443858C3A Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=40.107.20.76 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1710340301; cv=pass; b=vNO2xwmVpPDLivZsB8Gpqr/AG1Zab2CbE8zjPvIxr9YVaZPHeKtx8Q2YM+So63AmO9R3ZRxyhKS/glGeDPdYu0mJI9wL4d+5uiOIqHGH9NJmCLdDukjcBPWw5JNLZQW55D74oo1XXmFbDxuvTsQmXxO7PTUqYGyFeqBSterpwl8= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1710340301; c=relaxed/simple; bh=EaOKUuTIEnK5+H5aznPm8Ue5MxpfLoHX7SgAfyYVVGA=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=Owc+iaT2CU7MvbqplPMkp1f39DrVfO2bR2Utp/716sHij4JvZBEBTIiXU44r2QNk0G2iXFkmExjr+ub71jzobhblK/lGXhPSHWZ+bwxcfF/FIWVNnKc1BKPZSzNuEqY/tgtl6o+FG8jnrfhLg63RHfbmJUesaJyZ/CeW7JcnDB4= ARC-Authentication-Results: i=3; server2.sourceware.org ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=TjWlE2wmUk2kE2BO3aqpYWUsPMdPlLcqPTSLCdkcZkjEjgwAc8k/rR1RULsy0mEYeQ/2Ey/Koh6XDVOA/G5+xGKXkUa9tmXvWlxQpuseWTX/2kZCos96RyQy1AoOCC8lKBKzcjRBpFLfR1EqlsQ9rNth7Nb+7WntWl8uXddQE0h9L5JGK5JelAb4USoV2G7iDpJrA71XLKvlBjNFeBheG6pUcCQVc/Svknj8RT6QKnIC+BndOq3Ad9IGTo3n0bbW+WBosYZ5Hk1wBsL9wItQY94dOyp0yGDQIM5QOxMT0fmiljzfVOFhm2S1+jlznl5EAfuwou5ECZrUZybdU31wEw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hnh4UgpuhcwAqXok6wl8WNaVuVyh1E1ApmpGkb/IZ2I=; b=HUOy68h96ptmGK7vsGoBSXV8muoJAOIaCFE+DjIjCkxGH7Wzk0KT8VjwIC92nUnNHS3dsV6Zo9piNUCfQXoMSdF996UQWGAKGYxBUhtpEsVcGLBaSOdWH4Hc10jeWPSnVj0oR3koUN1lj6w+LblO2yw481XYOR8sO4hau2vpmP8YgqWDEosFpllJbR1WbmZ56tzSnONUxMZN6Cdq4+Kwhk7u+eyUzGzA6DXnsvK1GGsyryOqeXzcoeQ++VxzY2C1kQW88/Y+eKfXKPgBEO68OgqzBNe26vqFIfxXZtnRXMV4cfYq928PwDkEnAxOmRBQSR0WLFTIe/6hMBrLwr1idg== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hnh4UgpuhcwAqXok6wl8WNaVuVyh1E1ApmpGkb/IZ2I=; b=sUaSbgE7TNVhdXHdHXHAbKUTnHRlKnvC+pIU32JTYEEuIK5HeZC23gGqa5/qCgm5AWeZAPAd2CcfDovBxzYpRat3+UUaARMnULP0/r09aA1EAkGHVO28thASzoF9nUHPzQpC2BdJmScTvoGk3Sajoh8bgJxU7ArlsguKg+dFfpw= Received: from DB8PR04CA0028.eurprd04.prod.outlook.com (2603:10a6:10:110::38) by DU2PR08MB10085.eurprd08.prod.outlook.com (2603:10a6:10:496::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7362.35; Wed, 13 Mar 2024 14:31:34 +0000 Received: from DB5PEPF00014B98.eurprd02.prod.outlook.com (2603:10a6:10:110:cafe::ab) by DB8PR04CA0028.outlook.office365.com (2603:10a6:10:110::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7386.18 via Frontend Transport; Wed, 13 Mar 2024 14:31:34 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5PEPF00014B98.mail.protection.outlook.com (10.167.8.165) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7386.12 via Frontend Transport; Wed, 13 Mar 2024 14:31:33 +0000 Received: ("Tessian outbound 5fdc1e2b2c35:v276"); Wed, 13 Mar 2024 14:31:33 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 927985a74c063f66 X-CR-MTA-TID: 64aa7808 Received: from 502a96470429.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 57857CCA-F357-4FDC-B317-85988FBB5E63.1; Wed, 13 Mar 2024 14:31:26 +0000 Received: from EUR03-DBA-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 502a96470429.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 13 Mar 2024 14:31:26 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NmqwZgxvppOFjLka9wtCcVd6h4LSeUwy0Vu4JQDJ6SVR3sG2kzpzxn8h6bhRr8YO/V32VKGl4yR0YGCGljHPE7FOxM1OtQhSrPy1o3/iW6GdhY9kPtoG3Ab5ITN57elrZrfYcTNFkKHcp6lFu0Bu84bRjcxx7fKwEvnxUHxy2+vd42J0qTagbsRVSnrwvS2wb2+vT1WLB5nPoW6ZQMmmJMo8FukHDS6qyJ9agfSCjtSXhGnbJb68/VgARwI66HzoJB1+zfgZkR2yPSqNr9fUwEhxvdSKjtnWhJr5Jj+tQg/znpROrncrO2cpgHsmVnCRZhbMm50SyEK3Fb5FXGXwSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hnh4UgpuhcwAqXok6wl8WNaVuVyh1E1ApmpGkb/IZ2I=; b=naS8Bxo8oZtzURDhxGlBUmOz4RHbxeaE2X+v/N8+Fdrsy3rKS7zKcWd2/Sw1RT5zgc0V+g42b9f0E/ybSozpvo/dnFHpLF5dFZ1GMDjpJ6BJ2ANUUAuOs/UkoE6eYs3cyB5yMgtQTaypDBEqJo+DK7nYXCc/BCQa6piuXclLKbZ1gvZVQrNwarIeO+FvzRzucOwndnNiBqAnmLUWdWzoqxY+BHCRDWTv/5+iVQBrQoBsDyeYhscpwk+FdS0hjlB3vuPtI+ETequ4iV49OvRVEUfyRVg612WqAa8SsfS3uLUgmaGSy5FabjQLBBKiasglq7yKns4ipfxKQQVA7vLlTw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hnh4UgpuhcwAqXok6wl8WNaVuVyh1E1ApmpGkb/IZ2I=; b=sUaSbgE7TNVhdXHdHXHAbKUTnHRlKnvC+pIU32JTYEEuIK5HeZC23gGqa5/qCgm5AWeZAPAd2CcfDovBxzYpRat3+UUaARMnULP0/r09aA1EAkGHVO28thASzoF9nUHPzQpC2BdJmScTvoGk3Sajoh8bgJxU7ArlsguKg+dFfpw= Received: from PAWPR08MB8982.eurprd08.prod.outlook.com (2603:10a6:102:33f::20) by PA4PR08MB7595.eurprd08.prod.outlook.com (2603:10a6:102:271::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7362.36; Wed, 13 Mar 2024 14:31:25 +0000 Received: from PAWPR08MB8982.eurprd08.prod.outlook.com ([fe80::8b1b:5f28:5006:ac18]) by PAWPR08MB8982.eurprd08.prod.outlook.com ([fe80::8b1b:5f28:5006:ac18%4]) with mapi id 15.20.7362.035; Wed, 13 Mar 2024 14:31:25 +0000 From: Wilco Dijkstra To: 'GNU C Library' Subject: [PATCH] AArch64: Check kernel version for SVE ifuncs Thread-Topic: [PATCH] AArch64: Check kernel version for SVE ifuncs Thread-Index: AQHadVLx33HAZ4D2skuZZ4SgsA+U6w== Date: Wed, 13 Mar 2024 14:31:25 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; x-ms-traffictypediagnostic: PAWPR08MB8982:EE_|PA4PR08MB7595:EE_|DB5PEPF00014B98:EE_|DU2PR08MB10085:EE_ X-MS-Office365-Filtering-Correlation-Id: 9cf54599-ca7b-449c-4ec3-08dc436a4818 x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: oxaXrFIex4R6gvSLSWS3hPNl0x5jgrU9BN9xyda2nd9BltmXy1ErdwSFqtVa34JUn0UJVFKA18W4VpIsJJXxECkmeiiXVB+49+kAl8cQ5K6PILFPQlH1Nz9WHGAvvuHsJUEsrbQyUrq7CLK9pHdJHCPLHHGVjDWRScUh4nm4A64L/igGmNj4G9w/2YuM2yix8XVZtyQZvduF7MVdCvfETvNO3KLEXLTdShX3D0aNsMz+DWVoqdbFBB1fMAkDYh31S8GmTpp3vqBHds8WG9/YwcRc1xJ9i/yUp1L2XwRrwQhzAA9OXXDvenv6lt2cezeOhc5hbpxcc6es1Q6/Ex1TOwdztsRVPbs2cXezM0706FgUnPd6Bz7geUl+paxkuBd4IVo4QxO38ynIYqhcPk/fq9phnt/6m5+5LOlNPXai6XsASO72WnSBpIAmP7OLOoCl1GGMTnsqkO2hF1gcOARXQz81Xh/aNJDRbnR7jrhgae+6e7/KcigZxlBBIlWBn7ihL0b+FS7ZrpVydLVEk7YuzCRxCOlzVDbk97IJe+Cv4+hmNAs2VhPPNzf6VycMwzpStkxpLX+dINUU+RM82JRjizba3PsIHxNMrsmhktr7l9auPzHqRAhiajCI+hfpMjanqS7YAbFdL/fUOZctTzh+9cscldYtf6Fvyz4Hg3TULHDqm6MXh7isTt9Pve5MIPZYQCGXevFXpRT/BV+E3JoVGh8YzOIPCA5ri0gz1BCuHmo= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAWPR08MB8982.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(1800799015)(376005)(38070700009); DIR:OUT; SFP:1101; MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB7595 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5PEPF00014B98.eurprd02.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 23c387fd-4778-48e3-06af-08dc436a431b X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Ug0NhrwjKkBPWChqWYhZhXsL3HSeE0BhQPAxs1ylgatXrIFfQu1QhAnox6xmjsTZlexpiykCfFiWPK11aHhxzMNNVrXiYLkV+5j3p0QiXJDO39O5qqmsxhsJKcYQozMmiYMm6y3F+g/7kEk0A1kFzIToNPPMzi9k0lwL2LzMTmmI9qCSAjB9RcjQuH7ZewA/AI2LEM2SjZ4uumR1A05aL70IdOB/F3dveKyEzbzs9lKtZo6yL6Bh3sY2cPVa983PccPfM1fEGL+GyBdeE71frEIATfpDLNxw/EFeRzzkvwCUo23tMcR5n7NFf6y9kNPAOdCGfVuCJF3FevmFX+ZwMO9h1n2A9/hEQLbEgXr5fBjPq5yf4k2oiXpRHtIhFWQkbWf169Erwj6GCk60ZQRTOCrP2cGerlxGKxpG6b5P13l630hfExOpA4nff6Wj8Buu1Fol7uBiPaY/SuKeAa5DOLVr/J989lN21DpL1ga/NGU+BSOPaSfIdxysjTjMUls/XxJ+b+rKeeRBraZ8CAQceB9XtP5S0OvRdXJ/UVLckNUUiHudpPUnaaHrVDRMW2G6PA47/Sw7Zv1IHzrTX2X2i817lpV79cjuW1uafFDEUHAYjdhZRytPZAjpaKP1DrD9OML0b4Bau5SPuhh6r8UqPkJrxQFuV7S9d14mji9OF4gJ5V00ILNdp8nEwXtcJ1wmfjZiiJwCsBe6IZHr6oBnG41TjXYF5Cu/HaWrD+ebQ3F1O3sQl4Kkt48cyTO4RsPZ X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230031)(1800799015)(82310400014)(376005)(36860700004); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Mar 2024 14:31:33.4914 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9cf54599-ca7b-449c-4ec3-08dc436a4818 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DB5PEPF00014B98.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR08MB10085 X-Spam-Status: No, score=-10.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, KAM_LOTSOFHASH, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Older Linux kernels may disable SVE after certain syscalls. Calling the SVE-optimized memcpy afterwards will then cause a trap to reenable SVE. As a result, applications with a high use of syscalls may run slower with the SVE memcpy. Avoid this by checking the kernel version and enable the SVE-optimized memcpy/memmove ifuncs only on Linux kernel 6.2 or newer. Passes regress, OK for commit? diff --git a/sysdeps/aarch64/cpu-features.h b/sysdeps/aarch64/cpu-features.h index 77a782422af1b6e4b2af32bfebfda37874111510..5f2da91ebbd0adafb0d84ec503b0f902f566da5a 100644 --- a/sysdeps/aarch64/cpu-features.h +++ b/sysdeps/aarch64/cpu-features.h @@ -71,6 +71,7 @@ struct cpu_features /* Currently, the GLIBC memory tagging tunable only defines 8 bits. */ uint8_t mte_state; bool sve; + bool prefer_sve_ifuncs; bool mops; }; diff --git a/sysdeps/aarch64/multiarch/init-arch.h b/sysdeps/aarch64/multiarch/init-arch.h index c52860efb22d70eb4bdf356781f51c7de8ec67dc..61dc40088f4d9e5e06b57bdc7d54bde1e2a686a4 100644 --- a/sysdeps/aarch64/multiarch/init-arch.h +++ b/sysdeps/aarch64/multiarch/init-arch.h @@ -36,5 +36,7 @@ MTE_ENABLED (); \ bool __attribute__((unused)) sve = \ GLRO(dl_aarch64_cpu_features).sve; \ + bool __attribute__((unused)) prefer_sve_ifuncs = \ + GLRO(dl_aarch64_cpu_features).prefer_sve_ifuncs; \ bool __attribute__((unused)) mops = \ GLRO(dl_aarch64_cpu_features).mops; diff --git a/sysdeps/aarch64/multiarch/memcpy.c b/sysdeps/aarch64/multiarch/memcpy.c index d12eccfca51f4bcfef6ccf5aa286edb301e361ac..ce53567dab33c2f00b89b4069235abd4651811a6 100644 --- a/sysdeps/aarch64/multiarch/memcpy.c +++ b/sysdeps/aarch64/multiarch/memcpy.c @@ -47,7 +47,7 @@ select_memcpy_ifunc (void) { if (IS_A64FX (midr)) return __memcpy_a64fx; - return __memcpy_sve; + return prefer_sve_ifuncs ? __memcpy_sve : __memcpy_generic; } if (IS_THUNDERX (midr)) diff --git a/sysdeps/aarch64/multiarch/memmove.c b/sysdeps/aarch64/multiarch/memmove.c index 2081eeb4d40e0240e67a7b26b64576f44eaf18e3..fe95037be391896c7670ef606bf4d3ba7dfb6a00 100644 --- a/sysdeps/aarch64/multiarch/memmove.c +++ b/sysdeps/aarch64/multiarch/memmove.c @@ -47,7 +47,7 @@ select_memmove_ifunc (void) { if (IS_A64FX (midr)) return __memmove_a64fx; - return __memmove_sve; + return prefer_sve_ifuncs ? __memmove_sve : __memmove_generic; } if (IS_THUNDERX (midr)) diff --git a/sysdeps/unix/sysv/linux/aarch64/cpu-features.c b/sysdeps/unix/sysv/linux/aarch64/cpu-features.c index b1a3f673f067280bdacfddd92723a81e418023e5..13b02c45df80b493516b3c9d4acbbbffaa47af92 100644 --- a/sysdeps/unix/sysv/linux/aarch64/cpu-features.c +++ b/sysdeps/unix/sysv/linux/aarch64/cpu-features.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #define DCZID_DZP_MASK (1 << 4) @@ -62,6 +63,41 @@ get_midr_from_mcpu (const struct tunable_str_t *mcpu) return UINT64_MAX; } +/* Parse kernel version without calling any library functions. + Allow 2 digits for kernel version and 3 digits for major version, + separated by '.': "kk.mmm.". + Return kernel version * 1000 + major version, or -1 on failure. */ + +static inline int +kernel_version (void) +{ + struct utsname buf; + const char *p = &buf.release[0]; + int kernel = 0; + int major = 0; + + if (__uname (&buf) < 0) + return -1; + + if (*p >= '0' && *p <= '9') + kernel = (kernel * 10) + *p++ - '0'; + if (*p >= '0' && *p <= '9') + kernel = (kernel * 10) + *p++ - '0'; + if (*p != '.') + return -1; + p++; + if (*p >= '0' && *p <= '9') + major = (major * 10) + *p++ - '0'; + if (*p >= '0' && *p <= '9') + major = (major * 10) + *p++ - '0'; + if (*p >= '0' && *p <= '9') + major = (major * 10) + *p++ - '0'; + if (*p != '.' && *p != '\0') + return -1; + + return kernel * 1000 + major; +} + static inline void init_cpu_features (struct cpu_features *cpu_features) { @@ -126,6 +162,10 @@ init_cpu_features (struct cpu_features *cpu_features) /* Check if SVE is supported. */ cpu_features->sve = GLRO (dl_hwcap) & HWCAP_SVE; + /* Prefer using SVE in string ifuncs from Linux 6.2 onwards. */ + cpu_features->prefer_sve_ifuncs = + cpu_features->sve && kernel_version () >= 6002; + /* Check if MOPS is supported. */ cpu_features->mops = GLRO (dl_hwcap2) & HWCAP2_MOPS; }