From patchwork Fri Jun 24 22:29:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Noah Goldstein X-Patchwork-Id: 1648208 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=cRIBZLgS; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LVBd46cxmz9s0w for ; Sat, 25 Jun 2022 08:29:44 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 872FB3875A21 for ; Fri, 24 Jun 2022 22:29:42 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 872FB3875A21 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1656109782; bh=MHd3FGJNys3j0jaskcvVPyg76bxfD08z+noNFytd/Yo=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=cRIBZLgSMRJCl1Tz6CfJnloOco0OLvyG7XyeNmhwf3W87ITDtGOcwBInamfRRxT31 Lc58EV6JgLkFMnfJTdtIpS3pGiEkEeZI9zejhL3OxOC9UAMyh5k24sANZQr3J+GqrH wbBGZDwY81SK0wzn7RPZ151PQMhVVHHMFDF8v6ek= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) by sourceware.org (Postfix) with ESMTPS id C07D63875A0E for ; Fri, 24 Jun 2022 22:29:15 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org C07D63875A0E Received: by mail-pf1-x42f.google.com with SMTP id 65so3666934pfw.11 for ; Fri, 24 Jun 2022 15:29:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MHd3FGJNys3j0jaskcvVPyg76bxfD08z+noNFytd/Yo=; b=yaKYPIK0iBT/mEca7cvWV4lrPMSLjicuRVQ3OOkCuk4cczca725O6bvi/fkg1CBTTx AUE8y8gCdFX5jfYvWlFgJsLU6TXHIfWuFoWoEwMwhVG117LgamsHyE4TZkuKU3fU1dJc belcPHu/Iium5Bj5BKoXPHTAPJCP+M14eCExxHpOVHgIDudahXc2U8DIUYE0lJtyCwHU V3MzJdto4SmXWrmn1P9crCLKzvLlqivg7cJ1CrVUwy4MjExhEgzPL42NzgPBQOYOnEZZ rw9ZBvkoo7F7q33xW9ALEE/qZDIfmAm26pQlYXYn0lh7PSYKvXDfRVpRm2uYSsmOQiHr FzwA== X-Gm-Message-State: AJIora+gxDfImNwN1b+O+B6P5hKYpyMfVWVLTmmc48GGtoYzA019Lzav uYo5v7wdgB6HB+PzJ16hAeTOgO+v6dU= X-Google-Smtp-Source: AGRyM1vW1TuI+biX+0RlKM2AC0SVxmXMJ1IbKdg6/yjpifTMPdOT0pXVhZTQrgiOrwNEZxtUpz+XWQ== X-Received: by 2002:a63:698a:0:b0:40c:4714:2425 with SMTP id e132-20020a63698a000000b0040c47142425mr910263pgc.548.1656109753577; Fri, 24 Jun 2022 15:29:13 -0700 (PDT) Received: from noah-tgl.. ([192.55.60.37]) by smtp.gmail.com with ESMTPSA id x4-20020a170902a38400b0015e8d4eb1easm2329597pla.52.2022.06.24.15.29.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Jun 2022 15:29:13 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v4] x86: Fix backwards Prefer_No_VZEROUPPER check in ifunc-evex.h Date: Fri, 24 Jun 2022 15:29:10 -0700 Message-Id: <20220624222910.1359991-1-goldstein.w.n@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220624063653.2126416-3-goldstein.w.n@gmail.com> References: <20220624063653.2126416-3-goldstein.w.n@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Noah Goldstein via Libc-alpha From: Noah Goldstein Reply-To: Noah Goldstein Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" Add third argument to X86_ISA_CPU_FEATURES_ARCH_P macro so the runtime CPU_FEATURES_ARCH_P check can be inverted if the MINIMUM_X86_ISA_LEVEL is not high enough to constantly evaluate the check. Use this new macro to correct the backwards check in ifunc-evex.h --- sysdeps/x86/isa-ifunc-macros.h | 46 +++++++++++++++++++++++---- sysdeps/x86/isa-level.h | 32 ++++++++----------- sysdeps/x86_64/multiarch/ifunc-evex.h | 4 +-- 3 files changed, 56 insertions(+), 26 deletions(-) diff --git a/sysdeps/x86/isa-ifunc-macros.h b/sysdeps/x86/isa-ifunc-macros.h index ba6826d518..4cdc71e40e 100644 --- a/sysdeps/x86/isa-ifunc-macros.h +++ b/sysdeps/x86/isa-ifunc-macros.h @@ -56,15 +56,49 @@ # define X86_IFUNC_IMPL_ADD_V1(...) #endif -#define X86_ISA_CPU_FEATURE_CONST_CHECK_ENABLED(name) \ - ((name##_X86_ISA_LEVEL) <= MINIMUM_X86_ISA_LEVEL) +/* Both X86_ISA_CPU_FEATURE_USABLE_P and X86_ISA_CPU_FEATURES_ARCH_P + macros are wrappers for the the respective + CPU_FEATURE{S}_{USABLE|ARCH}_P runtime checks. They differ in two + ways. + + 1. The USABLE_P version is evaluated to true when the feature + is enabled. + + 2. The ARCH_P version has a third argument `not`. The `not` + argument can either be '!' or empty. If the feature is + enabled above an ISA level, the third argument should be empty + and the expression is evaluated to true when the feature is + enabled. If the feature is disabled above an ISA level, the + third argument should be `!` and the expression is evaluated + to true when the feature is disabled. + + i.e + + X86_ISA_CPU_FEATURE_USABLE_P(p, FEATURE): + A) default value of feature <= MINIMUM_X86_ISA_LEVEL -> True + B) runtime bit is set -> True + + X86_ISA_CPU_FEATURE_ARCH_P(p, FEATURE, ): + A) default value of feature <= MINIMUM_X86_ISA_LEVEL -> True + B) runtime bit is set -> True + + X86_ISA_CPU_FEATURE_ARCH_P(p, FEATURE, !): + A) default value of feature <= MINIMUM_X86_ISA_LEVEL -> True + B) runtime bit is NOT set -> True + + All cases not listed will evaluate to false. + + Only the A) cases will constantly evaluate. + */ #define X86_ISA_CPU_FEATURE_USABLE_P(ptr, name) \ - (X86_ISA_CPU_FEATURE_CONST_CHECK_ENABLED (name) \ + (((name##_X86_ISA_LEVEL) <= MINIMUM_X86_ISA_LEVEL) \ || CPU_FEATURE_USABLE_P (ptr, name)) -#define X86_ISA_CPU_FEATURES_ARCH_P(ptr, name) \ - (X86_ISA_CPU_FEATURE_CONST_CHECK_ENABLED (name) \ - || CPU_FEATURES_ARCH_P (ptr, name)) + +#define X86_ISA_CPU_FEATURES_ARCH_P(ptr, name, not) \ + (((name##_X86_ISA_LEVEL) <= MINIMUM_X86_ISA_LEVEL) \ + || not CPU_FEATURES_ARCH_P (ptr, name)) + #endif diff --git a/sysdeps/x86/isa-level.h b/sysdeps/x86/isa-level.h index 7cae11c228..73937fecdc 100644 --- a/sysdeps/x86/isa-level.h +++ b/sysdeps/x86/isa-level.h @@ -64,14 +64,8 @@ #define MINIMUM_X86_ISA_LEVEL \ (__X86_ISA_V1 + __X86_ISA_V2 + __X86_ISA_V3 + __X86_ISA_V4) - -/* - * CPU Features that are hard coded as enabled depending on ISA build - * level. - * - Values > 0 features are always ENABLED if: - * Value >= MINIMUM_X86_ISA_LEVEL - */ - +/* Depending on the minimum ISA level, a feature check result can be a + compile-time constant.. */ /* ISA level >= 4 guaranteed includes. */ #define AVX512VL_X86_ISA_LEVEL 4 @@ -81,20 +75,22 @@ #define AVX2_X86_ISA_LEVEL 3 #define BMI2_X86_ISA_LEVEL 3 -/* - * NB: This may not be fully assumable for ISA level >= 3. From - * looking over the architectures supported in cpu-features.h the - * following CPUs may have an issue with this being default set: - * - AMD Excavator - */ +/* NB: This feature is enabled when ISA level >= 3, which was disabled + for the following CPUs: + - AMD Excavator + when ISA level < 3. */ #define AVX_Fast_Unaligned_Load_X86_ISA_LEVEL 3 -/* - * KNL (the only cpu that sets this supported in cpu-features.h) - * builds with ISA V1 so this shouldn't harm any architectures. - */ +/* NB: This feature is disabled when ISA level >= 3, which was enabled + for the following CPUs: + - Intel KNL + when ISA level < 3. */ #define Prefer_No_VZEROUPPER_X86_ISA_LEVEL 3 + +/* ISA level >= 2 guaranteed includes. */ +#define Fast_Unaligned_Load_X86_ISA_LEVEL 2 + #define ISA_SHOULD_BUILD(isa_build_level) \ (MINIMUM_X86_ISA_LEVEL <= (isa_build_level) && IS_IN (libc)) \ || defined ISA_DEFAULT_IMPL diff --git a/sysdeps/x86_64/multiarch/ifunc-evex.h b/sysdeps/x86_64/multiarch/ifunc-evex.h index 856c6261f8..310cfd269f 100644 --- a/sysdeps/x86_64/multiarch/ifunc-evex.h +++ b/sysdeps/x86_64/multiarch/ifunc-evex.h @@ -37,7 +37,7 @@ IFUNC_SELECTOR (void) if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, AVX2) && X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, BMI2) && X86_ISA_CPU_FEATURES_ARCH_P (cpu_features, - AVX_Fast_Unaligned_Load)) + AVX_Fast_Unaligned_Load, )) { if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, AVX512VL) && X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, AVX512BW)) @@ -52,7 +52,7 @@ IFUNC_SELECTOR (void) return OPTIMIZE (avx2_rtm); if (X86_ISA_CPU_FEATURES_ARCH_P (cpu_features, - Prefer_No_VZEROUPPER)) + Prefer_No_VZEROUPPER, !)) return OPTIMIZE (avx2); }