From patchwork Wed Jun 15 00:25:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Noah Goldstein X-Patchwork-Id: 1643450 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=wKgPnjlH; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LN5gl5j77z9vGv for ; Wed, 15 Jun 2022 10:25:55 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id CFD98384145C for ; Wed, 15 Jun 2022 00:25:52 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org CFD98384145C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1655252752; bh=cxcKBp+vvOuHbvabuG4TCNDZEoGzg4QJmdwk429SRQY=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=wKgPnjlHwOZbT1KsOiJv79Fjt/T21rhQXaCwGybRrpgtLSC+56j3W4FF3fkwEKsHG W6I7QldwjTfGUaXlkDm8XwWLfcJIPs3SmtRQdKoWHf9K/NY/wktBTDUq2VzVsxc6y0 QXLSdvkgnTxLRTQeua7lrrqcgteEpAFL8w8XZnL4= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pj1-x1033.google.com (mail-pj1-x1033.google.com [IPv6:2607:f8b0:4864:20::1033]) by sourceware.org (Postfix) with ESMTPS id 1A01D3858413 for ; Wed, 15 Jun 2022 00:25:38 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 1A01D3858413 Received: by mail-pj1-x1033.google.com with SMTP id k12-20020a17090a404c00b001eaabc1fe5dso568272pjg.1 for ; Tue, 14 Jun 2022 17:25:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=cxcKBp+vvOuHbvabuG4TCNDZEoGzg4QJmdwk429SRQY=; b=ffraIyCefXsOYwr7yT0UpdG1bBjFkaYiLYXzNOUAINP6ymYEA/xoFV56uARTFDzKJO wZWybsyXMifg+LdDIU2bu4OW00A/FG87XWemgFEPsnoQJUy7Pkafofk0sVmuM0zyMyjJ ZEqEYFg7dqgtTILJy0amEcgFjdhaczPp018Z6gzUqgVWpewSwu2UCnjoW6P2RduQTizH EZhJA3QzX9XARwjOWbji9tR3yysKhyhwMqkyhDFRExazeuaFMSt0pNvFfsZE0NlY2ljO yVlm6BA2Jk7F78GUPXcL47yScqyzel1NpNXvcFcvGNL6Z2ADHp6H98MA4oc/a655114E eTJw== X-Gm-Message-State: AJIora8ukfln4RzOA60sXiG7iONwaN7XV27jFyaw6LcSKMjAvrTTulOT 0HMWwb0jAwusDVpdmw9JesC/+Wj4BOw= X-Google-Smtp-Source: AGRyM1tcNHjITBOcr49AylGhEQubFsl2EpVkDFe9Kh+uaHOVJo38ySgvFFnanwCq0suv64Uv/oCOLQ== X-Received: by 2002:a17:902:f548:b0:167:5c83:3adb with SMTP id h8-20020a170902f54800b001675c833adbmr6512404plf.70.1655252736871; Tue, 14 Jun 2022 17:25:36 -0700 (PDT) Received: from noah-tgl.. ([2600:1010:b00a:24b5:2ca1:5b17:18d:7e5e]) by smtp.gmail.com with ESMTPSA id p1-20020a170903248100b0016796cdd802sm7877530plw.19.2022.06.14.17.25.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Jun 2022 17:25:36 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v1 1/3] x86: Fix misordered logic for setting `rep_movsb_stop_threshold` Date: Tue, 14 Jun 2022 17:25:31 -0700 Message-Id: <20220615002533.1741934-1-goldstein.w.n@gmail.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Spam-Status: No, score=-12.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Noah Goldstein via Libc-alpha From: Noah Goldstein Reply-To: Noah Goldstein Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" Move the setting of `rep_movsb_stop_threshold` to after the tunables have been collected so that the `rep_movsb_stop_threshold` (which is used to redirect control flow to the non_temporal case) will use any user value for `non_temporal_threshold` (set using glibc.cpu.x86_non_temporal_threshold) --- sysdeps/x86/dl-cacheinfo.h | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/sysdeps/x86/dl-cacheinfo.h b/sysdeps/x86/dl-cacheinfo.h index f64a2fb0ba..cc3b840f9c 100644 --- a/sysdeps/x86/dl-cacheinfo.h +++ b/sysdeps/x86/dl-cacheinfo.h @@ -898,18 +898,6 @@ dl_init_cacheinfo (struct cpu_features *cpu_features) if (CPU_FEATURE_USABLE_P (cpu_features, FSRM)) rep_movsb_threshold = 2112; - unsigned long int rep_movsb_stop_threshold; - /* ERMS feature is implemented from AMD Zen3 architecture and it is - performing poorly for data above L2 cache size. Henceforth, adding - an upper bound threshold parameter to limit the usage of Enhanced - REP MOVSB operations and setting its value to L2 cache size. */ - if (cpu_features->basic.kind == arch_kind_amd) - rep_movsb_stop_threshold = core; - /* Setting the upper bound of ERMS to the computed value of - non-temporal threshold for architectures other than AMD. */ - else - rep_movsb_stop_threshold = non_temporal_threshold; - /* The default threshold to use Enhanced REP STOSB. */ unsigned long int rep_stosb_threshold = 2048; @@ -951,6 +939,18 @@ dl_init_cacheinfo (struct cpu_features *cpu_features) SIZE_MAX); #endif + unsigned long int rep_movsb_stop_threshold; + /* ERMS feature is implemented from AMD Zen3 architecture and it is + performing poorly for data above L2 cache size. Henceforth, adding + an upper bound threshold parameter to limit the usage of Enhanced + REP MOVSB operations and setting its value to L2 cache size. */ + if (cpu_features->basic.kind == arch_kind_amd) + rep_movsb_stop_threshold = core; + /* Setting the upper bound of ERMS to the computed value of + non-temporal threshold for architectures other than AMD. */ + else + rep_movsb_stop_threshold = non_temporal_threshold; + cpu_features->data_cache_size = data; cpu_features->shared_cache_size = shared; cpu_features->non_temporal_threshold = non_temporal_threshold; From patchwork Wed Jun 15 00:25:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Noah Goldstein X-Patchwork-Id: 1643451 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=BEk4aH06; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Received: from sourceware.org (ip-8-43-85-97.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LN5hZ1cbkz9vGv for ; Wed, 15 Jun 2022 10:26:38 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A84D7388454A for ; Wed, 15 Jun 2022 00:26:34 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org A84D7388454A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1655252794; bh=7oq86W5/nc3Zq4MSI5gp0JMK3Yxj7W3aqIMW+y8Jh40=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=BEk4aH06F4ipj5d1uWwgy+xKevRA3zCtba0OQZef17xouX+rsO28oEYBUxVzPP4BB jaa1ENEYRjEZI7Meyvjhb7UFsT/8Ut0xl0u5FdW8KvEkuKGPhtTW7FfhyiVC5HGhel tY2UI3CmAlYXvZmAQQqvYwpSPUYJzjEnxb/wYSqk= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by sourceware.org (Postfix) with ESMTPS id 611F23858413 for ; Wed, 15 Jun 2022 00:25:40 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 611F23858413 Received: by mail-pl1-x636.google.com with SMTP id r1so9045526plo.10 for ; Tue, 14 Jun 2022 17:25:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7oq86W5/nc3Zq4MSI5gp0JMK3Yxj7W3aqIMW+y8Jh40=; b=qMr6OyPIW7rdRR73zTQKxfNE00JnmT6VyOBqOxQXJJys1W5hR6BgltuDgjROfmHVVP z/epNJQnq08uBNFxiVxZuDLSWfHMYSBiUTPw2wDkxMxRdYDH+C9fMQUdfVbeJGIGIPql CsxeciaK8Z4PQNES5ladpR305zqG84THnH9Xv6CpAMal+RAs1My7RxAIFICDuNNjORkr l/NDo1oyNVq0lNhIw9VEHvfClzH76fdLCFobpYQOsZVrcAL44muSr7Us1rbCvDAU1Eel jSqJ+cqe/JYLdnRbvm8pNBnsthgphiQL4ZIWP+aq5Q7mVhkaaaNPpuxitHUvo82zaOsR UY2g== X-Gm-Message-State: AJIora98xM3x4pqdxsS+YsjzEss+2U0iOEPS7QpqVjS+y8ug9ZStiZpp JzfG6INxj/Yk9RyqbhN/OXpFzMWTcu4= X-Google-Smtp-Source: AGRyM1smI9LANBEk8Xor5TTf1w0aJaYtUA1z3+T+uo0QnkUeh7as1A8dCnH7/D1yUuSEJEXJIwbwlA== X-Received: by 2002:a17:903:240f:b0:168:f037:98c8 with SMTP id e15-20020a170903240f00b00168f03798c8mr4401901plo.53.1655252739295; Tue, 14 Jun 2022 17:25:39 -0700 (PDT) Received: from noah-tgl.. ([2600:1010:b00a:24b5:2ca1:5b17:18d:7e5e]) by smtp.gmail.com with ESMTPSA id p1-20020a170903248100b0016796cdd802sm7877530plw.19.2022.06.14.17.25.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Jun 2022 17:25:38 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v1 2/3] x86: Cleanup bounds checking in large memcpy case Date: Tue, 14 Jun 2022 17:25:32 -0700 Message-Id: <20220615002533.1741934-2-goldstein.w.n@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220615002533.1741934-1-goldstein.w.n@gmail.com> References: <20220615002533.1741934-1-goldstein.w.n@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Noah Goldstein via Libc-alpha From: Noah Goldstein Reply-To: Noah Goldstein Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" 1. Fix incorrect lower-bound threshold in L(large_memcpy_2x). Previously was using `__x86_rep_movsb_threshold` and should have been using `__x86_shared_non_temporal_threshold`. 2. Avoid reloading __x86_shared_non_temporal_threshold before the L(large_memcpy_4x) bounds check. 3. Document the second bounds check for L(large_memcpy_4x) more clearly. --- manual/tunables.texi | 2 +- sysdeps/x86/dl-cacheinfo.h | 8 +++-- .../multiarch/memmove-vec-unaligned-erms.S | 29 ++++++++++++++----- 3 files changed, 28 insertions(+), 11 deletions(-) diff --git a/manual/tunables.texi b/manual/tunables.texi index 1482412078..49daf3eb4a 100644 --- a/manual/tunables.texi +++ b/manual/tunables.texi @@ -47,7 +47,7 @@ glibc.malloc.mxfast: 0x0 (min: 0x0, max: 0xffffffffffffffff) glibc.elision.skip_lock_busy: 3 (min: -2147483648, max: 2147483647) glibc.malloc.top_pad: 0x0 (min: 0x0, max: 0xffffffffffffffff) glibc.cpu.x86_rep_stosb_threshold: 0x800 (min: 0x1, max: 0xffffffffffffffff) -glibc.cpu.x86_non_temporal_threshold: 0xc0000 (min: 0x0, max: 0xffffffffffffffff) +glibc.cpu.x86_non_temporal_threshold: 0xc0000 (min: 0x0, max: 0x0fffffffffffffff) glibc.cpu.x86_shstk: glibc.cpu.hwcap_mask: 0x6 (min: 0x0, max: 0xffffffffffffffff) glibc.malloc.mmap_max: 0 (min: -2147483648, max: 2147483647) diff --git a/sysdeps/x86/dl-cacheinfo.h b/sysdeps/x86/dl-cacheinfo.h index cc3b840f9c..a66152d9cc 100644 --- a/sysdeps/x86/dl-cacheinfo.h +++ b/sysdeps/x86/dl-cacheinfo.h @@ -915,9 +915,13 @@ dl_init_cacheinfo (struct cpu_features *cpu_features) shared = tunable_size; tunable_size = TUNABLE_GET (x86_non_temporal_threshold, long int, NULL); - /* NB: Ignore the default value 0. */ - if (tunable_size != 0) + /* NB: Ignore the default value 0. Saturate very large values at + LONG_MAX >> 4. */ + if (tunable_size != 0 && tunable_size <= (LONG_MAX >> 3)) non_temporal_threshold = tunable_size; + /* Saturate huge arguments. */ + else if (tunable_size != 0) + non_temporal_threshold = LONG_MAX >> 3; tunable_size = TUNABLE_GET (x86_rep_movsb_threshold, long int, NULL); if (tunable_size > minimum_rep_movsb_threshold) diff --git a/sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S b/sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S index af51177d5d..d1518b8bab 100644 --- a/sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S +++ b/sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S @@ -118,7 +118,13 @@ # define LARGE_LOAD_SIZE (VEC_SIZE * 4) #endif -/* Amount to shift rdx by to compare for memcpy_large_4x. */ +/* Amount to shift __x86_shared_non_temporal_threshold by for + bound for memcpy_large_4x. This is essentially use to to + indicate that the copy is far beyond the scope of L3 + (assuming no user config x86_non_temporal_threshold) and to + use a more aggressively unrolled loop. NB: before + increasing the value also update initialization of + x86_non_temporal_threshold. */ #ifndef LOG_4X_MEMCPY_THRESH # define LOG_4X_MEMCPY_THRESH 4 #endif @@ -724,9 +730,14 @@ L(skip_short_movsb_check): .p2align 4,, 10 #if (defined USE_MULTIARCH || VEC_SIZE == 16) && IS_IN (libc) L(large_memcpy_2x_check): - cmp __x86_rep_movsb_threshold(%rip), %RDX_LP - jb L(more_8x_vec_check) + /* Entry from L(large_memcpy_2x) has a redundant load of + __x86_shared_non_temporal_threshold(%rip). L(large_memcpy_2x) + is only use for the non-erms memmove which is generally less + common. */ L(large_memcpy_2x): + mov __x86_shared_non_temporal_threshold(%rip), %R11_LP + cmp %R11_LP, %RDX_LP + jb L(more_8x_vec_check) /* To reach this point it is impossible for dst > src and overlap. Remaining to check is src > dst and overlap. rcx already contains dst - src. Negate rcx to get src - dst. If @@ -774,18 +785,21 @@ L(large_memcpy_2x): /* ecx contains -(dst - src). not ecx will return dst - src - 1 which works for testing aliasing. */ notl %ecx + movq %rdx, %r10 testl $(PAGE_SIZE - VEC_SIZE * 8), %ecx jz L(large_memcpy_4x) - movq %rdx, %r10 - shrq $LOG_4X_MEMCPY_THRESH, %r10 - cmp __x86_shared_non_temporal_threshold(%rip), %r10 + /* r11 has __x86_shared_non_temporal_threshold. Shift it left + by LOG_4X_MEMCPY_THRESH to get L(large_memcpy_4x) threshold. + */ + shlq $LOG_4X_MEMCPY_THRESH, %r11 + cmp %r11, %rdx jae L(large_memcpy_4x) /* edx will store remainder size for copying tail. */ andl $(PAGE_SIZE * 2 - 1), %edx /* r10 stores outer loop counter. */ - shrq $((LOG_PAGE_SIZE + 1) - LOG_4X_MEMCPY_THRESH), %r10 + shrq $(LOG_PAGE_SIZE + 1), %r10 /* Copy 4x VEC at a time from 2 pages. */ .p2align 4 L(loop_large_memcpy_2x_outer): @@ -850,7 +864,6 @@ L(large_memcpy_2x_end): .p2align 4 L(large_memcpy_4x): - movq %rdx, %r10 /* edx will store remainder size for copying tail. */ andl $(PAGE_SIZE * 4 - 1), %edx /* r10 stores outer loop counter. */ From patchwork Wed Jun 15 00:25:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Noah Goldstein X-Patchwork-Id: 1643452 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=GBk+5nxi; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LN5jR6w8Cz9vGv for ; Wed, 15 Jun 2022 10:27:23 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 1BBCA384B82B for ; Wed, 15 Jun 2022 00:27:22 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 1BBCA384B82B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1655252842; bh=ajCvJTsH/tvpQt+PLBmfpjsPJIBwvL8M2w05/GHXrW4=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=GBk+5nxin6vr3B2TJD/LRRyNLMwnZrfhSDwqq0/IyWeerhpiGMF//Emk+gfx5UmED LtRm8owfCRsKEHgmGoRcsD2HZiVz2/ffLfcmHpCq8canQo+obHdPr5xdt5lCmO0IL/ uXCgpUNEwHIOiqK2nt6HVVffNj7jcQLfWXMMb60k= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by sourceware.org (Postfix) with ESMTPS id 3580E38300B0 for ; Wed, 15 Jun 2022 00:25:42 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 3580E38300B0 Received: by mail-pf1-x42d.google.com with SMTP id 187so9971641pfu.9 for ; Tue, 14 Jun 2022 17:25:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ajCvJTsH/tvpQt+PLBmfpjsPJIBwvL8M2w05/GHXrW4=; b=IxbrVbFnKIwo6wbEAyCL24fNI5nfkPcuIqp2FvUapECM/e5Yhj9UenogsYmZLQlUIo 86sfMnU6mg4NcUSwVjCHscuirT0ZOeZF2WMArtPl6MmRDwuRIOFZ6tfVLuDGcLvNSXNo dtXBKWwEdh761Fob+yjnqDMg9raePyDUTTJlSSLoza8niDqotg+02zjChuZ1ZrdwaWWN WyE8sbmBYe+8XvPiqQdFLqk0XqkybIxbg4w6BzFwrfd9rLyQruia7WlT/3m0wLWois7E ByPr4AVtf5ep8snyQWk5iC4HpWN8+Ba6IGMhLLSr8mSGTbPOrjAkNxQ74ME7eRT3VgZ5 x7OQ== X-Gm-Message-State: AOAM532y0otB64pP/yM9/1aSx1rNfHqo2XsNwJSHp9VhB+G+4Jca04Ur lnBEeGWzhFHTskbC1HL0N/kguGcpo5w= X-Google-Smtp-Source: ABdhPJwY39aT9nCmhckK6JsSCyzdCJXMybDbmLHLi/EzS167QOFPJu7SKeIY2kr60Vgx0dDIFmjx7w== X-Received: by 2002:a05:6a00:c89:b0:51c:2ad8:47ad with SMTP id a9-20020a056a000c8900b0051c2ad847admr7219646pfv.42.1655252741123; Tue, 14 Jun 2022 17:25:41 -0700 (PDT) Received: from noah-tgl.. ([2600:1010:b00a:24b5:2ca1:5b17:18d:7e5e]) by smtp.gmail.com with ESMTPSA id p1-20020a170903248100b0016796cdd802sm7877530plw.19.2022.06.14.17.25.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Jun 2022 17:25:40 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v1 3/3] x86: Add sse42 implementation to strcmp's ifunc Date: Tue, 14 Jun 2022 17:25:33 -0700 Message-Id: <20220615002533.1741934-3-goldstein.w.n@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220615002533.1741934-1-goldstein.w.n@gmail.com> References: <20220615002533.1741934-1-goldstein.w.n@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Noah Goldstein via Libc-alpha From: Noah Goldstein Reply-To: Noah Goldstein Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" This has been missing since the the ifuncs where added. The performance of SSE4.2 is preferable to to SSE2. Measured on Tigerlake with N = 20 runs. Geometric Mean of all benchmarks SSE4.2 / SSE2: 0.906 --- sysdeps/x86_64/multiarch/strcmp.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/sysdeps/x86_64/multiarch/strcmp.c b/sysdeps/x86_64/multiarch/strcmp.c index a248c2a6e6..9c1677724c 100644 --- a/sysdeps/x86_64/multiarch/strcmp.c +++ b/sysdeps/x86_64/multiarch/strcmp.c @@ -28,6 +28,7 @@ extern __typeof (REDIRECT_NAME) OPTIMIZE (sse2) attribute_hidden; extern __typeof (REDIRECT_NAME) OPTIMIZE (sse2_unaligned) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (sse42) attribute_hidden; extern __typeof (REDIRECT_NAME) OPTIMIZE (avx2) attribute_hidden; extern __typeof (REDIRECT_NAME) OPTIMIZE (avx2_rtm) attribute_hidden; extern __typeof (REDIRECT_NAME) OPTIMIZE (evex) attribute_hidden; @@ -52,6 +53,10 @@ IFUNC_SELECTOR (void) return OPTIMIZE (avx2); } + if (CPU_FEATURE_USABLE_P (cpu_features, SSE4_2) + && !CPU_FEATURES_ARCH_P (cpu_features, Slow_SSE4_2)) + return OPTIMIZE (sse42); + if (CPU_FEATURES_ARCH_P (cpu_features, Fast_Unaligned_Load)) return OPTIMIZE (sse2_unaligned);