From patchwork Wed Jun 8 21:34:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Noah Goldstein X-Patchwork-Id: 1640913 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=IQpn2rk2; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LJL9p4M4tz9sGC for ; Thu, 9 Jun 2022 07:35:26 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 109C838CA686 for ; Wed, 8 Jun 2022 21:35:19 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 109C838CA686 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1654724119; bh=+KI9fkWGAEcUjZr1jxYuPTwpQBn+0zZs/uRUHgKJAvM=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=IQpn2rk2kZjVJRXzXCQ+VYAsyyTYO65IJBKbi87Yb3dlFYG6L8kOxd4q4QdeD8Vtz BacPcdNILZr+fZ8Ew33wEKseNkhGTth5s4Yua8dEMsFO+fE9uAvbozosh9+3VpNRUr 9Vb5hsybydXfPwdvnQjgbFbfToo5jbLSPJTO9c3c= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by sourceware.org (Postfix) with ESMTPS id 875A9383A339 for ; Wed, 8 Jun 2022 21:35:03 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 875A9383A339 Received: by mail-pj1-x1032.google.com with SMTP id o6-20020a17090a0a0600b001e2c6566046so25098319pjo.0 for ; Wed, 08 Jun 2022 14:35:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+KI9fkWGAEcUjZr1jxYuPTwpQBn+0zZs/uRUHgKJAvM=; b=6scE++jXcmR1sE0Agb5c9XXJfXj/yrI16DiA3kmo64Hyyr6U+1kzKVptp5WkMrAMAh Pp3IS9FXwy0Z2YuULm3V5Jw4wyMjGztlOIflfDVRidRSD5TroxZmk3GukZF6Airpb4cI 4UXT8F6bCh1TEbOmVhQixMQZzgVFsYlzeMgDVhbslXE7ismNAvs+bc5B6fO6qmhrJXNJ GdEueutGppom8PHEuKsOzlVJMRq/iFC7qwKBwXMqn4ZWrRw+540I1ku4TYQmyfNUt/WI QY8EYa1cDoE5PBwIn+/S4+DoN6Qar9ypIabuSBMBJrSvi6n1qisfUliu4d89lIj29e7P ebUw== X-Gm-Message-State: AOAM532bv3EcZfS1+W1fo7I/vGIeAVOi8TvK/7xkgZqU1nneCAuAVcd3 Dg5Ov+GS4UrUFTpPSIq3QDIEpeJrafbNTw== X-Google-Smtp-Source: ABdhPJwVk0/oh23P9BR+0NhxkEB9gwHB2P/JJrGhcrKeIqrsioVRaKN9ciWKr6n/E8QAzFCBzBFgyg== X-Received: by 2002:a17:90b:314b:b0:1e3:1033:f555 with SMTP id ip11-20020a17090b314b00b001e31033f555mr1196012pjb.245.1654724102393; Wed, 08 Jun 2022 14:35:02 -0700 (PDT) Received: from noah-tgl.. ([2600:1010:b04a:6ef:9b9d:71d8:5e82:2442]) by smtp.gmail.com with ESMTPSA id 84-20020a621857000000b0050dc7628158sm15558733pfy.50.2022.06.08.14.35.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jun 2022 14:35:02 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v2] x86: Fix page cross case in rawmemchr-avx2 [BZ #29234] Date: Wed, 8 Jun 2022 14:34:59 -0700 Message-Id: <20220608213459.3348847-1-goldstein.w.n@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-12.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Noah Goldstein via Libc-alpha From: Noah Goldstein Reply-To: Noah Goldstein Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" commit 6dcbb7d95dded20153b12d76d2f4e0ef0cda4f35 Author: Noah Goldstein Date: Mon Jun 6 21:11:33 2022 -0700 x86: Shrink code size of memchr-avx2.S Changed how the page cross case aligned string (rdi) in rawmemchr. This was incompatible with how `L(cross_page_continue)` expected the pointer to be aligned and would cause rawmemchr to read data start started before the beginning of the string. What it would read was in valid memory but could count CHAR matches resulting in an incorrect return value. This commit fixes that issue by essentially reverting the changes to the L(page_cross) case as they didn't really matter. Test cases added and all pass with the new code (and where confirmed to fail with the old code). Reviewed-by: H.J. Lu --- string/test-rawmemchr.c | 57 +++++++++++++++++++++++++- sysdeps/x86_64/multiarch/memchr-avx2.S | 16 ++++---- 2 files changed, 64 insertions(+), 9 deletions(-) diff --git a/string/test-rawmemchr.c b/string/test-rawmemchr.c index cafb75298a..703e8ec27c 100644 --- a/string/test-rawmemchr.c +++ b/string/test-rawmemchr.c @@ -17,6 +17,7 @@ . */ #include +#include #define TEST_MAIN #define TEST_NAME "rawmemchr" @@ -50,13 +51,45 @@ do_one_test (impl_t *impl, const char *s, int c, char *exp_res) } } +static void +do_test_bz29234 (void) +{ + size_t i, j; + char *ptr_start; + char *buf = xmmap (0, 8192, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1); + + memset (buf, -1, 8192); + + ptr_start = buf + 4096 - 8; + + /* Out of range matches before the start of a page. */ + memset (ptr_start - 8, 0x1, 8); + + for (j = 0; j < 8; ++j) + { + for (i = 0; i < 128; ++i) + { + ptr_start[i + j] = 0x1; + + FOR_EACH_IMPL (impl, 0) + do_one_test (impl, (char *) (ptr_start + j), 0x1, + ptr_start + i + j); + + ptr_start[i + j] = 0xff; + } + } + + xmunmap (buf, 8192); +} + static void do_test (size_t align, size_t pos, size_t len, int seek_char) { size_t i; char *result; - align &= 7; + align &= getpagesize () - 1; if (align + len >= page_size) return; @@ -114,6 +147,13 @@ do_random_tests (void) } } + if (align) + { + p[align - 1] = seek_char; + if (align > 4) + p[align - 4] = seek_char; + } + assert (pos < len); size_t r = random (); if ((r & 31) == 0) @@ -129,6 +169,13 @@ do_random_tests (void) result, p); ret = 1; } + + if (align) + { + p[align - 1] = seek_char; + if (align > 4) + p[align - 4] = seek_char; + } } } @@ -150,14 +197,22 @@ test_main (void) do_test (i, 64, 256, 23); do_test (0, 16 << i, 2048, 0); do_test (i, 64, 256, 0); + + do_test (getpagesize () - i, 64, 256, 23); + do_test (getpagesize () - i, 64, 256, 0); } for (i = 1; i < 32; ++i) { do_test (0, i, i + 1, 23); do_test (0, i, i + 1, 0); + + do_test (getpagesize () - 7, i, i + 1, 23); + do_test (getpagesize () - i / 2, i, i + 1, 23); + do_test (getpagesize () - i, i, i + 1, 23); } do_random_tests (); + do_test_bz29234 (); return ret; } diff --git a/sysdeps/x86_64/multiarch/memchr-avx2.S b/sysdeps/x86_64/multiarch/memchr-avx2.S index 28a01280ec..c5a256eb37 100644 --- a/sysdeps/x86_64/multiarch/memchr-avx2.S +++ b/sysdeps/x86_64/multiarch/memchr-avx2.S @@ -409,19 +409,19 @@ L(cross_page_boundary): computer return address if byte is found or adjusting length if it is not and this is memchr. */ movq %rdi, %rcx - /* Align data to VEC_SIZE. ALGN_PTR_REG is rcx for memchr and rdi for - rawmemchr. */ - andq $-VEC_SIZE, %ALGN_PTR_REG - VPCMPEQ (%ALGN_PTR_REG), %ymm0, %ymm1 + /* Align data to VEC_SIZE - 1. ALGN_PTR_REG is rcx for memchr + and rdi for rawmemchr. */ + orq $(VEC_SIZE - 1), %ALGN_PTR_REG + VPCMPEQ -(VEC_SIZE - 1)(%ALGN_PTR_REG), %ymm0, %ymm1 vpmovmskb %ymm1, %eax # ifndef USE_AS_RAWMEMCHR /* Calculate length until end of page (length checked for a match). */ - leal VEC_SIZE(%ALGN_PTR_REG), %esi - subl %ERAW_PTR_REG, %esi -# ifdef USE_AS_WMEMCHR + leaq 1(%ALGN_PTR_REG), %rsi + subq %RRAW_PTR_REG, %rsi +# ifdef USE_AS_WMEMCHR /* NB: Divide bytes by 4 to get wchar_t count. */ shrl $2, %esi -# endif +# endif # endif /* Remove the leading bytes. */ sarxl %ERAW_PTR_REG, %eax, %eax