From patchwork Fri Oct 11 20:08:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Wakely X-Patchwork-Id: 1996283 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=fFCBPlM3; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XQHm765Fpz1xt1 for ; Sat, 12 Oct 2024 07:09:58 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 3365E3858433 for ; Fri, 11 Oct 2024 20:09:56 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTP id 2A4083857835 for ; Fri, 11 Oct 2024 20:09:28 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 2A4083857835 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 2A4083857835 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1728677370; cv=none; b=McI62yZLr0mjLnAIoIiwsND+owF5BRLeKsN719lrCHxGJRLZ6+BTbsjDjz1XN9zsTzPPi+UCfFwGgWlGyuTbwR11MYfE9Sk+o3M0nRY6J0SGlW/bVMsAnOIormRPMRtLc0djk3DMEwfeMsFNdUOky4tubY/xuKBNpUtOPh4nnw0= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1728677370; c=relaxed/simple; bh=+/nmFwjVlAQisc9cZnwo10arWNydt6WQRhKyMPEJCHM=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=g/0c3EJObUx/CC+eAU0MVl4A0p7qGCjIBz2DSxyx7nT8VKnfOQRqCHAc1TpFJLvqjHYek6nq5YOYKdUB4uxGmvZDu5WqpvK7znu6jS9nIH5UzZCFBouCIJu9PiRKNoo+Y4jebTXkyfX/hDHFSMJprp7O3y6yGidFvn9Isbt6LQg= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728677367; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DPS9yVmAWIR2zB7j+o7qBrfvy9uvN6XiU1kYUjBq2RI=; b=fFCBPlM3ytvKLBd6gXdQleWviYU8RVH8Ay67Vj4H0F5RkfjPipE60esBVarExhwZ+lsZ4H VJZZwEpzHFN8YZJKjiTpGsvQMNiaaYF7pmt8L8dnXCMbwtoUH3ZVh/w8oZwveZjbfiW0+k DVQqx2FvXxN9+VhZT0U9h1pxvEW+YYg= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-491-i-kCUNb2Pd6EFQB3jPEqWQ-1; Fri, 11 Oct 2024 16:09:24 -0400 X-MC-Unique: i-kCUNb2Pd6EFQB3jPEqWQ-1 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7DB281956057; Fri, 11 Oct 2024 20:09:23 +0000 (UTC) Received: from localhost (unknown [10.42.28.16]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9385C30001A3; Fri, 11 Oct 2024 20:09:21 +0000 (UTC) From: Jonathan Wakely To: libstdc++@gcc.gnu.org, gcc-patches@gcc.gnu.org Subject: [PATCH v4 1/2] libstdc++: Enable memcpy optimizations for distinct integral types [PR93059] Date: Fri, 11 Oct 2024 21:08:53 +0100 Message-ID: <20241011200920.1448131-1-jwakely@redhat.com> In-Reply-To: <20241011160103.1354851-1-jwakely@redhat.com> References: <20241011160103.1354851-1-jwakely@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Spam-Status: No, score=-12.0 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org Sigh, the v3 patch had a stray "#pragma GCC" in it somehow, so now here's v4. Unchanged apart from removing that. Testing powerpc64le-linux ... -- >8 -- Currently we only optimize std::copy, std::copy_n etc. to memmove when the source and destination types are the same. This means that we fail to optimize copying between distinct 1-byte types, e.g. copying from a buffer of unsigned char to a buffer of char8_t or vice versa. This patch adds more partial specializations of the __memcpyable trait so that we allow memcpy between integers of equal widths. This will enable memmove for copies between narrow character types and also between same-width types like int and unsigned. Enabling the optimization needs to be based on the width of the integer type, not just the size in bytes. This is because some targets define non-standard integral types such as __int20 in msp430, which has padding bits. It would not be safe to memcpy between e.g. __int20 and int32_t, even though sizeof(__int20) == sizeof(int32_t). A new trait is introduced to define the width, __memcpyable_integer, and then the __memcpyable trait compares the widths. It's safe to copy between signed and unsigned integers of the same width, because GCC only supports two's complement integers. I initially though it would be useful to define the specialization __memcpyable_integer to enable copying between narrow character types and std::byte. But that isn't possible with std::copy, because is_assignable is false. Optimized copies using memmove will already happen for copying std::byte to std::byte, because __memcpyable is true. libstdc++-v3/ChangeLog: PR libstdc++/93059 * include/bits/cpp_type_traits.h (__memcpyable): Add partial specialization for pointers to distinct types. (__memcpyable_integer): New trait to control which types can use cross-type memcpy optimizations. --- libstdc++-v3/include/bits/cpp_type_traits.h | 89 ++++++++++++++++++++- 1 file changed, 87 insertions(+), 2 deletions(-) diff --git a/libstdc++-v3/include/bits/cpp_type_traits.h b/libstdc++-v3/include/bits/cpp_type_traits.h index 060652afb18..2f9ce75e82c 100644 --- a/libstdc++-v3/include/bits/cpp_type_traits.h +++ b/libstdc++-v3/include/bits/cpp_type_traits.h @@ -434,8 +434,6 @@ __INT_N(__GLIBCXX_TYPE_INT_N_3) }; #endif - template struct iterator_traits; - // A type that is safe for use with memcpy, memmove, memcmp etc. template struct __is_nonvolatile_trivially_copyable @@ -459,16 +457,103 @@ __INT_N(__GLIBCXX_TYPE_INT_N_3) enum { __value = 0 }; }; + // Allow memcpy when source and destination are pointers to the same type. template struct __memcpyable<_Tp*, _Tp*> : __is_nonvolatile_trivially_copyable<_Tp> { }; + // Source pointer can be const. template struct __memcpyable<_Tp*, const _Tp*> : __is_nonvolatile_trivially_copyable<_Tp> { }; + template struct __memcpyable_integer; + + // For heterogeneous types, allow memcpy between equal-sized integers. + template + struct __memcpyable<_Tp*, _Up*> + { + enum { + __value = __memcpyable_integer<_Tp>::__width != 0 + && ((int)__memcpyable_integer<_Tp>::__width + == (int)__memcpyable_integer<_Up>::__width) + }; + }; + + // Specialization for const U* because __is_integer is never true. + template + struct __memcpyable<_Tp*, const _Up*> + : __memcpyable<_Tp*, _Up*> + { }; + + template + struct __memcpyable_integer + { + enum { + __width = __is_integer<_Tp>::__value ? (sizeof(_Tp) * __CHAR_BIT__) : 0 + }; + }; + + // Cannot memcpy volatile memory. + template + struct __memcpyable_integer + { enum { __width = 0 }; }; + + // Specializations for __intNN types with padding bits. +#if defined __GLIBCXX_TYPE_INT_N_0 && __GLIBCXX_BITSIZE_INT_N_0 % __CHAR_BIT__ + __extension__ + template<> + struct __memcpyable_integer<__GLIBCXX_TYPE_INT_N_0> + { enum { __width = __GLIBCXX_BITSIZE_INT_N_0 }; }; + __extension__ + template<> + struct __memcpyable_integer + { enum { __width = __GLIBCXX_BITSIZE_INT_N_0 }; }; +#endif +#if defined __GLIBCXX_TYPE_INT_N_1 && __GLIBCXX_BITSIZE_INT_N_1 % __CHAR_BIT__ + __extension__ + template<> + struct __memcpyable_integer<__GLIBCXX_TYPE_INT_N_1> + { enum { __width = __GLIBCXX_BITSIZE_INT_N_1 }; }; + __extension__ + template<> + struct __memcpyable_integer + { enum { __width = __GLIBCXX_BITSIZE_INT_N_1 }; }; +#endif +#if defined __GLIBCXX_TYPE_INT_N_2 && __GLIBCXX_BITSIZE_INT_N_2 % __CHAR_BIT__ + __extension__ + template<> + struct __memcpyable_integer<__GLIBCXX_TYPE_INT_N_2> + { enum { __width = __GLIBCXX_BITSIZE_INT_N_2 }; }; + __extension__ + template<> + struct __memcpyable_integer + { enum { __width = __GLIBCXX_BITSIZE_INT_N_2 }; }; +#endif +#if defined __GLIBCXX_TYPE_INT_N_3 && __GLIBCXX_BITSIZE_INT_N_3 % __CHAR_BIT__ + __extension__ + template<> + struct __memcpyable_integer<__GLIBCXX_TYPE_INT_N_3> + { enum { __width = __GLIBCXX_BITSIZE_INT_N_3 }; }; + __extension__ + template<> + struct __memcpyable_integer + { enum { __width = __GLIBCXX_BITSIZE_INT_N_3 }; }; +#endif + +#if defined __STRICT_ANSI__ && defined __SIZEOF_INT128__ + // In strict modes __is_integer<__int128> is false, + // but we want to allow memcpy between signed/unsigned __int128. + __extension__ + template<> + struct __memcpyable_integer<__int128> { enum { __width = 128 }; }; + __extension__ + template<> + struct __memcpyable_integer { enum { __width = 128 }; }; +#endif + // Whether two iterator types can be used with memcmp. // This trait only says it's well-formed to use memcmp, not that it // gives the right answer for a given algorithm. So for example, std::equal