From patchwork Thu Jun 8 09:00:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Noah Goldstein X-Patchwork-Id: 1792106 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=O7zzGCJ5; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QcJ966N6xz20X9 for ; Thu, 8 Jun 2023 19:01:14 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 465623856DDA for ; Thu, 8 Jun 2023 09:01:11 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 465623856DDA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1686214871; bh=K2awRqbEWgJMMjI6iUvz5HXm636ThgtSOMC/B3CZ2jQ=; h=To:Cc:Subject:Date:List-Id:List-Unsubscribe:List-Archive: List-Post:List-Help:List-Subscribe:From:Reply-To:From; b=O7zzGCJ5lJQhnjKc7hWCutuPRf//ZOBqZ3nI+noByOMPPdM8ZnjpCs/WTM9ekl4fY kQYdgNQlGmlul21rOeiiWGkN3owgHNPT897cSeZZyO5U3YoQWdQBLeD5yZkReQ2rEl X/h8/unB+pIT+TwEu5h4YhOsJJpl9X2uY2hc2Lec= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-ot1-x336.google.com (mail-ot1-x336.google.com [IPv6:2607:f8b0:4864:20::336]) by sourceware.org (Postfix) with ESMTPS id 4B2B53858C62 for ; Thu, 8 Jun 2023 09:00:54 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 4B2B53858C62 Received: by mail-ot1-x336.google.com with SMTP id 46e09a7af769-6af81142b6dso238265a34.2 for ; Thu, 08 Jun 2023 02:00:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686214853; x=1688806853; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=K2awRqbEWgJMMjI6iUvz5HXm636ThgtSOMC/B3CZ2jQ=; b=Twx7aHbEb/2xX1XOAheaFbRBmzg/7ZD8ho36osq9cb8RCuC9tTG639zfdARCSi+tE1 3hNIBQl0CNh62EoEg3sKe01RdkNscuIyXdsvK3/fmrm4GdwjCUN7MrGd43O5fuMbJzSJ YtqITbaJhPbq+S33S4x8E6p81sps5Fdm77T1oEu7fYaQ0SV7ELjM9K1oAqtqYzX9D6WT oB2M3wAtx/OXFwI76fZJK4ZuWpFQ87lMBbffTmvZYQMoY45szoVG0lJJllA13+R+7yC3 bE3ANhavzblWr8fFdJoztcOvmd7QFzXNDP7bpz+kKI1to1Tad+adzYMgjRBFJU2IvjpB Y7NQ== X-Gm-Message-State: AC+VfDzIWkmD6txIogvE5G6ouBEIq2efgIEcFpkjCTTS/gdjgRwkyUIK XAVvZ29xx7SklAk9mqVUFC+o5kQJzJI= X-Google-Smtp-Source: ACHHUZ5Os8HItrCiaPA0MJynWyMRJtb89jtjuptpsdEWpsgfERnfKgUCS+tSXh3rtQm1zOTD9jWSdQ== X-Received: by 2002:a05:6870:1844:b0:188:1338:fbb6 with SMTP id u4-20020a056870184400b001881338fbb6mr5321633oaf.36.1686214852961; Thu, 08 Jun 2023 02:00:52 -0700 (PDT) Received: from noahgold-desk.lan (2603-8080-1301-76c6-bbb0-ef3c-a689-4ab7.res6.spectrum.com. [2603:8080:1301:76c6:bbb0:ef3c:a689:4ab7]) by smtp.gmail.com with ESMTPSA id yu6-20020a05687c224600b0019eee680504sm611534oab.0.2023.06.08.02.00.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Jun 2023 02:00:52 -0700 (PDT) To: libc-alpha@sourceware.org Cc: goldstein.w.n@gmail.com, hjl.tools@gmail.com, carlos@systemhalted.org Subject: [PATCH v1 1/2] x86: Implement sched_yield syscall for x86 only. Date: Thu, 8 Jun 2023 04:00:49 -0500 Message-Id: <20230608090050.2056824-1-goldstein.w.n@gmail.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Spam-Status: No, score=-12.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Noah Goldstein via Libc-alpha From: Noah Goldstein Reply-To: Noah Goldstein Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" We slightly optimize it by using `vzeroall` before the actual syscall. This returns the SSE, AVX, and ZMM_HI256 xsave/xrstor states to the init-state which allows the imminent context switch to skip saving/restoring those states. --- .../unix/sysv/linux/x86_64/sched-yield-impl.h | 29 ++++++++++ sysdeps/unix/sysv/linux/x86_64/sched_yield.c | 56 +++++++++++++++++++ 2 files changed, 85 insertions(+) create mode 100644 sysdeps/unix/sysv/linux/x86_64/sched-yield-impl.h create mode 100644 sysdeps/unix/sysv/linux/x86_64/sched_yield.c diff --git a/sysdeps/unix/sysv/linux/x86_64/sched-yield-impl.h b/sysdeps/unix/sysv/linux/x86_64/sched-yield-impl.h new file mode 100644 index 0000000000..03622ccea4 --- /dev/null +++ b/sysdeps/unix/sysv/linux/x86_64/sched-yield-impl.h @@ -0,0 +1,29 @@ +/* Yield current process. Linux specific syscall. + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +static int TARGET +SCHED_YIELD (void) +{ + PREPARE_CONTEXT_SWITCH (); + return INLINE_SYSCALL_CALL (sched_yield); +} +#undef TARGET +#undef SCHED_YIELD +#undef PREPARE_CONTEXT_SWITCH diff --git a/sysdeps/unix/sysv/linux/x86_64/sched_yield.c b/sysdeps/unix/sysv/linux/x86_64/sched_yield.c new file mode 100644 index 0000000000..e87acf124b --- /dev/null +++ b/sysdeps/unix/sysv/linux/x86_64/sched_yield.c @@ -0,0 +1,56 @@ +/* clock_nanosleep for x86_64. + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* Only difference is if we have AVX, use vzeroall to clear inuse for SSE, AVX, + and ZMM_HI256 xsave/xrstor state. This enables the init-state optimization + saving overhead on context switches. */ + +#include +#if ISA_SHOULD_BUILD(4) +# include +# define TARGET __attribute__ ((target ("avx"))) +# define PREPARE_CONTEXT_SWITCH() _mm256_zeroall () +# define SCHED_YIELD __sched_yield_avx +# include "sched-yield-impl.h" +#endif +#if ISA_SHOULD_BUILD(2) +# define TARGET +# define PREPARE_CONTEXT_SWITCH() +# define SCHED_YIELD __sched_yield_generic +# include "sched-yield-impl.h" +#endif + +#include +#include + +static inline void * +__sched_yield_ifunc_selector (void) +{ +#if MINIMUM_X86_ISA_LEVEL >= 3 + return __sched_yield_avx; +#else + const struct cpu_features *cpu_features = __get_cpu_features (); + if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, AVX)) + return __sched_yield_avx; + return __sched_yield_generic; +#endif +} + +libc_ifunc (__sched_yield, __sched_yield_ifunc_selector ()); +libc_hidden_def (__sched_yield); +weak_alias (__sched_yield, sched_yield); From patchwork Thu Jun 8 09:00:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Noah Goldstein X-Patchwork-Id: 1792105 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=k+S59SHe; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QcJ966K8mz20WP for ; Thu, 8 Jun 2023 19:01:14 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 9206B3856956 for ; Thu, 8 Jun 2023 09:01:12 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 9206B3856956 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1686214872; bh=dP/6vK+ugjabfbZUfob+XWHOvM24vawxdqKupEo6neA=; h=To:Cc:Subject:Date:In-Reply-To:References:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=k+S59SHe1cDF9j4BNvlFUygHyIs8AmgWXeTKM/hYBtX09heor+sehBCfFiR2GhMM0 A3zhEbL1giQ0R9QknL6156JfByqPbktIs2C1mSZZ84hTnyssgbN/VZE2j8wehNAvlN 0quF7aS9YwmdjFqQYWuKrR/YQWqn2oeExFT3zip8= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-oa1-x36.google.com (mail-oa1-x36.google.com [IPv6:2001:4860:4864:20::36]) by sourceware.org (Postfix) with ESMTPS id 243CC3858C2F for ; Thu, 8 Jun 2023 09:00:56 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 243CC3858C2F Received: by mail-oa1-x36.google.com with SMTP id 586e51a60fabf-19f30256921so368792fac.1 for ; Thu, 08 Jun 2023 02:00:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686214855; x=1688806855; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dP/6vK+ugjabfbZUfob+XWHOvM24vawxdqKupEo6neA=; b=NRBCRBOc66yfO58wuOQaKLRZFCdtQv8XPntLiQqAAgrCdmEiMfPLoGYbHXIIOn1tA3 ZmSmAb3x3yT5F/tcfGsJdWxpO4yASE0ddqZH8JQ3HhcpJhaas27+cjiTz5TUA1l4WVdN f9hfC8T0qaGKK1y7KE9+5ukraRSad6MXsZcCqsNm2kualTVZBy8pDAw8kGkM7GWD8fdK E5dgnj7qwF6RnSc/Y1VlNrtsRjX9k35iPX6xTEKx4G+0ox/zOfXdOHglePwZYCn9LXeV CD+7vFtFmEgYa8GvSgyPqcqyPmCph+AHs0D5Y01BHwXGOmbXBpVzOcj6ECh94UIlHqU+ lW3Q== X-Gm-Message-State: AC+VfDwYsAYytEaQC77tW4QHPqg4uxo+nUS329DIQivne25F2MIwq/oD REtRfLLAqW8fX9dSPYilk7x11X0aFz0= X-Google-Smtp-Source: ACHHUZ4mCVr3eLHTw1BcZz6uh9M+OvZhxGQ9H/o/62Dk60YqIgzkTIoEjxUjawyEr7a0cGXIVD75pA== X-Received: by 2002:a05:6870:c690:b0:19f:55f7:310e with SMTP id cv16-20020a056870c69000b0019f55f7310emr9897326oab.13.1686214854847; Thu, 08 Jun 2023 02:00:54 -0700 (PDT) Received: from noahgold-desk.lan (2603-8080-1301-76c6-bbb0-ef3c-a689-4ab7.res6.spectrum.com. [2603:8080:1301:76c6:bbb0:ef3c:a689:4ab7]) by smtp.gmail.com with ESMTPSA id yu6-20020a05687c224600b0019eee680504sm611534oab.0.2023.06.08.02.00.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Jun 2023 02:00:54 -0700 (PDT) To: libc-alpha@sourceware.org Cc: goldstein.w.n@gmail.com, hjl.tools@gmail.com, carlos@systemhalted.org Subject: [PATCH v1 2/2] x86: Implement clock_nanosleep{_time64} syscall for x86 only. Date: Thu, 8 Jun 2023 04:00:50 -0500 Message-Id: <20230608090050.2056824-2-goldstein.w.n@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230608090050.2056824-1-goldstein.w.n@gmail.com> References: <20230608090050.2056824-1-goldstein.w.n@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Noah Goldstein via Libc-alpha From: Noah Goldstein Reply-To: Noah Goldstein Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" We slightly optimize it by using `vzeroall` before the actual syscall. This returns the SSE, AVX, and ZMM_HI256 xsave/xrstor states to the init-state which allows the imminent context switch to skip saving/restoring those states. --- sysdeps/unix/sysv/linux/clock_nanosleep.c | 33 ++++++-- .../unix/sysv/linux/kernel-posix-cpu-timers.h | 4 + .../unix/sysv/linux/x86_64/clock_nanosleep.c | 82 +++++++++++++++++++ 3 files changed, 113 insertions(+), 6 deletions(-) create mode 100644 sysdeps/unix/sysv/linux/x86_64/clock_nanosleep.c diff --git a/sysdeps/unix/sysv/linux/clock_nanosleep.c b/sysdeps/unix/sysv/linux/clock_nanosleep.c index ac2d810632..31a2aa24af 100644 --- a/sysdeps/unix/sysv/linux/clock_nanosleep.c +++ b/sysdeps/unix/sysv/linux/clock_nanosleep.c @@ -24,10 +24,19 @@ #include +#ifndef CLOCK_NANOSLEEP_TIME64 +# define CLOCK_NANOSLEEP_TIME64 __clock_nanosleep_time64 +# define CLOCK_NANOSLEEP __clock_nanosleep +# define STATIC +# define TARGET +# define MAKE_DEFS +# define PREPARE_CONTEXT_SWITCH() +#endif + /* We can simply use the syscall. The CPU clocks are not supported with this function. */ -int -__clock_nanosleep_time64 (clockid_t clock_id, int flags, +STATIC int TARGET +CLOCK_NANOSLEEP_TIME64 (clockid_t clock_id, int flags, const struct __timespec64 *req, struct __timespec64 *rem) { @@ -44,6 +53,7 @@ __clock_nanosleep_time64 (clockid_t clock_id, int flags, #endif int r; + PREPARE_CONTEXT_SWITCH (); #ifdef __ASSUME_TIME64_SYSCALLS r = INTERNAL_SYSCALL_CANCEL (clock_nanosleep_time64, clock_id, flags, req, rem); @@ -72,17 +82,19 @@ __clock_nanosleep_time64 (clockid_t clock_id, int flags, } #if __TIMESIZE != 64 +# ifdef MAKE_DEFS libc_hidden_def (__clock_nanosleep_time64) +# endif -int -__clock_nanosleep (clockid_t clock_id, int flags, const struct timespec *req, +STATIC int +CLOCK_NANOSLEEP (clockid_t clock_id, int flags, const struct timespec *req, struct timespec *rem) { int r; struct __timespec64 treq64, trem64; treq64 = valid_timespec_to_timespec64 (*req); - r = __clock_nanosleep_time64 (clock_id, flags, &treq64, + r = CLOCK_NANOSLEEP_TIME64 (clock_id, flags, &treq64, rem != NULL ? &trem64 : NULL); if (r == EINTR && rem != NULL && (flags & TIMER_ABSTIME) == 0) @@ -91,11 +103,20 @@ __clock_nanosleep (clockid_t clock_id, int flags, const struct timespec *req, return r; } #endif +#ifdef MAKE_DEFS libc_hidden_def (__clock_nanosleep) versioned_symbol (libc, __clock_nanosleep, clock_nanosleep, GLIBC_2_17); /* clock_nanosleep moved to libc in version 2.17; old binaries may expect the symbol version it had in librt. */ -#if SHLIB_COMPAT (libc, GLIBC_2_2, GLIBC_2_17) +# if SHLIB_COMPAT (libc, GLIBC_2_2, GLIBC_2_17) strong_alias (__clock_nanosleep, __clock_nanosleep_2); compat_symbol (libc, __clock_nanosleep_2, clock_nanosleep, GLIBC_2_2); +# endif #endif + +#undef CLOCK_NANOSLEEP_TIME64 +#undef CLOCK_NANOSLEEP +#undef STATIC +#undef TARGET +#undef MAKE_DEFS +#undef PREPARE_CONTEXT_SWITCH diff --git a/sysdeps/unix/sysv/linux/kernel-posix-cpu-timers.h b/sysdeps/unix/sysv/linux/kernel-posix-cpu-timers.h index bea1e0e62d..76a3be9e0d 100644 --- a/sysdeps/unix/sysv/linux/kernel-posix-cpu-timers.h +++ b/sysdeps/unix/sysv/linux/kernel-posix-cpu-timers.h @@ -1,3 +1,6 @@ +#ifndef _KERNEL_POSIX_CPU_TIMERS_H +#define _KERNEL_POSIX_CPU_TIMERS_H + /* Parameters for the Linux kernel ABI for CPU clocks, the bit fields within a clockid: @@ -34,3 +37,4 @@ make_thread_cpuclock (unsigned int tid, clockid_t clock) #define PROCESS_CLOCK make_process_cpuclock (0, CPUCLOCK_SCHED) #define THREAD_CLOCK make_thread_cpuclock (0, CPUCLOCK_SCHED) +#endif diff --git a/sysdeps/unix/sysv/linux/x86_64/clock_nanosleep.c b/sysdeps/unix/sysv/linux/x86_64/clock_nanosleep.c new file mode 100644 index 0000000000..ae9a7d1ead --- /dev/null +++ b/sysdeps/unix/sysv/linux/x86_64/clock_nanosleep.c @@ -0,0 +1,82 @@ +/* clock_nanosleep for x86_64. + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* Only difference is if we have AVX, use vzeroall to clear inuse for SSE, AVX, + and ZMM_HI256 xsave/xrstor state. This enables the init-state optimization + saving overhead on context switches. */ + +#include +#include +#if ISA_SHOULD_BUILD(4) +# include +# define TARGET __attribute__ ((target ("avx"))) +# define PREPARE_CONTEXT_SWITCH() _mm256_zeroall () +# define CLOCK_NANOSLEEP_TIME64 __clock_nanosleep_time64_avx +# define CLOCK_NANOSLEEP __clock_nanosleep_avx +# define STATIC static +# include +#endif +#if ISA_SHOULD_BUILD(2) +# define TARGET +# define PREPARE_CONTEXT_SWITCH() +# define CLOCK_NANOSLEEP_TIME64 __clock_nanosleep_time64_generic +# define CLOCK_NANOSLEEP __clock_nanosleep_generic +# define STATIC static +# include +#endif + +#include +#include + +static inline void * +__clock_nanosleep_time64_ifunc_selector (void) +{ +#if MINIMUM_X86_ISA_LEVEL >= 3 + return __clock_nanosleep_time64_avx; +#else + const struct cpu_features *cpu_features = __get_cpu_features (); + if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, AVX)) + return __clock_nanosleep_time64_avx; + return __clock_nanosleep_time64_generic; +#endif +} + +libc_ifunc (__clock_nanosleep_time64, + __clock_nanosleep_time64_ifunc_selector ()); +#if __TIMESIZE != 64 +libc_hidden_def (__clock_nanosleep_time64); +static inline void * +__clock_nanosleep_ifunc_selector (void) +{ +# if MINIMUM_X86_ISA_LEVEL >= 3 + return __clock_nanosleep_avx; +# else + const struct cpu_features *cpu_features = __get_cpu_features (); + if (X86_ISA_CPU_FEATURE_USABLE_P (cpu_features, AVX)) + return __clock_nanosleep_avx; + return __clock_nanosleep_generic; +# endif +} +libc_ifunc (__clock_nanosleep, __clock_nanosleep_ifunc_selector ()); +#endif +libc_hidden_def (__clock_nanosleep); +versioned_symbol (libc, __clock_nanosleep, clock_nanosleep, GLIBC_2_17); +#if SHLIB_COMPAT(libc, GLIBC_2_2, GLIBC_2_17) +strong_alias (__clock_nanosleep, __clock_nanosleep_2); +compat_symbol (libc, __clock_nanosleep_2, clock_nanosleep, GLIBC_2_2); +#endif