From patchwork Thu Oct 17 21:55:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 1998794 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=uKQDXOec; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-2354-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XV1qC4YtWz1xvc for ; Fri, 18 Oct 2024 08:55:35 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XV1q84nrfz2yNJ; Fri, 18 Oct 2024 08:55:32 +1100 (AEDT) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip=139.178.84.217 ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1729202132; cv=none; b=nC5GLrShSEaSrhpy+yi27Q1wDU5wvWKDehniTqG26HMpZKM3Nxcw5jsPRY2KgjjmhVyVBBvwWXH8JrUtom/q9c+jXIBDu0/gnyIzQn7h63ecCoLRAbvliHe4nA+GuHuQ5TQ+7Cgg2xN90YA3UUcpiCxkr5drXesoHMFb0GcRfRkRZanHOjBaZG6qKZRPHcZlWsRLQgfn3/e9yep6bjWAzkk34nl1weQMyRtAYFH6gsNKXZDVNTlFDZfSYvKNHG3pTFBs+8eGLRjguKUl97gYRHl2DoXkD7/YVJEEWX80E3G8Ao471bbFdP8/In3PPUSItnm+z8thkpGmcVSzEdDGug== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1729202132; c=relaxed/relaxed; bh=xZSrk6yZJlcU1f/diA7MgQsxzKkKRPfd3XHK4KyCq8s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ba8uTJinGxBX2lZOud9hwNcXAIvKbWTG3rKNaOUSNggy/ZBfv04Uw9QGC9ai4TVt8/OS72h2+KercriE9upBI9okt6gaKhYtTCzWv6zAJ0fyOP7HAUxEoftbaA8Z2RWiY2UgcKztKepUwg7Q/m55NZ9+Fm6JcKxgVS3o+zesvDNGCHNMWLtvWthNdKjMhMNl/EReWqPSi6Ev8APLpAe5Le0FofEyvt8woMhbkFmQE/xxUQo66FK9c4H4G5owgvpNw0ewJnQSyIDrycLwCAlUzHrqrvOw0I94TR2nD0KM9rLw9Lqh3UxRIyq3zi5fCt0hnSgnGV8Kf8Twhh47Lf8F6w== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=kernel.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=uKQDXOec; dkim-atps=neutral; spf=pass (client-ip=139.178.84.217; helo=dfw.source.kernel.org; envelope-from=jpoimboe@kernel.org; receiver=lists.ozlabs.org) smtp.mailfrom=kernel.org Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=kernel.org Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=uKQDXOec; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=kernel.org (client-ip=139.178.84.217; helo=dfw.source.kernel.org; envelope-from=jpoimboe@kernel.org; receiver=lists.ozlabs.org) Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XV1q71f1xz2yFJ for ; Fri, 18 Oct 2024 08:55:31 +1100 (AEDT) Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 147B95C5AF5; Thu, 17 Oct 2024 21:55:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BC073C4CED0; Thu, 17 Oct 2024 21:55:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729202128; bh=NSb0CuZFRxLsMtRmJayyQ03cyCTlxrCnGaQiC+w0oD4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uKQDXOecdhFeWVcjmHYzxDjck9if1H7eA1+6f7NoHaSKzm99ot5/BncF/3PLUnbAh RsJ7nB42/0UWG4+wIvyIna1c/vVySuR/JHzeYr/MJO63eiW2SKqcm4YmcZILD0EMv4 YHTgd4DfquTsJNoGTCEUdsUNZ9188sXWqzp2a6fQoIM72e1F929co1n+Mw9ljovj0q wDdpYy1Vg6vEUx1jqAnmhgtzpw6DlmcpWR6hwWzIPCvf3bRNulPzegEizm04h1/bF+ cIsU8uiR2lVTLesSQzVBqpt6zepx/S73bzxCEFmtSxfFgGIy3L8IZsoalquT9dLCyn gzwgKm5qNOFFg== From: Josh Poimboeuf To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, Thomas Gleixner , Borislav Petkov , Peter Zijlstra , Pawan Gupta , Waiman Long , Dave Hansen , Ingo Molnar , Linus Torvalds , Michael Ellerman , linuxppc-dev@lists.ozlabs.org, Andrew Cooper , Mark Rutland , "Kirill A . Shutemov" Subject: [PATCH v2 1/6] x86/uaccess: Avoid barrier_nospec() in copy_from_user() Date: Thu, 17 Oct 2024 14:55:20 -0700 Message-ID: X-Mailer: git-send-email 2.47.0 In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-5.3 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,SPF_HELO_NONE, SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on lists.ozlabs.org For x86-64, the barrier_nospec() in copy_from_user() is overkill and painfully slow. Instead, use pointer masking to force the user pointer to a non-kernel value in speculative paths. To avoid regressing powerpc, move the barrier_nospec() to the powerpc raw_copy_from_user() implementation so there's no functional change. Signed-off-by: Josh Poimboeuf --- arch/powerpc/include/asm/uaccess.h | 2 ++ arch/x86/include/asm/uaccess_64.h | 7 ++++--- arch/x86/lib/getuser.S | 2 +- arch/x86/lib/putuser.S | 2 +- include/linux/uaccess.h | 6 ------ 5 files changed, 8 insertions(+), 11 deletions(-) diff --git a/arch/powerpc/include/asm/uaccess.h b/arch/powerpc/include/asm/uaccess.h index 4f5a46a77fa2..12abb8bf5eda 100644 --- a/arch/powerpc/include/asm/uaccess.h +++ b/arch/powerpc/include/asm/uaccess.h @@ -7,6 +7,7 @@ #include #include #include +#include #ifdef __powerpc64__ /* We use TASK_SIZE_USER64 as TASK_SIZE is not constant */ @@ -341,6 +342,7 @@ static inline unsigned long raw_copy_from_user(void *to, { unsigned long ret; + barrier_nospec(); allow_read_from_user(from, n); ret = __copy_tofrom_user((__force void __user *)to, from, n); prevent_read_from_user(from, n); diff --git a/arch/x86/include/asm/uaccess_64.h b/arch/x86/include/asm/uaccess_64.h index afce8ee5d7b7..61693028ea2b 100644 --- a/arch/x86/include/asm/uaccess_64.h +++ b/arch/x86/include/asm/uaccess_64.h @@ -54,11 +54,11 @@ static inline unsigned long __untagged_addr_remote(struct mm_struct *mm, #define valid_user_address(x) ((__force long)(x) >= 0) /* - * Masking the user address is an alternative to a conditional - * user_access_begin that can avoid the fencing. This only works - * for dense accesses starting at the address. + * If it's a kernel address, force it to all 1's. This prevents a mispredicted + * access_ok() from speculatively accessing kernel space. */ #define mask_user_address(x) ((typeof(x))((long)(x)|((long)(x)>>63))) + #define masked_user_access_begin(x) ({ \ __auto_type __masked_ptr = (x); \ __masked_ptr = mask_user_address(__masked_ptr); \ @@ -133,6 +133,7 @@ copy_user_generic(void *to, const void *from, unsigned long len) static __always_inline __must_check unsigned long raw_copy_from_user(void *dst, const void __user *src, unsigned long size) { + src = mask_user_address(src); return copy_user_generic(dst, (__force void *)src, size); } diff --git a/arch/x86/lib/getuser.S b/arch/x86/lib/getuser.S index d066aecf8aeb..094224ec9dca 100644 --- a/arch/x86/lib/getuser.S +++ b/arch/x86/lib/getuser.S @@ -39,7 +39,7 @@ .macro check_range size:req .if IS_ENABLED(CONFIG_X86_64) - mov %rax, %rdx + mov %rax, %rdx /* mask_user_address() */ sar $63, %rdx or %rdx, %rax .else diff --git a/arch/x86/lib/putuser.S b/arch/x86/lib/putuser.S index 975c9c18263d..09b7e37934ab 100644 --- a/arch/x86/lib/putuser.S +++ b/arch/x86/lib/putuser.S @@ -34,7 +34,7 @@ .macro check_range size:req .if IS_ENABLED(CONFIG_X86_64) - mov %rcx, %rbx + mov %rcx, %rbx /* mask_user_address() */ sar $63, %rbx or %rbx, %rcx .else diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h index 39c7cf82b0c2..dda9725a9559 100644 --- a/include/linux/uaccess.h +++ b/include/linux/uaccess.h @@ -160,12 +160,6 @@ _inline_copy_from_user(void *to, const void __user *from, unsigned long n) unsigned long res = n; might_fault(); if (!should_fail_usercopy() && likely(access_ok(from, n))) { - /* - * Ensure that bad access_ok() speculation will not - * lead to nasty side effects *after* the copy is - * finished: - */ - barrier_nospec(); instrument_copy_from_user_before(to, from, n); res = raw_copy_from_user(to, from, n); instrument_copy_from_user_after(to, from, n, res); From patchwork Thu Oct 17 21:55:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 1998798 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=tAf3RJLG; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-2358-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XV1qd4wjHz1xvc for ; Fri, 18 Oct 2024 08:55:57 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XV1qB5xgkz3bjb; Fri, 18 Oct 2024 08:55:34 +1100 (AEDT) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2604:1380:4641:c500::1" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1729202134; cv=none; b=dSv7kvHQmJlYtDVKM+10utjWUExRL3t7QI3IvzrA3fFjmMSfpDy33LzzgUZcXTPWoon3wWbM9yA/IN1cHXIFOV5wvfzJFMlSP7Xt5Q5HThLt4FtqGGYYP+iNeL5KchnOCY9GImsyuSHiz2LIzU2tLs8SvOHZkE0fzg+nG99n3aXXGm9tyGrgqEfMYGm8wdjKUpOT6Lq0fiA25UKMQg7tsLajY+qyXl4G8MBJ7MAYKdlVHsiyXCzP+hQzEGUmxG6zKvKBQey9P4kSWCjta43zw93KiAiO3dRpWsU6FGQBj7tQlSG9cM1M/0stLOjlmjyd8vvegtC9ywEDsTZ0tmtJJQ== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1729202134; c=relaxed/relaxed; bh=N7Ujn6q3OmLUGAa7ezEiS1HEORyBXoO92oufTsSydHk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RNdgllPFx2dnq6MlfpN+JPSjNk+SgH88pzMwScagg5EBxD+P9OA+RrRaPR0ZTgX6tqoMJvezL3H01QPZPLnfa7rVGozlfDO8cRu7PMfAPXXTkFS9GmSinOw2pMguQZotJ/WKD7MhPLkXj2u7RVcCYS+1jrCH7mO4Yn4gHbgTc6FPI9SNgYRhw4D3nyIU6XGiMjjN55Q30Q8pyyDwWRKcR7aswlUJP/9XrCgrG2YYXM9mE7V34WLPaNYBsWU7GiSnUYwdntcaFku/5lG6uy/ZKZcTF9lsFXOY/4NUYvB/s/J8qU39jKyzrBeyyrzObxtqIAhKZY7sL+7w9TMMVjR48w== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=kernel.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=tAf3RJLG; dkim-atps=neutral; spf=pass (client-ip=2604:1380:4641:c500::1; helo=dfw.source.kernel.org; envelope-from=jpoimboe@kernel.org; receiver=lists.ozlabs.org) smtp.mailfrom=kernel.org Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=kernel.org Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=tAf3RJLG; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=kernel.org (client-ip=2604:1380:4641:c500::1; helo=dfw.source.kernel.org; envelope-from=jpoimboe@kernel.org; receiver=lists.ozlabs.org) Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XV1q82wc7z2yN4 for ; Fri, 18 Oct 2024 08:55:32 +1100 (AEDT) Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 960E25C5A92; Thu, 17 Oct 2024 21:55:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 54825C4CED2; Thu, 17 Oct 2024 21:55:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729202128; bh=xfgOc7lTaXmUYWAtPlY5EDc6fxKMFQn54g5tpKh/iYU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tAf3RJLGarfzTLeMnQbadhPeskZRMtA2DGorwTgKO/UVRrLECKX9Jb53OUjyLQ5O+ RkMS9uNs6oHcwAc5nf9SB5TTqo/0q+P++YVF9PieqhHBFsJ2Nn02Vc/zATf8WMNWcz 6A3fk34YHGLyWGbMrzecfgFFgW+aaH/n3sO1DpZuPl71DDI7CTZA4NQnXre7u48w/+ tHTjzHDN8gLnQBYjN1UK5+GwCyXFeWQcQKOC3DYKqu6iGxuqv2vsi+5ltcUxGlZi4K RqdTPM+H+IpoqnFYdA19gi+j6Oo8okTqCyrGJW6uWOuVKNAELOeDiY0xGGZ35JI8+2 XXGaZkCCJuo+Q== From: Josh Poimboeuf To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, Thomas Gleixner , Borislav Petkov , Peter Zijlstra , Pawan Gupta , Waiman Long , Dave Hansen , Ingo Molnar , Linus Torvalds , Michael Ellerman , linuxppc-dev@lists.ozlabs.org, Andrew Cooper , Mark Rutland , "Kirill A . Shutemov" Subject: [PATCH v2 2/6] x86/uaccess: Avoid barrier_nospec() in __get_user() Date: Thu, 17 Oct 2024 14:55:21 -0700 Message-ID: <0777ac8e8c8d669fa56971dcba68b6f1c1980d39.1729201904.git.jpoimboe@kernel.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on lists.ozlabs.org On 64-bit, the barrier_nospec() in __get_user() is overkill and painfully slow. Instead, use pointer masking to force the user pointer to a non-kernel value in speculative paths. Doing so makes get_user() and __get_user() identical in behavior, so converge their implementations. Signed-off-by: Josh Poimboeuf --- arch/x86/lib/getuser.S | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/arch/x86/lib/getuser.S b/arch/x86/lib/getuser.S index 094224ec9dca..7c9bf8f0b3ac 100644 --- a/arch/x86/lib/getuser.S +++ b/arch/x86/lib/getuser.S @@ -105,6 +105,26 @@ SYM_FUNC_START(__get_user_8) SYM_FUNC_END(__get_user_8) EXPORT_SYMBOL(__get_user_8) +#ifdef CONFIG_X86_64 + +/* + * On x86-64, get_user() does address masking rather than a conditional + * bounds check so there's no functional difference with __get_user(). + */ +SYM_FUNC_ALIAS(__get_user_nocheck_1, __get_user_1); +EXPORT_SYMBOL(__get_user_nocheck_1); + +SYM_FUNC_ALIAS(__get_user_nocheck_2, __get_user_2); +EXPORT_SYMBOL(__get_user_nocheck_2); + +SYM_FUNC_ALIAS(__get_user_nocheck_4, __get_user_4); +EXPORT_SYMBOL(__get_user_nocheck_4); + +SYM_FUNC_ALIAS(__get_user_nocheck_8, __get_user_8); +EXPORT_SYMBOL(__get_user_nocheck_8); + +#else /* CONFIG_X86_32 */ + /* .. and the same for __get_user, just without the range checks */ SYM_FUNC_START(__get_user_nocheck_1) ASM_STAC @@ -139,19 +159,16 @@ EXPORT_SYMBOL(__get_user_nocheck_4) SYM_FUNC_START(__get_user_nocheck_8) ASM_STAC ASM_BARRIER_NOSPEC -#ifdef CONFIG_X86_64 - UACCESS movq (%_ASM_AX),%rdx -#else xor %ecx,%ecx UACCESS movl (%_ASM_AX),%edx UACCESS movl 4(%_ASM_AX),%ecx -#endif xor %eax,%eax ASM_CLAC RET SYM_FUNC_END(__get_user_nocheck_8) EXPORT_SYMBOL(__get_user_nocheck_8) +#endif /* CONFIG_X86_32 */ SYM_CODE_START_LOCAL(__get_user_handle_exception) ASM_CLAC From patchwork Thu Oct 17 21:55:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 1998796 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=RwSRDs6W; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-2356-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XV1qR1Ttfz1xvc for ; Fri, 18 Oct 2024 08:55:47 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XV1qB169Sz3bbW; Fri, 18 Oct 2024 08:55:34 +1100 (AEDT) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip=147.75.193.91 ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1729202134; cv=none; b=obpCNGKwJg9A8qwcpDDxXihQJvX00Op7zqsIvhA0ncRGpyz09JQKe059i+hUpskqmMhG6m6ntQjfkQRFXpJB0IjrweIKA2+XZ2KBZSdqnfv7nbJ6kGmpmi7kf5u3LunXwB8KCEwn4kx2eAJqicQ06I6i8Etpo26Y/hLSkgxSYKHCMzR4/UDnq/WI4E70XMrsg0wU/gwTSxnYBANL1NJJTfsBNg3SKqRaWh/KjzBAyMS3j37Eq8bNV04qcoGnJjzfixNato0blxTyNjsq7m4MqJuScGeZw9heeLjVGuzp2azs9xa0NvVQ1tnw95+vXB6YllogkF0FsgwGHtZMjhCF/g== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1729202134; c=relaxed/relaxed; bh=/jmJa/gwHFWlu3zXGxyo3KrVsPwIWewIXZFfCfaPzA0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CutvdTvrs7oU/iQGTtHPFUFySdwJARkutcLr0ylwBJT93Xc134/QuUEogHlDrOvvr0CA+VKGMsMp5jnrAGoDCVJrkHARw7XvUlz1sGeNTmLMjlacc/AvP0FEveeWb6yl0AHRjfnJpKdc4NI4qKaPKKiPguQomyIPPvVD60ACMmlMn3xuBA6VcKLV1x1xQIC+uLxvKUv8KDYjNNW0Q8Z7vgQmuqHffXDoavbCBNPCEhTlMG6rL8R6wpv+ZSew4eA2dtqDxF17zGTC3A6yTwrf/LldbveA7ch7fd9s/YnZUkljKsCD8fl453MbYzFEeoHRi0dj+fQaKRQMOPNh5YkDNA== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=kernel.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=RwSRDs6W; dkim-atps=neutral; spf=pass (client-ip=147.75.193.91; helo=nyc.source.kernel.org; envelope-from=jpoimboe@kernel.org; receiver=lists.ozlabs.org) smtp.mailfrom=kernel.org Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=kernel.org Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=RwSRDs6W; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=kernel.org (client-ip=147.75.193.91; helo=nyc.source.kernel.org; envelope-from=jpoimboe@kernel.org; receiver=lists.ozlabs.org) Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XV1q80Mvnz2yGZ for ; Fri, 18 Oct 2024 08:55:31 +1100 (AEDT) Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id A2FF4A444A9; Thu, 17 Oct 2024 21:55:20 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E136DC4CED0; Thu, 17 Oct 2024 21:55:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729202129; bh=+NVAfBL5XC4O3/+bvmSlB+aCF1KvxK5rqTMlDvFps8s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RwSRDs6W96PHO2LRpbnWJ0OAi+5QM78NCMgPnwsZ6OZ0V7sPQULC5WebYPZVd5d3l FEbxM2R0ClBXjbJm5zZ85m/WdRyqIGWUpJG/xqyxJHWhKORX+n8iGN4M6qwy7uf8s1 /YDH9Ssdat0h9Zn+uKLHRCPkjKL0FtvgGVq0TvnDjm2Uy1Qpqp4d7T5E6kHkevZe2g E72VfcMW9JI8XGhLsRfh/X5RIhe9aMYhRvjmDchnYZg2OEwwYPL0YBiLJKd8BTVkBm r84u32waSE4V6iFBM1gjUKQTZ6gCjonbVPlRHOtWM6g8iJpjGXgSOKuqqbdAWwXPP6 vbrG1Orde5mXw== From: Josh Poimboeuf To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, Thomas Gleixner , Borislav Petkov , Peter Zijlstra , Pawan Gupta , Waiman Long , Dave Hansen , Ingo Molnar , Linus Torvalds , Michael Ellerman , linuxppc-dev@lists.ozlabs.org, Andrew Cooper , Mark Rutland , "Kirill A . Shutemov" Subject: [PATCH v2 3/6] x86/uaccess: Rearrange putuser.S Date: Thu, 17 Oct 2024 14:55:22 -0700 Message-ID: <7818233ecd726628a3eb9cbb5ed0ba831e69af4b.1729201904.git.jpoimboe@kernel.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-0.3 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on lists.ozlabs.org Separate __put_user_*() from __put_user_nocheck_*() to make the layout similar to getuser.S. This will also make it easier to do a subsequent change. No functional changes. Signed-off-by: Josh Poimboeuf --- arch/x86/lib/putuser.S | 67 ++++++++++++++++++++++-------------------- 1 file changed, 35 insertions(+), 32 deletions(-) diff --git a/arch/x86/lib/putuser.S b/arch/x86/lib/putuser.S index 09b7e37934ab..cb137e0286be 100644 --- a/arch/x86/lib/putuser.S +++ b/arch/x86/lib/putuser.S @@ -54,59 +54,32 @@ SYM_FUNC_START(__put_user_1) SYM_FUNC_END(__put_user_1) EXPORT_SYMBOL(__put_user_1) -SYM_FUNC_START(__put_user_nocheck_1) - ASM_STAC -2: movb %al,(%_ASM_CX) - xor %ecx,%ecx - ASM_CLAC - RET -SYM_FUNC_END(__put_user_nocheck_1) -EXPORT_SYMBOL(__put_user_nocheck_1) - SYM_FUNC_START(__put_user_2) check_range size=2 ASM_STAC -3: movw %ax,(%_ASM_CX) +2: movw %ax,(%_ASM_CX) xor %ecx,%ecx ASM_CLAC RET SYM_FUNC_END(__put_user_2) EXPORT_SYMBOL(__put_user_2) -SYM_FUNC_START(__put_user_nocheck_2) - ASM_STAC -4: movw %ax,(%_ASM_CX) - xor %ecx,%ecx - ASM_CLAC - RET -SYM_FUNC_END(__put_user_nocheck_2) -EXPORT_SYMBOL(__put_user_nocheck_2) - SYM_FUNC_START(__put_user_4) check_range size=4 ASM_STAC -5: movl %eax,(%_ASM_CX) +3: movl %eax,(%_ASM_CX) xor %ecx,%ecx ASM_CLAC RET SYM_FUNC_END(__put_user_4) EXPORT_SYMBOL(__put_user_4) -SYM_FUNC_START(__put_user_nocheck_4) - ASM_STAC -6: movl %eax,(%_ASM_CX) - xor %ecx,%ecx - ASM_CLAC - RET -SYM_FUNC_END(__put_user_nocheck_4) -EXPORT_SYMBOL(__put_user_nocheck_4) - SYM_FUNC_START(__put_user_8) check_range size=8 ASM_STAC -7: mov %_ASM_AX,(%_ASM_CX) +4: mov %_ASM_AX,(%_ASM_CX) #ifdef CONFIG_X86_32 -8: movl %edx,4(%_ASM_CX) +5: movl %edx,4(%_ASM_CX) #endif xor %ecx,%ecx ASM_CLAC @@ -114,6 +87,34 @@ SYM_FUNC_START(__put_user_8) SYM_FUNC_END(__put_user_8) EXPORT_SYMBOL(__put_user_8) +/* .. and the same for __put_user, just without the range checks */ +SYM_FUNC_START(__put_user_nocheck_1) + ASM_STAC +6: movb %al,(%_ASM_CX) + xor %ecx,%ecx + ASM_CLAC + RET +SYM_FUNC_END(__put_user_nocheck_1) +EXPORT_SYMBOL(__put_user_nocheck_1) + +SYM_FUNC_START(__put_user_nocheck_2) + ASM_STAC +7: movw %ax,(%_ASM_CX) + xor %ecx,%ecx + ASM_CLAC + RET +SYM_FUNC_END(__put_user_nocheck_2) +EXPORT_SYMBOL(__put_user_nocheck_2) + +SYM_FUNC_START(__put_user_nocheck_4) + ASM_STAC +8: movl %eax,(%_ASM_CX) + xor %ecx,%ecx + ASM_CLAC + RET +SYM_FUNC_END(__put_user_nocheck_4) +EXPORT_SYMBOL(__put_user_nocheck_4) + SYM_FUNC_START(__put_user_nocheck_8) ASM_STAC 9: mov %_ASM_AX,(%_ASM_CX) @@ -137,11 +138,13 @@ SYM_CODE_END(__put_user_handle_exception) _ASM_EXTABLE_UA(2b, __put_user_handle_exception) _ASM_EXTABLE_UA(3b, __put_user_handle_exception) _ASM_EXTABLE_UA(4b, __put_user_handle_exception) +#ifdef CONFIG_X86_32 _ASM_EXTABLE_UA(5b, __put_user_handle_exception) +#endif _ASM_EXTABLE_UA(6b, __put_user_handle_exception) _ASM_EXTABLE_UA(7b, __put_user_handle_exception) + _ASM_EXTABLE_UA(8b, __put_user_handle_exception) _ASM_EXTABLE_UA(9b, __put_user_handle_exception) #ifdef CONFIG_X86_32 - _ASM_EXTABLE_UA(8b, __put_user_handle_exception) _ASM_EXTABLE_UA(10b, __put_user_handle_exception) #endif From patchwork Thu Oct 17 21:55:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 1998797 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=kMzCg3kq; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-2357-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XV1qV5Bv6z1xvc for ; Fri, 18 Oct 2024 08:55:50 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XV1qB2WVJz3bgf; Fri, 18 Oct 2024 08:55:34 +1100 (AEDT) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip=147.75.193.91 ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1729202134; cv=none; b=fz51H+CBWgrxGel+q5PncMeZYqpS0LI1W7xlXy94lBKKS2lUShQi/d0XSN+dMrkIXynxXx7FJwJEgv/eZco4ZqtplEoORn7buNkmRru2tCbrChFavuCbLwf6pyfelRfOExD+odw2TVhmry1VmZiv0hs1Ih9AOBH3yZxvV+CTf2kv57zGUMksAZZxvV47QgSWyZJy2lK1lVX4jzxaV1Cgrz7h/IQ35mGpHw7ZHmdjf9X86TnHxhtNybnD9rZ/0qosuS8qi8qrFO8Upidam4ieWEGgPYHQKO0R/4vu0JhBzpPPjhyEcMI0sWt6VjadPseIMiYxhnodXAH/oX+jlyMCAg== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1729202134; c=relaxed/relaxed; bh=iUfPZ2q+NIS2dyY6unsVAimEzY3gCmEEgLnL341hcno=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SYR2P3BD5sgLBnd1WOLKAfU4GzBlCAjiU6bMv8CpajS2176GHLrugYKxfTyXozZk40ObSJGgWYgscRuJEj1qvseodIveH57kt4x8WXm4QmKVBlj70acPE9Yeh9d/uiCAMuStLEUcaCaBB0RevjDvEtmPMHTwxNAdZiIe7QelwqXSuIaC7AfhPkhWpi1s2zT3hUJY0y3SyvvdlwORR4BpeLCniofkorT5ACkUPkv2eaL7xWZCy9Pp1Yom7Mv5qhFTZnOnfUHtdYRxepSYI3y6d2Lv9E/rvxtfsTzYtR+0xLHvC6Dg3EdHhGjODLLuDyvOQP+hUHWn2ufQapTFoTdyLQ== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=kernel.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=kMzCg3kq; dkim-atps=neutral; spf=pass (client-ip=147.75.193.91; helo=nyc.source.kernel.org; envelope-from=jpoimboe@kernel.org; receiver=lists.ozlabs.org) smtp.mailfrom=kernel.org Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=kernel.org Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=kMzCg3kq; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=kernel.org (client-ip=147.75.193.91; helo=nyc.source.kernel.org; envelope-from=jpoimboe@kernel.org; receiver=lists.ozlabs.org) Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XV1q85MKmz2yQn for ; Fri, 18 Oct 2024 08:55:32 +1100 (AEDT) Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 1EC42A444AF; Thu, 17 Oct 2024 21:55:21 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7A02EC4CED4; Thu, 17 Oct 2024 21:55:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729202129; bh=NMUERTfe8IitvLeucMcKK8QffKkIIRs7lelNieDdT8Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kMzCg3kqxX5332ygSUeJINGDmjiYNVsMfc7/7dzsAm06HCcjkVGMntsgcp+IjM/hL RWPWigIMfoZYM/d8HrkVmaWzOWlxpmFWU+0rbXuGpt/rbahjspkRqroToOQK7omUQ0 CWgPseIrIFd1H+4fxrsVmrG/4XMCBQKu/BjkSyy9/H784GkUvxWYX+1c7y4jtYw8xz S1ZgR2yDDjpu2629dLL8w54cSK2EIS4rdxvQ4vQO/Hksj4o5MokUFoSuMrhT91SS8v oMDBzwzfliCdS/84MB41FTSlPz99wiwk7WJV0Z3ZdtGpx+p7eqbS+1o5RAP8MuGZ47 OnD3DSEsxO3fg== From: Josh Poimboeuf To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, Thomas Gleixner , Borislav Petkov , Peter Zijlstra , Pawan Gupta , Waiman Long , Dave Hansen , Ingo Molnar , Linus Torvalds , Michael Ellerman , linuxppc-dev@lists.ozlabs.org, Andrew Cooper , Mark Rutland , "Kirill A . Shutemov" Subject: [PATCH v2 4/6] x86/uaccess: Add user pointer masking to __put_user() Date: Thu, 17 Oct 2024 14:55:23 -0700 Message-ID: X-Mailer: git-send-email 2.47.0 In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-0.3 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on lists.ozlabs.org Add user pointer masking to __put_user() to mitigate Spectre v1. A write in a mispredicted access_ok() branch to a user-controlled kernel address can populate the rest of the affected cache line with kernel data. This makes its behavior identical to put_user(), so converge their implementations. Signed-off-by: Josh Poimboeuf --- arch/x86/lib/putuser.S | 27 ++++++++++++++++++++++----- 1 file changed, 22 insertions(+), 5 deletions(-) diff --git a/arch/x86/lib/putuser.S b/arch/x86/lib/putuser.S index cb137e0286be..1b122261b7aa 100644 --- a/arch/x86/lib/putuser.S +++ b/arch/x86/lib/putuser.S @@ -87,7 +87,26 @@ SYM_FUNC_START(__put_user_8) SYM_FUNC_END(__put_user_8) EXPORT_SYMBOL(__put_user_8) -/* .. and the same for __put_user, just without the range checks */ +#ifdef CONFIG_X86_64 + +/* + * On x86-64, put_user() does address masking rather than a conditional + * bounds check so there's no functional difference with __put_user(). + */ +SYM_FUNC_ALIAS(__put_user_nocheck_1, __put_user_1); +EXPORT_SYMBOL(__put_user_nocheck_1); + +SYM_FUNC_ALIAS(__put_user_nocheck_2, __put_user_2); +EXPORT_SYMBOL(__put_user_nocheck_2); + +SYM_FUNC_ALIAS(__put_user_nocheck_4, __put_user_4); +EXPORT_SYMBOL(__put_user_nocheck_4); + +SYM_FUNC_ALIAS(__put_user_nocheck_8, __put_user_8); +EXPORT_SYMBOL(__put_user_nocheck_8); + +#else /* CONFIG_X86_32 */ + SYM_FUNC_START(__put_user_nocheck_1) ASM_STAC 6: movb %al,(%_ASM_CX) @@ -118,15 +137,15 @@ EXPORT_SYMBOL(__put_user_nocheck_4) SYM_FUNC_START(__put_user_nocheck_8) ASM_STAC 9: mov %_ASM_AX,(%_ASM_CX) -#ifdef CONFIG_X86_32 10: movl %edx,4(%_ASM_CX) -#endif xor %ecx,%ecx ASM_CLAC RET SYM_FUNC_END(__put_user_nocheck_8) EXPORT_SYMBOL(__put_user_nocheck_8) +#endif /* CONFIG_X86_32 */ + SYM_CODE_START_LOCAL(__put_user_handle_exception) ASM_CLAC .Lbad_put_user: @@ -140,11 +159,9 @@ SYM_CODE_END(__put_user_handle_exception) _ASM_EXTABLE_UA(4b, __put_user_handle_exception) #ifdef CONFIG_X86_32 _ASM_EXTABLE_UA(5b, __put_user_handle_exception) -#endif _ASM_EXTABLE_UA(6b, __put_user_handle_exception) _ASM_EXTABLE_UA(7b, __put_user_handle_exception) _ASM_EXTABLE_UA(8b, __put_user_handle_exception) _ASM_EXTABLE_UA(9b, __put_user_handle_exception) -#ifdef CONFIG_X86_32 _ASM_EXTABLE_UA(10b, __put_user_handle_exception) #endif From patchwork Thu Oct 17 21:55:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 1998799 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=QScC8JXg; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-2359-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XV1qk5w4Fz1xvc for ; Fri, 18 Oct 2024 08:56:02 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XV1qC1cD8z3blc; Fri, 18 Oct 2024 08:55:35 +1100 (AEDT) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip=147.75.193.91 ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1729202135; cv=none; b=jb30RSHwv2kL04UsZ6jYLHBFD8rBRYeNPB0s8+EyAFr+7JYgmXQY5FfRdG/F1X6ayIyttxGQzE+kvYmIjsJTEKL9ebLJoew66sKyWrnkaDBORzLClHIMp8rxW9uaqkXdc0MlRrSPoGn5ta3tgLRHIr74azNRpCjiVnB/rftyx2r+e5++bK49/J4EXw+5BoMJX3u005MX9ukvMD1abz7JkH+7BCs9jgwlSF4nu43KN3VdxmlOJYGgdIEYwVPhG6Sz4iTVWPRoo0vxalxMpdJgoLbwpbQKRBGHr+GjlS7+Wz3INBTT828RABLj440IQSjn7WpLZ7jeEDyKZfyZY5v9HQ== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1729202135; c=relaxed/relaxed; bh=Z2uuxEKJxpJdB/cl/bli/5bMWlpijsXLHtGSalHPI34=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Pt03adONWP9fCPEpE5bKjSdJPEQszSDc/AXu4Fq/DPgLxi9v7N11E1Hi5KRky9bvmTnolumlYUtKUD1+eoeOHgZikY9/Eoj0u6TNelFnzx5H+IttanZlo1jZJYsVIRrjx7IKN4x/U0wm8oLSfRpntJ1yhDEnnTQ5+Y9UvnHy0X2f0OPr9bD0ebRd1YtwZRTRIvskzMwHQKgwc90BJVtmeheg/aaLBxUogS3vvTluqjj1qQZx7jQESqCu9PPNimtXzB6Tp3QmKU0eqmMDiEXwPWzOUJ29X48f3+ZNg9yeKHDBm3Wr8a2enl+Z3F6QvVRMAarJkGeXKzuo3VeK0M652Q== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=kernel.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=QScC8JXg; dkim-atps=neutral; spf=pass (client-ip=147.75.193.91; helo=nyc.source.kernel.org; envelope-from=jpoimboe@kernel.org; receiver=lists.ozlabs.org) smtp.mailfrom=kernel.org Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=kernel.org Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=QScC8JXg; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=kernel.org (client-ip=147.75.193.91; helo=nyc.source.kernel.org; envelope-from=jpoimboe@kernel.org; receiver=lists.ozlabs.org) Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XV1q93Kkcz30CD for ; Fri, 18 Oct 2024 08:55:32 +1100 (AEDT) Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id A27F0A444AA; Thu, 17 Oct 2024 21:55:21 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 15429C4CED0; Thu, 17 Oct 2024 21:55:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729202130; bh=3+6cqjky5SJ8pbbT1W6aRTHDrkhRTtSUbUYAfyoehbs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QScC8JXg1nhIsJfvLMXZp0T/67JQHRKuGALw/DEQDyZqG5bhTI5VunzUliAmaDCTy c/IGDphRQwyP6so/z6M/bZGusu6oMxqktvU2B/PxIyBH09VpBrFCizxBN/yNOraMcH LPv7C82XvgQjd7Cw0C7dkk81Qv4RG68Udm1ukCLHRDJEctcSNXt1xTWyxp9Smy6ggB irDMBk+hdz68zwzRFKTky05dws961rQB+l/+CuG5u5sysjQN0L6VF+X+Xe6jAzw8Zr 96AOyOlz/q0+wljnFl74MsTItpZfzA1wWN+6rVInJrwOm+isaX+aJ/EEDqOw+Ww0XL 9jOJGvTtKjZ3Q== From: Josh Poimboeuf To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, Thomas Gleixner , Borislav Petkov , Peter Zijlstra , Pawan Gupta , Waiman Long , Dave Hansen , Ingo Molnar , Linus Torvalds , Michael Ellerman , linuxppc-dev@lists.ozlabs.org, Andrew Cooper , Mark Rutland , "Kirill A . Shutemov" Subject: [PATCH v2 5/6] x86/uaccess: Add user pointer masking to copy_to_user() Date: Thu, 17 Oct 2024 14:55:24 -0700 Message-ID: <6500dcd8e7700b4dfe5de4f82ed2da19edc23c58.1729201904.git.jpoimboe@kernel.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-0.3 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on lists.ozlabs.org Add user pointer masking to copy_to_user() to mitigate Spectre v1. A write in a mispredicted access_ok() branch to a user-controlled kernel address can populate the rest of the affected cache line with kernel data. Signed-off-by: Josh Poimboeuf --- arch/x86/include/asm/uaccess_64.h | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/include/asm/uaccess_64.h b/arch/x86/include/asm/uaccess_64.h index 61693028ea2b..0587830a47e1 100644 --- a/arch/x86/include/asm/uaccess_64.h +++ b/arch/x86/include/asm/uaccess_64.h @@ -140,6 +140,7 @@ raw_copy_from_user(void *dst, const void __user *src, unsigned long size) static __always_inline __must_check unsigned long raw_copy_to_user(void __user *dst, const void *src, unsigned long size) { + dst = mask_user_address(dst); return copy_user_generic((__force void *)dst, src, size); } From patchwork Thu Oct 17 21:55:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 1998800 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=TxSb0QpA; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-2360-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XV1qs3WTcz1xvc for ; Fri, 18 Oct 2024 08:56:09 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XV1qC5JGZz3bmC; Fri, 18 Oct 2024 08:55:35 +1100 (AEDT) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip=147.75.193.91 ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1729202135; cv=none; b=MiKbadhQFLb3TzJjFK638S/qQXPphZfcsOOuNsni0V5G/CZFnXouKcq97Ql+Q7Cc1ECYf1VJpBd48dMuSHLl2MbCz449oSA58gQLv80C3pJrMDSdRlBV6xrO9xU90dcoMCOzaTKRom1tD2xJQ0RC+l190rkyW3Nvm8JHfQoa6RcohCk2JyfF1gxJHsJx0RKUCrJ2JCy1qjUgw92bCqFkf//s0snGBsGhthZortTTi42xWaKTF6JvJ+rbAqvzQPTHSgA264Yi+SBGIov3VuenFIQnJxeu/tmPF2k/UnX1S4W5KeBHmBwxUlT99/ceNEKRKnU3MU0gmkkinDpGX/9Kgw== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1729202135; c=relaxed/relaxed; bh=t+fp9HkqB7t9T4kXp25tzRMFvZmMzZx5AUMGMn2tzpM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=El4JZfkATKg7gs7ohLDkwwnmLlxUpnM3sM7fa0fUFr90r9uFzElpdLGZzTa9epoEfTdxxWa9jroUHKVkM6UO0eJiK6QWyTL9TjjrKb6aVRBJDi3TAmhNs6+IdidpuK/4j2ZEu4Bz8oU1Lz7F0WBApRtE63f6shNSkbJXQ7qLmyry1Hz+uV7VmkHaB1+kXfShxij0NR4HIaIg7SZSycXNkwznILbcldl3r0WCKI+j7YDK79yH/YzAed4lf+wt2HCqFzcIwzx6ZB36Hx4yGQwF8ku3cfCp8+7iYu6O0mGsmQ4XA3yRdI9/u4i6vali1TbzBu6YcrngDq/q8jeqMq2l6Q== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=kernel.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=TxSb0QpA; dkim-atps=neutral; spf=pass (client-ip=147.75.193.91; helo=nyc.source.kernel.org; envelope-from=jpoimboe@kernel.org; receiver=lists.ozlabs.org) smtp.mailfrom=kernel.org Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=kernel.org Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=TxSb0QpA; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=kernel.org (client-ip=147.75.193.91; helo=nyc.source.kernel.org; envelope-from=jpoimboe@kernel.org; receiver=lists.ozlabs.org) Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XV1q964ngz2yFK for ; Fri, 18 Oct 2024 08:55:33 +1100 (AEDT) Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 49435A444B3; Thu, 17 Oct 2024 21:55:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A17A8C4CEC3; Thu, 17 Oct 2024 21:55:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729202131; bh=opTfvoucneR4FnsToYqJIQ2vAsIrtZ9GVWhav979RCg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TxSb0QpAIMqiN4HI9o5K6GW+rkBMV2URDlO14dMxDoDJzEWfPvquUxVQ5hm8pQmkx 43qcYbZDzeLJqryctoK0nWFeX51ZT5JQwsA3zcUlKHjW/V3/WYXNA5dHkSBiMDmR7r FNs/BZvZOhDnougKtZALz8ZZxadM48pBr24W2appoYcj7pUuIr1FDlDmmJO89CXqs6 6CuDXUronT/fSOE0ULA/lFjCU6Ot0QmXBgvr9vPWd+isX+b8RkNv6gPKtPgq6GKI8h KiFV2651yGF27415Tjt3LZ2FXdVzMG11jflBg3CcVtQcyqZWDr3K2Lwl4y4ABsU9bV VX+CnZpzvHY4A== From: Josh Poimboeuf To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, Thomas Gleixner , Borislav Petkov , Peter Zijlstra , Pawan Gupta , Waiman Long , Dave Hansen , Ingo Molnar , Linus Torvalds , Michael Ellerman , linuxppc-dev@lists.ozlabs.org, Andrew Cooper , Mark Rutland , "Kirill A . Shutemov" Subject: [PATCH v2 6/6] x86/uaccess: Add user pointer masking to clear_user() Date: Thu, 17 Oct 2024 14:55:25 -0700 Message-ID: <7db4ec5c9444e4b76d45a189fdd37f6483c06bef.1729201904.git.jpoimboe@kernel.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-0.3 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on lists.ozlabs.org Add user pointer masking to clear_user() to mitigate Spectre v1. A write in a mispredicted access_ok() branch to a user-controlled kernel address can populate the rest of the affected cache line with kernel data. Signed-off-by: Josh Poimboeuf --- arch/x86/include/asm/uaccess_64.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/include/asm/uaccess_64.h b/arch/x86/include/asm/uaccess_64.h index 0587830a47e1..8027db7f68c2 100644 --- a/arch/x86/include/asm/uaccess_64.h +++ b/arch/x86/include/asm/uaccess_64.h @@ -199,7 +199,7 @@ static __always_inline __must_check unsigned long __clear_user(void __user *addr static __always_inline unsigned long clear_user(void __user *to, unsigned long n) { if (__access_ok(to, n)) - return __clear_user(to, n); + return __clear_user(mask_user_address(to), n); return n; } #endif /* _ASM_X86_UACCESS_64_H */