From patchwork Wed Apr 17 23:53:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yuxuan Luo X-Patchwork-Id: 1924775 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=185.125.189.65; helo=lists.ubuntu.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=patchwork.ozlabs.org) Received: from lists.ubuntu.com (lists.ubuntu.com [185.125.189.65]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VKd6w2tpvz1yZx for ; Thu, 18 Apr 2024 09:54:32 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=lists.ubuntu.com) by lists.ubuntu.com with esmtp (Exim 4.86_2) (envelope-from ) id 1rxF6g-00066I-2n; Wed, 17 Apr 2024 23:54:26 +0000 Received: from smtp-relay-internal-1.internal ([10.131.114.114] helo=smtp-relay-internal-1.canonical.com) by lists.ubuntu.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1rxF6T-0005gm-BR for kernel-team@lists.ubuntu.com; Wed, 17 Apr 2024 23:54:13 +0000 Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id 2C17A3F36F for ; Wed, 17 Apr 2024 23:54:13 +0000 (UTC) Received: by mail-qt1-f198.google.com with SMTP id d75a77b69052e-434e823ba24so2909021cf.2 for ; Wed, 17 Apr 2024 16:54:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713398051; x=1714002851; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Wt5Lkh5a1X7wf3SuQP80lsLT20ongQx1QHaNAi78Fag=; b=k9rHv/DG6n6QKo9KY6h8Q40qANetxHK2etmJX7fIMiz18JQJaUPaPle4t7ZnDnqp9F Gdatp9M6CSeeCQeMd9Mmx8XYmd9WilpEFfq/DnXfRaAa8zHRHL0gAwiQYxNPvE3YRqNI iXz0KuUcgcuB03edH5W155pprXxOZomtvbsveUOwXVRCfz7O+SMJDd0ECdzHfzxpvkqN 2bqYF0TiwjwBvvOewFjTwtZKm1WjAuekcOYSW4P1yF8fY85pnPDKLsLzZ/WZuEabbPge Bc9GX+Nq4z4mYTRijmKeO+xCwa8wiRh6lnSTiys7ekz0JJ6lnNeUNF5kfktxtHsQIx1m oS6A== X-Gm-Message-State: AOJu0Yyuwld2X8Gsczs7ctghvmDA25yPvfCySS6NS/ZGg+FtLqZTbYZ4 1BeiKzbmfyfTGxsonNvoQuzRTGUZ+9mlIy3bKybfvmS1FJh4O2uge1t8Pgs+OtyQ6Ojo6dnbB8z hmiY4Nn/9ZVI6lU/+cRwQSYY7kQ8XLSRLwIMapRKpMzZ/e4syZksQWBRyC2zi1jBdf/kuQjPEoJ jtQoUv/ciGF98C X-Received: by 2002:ac8:5dcb:0:b0:434:a4fd:29f0 with SMTP id e11-20020ac85dcb000000b00434a4fd29f0mr1175943qtx.65.1713398051160; Wed, 17 Apr 2024 16:54:11 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH1sWzPmkvIm0ruYEosG9oP1s7bcRW4o5wtG22h21u/KD6COspPKVP0stsUbULOnHyAvTsGGg== X-Received: by 2002:ac8:5dcb:0:b0:434:a4fd:29f0 with SMTP id e11-20020ac85dcb000000b00434a4fd29f0mr1175925qtx.65.1713398050733; Wed, 17 Apr 2024 16:54:10 -0700 (PDT) Received: from cache-ubuntu.hsd1.nj.comcast.net ([2001:67c:1562:8007::aac:4795]) by smtp.gmail.com with ESMTPSA id x27-20020a05620a099b00b0078d6a0d5728sm129208qkx.29.2024.04.17.16.54.10 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Apr 2024 16:54:10 -0700 (PDT) From: Yuxuan Luo To: kernel-team@lists.ubuntu.com Subject: [PATCH 05/11] x86/syscall: Don't force use of indirect calls for system calls Date: Wed, 17 Apr 2024 19:53:55 -0400 Message-Id: <20240417235401.243631-6-yuxuan.luo@canonical.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240417235401.243631-1-yuxuan.luo@canonical.com> References: <20240417235401.243631-1-yuxuan.luo@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Linus Torvalds Make build a switch statement instead, and the compiler can either decide to generate an indirect jump, or - more likely these days due to mitigations - just a series of conditional branches. Yes, the conditional branches also have branch prediction, but the branch prediction is much more controlled, in that it just causes speculatively running the wrong system call (harmless), rather than speculatively running possibly wrong random less controlled code gadgets. This doesn't mitigate other indirect calls, but the system call indirection is the first and most easily triggered case. Signed-off-by: Linus Torvalds Signed-off-by: Daniel Sneddon Signed-off-by: Thomas Gleixner Reviewed-by: Josh Poimboeuf (backported from commit 1e3ad78334a69b36e107232e337f9d693dcc9df2) [yuxuan.luo: common.c: - substitute *sys_call_table[nr]() with corresponding *sys_call(regs, nr). - For do_syscall_irqs_on()/ia32_sys_call(), substitute both and we’ll define different ia32_sys_call() in syscall_32.c using macros. syscall_x32.c: - x32_sys_call() should move to syscall_64.c since they both rely on __x64_ system calls. syscall_32.c: - Focal tree is using __SYSCALL_I386 rather than __SYSCALL, substitute __SYSCALL with __SYSCALL_I386 in #define lines. - For the case where CONFIG_IA32_EMULATION=n, expand regs to six variables. - Since syscall table in Focal includes arch name in syscalls prefixes already, change 'case nr: return __x64_##sym()' to '... return sym()'. This applies to syscall_64.c as well. syscall_64.c: - For x64_sys_call(), substitute __SYSCALL with __SYSCALL_64. - For x32_sys_call(), wrap it with #ifdef CONFIG_X86_X32_ABI to comply with x32_sys_call() in common.c. Use macros to ignored __SYSCALL_64 lines and substitute __SYSCALL_X32 lines. ] CVE-2024-2201 Signed-off-by: Yuxuan Luo --- arch/x86/entry/common.c | 11 ++++------- arch/x86/entry/syscall_32.c | 33 +++++++++++++++++++++++++++++++++ arch/x86/entry/syscall_64.c | 27 +++++++++++++++++++++++++++ arch/x86/include/asm/syscall.h | 4 ++++ 4 files changed, 68 insertions(+), 7 deletions(-) diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c index 721109c0cf994..dec36a7133821 100644 --- a/arch/x86/entry/common.c +++ b/arch/x86/entry/common.c @@ -288,13 +288,13 @@ __visible void do_syscall_64(unsigned long nr, struct pt_regs *regs) if (likely(nr < NR_syscalls)) { nr = array_index_nospec(nr, NR_syscalls); - regs->ax = sys_call_table[nr](regs); + regs->ax = x64_sys_call(regs, nr); #ifdef CONFIG_X86_X32_ABI } else if (likely((nr & __X32_SYSCALL_BIT) && (nr & ~__X32_SYSCALL_BIT) < X32_NR_syscalls)) { nr = array_index_nospec(nr & ~__X32_SYSCALL_BIT, X32_NR_syscalls); - regs->ax = x32_sys_call_table[nr](regs); + regs->ax = x32_sys_call(regs, nr); #endif } @@ -331,7 +331,7 @@ static __always_inline void do_syscall_32_irqs_on(struct pt_regs *regs) if (likely(nr < IA32_NR_syscalls)) { nr = array_index_nospec(nr, IA32_NR_syscalls); #ifdef CONFIG_IA32_EMULATION - regs->ax = ia32_sys_call_table[nr](regs); + regs->ax = ia32_sys_call(regs, nr); #else /* * It's possible that a 32-bit syscall implementation @@ -339,10 +339,7 @@ static __always_inline void do_syscall_32_irqs_on(struct pt_regs *regs) * the high bits are zero. Make sure we zero-extend all * of the args. */ - regs->ax = ia32_sys_call_table[nr]( - (unsigned int)regs->bx, (unsigned int)regs->cx, - (unsigned int)regs->dx, (unsigned int)regs->si, - (unsigned int)regs->di, (unsigned int)regs->bp); + regs->ax = ia32_sys_call(regs, nr); #endif /* CONFIG_IA32_EMULATION */ } diff --git a/arch/x86/entry/syscall_32.c b/arch/x86/entry/syscall_32.c index 7d17b3addbbb3..e4e186ab20567 100644 --- a/arch/x86/entry/syscall_32.c +++ b/arch/x86/entry/syscall_32.c @@ -30,3 +30,36 @@ __visible const sys_call_ptr_t ia32_sys_call_table[__NR_syscall_compat_max+1] = [0 ... __NR_syscall_compat_max] = &__sys_ni_syscall, #include }; +#undef __SYSCALL_I386 + +#ifdef CONFIG_IA32_EMULATION + +#define __SYSCALL_I386(nr, sym, qual) case nr: return sym(regs); + +long ia32_sys_call(const struct pt_regs *regs, unsigned int nr) +{ + switch (nr) { + #include + default: return __ia32_sys_ni_syscall(regs); + } +}; + +#else /* CONFIG_IA32_EMULATION */ + +#define __SYSCALL_I386(nr, sym, qual) case nr: return sym( \ + (unsigned int)regs->bx, (unsigned int)regs->cx, \ + (unsigned int)regs->dx, (unsigned int)regs->si, \ + (unsigned int)regs->di, (unsigned int)regs->bp); + +long ia32_sys_call(const struct pt_regs *regs, unsigned int nr) +{ + switch (nr) { + #include + default: return __ia32_sys_ni_syscall( + (unsigned int)regs->bx, (unsigned int)regs->cx, + (unsigned int)regs->dx, (unsigned int)regs->si, + (unsigned int)regs->di, (unsigned int)regs->bp); + } +}; + +#endif /* CONFIG_IA32_EMULATION */ diff --git a/arch/x86/entry/syscall_64.c b/arch/x86/entry/syscall_64.c index adf619a856e8d..197835442c265 100644 --- a/arch/x86/entry/syscall_64.c +++ b/arch/x86/entry/syscall_64.c @@ -54,3 +54,30 @@ asmlinkage const sys_call_ptr_t x32_sys_call_table[__NR_syscall_x32_max+1] = { #undef __SYSCALL_X32 #endif + +#define __SYSCALL_64(nr, sym, qual) case nr: return sym(regs); +#define __SYSCALL_X32(nr, sym, qual) + +long x64_sys_call(const struct pt_regs *regs, unsigned int nr) +{ + switch (nr) { + #include + default: return __x64_sys_ni_syscall(regs); + } +}; + +#ifdef CONFIG_X86_X32_ABI + +#define __SYSCALL_64(nr, sym, qual) +#define __SYSCALL_X32(nr, sym, qual) case nr: return sym(regs); +long x32_sys_call(const struct pt_regs *regs, unsigned int nr) +{ + switch (nr) { + #include + default: return __x64_sys_ni_syscall(regs); + } +} +#undef __SYSCALL_64 +#undef __SYSCALL_X32 + +#endif /* CONFIG_X86_X32_ABI */ diff --git a/arch/x86/include/asm/syscall.h b/arch/x86/include/asm/syscall.h index 8db3fdb6102ec..5d131e7206a26 100644 --- a/arch/x86/include/asm/syscall.h +++ b/arch/x86/include/asm/syscall.h @@ -40,6 +40,10 @@ extern const sys_call_ptr_t ia32_sys_call_table[]; extern const sys_call_ptr_t x32_sys_call_table[]; #endif +extern long ia32_sys_call(const struct pt_regs *, unsigned int nr); +extern long x32_sys_call(const struct pt_regs *, unsigned int nr); +extern long x64_sys_call(const struct pt_regs *, unsigned int nr); + /* * Only the low 32 bits of orig_ax are meaningful, so we return int. * This importantly ignores the high bits on 64-bit, so comparisons