From patchwork Thu Jun 8 02:10:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cengiz Can X-Patchwork-Id: 1791975 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=J0CnQA0o; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Qc74Z0y19z20WP for ; Thu, 8 Jun 2023 12:11:42 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1q757g-0001Pk-KE; Thu, 08 Jun 2023 02:11:36 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1q757e-0001Md-As for kernel-team@lists.ubuntu.com; Thu, 08 Jun 2023 02:11:34 +0000 Received: from mail-ej1-f71.google.com (mail-ej1-f71.google.com [209.85.218.71]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 957663F117 for ; Thu, 8 Jun 2023 02:11:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1686190292; bh=ltyPjrqzKPXo0tdGV5OFaNUrDPXGxeHJ2CwJSk7ED7k=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=J0CnQA0olf72C4gT2QT26ieuZBxPf6WvjwsZhjGMVDm9kChpvJR1++BekGLGOEaE9 llok4EITlmHURGrxgSeBD8SmV/4RGvoHHeo5BJuMBegeojPHsJBOLdrEzSbXtphmnQ 5lxSZIwUS+h64d4J8rYWvoPAdUIFg7Oi4iCstjteR4435nKfM7dOdKRvZDrcUky4CN SbTTdSNAMm26c9NtnCDcUo621ErfQkZEXqcF3/n9KoBkHVcVX4APCkbTL+CyNdhH+N NrkrwSnLcf2iiuovTPYE8sd82gMhkCE0UXIl66Mwafiz9DtJseprEHciMFlJTGWkpq lCkccDlyl8Bbg== Received: by mail-ej1-f71.google.com with SMTP id a640c23a62f3a-97467e06580so19068966b.1 for ; Wed, 07 Jun 2023 19:11:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686190292; x=1688782292; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ltyPjrqzKPXo0tdGV5OFaNUrDPXGxeHJ2CwJSk7ED7k=; b=aqKRmjPxmvff56jFkSn+cRYet/Tz2WDyi4a6xS5ZoXttbgqtr+61F4Syx9P0WEOWI2 VCxGzk5PMqtkhCTY1EZ/F0ZC7oq8TljvVMaNSp5J7/l/LOBxuOx+Ea58XJ1fPecwRP+i 2z8r7FeVCP4ZNUCBu8SSp8YQboEWA25PMQznpHTfml0DWI/AgGtRWINJI8g3o2d20/ko +iX7dghosz0Dbe2D03Y+GWy5i2KIcmeNkQczM6li3p0gV8mAol+EcQq4RhyqGDwdOKVb F4QeJrOwat84dS9RYXqQwwtAZpSbLa4nVs6Qfh52xuNZFqQ1EP2qJiUDygHh/xlO7/c9 P1Bw== X-Gm-Message-State: AC+VfDycBSeClHG7cqdF6LsgddzuTUnC9agUJmbLvvhtI48b8bnmLy1H AU9zAS7BVwIPPnw75Pb6hjLEAQbwleU5BlesroulGoo2LFst6a+tHVqWL9U3cpjoIH38Ue5CB8i kgnDnsdE9cia3+cAhRSF+IqnpRGOnSKR6xy6o/74zJaqJjYwa3kgKKyQ= X-Received: by 2002:a17:907:6ea0:b0:96f:cde5:5f5e with SMTP id sh32-20020a1709076ea000b0096fcde55f5emr9425260ejc.29.1686190292129; Wed, 07 Jun 2023 19:11:32 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4J9vNSy/0iXV0o1ILoWTZ7CpvVB+53toCz5L04rOyXOdJxAVjVETynJsO50wvHqErCZxz5bw== X-Received: by 2002:a17:907:6ea0:b0:96f:cde5:5f5e with SMTP id sh32-20020a1709076ea000b0096fcde55f5emr9425252ejc.29.1686190291819; Wed, 07 Jun 2023 19:11:31 -0700 (PDT) Received: from localhost ([82.222.124.85]) by smtp.gmail.com with ESMTPSA id bv15-20020a170906b1cf00b009596e7e0dbasm36341ejb.162.2023.06.07.19.11.31 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jun 2023 19:11:31 -0700 (PDT) From: Cengiz Can To: kernel-team@lists.ubuntu.com Subject: [SRU Jammy, OEM-5.17, Kinetic PATCH 2/5] x86/mm: Randomize per-cpu entry area Date: Thu, 8 Jun 2023 05:10:53 +0300 Message-Id: <20230608021055.203634-4-cengiz.can@canonical.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230608021055.203634-1-cengiz.can@canonical.com> References: <20230608021055.203634-1-cengiz.can@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Peter Zijlstra Seth found that the CPU-entry-area; the piece of per-cpu data that is mapped into the userspace page-tables for kPTI is not subject to any randomization -- irrespective of kASLR settings. On x86_64 a whole P4D (512 GB) of virtual address space is reserved for this structure, which is plenty large enough to randomize things a little. As such, use a straight forward randomization scheme that avoids duplicates to spread the existing CPUs over the available space. [ bp: Fix le build. ] Reported-by: Seth Jenkins Reviewed-by: Kees Cook Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Dave Hansen Signed-off-by: Borislav Petkov CVE-2023-0597 (backported from commit 97e3d26b5e5f371b3ee223d94dd123e6c442ba80) [cengizcan: include random.h for newly introduced `prandom_u32_max` call] Signed-off-by: Cengiz Can --- arch/x86/include/asm/cpu_entry_area.h | 4 --- arch/x86/include/asm/pgtable_areas.h | 8 ++++- arch/x86/kernel/hw_breakpoint.c | 2 +- arch/x86/mm/cpu_entry_area.c | 47 ++++++++++++++++++++++++--- 4 files changed, 51 insertions(+), 10 deletions(-) diff --git a/arch/x86/include/asm/cpu_entry_area.h b/arch/x86/include/asm/cpu_entry_area.h index 75efc4c6f076..462fc34f1317 100644 --- a/arch/x86/include/asm/cpu_entry_area.h +++ b/arch/x86/include/asm/cpu_entry_area.h @@ -130,10 +130,6 @@ struct cpu_entry_area { }; #define CPU_ENTRY_AREA_SIZE (sizeof(struct cpu_entry_area)) -#define CPU_ENTRY_AREA_ARRAY_SIZE (CPU_ENTRY_AREA_SIZE * NR_CPUS) - -/* Total size includes the readonly IDT mapping page as well: */ -#define CPU_ENTRY_AREA_TOTAL_SIZE (CPU_ENTRY_AREA_ARRAY_SIZE + PAGE_SIZE) DECLARE_PER_CPU(struct cpu_entry_area *, cpu_entry_area); DECLARE_PER_CPU(struct cea_exception_stacks *, cea_exception_stacks); diff --git a/arch/x86/include/asm/pgtable_areas.h b/arch/x86/include/asm/pgtable_areas.h index d34cce1b995c..4f056fb88174 100644 --- a/arch/x86/include/asm/pgtable_areas.h +++ b/arch/x86/include/asm/pgtable_areas.h @@ -11,6 +11,12 @@ #define CPU_ENTRY_AREA_RO_IDT_VADDR ((void *)CPU_ENTRY_AREA_RO_IDT) -#define CPU_ENTRY_AREA_MAP_SIZE (CPU_ENTRY_AREA_PER_CPU + CPU_ENTRY_AREA_ARRAY_SIZE - CPU_ENTRY_AREA_BASE) +#ifdef CONFIG_X86_32 +#define CPU_ENTRY_AREA_MAP_SIZE (CPU_ENTRY_AREA_PER_CPU + \ + (CPU_ENTRY_AREA_SIZE * NR_CPUS) - \ + CPU_ENTRY_AREA_BASE) +#else +#define CPU_ENTRY_AREA_MAP_SIZE P4D_SIZE +#endif #endif /* _ASM_X86_PGTABLE_AREAS_H */ diff --git a/arch/x86/kernel/hw_breakpoint.c b/arch/x86/kernel/hw_breakpoint.c index 668a4a6533d9..bbb0f737aab1 100644 --- a/arch/x86/kernel/hw_breakpoint.c +++ b/arch/x86/kernel/hw_breakpoint.c @@ -266,7 +266,7 @@ static inline bool within_cpu_entry(unsigned long addr, unsigned long end) /* CPU entry erea is always used for CPU entry */ if (within_area(addr, end, CPU_ENTRY_AREA_BASE, - CPU_ENTRY_AREA_TOTAL_SIZE)) + CPU_ENTRY_AREA_MAP_SIZE)) return true; /* diff --git a/arch/x86/mm/cpu_entry_area.c b/arch/x86/mm/cpu_entry_area.c index d7081b1accca..84495023ec2c 100644 --- a/arch/x86/mm/cpu_entry_area.c +++ b/arch/x86/mm/cpu_entry_area.c @@ -5,6 +5,7 @@ #include #include #include +#include #include #include @@ -16,16 +17,53 @@ static DEFINE_PER_CPU_PAGE_ALIGNED(struct entry_stack_page, entry_stack_storage) #ifdef CONFIG_X86_64 static DEFINE_PER_CPU_PAGE_ALIGNED(struct exception_stacks, exception_stacks); DEFINE_PER_CPU(struct cea_exception_stacks*, cea_exception_stacks); -#endif -#ifdef CONFIG_X86_32 +static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, _cea_offset); + +static __always_inline unsigned int cea_offset(unsigned int cpu) +{ + return per_cpu(_cea_offset, cpu); +} + +static __init void init_cea_offsets(void) +{ + unsigned int max_cea; + unsigned int i, j; + + max_cea = (CPU_ENTRY_AREA_MAP_SIZE - PAGE_SIZE) / CPU_ENTRY_AREA_SIZE; + + /* O(sodding terrible) */ + for_each_possible_cpu(i) { + unsigned int cea; + +again: + cea = prandom_u32_max(max_cea); + + for_each_possible_cpu(j) { + if (cea_offset(j) == cea) + goto again; + + if (i == j) + break; + } + + per_cpu(_cea_offset, i) = cea; + } +} +#else /* !X86_64 */ DECLARE_PER_CPU_PAGE_ALIGNED(struct doublefault_stack, doublefault_stack); + +static __always_inline unsigned int cea_offset(unsigned int cpu) +{ + return cpu; +} +static inline void init_cea_offsets(void) { } #endif /* Is called from entry code, so must be noinstr */ noinstr struct cpu_entry_area *get_cpu_entry_area(int cpu) { - unsigned long va = CPU_ENTRY_AREA_PER_CPU + cpu * CPU_ENTRY_AREA_SIZE; + unsigned long va = CPU_ENTRY_AREA_PER_CPU + cea_offset(cpu) * CPU_ENTRY_AREA_SIZE; BUILD_BUG_ON(sizeof(struct cpu_entry_area) % PAGE_SIZE != 0); return (struct cpu_entry_area *) va; @@ -211,7 +249,6 @@ static __init void setup_cpu_entry_area_ptes(void) /* The +1 is for the readonly IDT: */ BUILD_BUG_ON((CPU_ENTRY_AREA_PAGES+1)*PAGE_SIZE != CPU_ENTRY_AREA_MAP_SIZE); - BUILD_BUG_ON(CPU_ENTRY_AREA_TOTAL_SIZE != CPU_ENTRY_AREA_MAP_SIZE); BUG_ON(CPU_ENTRY_AREA_BASE & ~PMD_MASK); start = CPU_ENTRY_AREA_BASE; @@ -227,6 +264,8 @@ void __init setup_cpu_entry_areas(void) { unsigned int cpu; + init_cea_offsets(); + setup_cpu_entry_area_ptes(); for_each_possible_cpu(cpu)