From patchwork Mon Dec 27 14:59:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 1573384 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JN0vn0TV0z9sRN for ; Tue, 28 Dec 2021 01:50:29 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4JN0vj49WDz3cM5 for ; Tue, 28 Dec 2021 01:50:25 +1100 (AEDT) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=huawei.com (client-ip=45.249.212.188; helo=szxga02-in.huawei.com; envelope-from=wangkefeng.wang@huawei.com; receiver=) Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4JN0v55Qqwz2xF0 for ; Tue, 28 Dec 2021 01:49:52 +1100 (AEDT) Received: from dggpemm500020.china.huawei.com (unknown [172.30.72.57]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4JN0sq5Vv9z91vr; Mon, 27 Dec 2021 22:48:47 +0800 (CST) Received: from dggpemm500001.china.huawei.com (7.185.36.107) by dggpemm500020.china.huawei.com (7.185.36.49) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Mon, 27 Dec 2021 22:49:42 +0800 Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm500001.china.huawei.com (7.185.36.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Mon, 27 Dec 2021 22:49:41 +0800 From: Kefeng Wang To: Jonathan Corbet , Andrew Morton , , , , , , Subject: [PATCH v2 1/3] mm: vmalloc: Let user to control huge vmalloc default behavior Date: Mon, 27 Dec 2021 22:59:01 +0800 Message-ID: <20211227145903.187152-2-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20211227145903.187152-1-wangkefeng.wang@huawei.com> References: <20211227145903.187152-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm500001.china.huawei.com (7.185.36.107) X-CFilter-Loop: Reflected X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kefeng Wang , Matthew Wilcox , Catalin Marinas , Dave Hansen , Nicholas Piggin , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Paul Mackerras , Thomas Gleixner , Will Deacon Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Introduce HUGE_VMALLOC_DEFAULT_ENABLED and make it default y, this let user to choose whether or not enable huge vmalloc mappings by default. Meanwhile, add new hugevmalloc=on/off parameter to enable or disable this feature at boot time, nohugevmalloc is still supported and equivalent to hugevmalloc=off. Signed-off-by: Kefeng Wang --- .../admin-guide/kernel-parameters.txt | 12 ++++++++++++ mm/Kconfig | 8 ++++++++ mm/vmalloc.c | 18 +++++++++++++++++- 3 files changed, 37 insertions(+), 1 deletion(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index a069d8fe2fee..7b2f900fd243 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -1638,6 +1638,18 @@ If both parameters are enabled, hugetlb_free_vmemmap takes precedence over memory_hotplug.memmap_on_memory. + + hugevmalloc= [PPC] Reguires CONFIG_HAVE_ARCH_HUGE_VMALLOC + Format: { on | off } + Default set by CONFIG_HUGE_VMALLOC_DEFAULT_ENABLED. + + This parameter enables/disables kernel huge vmalloc + mappings at boot time. + + on: Enable the feature + off: Disable the feature + Equivalent to: nohugevmalloc + hung_task_panic= [KNL] Should the hung task detector generate panics. Format: 0 | 1 diff --git a/mm/Kconfig b/mm/Kconfig index a99bd499ef51..8d8a92f22905 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -262,6 +262,14 @@ config HUGETLB_PAGE_SIZE_VARIABLE HUGETLB_PAGE_ORDER when there are multiple HugeTLB page sizes available on a platform. +config HUGE_VMALLOC_DEFAULT_ENABLED + bool "Enable huge vmalloc mappings by default" + default y + depends on HAVE_ARCH_HUGE_VMALLOC + help + Enable huge vmalloc mappings by default, this value could be overridden + by hugevmalloc=off|on. + config CONTIG_ALLOC def_bool (MEMORY_ISOLATION && COMPACTION) || CMA diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 9bf838817a47..0d0f8deb5639 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -60,7 +60,7 @@ static const unsigned int ioremap_max_page_shift = PAGE_SHIFT; #endif /* CONFIG_HAVE_ARCH_HUGE_VMAP */ #ifdef CONFIG_HAVE_ARCH_HUGE_VMALLOC -static bool __ro_after_init vmap_allow_huge = true; +static bool __ro_after_init vmap_allow_huge = IS_ENABLED(CONFIG_HUGE_VMALLOC_DEFAULT_ENABLED); static int __init set_nohugevmalloc(char *str) { @@ -68,6 +68,22 @@ static int __init set_nohugevmalloc(char *str) return 0; } early_param("nohugevmalloc", set_nohugevmalloc); + +static int __init set_hugevmalloc(char *str) +{ + if (!str) + return -EINVAL; + + if (!strcmp(str, "on")) + vmap_allow_huge = true; + else if (!strcmp(str, "off")) + vmap_allow_huge = false; + else + return -EINVAL; + + return 0; +} +early_param("hugevmalloc", set_hugevmalloc); #else /* CONFIG_HAVE_ARCH_HUGE_VMALLOC */ static const bool vmap_allow_huge = false; #endif /* CONFIG_HAVE_ARCH_HUGE_VMALLOC */ From patchwork Mon Dec 27 14:59:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 1573387 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JN0wc5fk7z9sRN for ; Tue, 28 Dec 2021 01:51:12 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4JN0wc4p5kz3c9f for ; Tue, 28 Dec 2021 01:51:12 +1100 (AEDT) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=huawei.com (client-ip=45.249.212.255; helo=szxga08-in.huawei.com; envelope-from=wangkefeng.wang@huawei.com; receiver=) Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4JN0v55dpGz2xF1 for ; Tue, 28 Dec 2021 01:49:51 +1100 (AEDT) Received: from dggpemm500022.china.huawei.com (unknown [172.30.72.54]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4JN0q76hTkz1DKFm; Mon, 27 Dec 2021 22:46:27 +0800 (CST) Received: from dggpemm500001.china.huawei.com (7.185.36.107) by dggpemm500022.china.huawei.com (7.185.36.162) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Mon, 27 Dec 2021 22:49:43 +0800 Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm500001.china.huawei.com (7.185.36.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Mon, 27 Dec 2021 22:49:42 +0800 From: Kefeng Wang To: Jonathan Corbet , Andrew Morton , , , , , , Subject: [PATCH v2 2/3] arm64: Support huge vmalloc mappings Date: Mon, 27 Dec 2021 22:59:02 +0800 Message-ID: <20211227145903.187152-3-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20211227145903.187152-1-wangkefeng.wang@huawei.com> References: <20211227145903.187152-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm500001.china.huawei.com (7.185.36.107) X-CFilter-Loop: Reflected X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kefeng Wang , Matthew Wilcox , Catalin Marinas , Dave Hansen , Nicholas Piggin , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Paul Mackerras , Thomas Gleixner , Will Deacon Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" This patch select HAVE_ARCH_HUGE_VMALLOC to let arm64 support huge vmalloc mappings. Cc: Catalin Marinas Cc: Will Deacon Signed-off-by: Kefeng Wang --- Documentation/admin-guide/kernel-parameters.txt | 4 ++-- arch/arm64/Kconfig | 1 + arch/arm64/kernel/module.c | 5 +++-- 3 files changed, 6 insertions(+), 4 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 7b2f900fd243..e3f9fd7ec106 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -1639,7 +1639,7 @@ precedence over memory_hotplug.memmap_on_memory. - hugevmalloc= [PPC] Reguires CONFIG_HAVE_ARCH_HUGE_VMALLOC + hugevmalloc= [KNL,PPC,ARM64] Reguires CONFIG_HAVE_ARCH_HUGE_VMALLOC Format: { on | off } Default set by CONFIG_HUGE_VMALLOC_DEFAULT_ENABLED. @@ -3424,7 +3424,7 @@ nohugeiomap [KNL,X86,PPC,ARM64] Disable kernel huge I/O mappings. - nohugevmalloc [PPC] Disable kernel huge vmalloc mappings. + nohugevmalloc [KNL,PPC,ARM64] Disable kernel huge vmalloc mappings. nosmt [KNL,S390] Disable symmetric multithreading (SMT). Equivalent to smt=1. diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 3bb0b67292b5..c34bbb4482b0 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -142,6 +142,7 @@ config ARM64 select HAVE_ARCH_AUDITSYSCALL select HAVE_ARCH_BITREVERSE select HAVE_ARCH_COMPILER_H + select HAVE_ARCH_HUGE_VMALLOC select HAVE_ARCH_HUGE_VMAP select HAVE_ARCH_JUMP_LABEL select HAVE_ARCH_JUMP_LABEL_RELATIVE diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c index 309a27553c87..af7b4cbace2b 100644 --- a/arch/arm64/kernel/module.c +++ b/arch/arm64/kernel/module.c @@ -36,7 +36,8 @@ void *module_alloc(unsigned long size) module_alloc_end = MODULES_END; p = __vmalloc_node_range(size, MODULE_ALIGN, module_alloc_base, - module_alloc_end, gfp_mask, PAGE_KERNEL, VM_DEFER_KMEMLEAK, + module_alloc_end, gfp_mask, PAGE_KERNEL, + VM_DEFER_KMEMLEAK | VM_NO_HUGE_VMAP, NUMA_NO_NODE, __builtin_return_address(0)); if (!p && IS_ENABLED(CONFIG_ARM64_MODULE_PLTS) && @@ -55,7 +56,7 @@ void *module_alloc(unsigned long size) */ p = __vmalloc_node_range(size, MODULE_ALIGN, module_alloc_base, module_alloc_base + SZ_2G, GFP_KERNEL, - PAGE_KERNEL, 0, NUMA_NO_NODE, + PAGE_KERNEL, VM_NO_HUGE_VMAP, NUMA_NO_NODE, __builtin_return_address(0)); if (p && (kasan_module_alloc(p, size, gfp_mask) < 0)) { From patchwork Mon Dec 27 14:59:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 1573386 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JN0w92GbXz9sRN for ; Tue, 28 Dec 2021 01:50:49 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4JN0w91SWnz3cY2 for ; Tue, 28 Dec 2021 01:50:49 +1100 (AEDT) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=huawei.com (client-ip=45.249.212.189; helo=szxga03-in.huawei.com; envelope-from=wangkefeng.wang@huawei.com; receiver=) Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4JN0v55hKbz2xRn for ; Tue, 28 Dec 2021 01:49:53 +1100 (AEDT) Received: from dggpemm500023.china.huawei.com (unknown [172.30.72.54]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4JN0r63gHfz8w6s; Mon, 27 Dec 2021 22:47:18 +0800 (CST) Received: from dggpemm500001.china.huawei.com (7.185.36.107) by dggpemm500023.china.huawei.com (7.185.36.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Mon, 27 Dec 2021 22:49:44 +0800 Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm500001.china.huawei.com (7.185.36.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Mon, 27 Dec 2021 22:49:43 +0800 From: Kefeng Wang To: Jonathan Corbet , Andrew Morton , , , , , , Subject: [PATCH v2 3/3] x86: Support huge vmalloc mappings Date: Mon, 27 Dec 2021 22:59:03 +0800 Message-ID: <20211227145903.187152-4-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20211227145903.187152-1-wangkefeng.wang@huawei.com> References: <20211227145903.187152-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm500001.china.huawei.com (7.185.36.107) X-CFilter-Loop: Reflected X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kefeng Wang , Matthew Wilcox , Catalin Marinas , Dave Hansen , Nicholas Piggin , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Paul Mackerras , Thomas Gleixner , Will Deacon Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" This patch select HAVE_ARCH_HUGE_VMALLOC to let X86_64 and X86_PAE support huge vmalloc mappings. Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: Dave Hansen Cc: "H. Peter Anvin" Signed-off-by: Kefeng Wang --- Documentation/admin-guide/kernel-parameters.txt | 4 ++-- arch/x86/Kconfig | 1 + arch/x86/kernel/module.c | 4 ++-- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index e3f9fd7ec106..ffce6591ae64 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -1639,7 +1639,7 @@ precedence over memory_hotplug.memmap_on_memory. - hugevmalloc= [KNL,PPC,ARM64] Reguires CONFIG_HAVE_ARCH_HUGE_VMALLOC + hugevmalloc= [KNL,PPC,ARM64,X86] Reguires CONFIG_HAVE_ARCH_HUGE_VMALLOC Format: { on | off } Default set by CONFIG_HUGE_VMALLOC_DEFAULT_ENABLED. @@ -3424,7 +3424,7 @@ nohugeiomap [KNL,X86,PPC,ARM64] Disable kernel huge I/O mappings. - nohugevmalloc [KNL,PPC,ARM64] Disable kernel huge vmalloc mappings. + nohugevmalloc [KNL,PPC,ARM64,X86] Disable kernel huge vmalloc mappings. nosmt [KNL,S390] Disable symmetric multithreading (SMT). Equivalent to smt=1. diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index ebe8fc76949a..f6bf6675bbe7 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -157,6 +157,7 @@ config X86 select HAVE_ACPI_APEI_NMI if ACPI select HAVE_ALIGNED_STRUCT_PAGE if SLUB select HAVE_ARCH_AUDITSYSCALL + select HAVE_ARCH_HUGE_VMALLOC if HAVE_ARCH_HUGE_VMAP select HAVE_ARCH_HUGE_VMAP if X86_64 || X86_PAE select HAVE_ARCH_JUMP_LABEL select HAVE_ARCH_JUMP_LABEL_RELATIVE diff --git a/arch/x86/kernel/module.c b/arch/x86/kernel/module.c index 95fa745e310a..6bf5cb7d876a 100644 --- a/arch/x86/kernel/module.c +++ b/arch/x86/kernel/module.c @@ -75,8 +75,8 @@ void *module_alloc(unsigned long size) p = __vmalloc_node_range(size, MODULE_ALIGN, MODULES_VADDR + get_module_load_offset(), - MODULES_END, gfp_mask, - PAGE_KERNEL, VM_DEFER_KMEMLEAK, NUMA_NO_NODE, + MODULES_END, gfp_mask, PAGE_KERNEL, + VM_DEFER_KMEMLEAK | VM_NO_HUGE_VMAP, NUMA_NO_NODE, __builtin_return_address(0)); if (p && (kasan_module_alloc(p, size, gfp_mask) < 0)) { vfree(p);