| Message ID | 20250313135003.836600-11-rppt@kernel.org |
|---|---|
| State | New |
| Headers | show |
| Series | arch, mm: reduce code duplication in mem_init() | expand |
Hi Mike, Andrew, On Thu, Mar 13 2025, Mike Rapoport wrote: > From: "Mike Rapoport (Microsoft)" <rppt@kernel.org> > > high_memory defines upper bound on the directly mapped memory. > This bound is defined by the beginning of ZONE_HIGHMEM when a system has > high memory and by the end of memory otherwise. > > All this is known to generic memory management initialization code that > can set high_memory while initializing core mm structures. > > Add a generic calculation of high_memory to free_area_init() and remove > per-architecture calculation except for the architectures that set and > use high_memory earlier than that. > > Acked-by: Dave Hansen <dave.hansen@linux.intel.com> # x86 > Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org> > --- > arch/alpha/mm/init.c | 1 - > arch/arc/mm/init.c | 2 -- > arch/arm64/mm/init.c | 2 -- > arch/csky/mm/init.c | 1 - > arch/hexagon/mm/init.c | 6 ------ > arch/loongarch/kernel/numa.c | 1 - > arch/loongarch/mm/init.c | 2 -- > arch/microblaze/mm/init.c | 2 -- > arch/mips/mm/init.c | 2 -- > arch/nios2/mm/init.c | 6 ------ > arch/openrisc/mm/init.c | 2 -- > arch/parisc/mm/init.c | 1 - > arch/riscv/mm/init.c | 1 - > arch/s390/mm/init.c | 2 -- > arch/sh/mm/init.c | 7 ------- > arch/sparc/mm/init_32.c | 1 - > arch/sparc/mm/init_64.c | 2 -- > arch/um/kernel/um_arch.c | 1 - > arch/x86/kernel/setup.c | 2 -- > arch/x86/mm/init_32.c | 3 --- > arch/x86/mm/numa_32.c | 3 --- > arch/xtensa/mm/init.c | 2 -- > mm/memory.c | 8 -------- > mm/mm_init.c | 30 ++++++++++++++++++++++++++++++ > mm/nommu.c | 2 -- > 25 files changed, 30 insertions(+), 62 deletions(-) This patch causes a BUG() when built with CONFIG_DEBUG_VIRTUAL and passing in the cma= commandline parameter: ------------[ cut here ]------------ kernel BUG at arch/x86/mm/physaddr.c:23! ception 0x06 IP 10:ffffffff812ebbf8 error 0 cr2 0xffff88903ffff000 CPU: 0 UID: 0 PID: 0 Comm: swapper Not tainted 6.15.0-rc6+ #231 PREEMPT(undef) Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.3-1-1 04/01/2014 RIP: 0010:__phys_addr+0x58/0x60 Code: 01 48 89 c2 48 d3 ea 48 85 d2 75 05 e9 91 52 cf 00 0f 0b 48 3d ff ff ff 1f 77 0f 48 8b 05 20 54 55 01 48 01 d0 e9 78 52 cf 00 <0f> 0b 90 0f 1f 44 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 RSP: 0000:ffffffff82803dd8 EFLAGS: 00010006 ORIG_RAX: 0000000000000000 RAX: 000000007fffffff RBX: 00000000ffffffff RCX: 0000000000000000 RDX: 000000007fffffff RSI: 0000000280000000 RDI: ffffffffffffffff RBP: ffffffff82803e68 R08: 0000000000000000 R09: 0000000000000000 R10: ffffffff83153180 R11: ffffffff82803e48 R12: ffffffff83c9aed0 R13: 0000000000000000 R14: 0000001040000000 R15: 0000000000000000 FS: 0000000000000000(0000) GS:0000000000000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: ffff88903ffff000 CR3: 0000000002838000 CR4: 00000000000000b0 Call Trace: <TASK> ? __cma_declare_contiguous_nid+0x6e/0x340 ? cma_declare_contiguous_nid+0x33/0x70 ? dma_contiguous_reserve_area+0x2f/0x70 ? setup_arch+0x6f1/0x870 ? start_kernel+0x52/0x4b0 ? x86_64_start_reservations+0x29/0x30 ? x86_64_start_kernel+0x7c/0x80 ? common_startup_64+0x13e/0x141 The reason is that __cma_declare_contiguous_nid() does: highmem_start = __pa(high_memory - 1) + 1; If dma_contiguous_reserve_area() (or any other CMA declaration) is called before free_area_init(), high_memory is uninitialized. Without CONFIG_DEBUG_VIRTUAL, it will likely work but use the wrong value for highmem_start. Among the architectures this patch touches, the below call dma_contiguous_reserve_area() _before_ free_area_init(): - x86 - s390 - mips - riscv - xtensa - loongarch - csky The below call it _after_ free_area_init(): - arm64 And the below don't call it at all: - sparc - nios2 - openrisc - hexagon - sh - um - alpha One possible fix would be to move the calls to dma_contiguous_reserve_area() after free_area_init(). On x86, it would look like the diff below. The obvious downside is that moving the call later increases the chances of allocation failure. I'm not sure how much that actually matters, but at least on x86, that means crash kernel and hugetlb reservations go before DMA reservation. Also, adding a patch like that at rc7 is a bit risky. The other option would be to revert this. I tried a revert, but it isn't trivial. It runs into merge conflicts in pretty much all of the arch files. Maybe reverting patches 11, 12, and 13 as well would make it easier but I didn't try that. Which option should we take? If we want to move dma_contiguous_reserve_area() a bit further down the line then I can send a patch doing that on the rest of the architectures. Otherwise I can try my hand at the revert. --- 8< --- diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 9d2a13b37833..ca6928dde0c9 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -1160,7 +1160,6 @@ void __init setup_arch(char **cmdline_p) x86_flattree_get_config(); initmem_init(); - dma_contiguous_reserve(max_pfn_mapped << PAGE_SHIFT); if (boot_cpu_has(X86_FEATURE_GBPAGES)) { hugetlb_cma_reserve(PUD_SHIFT - PAGE_SHIFT); @@ -1178,6 +1177,8 @@ void __init setup_arch(char **cmdline_p) x86_init.paging.pagetable_init(); + dma_contiguous_reserve(max_pfn_mapped << PAGE_SHIFT); + kasan_init(); /*
Hi Pratyush, On Fri, May 16, 2025 at 05:28:17PM +0200, Pratyush Yadav wrote: > Hi Mike, Andrew, > > On Thu, Mar 13 2025, Mike Rapoport wrote: > > > From: "Mike Rapoport (Microsoft)" <rppt@kernel.org> > > > > high_memory defines upper bound on the directly mapped memory. > > This bound is defined by the beginning of ZONE_HIGHMEM when a system has > > high memory and by the end of memory otherwise. > > > > All this is known to generic memory management initialization code that > > can set high_memory while initializing core mm structures. > > > > Add a generic calculation of high_memory to free_area_init() and remove > > per-architecture calculation except for the architectures that set and > > use high_memory earlier than that. > > > > Acked-by: Dave Hansen <dave.hansen@linux.intel.com> # x86 > > Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org> > > --- > > arch/alpha/mm/init.c | 1 - > > arch/arc/mm/init.c | 2 -- > > arch/arm64/mm/init.c | 2 -- > > arch/csky/mm/init.c | 1 - > > arch/hexagon/mm/init.c | 6 ------ > > arch/loongarch/kernel/numa.c | 1 - > > arch/loongarch/mm/init.c | 2 -- > > arch/microblaze/mm/init.c | 2 -- > > arch/mips/mm/init.c | 2 -- > > arch/nios2/mm/init.c | 6 ------ > > arch/openrisc/mm/init.c | 2 -- > > arch/parisc/mm/init.c | 1 - > > arch/riscv/mm/init.c | 1 - > > arch/s390/mm/init.c | 2 -- > > arch/sh/mm/init.c | 7 ------- > > arch/sparc/mm/init_32.c | 1 - > > arch/sparc/mm/init_64.c | 2 -- > > arch/um/kernel/um_arch.c | 1 - > > arch/x86/kernel/setup.c | 2 -- > > arch/x86/mm/init_32.c | 3 --- > > arch/x86/mm/numa_32.c | 3 --- > > arch/xtensa/mm/init.c | 2 -- > > mm/memory.c | 8 -------- > > mm/mm_init.c | 30 ++++++++++++++++++++++++++++++ > > mm/nommu.c | 2 -- > > 25 files changed, 30 insertions(+), 62 deletions(-) > > This patch causes a BUG() when built with CONFIG_DEBUG_VIRTUAL and > passing in the cma= commandline parameter: > > ------------[ cut here ]------------ > kernel BUG at arch/x86/mm/physaddr.c:23! > ception 0x06 IP 10:ffffffff812ebbf8 error 0 cr2 0xffff88903ffff000 > CPU: 0 UID: 0 PID: 0 Comm: swapper Not tainted 6.15.0-rc6+ #231 PREEMPT(undef) > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.3-1-1 04/01/2014 > RIP: 0010:__phys_addr+0x58/0x60 > Code: 01 48 89 c2 48 d3 ea 48 85 d2 75 05 e9 91 52 cf 00 0f 0b 48 3d ff ff ff 1f 77 0f 48 8b 05 20 54 55 01 48 01 d0 e9 78 52 cf 00 <0f> 0b 90 0f 1f 44 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 > RSP: 0000:ffffffff82803dd8 EFLAGS: 00010006 ORIG_RAX: 0000000000000000 > RAX: 000000007fffffff RBX: 00000000ffffffff RCX: 0000000000000000 > RDX: 000000007fffffff RSI: 0000000280000000 RDI: ffffffffffffffff > RBP: ffffffff82803e68 R08: 0000000000000000 R09: 0000000000000000 > R10: ffffffff83153180 R11: ffffffff82803e48 R12: ffffffff83c9aed0 > R13: 0000000000000000 R14: 0000001040000000 R15: 0000000000000000 > FS: 0000000000000000(0000) GS:0000000000000000(0000) knlGS:0000000000000000 > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > CR2: ffff88903ffff000 CR3: 0000000002838000 CR4: 00000000000000b0 > Call Trace: > <TASK> > ? __cma_declare_contiguous_nid+0x6e/0x340 > ? cma_declare_contiguous_nid+0x33/0x70 > ? dma_contiguous_reserve_area+0x2f/0x70 > ? setup_arch+0x6f1/0x870 > ? start_kernel+0x52/0x4b0 > ? x86_64_start_reservations+0x29/0x30 > ? x86_64_start_kernel+0x7c/0x80 > ? common_startup_64+0x13e/0x141 > > The reason is that __cma_declare_contiguous_nid() does: > > highmem_start = __pa(high_memory - 1) + 1; > > If dma_contiguous_reserve_area() (or any other CMA declaration) is > called before free_area_init(), high_memory is uninitialized. Without > CONFIG_DEBUG_VIRTUAL, it will likely work but use the wrong value for > highmem_start. > > Among the architectures this patch touches, the below call > dma_contiguous_reserve_area() _before_ free_area_init(): > > - x86 > - s390 > - mips > - riscv > - xtensa > - loongarch > - csky For most of those this patch didn't really change anything because they initialized high_memory in mem_init() which is a part of free_area_init(). In those cases cma just did highmem_start = __pa(-1) + 1; and everyone was happy :) > The below call it _after_ free_area_init(): > - arm64 > > And the below don't call it at all: > - sparc > - nios2 > - openrisc > - hexagon > - sh > - um > - alpha > > One possible fix would be to move the calls to > dma_contiguous_reserve_area() after free_area_init(). On x86, it would > look like the diff below. The obvious downside is that moving the call > later increases the chances of allocation failure. I'm not sure how much > that actually matters, but at least on x86, that means crash kernel and > hugetlb reservations go before DMA reservation. Also, adding a patch > like that at rc7 is a bit risky. I don't think there's a risk of allocation failure, but moving things around in setup_arch() is always risky :) > The other option would be to revert this. I tried a revert, but it isn't > trivial. It runs into merge conflicts in pretty much all of the arch > files. Maybe reverting patches 11, 12, and 13 as well would make it > easier but I didn't try that. What I think we can do is to add this to mm/cma.c (not even compile tested) diff --git a/mm/cma.c b/mm/cma.c index 15632939f20a..c04be488b099 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -608,7 +608,10 @@ static int __init __cma_declare_contiguous_nid(phys_addr_t *basep, * complain. Find the boundary by adding one to the last valid * address. */ - highmem_start = __pa(high_memory - 1) + 1; + if (IS_ENABLED(CONFIG_HIGHMEM)) + highmem_start = __pa(high_memory - 1) + 1; + else + highmem_start = memblock_end_of_DRAM(); pr_debug("%s(size %pa, base %pa, limit %pa alignment %pa)\n", __func__, &size, &base, &limit, &alignment); so that highmem_start in __cma_declare_contiguous_nid() will be always correct for !HIGHMEM configs and then restore setting of highmem_start in mips::paging_init() as mips is the only architecture that actually set high_memory before free_area_init() before this patch. (for 32 bit configs of x86 there alrady a fixup d893aca973c3 ("x86/mm: restore early initialization of high_memory for 32-bits"))
Hi Mike, On 5/16/25 19:01, Mike Rapoport wrote: > Hi Pratyush, > > On Fri, May 16, 2025 at 05:28:17PM +0200, Pratyush Yadav wrote: >> Hi Mike, Andrew, >> >> On Thu, Mar 13 2025, Mike Rapoport wrote: >> >>> From: "Mike Rapoport (Microsoft)" <rppt@kernel.org> >>> >>> high_memory defines upper bound on the directly mapped memory. >>> This bound is defined by the beginning of ZONE_HIGHMEM when a system has >>> high memory and by the end of memory otherwise. >>> >>> All this is known to generic memory management initialization code that >>> can set high_memory while initializing core mm structures. >>> >>> Add a generic calculation of high_memory to free_area_init() and remove >>> per-architecture calculation except for the architectures that set and >>> use high_memory earlier than that. >>> >>> Acked-by: Dave Hansen <dave.hansen@linux.intel.com> # x86 >>> Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org> >>> --- >>> arch/alpha/mm/init.c | 1 - >>> arch/arc/mm/init.c | 2 -- >>> arch/arm64/mm/init.c | 2 -- >>> arch/csky/mm/init.c | 1 - >>> arch/hexagon/mm/init.c | 6 ------ >>> arch/loongarch/kernel/numa.c | 1 - >>> arch/loongarch/mm/init.c | 2 -- >>> arch/microblaze/mm/init.c | 2 -- >>> arch/mips/mm/init.c | 2 -- >>> arch/nios2/mm/init.c | 6 ------ >>> arch/openrisc/mm/init.c | 2 -- >>> arch/parisc/mm/init.c | 1 - >>> arch/riscv/mm/init.c | 1 - >>> arch/s390/mm/init.c | 2 -- >>> arch/sh/mm/init.c | 7 ------- >>> arch/sparc/mm/init_32.c | 1 - >>> arch/sparc/mm/init_64.c | 2 -- >>> arch/um/kernel/um_arch.c | 1 - >>> arch/x86/kernel/setup.c | 2 -- >>> arch/x86/mm/init_32.c | 3 --- >>> arch/x86/mm/numa_32.c | 3 --- >>> arch/xtensa/mm/init.c | 2 -- >>> mm/memory.c | 8 -------- >>> mm/mm_init.c | 30 ++++++++++++++++++++++++++++++ >>> mm/nommu.c | 2 -- >>> 25 files changed, 30 insertions(+), 62 deletions(-) >> This patch causes a BUG() when built with CONFIG_DEBUG_VIRTUAL and >> passing in the cma= commandline parameter: >> >> ------------[ cut here ]------------ >> kernel BUG at arch/x86/mm/physaddr.c:23! >> ception 0x06 IP 10:ffffffff812ebbf8 error 0 cr2 0xffff88903ffff000 >> CPU: 0 UID: 0 PID: 0 Comm: swapper Not tainted 6.15.0-rc6+ #231 PREEMPT(undef) >> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.3-1-1 04/01/2014 >> RIP: 0010:__phys_addr+0x58/0x60 >> Code: 01 48 89 c2 48 d3 ea 48 85 d2 75 05 e9 91 52 cf 00 0f 0b 48 3d ff ff ff 1f 77 0f 48 8b 05 20 54 55 01 48 01 d0 e9 78 52 cf 00 <0f> 0b 90 0f 1f 44 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 >> RSP: 0000:ffffffff82803dd8 EFLAGS: 00010006 ORIG_RAX: 0000000000000000 >> RAX: 000000007fffffff RBX: 00000000ffffffff RCX: 0000000000000000 >> RDX: 000000007fffffff RSI: 0000000280000000 RDI: ffffffffffffffff >> RBP: ffffffff82803e68 R08: 0000000000000000 R09: 0000000000000000 >> R10: ffffffff83153180 R11: ffffffff82803e48 R12: ffffffff83c9aed0 >> R13: 0000000000000000 R14: 0000001040000000 R15: 0000000000000000 >> FS: 0000000000000000(0000) GS:0000000000000000(0000) knlGS:0000000000000000 >> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >> CR2: ffff88903ffff000 CR3: 0000000002838000 CR4: 00000000000000b0 >> Call Trace: >> <TASK> >> ? __cma_declare_contiguous_nid+0x6e/0x340 >> ? cma_declare_contiguous_nid+0x33/0x70 >> ? dma_contiguous_reserve_area+0x2f/0x70 >> ? setup_arch+0x6f1/0x870 >> ? start_kernel+0x52/0x4b0 >> ? x86_64_start_reservations+0x29/0x30 >> ? x86_64_start_kernel+0x7c/0x80 >> ? common_startup_64+0x13e/0x141 >> >> The reason is that __cma_declare_contiguous_nid() does: >> >> highmem_start = __pa(high_memory - 1) + 1; >> >> If dma_contiguous_reserve_area() (or any other CMA declaration) is >> called before free_area_init(), high_memory is uninitialized. Without >> CONFIG_DEBUG_VIRTUAL, it will likely work but use the wrong value for >> highmem_start. >> >> Among the architectures this patch touches, the below call >> dma_contiguous_reserve_area() _before_ free_area_init(): >> >> - x86 >> - s390 >> - mips >> - riscv >> - xtensa >> - loongarch >> - csky > For most of those this patch didn't really change anything because they > initialized high_memory in mem_init() which is a part of free_area_init(). > In those cases cma just did > > highmem_start = __pa(-1) + 1; > > and everyone was happy :) > >> The below call it _after_ free_area_init(): >> - arm64 >> >> And the below don't call it at all: >> - sparc >> - nios2 >> - openrisc >> - hexagon >> - sh >> - um >> - alpha >> >> One possible fix would be to move the calls to >> dma_contiguous_reserve_area() after free_area_init(). On x86, it would >> look like the diff below. The obvious downside is that moving the call >> later increases the chances of allocation failure. I'm not sure how much >> that actually matters, but at least on x86, that means crash kernel and >> hugetlb reservations go before DMA reservation. Also, adding a patch >> like that at rc7 is a bit risky. > I don't think there's a risk of allocation failure, but moving things > around in setup_arch() is always risky :) > >> The other option would be to revert this. I tried a revert, but it isn't >> trivial. It runs into merge conflicts in pretty much all of the arch >> files. Maybe reverting patches 11, 12, and 13 as well would make it >> easier but I didn't try that. > What I think we can do is to add this to mm/cma.c (not even compile tested) > > diff --git a/mm/cma.c b/mm/cma.c > index 15632939f20a..c04be488b099 100644 > --- a/mm/cma.c > +++ b/mm/cma.c > @@ -608,7 +608,10 @@ static int __init __cma_declare_contiguous_nid(phys_addr_t *basep, > * complain. Find the boundary by adding one to the last valid > * address. > */ > - highmem_start = __pa(high_memory - 1) + 1; > + if (IS_ENABLED(CONFIG_HIGHMEM)) > + highmem_start = __pa(high_memory - 1) + 1; > + else > + highmem_start = memblock_end_of_DRAM(); > pr_debug("%s(size %pa, base %pa, limit %pa alignment %pa)\n", > __func__, &size, &base, &limit, &alignment); I encountered the same error as Pratyush and the above diff fixes it: do you plan on sending this fix for 6.15? If so, you can add: Tested-by: Alexandre Ghiti <alexghiti@rivosinc.com> If not, let me know how you want to proceed :) Thanks, Alex > > so that highmem_start in __cma_declare_contiguous_nid() will be always > correct for !HIGHMEM configs and then restore setting of highmem_start in > mips::paging_init() as mips is the only architecture that actually set > high_memory before free_area_init() before this patch. > > (for 32 bit configs of x86 there alrady a fixup d893aca973c3 ("x86/mm: restore > early initialization of high_memory for 32-bits")) >
Hi Alexandre, On Mon, May 19, 2025 at 05:54:23PM +0200, Alexandre Ghiti wrote: > Hi Mike, > > I encountered the same error as Pratyush and the above diff fixes it: do you > plan on sending this fix for 6.15? > > If so, you can add: > > Tested-by: Alexandre Ghiti <alexghiti@rivosinc.com> Thanks! Here's the patch: https://lore.kernel.org/linux-mm/20250519171805.1288393-1-rppt@kernel.org > Thanks, > Alex
diff --git a/arch/alpha/mm/init.c b/arch/alpha/mm/init.c index ec0eeae9c653..3ab2d2f3c917 100644 --- a/arch/alpha/mm/init.c +++ b/arch/alpha/mm/init.c @@ -276,7 +276,6 @@ srm_paging_stop (void) void __init mem_init(void) { - high_memory = (void *) __va(max_low_pfn * PAGE_SIZE); memblock_free_all(); } diff --git a/arch/arc/mm/init.c b/arch/arc/mm/init.c index 7ef883d58dc1..05025122e965 100644 --- a/arch/arc/mm/init.c +++ b/arch/arc/mm/init.c @@ -150,8 +150,6 @@ void __init setup_arch_memory(void) */ max_zone_pfn[ZONE_HIGHMEM] = max_high_pfn; - high_memory = (void *)(min_high_pfn << PAGE_SHIFT); - arch_pfn_offset = min(min_low_pfn, min_high_pfn); kmap_init(); #endif /* CONFIG_HIGHMEM */ diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 9c0b8d9558fc..a48fcccd67fa 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -314,8 +314,6 @@ void __init arm64_memblock_init(void) } early_init_fdt_scan_reserved_mem(); - - high_memory = __va(memblock_end_of_DRAM() - 1) + 1; } void __init bootmem_init(void) diff --git a/arch/csky/mm/init.c b/arch/csky/mm/init.c index ba6694d6170a..a22801aa503a 100644 --- a/arch/csky/mm/init.c +++ b/arch/csky/mm/init.c @@ -47,7 +47,6 @@ void __init mem_init(void) #ifdef CONFIG_HIGHMEM unsigned long tmp; #endif - high_memory = (void *) __va(max_low_pfn << PAGE_SHIFT); memblock_free_all(); diff --git a/arch/hexagon/mm/init.c b/arch/hexagon/mm/init.c index 508bb6a8dcc9..d412c2314509 100644 --- a/arch/hexagon/mm/init.c +++ b/arch/hexagon/mm/init.c @@ -100,12 +100,6 @@ static void __init paging_init(void) * initial kernel segment table's physical address. */ init_mm.context.ptbase = __pa(init_mm.pgd); - - /* - * Start of high memory area. Will probably need something more - * fancy if we... get more fancy. - */ - high_memory = (void *)((bootmem_lastpg + 1) << PAGE_SHIFT); } #ifndef DMA_RESERVE diff --git a/arch/loongarch/kernel/numa.c b/arch/loongarch/kernel/numa.c index 84fe7f854820..8eb489725b1a 100644 --- a/arch/loongarch/kernel/numa.c +++ b/arch/loongarch/kernel/numa.c @@ -389,7 +389,6 @@ void __init paging_init(void) void __init mem_init(void) { - high_memory = (void *) __va(max_low_pfn << PAGE_SHIFT); memblock_free_all(); } diff --git a/arch/loongarch/mm/init.c b/arch/loongarch/mm/init.c index 00449df50db1..6affa3609188 100644 --- a/arch/loongarch/mm/init.c +++ b/arch/loongarch/mm/init.c @@ -78,8 +78,6 @@ void __init paging_init(void) void __init mem_init(void) { - high_memory = (void *) __va(max_low_pfn << PAGE_SHIFT); - memblock_free_all(); } #endif /* !CONFIG_NUMA */ diff --git a/arch/microblaze/mm/init.c b/arch/microblaze/mm/init.c index 857cd2b44bcf..7e2e342e84c5 100644 --- a/arch/microblaze/mm/init.c +++ b/arch/microblaze/mm/init.c @@ -120,8 +120,6 @@ void __init setup_memory(void) void __init mem_init(void) { - high_memory = (void *)__va(memory_start + lowmem_size - 1); - /* this will put all memory onto the freelists */ memblock_free_all(); #ifdef CONFIG_HIGHMEM diff --git a/arch/mips/mm/init.c b/arch/mips/mm/init.c index eb61a73520a0..ed9dde6a00f7 100644 --- a/arch/mips/mm/init.c +++ b/arch/mips/mm/init.c @@ -417,7 +417,6 @@ void __init paging_init(void) max_zone_pfns[ZONE_HIGHMEM] = max_low_pfn; } #endif - high_memory = (void *) __va(max_low_pfn << PAGE_SHIFT); free_area_init(max_zone_pfns); } @@ -469,7 +468,6 @@ void __init mem_init(void) #else /* CONFIG_NUMA */ void __init mem_init(void) { - high_memory = (void *) __va(get_num_physpages() << PAGE_SHIFT); setup_zero_pages(); /* This comes from node 0 */ memblock_free_all(); } diff --git a/arch/nios2/mm/init.c b/arch/nios2/mm/init.c index 3cafa87ead9e..4ba8dfa0d238 100644 --- a/arch/nios2/mm/init.c +++ b/arch/nios2/mm/init.c @@ -62,12 +62,6 @@ void __init paging_init(void) void __init mem_init(void) { - unsigned long end_mem = memory_end; /* this must not include - kernel stack at top */ - - end_mem &= PAGE_MASK; - high_memory = __va(end_mem); - /* this will put all memory onto the freelists */ memblock_free_all(); } diff --git a/arch/openrisc/mm/init.c b/arch/openrisc/mm/init.c index 9093c336e158..72c5952607ac 100644 --- a/arch/openrisc/mm/init.c +++ b/arch/openrisc/mm/init.c @@ -193,8 +193,6 @@ void __init mem_init(void) { BUG_ON(!mem_map); - high_memory = (void *)__va(max_low_pfn * PAGE_SIZE); - /* clear the zero-page */ memset((void *)empty_zero_page, 0, PAGE_SIZE); diff --git a/arch/parisc/mm/init.c b/arch/parisc/mm/init.c index 2cdfc0b1195c..4fbe354dc9b4 100644 --- a/arch/parisc/mm/init.c +++ b/arch/parisc/mm/init.c @@ -562,7 +562,6 @@ void __init mem_init(void) BUILD_BUG_ON(TMPALIAS_MAP_START >= 0x80000000); #endif - high_memory = __va((max_pfn << PAGE_SHIFT)); memblock_free_all(); #ifdef CONFIG_PA11 diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index 157c9ca51541..ac6d41e86243 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -295,7 +295,6 @@ static void __init setup_bootmem(void) phys_ram_end = memblock_end_of_DRAM(); min_low_pfn = PFN_UP(phys_ram_base); max_low_pfn = max_pfn = PFN_DOWN(phys_ram_end); - high_memory = (void *)(__va(PFN_PHYS(max_low_pfn))); dma32_phys_limit = min(4UL * SZ_1G, (unsigned long)PFN_PHYS(max_low_pfn)); diff --git a/arch/s390/mm/init.c b/arch/s390/mm/init.c index 36fed6dffb38..0c485884d373 100644 --- a/arch/s390/mm/init.c +++ b/arch/s390/mm/init.c @@ -159,8 +159,6 @@ void __init mem_init(void) cpumask_set_cpu(0, &init_mm.context.cpu_attach_mask); cpumask_set_cpu(0, mm_cpumask(&init_mm)); - high_memory = (void *) __va(max_low_pfn * PAGE_SIZE); - pv_init(); kfence_split_mapping(); diff --git a/arch/sh/mm/init.c b/arch/sh/mm/init.c index 72aea5cd1b85..6d459ffba4bc 100644 --- a/arch/sh/mm/init.c +++ b/arch/sh/mm/init.c @@ -330,13 +330,6 @@ unsigned int mem_init_done = 0; void __init mem_init(void) { - pg_data_t *pgdat; - - high_memory = NULL; - for_each_online_pgdat(pgdat) - high_memory = max_t(void *, high_memory, - __va(pgdat_end_pfn(pgdat) << PAGE_SHIFT)); - memblock_free_all(); /* Set this up early, so we can take care of the zero page */ diff --git a/arch/sparc/mm/init_32.c b/arch/sparc/mm/init_32.c index 6b58da14edc6..81a468a9c223 100644 --- a/arch/sparc/mm/init_32.c +++ b/arch/sparc/mm/init_32.c @@ -275,7 +275,6 @@ void __init mem_init(void) taint_real_pages(); - high_memory = __va(max_low_pfn << PAGE_SHIFT); memblock_free_all(); for (i = 0; sp_banks[i].num_bytes != 0; i++) { diff --git a/arch/sparc/mm/init_64.c b/arch/sparc/mm/init_64.c index 05882bca5b73..34d46adb9571 100644 --- a/arch/sparc/mm/init_64.c +++ b/arch/sparc/mm/init_64.c @@ -2505,8 +2505,6 @@ static void __init register_page_bootmem_info(void) } void __init mem_init(void) { - high_memory = __va(last_valid_pfn << PAGE_SHIFT); - memblock_free_all(); /* diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c index 6414cbf00572..f24a3ce37ab7 100644 --- a/arch/um/kernel/um_arch.c +++ b/arch/um/kernel/um_arch.c @@ -385,7 +385,6 @@ int __init linux_main(int argc, char **argv, char **envp) high_physmem = uml_physmem + physmem_size; end_iomem = high_physmem + iomem_size; - high_memory = (void *) end_iomem; start_vm = VMALLOC_START; diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index cebee310e200..5c9ec876915e 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -972,8 +972,6 @@ void __init setup_arch(char **cmdline_p) max_low_pfn = e820__end_of_low_ram_pfn(); else max_low_pfn = max_pfn; - - high_memory = (void *)__va(max_pfn * PAGE_SIZE - 1) + 1; #endif /* Find and reserve MPTABLE area */ diff --git a/arch/x86/mm/init_32.c b/arch/x86/mm/init_32.c index 6d2f8cb9451e..801b659ead0c 100644 --- a/arch/x86/mm/init_32.c +++ b/arch/x86/mm/init_32.c @@ -643,9 +643,6 @@ void __init initmem_init(void) highstart_pfn = max_low_pfn; printk(KERN_NOTICE "%ldMB HIGHMEM available.\n", pages_to_mb(highend_pfn - highstart_pfn)); - high_memory = (void *) __va(highstart_pfn * PAGE_SIZE - 1) + 1; -#else - high_memory = (void *) __va(max_low_pfn * PAGE_SIZE - 1) + 1; #endif memblock_set_node(0, PHYS_ADDR_MAX, &memblock.memory, 0); diff --git a/arch/x86/mm/numa_32.c b/arch/x86/mm/numa_32.c index 65fda406e6f2..442ef3facff0 100644 --- a/arch/x86/mm/numa_32.c +++ b/arch/x86/mm/numa_32.c @@ -41,9 +41,6 @@ void __init initmem_init(void) highstart_pfn = max_low_pfn; printk(KERN_NOTICE "%ldMB HIGHMEM available.\n", pages_to_mb(highend_pfn - highstart_pfn)); - high_memory = (void *) __va(highstart_pfn * PAGE_SIZE - 1) + 1; -#else - high_memory = (void *) __va(max_low_pfn * PAGE_SIZE - 1) + 1; #endif printk(KERN_NOTICE "%ldMB LOWMEM available.\n", pages_to_mb(max_low_pfn)); diff --git a/arch/xtensa/mm/init.c b/arch/xtensa/mm/init.c index 9f1b0d5fccc7..9b662477b3d4 100644 --- a/arch/xtensa/mm/init.c +++ b/arch/xtensa/mm/init.c @@ -164,8 +164,6 @@ void __init mem_init(void) { free_highpages(); - high_memory = (void *)__va(max_low_pfn << PAGE_SHIFT); - memblock_free_all(); } diff --git a/mm/memory.c b/mm/memory.c index 126fdd3001e3..2351f3f6b9ed 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -113,14 +113,6 @@ static __always_inline bool vmf_orig_pte_uffd_wp(struct vm_fault *vmf) return pte_marker_uffd_wp(vmf->orig_pte); } -/* - * A number of key systems in x86 including ioremap() rely on the assumption - * that high_memory defines the upper bound on direct map memory, then end - * of ZONE_NORMAL. - */ -void *high_memory; -EXPORT_SYMBOL(high_memory); - /* * Randomize the address space (stacks, mmaps, brk, etc.). * diff --git a/mm/mm_init.c b/mm/mm_init.c index 50a93714e1c6..61ac0cea711a 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -44,6 +44,13 @@ struct page *mem_map; EXPORT_SYMBOL(mem_map); #endif +/* + * high_memory defines the upper bound on direct map memory, then end + * of ZONE_NORMAL. + */ +void *high_memory; +EXPORT_SYMBOL(high_memory); + #ifdef CONFIG_DEBUG_MEMORY_INIT int __meminitdata mminit_loglevel; @@ -1756,6 +1763,27 @@ static bool arch_has_descending_max_zone_pfns(void) return IS_ENABLED(CONFIG_ARC) && !IS_ENABLED(CONFIG_ARC_HAS_PAE40); } +static void set_high_memory(void) +{ + phys_addr_t highmem = memblock_end_of_DRAM(); + + /* + * Some architectures (e.g. ARM) set high_memory very early and + * use it in arch setup code. + * If an architecture already set high_memory don't overwrite it + */ + if (high_memory) + return; + +#ifdef CONFIG_HIGHMEM + if (arch_has_descending_max_zone_pfns() || + highmem > PFN_PHYS(arch_zone_lowest_possible_pfn[ZONE_HIGHMEM])) + highmem = PFN_PHYS(arch_zone_lowest_possible_pfn[ZONE_HIGHMEM]); +#endif + + high_memory = phys_to_virt(highmem - 1) + 1; +} + /** * free_area_init - Initialise all pg_data_t and zone data * @max_zone_pfn: an array of max PFNs for each zone @@ -1875,6 +1903,8 @@ void __init free_area_init(unsigned long *max_zone_pfn) /* disable hash distribution for systems with a single node */ fixup_hashdist(); + + set_high_memory(); } /** diff --git a/mm/nommu.c b/mm/nommu.c index f0209dd26dfa..b9783638fbd4 100644 --- a/mm/nommu.c +++ b/mm/nommu.c @@ -42,8 +42,6 @@ #include <asm/mmu_context.h> #include "internal.h" -void *high_memory; -EXPORT_SYMBOL(high_memory); unsigned long highest_memmap_pfn; int sysctl_nr_trim_pages = CONFIG_NOMMU_INITIAL_TRIM_EXCESS; int heap_stack_gap = 0;