| Message ID | 20240730064712.3714387-1-alexs@kernel.org (mailing list archive) |
|---|---|
| Headers | show |
| Series | use struct ptdesc to replace pgtable_t | expand |
BTW, the patchset based on the latest git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm mm-unstable branch. On 7/30/24 2:46 PM, alexs@kernel.org wrote: > From: Alex Shi <alexs@kernel.org> > > We have struct ptdesc for page table descriptor a year ago, but it > has no much usages in kernel, while pgtable_t is used widely. > > The pgtable_t is typedefed as 'pte_t *' in sparc, s390, powerpc and m68k > except SUN3, others archs are all same as 'struct page *'. > > These blocks the conception and code update for page table descriptor to > struct ptdesc. > > So, the simple idea to push the ptdesc conception forward is to update > all pgtable_t by ptdesc or pte_t pointer. But this needs widely > knowledges for most all of different archs. Common code change is easy > for include/ and mm/ directory, but it's hard in all archs. > > Thanks for intel LKP framework, I fixed most all of build issues except > a bug on powerpc which reports a "struct ptdesc *" incompatible with > struct ptdesc *' pointer issue... > > Another trouble is pmd_pgtable() conversion in the last patch. > Maybe some of arch need define theirself own pmd_ptdesc()? > > This patchset is immature, even except above 2 issues, I just tested > virutal machine booting and kselftest mm on x86 and arm64. > > Anyway any input are appreciated! > > Thanks > Alex > > Alex Shi (18): > mm/pgtable: use ptdesc in pte_free_now/pte_free_defer > mm/pgtable: convert ptdesc.pmd_huge_pte to ptdesc pointer > fs/dax: use ptdesc in dax_pmd_load_hole > mm/thp: use ptdesc pointer in __do_huge_pmd_anonymous_page > mm/thp: use ptdesc in do_huge_pmd_anonymous_page > mm/thp: convert insert_pfn_pmd and its caller to use ptdesc > mm/thp: use ptdesc in copy_huge_pmd > mm/memory: use ptdesc in __pte_alloc > mm/pgtable: fully use ptdesc in pte_alloc_one series functions > mm/pgtable: pass ptdesc to pte_free() > mm/pgtable: introduce ptdesc_pfn and use ptdesc in free_pte_range() > mm/thp: pass ptdesc to set_huge_zero_folio function > mm/pgtable: return ptdesc pointer in pgtable_trans_huge_withdraw > mm/pgtable: use ptdesc in pgtable_trans_huge_deposit > mm/pgtable: pass ptdesc to pmd_populate > mm/pgtable: pass ptdesc to pmd_install > mm: convert vmf.prealloc_pte to struct ptdesc pointer > mm/pgtable: pass ptdesc in pte_free_defer > > arch/alpha/include/asm/pgalloc.h | 4 +- > arch/arc/include/asm/pgalloc.h | 4 +- > arch/arm/include/asm/pgalloc.h | 13 +-- > arch/arm/include/asm/tlb.h | 4 +- > arch/arm/mm/pgd.c | 2 +- > arch/arm64/include/asm/pgalloc.h | 4 +- > arch/arm64/include/asm/tlb.h | 4 +- > arch/csky/include/asm/pgalloc.h | 4 +- > arch/hexagon/include/asm/pgalloc.h | 8 +- > arch/loongarch/include/asm/pgalloc.h | 8 +- > arch/m68k/include/asm/motorola_pgalloc.h | 12 +- > arch/m68k/include/asm/sun3_pgalloc.h | 4 +- > arch/microblaze/include/asm/pgalloc.h | 2 +- > arch/mips/include/asm/pgalloc.h | 4 +- > arch/nios2/include/asm/pgalloc.h | 4 +- > arch/openrisc/include/asm/pgalloc.h | 8 +- > arch/parisc/include/asm/pgalloc.h | 2 +- > arch/powerpc/include/asm/book3s/32/pgalloc.h | 4 +- > arch/powerpc/include/asm/book3s/64/hash-4k.h | 4 +- > arch/powerpc/include/asm/book3s/64/hash-64k.h | 4 +- > arch/powerpc/include/asm/book3s/64/pgalloc.h | 4 +- > arch/powerpc/include/asm/book3s/64/pgtable.h | 8 +- > arch/powerpc/include/asm/book3s/64/radix.h | 4 +- > arch/powerpc/include/asm/pgalloc.h | 8 +- > arch/powerpc/mm/book3s64/hash_pgtable.c | 10 +- > arch/powerpc/mm/book3s64/radix_pgtable.c | 10 +- > arch/riscv/include/asm/pgalloc.h | 8 +- > arch/s390/include/asm/pgalloc.h | 4 +- > arch/s390/include/asm/pgtable.h | 4 +- > arch/s390/mm/pgalloc.c | 2 +- > arch/s390/mm/pgtable.c | 14 +-- > arch/sh/include/asm/pgalloc.h | 4 +- > arch/sparc/include/asm/pgalloc_32.h | 6 +- > arch/sparc/include/asm/pgalloc_64.h | 2 +- > arch/sparc/include/asm/pgtable_64.h | 4 +- > arch/sparc/mm/init_64.c | 2 +- > arch/sparc/mm/srmmu.c | 6 +- > arch/sparc/mm/tlb.c | 14 +-- > arch/x86/include/asm/pgalloc.h | 10 +- > arch/x86/mm/pgtable.c | 8 +- > arch/xtensa/include/asm/pgalloc.h | 12 +- > fs/dax.c | 14 +-- > include/asm-generic/pgalloc.h | 10 +- > include/linux/mm.h | 16 ++- > include/linux/mm_types.h | 4 +- > include/linux/pgtable.h | 6 +- > mm/debug_vm_pgtable.c | 6 +- > mm/huge_memory.c | 103 +++++++++--------- > mm/internal.h | 2 +- > mm/khugepaged.c | 14 +-- > mm/memory.c | 15 +-- > mm/mremap.c | 2 +- > mm/pgtable-generic.c | 37 +++---- > 53 files changed, 240 insertions(+), 236 deletions(-) >
Hi, Le 30/07/2024 à 08:46, alexs@kernel.org a écrit : > From: Alex Shi <alexs@kernel.org> > > We have struct ptdesc for page table descriptor a year ago, but it > has no much usages in kernel, while pgtable_t is used widely. > > The pgtable_t is typedefed as 'pte_t *' in sparc, s390, powerpc and m68k > except SUN3, others archs are all same as 'struct page *'. > > These blocks the conception and code update for page table descriptor to > struct ptdesc. > > So, the simple idea to push the ptdesc conception forward is to update > all pgtable_t by ptdesc or pte_t pointer. But this needs widely > knowledges for most all of different archs. Common code change is easy > for include/ and mm/ directory, but it's hard in all archs. > > Thanks for intel LKP framework, I fixed most all of build issues except > a bug on powerpc which reports a "struct ptdesc *" incompatible with > struct ptdesc *' pointer issue... Can you tell more about that problem on powerpc ? Which defconfig for instance ? > > Another trouble is pmd_pgtable() conversion in the last patch. > Maybe some of arch need define theirself own pmd_ptdesc()? > > This patchset is immature, even except above 2 issues, I just tested > virutal machine booting and kselftest mm on x86 and arm64. > > Anyway any input are appreciated! Can you tell on which tree you based this series ? Last patch doesn't apply on 6.11-rc1: Applying: mm/pgtable: pass ptdesc in pte_free_defer error: sha1 information is lacking or useless (mm/khugepaged.c). error: could not build fake ancestor Patch failed at 0019 mm/pgtable: pass ptdesc in pte_free_defer Following hunk is the problem: diff a/mm/khugepaged.c b/mm/khugepaged.c (rejected hunks) @@ -1777,7 +1777,7 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) if (retracted) { mm_dec_nr_ptes(mm); page_table_check_pte_clear_range(mm, addr, pgt_pmd); - pte_free_defer(mm, pmd_pgtable(pgt_pmd)); + pte_free_defer(mm, pmd_ptdesc(&pgt_pmd)); } } i_mmap_unlock_read(mapping); Christophe > > Thanks > Alex > > Alex Shi (18): > mm/pgtable: use ptdesc in pte_free_now/pte_free_defer > mm/pgtable: convert ptdesc.pmd_huge_pte to ptdesc pointer > fs/dax: use ptdesc in dax_pmd_load_hole > mm/thp: use ptdesc pointer in __do_huge_pmd_anonymous_page > mm/thp: use ptdesc in do_huge_pmd_anonymous_page > mm/thp: convert insert_pfn_pmd and its caller to use ptdesc > mm/thp: use ptdesc in copy_huge_pmd > mm/memory: use ptdesc in __pte_alloc > mm/pgtable: fully use ptdesc in pte_alloc_one series functions > mm/pgtable: pass ptdesc to pte_free() > mm/pgtable: introduce ptdesc_pfn and use ptdesc in free_pte_range() > mm/thp: pass ptdesc to set_huge_zero_folio function > mm/pgtable: return ptdesc pointer in pgtable_trans_huge_withdraw > mm/pgtable: use ptdesc in pgtable_trans_huge_deposit > mm/pgtable: pass ptdesc to pmd_populate > mm/pgtable: pass ptdesc to pmd_install > mm: convert vmf.prealloc_pte to struct ptdesc pointer > mm/pgtable: pass ptdesc in pte_free_defer > > arch/alpha/include/asm/pgalloc.h | 4 +- > arch/arc/include/asm/pgalloc.h | 4 +- > arch/arm/include/asm/pgalloc.h | 13 +-- > arch/arm/include/asm/tlb.h | 4 +- > arch/arm/mm/pgd.c | 2 +- > arch/arm64/include/asm/pgalloc.h | 4 +- > arch/arm64/include/asm/tlb.h | 4 +- > arch/csky/include/asm/pgalloc.h | 4 +- > arch/hexagon/include/asm/pgalloc.h | 8 +- > arch/loongarch/include/asm/pgalloc.h | 8 +- > arch/m68k/include/asm/motorola_pgalloc.h | 12 +- > arch/m68k/include/asm/sun3_pgalloc.h | 4 +- > arch/microblaze/include/asm/pgalloc.h | 2 +- > arch/mips/include/asm/pgalloc.h | 4 +- > arch/nios2/include/asm/pgalloc.h | 4 +- > arch/openrisc/include/asm/pgalloc.h | 8 +- > arch/parisc/include/asm/pgalloc.h | 2 +- > arch/powerpc/include/asm/book3s/32/pgalloc.h | 4 +- > arch/powerpc/include/asm/book3s/64/hash-4k.h | 4 +- > arch/powerpc/include/asm/book3s/64/hash-64k.h | 4 +- > arch/powerpc/include/asm/book3s/64/pgalloc.h | 4 +- > arch/powerpc/include/asm/book3s/64/pgtable.h | 8 +- > arch/powerpc/include/asm/book3s/64/radix.h | 4 +- > arch/powerpc/include/asm/pgalloc.h | 8 +- > arch/powerpc/mm/book3s64/hash_pgtable.c | 10 +- > arch/powerpc/mm/book3s64/radix_pgtable.c | 10 +- > arch/riscv/include/asm/pgalloc.h | 8 +- > arch/s390/include/asm/pgalloc.h | 4 +- > arch/s390/include/asm/pgtable.h | 4 +- > arch/s390/mm/pgalloc.c | 2 +- > arch/s390/mm/pgtable.c | 14 +-- > arch/sh/include/asm/pgalloc.h | 4 +- > arch/sparc/include/asm/pgalloc_32.h | 6 +- > arch/sparc/include/asm/pgalloc_64.h | 2 +- > arch/sparc/include/asm/pgtable_64.h | 4 +- > arch/sparc/mm/init_64.c | 2 +- > arch/sparc/mm/srmmu.c | 6 +- > arch/sparc/mm/tlb.c | 14 +-- > arch/x86/include/asm/pgalloc.h | 10 +- > arch/x86/mm/pgtable.c | 8 +- > arch/xtensa/include/asm/pgalloc.h | 12 +- > fs/dax.c | 14 +-- > include/asm-generic/pgalloc.h | 10 +- > include/linux/mm.h | 16 ++- > include/linux/mm_types.h | 4 +- > include/linux/pgtable.h | 6 +- > mm/debug_vm_pgtable.c | 6 +- > mm/huge_memory.c | 103 +++++++++--------- > mm/internal.h | 2 +- > mm/khugepaged.c | 14 +-- > mm/memory.c | 15 +-- > mm/mremap.c | 2 +- > mm/pgtable-generic.c | 37 +++---- > 53 files changed, 240 insertions(+), 236 deletions(-) >
On Tue, Jul 30, 2024 at 02:46:54PM +0800, alexs@kernel.org wrote: > We have struct ptdesc for page table descriptor a year ago, but it > has no much usages in kernel, while pgtable_t is used widely. Hum, I thought I responded to this to point out the problem, but I don't see the response anywhere, so I'll try again. > The pgtable_t is typedefed as 'pte_t *' in sparc, s390, powerpc and m68k > except SUN3, others archs are all same as 'struct page *'. And there's a very good reason for that. On s390 and powerpc (I cannot speak to the sparc/m68k), each page table is (potentially) smaller than PAGE_SIZE. So we cannot do what your patch purports to do, as we would not know whether we're referring to the first or subsequent page tables contained within a page. Maybe at some point in the distant future we'll be able to allocate a ptdesc per page table instead of per page allocated for use by page tables. But we cannot do that yet.
On 8/15/24 1:13 AM, Matthew Wilcox wrote: > On Tue, Jul 30, 2024 at 02:46:54PM +0800, alexs@kernel.org wrote: >> We have struct ptdesc for page table descriptor a year ago, but it >> has no much usages in kernel, while pgtable_t is used widely. > > Hum, I thought I responded to this to point out the problem, but > I don't see the response anywhere, so I'll try again. > >> The pgtable_t is typedefed as 'pte_t *' in sparc, s390, powerpc and m68k >> except SUN3, others archs are all same as 'struct page *'. > > And there's a very good reason for that. On s390 and powerpc (I cannot > speak to the sparc/m68k), each page table is (potentially) smaller > than PAGE_SIZE. So we cannot do what your patch purports to do, as > we would not know whether we're referring to the first or subsequent > page tables contained within a page. >> Maybe at some point in the distant future we'll be able to allocate > a ptdesc per page table instead of per page allocated for use by page > tables. But we cannot do that yet. Got it. Thanks a lot for the info!
On 8/15/24 12:50 AM, LEROY Christophe wrote: > Hi, > > Le 30/07/2024 à 08:46, alexs@kernel.org a écrit : >> From: Alex Shi <alexs@kernel.org> >> >> We have struct ptdesc for page table descriptor a year ago, but it >> has no much usages in kernel, while pgtable_t is used widely. >> >> The pgtable_t is typedefed as 'pte_t *' in sparc, s390, powerpc and m68k >> except SUN3, others archs are all same as 'struct page *'. >> >> These blocks the conception and code update for page table descriptor to >> struct ptdesc. >> >> So, the simple idea to push the ptdesc conception forward is to update >> all pgtable_t by ptdesc or pte_t pointer. But this needs widely >> knowledges for most all of different archs. Common code change is easy >> for include/ and mm/ directory, but it's hard in all archs. >> >> Thanks for intel LKP framework, I fixed most all of build issues except >> a bug on powerpc which reports a "struct ptdesc *" incompatible with >> struct ptdesc *' pointer issue... > > Can you tell more about that problem on powerpc ? Which defconfig for > instance ? > Hi Leroy, Thanks a lot for response. The detailed error and code tree is from LKP: On 8/3/24 11:26 AM, kernel test robot wrote: > tree: https://github.com/alexshi/linux.git ptdesc > head: 3735e634f8feb67c941d17222b53906ace2fd2b8 > commit: 8ac2ea5f299444d065b292043aed0a1e94996159 [14/17] mm/pgtable: use ptdesc in pgtable_trans_huge_deposit > config: powerpc-allyesconfig (https://download.01.org/0day-ci/archive/20240803/202408031130.UnwgnuLS-lkp@intel.com/config) > compiler: clang version 20.0.0git (https://github.com/llvm/llvm-project 423aec6573df4424f90555468128e17073ddc69e) > reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240803/202408031130.UnwgnuLS-lkp@intel.com/reproduce) > In file included from arch/powerpc/include/asm/book3s/64/mmu-hash.h:20: >>> arch/powerpc/include/asm/book3s/64/pgtable.h:1372:52: error: incompatible pointer types passing 'struct ptdesc *' to parameter of type 'struct ptdesc *' [-Werror,-Wincompatible-pointer-types] >> >> Another trouble is pmd_pgtable() conversion in the last patch. >> Maybe some of arch need define theirself own pmd_ptdesc()? >> >> This patchset is immature, even except above 2 issues, I just tested >> virutal machine booting and kselftest mm on x86 and arm64. >> >> Anyway any input are appreciated! > > Can you tell on which tree you based this series ? Last patch doesn't > apply on 6.11-rc1: The code based on akpm/mm-unstable tree, not upstream, you can check the code on: https://github.com/alexshi/linux.git ptdesc Thanks a lot! Alex
From: Alex Shi <alexs@kernel.org> We have struct ptdesc for page table descriptor a year ago, but it has no much usages in kernel, while pgtable_t is used widely. The pgtable_t is typedefed as 'pte_t *' in sparc, s390, powerpc and m68k except SUN3, others archs are all same as 'struct page *'. These blocks the conception and code update for page table descriptor to struct ptdesc. So, the simple idea to push the ptdesc conception forward is to update all pgtable_t by ptdesc or pte_t pointer. But this needs widely knowledges for most all of different archs. Common code change is easy for include/ and mm/ directory, but it's hard in all archs. Thanks for intel LKP framework, I fixed most all of build issues except a bug on powerpc which reports a "struct ptdesc *" incompatible with struct ptdesc *' pointer issue... Another trouble is pmd_pgtable() conversion in the last patch. Maybe some of arch need define theirself own pmd_ptdesc()? This patchset is immature, even except above 2 issues, I just tested virutal machine booting and kselftest mm on x86 and arm64. Anyway any input are appreciated! Thanks Alex Alex Shi (18): mm/pgtable: use ptdesc in pte_free_now/pte_free_defer mm/pgtable: convert ptdesc.pmd_huge_pte to ptdesc pointer fs/dax: use ptdesc in dax_pmd_load_hole mm/thp: use ptdesc pointer in __do_huge_pmd_anonymous_page mm/thp: use ptdesc in do_huge_pmd_anonymous_page mm/thp: convert insert_pfn_pmd and its caller to use ptdesc mm/thp: use ptdesc in copy_huge_pmd mm/memory: use ptdesc in __pte_alloc mm/pgtable: fully use ptdesc in pte_alloc_one series functions mm/pgtable: pass ptdesc to pte_free() mm/pgtable: introduce ptdesc_pfn and use ptdesc in free_pte_range() mm/thp: pass ptdesc to set_huge_zero_folio function mm/pgtable: return ptdesc pointer in pgtable_trans_huge_withdraw mm/pgtable: use ptdesc in pgtable_trans_huge_deposit mm/pgtable: pass ptdesc to pmd_populate mm/pgtable: pass ptdesc to pmd_install mm: convert vmf.prealloc_pte to struct ptdesc pointer mm/pgtable: pass ptdesc in pte_free_defer arch/alpha/include/asm/pgalloc.h | 4 +- arch/arc/include/asm/pgalloc.h | 4 +- arch/arm/include/asm/pgalloc.h | 13 +-- arch/arm/include/asm/tlb.h | 4 +- arch/arm/mm/pgd.c | 2 +- arch/arm64/include/asm/pgalloc.h | 4 +- arch/arm64/include/asm/tlb.h | 4 +- arch/csky/include/asm/pgalloc.h | 4 +- arch/hexagon/include/asm/pgalloc.h | 8 +- arch/loongarch/include/asm/pgalloc.h | 8 +- arch/m68k/include/asm/motorola_pgalloc.h | 12 +- arch/m68k/include/asm/sun3_pgalloc.h | 4 +- arch/microblaze/include/asm/pgalloc.h | 2 +- arch/mips/include/asm/pgalloc.h | 4 +- arch/nios2/include/asm/pgalloc.h | 4 +- arch/openrisc/include/asm/pgalloc.h | 8 +- arch/parisc/include/asm/pgalloc.h | 2 +- arch/powerpc/include/asm/book3s/32/pgalloc.h | 4 +- arch/powerpc/include/asm/book3s/64/hash-4k.h | 4 +- arch/powerpc/include/asm/book3s/64/hash-64k.h | 4 +- arch/powerpc/include/asm/book3s/64/pgalloc.h | 4 +- arch/powerpc/include/asm/book3s/64/pgtable.h | 8 +- arch/powerpc/include/asm/book3s/64/radix.h | 4 +- arch/powerpc/include/asm/pgalloc.h | 8 +- arch/powerpc/mm/book3s64/hash_pgtable.c | 10 +- arch/powerpc/mm/book3s64/radix_pgtable.c | 10 +- arch/riscv/include/asm/pgalloc.h | 8 +- arch/s390/include/asm/pgalloc.h | 4 +- arch/s390/include/asm/pgtable.h | 4 +- arch/s390/mm/pgalloc.c | 2 +- arch/s390/mm/pgtable.c | 14 +-- arch/sh/include/asm/pgalloc.h | 4 +- arch/sparc/include/asm/pgalloc_32.h | 6 +- arch/sparc/include/asm/pgalloc_64.h | 2 +- arch/sparc/include/asm/pgtable_64.h | 4 +- arch/sparc/mm/init_64.c | 2 +- arch/sparc/mm/srmmu.c | 6 +- arch/sparc/mm/tlb.c | 14 +-- arch/x86/include/asm/pgalloc.h | 10 +- arch/x86/mm/pgtable.c | 8 +- arch/xtensa/include/asm/pgalloc.h | 12 +- fs/dax.c | 14 +-- include/asm-generic/pgalloc.h | 10 +- include/linux/mm.h | 16 ++- include/linux/mm_types.h | 4 +- include/linux/pgtable.h | 6 +- mm/debug_vm_pgtable.c | 6 +- mm/huge_memory.c | 103 +++++++++--------- mm/internal.h | 2 +- mm/khugepaged.c | 14 +-- mm/memory.c | 15 +-- mm/mremap.c | 2 +- mm/pgtable-generic.c | 37 +++---- 53 files changed, 240 insertions(+), 236 deletions(-)