From patchwork Tue Jul 30 07:27:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 1966501 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=Vja3T545; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WYBQc3Frnz1ybV for ; Tue, 30 Jul 2024 20:33:28 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=Vja3T545; dkim-atps=neutral Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4WYBQc2Bs9z3cds for ; Tue, 30 Jul 2024 20:33:28 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=Vja3T545; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=kernel.org (client-ip=139.178.84.217; helo=dfw.source.kernel.org; envelope-from=alexs@kernel.org; receiver=lists.ozlabs.org) Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4WY6BC3g7cz2yN3 for ; Tue, 30 Jul 2024 17:22:27 +1000 (AEST) Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 3321061DAC; Tue, 30 Jul 2024 07:22:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B5824C4AF09; Tue, 30 Jul 2024 07:22:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722324144; bh=HFILjDyuOkFueGdpeZ+expeCNrW1pem0IxRTIfXPmJE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Vja3T5454h4Q+VtrSRmXah9kscyuQ0sdUaWF4cc6GQNdR4jZLqcvzt/NApj4QHVhI pqrlzT+GWHsnAid/s7t2kxwJYFUtrUPlxfokjC/Txb/CQVYpaKg0WAWtESCPzRq71x NXeojkPNmEJrPbsYGdO5TaC1GuWCiXQv76PuZ6FU1/c+ZAWYWlluAS+r9mBDcGbOt5 1D/+y4VcHKQVl6RBo7gAGoNGP9Euq87+ecG3V1+RQ5tu2ME9dlPBD4ssUxQBUeIFyI uYUT3ZAY3PQJ/mREouQl40/x4dEbUyqiFsmeMNBhlEBFnnYo1M91KLOZ77KIeR/1sW TD6wdpb7tGFTQ== From: alexs@kernel.org To: Will Deacon , "Aneesh Kumar K . V" , Nick Piggin , Peter Zijlstra , Russell King , Catalin Marinas , Brian Cain , WANG Xuerui , Geert Uytterhoeven , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Michael Ellerman , Naveen N Rao , Paul Walmsley , Albert Ou , Thomas Gleixner , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Bibo Mao , Baolin Wang , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, loongarch@lists.linux.dev, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Qi Zheng , Vishal Moola , "Aneesh Kumar K . V" , Kemeng Shi , Lance Yang , Peter Xu , Barry Song , linux-s390@vger.kernel.org Subject: [RFC PATCH 11/18] mm/pgtable: introduce ptdesc_pfn and use ptdesc in free_pte_range() Date: Tue, 30 Jul 2024 15:27:12 +0800 Message-ID: <20240730072719.3715016-1-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240730064712.3714387-1-alexs@kernel.org> References: <20240730064712.3714387-1-alexs@kernel.org> MIME-Version: 1.0 X-Mailman-Approved-At: Tue, 30 Jul 2024 20:25:54 +1000 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ryan Roberts , Alexandre Ghiti , Alex Shi , David Hildenbrand , Anup Patel , Hugh Dickins , Christophe Leroy , Samuel Holland , "Naveen N . Rao" , Palmer Dabbelt , Matthew Wilcox , Jisheng Zhang , Breno Leitao , Guo Ren , Alexander Gordeev , Gerald Schaefer , Josh Poimboeuf , Mike Rapoport , Oscar Salvador Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" From: Alex Shi Replace pgtable_t by ptdesc in free_pte_range and it's callee pte_free_tlb series functions. And save some converters now. We have to use type casting for pmd_pgtable() instead of page_ptdesc() helper since different arch has different type of pgtable_t. btw, we can not simplify pmd_ptdesc() via replace pmd_pgtable_page by pmd_page, since some arch may have no pmd_page yet. Signed-off-by: Alex Shi Cc: Anup Patel Cc: Samuel Holland Cc: Jisheng Zhang Cc: Alexandre Ghiti Cc: Oscar Salvador Cc: Palmer Dabbelt Cc: Guo Ren Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-openrisc@vger.kernel.org Cc: linux-m68k@lists.linux-m68k.org Cc: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-mm@kvack.org Cc: linux-arch@vger.kernel.org Cc: Andy Lutomirski Cc: H. Peter Anvin Cc: x86@kernel.org Cc: Dave Hansen Cc: Borislav Petkov Cc: Thomas Gleixner Cc: Naveen N. Rao Cc: Christophe Leroy Cc: Michael Ellerman Cc: Stafford Horne Cc: Stefan Kristiansson Cc: Jonas Bonn Cc: Geert Uytterhoeven Cc: Catalin Marinas Cc: Russell King Cc: Peter Zijlstra Cc: Nick Piggin Cc: Aneesh Kumar K.V Cc: Will Deacon Cc: Breno Leitao Cc: Josh Poimboeuf Cc: Vishal Moola Cc: Mike Rapoport --- arch/arm/include/asm/tlb.h | 4 +--- arch/arm64/include/asm/tlb.h | 4 +--- arch/csky/include/asm/pgalloc.h | 4 ++-- arch/hexagon/include/asm/pgalloc.h | 4 ++-- arch/loongarch/include/asm/pgalloc.h | 4 ++-- arch/m68k/include/asm/motorola_pgalloc.h | 4 ++-- arch/openrisc/include/asm/pgalloc.h | 4 ++-- arch/powerpc/include/asm/book3s/32/pgalloc.h | 2 +- arch/powerpc/include/asm/book3s/64/pgalloc.h | 2 +- arch/riscv/include/asm/pgalloc.h | 8 +++----- arch/x86/include/asm/pgalloc.h | 4 ++-- arch/x86/mm/pgtable.c | 6 +++--- include/linux/mm.h | 14 ++++++++++++++ mm/memory.c | 3 ++- 14 files changed, 38 insertions(+), 29 deletions(-) diff --git a/arch/arm/include/asm/tlb.h b/arch/arm/include/asm/tlb.h index f40d06ad5d2a..ed6aa4255518 100644 --- a/arch/arm/include/asm/tlb.h +++ b/arch/arm/include/asm/tlb.h @@ -37,10 +37,8 @@ static inline void __tlb_remove_table(void *_table) #include static inline void -__pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte, unsigned long addr) +__pte_free_tlb(struct mmu_gather *tlb, struct ptdesc *ptdesc, unsigned long addr) { - struct ptdesc *ptdesc = page_ptdesc(pte); - pagetable_pte_dtor(ptdesc); #ifndef CONFIG_ARM_LPAE diff --git a/arch/arm64/include/asm/tlb.h b/arch/arm64/include/asm/tlb.h index a947c6e784ed..cee7234af6e7 100644 --- a/arch/arm64/include/asm/tlb.h +++ b/arch/arm64/include/asm/tlb.h @@ -77,11 +77,9 @@ static inline void tlb_flush(struct mmu_gather *tlb) last_level, tlb_level); } -static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte, +static inline void __pte_free_tlb(struct mmu_gather *tlb, struct ptdesc *ptdesc, unsigned long addr) { - struct ptdesc *ptdesc = page_ptdesc(pte); - pagetable_pte_dtor(ptdesc); tlb_remove_ptdesc(tlb, ptdesc); } diff --git a/arch/csky/include/asm/pgalloc.h b/arch/csky/include/asm/pgalloc.h index 9c84c9012e53..b24b4611436e 100644 --- a/arch/csky/include/asm/pgalloc.h +++ b/arch/csky/include/asm/pgalloc.h @@ -63,8 +63,8 @@ static inline pgd_t *pgd_alloc(struct mm_struct *mm) #define __pte_free_tlb(tlb, pte, address) \ do { \ - pagetable_pte_dtor(page_ptdesc(pte)); \ - tlb_remove_page_ptdesc(tlb, page_ptdesc(pte)); \ + pagetable_pte_dtor(pte); \ + tlb_remove_page_ptdesc(tlb, pte); \ } while (0) extern void pagetable_init(void); diff --git a/arch/hexagon/include/asm/pgalloc.h b/arch/hexagon/include/asm/pgalloc.h index 55988625e6fb..a3e082e54b74 100644 --- a/arch/hexagon/include/asm/pgalloc.h +++ b/arch/hexagon/include/asm/pgalloc.h @@ -89,8 +89,8 @@ static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, #define __pte_free_tlb(tlb, pte, addr) \ do { \ - pagetable_pte_dtor((page_ptdesc(pte))); \ - tlb_remove_page_ptdesc((tlb), (page_ptdesc(pte))); \ + pagetable_pte_dtor((pte)); \ + tlb_remove_page_ptdesc((tlb), (pte)); \ } while (0) #endif diff --git a/arch/loongarch/include/asm/pgalloc.h b/arch/loongarch/include/asm/pgalloc.h index 4e2d6b7ca2ee..c96d7160babc 100644 --- a/arch/loongarch/include/asm/pgalloc.h +++ b/arch/loongarch/include/asm/pgalloc.h @@ -46,8 +46,8 @@ extern pgd_t *pgd_alloc(struct mm_struct *mm); #define __pte_free_tlb(tlb, pte, address) \ do { \ - pagetable_pte_dtor(page_ptdesc(pte)); \ - tlb_remove_page_ptdesc((tlb), page_ptdesc(pte)); \ + pagetable_pte_dtor(pte); \ + tlb_remove_page_ptdesc((tlb), pte); \ } while (0) #ifndef __PAGETABLE_PMD_FOLDED diff --git a/arch/m68k/include/asm/motorola_pgalloc.h b/arch/m68k/include/asm/motorola_pgalloc.h index f6bb375971dc..f9ee5ec4574d 100644 --- a/arch/m68k/include/asm/motorola_pgalloc.h +++ b/arch/m68k/include/asm/motorola_pgalloc.h @@ -44,10 +44,10 @@ static inline void pte_free(struct mm_struct *mm, struct ptdesc *ptdesc) free_pointer_table(ptdesc_page(ptdesc), TABLE_PTE); } -static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pgtable, +static inline void __pte_free_tlb(struct mmu_gather *tlb, struct ptdesc *ptdesc, unsigned long address) { - free_pointer_table(pgtable, TABLE_PTE); + free_pointer_table(ptdesc_page(ptdesc), TABLE_PTE); } diff --git a/arch/openrisc/include/asm/pgalloc.h b/arch/openrisc/include/asm/pgalloc.h index c6a73772a546..2251d940c3d8 100644 --- a/arch/openrisc/include/asm/pgalloc.h +++ b/arch/openrisc/include/asm/pgalloc.h @@ -68,8 +68,8 @@ extern pte_t *pte_alloc_one_kernel(struct mm_struct *mm); #define __pte_free_tlb(tlb, pte, addr) \ do { \ - pagetable_pte_dtor(page_ptdesc(pte)); \ - tlb_remove_page_ptdesc((tlb), (page_ptdesc(pte))); \ + pagetable_pte_dtor(pte); \ + tlb_remove_page_ptdesc((tlb), (pte)); \ } while (0) #endif diff --git a/arch/powerpc/include/asm/book3s/32/pgalloc.h b/arch/powerpc/include/asm/book3s/32/pgalloc.h index dd4eb3063175..a435c84d1f9a 100644 --- a/arch/powerpc/include/asm/book3s/32/pgalloc.h +++ b/arch/powerpc/include/asm/book3s/32/pgalloc.h @@ -64,7 +64,7 @@ static inline void __tlb_remove_table(void *_table) pgtable_free(table, shift); } -static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t table, +static inline void __pte_free_tlb(struct mmu_gather *tlb, struct ptdesc *table, unsigned long address) { pgtable_free_tlb(tlb, table, 0); diff --git a/arch/powerpc/include/asm/book3s/64/pgalloc.h b/arch/powerpc/include/asm/book3s/64/pgalloc.h index eb7d2ca59f62..675eca34fe40 100644 --- a/arch/powerpc/include/asm/book3s/64/pgalloc.h +++ b/arch/powerpc/include/asm/book3s/64/pgalloc.h @@ -167,7 +167,7 @@ static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmd, *pmd = __pmd(__pgtable_ptr_val(pte_page) | PMD_VAL_BITS); } -static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t table, +static inline void __pte_free_tlb(struct mmu_gather *tlb, struct ptdesc *table, unsigned long address) { pgtable_free_tlb(tlb, table, PTE_INDEX); diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgalloc.h index f52264304f77..63596efcd528 100644 --- a/arch/riscv/include/asm/pgalloc.h +++ b/arch/riscv/include/asm/pgalloc.h @@ -183,13 +183,11 @@ static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd, #endif /* __PAGETABLE_PMD_FOLDED */ -static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte, +static inline void __pte_free_tlb(struct mmu_gather *tlb, struct ptdesc *pte, unsigned long addr) { - struct ptdesc *ptdesc = page_ptdesc(pte); - - pagetable_pte_dtor(ptdesc); - riscv_tlb_remove_ptdesc(tlb, ptdesc); + pagetable_pte_dtor(pte); + riscv_tlb_remove_ptdesc(tlb, pte); } #endif /* CONFIG_MMU */ diff --git a/arch/x86/include/asm/pgalloc.h b/arch/x86/include/asm/pgalloc.h index 497c757b5b98..06a9a5867a86 100644 --- a/arch/x86/include/asm/pgalloc.h +++ b/arch/x86/include/asm/pgalloc.h @@ -53,9 +53,9 @@ extern void pgd_free(struct mm_struct *mm, pgd_t *pgd); extern struct ptdesc *pte_alloc_one(struct mm_struct *); -extern void ___pte_free_tlb(struct mmu_gather *tlb, struct page *pte); +extern void ___pte_free_tlb(struct mmu_gather *tlb, struct ptdesc *pte); -static inline void __pte_free_tlb(struct mmu_gather *tlb, struct page *pte, +static inline void __pte_free_tlb(struct mmu_gather *tlb, struct ptdesc *pte, unsigned long address) { ___pte_free_tlb(tlb, pte); diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index c27d15cd01b9..3cf9c0d25dbd 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -50,10 +50,10 @@ static int __init setup_userpte(char *arg) } early_param("userpte", setup_userpte); -void ___pte_free_tlb(struct mmu_gather *tlb, struct page *pte) +void ___pte_free_tlb(struct mmu_gather *tlb, struct ptdesc *pte) { - pagetable_pte_dtor(page_ptdesc(pte)); - paravirt_release_pte(page_to_pfn(pte)); + pagetable_pte_dtor(pte); + paravirt_release_pte(ptdesc_pfn(pte)); paravirt_tlb_remove_table(tlb, pte); } diff --git a/include/linux/mm.h b/include/linux/mm.h index 381750f41767..7424f964dff3 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2859,6 +2859,20 @@ static inline bool pagetable_is_reserved(struct ptdesc *pt) return folio_test_reserved(ptdesc_folio(pt)); } +/** + * ptdesc_pfn - Return the Page Frame Number of a ptdesc. + * @ptdesc: The ptdesc. + * + * A ptdesc may contain multiple pages. The pages have consecutive + * Page Frame Numbers. + * + * Return: The Page Frame Number of the first page in the ptdesc. + */ +static inline unsigned long ptdesc_pfn(struct ptdesc *ptdesc) +{ + return page_to_pfn(ptdesc_page(ptdesc)); +} + /** * pagetable_alloc - Allocate pagetables * @gfp: GFP flags diff --git a/mm/memory.c b/mm/memory.c index 3014168e7296..27c2f63b7487 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -189,7 +189,8 @@ void mm_trace_rss_stat(struct mm_struct *mm, int member) static void free_pte_range(struct mmu_gather *tlb, pmd_t *pmd, unsigned long addr) { - pgtable_t token = pmd_pgtable(*pmd); + struct ptdesc *token = (struct ptdesc *)pmd_pgtable(*pmd); + pmd_clear(pmd); pte_free_tlb(tlb, token, addr); mm_dec_nr_ptes(tlb->mm);