From patchwork Tue Jul 30 06:46:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 1966330 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=uXb4byD4; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=mail.ozlabs.org; envelope-from=srs0=zku4=o6=vger.kernel.org=sparclinux+bounces-1758-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WY5KJ2YBsz1yYq for ; Tue, 30 Jul 2024 16:43:30 +1000 (AEST) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4WY5KF200lz4wcr for ; Tue, 30 Jul 2024 16:43:29 +1000 (AEST) Received: by gandalf.ozlabs.org (Postfix) id 4WY5KF1x8Mz4x6n; Tue, 30 Jul 2024 16:43:29 +1000 (AEST) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=uXb4byD4; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45d1:ec00::1; helo=ny.mirrors.kernel.org; envelope-from=sparclinux+bounces-1758-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org [IPv6:2604:1380:45d1:ec00::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4WY5KD5ntrz4wcr for ; Tue, 30 Jul 2024 16:43:28 +1000 (AEST) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id EEB011C229B9 for ; Tue, 30 Jul 2024 06:43:26 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 43AD718F2E7; Tue, 30 Jul 2024 06:43:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="uXb4byD4" X-Original-To: sparclinux@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F42022563; Tue, 30 Jul 2024 06:43:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722321794; cv=none; b=niOhqBaFvq0kIcQoCMH61TdBVvAofdeX8zZFFuYbIJx4zwaDrY9MDkUf8FNliG0V7HtpyrrHhb6Sq7GJhpZ7/12imhK88nbAIuf32H+TiHgBrjl8cVTEQcDQiHuR5X3CWovdGKDk3usBqBxr2DOILp0iuDjG7lo5Ye2ifv6h3d8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722321794; c=relaxed/simple; bh=9NZb0rrKQqKYXT4j44hct+A4v/K9XblH1QcI1NJ86wY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eI4E/ejlpGi+tb2Lu4lXHcXCjowazjh37bFkPlIojh1kbrsKbMO4ropTLsiUASbCN47dmDWsV2rWF0q+/xHQpe9RIhjaiteq9SRMJA5bR/bb9g0bP2myOoTTD6rZqanOTi7LVlSGYoMdQLRhSipAHhPX9XBA7Og06jS6rl0pn0w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=uXb4byD4; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 469FBC32782; Tue, 30 Jul 2024 06:43:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722321793; bh=9NZb0rrKQqKYXT4j44hct+A4v/K9XblH1QcI1NJ86wY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uXb4byD4QEB/pykGQSUxfRgInQG2nhMHKXm5xZNegBtn3rrGWMaLewVYiqUNRYDtJ aQ+opzP1rC4aJUxCXkL2wAnqAIW3nIQYdRxHYnksJ0PJsaNlzE/MV3jYxsY7CWC6i6 A55O5dfE29WFwTXyQHh6asj1EX4qez2hBgWChYq1nW9wZhzPJt7adzM3dujj4fTqjW GPMlZQsWyciJ/EykTgqqAmIjcZelbGlK8I8ok7QSAhrcTaitTlPHXoD4wXFz71j46R BpZv8MZ6nqA/BUfQ66+uiCN6lG0xAGpqKsFp7FzaM7qSm50/Hdt7uFpAhZjkPrnNQ4 JZOzjdQ+jlf0A== From: alexs@kernel.org To: Will Deacon , "Aneesh Kumar K . V" , Nick Piggin , Peter Zijlstra , Russell King , Catalin Marinas , Brian Cain , WANG Xuerui , Geert Uytterhoeven , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Michael Ellerman , Naveen N Rao , Paul Walmsley , Albert Ou , Thomas Gleixner , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Bibo Mao , Baolin Wang , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, loongarch@lists.linux.dev, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Qi Zheng , Vishal Moola , "Aneesh Kumar K . V" , Kemeng Shi , Lance Yang , Peter Xu , Barry Song , linux-s390@vger.kernel.org Cc: Guo Ren , Christophe Leroy , Palmer Dabbelt , Mike Rapoport , Oscar Salvador , Alexandre Ghiti , Jisheng Zhang , Samuel Holland , Anup Patel , Josh Poimboeuf , Breno Leitao , Alexander Gordeev , Gerald Schaefer , Hugh Dickins , David Hildenbrand , Ryan Roberts , Matthew Wilcox , Alex Shi , sparclinux@vger.kernel.org, Andreas Larsson , "David S . Miller" , Zi Yan Subject: [RFC PATCH 02/18] mm/pgtable: convert ptdesc.pmd_huge_pte to ptdesc pointer Date: Tue, 30 Jul 2024 14:46:56 +0800 Message-ID: <20240730064712.3714387-3-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240730064712.3714387-1-alexs@kernel.org> References: <20240730064712.3714387-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: sparclinux@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Alex Shi folio/page.pmd_huge_pte is a pointer to pagetable descriptor: pgtable_t. In most arch, it is a typedef of 'struct page *'. But we have ptdesc now, tt's better to convert it to right one: struct ptdesc pointer. Different from others, s390/sparc use typedef 'pte_t *' as pgtable_t, so they need different cost in their arch. Thanks for lkp found the build issue for s390/sparc, it fixed now. Signed-off-by: Alex Shi Cc: linux-mm@kvack.org Cc: sparclinux@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: linux-s390@vger.kernel.org Cc: Ryan Roberts Cc: Andreas Larsson Cc: David S. Miller Cc: Sven Schnelle Cc: Christian Borntraeger Cc: Vasily Gorbik Cc: Heiko Carstens Cc: Gerald Schaefer Cc: Alexander Gordeev Cc: Zi Yan Cc: Matthew Wilcox Cc: Mike Rapoport Cc: David Hildenbrand --- arch/s390/mm/pgtable.c | 6 +++--- arch/sparc/mm/tlb.c | 6 +++--- include/linux/mm_types.h | 4 ++-- mm/pgtable-generic.c | 16 ++++++++-------- 4 files changed, 16 insertions(+), 16 deletions(-) diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c index 2c944bafb030..201d350abd1e 100644 --- a/arch/s390/mm/pgtable.c +++ b/arch/s390/mm/pgtable.c @@ -574,7 +574,7 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, INIT_LIST_HEAD(lh); else list_add(lh, (struct list_head *) pmd_huge_pte(mm, pmdp)); - pmd_huge_pte(mm, pmdp) = pgtable; + pmd_huge_pte(mm, pmdp) = (struct ptdesc *)pgtable; } pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) @@ -586,12 +586,12 @@ pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) assert_spin_locked(pmd_lockptr(mm, pmdp)); /* FIFO */ - pgtable = pmd_huge_pte(mm, pmdp); + pgtable = (pte_t *)pmd_huge_pte(mm, pmdp); lh = (struct list_head *) pgtable; if (list_empty(lh)) pmd_huge_pte(mm, pmdp) = NULL; else { - pmd_huge_pte(mm, pmdp) = (pgtable_t) lh->next; + pmd_huge_pte(mm, pmdp) = (struct ptdesc *) lh->next; list_del(lh); } ptep = (pte_t *) pgtable; diff --git a/arch/sparc/mm/tlb.c b/arch/sparc/mm/tlb.c index 8648a50afe88..903825b4c997 100644 --- a/arch/sparc/mm/tlb.c +++ b/arch/sparc/mm/tlb.c @@ -278,7 +278,7 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, INIT_LIST_HEAD(lh); else list_add(lh, (struct list_head *) pmd_huge_pte(mm, pmdp)); - pmd_huge_pte(mm, pmdp) = pgtable; + pmd_huge_pte(mm, pmdp) = (struct ptdesc *)pgtable; } pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) @@ -289,12 +289,12 @@ pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) assert_spin_locked(&mm->page_table_lock); /* FIFO */ - pgtable = pmd_huge_pte(mm, pmdp); + pgtable = (pte_t *)pmd_huge_pte(mm, pmdp); lh = (struct list_head *) pgtable; if (list_empty(lh)) pmd_huge_pte(mm, pmdp) = NULL; else { - pmd_huge_pte(mm, pmdp) = (pgtable_t) lh->next; + pmd_huge_pte(mm, pmdp) = (struct ptdesc *) lh->next; list_del(lh); } pte_val(pgtable[0]) = 0; diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 485424979254..2e3eddf6edc9 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -462,7 +462,7 @@ struct ptdesc { struct list_head pt_list; struct { unsigned long _pt_pad_1; - pgtable_t pmd_huge_pte; + struct ptdesc *pmd_huge_pte; }; }; unsigned long __page_mapping; @@ -948,7 +948,7 @@ struct mm_struct { struct mmu_notifier_subscriptions *notifier_subscriptions; #endif #if defined(CONFIG_TRANSPARENT_HUGEPAGE) && !USE_SPLIT_PMD_PTLOCKS - pgtable_t pmd_huge_pte; /* protected by page_table_lock */ + struct ptdesc *pmd_huge_pte; /* protected by page_table_lock */ #endif #ifdef CONFIG_NUMA_BALANCING /* diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index 2ce714f1dd15..f34a8d115f5b 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -171,8 +171,8 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, if (!pmd_huge_pte(mm, pmdp)) INIT_LIST_HEAD(&pgtable->lru); else - list_add(&pgtable->lru, &pmd_huge_pte(mm, pmdp)->lru); - pmd_huge_pte(mm, pmdp) = pgtable; + list_add(&pgtable->lru, &pmd_huge_pte(mm, pmdp)->pt_list); + pmd_huge_pte(mm, pmdp) = page_ptdesc(pgtable); } #endif @@ -180,17 +180,17 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, /* no "address" argument so destroys page coloring of some arch */ pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) { - pgtable_t pgtable; + struct ptdesc *ptdesc; assert_spin_locked(pmd_lockptr(mm, pmdp)); /* FIFO */ - pgtable = pmd_huge_pte(mm, pmdp); - pmd_huge_pte(mm, pmdp) = list_first_entry_or_null(&pgtable->lru, - struct page, lru); + ptdesc = pmd_huge_pte(mm, pmdp); + pmd_huge_pte(mm, pmdp) = list_first_entry_or_null(&ptdesc->pt_list, + struct ptdesc, pt_list); if (pmd_huge_pte(mm, pmdp)) - list_del(&pgtable->lru); - return pgtable; + list_del(&ptdesc->pt_list); + return ptdesc_page(ptdesc); } #endif From patchwork Tue Jul 30 06:47:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 1966331 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=XU5gA41f; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=mail.ozlabs.org; envelope-from=srs0=xwoy=o6=vger.kernel.org=sparclinux+bounces-1759-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WY5Mw5YsGz1ybX for ; Tue, 30 Jul 2024 16:45:48 +1000 (AEST) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4WY5Ms0W2yz4x6r for ; Tue, 30 Jul 2024 16:45:45 +1000 (AEST) Received: by gandalf.ozlabs.org (Postfix) id 4WY5Ms0Qckz4x7H; Tue, 30 Jul 2024 16:45:45 +1000 (AEST) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=XU5gA41f; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=147.75.199.223; helo=ny.mirrors.kernel.org; envelope-from=sparclinux+bounces-1759-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org [147.75.199.223]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4WY5Mr4N2mz4x6r for ; Tue, 30 Jul 2024 16:45:44 +1000 (AEST) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 909C31C22CED for ; Tue, 30 Jul 2024 06:45:42 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8EB9D18FDCF; Tue, 30 Jul 2024 06:44:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="XU5gA41f" X-Original-To: sparclinux@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4630818E741; Tue, 30 Jul 2024 06:44:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722321892; cv=none; b=MXKvlZeeJma8SVQjQgPJWjYbrpPaZ4v6MvBhrTxQou0I+3KdJ1ib8Q+cerQ9toIDxddc+IZh1vozsJCdVZZQVq65t7qbItZ3Va7Z5/Nw55r2QIhB/nGRHhrYreFokfsVhyv1LOo4U5WbLoGrlL/fbx1fQRXykYdng+Fm/qm/h9g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722321892; c=relaxed/simple; bh=DVj1dF/3LIG4QvumgddsFqj1lx0SQkF4MqPzNl7vGkQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NLBxRjRRSIabzDQiJmMVTg2NAiv0bF7pCb+bX6SQyYFW1bk6YTPUNBxuTtupCJeDelHkPz/0fNjwOpMUsqKsNoBmUcbVrwlbc36ny7p1WzHsGb2GoNtjLKtxXnCkt4jaTvIjBU2/83OHev3KjaMTa0IkRG0txp0NfvUXWJZo5kE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=XU5gA41f; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 03182C32782; Tue, 30 Jul 2024 06:44:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722321891; bh=DVj1dF/3LIG4QvumgddsFqj1lx0SQkF4MqPzNl7vGkQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XU5gA41fGeeGuG84jrc0u9UbdT/1OsrKiK6gQ3j2Dsfzspr/VFoGGY+9jRktUTxPK eCGcueThv1EkrEWTBCdEOfYYfg98cNkh9aL9IF5PQmYQLJsXHenG6BTEDfpnx4HRTt TG8HbzuCVQ0Yi4yCrfqsrt0bVyQPCbBoH3u3b1xRrZCKzGXrJd+WVkan/f/ggLnn2L OAxW9p4eTi5ixJZtACGTpHef4YO3O2bTkOma9P1V+YtTDRQRofXG8ycaO2YYLPUoLA G8KSSn7XyBfjWFhT9lu+2dLE6tceUhkIIcFz3EtYj5naYaEqlGB/vOaIaJXYdxgv9q dGnAH3je2sPvA== From: alexs@kernel.org To: Will Deacon , "Aneesh Kumar K . V" , Nick Piggin , Peter Zijlstra , Russell King , Catalin Marinas , Brian Cain , WANG Xuerui , Geert Uytterhoeven , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Michael Ellerman , Naveen N Rao , Paul Walmsley , Albert Ou , Thomas Gleixner , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Bibo Mao , Baolin Wang , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, loongarch@lists.linux.dev, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Qi Zheng , Vishal Moola , "Aneesh Kumar K . V" , Kemeng Shi , Lance Yang , Peter Xu , Barry Song , linux-s390@vger.kernel.org Cc: Guo Ren , Christophe Leroy , Palmer Dabbelt , Mike Rapoport , Oscar Salvador , Alexandre Ghiti , Jisheng Zhang , Samuel Holland , Anup Patel , Josh Poimboeuf , Breno Leitao , Alexander Gordeev , Gerald Schaefer , Hugh Dickins , David Hildenbrand , Ryan Roberts , Matthew Wilcox , Alex Shi , nvdimm@lists.linux.dev, linux-fsdevel@vger.kernel.org, sparclinux@vger.kernel.org, Dawei Li , Arnd Bergmann , Christian Brauner , Alexander Viro , Jan Kara , Dan Williams , Max Filippov , Chris Zankel , "David S . Miller" , "Naveen N . Rao" , Bjorn Helgaas , Sam Ravnborg , Jason Gunthorpe Subject: [RFC PATCH 09/18] mm/pgtable: fully use ptdesc in pte_alloc_one series functions Date: Tue, 30 Jul 2024 14:47:03 +0800 Message-ID: <20240730064712.3714387-10-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240730064712.3714387-1-alexs@kernel.org> References: <20240730064712.3714387-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: sparclinux@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Alex Shi Replace pgtable_t and struct page by ptdesc in pte_alloc_one series functions. Signed-off-by: Alex Shi Cc: linux-mm@kvack.org Cc: linux-arch@vger.kernel.org Cc: nvdimm@lists.linux.dev Cc: linux-fsdevel@vger.kernel.org Cc: sparclinux@vger.kernel.org Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org Cc: Dawei Li Cc: Vishal Moola Cc: Arnd Bergmann Cc: Christian Brauner Cc: Alexander Viro Cc: Jan Kara Cc: Matthew Wilcox Cc: Dan Williams Cc: Max Filippov Cc: Chris Zankel Cc: Peter Zijlstra Cc: Andy Lutomirski Cc: H. Peter Anvin Cc: x86@kernel.org Cc: Dave Hansen Cc: Borislav Petkov Cc: Thomas Gleixner Cc: David S. Miller Cc: Naveen N. Rao Cc: Christophe Leroy Cc: Nicholas Piggin Cc: Michael Ellerman Cc: Russell King Cc: Breno Leitao Cc: Josh Poimboeuf Cc: Bjorn Helgaas Cc: Sam Ravnborg Cc: Peter Xu Cc: Jason Gunthorpe Cc: Mike Rapoport Cc: Hugh Dickins --- arch/arm/include/asm/pgalloc.h | 9 ++++----- arch/powerpc/include/asm/pgalloc.h | 4 ++-- arch/s390/include/asm/pgalloc.h | 2 +- arch/sparc/include/asm/pgalloc_32.h | 2 +- arch/sparc/include/asm/pgalloc_64.h | 2 +- arch/sparc/mm/init_64.c | 2 +- arch/sparc/mm/srmmu.c | 4 ++-- arch/x86/include/asm/pgalloc.h | 2 +- arch/x86/mm/pgtable.c | 2 +- arch/xtensa/include/asm/pgalloc.h | 12 ++++++------ fs/dax.c | 2 +- include/asm-generic/pgalloc.h | 6 +++--- mm/huge_memory.c | 8 ++++---- mm/memory.c | 8 ++++---- 14 files changed, 32 insertions(+), 33 deletions(-) diff --git a/arch/arm/include/asm/pgalloc.h b/arch/arm/include/asm/pgalloc.h index a17f01235c29..e8501a6c3336 100644 --- a/arch/arm/include/asm/pgalloc.h +++ b/arch/arm/include/asm/pgalloc.h @@ -91,16 +91,15 @@ pte_alloc_one_kernel(struct mm_struct *mm) #define PGTABLE_HIGHMEM 0 #endif -static inline pgtable_t -pte_alloc_one(struct mm_struct *mm) +static inline struct ptdesc *pte_alloc_one(struct mm_struct *mm) { - struct page *pte; + struct ptdesc *pte; pte = __pte_alloc_one(mm, GFP_PGTABLE_USER | PGTABLE_HIGHMEM); if (!pte) return NULL; - if (!PageHighMem(pte)) - clean_pte_table(page_address(pte)); + if (!PageHighMem(ptdesc_page(pte))) + clean_pte_table(ptdesc_address(pte)); return pte; } diff --git a/arch/powerpc/include/asm/pgalloc.h b/arch/powerpc/include/asm/pgalloc.h index 3a971e2a8c73..37512f344b37 100644 --- a/arch/powerpc/include/asm/pgalloc.h +++ b/arch/powerpc/include/asm/pgalloc.h @@ -27,9 +27,9 @@ static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm) return (pte_t *)pte_fragment_alloc(mm, 1); } -static inline pgtable_t pte_alloc_one(struct mm_struct *mm) +static inline struct ptdesc *pte_alloc_one(struct mm_struct *mm) { - return (pgtable_t)pte_fragment_alloc(mm, 0); + return (struct ptdesc *)pte_fragment_alloc(mm, 0); } void pte_frag_destroy(void *pte_frag); diff --git a/arch/s390/include/asm/pgalloc.h b/arch/s390/include/asm/pgalloc.h index 7b84ef6dc4b6..771494526f6e 100644 --- a/arch/s390/include/asm/pgalloc.h +++ b/arch/s390/include/asm/pgalloc.h @@ -137,7 +137,7 @@ static inline void pmd_populate(struct mm_struct *mm, * page table entry allocation/free routines. */ #define pte_alloc_one_kernel(mm) ((pte_t *)page_table_alloc(mm)) -#define pte_alloc_one(mm) ((pte_t *)page_table_alloc(mm)) +#define pte_alloc_one(mm) ((struct ptdesc *)page_table_alloc(mm)) #define pte_free_kernel(mm, pte) page_table_free(mm, (unsigned long *) pte) #define pte_free(mm, pte) page_table_free(mm, (unsigned long *) pte) diff --git a/arch/sparc/include/asm/pgalloc_32.h b/arch/sparc/include/asm/pgalloc_32.h index 4f73e87b22a3..bc3ef54d9564 100644 --- a/arch/sparc/include/asm/pgalloc_32.h +++ b/arch/sparc/include/asm/pgalloc_32.h @@ -55,7 +55,7 @@ static inline void free_pmd_fast(pmd_t * pmd) void pmd_set(pmd_t *pmdp, pte_t *ptep); #define pmd_populate_kernel pmd_populate -pgtable_t pte_alloc_one(struct mm_struct *mm); +struct ptdesc *pte_alloc_one(struct mm_struct *mm); static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm) { diff --git a/arch/sparc/include/asm/pgalloc_64.h b/arch/sparc/include/asm/pgalloc_64.h index caa7632be4c2..285aa7958912 100644 --- a/arch/sparc/include/asm/pgalloc_64.h +++ b/arch/sparc/include/asm/pgalloc_64.h @@ -61,7 +61,7 @@ static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd) } pte_t *pte_alloc_one_kernel(struct mm_struct *mm); -pgtable_t pte_alloc_one(struct mm_struct *mm); +struct ptdesc *pte_alloc_one(struct mm_struct *mm); void pte_free_kernel(struct mm_struct *mm, pte_t *pte); void pte_free(struct mm_struct *mm, pgtable_t ptepage); diff --git a/arch/sparc/mm/init_64.c b/arch/sparc/mm/init_64.c index 53d7cb5bbffe..e1b33f996469 100644 --- a/arch/sparc/mm/init_64.c +++ b/arch/sparc/mm/init_64.c @@ -2900,7 +2900,7 @@ pte_t *pte_alloc_one_kernel(struct mm_struct *mm) return pte; } -pgtable_t pte_alloc_one(struct mm_struct *mm) +struct ptdesc *pte_alloc_one(struct mm_struct *mm) { struct ptdesc *ptdesc = pagetable_alloc(GFP_KERNEL | __GFP_ZERO, 0); diff --git a/arch/sparc/mm/srmmu.c b/arch/sparc/mm/srmmu.c index 9df51a62333d..60bb8628bb1f 100644 --- a/arch/sparc/mm/srmmu.c +++ b/arch/sparc/mm/srmmu.c @@ -346,7 +346,7 @@ pgd_t *get_pgd_fast(void) * Alignments up to the page size are the same for physical and virtual * addresses of the nocache area. */ -pgtable_t pte_alloc_one(struct mm_struct *mm) +struct ptdesc *pte_alloc_one(struct mm_struct *mm) { pte_t *ptep; struct page *page; @@ -362,7 +362,7 @@ pgtable_t pte_alloc_one(struct mm_struct *mm) } spin_unlock(&mm->page_table_lock); - return ptep; + return (struct ptdesc *)ptep; } void pte_free(struct mm_struct *mm, pgtable_t ptep) diff --git a/arch/x86/include/asm/pgalloc.h b/arch/x86/include/asm/pgalloc.h index dcd836b59beb..497c757b5b98 100644 --- a/arch/x86/include/asm/pgalloc.h +++ b/arch/x86/include/asm/pgalloc.h @@ -51,7 +51,7 @@ extern gfp_t __userpte_alloc_gfp; extern pgd_t *pgd_alloc(struct mm_struct *); extern void pgd_free(struct mm_struct *mm, pgd_t *pgd); -extern pgtable_t pte_alloc_one(struct mm_struct *); +extern struct ptdesc *pte_alloc_one(struct mm_struct *); extern void ___pte_free_tlb(struct mmu_gather *tlb, struct page *pte); diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index 93e54ba91fbf..c27d15cd01b9 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -28,7 +28,7 @@ void paravirt_tlb_remove_table(struct mmu_gather *tlb, void *table) gfp_t __userpte_alloc_gfp = GFP_PGTABLE_USER | PGTABLE_HIGHMEM; -pgtable_t pte_alloc_one(struct mm_struct *mm) +struct ptdesc *pte_alloc_one(struct mm_struct *mm) { return __pte_alloc_one(mm, __userpte_alloc_gfp); } diff --git a/arch/xtensa/include/asm/pgalloc.h b/arch/xtensa/include/asm/pgalloc.h index 7fc0f9126dd3..a9206c02956e 100644 --- a/arch/xtensa/include/asm/pgalloc.h +++ b/arch/xtensa/include/asm/pgalloc.h @@ -51,15 +51,15 @@ static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm) return ptep; } -static inline pgtable_t pte_alloc_one(struct mm_struct *mm) +static inline struct ptdesc *pte_alloc_one(struct mm_struct *mm) { - struct page *page; + struct ptdesc *ptdesc; - page = __pte_alloc_one(mm, GFP_PGTABLE_USER); - if (!page) + ptdesc = __pte_alloc_one(mm, GFP_PGTABLE_USER); + if (!ptdesc) return NULL; - ptes_clear(page_address(page)); - return page; + ptes_clear(ptdesc_address(ptdesc)); + return ptdesc; } #endif /* CONFIG_MMU */ diff --git a/fs/dax.c b/fs/dax.c index 6f7cea248206..51cbc08b22e7 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -1222,7 +1222,7 @@ static vm_fault_t dax_pmd_load_hole(struct xa_state *xas, struct vm_fault *vmf, DAX_PMD | DAX_ZERO_PAGE); if (arch_needs_pgtable_deposit()) { - ptdesc = page_ptdesc(pte_alloc_one(vma->vm_mm)); + ptdesc = pte_alloc_one(vma->vm_mm); if (!ptdesc) return VM_FAULT_OOM; } diff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h index 7c48f5fbf8aa..1a4070f8d5dd 100644 --- a/include/asm-generic/pgalloc.h +++ b/include/asm-generic/pgalloc.h @@ -63,7 +63,7 @@ static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte) * * Return: `struct page` referencing the ptdesc or %NULL on error */ -static inline pgtable_t __pte_alloc_one_noprof(struct mm_struct *mm, gfp_t gfp) +static inline struct ptdesc *__pte_alloc_one_noprof(struct mm_struct *mm, gfp_t gfp) { struct ptdesc *ptdesc; @@ -75,7 +75,7 @@ static inline pgtable_t __pte_alloc_one_noprof(struct mm_struct *mm, gfp_t gfp) return NULL; } - return ptdesc_page(ptdesc); + return ptdesc; } #define __pte_alloc_one(...) alloc_hooks(__pte_alloc_one_noprof(__VA_ARGS__)) @@ -88,7 +88,7 @@ static inline pgtable_t __pte_alloc_one_noprof(struct mm_struct *mm, gfp_t gfp) * * Return: `struct page` referencing the ptdesc or %NULL on error */ -static inline pgtable_t pte_alloc_one_noprof(struct mm_struct *mm) +static inline struct ptdesc *pte_alloc_one_noprof(struct mm_struct *mm) { return __pte_alloc_one_noprof(mm, GFP_PGTABLE_USER); } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 236e1582d97e..6274eb7559ac 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -959,7 +959,7 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf, } folio_throttle_swaprate(folio, gfp); - ptdesc = page_ptdesc(pte_alloc_one(vma->vm_mm)); + ptdesc = pte_alloc_one(vma->vm_mm); if (unlikely(!ptdesc)) { ret = VM_FAULT_OOM; goto release; @@ -1091,7 +1091,7 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) struct folio *zero_folio; vm_fault_t ret; - ptdesc = page_ptdesc(pte_alloc_one(vma->vm_mm)); + ptdesc = pte_alloc_one(vma->vm_mm); if (unlikely(!ptdesc)) return VM_FAULT_OOM; zero_folio = mm_get_huge_zero_folio(vma->vm_mm); @@ -1213,7 +1213,7 @@ vm_fault_t vmf_insert_pfn_pmd(struct vm_fault *vmf, pfn_t pfn, bool write) return VM_FAULT_SIGBUS; if (arch_needs_pgtable_deposit()) { - ptdesc = page_ptdesc(pte_alloc_one(vma->vm_mm)); + ptdesc = pte_alloc_one(vma->vm_mm); if (!ptdesc) return VM_FAULT_OOM; } @@ -1376,7 +1376,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, if (!vma_is_anonymous(dst_vma)) return 0; - ptdesc = page_ptdesc(pte_alloc_one(dst_mm)); + ptdesc = pte_alloc_one(dst_mm); if (unlikely(!ptdesc)) goto out; diff --git a/mm/memory.c b/mm/memory.c index 5b01d94a0b5f..37529e0a9ce2 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -445,7 +445,7 @@ void pmd_install(struct mm_struct *mm, pmd_t *pmd, pgtable_t *pte) int __pte_alloc(struct mm_struct *mm, pmd_t *pmd) { - struct ptdesc *ptdesc = page_ptdesc(pte_alloc_one(mm)); + struct ptdesc *ptdesc = pte_alloc_one(mm); if (!ptdesc) return -ENOMEM; @@ -4647,7 +4647,7 @@ static vm_fault_t __do_fault(struct vm_fault *vmf) * # flush A, B to clear the writeback */ if (pmd_none(*vmf->pmd) && !vmf->prealloc_pte) { - vmf->prealloc_pte = pte_alloc_one(vma->vm_mm); + vmf->prealloc_pte = ptdesc_page(pte_alloc_one(vma->vm_mm)); if (!vmf->prealloc_pte) return VM_FAULT_OOM; } @@ -4725,7 +4725,7 @@ vm_fault_t do_set_pmd(struct vm_fault *vmf, struct page *page) * related to pte entry. Use the preallocated table for that. */ if (arch_needs_pgtable_deposit() && !vmf->prealloc_pte) { - vmf->prealloc_pte = pte_alloc_one(vma->vm_mm); + vmf->prealloc_pte = ptdesc_page(pte_alloc_one(vma->vm_mm)); if (!vmf->prealloc_pte) return VM_FAULT_OOM; } @@ -5010,7 +5010,7 @@ static vm_fault_t do_fault_around(struct vm_fault *vmf) pte_off + vma_pages(vmf->vma) - vma_off) - 1; if (pmd_none(*vmf->pmd)) { - vmf->prealloc_pte = pte_alloc_one(vmf->vma->vm_mm); + vmf->prealloc_pte = ptdesc_page(pte_alloc_one(vmf->vma->vm_mm)); if (!vmf->prealloc_pte) return VM_FAULT_OOM; } From patchwork Tue Jul 30 06:47:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 1966332 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=otPDGIS+; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=mail.ozlabs.org; envelope-from=srs0=quqm=o6=vger.kernel.org=sparclinux+bounces-1760-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WY5ND4ZN6z1ybX for ; Tue, 30 Jul 2024 16:46:04 +1000 (AEST) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4WY5ND48NBz4x3d for ; Tue, 30 Jul 2024 16:46:04 +1000 (AEST) Received: by gandalf.ozlabs.org (Postfix) id 4WY5ND44mVz4x6n; Tue, 30 Jul 2024 16:46:04 +1000 (AEST) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=otPDGIS+; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=139.178.88.99; helo=sv.mirrors.kernel.org; envelope-from=sparclinux+bounces-1760-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [139.178.88.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4WY5ND1qj9z4x3d for ; Tue, 30 Jul 2024 16:46:04 +1000 (AEST) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 052C6284F73 for ; Tue, 30 Jul 2024 06:46:03 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 432BF190674; Tue, 30 Jul 2024 06:45:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="otPDGIS+" X-Original-To: sparclinux@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ED832190467; Tue, 30 Jul 2024 06:45:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722321908; cv=none; b=bse2Q0PKf1SQ7Dcz+NvQmlejy85zq720Spg2PiQytegKrSPV825KX0hlHPdOT5f2MzAJd2tygXzlWK0hGwGWX4zmN8n96i7e85WAkcDfcY/rdyxZK21uPzKDMc5tZjLnp0rvY3ex0xpFtd2taHbXMaB5FfXZ8nQy53H/w4kzS3M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722321908; c=relaxed/simple; bh=W5tYBso4UoPS/fKSVOluC4Ch4cJZaP/H2xHcE2b1FHU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sOMPA+O+1vwfracG62r5W9DXfbU1r7fxZ13o9EwneulrRBV8FiUBWJ5eWB5mvJOXzXaYdn5AujP61bebIrM0nrsTk17550M4+JuG+/kxehsGr8KQlRnIcjQ7iljPZyKs+86au/nQYl0eL3/oNDuzzH3tgQkCS8PsRgcvteROSH8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=otPDGIS+; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 59076C4AF0E; Tue, 30 Jul 2024 06:44:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722321907; bh=W5tYBso4UoPS/fKSVOluC4Ch4cJZaP/H2xHcE2b1FHU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=otPDGIS+BP+zJU1WerV1XStPgq7KbeiiHRdcUEyZGtj4yf/zK6A7xDzbi5ECaayAt YVSoB/3bQTSLE8x8/QN40AJ0s2PdSq3df6HoYbaIj4z13Vl5vZw9N5ye4PEjdht//7 bqv0egHAAAYkaLavBipdxSHoduTULRxYgtDM4b4mjLw9jYnjamZKyZ8Y6tyDakNpzC Y0Ij1GnGY4gY870o0InOzrbMFIggAcxm9mvmF741jpwmx9ZohF6o5cMeki7Psq4dLD /4dfI13pYO7FfkQIWJrOHrtaZ1MHS6fgo+N7tVvA1Q6NzphZJuPFR1n3WFSvbuCYFD UKCxIcO9HG0sA== From: alexs@kernel.org To: Will Deacon , "Aneesh Kumar K . V" , Nick Piggin , Peter Zijlstra , Russell King , Catalin Marinas , Brian Cain , WANG Xuerui , Geert Uytterhoeven , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Michael Ellerman , Naveen N Rao , Paul Walmsley , Albert Ou , Thomas Gleixner , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Bibo Mao , Baolin Wang , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, loongarch@lists.linux.dev, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Qi Zheng , Vishal Moola , "Aneesh Kumar K . V" , Kemeng Shi , Lance Yang , Peter Xu , Barry Song , linux-s390@vger.kernel.org Cc: Guo Ren , Christophe Leroy , Palmer Dabbelt , Mike Rapoport , Oscar Salvador , Alexandre Ghiti , Jisheng Zhang , Samuel Holland , Anup Patel , Josh Poimboeuf , Breno Leitao , Alexander Gordeev , Gerald Schaefer , Hugh Dickins , David Hildenbrand , Ryan Roberts , Matthew Wilcox , Alex Shi , nvdimm@lists.linux.dev, linux-fsdevel@vger.kernel.org, sparclinux@vger.kernel.org, Bjorn Helgaas , Arnd Bergmann , Christian Brauner , Alexander Viro , Jan Kara , Dan Williams , "David S . Miller" , "Naveen N . Rao" , Dawei Li Subject: [RFC PATCH 10/18] mm/pgtable: pass ptdesc to pte_free() Date: Tue, 30 Jul 2024 14:47:04 +0800 Message-ID: <20240730064712.3714387-11-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240730064712.3714387-1-alexs@kernel.org> References: <20240730064712.3714387-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: sparclinux@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Alex Shi Now we could remove couple of page<->ptdesc converters now. Signed-off-by: Alex Shi Cc: linux-mm@kvack.org Cc: linux-arch@vger.kernel.org Cc: nvdimm@lists.linux.dev Cc: linux-fsdevel@vger.kernel.org Cc: sparclinux@vger.kernel.org Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-m68k@lists.linux-m68k.org Cc: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org Cc: Bjorn Helgaas Cc: Vishal Moola Cc: Arnd Bergmann Cc: Christian Brauner Cc: Alexander Viro Cc: Jan Kara Cc: Matthew Wilcox Cc: Dan Williams Cc: David S. Miller Cc: Naveen N. Rao Cc: Christophe Leroy Cc: Nicholas Piggin Cc: Michael Ellerman Cc: Geert Uytterhoeven Cc: Russell King Cc: Mike Rapoport Cc: Dawei Li Cc: Hugh Dickins --- arch/arm/mm/pgd.c | 2 +- arch/m68k/include/asm/motorola_pgalloc.h | 4 ++-- arch/powerpc/include/asm/book3s/64/pgalloc.h | 2 +- arch/powerpc/include/asm/pgalloc.h | 2 +- arch/sparc/include/asm/pgalloc_32.h | 2 +- arch/sparc/mm/srmmu.c | 2 +- fs/dax.c | 2 +- include/asm-generic/pgalloc.h | 4 +--- mm/debug_vm_pgtable.c | 2 +- mm/huge_memory.c | 20 ++++++++++---------- mm/memory.c | 4 ++-- mm/pgtable-generic.c | 2 +- 12 files changed, 23 insertions(+), 25 deletions(-) diff --git a/arch/arm/mm/pgd.c b/arch/arm/mm/pgd.c index f8e9bc58a84f..c384b734d752 100644 --- a/arch/arm/mm/pgd.c +++ b/arch/arm/mm/pgd.c @@ -168,7 +168,7 @@ void pgd_free(struct mm_struct *mm, pgd_t *pgd_base) pte = pmd_pgtable(*pmd); pmd_clear(pmd); - pte_free(mm, pte); + pte_free(mm, page_ptdesc(pte)); mm_dec_nr_ptes(mm); no_pmd: pud_clear(pud); diff --git a/arch/m68k/include/asm/motorola_pgalloc.h b/arch/m68k/include/asm/motorola_pgalloc.h index 74a817d9387f..f6bb375971dc 100644 --- a/arch/m68k/include/asm/motorola_pgalloc.h +++ b/arch/m68k/include/asm/motorola_pgalloc.h @@ -39,9 +39,9 @@ static inline pgtable_t pte_alloc_one(struct mm_struct *mm) return get_pointer_table(TABLE_PTE); } -static inline void pte_free(struct mm_struct *mm, pgtable_t pgtable) +static inline void pte_free(struct mm_struct *mm, struct ptdesc *ptdesc) { - free_pointer_table(pgtable, TABLE_PTE); + free_pointer_table(ptdesc_page(ptdesc), TABLE_PTE); } static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pgtable, diff --git a/arch/powerpc/include/asm/book3s/64/pgalloc.h b/arch/powerpc/include/asm/book3s/64/pgalloc.h index dd2cff53a111..eb7d2ca59f62 100644 --- a/arch/powerpc/include/asm/book3s/64/pgalloc.h +++ b/arch/powerpc/include/asm/book3s/64/pgalloc.h @@ -162,7 +162,7 @@ static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, } static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmd, - pgtable_t pte_page) + struct ptdesc *pte_page) { *pmd = __pmd(__pgtable_ptr_val(pte_page) | PMD_VAL_BITS); } diff --git a/arch/powerpc/include/asm/pgalloc.h b/arch/powerpc/include/asm/pgalloc.h index 37512f344b37..12520521163e 100644 --- a/arch/powerpc/include/asm/pgalloc.h +++ b/arch/powerpc/include/asm/pgalloc.h @@ -40,7 +40,7 @@ static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte) pte_fragment_free((unsigned long *)pte, 1); } -static inline void pte_free(struct mm_struct *mm, pgtable_t ptepage) +static inline void pte_free(struct mm_struct *mm, struct ptdesc *ptepage) { pte_fragment_free((unsigned long *)ptepage, 0); } diff --git a/arch/sparc/include/asm/pgalloc_32.h b/arch/sparc/include/asm/pgalloc_32.h index bc3ef54d9564..addaade56f21 100644 --- a/arch/sparc/include/asm/pgalloc_32.h +++ b/arch/sparc/include/asm/pgalloc_32.h @@ -71,7 +71,7 @@ static inline void free_pte_fast(pte_t *pte) #define pte_free_kernel(mm, pte) free_pte_fast(pte) -void pte_free(struct mm_struct * mm, pgtable_t pte); +void pte_free(struct mm_struct *mm, struct ptdesc *pte); #define __pte_free_tlb(tlb, pte, addr) pte_free((tlb)->mm, pte) #endif /* _SPARC_PGALLOC_H */ diff --git a/arch/sparc/mm/srmmu.c b/arch/sparc/mm/srmmu.c index 60bb8628bb1f..05be7d86eda3 100644 --- a/arch/sparc/mm/srmmu.c +++ b/arch/sparc/mm/srmmu.c @@ -365,7 +365,7 @@ struct ptdesc *pte_alloc_one(struct mm_struct *mm) return (struct ptdesc *)ptep; } -void pte_free(struct mm_struct *mm, pgtable_t ptep) +void pte_free(struct mm_struct *mm, struct ptdesc *ptep) { struct page *page; diff --git a/fs/dax.c b/fs/dax.c index 51cbc08b22e7..61b9bd5200da 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -1246,7 +1246,7 @@ static vm_fault_t dax_pmd_load_hole(struct xa_state *xas, struct vm_fault *vmf, fallback: if (ptdesc) - pte_free(vma->vm_mm, ptdesc_page(ptdesc)); + pte_free(vma->vm_mm, ptdesc); trace_dax_pmd_load_hole_fallback(inode, vmf, zero_folio, *entry); return VM_FAULT_FALLBACK; } diff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h index 1a4070f8d5dd..5f249ec9d289 100644 --- a/include/asm-generic/pgalloc.h +++ b/include/asm-generic/pgalloc.h @@ -105,10 +105,8 @@ static inline struct ptdesc *pte_alloc_one_noprof(struct mm_struct *mm) * @mm: the mm_struct of the current context * @pte_page: the `struct page` referencing the ptdesc */ -static inline void pte_free(struct mm_struct *mm, struct page *pte_page) +static inline void pte_free(struct mm_struct *mm, struct ptdesc *ptdesc) { - struct ptdesc *ptdesc = page_ptdesc(pte_page); - pagetable_pte_dtor(ptdesc); pagetable_free(ptdesc); } diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c index e4969fb54da3..f256bc816744 100644 --- a/mm/debug_vm_pgtable.c +++ b/mm/debug_vm_pgtable.c @@ -1049,7 +1049,7 @@ static void __init destroy_args(struct pgtable_debug_args *args) /* Free page table entries */ if (args->start_ptep) { - pte_free(args->mm, args->start_ptep); + pte_free(args->mm, page_ptdesc(args->start_ptep)); mm_dec_nr_ptes(args->mm); } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 6274eb7559ac..dc323453fa02 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -987,7 +987,7 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf, if (userfaultfd_missing(vma)) { spin_unlock(vmf->ptl); folio_put(folio); - pte_free(vma->vm_mm, ptdesc_page(ptdesc)); + pte_free(vma->vm_mm, ptdesc); ret = handle_userfault(vmf, VM_UFFD_MISSING); VM_BUG_ON(ret & VM_FAULT_FALLBACK); return ret; @@ -1013,7 +1013,7 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf, spin_unlock(vmf->ptl); release: if (ptdesc) - pte_free(vma->vm_mm, ptdesc_page(ptdesc)); + pte_free(vma->vm_mm, ptdesc); folio_put(folio); return ret; @@ -1096,7 +1096,7 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) return VM_FAULT_OOM; zero_folio = mm_get_huge_zero_folio(vma->vm_mm); if (unlikely(!zero_folio)) { - pte_free(vma->vm_mm, ptdesc_page(ptdesc)); + pte_free(vma->vm_mm, ptdesc); count_vm_event(THP_FAULT_FALLBACK); return VM_FAULT_FALLBACK; } @@ -1106,10 +1106,10 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) ret = check_stable_address_space(vma->vm_mm); if (ret) { spin_unlock(vmf->ptl); - pte_free(vma->vm_mm, ptdesc_page(ptdesc)); + pte_free(vma->vm_mm, ptdesc); } else if (userfaultfd_missing(vma)) { spin_unlock(vmf->ptl); - pte_free(vma->vm_mm, ptdesc_page(ptdesc)); + pte_free(vma->vm_mm, ptdesc); ret = handle_userfault(vmf, VM_UFFD_MISSING); VM_BUG_ON(ret & VM_FAULT_FALLBACK); } else { @@ -1120,7 +1120,7 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) } } else { spin_unlock(vmf->ptl); - pte_free(vma->vm_mm, ptdesc_page(ptdesc)); + pte_free(vma->vm_mm, ptdesc); } return ret; } @@ -1178,7 +1178,7 @@ static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr, out_unlock: spin_unlock(ptl); if (ptdesc) - pte_free(mm, ptdesc_page(ptdesc)); + pte_free(mm, ptdesc); } /** @@ -1414,7 +1414,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, #endif if (unlikely(!pmd_trans_huge(pmd))) { - pte_free(dst_mm, ptdesc_page(ptdesc)); + pte_free(dst_mm, ptdesc); goto out_unlock; } /* @@ -1440,7 +1440,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, if (unlikely(folio_try_dup_anon_rmap_pmd(src_folio, src_page, src_vma))) { /* Page maybe pinned: split and retry the fault on PTEs. */ folio_put(src_folio); - pte_free(dst_mm, ptdesc_page(ptdesc)); + pte_free(dst_mm, ptdesc); spin_unlock(src_ptl); spin_unlock(dst_ptl); __split_huge_pmd(src_vma, src_pmd, addr, false, NULL); @@ -1830,7 +1830,7 @@ static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) pgtable_t pgtable; pgtable = pgtable_trans_huge_withdraw(mm, pmd); - pte_free(mm, pgtable); + pte_free(mm, page_ptdesc(pgtable)); mm_dec_nr_ptes(mm); } diff --git a/mm/memory.c b/mm/memory.c index 37529e0a9ce2..3014168e7296 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -451,7 +451,7 @@ int __pte_alloc(struct mm_struct *mm, pmd_t *pmd) pmd_install(mm, pmd, (pgtable_t *)&ptdesc); if (ptdesc) - pte_free(mm, ptdesc_page(ptdesc)); + pte_free(mm, ptdesc); return 0; } @@ -5196,7 +5196,7 @@ static vm_fault_t do_fault(struct vm_fault *vmf) /* preallocated pagetable is unused: free it */ if (vmf->prealloc_pte) { - pte_free(vm_mm, vmf->prealloc_pte); + pte_free(vm_mm, page_ptdesc(vmf->prealloc_pte)); vmf->prealloc_pte = NULL; } return ret; diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index f34a8d115f5b..92245a32656b 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -241,7 +241,7 @@ static void pte_free_now(struct rcu_head *head) struct ptdesc *ptdesc; ptdesc = container_of(head, struct ptdesc, pt_rcu_head); - pte_free(NULL /* mm not passed and not used */, (pgtable_t)ptdesc); + pte_free(NULL /* mm not passed and not used */, ptdesc); } void pte_free_defer(struct mm_struct *mm, pgtable_t pgtable) From patchwork Tue Jul 30 07:27:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 1966360 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=VX1gqElX; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=mail.ozlabs.org; envelope-from=srs0=zsvw=o6=vger.kernel.org=sparclinux+bounces-1761-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WY6C46SWjz1ybX for ; Tue, 30 Jul 2024 17:23:12 +1000 (AEST) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4WY6C375GLz4x6r for ; Tue, 30 Jul 2024 17:23:11 +1000 (AEST) Received: by gandalf.ozlabs.org (Postfix) id 4WY6C372B1z4x82; Tue, 30 Jul 2024 17:23:11 +1000 (AEST) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=VX1gqElX; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=139.178.88.99; helo=sv.mirrors.kernel.org; envelope-from=sparclinux+bounces-1761-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [139.178.88.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4WY6C31zb3z4x6r for ; Tue, 30 Jul 2024 17:23:11 +1000 (AEST) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id DA6CB284A96 for ; Tue, 30 Jul 2024 07:23:09 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E06DF18FDB8; Tue, 30 Jul 2024 07:22:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="VX1gqElX" X-Original-To: sparclinux@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9E12318EFF4; Tue, 30 Jul 2024 07:22:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722324175; cv=none; b=IQKj89tzOiRt/p4ZfsHLkp01GdU94fE3JeXyzr3Ows0As+C2B+pU1M65jiV9WmmxWTlkZ/D0F1b4yl8NfSvvTBHOQyvsrK1jFffPnd2JZA1tvSayXM2OqV4m7vohkN+ib+GLUk1c6q6cmxQ3pxjz2Wg9hGcsAiz3+6z2va5q0bc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722324175; c=relaxed/simple; bh=0TbHGOJe6xjBQAxgBmZt5/6tHpzUCuFv7Fnp2SPF0/s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=N+L48vj02aLjkJvp5oeKeGy2aSfbhIMv58J/Sy3yM8BJSUEpPz2q93gJShBWd6e8AxeELtQ1TFhKYTb2OCxALoHuysIvXgf+kllOzs76EpH+88CzcrVpkTSbZQ34Zu5t1jB0RcHrErqShKAO53jVWh1iG9wedZBvXLi5qPvpD1Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=VX1gqElX; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 02314C4AF18; Tue, 30 Jul 2024 07:22:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722324175; bh=0TbHGOJe6xjBQAxgBmZt5/6tHpzUCuFv7Fnp2SPF0/s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VX1gqElX7WCCLiUWy2ntLa+Ucwphw4/QSxbhHNT0raPC4/UkeHmzo9W0nHE9Mpecx GiufKIepmOHPcYPzSaWYuLKiSsVBbybFMQZ9B61DOBMKp6hANUQcKP3+nRBipeuwBf ugpyKDaDVAkIMoeBe17I8tOhrD6R4mB8TeoZK7sFzVjOFfs948DRJNpd6K2obeDmkP Jo8Bqwcsk9Y8AJMGzmYkRltiRneJQdQnfGXx+EqBl49hPEStt5gmDVcD28UqmIHves ETiZ03dTkJdj+XtouaSH1+gTnn7AFo+OzJm5Opx0xYfz4rxDS0SAqH+YYi9e12gtZu JdZQXBJ9yyREw== From: alexs@kernel.org To: Will Deacon , "Aneesh Kumar K . V" , Nick Piggin , Peter Zijlstra , Russell King , Catalin Marinas , Brian Cain , WANG Xuerui , Geert Uytterhoeven , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Michael Ellerman , Naveen N Rao , Paul Walmsley , Albert Ou , Thomas Gleixner , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Bibo Mao , Baolin Wang , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, loongarch@lists.linux.dev, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Qi Zheng , Vishal Moola , "Aneesh Kumar K . V" , Kemeng Shi , Lance Yang , Peter Xu , Barry Song , linux-s390@vger.kernel.org Cc: Guo Ren , Christophe Leroy , Palmer Dabbelt , Mike Rapoport , Oscar Salvador , Alexandre Ghiti , Jisheng Zhang , Samuel Holland , Anup Patel , Josh Poimboeuf , Breno Leitao , Alexander Gordeev , Gerald Schaefer , Hugh Dickins , David Hildenbrand , Ryan Roberts , Matthew Wilcox , Alex Shi , "Naveen N . Rao" , sparclinux@vger.kernel.org, Kinsey Ho , Benjamin Gray , Andreas Larsson , "David S . Miller" , Jason Gunthorpe Subject: [RFC PATCH 13/18] mm/pgtable: return ptdesc pointer in pgtable_trans_huge_withdraw Date: Tue, 30 Jul 2024 15:27:14 +0800 Message-ID: <20240730072719.3715016-3-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240730072719.3715016-1-alexs@kernel.org> References: <20240730064712.3714387-1-alexs@kernel.org> <20240730072719.3715016-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: sparclinux@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Alex Shi Way to replace pgtable_t aka struct page in most of archs. Signed-off-by: Alex Shi Cc: linux-mm@kvack.org Cc: sparclinux@vger.kernel.org Cc: linux-s390@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: linuxppc-dev@lists.ozlabs.org Cc: Barry Song Cc: Lance Yang Cc: Kinsey Ho Cc: Aneesh Kumar K.V Cc: Benjamin Gray Cc: Andreas Larsson Cc: David S. Miller Cc: Sven Schnelle Cc: Christian Borntraeger Cc: Vasily Gorbik Cc: Heiko Carstens Cc: Gerald Schaefer Cc: Alexander Gordeev Cc: Naveen N. Rao Cc: Nicholas Piggin Cc: Ryan Roberts Cc: Matthew Wilcox Cc: David Hildenbrand Cc: Jason Gunthorpe Cc: Aneesh Kumar K.V Cc: Peter Xu Cc: Mike Rapoport Cc: Christophe Leroy Cc: Michael Ellerman --- arch/powerpc/include/asm/book3s/64/hash-4k.h | 4 +-- arch/powerpc/include/asm/book3s/64/hash-64k.h | 4 +-- arch/powerpc/include/asm/book3s/64/pgtable.h | 2 +- arch/powerpc/include/asm/book3s/64/radix.h | 4 +-- arch/powerpc/mm/book3s64/hash_pgtable.c | 4 +-- arch/powerpc/mm/book3s64/radix_pgtable.c | 4 +-- arch/s390/include/asm/pgtable.h | 2 +- arch/s390/mm/pgtable.c | 4 +-- arch/sparc/include/asm/pgtable_64.h | 2 +- arch/sparc/mm/tlb.c | 4 +-- include/linux/pgtable.h | 2 +- mm/huge_memory.c | 35 ++++++++++--------- mm/pgtable-generic.c | 4 +-- 13 files changed, 38 insertions(+), 37 deletions(-) diff --git a/arch/powerpc/include/asm/book3s/64/hash-4k.h b/arch/powerpc/include/asm/book3s/64/hash-4k.h index c654c376ef8b..3a99a0229c37 100644 --- a/arch/powerpc/include/asm/book3s/64/hash-4k.h +++ b/arch/powerpc/include/asm/book3s/64/hash-4k.h @@ -133,8 +133,8 @@ extern unsigned long hash__pmd_hugepage_update(struct mm_struct *mm, extern pmd_t hash__pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp); extern void hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, - pgtable_t pgtable); -extern pgtable_t hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp); + struct ptdesc *ptdesc); +extern struct ptdesc *hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp); extern pmd_t hash__pmdp_huge_get_and_clear(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp); extern int hash__has_transparent_hugepage(void); diff --git a/arch/powerpc/include/asm/book3s/64/hash-64k.h b/arch/powerpc/include/asm/book3s/64/hash-64k.h index 0bf6fd0bf42a..8f497e1617bd 100644 --- a/arch/powerpc/include/asm/book3s/64/hash-64k.h +++ b/arch/powerpc/include/asm/book3s/64/hash-64k.h @@ -274,8 +274,8 @@ extern unsigned long hash__pmd_hugepage_update(struct mm_struct *mm, extern pmd_t hash__pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp); extern void hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, - pgtable_t pgtable); -extern pgtable_t hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp); + struct ptdesc *ptdesc); +extern struct ptdesc *hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp); extern pmd_t hash__pmdp_huge_get_and_clear(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp); extern int hash__has_transparent_hugepage(void); diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h index 519b1743a0f4..0ee440b819d7 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -1373,7 +1373,7 @@ static inline void pgtable_trans_huge_deposit(struct mm_struct *mm, } #define __HAVE_ARCH_PGTABLE_WITHDRAW -static inline pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, +static inline struct ptdesc *pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) { if (radix_enabled()) diff --git a/arch/powerpc/include/asm/book3s/64/radix.h b/arch/powerpc/include/asm/book3s/64/radix.h index 8f55ff74bb68..a8630b249f4c 100644 --- a/arch/powerpc/include/asm/book3s/64/radix.h +++ b/arch/powerpc/include/asm/book3s/64/radix.h @@ -291,8 +291,8 @@ extern unsigned long radix__pud_hugepage_update(struct mm_struct *mm, unsigned l extern pmd_t radix__pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp); extern void radix__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, - pgtable_t pgtable); -extern pgtable_t radix__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp); + struct ptdesc *ptdesc); +extern struct ptdesc *radix__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp); extern pmd_t radix__pmdp_huge_get_and_clear(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp); pud_t radix__pudp_huge_get_and_clear(struct mm_struct *mm, diff --git a/arch/powerpc/mm/book3s64/hash_pgtable.c b/arch/powerpc/mm/book3s64/hash_pgtable.c index 988948d69bc1..35562d1f4267 100644 --- a/arch/powerpc/mm/book3s64/hash_pgtable.c +++ b/arch/powerpc/mm/book3s64/hash_pgtable.c @@ -284,7 +284,7 @@ void hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, smp_wmb(); } -pgtable_t hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) +struct ptdesc *hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) { pgtable_t pgtable; pgtable_t *pgtable_slot; @@ -302,7 +302,7 @@ pgtable_t hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) * zero out the content on withdraw. */ memset(pgtable, 0, PTE_FRAG_SIZE); - return pgtable; + return (struct ptdesc *)pgtable; } /* diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c index b0d927009af8..3b9bb19510e3 100644 --- a/arch/powerpc/mm/book3s64/radix_pgtable.c +++ b/arch/powerpc/mm/book3s64/radix_pgtable.c @@ -1492,7 +1492,7 @@ void radix__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, pmd_huge_pte(mm, pmdp) = pgtable; } -pgtable_t radix__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) +struct ptdesc *radix__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) { pte_t *ptep; pgtable_t pgtable; @@ -1513,7 +1513,7 @@ pgtable_t radix__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) *ptep = __pte(0); ptep++; *ptep = __pte(0); - return pgtable; + return (struct ptdesc *)pgtable; } pmd_t radix__pmdp_huge_get_and_clear(struct mm_struct *mm, diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h index 3fa280d0672a..cf0baf4bfe5c 100644 --- a/arch/s390/include/asm/pgtable.h +++ b/arch/s390/include/asm/pgtable.h @@ -1738,7 +1738,7 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, pgtable_t pgtable); #define __HAVE_ARCH_PGTABLE_WITHDRAW -pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp); +struct ptdesc *pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp); #define __HAVE_ARCH_PMDP_SET_ACCESS_FLAGS static inline int pmdp_set_access_flags(struct vm_area_struct *vma, diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c index 201d350abd1e..b9016ee145cb 100644 --- a/arch/s390/mm/pgtable.c +++ b/arch/s390/mm/pgtable.c @@ -577,7 +577,7 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, pmd_huge_pte(mm, pmdp) = (struct ptdesc *)pgtable; } -pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) +struct ptdesc *pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) { struct list_head *lh; pgtable_t pgtable; @@ -598,7 +598,7 @@ pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) set_pte(ptep, __pte(_PAGE_INVALID)); ptep++; set_pte(ptep, __pte(_PAGE_INVALID)); - return pgtable; + return (struct ptdesc *)pgtable; } #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ diff --git a/arch/sparc/include/asm/pgtable_64.h b/arch/sparc/include/asm/pgtable_64.h index 3fe429d73a65..bfefd678e220 100644 --- a/arch/sparc/include/asm/pgtable_64.h +++ b/arch/sparc/include/asm/pgtable_64.h @@ -998,7 +998,7 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, pgtable_t pgtable); #define __HAVE_ARCH_PGTABLE_WITHDRAW -pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp); +struct ptdesc *pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp); #endif /* diff --git a/arch/sparc/mm/tlb.c b/arch/sparc/mm/tlb.c index 903825b4c997..bd2d3b1f6ba3 100644 --- a/arch/sparc/mm/tlb.c +++ b/arch/sparc/mm/tlb.c @@ -281,7 +281,7 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, pmd_huge_pte(mm, pmdp) = (struct ptdesc *)pgtable; } -pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) +struct ptdesc *pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) { struct list_head *lh; pgtable_t pgtable; @@ -300,6 +300,6 @@ pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) pte_val(pgtable[0]) = 0; pte_val(pgtable[1]) = 0; - return pgtable; + return (struct ptdesc *)pgtable; } #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 2a6a3cccfc36..3fa7b93580a3 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -929,7 +929,7 @@ extern void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, #endif #ifndef __HAVE_ARCH_PGTABLE_WITHDRAW -extern pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp); +extern struct ptdesc *pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp); #endif #ifndef arch_needs_pgtable_deposit diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 1c121ec85447..4dc36910c8aa 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1827,10 +1827,10 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) { - pgtable_t pgtable; + struct ptdesc *ptdesc; - pgtable = pgtable_trans_huge_withdraw(mm, pmd); - pte_free(mm, page_ptdesc(pgtable)); + ptdesc = pgtable_trans_huge_withdraw(mm, pmd); + pte_free(mm, ptdesc); mm_dec_nr_ptes(mm); } @@ -1959,9 +1959,10 @@ bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr, VM_BUG_ON(!pmd_none(*new_pmd)); if (pmd_move_must_withdraw(new_ptl, old_ptl, vma)) { - pgtable_t pgtable; - pgtable = pgtable_trans_huge_withdraw(mm, old_pmd); - pgtable_trans_huge_deposit(mm, new_pmd, pgtable); + struct ptdesc *ptdesc; + + ptdesc = pgtable_trans_huge_withdraw(mm, old_pmd); + pgtable_trans_huge_deposit(mm, new_pmd, ptdesc_page(ptdesc)); } pmd = move_soft_dirty_pmd(pmd); set_pmd_at(mm, new_addr, new_pmd, pmd); @@ -2130,7 +2131,7 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, pm struct folio *src_folio; struct anon_vma *src_anon_vma; spinlock_t *src_ptl, *dst_ptl; - pgtable_t src_pgtable; + struct ptdesc *src_ptdesc; struct mmu_notifier_range range; int err = 0; @@ -2234,8 +2235,8 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, pm } set_pmd_at(mm, dst_addr, dst_pmd, _dst_pmd); - src_pgtable = pgtable_trans_huge_withdraw(mm, src_pmd); - pgtable_trans_huge_deposit(mm, dst_pmd, src_pgtable); + src_ptdesc = pgtable_trans_huge_withdraw(mm, src_pmd); + pgtable_trans_huge_deposit(mm, dst_pmd, ptdesc_page(src_ptdesc)); unlock_ptls: double_pt_unlock(src_ptl, dst_ptl); if (src_anon_vma) { @@ -2347,7 +2348,7 @@ static void __split_huge_zero_page_pmd(struct vm_area_struct *vma, unsigned long haddr, pmd_t *pmd) { struct mm_struct *mm = vma->vm_mm; - pgtable_t pgtable; + struct ptdesc *ptdesc; pmd_t _pmd, old_pmd; unsigned long addr; pte_t *pte; @@ -2363,8 +2364,8 @@ static void __split_huge_zero_page_pmd(struct vm_area_struct *vma, */ old_pmd = pmdp_huge_clear_flush(vma, haddr, pmd); - pgtable = pgtable_trans_huge_withdraw(mm, pmd); - pmd_populate(mm, &_pmd, pgtable); + ptdesc = pgtable_trans_huge_withdraw(mm, pmd); + pmd_populate(mm, &_pmd, ptdesc_page(ptdesc)); pte = pte_offset_map(&_pmd, haddr); VM_BUG_ON(!pte); @@ -2381,7 +2382,7 @@ static void __split_huge_zero_page_pmd(struct vm_area_struct *vma, } pte_unmap(pte - 1); smp_wmb(); /* make pte visible before pmd */ - pmd_populate(mm, pmd, pgtable); + pmd_populate(mm, pmd, ptdesc_page(ptdesc)); } static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, @@ -2390,7 +2391,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, struct mm_struct *mm = vma->vm_mm; struct folio *folio; struct page *page; - pgtable_t pgtable; + struct ptdesc *ptdesc; pmd_t old_pmd, _pmd; bool young, write, soft_dirty, pmd_migration = false, uffd_wp = false; bool anon_exclusive = false, dirty = false; @@ -2535,8 +2536,8 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, * Withdraw the table only after we mark the pmd entry invalid. * This's critical for some architectures (Power). */ - pgtable = pgtable_trans_huge_withdraw(mm, pmd); - pmd_populate(mm, &_pmd, pgtable); + ptdesc = pgtable_trans_huge_withdraw(mm, pmd); + pmd_populate(mm, &_pmd, ptdesc_page(ptdesc)); pte = pte_offset_map(&_pmd, haddr); VM_BUG_ON(!pte); @@ -2601,7 +2602,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, put_page(page); smp_wmb(); /* make pte visible before pmd */ - pmd_populate(mm, pmd, pgtable); + pmd_populate(mm, pmd, ptdesc_page(ptdesc)); } void split_huge_pmd_locked(struct vm_area_struct *vma, unsigned long address, diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index 92245a32656b..de1ed30fea16 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -178,7 +178,7 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, #ifndef __HAVE_ARCH_PGTABLE_WITHDRAW /* no "address" argument so destroys page coloring of some arch */ -pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) +struct ptdesc *pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) { struct ptdesc *ptdesc; @@ -190,7 +190,7 @@ pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) struct ptdesc, pt_list); if (pmd_huge_pte(mm, pmdp)) list_del(&ptdesc->pt_list); - return ptdesc_page(ptdesc); + return ptdesc; } #endif From patchwork Tue Jul 30 07:27:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 1966362 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=HHL0szqj; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=mail.ozlabs.org; envelope-from=srs0=a05n=o6=vger.kernel.org=sparclinux+bounces-1762-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WY6Cd64Bdz1ybX for ; Tue, 30 Jul 2024 17:23:41 +1000 (AEST) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4WY6Cc1jmJz4x3d for ; Tue, 30 Jul 2024 17:23:40 +1000 (AEST) Received: by gandalf.ozlabs.org (Postfix) id 4WY6Cc1blTz4x87; Tue, 30 Jul 2024 17:23:40 +1000 (AEST) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=HHL0szqj; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45e3:2400::1; helo=sv.mirrors.kernel.org; envelope-from=sparclinux+bounces-1762-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [IPv6:2604:1380:45e3:2400::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4WY6Cb4sklz4x3d for ; Tue, 30 Jul 2024 17:23:39 +1000 (AEST) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 5ADE1285720 for ; Tue, 30 Jul 2024 07:23:38 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D89D018FDA6; Tue, 30 Jul 2024 07:23:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HHL0szqj" X-Original-To: sparclinux@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A66E18F2CA; Tue, 30 Jul 2024 07:23:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722324192; cv=none; b=Vc14RansOuL9tjHtWhxts6vh4l9nJE0oZsNRAXXt4+F8c0XjEp8lVzfivexYf6flDGqaB5DWliQRwCkqEer4Wb2obpXpdwD7tuHSSGprslhrRQOROFcTqwmetsXceLDuNjBG4QcTjds3M4uzUg0J7StQkSZq7v3ihF7SaWeAcg0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722324192; c=relaxed/simple; bh=+91Y+/MLNk1WBxXTysrg77L3vkr/WGy2+/WN+U1aHfM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=l7NgAqCdW+xjr9lTpovLN68h/AaP1z7xjRYbvN/p8ILy9OCQ1FlMj+x+WnkaolQJkOns2T+bCp88waXSp0qSFY4Zn1dP/z7aZUloqnSpFUxzMH9ZZr4HWQ9BLDTIr2NLecsl8CP2AjQkM4vDk47k8x1JoHqIcyjUI8Znf0FnHp4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HHL0szqj; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id B21F9C4AF16; Tue, 30 Jul 2024 07:22:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722324192; bh=+91Y+/MLNk1WBxXTysrg77L3vkr/WGy2+/WN+U1aHfM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HHL0szqjb9e2c7ISow55+gL3zMTJ7Hi70dgqGvtn04IgFlKSoCDbSjvq5pKd3SALU BNdZFk3RcqpWHqICH7MEB7lMmy5GUPrdSGvEDrygCOQcEoLTdfx6YJDQCsmkTky0Ql Dd6ra4zjUVvYJyB9u9IbpBdyrSpsi7fah8Mpxsshed3fhBinvJRbysQV62PvMQ39yN malm4RfG9bAhGyLJJu3pI2OdmRtcqaC1PB2x/qfzEDiHAkmrq0gPhjyLfii9l1DWIZ wHmNuR5GcmhGyMxkdgUgoHi1zrgdIL924cLjlxVp5VoiwZM31oi9eGVEUa4eTSyk8D bxsp/h7xWID4w== From: alexs@kernel.org To: Will Deacon , "Aneesh Kumar K . V" , Nick Piggin , Peter Zijlstra , Russell King , Catalin Marinas , Brian Cain , WANG Xuerui , Geert Uytterhoeven , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Michael Ellerman , Naveen N Rao , Paul Walmsley , Albert Ou , Thomas Gleixner , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Bibo Mao , Baolin Wang , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, loongarch@lists.linux.dev, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Qi Zheng , Vishal Moola , "Aneesh Kumar K . V" , Kemeng Shi , Lance Yang , Peter Xu , Barry Song , linux-s390@vger.kernel.org Cc: Guo Ren , Christophe Leroy , Palmer Dabbelt , Mike Rapoport , Oscar Salvador , Alexandre Ghiti , Jisheng Zhang , Samuel Holland , Anup Patel , Josh Poimboeuf , Breno Leitao , Alexander Gordeev , Gerald Schaefer , Hugh Dickins , David Hildenbrand , Ryan Roberts , Matthew Wilcox , Alex Shi , "Naveen N . Rao" , nvdimm@lists.linux.dev, linux-fsdevel@vger.kernel.org, sparclinux@vger.kernel.org, Kinsey Ho , Ingo Molnar , Christian Brauner , Alexander Viro , Jan Kara , Dan Williams , Andreas Larsson , "David S . Miller" , Jason Gunthorpe Subject: [RFC PATCH 14/18] mm/pgtable: use ptdesc in pgtable_trans_huge_deposit Date: Tue, 30 Jul 2024 15:27:15 +0800 Message-ID: <20240730072719.3715016-4-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240730072719.3715016-1-alexs@kernel.org> References: <20240730064712.3714387-1-alexs@kernel.org> <20240730072719.3715016-1-alexs@kernel.org> Precedence: bulk X-Mailing-List: sparclinux@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Alex Shi A step to replace pgtable_t to struct ptdesc. Signed-off-by: Alex Shi Cc: linux-mm@kvack.org Cc: nvdimm@lists.linux.dev Cc: linux-fsdevel@vger.kernel.org Cc: sparclinux@vger.kernel.org Cc: linux-s390@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: linuxppc-dev@lists.ozlabs.org Cc: Barry Song Cc: Lance Yang Cc: Hugh Dickins Cc: Kinsey Ho Cc: Ingo Molnar Cc: Aneesh Kumar K.V Cc: Christian Brauner Cc: Alexander Viro Cc: Jan Kara Cc: Dan Williams Cc: Andreas Larsson Cc: David S. Miller Cc: Sven Schnelle Cc: Christian Borntraeger Cc: Vasily Gorbik Cc: Heiko Carstens Cc: Gerald Schaefer Cc: Alexander Gordeev Cc: Naveen N. Rao Cc: Nicholas Piggin Cc: Ryan Roberts Cc: David Hildenbrand Cc: Jason Gunthorpe Cc: Aneesh Kumar K.V Cc: Mike Rapoport Cc: Peter Xu Cc: Matthew Wilcox Cc: Christophe Leroy Cc: Michael Ellerman --- arch/powerpc/include/asm/book3s/64/pgtable.h | 6 +++--- arch/powerpc/mm/book3s64/hash_pgtable.c | 6 +++--- arch/powerpc/mm/book3s64/radix_pgtable.c | 6 +++--- arch/s390/include/asm/pgtable.h | 2 +- arch/s390/mm/pgtable.c | 6 +++--- arch/sparc/include/asm/pgtable_64.h | 2 +- arch/sparc/mm/tlb.c | 6 +++--- fs/dax.c | 2 +- include/linux/pgtable.h | 2 +- mm/debug_vm_pgtable.c | 2 +- mm/huge_memory.c | 14 +++++++------- mm/khugepaged.c | 2 +- mm/memory.c | 2 +- mm/pgtable-generic.c | 8 ++++---- 14 files changed, 33 insertions(+), 33 deletions(-) diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h index 0ee440b819d7..cf44e2440825 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -1365,11 +1365,11 @@ pud_t pudp_huge_get_and_clear_full(struct vm_area_struct *vma, #define __HAVE_ARCH_PGTABLE_DEPOSIT static inline void pgtable_trans_huge_deposit(struct mm_struct *mm, - pmd_t *pmdp, pgtable_t pgtable) + pmd_t *pmdp, struct ptdesc *ptdesc) { if (radix_enabled()) - return radix__pgtable_trans_huge_deposit(mm, pmdp, pgtable); - return hash__pgtable_trans_huge_deposit(mm, pmdp, pgtable); + return radix__pgtable_trans_huge_deposit(mm, pmdp, ptdesc); + return hash__pgtable_trans_huge_deposit(mm, pmdp, ptdesc); } #define __HAVE_ARCH_PGTABLE_WITHDRAW diff --git a/arch/powerpc/mm/book3s64/hash_pgtable.c b/arch/powerpc/mm/book3s64/hash_pgtable.c index 35562d1f4267..8fd2c833dc3d 100644 --- a/arch/powerpc/mm/book3s64/hash_pgtable.c +++ b/arch/powerpc/mm/book3s64/hash_pgtable.c @@ -265,16 +265,16 @@ pmd_t hash__pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long addres * the base page size hptes */ void hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, - pgtable_t pgtable) + struct ptdesc *ptdesc) { - pgtable_t *pgtable_slot; + pte_t **pgtable_slot; assert_spin_locked(pmd_lockptr(mm, pmdp)); /* * we store the pgtable in the second half of PMD */ pgtable_slot = (pgtable_t *)pmdp + PTRS_PER_PMD; - *pgtable_slot = pgtable; + *pgtable_slot = (pte_t)ptdesc; /* * expose the deposited pgtable to other cpus. * before we set the hugepage PTE at pmd level diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c index 3b9bb19510e3..c33e860966ad 100644 --- a/arch/powerpc/mm/book3s64/radix_pgtable.c +++ b/arch/powerpc/mm/book3s64/radix_pgtable.c @@ -1478,9 +1478,9 @@ pmd_t radix__pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long addre * list_head memory area. */ void radix__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, - pgtable_t pgtable) + struct ptdesc *ptdesc) { - struct list_head *lh = (struct list_head *) pgtable; + struct list_head *lh = (struct list_head *)ptdesc; assert_spin_locked(pmd_lockptr(mm, pmdp)); @@ -1489,7 +1489,7 @@ void radix__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, INIT_LIST_HEAD(lh); else list_add(lh, (struct list_head *) pmd_huge_pte(mm, pmdp)); - pmd_huge_pte(mm, pmdp) = pgtable; + pmd_huge_pte(mm, pmdp) = ptdesc_page(ptdesc); } struct ptdesc *radix__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h index cf0baf4bfe5c..d7b635f5e1e7 100644 --- a/arch/s390/include/asm/pgtable.h +++ b/arch/s390/include/asm/pgtable.h @@ -1735,7 +1735,7 @@ pud_t pudp_xchg_direct(struct mm_struct *, unsigned long, pud_t *, pud_t); #define __HAVE_ARCH_PGTABLE_DEPOSIT void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, - pgtable_t pgtable); + struct ptdesc *ptdesc); #define __HAVE_ARCH_PGTABLE_WITHDRAW struct ptdesc *pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp); diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c index b9016ee145cb..cf1a6aeb66d4 100644 --- a/arch/s390/mm/pgtable.c +++ b/arch/s390/mm/pgtable.c @@ -563,9 +563,9 @@ EXPORT_SYMBOL(pudp_xchg_direct); #ifdef CONFIG_TRANSPARENT_HUGEPAGE void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, - pgtable_t pgtable) + struct ptdesc *ptdesc) { - struct list_head *lh = (struct list_head *) pgtable; + struct list_head *lh = (struct list_head *)ptdesc; assert_spin_locked(pmd_lockptr(mm, pmdp)); @@ -574,7 +574,7 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, INIT_LIST_HEAD(lh); else list_add(lh, (struct list_head *) pmd_huge_pte(mm, pmdp)); - pmd_huge_pte(mm, pmdp) = (struct ptdesc *)pgtable; + pmd_huge_pte(mm, pmdp) = ptdesc; } struct ptdesc *pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) diff --git a/arch/sparc/include/asm/pgtable_64.h b/arch/sparc/include/asm/pgtable_64.h index bfefd678e220..c71be5ef8b06 100644 --- a/arch/sparc/include/asm/pgtable_64.h +++ b/arch/sparc/include/asm/pgtable_64.h @@ -995,7 +995,7 @@ extern pmd_t pmdp_invalidate(struct vm_area_struct *vma, unsigned long address, #define __HAVE_ARCH_PGTABLE_DEPOSIT void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, - pgtable_t pgtable); + struct ptdesc *ptdesc); #define __HAVE_ARCH_PGTABLE_WITHDRAW struct ptdesc *pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp); diff --git a/arch/sparc/mm/tlb.c b/arch/sparc/mm/tlb.c index bd2d3b1f6ba3..eeed4427f524 100644 --- a/arch/sparc/mm/tlb.c +++ b/arch/sparc/mm/tlb.c @@ -267,9 +267,9 @@ pmd_t pmdp_invalidate(struct vm_area_struct *vma, unsigned long address, } void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, - pgtable_t pgtable) + struct ptdesc *ptdesc) { - struct list_head *lh = (struct list_head *) pgtable; + struct list_head *lh = (struct list_head *)ptdesc; assert_spin_locked(&mm->page_table_lock); @@ -278,7 +278,7 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, INIT_LIST_HEAD(lh); else list_add(lh, (struct list_head *) pmd_huge_pte(mm, pmdp)); - pmd_huge_pte(mm, pmdp) = (struct ptdesc *)pgtable; + pmd_huge_pte(mm, pmdp) = ptdesc; } struct ptdesc *pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) diff --git a/fs/dax.c b/fs/dax.c index 61b9bd5200da..4b4e6acb0efc 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -1234,7 +1234,7 @@ static vm_fault_t dax_pmd_load_hole(struct xa_state *xas, struct vm_fault *vmf, } if (ptdesc) { - pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, ptdesc_page(ptdesc)); + pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, ptdesc); mm_inc_nr_ptes(vma->vm_mm); } pmd_entry = mk_pmd(&zero_folio->page, vmf->vma->vm_page_prot); diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 3fa7b93580a3..9d256c548f5e 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -925,7 +925,7 @@ static inline pmd_t pmdp_collapse_flush(struct vm_area_struct *vma, #ifndef __HAVE_ARCH_PGTABLE_DEPOSIT extern void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, - pgtable_t pgtable); + struct ptdesc *ptdesc); #endif #ifndef __HAVE_ARCH_PGTABLE_WITHDRAW diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c index f256bc816744..8550eec32aba 100644 --- a/mm/debug_vm_pgtable.c +++ b/mm/debug_vm_pgtable.c @@ -225,7 +225,7 @@ static void __init pmd_advanced_tests(struct pgtable_debug_args *args) /* Align the address wrt HPAGE_PMD_SIZE */ vaddr &= HPAGE_PMD_MASK; - pgtable_trans_huge_deposit(args->mm, args->pmdp, args->start_ptep); + pgtable_trans_huge_deposit(args->mm, args->pmdp, page_ptdesc(args->start_ptep)); pmd = pfn_pmd(args->pmd_pfn, args->page_prot); set_pmd_at(args->mm, vaddr, args->pmdp, pmd); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 4dc36910c8aa..aac67e8a8cc8 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -997,7 +997,7 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf, entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma); folio_add_new_anon_rmap(folio, vma, haddr, RMAP_EXCLUSIVE); folio_add_lru_vma(folio, vma); - pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, ptdesc_page(ptdesc)); + pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, ptdesc); set_pmd_at(vma->vm_mm, haddr, vmf->pmd, entry); update_mmu_cache_pmd(vma, vmf->address, vmf->pmd); add_mm_counter(vma->vm_mm, MM_ANONPAGES, HPAGE_PMD_NR); @@ -1064,7 +1064,7 @@ static void set_huge_zero_folio(struct ptdesc *ptdesc, struct mm_struct *mm, return; entry = mk_pmd(&zero_folio->page, vma->vm_page_prot); entry = pmd_mkhuge(entry); - pgtable_trans_huge_deposit(mm, pmd, ptdesc_page(ptdesc)); + pgtable_trans_huge_deposit(mm, pmd, ptdesc); set_pmd_at(mm, haddr, pmd, entry); mm_inc_nr_ptes(mm); } @@ -1167,7 +1167,7 @@ static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr, } if (ptdesc) { - pgtable_trans_huge_deposit(mm, pmd, ptdesc_page(ptdesc)); + pgtable_trans_huge_deposit(mm, pmd, ptdesc); mm_inc_nr_ptes(mm); ptdesc = NULL; } @@ -1404,7 +1404,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, } add_mm_counter(dst_mm, MM_ANONPAGES, HPAGE_PMD_NR); mm_inc_nr_ptes(dst_mm); - pgtable_trans_huge_deposit(dst_mm, dst_pmd, ptdesc_page(ptdesc)); + pgtable_trans_huge_deposit(dst_mm, dst_pmd, ptdesc); if (!userfaultfd_wp(dst_vma)) pmd = pmd_swp_clear_uffd_wp(pmd); set_pmd_at(dst_mm, addr, dst_pmd, pmd); @@ -1449,7 +1449,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, add_mm_counter(dst_mm, MM_ANONPAGES, HPAGE_PMD_NR); out_zero_page: mm_inc_nr_ptes(dst_mm); - pgtable_trans_huge_deposit(dst_mm, dst_pmd, ptdesc_page(ptdesc)); + pgtable_trans_huge_deposit(dst_mm, dst_pmd, ptdesc); pmdp_set_wrprotect(src_mm, addr, src_pmd); if (!userfaultfd_wp(dst_vma)) pmd = pmd_clear_uffd_wp(pmd); @@ -1962,7 +1962,7 @@ bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr, struct ptdesc *ptdesc; ptdesc = pgtable_trans_huge_withdraw(mm, old_pmd); - pgtable_trans_huge_deposit(mm, new_pmd, ptdesc_page(ptdesc)); + pgtable_trans_huge_deposit(mm, new_pmd, ptdesc); } pmd = move_soft_dirty_pmd(pmd); set_pmd_at(mm, new_addr, new_pmd, pmd); @@ -2236,7 +2236,7 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, pm set_pmd_at(mm, dst_addr, dst_pmd, _dst_pmd); src_ptdesc = pgtable_trans_huge_withdraw(mm, src_pmd); - pgtable_trans_huge_deposit(mm, dst_pmd, ptdesc_page(src_ptdesc)); + pgtable_trans_huge_deposit(mm, dst_pmd, src_ptdesc); unlock_ptls: double_pt_unlock(src_ptl, dst_ptl); if (src_anon_vma) { diff --git a/mm/khugepaged.c b/mm/khugepaged.c index f3b3db104615..48a54269472e 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1232,7 +1232,7 @@ static int collapse_huge_page(struct mm_struct *mm, unsigned long address, BUG_ON(!pmd_none(*pmd)); folio_add_new_anon_rmap(folio, vma, address, RMAP_EXCLUSIVE); folio_add_lru_vma(folio, vma); - pgtable_trans_huge_deposit(mm, pmd, pgtable); + pgtable_trans_huge_deposit(mm, pmd, page_ptdesc(pgtable)); set_pmd_at(mm, address, pmd, _pmd); update_mmu_cache_pmd(vma, address, pmd); spin_unlock(pmd_ptl); diff --git a/mm/memory.c b/mm/memory.c index 27c2f63b7487..956cfe5f644d 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4687,7 +4687,7 @@ static void deposit_prealloc_pte(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; - pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, vmf->prealloc_pte); + pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, page_ptdesc(vmf->prealloc_pte)); /* * We are going to consume the prealloc table, * count that as nr_ptes. diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index de1ed30fea16..5e763682941d 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -163,16 +163,16 @@ pud_t pudp_huge_clear_flush(struct vm_area_struct *vma, unsigned long address, #ifndef __HAVE_ARCH_PGTABLE_DEPOSIT void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, - pgtable_t pgtable) + struct ptdesc *ptdesc) { assert_spin_locked(pmd_lockptr(mm, pmdp)); /* FIFO */ if (!pmd_huge_pte(mm, pmdp)) - INIT_LIST_HEAD(&pgtable->lru); + INIT_LIST_HEAD(&ptdesc->pt_list); else - list_add(&pgtable->lru, &pmd_huge_pte(mm, pmdp)->pt_list); - pmd_huge_pte(mm, pmdp) = page_ptdesc(pgtable); + list_add(&ptdesc->pt_list, &pmd_huge_pte(mm, pmdp)->pt_list); + pmd_huge_pte(mm, pmdp) = ptdesc; } #endif