[v13,29/85] KVM: pfncache: Precisely track refcounted pages

Message ID	20241010182427.1434605-30-seanjc@google.com
State	Accepted
Headers	show Return-Path: <kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org> Date: Thu, 10 Oct 2024 11:23:31 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> Message-ID: <20241010182427.1434605-30-seanjc@google.com> Subject: [PATCH v13 29/85] KVM: pfncache: Precisely track refcounted pages From: Sean Christopherson <seanjc@google.com> To: Paolo Bonzini <pbonzini@redhat.com>, Marc Zyngier <maz@kernel.org>, Oliver Upton <oliver.upton@linux.dev>, Tianrui Zhao <zhaotianrui@loongson.cn>, Bibo Mao <maobibo@loongson.cn>, Huacai Chen <chenhuacai@kernel.org>, Michael Ellerman <mpe@ellerman.id.au>, Anup Patel <anup@brainfault.org>, Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>, Christian Borntraeger <borntraeger@linux.ibm.com>, Janosch Frank <frankja@linux.ibm.com>, Claudio Imbrenda <imbrenda@linux.ibm.com>, Sean Christopherson <seanjc@google.com> Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " <alex.bennee@linaro.org>, Yan Zhao <yan.y.zhao@intel.com>, David Matlack <dmatlack@google.com>, David Stevens <stevensd@chromium.org>, Andrew Jones <ajones@ventanamicro.com> Precedence: list Reply-To: Sean Christopherson <seanjc@google.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "kvm-riscv" <kvm-riscv-bounces@lists.infradead.org> Errors-To: kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org
Series	KVM: Stop grabbing references to PFNMAP'd pages \| expand [v13,00/85] KVM: Stop grabbing references to PFNMAP'd pages [v13,01/85] KVM: Drop KVM_ERR_PTR_BAD_PAGE and instead return NULL to indicate an error [v13,02/85] KVM: Allow calling kvm_release_page_{clean,dirty}() on a NULL page pointer [v13,03/85] KVM: Add kvm_release_page_unused() API to put pages that KVM never consumes [v13,04/85] KVM: x86/mmu: Skip the "try unsync" path iff the old SPTE was a leaf SPTE [v13,05/85] KVM: x86/mmu: Don't overwrite shadow-present MMU SPTEs when prefaulting [v13,06/85] KVM: x86/mmu: Invert @can_unsync and renamed to @synchronizing [v13,07/85] KVM: x86/mmu: Mark new SPTE as Accessed when synchronizing existing SPTE [v13,08/85] KVM: x86/mmu: Mark folio dirty when creating SPTE, not when zapping/modifying [v13,09/85] KVM: x86/mmu: Mark page/folio accessed only when zapping leaf SPTEs [v13,10/85] KVM: x86/mmu: Use gfn_to_page_many_atomic() when prefetching indirect PTEs [v13,11/85] KVM: Rename gfn_to_page_many_atomic() to kvm_prefetch_pages() [v13,12/85] KVM: Drop @atomic param from gfn=>pfn and hva=>pfn APIs [v13,13/85] KVM: Annotate that all paths in hva_to_pfn() might sleep [v13,14/85] KVM: Return ERR_SIGPENDING from hva_to_pfn() if GUP returns -EGAIN [v13,15/85] KVM: Drop extra GUP (via check_user_page_hwpoison()) to detect poisoned page [v13,16/85] KVM: Replace "async" pointer in gfn=>pfn with "no_wait" and error code [v13,17/85] KVM: x86/mmu: Drop kvm_page_fault.hva, i.e. don't track intermediate hva [v13,18/85] KVM: Drop unused "hva" pointer from __gfn_to_pfn_memslot() [v13,19/85] KVM: Introduce kvm_follow_pfn() to eventually replace "gfn_to_pfn" APIs [v13,20/85] KVM: Remove pointless sanity check on @map param to kvm_vcpu_(un)map() [v13,21/85] KVM: Explicitly initialize all fields at the start of kvm_vcpu_map() [v13,22/85] KVM: Use NULL for struct page pointer to indicate mremapped memory [v13,23/85] KVM: nVMX: Rely on kvm_vcpu_unmap() to track validity of eVMCS mapping [v13,24/85] KVM: nVMX: Drop pointless msr_bitmap_map field from struct nested_vmx [v13,25/85] KVM: nVMX: Add helper to put (unmap) vmcs12 pages [v13,26/85] KVM: Use plain "struct page" pointer instead of single-entry array [v13,27/85] KVM: Provide refcounted page as output field in struct kvm_follow_pfn [v13,28/85] KVM: Move kvm_{set,release}_page_{clean,dirty}() helpers up in kvm_main.c [v13,29/85] KVM: pfncache: Precisely track refcounted pages [v13,30/85] KVM: Migrate kvm_vcpu_map() to kvm_follow_pfn() [v13,31/85] KVM: Pin (as in FOLL_PIN) pages during kvm_vcpu_map() [v13,32/85] KVM: nVMX: Mark vmcs12's APIC access page dirty when unmapping [v13,33/85] KVM: Pass in write/dirty to kvm_vcpu_map(), not kvm_vcpu_unmap() [v13,34/85] KVM: Get writable mapping for __kvm_vcpu_map() only when necessary [v13,35/85] KVM: Disallow direct access (w/o mmu_notifier) to unpinned pfn by default [v13,36/85] KVM: x86: Don't fault-in APIC access page during initial allocation [v13,37/85] KVM: x86/mmu: Add "mmu" prefix fault-in helpers to free up generic names [v13,38/85] KVM: x86/mmu: Put direct prefetched pages via kvm_release_page_clean() [v13,39/85] KVM: x86/mmu: Add common helper to handle prefetching SPTEs [v13,40/85] KVM: x86/mmu: Add helper to "finish" handling a guest page fault [v13,41/85] KVM: x86/mmu: Mark pages/folios dirty at the origin of make_spte() [v13,42/85] KVM: Move declarations of memslot accessors up in kvm_host.h [v13,43/85] KVM: Add kvm_faultin_pfn() to specifically service guest page faults [v13,44/85] KVM: x86/mmu: Convert page fault paths to kvm_faultin_pfn() [v13,45/85] KVM: guest_memfd: Pass index, not gfn, to __kvm_gmem_get_pfn() [v13,46/85] KVM: guest_memfd: Provide "struct page" as output from kvm_gmem_get_pfn() [v13,47/85] KVM: x86/mmu: Put refcounted pages instead of blindly releasing pfns [v13,48/85] KVM: x86/mmu: Don't mark unused faultin pages as accessed [v13,49/85] KVM: Move x86's API to release a faultin page to common KVM [v13,50/85] KVM: VMX: Hold mmu_lock until page is released when updating APIC access page [v13,51/85] KVM: VMX: Use __kvm_faultin_page() to get APIC access page/pfn [v13,52/85] KVM: PPC: e500: Mark "struct page" dirty in kvmppc_e500_shadow_map() [v13,53/85] KVM: PPC: e500: Mark "struct page" pfn accessed before dropping mmu_lock [v13,54/85] KVM: PPC: e500: Use __kvm_faultin_pfn() to handle page faults [v13,55/85] KVM: arm64: Mark "struct page" pfns accessed/dirty before dropping mmu_lock [v13,56/85] KVM: arm64: Use __kvm_faultin_pfn() to handle memory aborts [v13,57/85] KVM: RISC-V: Mark "struct page" pfns dirty iff a stage-2 PTE is installed [v13,58/85] KVM: RISC-V: Mark "struct page" pfns accessed before dropping mmu_lock [v13,59/85] KVM: RISC-V: Use kvm_faultin_pfn() when mapping pfns into the guest [v13,60/85] KVM: PPC: Use __kvm_faultin_pfn() to handle page faults on Book3s HV [v13,61/85] KVM: PPC: Use __kvm_faultin_pfn() to handle page faults on Book3s Radix [v13,62/85] KVM: PPC: Drop unused @kvm_ro param from kvmppc_book3s_instantiate_page() [v13,63/85] KVM: PPC: Book3S: Mark "struct page" pfns dirty/accessed after installing PTE [v13,64/85] KVM: PPC: Use kvm_faultin_pfn() to handle page faults on Book3s PR [v13,65/85] KVM: LoongArch: Mark "struct page" pfns dirty only in "slow" page fault path [v13,66/85] KVM: LoongArch: Mark "struct page" pfns accessed only in "slow" page fault path [v13,67/85] KVM: LoongArch: Mark "struct page" pfn accessed before dropping mmu_lock [v13,68/85] KVM: LoongArch: Use kvm_faultin_pfn() to map pfns into the guest [v13,69/85] KVM: MIPS: Mark "struct page" pfns dirty only in "slow" page fault path [v13,70/85] KVM: MIPS: Mark "struct page" pfns accessed only in "slow" page fault path [v13,71/85] KVM: MIPS: Mark "struct page" pfns accessed prior to dropping mmu_lock [v13,72/85] KVM: MIPS: Use kvm_faultin_pfn() to map pfns into the guest [v13,73/85] KVM: PPC: Remove extra get_page() to fix page refcount leak [v13,74/85] KVM: PPC: Use kvm_vcpu_map() to map guest memory to patch dcbz instructions [v13,75/85] KVM: Convert gfn_to_page() to use kvm_follow_pfn() [v13,76/85] KVM: Add support for read-only usage of gfn_to_page() [v13,77/85] KVM: arm64: Use __gfn_to_page() when copying MTE tags to/from userspace [v13,78/85] KVM: PPC: Explicitly require struct page memory for Ultravisor sharing [v13,79/85] KVM: Drop gfn_to_pfn() APIs now that all users are gone [v13,80/85] KVM: s390: Use kvm_release_page_dirty() to unpin "struct page" memory [v13,81/85] KVM: Make kvm_follow_pfn.refcounted_page a required field [v13,82/85] KVM: x86/mmu: Don't mark "struct page" accessed when zapping SPTEs [v13,83/85] KVM: arm64: Don't mark "struct page" accessed when making SPTE young [v13,84/85] KVM: Drop APIs that manipulate "struct page" via pfns [v13,85/85] KVM: Don't grab reference on VM_MIXEDMAP pfns that have a "struct page"

Message ID

20241010182427.1434605-30-seanjc@google.com

State

Accepted

Headers

Date: Thu, 10 Oct 2024 11:23:31 -0700
In-Reply-To: <20241010182427.1434605-1-seanjc@google.com>
Mime-Version: 1.0
References: <20241010182427.1434605-1-seanjc@google.com>
Message-ID: <20241010182427.1434605-30-seanjc@google.com>
Subject: [PATCH v13 29/85] KVM: pfncache: Precisely track refcounted pages
From: Sean Christopherson <seanjc@google.com>
To: Paolo Bonzini <pbonzini@redhat.com>, Marc Zyngier <maz@kernel.org>,
	Oliver Upton <oliver.upton@linux.dev>,
 Tianrui Zhao <zhaotianrui@loongson.cn>,
	Bibo Mao <maobibo@loongson.cn>, Huacai Chen <chenhuacai@kernel.org>,
	Michael Ellerman <mpe@ellerman.id.au>, Anup Patel <anup@brainfault.org>,
	Paul Walmsley <paul.walmsley@sifive.com>,
 Palmer Dabbelt <palmer@dabbelt.com>,
	Albert Ou <aou@eecs.berkeley.edu>,
 Christian Borntraeger <borntraeger@linux.ibm.com>,
	Janosch Frank <frankja@linux.ibm.com>,
 Claudio Imbrenda <imbrenda@linux.ibm.com>,
	Sean Christopherson <seanjc@google.com>
Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
  kvmarm@lists.linux.dev, loongarch@lists.linux.dev,
 linux-mips@vger.kernel.org,  linuxppc-dev@lists.ozlabs.org,
 kvm-riscv@lists.infradead.org,  linux-riscv@lists.infradead.org,
 linux-kernel@vger.kernel.org,
  " =?utf-8?q?Alex_Benn=C3=A9e?= " <alex.bennee@linaro.org>,
 Yan Zhao <yan.y.zhao@intel.com>,  David Matlack <dmatlack@google.com>,
 David Stevens <stevensd@chromium.org>,
  Andrew Jones <ajones@ventanamicro.com>
Precedence: list
Reply-To: Sean Christopherson <seanjc@google.com>
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
Sender: "kvm-riscv" <kvm-riscv-bounces@lists.infradead.org>
Errors-To: kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org

Series

KVM: Stop grabbing references to PFNMAP'd pages | expand

Commit Message

Sean Christopherson Oct. 10, 2024, 6:23 p.m. UTC

Track refcounted struct page memory using kvm_follow_pfn.refcounted_page
instead of relying on kvm_release_pfn_clean() to correctly detect that the
pfn is associated with a struct page.

Tested-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 virt/kvm/pfncache.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c
index 067daf9ad6ef..728d2c1b488a 100644
--- a/virt/kvm/pfncache.c
+++ b/virt/kvm/pfncache.c
@@ -159,11 +159,14 @@  static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc)
 	kvm_pfn_t new_pfn = KVM_PFN_ERR_FAULT;
 	void *new_khva = NULL;
 	unsigned long mmu_seq;
+	struct page *page;
+
 	struct kvm_follow_pfn kfp = {
 		.slot = gpc->memslot,
 		.gfn = gpa_to_gfn(gpc->gpa),
 		.flags = FOLL_WRITE,
 		.hva = gpc->uhva,
+		.refcounted_page = &page,
 	};
 
 	lockdep_assert_held(&gpc->refresh_lock);
@@ -198,7 +201,7 @@  static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc)
 			if (new_khva != old_khva)
 				gpc_unmap(new_pfn, new_khva);
 
-			kvm_release_pfn_clean(new_pfn);
+			kvm_release_page_unused(page);
 
 			cond_resched();
 		}
@@ -218,7 +221,7 @@  static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc)
 			new_khva = gpc_map(new_pfn);
 
 		if (!new_khva) {
-			kvm_release_pfn_clean(new_pfn);
+			kvm_release_page_unused(page);
 			goto out_error;
 		}
 
@@ -236,11 +239,11 @@  static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc)
 	gpc->khva = new_khva + offset_in_page(gpc->uhva);
 
 	/*
-	 * Put the reference to the _new_ pfn.  The pfn is now tracked by the
+	 * Put the reference to the _new_ page.  The page is now tracked by the
 	 * cache and can be safely migrated, swapped, etc... as the cache will
 	 * invalidate any mappings in response to relevant mmu_notifier events.
 	 */
-	kvm_release_pfn_clean(new_pfn);
+	kvm_release_page_clean(page);
 
 	return 0;

[v13,29/85] KVM: pfncache: Precisely track refcounted pages

Commit Message

Patch