From patchwork Thu Jun 3 16:43:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 1487353 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=oZWknSme; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4FwsD971BGz9sWQ for ; Fri, 4 Jun 2021 02:43:53 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229695AbhFCQph (ORCPT ); Thu, 3 Jun 2021 12:45:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45050 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229762AbhFCQph (ORCPT ); Thu, 3 Jun 2021 12:45:37 -0400 Received: from mail-pg1-x52f.google.com (mail-pg1-x52f.google.com [IPv6:2607:f8b0:4864:20::52f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9DF88C061756 for ; Thu, 3 Jun 2021 09:43:52 -0700 (PDT) Received: by mail-pg1-x52f.google.com with SMTP id r1so5581171pgk.8 for ; Thu, 03 Jun 2021 09:43:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=9Q2DATu13P1pzwQHWEVjGMuJXKz5nnidG53u8t/T69Y=; b=oZWknSmeVZZfZwya3xUfjHMkrSowYXFuz2O2dLlUa9GiJ/jfDEeFrwHMXwjBA4wEI+ KgJVTITaDxvaEi2nzAHfwWTEAk9ONAyP6YJm2+/EYC4K3inpb83m3E64GzZtZ9U1kDBl lZo5D5cvDzPWuhMzO9hZfc3PzZmWok9dwf5rg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=9Q2DATu13P1pzwQHWEVjGMuJXKz5nnidG53u8t/T69Y=; b=gsNAczI+77TN0X3XWiG2aUatlKMb8r9n84Cb7Oh2p8G3Kc6Y+xCZdDH7WsCPmi16cs ac/iSciZDD2uUnLYOA+Jl8XzQk922TyxazO2HanF+wbwspu0JQKsJsxRkSGQSk3/vhc9 rb5IB0Yu7PU8zBv8cYGwUmv+I6ZL008YSBVUq3Hp5gFPa76iEpTGA7JeEu2a2Yx9VEmL gFBIVkRSIOtQ84QcDRKhilSeCBdAdJdhrQx064q6n3fe7eACSLCewpvztBf9zBzkjqGw 16liOzr/qXDImxzcquMjmVTZ22oMhO1XRkRFEQApkSiCHfYRJZG6WxAqMZC0yvFNd3H8 BkyQ== X-Gm-Message-State: AOAM530jLJ8zFilv24m8BLO1E5BfpWtz43aXhJ2fbf3A1q6RHabswcQx fLcwV6he/Hf5mmrqb4xcGy3PsTE311lr1g== X-Google-Smtp-Source: ABdhPJx7I5PtkMSKl9b8zDas4WwIKw7SMCpW5CkmtGzDxyA7T/ENxk1ZY0pC15QcFzNPM4JDV5yvgQ== X-Received: by 2002:a63:dc4e:: with SMTP id f14mr368523pgj.378.1622738632157; Thu, 03 Jun 2021 09:43:52 -0700 (PDT) Received: from senozhatsky.flets-east.jp ([2409:10:2e40:5100:1585:76c1:e367:901a]) by smtp.gmail.com with ESMTPSA id 76sm2749589pfy.82.2021.06.03.09.43.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Jun 2021 09:43:51 -0700 (PDT) From: Sergey Senozhatsky To: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Jim Mattson Cc: Marc Zyngier , Huacai Chen , Paul Mackerras , Christian Borntraeger , Suleiman Souhlal , x86@kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-ppc@vger.kernel.org, linux-s390@vger.kernel.org, Sergey Senozhatsky Subject: [RFC][PATCH] kvm: add suspend pm-notifier Date: Fri, 4 Jun 2021 01:43:15 +0900 Message-Id: <20210603164315.682994-1-senozhatsky@chromium.org> X-Mailer: git-send-email 2.32.0.rc0.204.g9fa02ecfa5-goog MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org Add KVM suspend/hibernate PM-notifier which lets architectures to implement arch-specific VM suspend code. For instance, on x86 this sets PVCLOCK_GUEST_STOPPED on all the VCPUs. Our case is that user puts the host system into sleep multiple times a day (e.g. closes the laptop's lid) so we need a reliable way to suspend VMs properly. Signed-off-by: Sergey Senozhatsky --- arch/arm64/kvm/arm.c | 4 ++++ arch/mips/kvm/mips.c | 4 ++++ arch/powerpc/kvm/powerpc.c | 4 ++++ arch/s390/kvm/kvm-s390.c | 4 ++++ arch/x86/kvm/x86.c | 21 ++++++++++++++++++++ include/linux/kvm_host.h | 8 ++++++++ virt/kvm/kvm_main.c | 40 ++++++++++++++++++++++++++++++++++++++ 7 files changed, 85 insertions(+) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 1126eae27400..547dbe44d039 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1311,6 +1311,10 @@ static int kvm_vm_ioctl_set_device_addr(struct kvm *kvm, } } +void kvm_arch_pm_notifier(struct kvm *kvm) +{ +} + long kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) { diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c index 4d4af97dcc88..d4408acd2be6 100644 --- a/arch/mips/kvm/mips.c +++ b/arch/mips/kvm/mips.c @@ -980,6 +980,10 @@ void kvm_arch_flush_remote_tlbs_memslot(struct kvm *kvm, kvm_flush_remote_tlbs(kvm); } +void kvm_arch_pm_notifier(struct kvm *kvm) +{ +} + long kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) { long r; diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c index a2a68a958fa0..96e8a7b6fcf0 100644 --- a/arch/powerpc/kvm/powerpc.c +++ b/arch/powerpc/kvm/powerpc.c @@ -2334,6 +2334,10 @@ static int kvmppc_get_cpu_char(struct kvm_ppc_cpu_char *cp) } #endif +void kvm_arch_pm_notifier(struct kvm *kvm) +{ +} + long kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) { diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c index 1296fc10f80c..c5f86fc1e497 100644 --- a/arch/s390/kvm/kvm-s390.c +++ b/arch/s390/kvm/kvm-s390.c @@ -2367,6 +2367,10 @@ static int kvm_s390_handle_pv(struct kvm *kvm, struct kvm_pv_cmd *cmd) return r; } +void kvm_arch_pm_notifier(struct kvm *kvm) +{ +} + long kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) { diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index bbc4e04e67ad..3f3d6497593f 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -5613,6 +5613,27 @@ static int kvm_vm_ioctl_set_msr_filter(struct kvm *kvm, void __user *argp) return 0; } +void kvm_arch_pm_notifier(struct kvm *kvm) +{ +#ifdef CONFIG_PM + int c; + + mutex_lock(&kvm->lock); + for (c = 0; c < kvm->created_vcpus; c++) { + struct kvm_vcpu *vcpu = kvm->vcpus[c]; + int r; + + if (!vcpu) + continue; + r = kvm_set_guest_paused(vcpu); + if (!r) + continue; + pr_err("Failed to suspend VCPU-%d: %d\n", vcpu->vcpu_id, r); + } + mutex_unlock(&kvm->lock); +#endif +} + long kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) { diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 2f34487e21f2..86695320a6b7 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -37,6 +37,8 @@ #include #include +#include + #ifndef KVM_MAX_VCPU_ID #define KVM_MAX_VCPU_ID KVM_MAX_VCPUS #endif @@ -579,6 +581,10 @@ struct kvm { pid_t userspace_pid; unsigned int max_halt_poll_ns; u32 dirty_ring_size; + +#ifdef CONFIG_PM + struct notifier_block pm_notifier; +#endif }; #define kvm_err(fmt, ...) \ @@ -992,6 +998,8 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu); void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu); void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu); +void kvm_arch_pm_notifier(struct kvm *kvm); + #ifdef __KVM_HAVE_ARCH_VCPU_DEBUGFS void kvm_arch_create_vcpu_debugfs(struct kvm_vcpu *vcpu, struct dentry *debugfs_dentry); #endif diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 6b4feb92dc79..86925ab7d162 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -51,6 +51,7 @@ #include #include #include +#include #include #include @@ -779,6 +780,43 @@ static int kvm_init_mmu_notifier(struct kvm *kvm) #endif /* CONFIG_MMU_NOTIFIER && KVM_ARCH_WANT_MMU_NOTIFIER */ +#ifdef CONFIG_PM +static int kvm_pm_notifier_call(struct notifier_block *bl, + unsigned long state, + void *unused) +{ + struct kvm *kvm = container_of(bl, struct kvm, pm_notifier); + + switch (state) { + case PM_HIBERNATION_PREPARE: + case PM_SUSPEND_PREPARE: + kvm_arch_pm_notifier(kvm); + break; + } + return NOTIFY_DONE; +} + +static void kvm_init_pm_notifier(struct kvm *kvm) +{ + kvm->pm_notifier.notifier_call = kvm_pm_notifier_call; + kvm->pm_notifier.priority = INT_MAX; + register_pm_notifier(&kvm->pm_notifier); +} + +static void kvm_destroy_pm_notifier(struct kvm *kvm) +{ + unregister_pm_notifier(&kvm->pm_notifier); +} +#else +static void kvm_init_pm_notifier(struct kvm *kvm) +{ +} + +static void kvm_destroy_pm_notifier(struct kvm *kvm) +{ +} +#endif /* CONFIG_PM */ + static struct kvm_memslots *kvm_alloc_memslots(void) { int i; @@ -962,6 +1000,7 @@ static struct kvm *kvm_create_vm(unsigned long type) mutex_unlock(&kvm_lock); preempt_notifier_inc(); + kvm_init_pm_notifier(kvm); return kvm; @@ -1009,6 +1048,7 @@ static void kvm_destroy_vm(struct kvm *kvm) int i; struct mm_struct *mm = kvm->mm; + kvm_destroy_pm_notifier(kvm); kvm_uevent_notify_change(KVM_EVENT_DESTROY_VM, kvm); kvm_destroy_vm_debugfs(kvm); kvm_arch_sync_events(kvm);