From patchwork Thu May 20 13:41:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Righi X-Patchwork-Id: 1481644 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Fm9rN26fpz9sfG; Thu, 20 May 2021 23:41:40 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1ljivg-0002YL-NT; Thu, 20 May 2021 13:41:36 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1ljive-0002XQ-6e for kernel-team@lists.ubuntu.com; Thu, 20 May 2021 13:41:34 +0000 Received: from mail-ed1-f69.google.com ([209.85.208.69]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1ljivd-0000rN-TU for kernel-team@lists.ubuntu.com; Thu, 20 May 2021 13:41:33 +0000 Received: by mail-ed1-f69.google.com with SMTP id ba15-20020a0564021acfb029038d3b33d7ffso6896400edb.23 for ; Thu, 20 May 2021 06:41:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mQCXrPkZtKhT0P2FJujFVOle3QSn2BUbGwn7wswzipw=; b=K0pFRv3Me40kGh3SIrKosN28Nf9KLDcYlRfq/i+srdcPm4K5tBylcAxgHmXqP2DmYf 7lTWygW/uEp4X3JEKT2nLLvI6AkVu/kJx4yZ69dzDCYdUOoA7jqpPCTMmV7NgdI8O16r 3ZaA+AYFnfuSvI1d2Gbz7T/prvzH1BtpC+5i0U4LW0e9o+O9Pxlm+Hh2nzflvQSHVvbu er9My7x/fKC6dK8ALLoMzTt/5dU3fCpdE9jwWqa2t/SDWungXSeAqM6rMjbC+dIYcffC MGyioWSJA7DIECDieZqp3GTLGmsvfGDAG8a7KLQ95M8NotBtRxUKpoTt4VVSK8HlqXuS rZIA== X-Gm-Message-State: AOAM532Vvo1yYm5Ac7HPWH5fgYD+QAp0Vpubza11PxZfFcklgcZ7Sang 0Es2Q71M3uMXekDpmRquRr/6cyiaqlwsl+B2lsaOHgBjiocQgzHCpZHhbAoSHn+dIYXyMhUQq+l pGtt3YBcO7x0ltgWHthrgTjXWKBG22YcwAVxus7pAwA== X-Received: by 2002:aa7:cfcd:: with SMTP id r13mr5148931edy.177.1621518093625; Thu, 20 May 2021 06:41:33 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyRaIGG4gNqoowN5l9tmCDMnLMCWhRxoZFvwogdVbDHoKOgF+dB9XplD+B0HGoRUDfIVfwaFw== X-Received: by 2002:aa7:cfcd:: with SMTP id r13mr5148916edy.177.1621518093428; Thu, 20 May 2021 06:41:33 -0700 (PDT) Received: from xps-13-7390.homenet.telecomitalia.it (host-87-19-3-42.retail.telecomitalia.it. [87.19.3.42]) by smtp.gmail.com with ESMTPSA id l6sm1417576ejc.92.2021.05.20.06.41.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 06:41:33 -0700 (PDT) From: Andrea Righi To: kernel-team@lists.ubuntu.com Subject: [SRU][H][PATCH 1/5] x86/kvm: Fix pr_info() for async PF setup/teardown Date: Thu, 20 May 2021 15:41:26 +0200 Message-Id: <20210520134130.40105-2-andrea.righi@canonical.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520134130.40105-1-andrea.righi@canonical.com> References: <20210520134130.40105-1-andrea.righi@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Vitaly Kuznetsov 'pr_fmt' already has 'kvm-guest: ' so 'KVM' prefix is redundant. "Unregister pv shared memory" is very ambiguous, it's hard to say which particular PV feature it relates to. Signed-off-by: Vitaly Kuznetsov Message-Id: <20210414123544.1060604-2-vkuznets@redhat.com> Signed-off-by: Paolo Bonzini (cherry picked from commit 0a269a008f837e76ce285679ab3005059fadc2a6) Signed-off-by: Andrea Righi --- arch/x86/kernel/kvm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 5e78e01ca3b4..2047e751e15f 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -345,7 +345,7 @@ static void kvm_guest_cpu_init(void) wrmsrl(MSR_KVM_ASYNC_PF_EN, pa); __this_cpu_write(apf_reason.enabled, 1); - pr_info("KVM setup async PF for cpu %d\n", smp_processor_id()); + pr_info("setup async PF for cpu %d\n", smp_processor_id()); } if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) { @@ -371,7 +371,7 @@ static void kvm_pv_disable_apf(void) wrmsrl(MSR_KVM_ASYNC_PF_EN, 0); __this_cpu_write(apf_reason.enabled, 0); - pr_info("Unregister pv shared memory for cpu %d\n", smp_processor_id()); + pr_info("disable async PF for cpu %d\n", smp_processor_id()); } static void kvm_pv_guest_cpu_reboot(void *unused) From patchwork Thu May 20 13:41:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Righi X-Patchwork-Id: 1481646 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Fm9rP3DGrz9sj5; Thu, 20 May 2021 23:41:41 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1ljivg-0002Yp-Uu; Thu, 20 May 2021 13:41:36 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1ljive-0002Xj-TI for kernel-team@lists.ubuntu.com; Thu, 20 May 2021 13:41:34 +0000 Received: from mail-ej1-f70.google.com ([209.85.218.70]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1ljive-0000s9-KF for kernel-team@lists.ubuntu.com; Thu, 20 May 2021 13:41:34 +0000 Received: by mail-ej1-f70.google.com with SMTP id 16-20020a1709063010b029037417ca2d43so5024511ejz.5 for ; Thu, 20 May 2021 06:41:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TzkWuiuoSDk6Cpyg01VGhgr4fWi8PguXY1aqoIxxemw=; b=Tlkec4Z3Q5WDAvMHa1kRVoTPazYmCUFM709V/PFqFsuP3FC+OBPJ8L/fXQGs9YDJjg 9+mt6a7RDxfQwXh6vC0W0iEhyEp4a0k1xD1dU2uN0KuNpnq2/jB++i+g+5+zTRX9xzsm fQ+ihPMv2nb03+jLoT7QTJZkXdIvsFQ/KnxjcUW8s4JqjELiMCn1SqFzjanhzwuzbKrB 93eAPqvWx/U1hT57MModeli0GyWyCkzz112FtIflpVONHKndjmU1VAwfMBA+s72o83/J 0k9AO0gGjxMFHOGDlQr3zmFBONBjuYLaAyYci99Z4myv34pninMNBuasQi2ULenZ37R9 rSzw== X-Gm-Message-State: AOAM5311Ixcn5F2BfuMaAdvf81EUwLQr8qbZSVxlgb6EC52I1t+wKlqL /BXAuC8G0qCxeUaYQlVqWMuFQG/ZarhZuyOtprfQBV98a+KtjZO6Wx1KJJ7pzfVj0+zpB6l9VY8 uREKkc+XttolPzBauP4xx1yqjaGKzDB8ZrTMItQC6Uw== X-Received: by 2002:a17:906:a245:: with SMTP id bi5mr4983289ejb.316.1621518094319; Thu, 20 May 2021 06:41:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw5IHBVkyv3HYhfaumS+RgFCTM+GuX7O+yDuOyJ0LBoeciUHwPQ7vNqUPCNLpHuJ94ehXKhAg== X-Received: by 2002:a17:906:a245:: with SMTP id bi5mr4983275ejb.316.1621518094077; Thu, 20 May 2021 06:41:34 -0700 (PDT) Received: from xps-13-7390.homenet.telecomitalia.it (host-87-19-3-42.retail.telecomitalia.it. [87.19.3.42]) by smtp.gmail.com with ESMTPSA id l6sm1417576ejc.92.2021.05.20.06.41.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 06:41:33 -0700 (PDT) From: Andrea Righi To: kernel-team@lists.ubuntu.com Subject: [SRU][H][PATCH 2/5] x86/kvm: Teardown PV features on boot CPU as well Date: Thu, 20 May 2021 15:41:27 +0200 Message-Id: <20210520134130.40105-3-andrea.righi@canonical.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520134130.40105-1-andrea.righi@canonical.com> References: <20210520134130.40105-1-andrea.righi@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Vitaly Kuznetsov Various PV features (Async PF, PV EOI, steal time) work through memory shared with hypervisor and when we restore from hibernation we must properly teardown all these features to make sure hypervisor doesn't write to stale locations after we jump to the previously hibernated kernel (which can try to place anything there). For secondary CPUs the job is already done by kvm_cpu_down_prepare(), register syscore ops to do the same for boot CPU. Signed-off-by: Vitaly Kuznetsov Message-Id: <20210414123544.1060604-3-vkuznets@redhat.com> Signed-off-by: Paolo Bonzini (backported from commit 8b79feffeca28c5459458fe78676b081e87c93a4) Signed-off-by: Andrea Righi --- arch/x86/kernel/kvm.c | 57 +++++++++++++++++++++++++++++++------------ 1 file changed, 41 insertions(+), 16 deletions(-) diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 2047e751e15f..d2448c1831e0 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -26,6 +26,7 @@ #include #include #include +#include #include #include #include @@ -451,6 +452,25 @@ static void __init sev_map_percpu_data(void) } } +static void kvm_guest_cpu_offline(void) +{ + kvm_disable_steal_time(); + if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) + wrmsrl(MSR_KVM_PV_EOI_EN, 0); + kvm_pv_disable_apf(); + apf_task_wake_all(); +} + +static int kvm_cpu_online(unsigned int cpu) +{ + unsigned long flags; + + local_irq_save(flags); + kvm_guest_cpu_init(); + local_irq_restore(flags); + return 0; +} + static bool pv_tlb_flush_supported(void) { return (kvm_para_has_feature(KVM_FEATURE_PV_TLB_FLUSH) && @@ -587,31 +607,34 @@ static void __init kvm_smp_prepare_boot_cpu(void) kvm_spinlock_init(); } -static void kvm_guest_cpu_offline(void) +static int kvm_cpu_down_prepare(unsigned int cpu) { - kvm_disable_steal_time(); - if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) - wrmsrl(MSR_KVM_PV_EOI_EN, 0); - kvm_pv_disable_apf(); - apf_task_wake_all(); -} + unsigned long flags; -static int kvm_cpu_online(unsigned int cpu) -{ - local_irq_disable(); - kvm_guest_cpu_init(); - local_irq_enable(); + local_irq_save(flags); + kvm_guest_cpu_offline(); + local_irq_restore(flags); return 0; } -static int kvm_cpu_down_prepare(unsigned int cpu) +#endif + +static int kvm_suspend(void) { - local_irq_disable(); kvm_guest_cpu_offline(); - local_irq_enable(); + return 0; } -#endif + +static void kvm_resume(void) +{ + kvm_cpu_online(raw_smp_processor_id()); +} + +static struct syscore_ops kvm_syscore_ops = { + .suspend = kvm_suspend, + .resume = kvm_resume, +}; static void kvm_flush_tlb_others(const struct cpumask *cpumask, const struct flush_tlb_info *info) @@ -681,6 +704,8 @@ static void __init kvm_guest_init(void) kvm_guest_cpu_init(); #endif + register_syscore_ops(&kvm_syscore_ops); + /* * Hard lockup detection is enabled by default. Disable it, as guests * can get false positives too easily, for example if the host is From patchwork Thu May 20 13:41:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Righi X-Patchwork-Id: 1481645 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Fm9rR4L6Xz9sX2; Thu, 20 May 2021 23:41:43 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1ljivi-0002Zj-Ej; Thu, 20 May 2021 13:41:38 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1ljivf-0002Xz-EQ for kernel-team@lists.ubuntu.com; Thu, 20 May 2021 13:41:35 +0000 Received: from mail-ed1-f71.google.com ([209.85.208.71]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1ljivf-0000sT-5b for kernel-team@lists.ubuntu.com; Thu, 20 May 2021 13:41:35 +0000 Received: by mail-ed1-f71.google.com with SMTP id s20-20020a0564025214b029038752a2d8f3so9531204edd.2 for ; Thu, 20 May 2021 06:41:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Z0J+6eLeMi2l/oWFG9q6t5PpcnSZSubNtTvV86mjow8=; b=ZXDXiiS8ydemQecEFa0MDMDWYNO9sMwJQIcB1y8STyYxOVrGGGf39zBW53W+hz6JXJ 9QVtT4igSdcA/psgjtvm0ZMaD1mQouVrnjXRp0eN0ktHtxV2awM8TGzx1A5ObMTBTxb4 7rSPqoTWpUvWxrcS7/FC/mkOIDZoRnQOF3LZiLPaVb4nScEs/00EntqM0YiOmigFr4Mo ZYiWuslBJyLsYtkz7y/+3mzIRtuJJz1m0Yk6muhoNtlNifYcMn8rpIurd7VpL8TVsZTF FLF7PZsLdWD6LBcammirtVrRhaXhNeJZjaWVtUldRxd+aTnZ8yVD4Yf0vpwX9DVVhJlr vS+A== X-Gm-Message-State: AOAM533Fbyih1MmGpzlphQGnPu4eS072VKdWuKY3rTROu+L1JURursEA GlA5BlQt/aa8U5clZ5/M4jYRlNnqAh3rcRuhWw9owOjmg3pe/mft6TspyfxALdoM/7tmykbUtqW MBrlVrNSosM0UsGWY3tUq7Iv5IJHk9WyEajZEroaxow== X-Received: by 2002:a17:907:c1f:: with SMTP id ga31mr4737662ejc.534.1621518094863; Thu, 20 May 2021 06:41:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxoL5uHymPZR1QL+FGVDQoXxWCy8fzd6VWkU8O4oqQvXLN/k97vCTkSumEzr5dQZnlel9ZmYg== X-Received: by 2002:a17:907:c1f:: with SMTP id ga31mr4737647ejc.534.1621518094682; Thu, 20 May 2021 06:41:34 -0700 (PDT) Received: from xps-13-7390.homenet.telecomitalia.it (host-87-19-3-42.retail.telecomitalia.it. [87.19.3.42]) by smtp.gmail.com with ESMTPSA id l6sm1417576ejc.92.2021.05.20.06.41.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 06:41:34 -0700 (PDT) From: Andrea Righi To: kernel-team@lists.ubuntu.com Subject: [SRU][H][PATCH 3/5] x86/kvm: Disable kvmclock on all CPUs on shutdown Date: Thu, 20 May 2021 15:41:28 +0200 Message-Id: <20210520134130.40105-4-andrea.righi@canonical.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520134130.40105-1-andrea.righi@canonical.com> References: <20210520134130.40105-1-andrea.righi@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Vitaly Kuznetsov Currenly, we disable kvmclock from machine_shutdown() hook and this only happens for boot CPU. We need to disable it for all CPUs to guard against memory corruption e.g. on restore from hibernate. Note, writing '0' to kvmclock MSR doesn't clear memory location, it just prevents hypervisor from updating the location so for the short while after write and while CPU is still alive, the clock remains usable and correct so we don't need to switch to some other clocksource. Signed-off-by: Vitaly Kuznetsov Message-Id: <20210414123544.1060604-4-vkuznets@redhat.com> Signed-off-by: Paolo Bonzini (cherry picked from commit c02027b5742b5aa804ef08a4a9db433295533046) Signed-off-by: Andrea Righi --- arch/x86/include/asm/kvm_para.h | 4 ++-- arch/x86/kernel/kvm.c | 1 + arch/x86/kernel/kvmclock.c | 5 +---- 3 files changed, 4 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h index 338119852512..9c56e0defd45 100644 --- a/arch/x86/include/asm/kvm_para.h +++ b/arch/x86/include/asm/kvm_para.h @@ -7,8 +7,6 @@ #include #include -extern void kvmclock_init(void); - #ifdef CONFIG_KVM_GUEST bool kvm_check_and_clear_guest_paused(void); #else @@ -86,6 +84,8 @@ static inline long kvm_hypercall4(unsigned int nr, unsigned long p1, } #ifdef CONFIG_KVM_GUEST +void kvmclock_init(void); +void kvmclock_disable(void); bool kvm_para_available(void); unsigned int kvm_arch_para_features(void); unsigned int kvm_arch_para_hints(void); diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index d2448c1831e0..464eb2a13c89 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -459,6 +459,7 @@ static void kvm_guest_cpu_offline(void) wrmsrl(MSR_KVM_PV_EOI_EN, 0); kvm_pv_disable_apf(); apf_task_wake_all(); + kvmclock_disable(); } static int kvm_cpu_online(unsigned int cpu) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 1fc0962c89c0..cf869de98eec 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -220,11 +220,9 @@ static void kvm_crash_shutdown(struct pt_regs *regs) } #endif -static void kvm_shutdown(void) +void kvmclock_disable(void) { native_write_msr(msr_kvm_system_time, 0, 0); - kvm_disable_steal_time(); - native_machine_shutdown(); } static void __init kvmclock_init_mem(void) @@ -351,7 +349,6 @@ void __init kvmclock_init(void) #endif x86_platform.save_sched_clock_state = kvm_save_sched_clock_state; x86_platform.restore_sched_clock_state = kvm_restore_sched_clock_state; - machine_ops.shutdown = kvm_shutdown; #ifdef CONFIG_KEXEC_CORE machine_ops.crash_shutdown = kvm_crash_shutdown; #endif From patchwork Thu May 20 13:41:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Righi X-Patchwork-Id: 1481647 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Fm9rT5PWKz9srX; Thu, 20 May 2021 23:41:43 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1ljivi-0002Zx-ND; Thu, 20 May 2021 13:41:38 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1ljivg-0002YA-8x for kernel-team@lists.ubuntu.com; Thu, 20 May 2021 13:41:36 +0000 Received: from mail-ed1-f69.google.com ([209.85.208.69]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1ljivg-0000si-0X for kernel-team@lists.ubuntu.com; Thu, 20 May 2021 13:41:36 +0000 Received: by mail-ed1-f69.google.com with SMTP id x3-20020a50ba830000b029038caed0dd2eso9570489ede.7 for ; Thu, 20 May 2021 06:41:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FoA8oNF8q+Ov8/uB0OSvTWFmix7Ippy2Y36qWcQKbbQ=; b=OI8yy7VfOQhcCTX93AJYwckzw3oZtfA9D1B5e/mZz/ISZA4nYyAyM1plQOYnsdjypL qWuaf9pLKsUAm67/8gbyhG0XcDLuC4O6lEa4gscaBmXAa7GJhGretLGMQ2aUDYYH03vs mulEiuvyuVK1WfCf2yC9w6GcujSNj2QmKktnicIjFVkEi/eUz3T8JZzqQVhLiaPwRFa6 esyCSk/GniRaBfl5+J24KNaezLjOQcnfYlhnm0iUzmqhMBMlfpYirzxyH2VcfevFzRlG ILCFOoXLWfQBgl0sgtIFmbqXvsxFqzxJ2qmKuRzH3mYeqRo99sHbTjpyUMO9jewGjBEY unuw== X-Gm-Message-State: AOAM531L71cGqL321SuQF3QaqYazvOZO4UIJxhMH6FsaTqjulfA6E381 J3EWovzIxYFCc8HF1DquOkwJJfKIzyjTB8iWAdKSAwFWsGcE6HiY1dZKicxLb49SDZAtzVdeN1i AMRbM2TDEG+DAqdqJtovaTSD4E1uWr7nqiDd7Wqa2ZQ== X-Received: by 2002:a17:906:79c8:: with SMTP id m8mr4889539ejo.260.1621518095603; Thu, 20 May 2021 06:41:35 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxu9siKmYZjnVULMNzCYZ4bXFaKqssf0b+PrxVmG2Rid6PU8BzZtfN5hoc9SkkN8QnT6/KZ8w== X-Received: by 2002:a17:906:79c8:: with SMTP id m8mr4889509ejo.260.1621518095273; Thu, 20 May 2021 06:41:35 -0700 (PDT) Received: from xps-13-7390.homenet.telecomitalia.it (host-87-19-3-42.retail.telecomitalia.it. [87.19.3.42]) by smtp.gmail.com with ESMTPSA id l6sm1417576ejc.92.2021.05.20.06.41.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 06:41:35 -0700 (PDT) From: Andrea Righi To: kernel-team@lists.ubuntu.com Subject: [SRU][H][PATCH 4/5] x86/kvm: Disable all PV features on crash Date: Thu, 20 May 2021 15:41:29 +0200 Message-Id: <20210520134130.40105-5-andrea.righi@canonical.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520134130.40105-1-andrea.righi@canonical.com> References: <20210520134130.40105-1-andrea.righi@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Vitaly Kuznetsov Crash shutdown handler only disables kvmclock and steal time, other PV features remain active so we risk corrupting memory or getting some side-effects in kdump kernel. Move crash handler to kvm.c and unify with CPU offline. Signed-off-by: Vitaly Kuznetsov Message-Id: <20210414123544.1060604-5-vkuznets@redhat.com> Signed-off-by: Paolo Bonzini (backported from commit 3d6b84132d2a57b5a74100f6923a8feb679ac2ce) Signed-off-by: Andrea Righi --- arch/x86/include/asm/kvm_para.h | 6 ----- arch/x86/kernel/kvm.c | 44 ++++++++++++++++++++++++--------- arch/x86/kernel/kvmclock.c | 21 ---------------- 3 files changed, 32 insertions(+), 39 deletions(-) diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h index 9c56e0defd45..69299878b200 100644 --- a/arch/x86/include/asm/kvm_para.h +++ b/arch/x86/include/asm/kvm_para.h @@ -92,7 +92,6 @@ unsigned int kvm_arch_para_hints(void); void kvm_async_pf_task_wait_schedule(u32 token); void kvm_async_pf_task_wake(u32 token); u32 kvm_read_and_reset_apf_flags(void); -void kvm_disable_steal_time(void); bool __kvm_handle_async_pf(struct pt_regs *regs, u32 token); DECLARE_STATIC_KEY_FALSE(kvm_async_pf_enabled); @@ -137,11 +136,6 @@ static inline u32 kvm_read_and_reset_apf_flags(void) return 0; } -static inline void kvm_disable_steal_time(void) -{ - return; -} - static __always_inline bool kvm_handle_async_pf(struct pt_regs *regs, u32 token) { return false; diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 464eb2a13c89..147281852ca9 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -38,6 +38,7 @@ #include #include #include +#include #include DEFINE_STATIC_KEY_FALSE(kvm_async_pf_enabled); @@ -375,6 +376,14 @@ static void kvm_pv_disable_apf(void) pr_info("disable async PF for cpu %d\n", smp_processor_id()); } +static void kvm_disable_steal_time(void) +{ + if (!has_steal_clock) + return; + + wrmsr(MSR_KVM_STEAL_TIME, 0, 0); +} + static void kvm_pv_guest_cpu_reboot(void *unused) { /* @@ -417,14 +426,6 @@ static u64 kvm_steal_clock(int cpu) return steal; } -void kvm_disable_steal_time(void) -{ - if (!has_steal_clock) - return; - - wrmsr(MSR_KVM_STEAL_TIME, 0, 0); -} - static inline void __set_percpu_decrypted(void *ptr, unsigned long size) { early_set_memory_decrypted((unsigned long) ptr, size); @@ -452,13 +453,14 @@ static void __init sev_map_percpu_data(void) } } -static void kvm_guest_cpu_offline(void) +static void kvm_guest_cpu_offline(bool shutdown) { kvm_disable_steal_time(); if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) wrmsrl(MSR_KVM_PV_EOI_EN, 0); kvm_pv_disable_apf(); - apf_task_wake_all(); + if (!shutdown) + apf_task_wake_all(); kvmclock_disable(); } @@ -613,7 +615,7 @@ static int kvm_cpu_down_prepare(unsigned int cpu) unsigned long flags; local_irq_save(flags); - kvm_guest_cpu_offline(); + kvm_guest_cpu_offline(false); local_irq_restore(flags); return 0; } @@ -622,7 +624,7 @@ static int kvm_cpu_down_prepare(unsigned int cpu) static int kvm_suspend(void) { - kvm_guest_cpu_offline(); + kvm_guest_cpu_offline(false); return 0; } @@ -663,6 +665,20 @@ static void kvm_flush_tlb_others(const struct cpumask *cpumask, native_flush_tlb_others(flushmask, info); } +/* + * After a PV feature is registered, the host will keep writing to the + * registered memory location. If the guest happens to shutdown, this memory + * won't be valid. In cases like kexec, in which you install a new kernel, this + * means a random memory location will be kept being written. + */ +#ifdef CONFIG_KEXEC_CORE +static void kvm_crash_shutdown(struct pt_regs *regs) +{ + kvm_guest_cpu_offline(true); + native_machine_crash_shutdown(regs); +} +#endif + static void __init kvm_guest_init(void) { int i; @@ -705,6 +721,10 @@ static void __init kvm_guest_init(void) kvm_guest_cpu_init(); #endif +#ifdef CONFIG_KEXEC_CORE + machine_ops.crash_shutdown = kvm_crash_shutdown; +#endif + register_syscore_ops(&kvm_syscore_ops); /* diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index cf869de98eec..b825c87c12ef 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -20,7 +20,6 @@ #include #include #include -#include #include static int kvmclock __initdata = 1; @@ -203,23 +202,6 @@ static void kvm_setup_secondary_clock(void) } #endif -/* - * After the clock is registered, the host will keep writing to the - * registered memory location. If the guest happens to shutdown, this memory - * won't be valid. In cases like kexec, in which you install a new kernel, this - * means a random memory location will be kept being written. So before any - * kind of shutdown from our side, we unregister the clock by writing anything - * that does not have the 'enable' bit set in the msr - */ -#ifdef CONFIG_KEXEC_CORE -static void kvm_crash_shutdown(struct pt_regs *regs) -{ - native_write_msr(msr_kvm_system_time, 0, 0); - kvm_disable_steal_time(); - native_machine_crash_shutdown(regs); -} -#endif - void kvmclock_disable(void) { native_write_msr(msr_kvm_system_time, 0, 0); @@ -349,9 +331,6 @@ void __init kvmclock_init(void) #endif x86_platform.save_sched_clock_state = kvm_save_sched_clock_state; x86_platform.restore_sched_clock_state = kvm_restore_sched_clock_state; -#ifdef CONFIG_KEXEC_CORE - machine_ops.crash_shutdown = kvm_crash_shutdown; -#endif kvm_get_preset_lpj(); /* From patchwork Thu May 20 13:41:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Righi X-Patchwork-Id: 1481648 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Fm9rV5pRbz9ssP; Thu, 20 May 2021 23:41:46 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1ljivk-0002bh-OJ; Thu, 20 May 2021 13:41:40 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1ljivg-0002YK-Nf for kernel-team@lists.ubuntu.com; Thu, 20 May 2021 13:41:36 +0000 Received: from mail-ej1-f71.google.com ([209.85.218.71]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1ljivg-0000sr-G1 for kernel-team@lists.ubuntu.com; Thu, 20 May 2021 13:41:36 +0000 Received: by mail-ej1-f71.google.com with SMTP id bi3-20020a170906a243b02903933c4d9132so5035593ejb.11 for ; Thu, 20 May 2021 06:41:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=v94kv0VaKIqgUGzoyATIUfzXHLgVrJzn/O/ymf2kgqs=; b=UU3gkZUcFM4B4Ykn7M0vW8uVW00ymJlcoTalmXXemQKUTfKRDwQiX/uGgSwxg1cqe0 CzJnrtw6ThgnMmNgyWfaxIZtzOVwe4XOtvT5kCT3GMcCcFL8MZMyJFE6lfH7eZU7f4WF SzRcDGl31ZS9OWzi9rt6u/5eRvLJZBQFVIxA9sXXt6C0Rmo1q0Aw2bYoigQ8WiYkxzR3 obf+OEzVR/93abZ8XBCzk+c46GKq8CftRr3/idyR5u2SaK8vE9UpNSGKz2n/4gx++t9H CWS3W09wdecAjyKUFmkuxYDJdpMpNAFHChUNqHDly09nBcWi1c4SDKuD6UfbQqFaPZgc 5CAA== X-Gm-Message-State: AOAM531H/AhATLBfWlghwGqeHRLOR8Ormimnyu2a+xW4wAk03tMVb/Ps ZnC1OtiYTCsf8YKzylEtZMaUfUmSlhLO6F6mEwrRH7p7ZCGBkj7m0n+ZhVYDzWdZ20QApKXUx1G PuDNtJh25wQFWVLHbwqn41EjAi4wtS/jtu/PHlw6BBQ== X-Received: by 2002:a17:906:7c9:: with SMTP id m9mr4715491ejc.98.1621518096197; Thu, 20 May 2021 06:41:36 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx+fLdB3v7IequYTHePGwT8E1P4SKWSn3vojYD+cNwUrbZy6euCQLkO20XDh3BtNUUCbuan3A== X-Received: by 2002:a17:906:7c9:: with SMTP id m9mr4715472ejc.98.1621518095895; Thu, 20 May 2021 06:41:35 -0700 (PDT) Received: from xps-13-7390.homenet.telecomitalia.it (host-87-19-3-42.retail.telecomitalia.it. [87.19.3.42]) by smtp.gmail.com with ESMTPSA id l6sm1417576ejc.92.2021.05.20.06.41.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 06:41:35 -0700 (PDT) From: Andrea Righi To: kernel-team@lists.ubuntu.com Subject: [SRU][H][PATCH 5/5] x86/kvm: Unify kvm_pv_guest_cpu_reboot() with kvm_guest_cpu_offline() Date: Thu, 20 May 2021 15:41:30 +0200 Message-Id: <20210520134130.40105-6-andrea.righi@canonical.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520134130.40105-1-andrea.righi@canonical.com> References: <20210520134130.40105-1-andrea.righi@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Vitaly Kuznetsov Simplify the code by making PV features shutdown happen in one place. Signed-off-by: Vitaly Kuznetsov Message-Id: <20210414123544.1060604-6-vkuznets@redhat.com> Signed-off-by: Paolo Bonzini (backported from commit 384fc672f528d3b84eacd9a86ecf35df3363b8ba) Signed-off-by: Andrea Righi --- arch/x86/kernel/kvm.c | 42 +++++++++++++++++------------------------- 1 file changed, 17 insertions(+), 25 deletions(-) diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 147281852ca9..478e43c55cb3 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -384,31 +384,6 @@ static void kvm_disable_steal_time(void) wrmsr(MSR_KVM_STEAL_TIME, 0, 0); } -static void kvm_pv_guest_cpu_reboot(void *unused) -{ - /* - * We disable PV EOI before we load a new kernel by kexec, - * since MSR_KVM_PV_EOI_EN stores a pointer into old kernel's memory. - * New kernel can re-enable when it boots. - */ - if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) - wrmsrl(MSR_KVM_PV_EOI_EN, 0); - kvm_pv_disable_apf(); - kvm_disable_steal_time(); -} - -static int kvm_pv_reboot_notify(struct notifier_block *nb, - unsigned long code, void *unused) -{ - if (code == SYS_RESTART) - on_each_cpu(kvm_pv_guest_cpu_reboot, NULL, 1); - return NOTIFY_DONE; -} - -static struct notifier_block kvm_pv_reboot_nb = { - .notifier_call = kvm_pv_reboot_notify, -}; - static u64 kvm_steal_clock(int cpu) { u64 steal; @@ -665,6 +640,23 @@ static void kvm_flush_tlb_others(const struct cpumask *cpumask, native_flush_tlb_others(flushmask, info); } +static void kvm_pv_guest_cpu_reboot(void *unused) +{ + kvm_guest_cpu_offline(true); +} + +static int kvm_pv_reboot_notify(struct notifier_block *nb, + unsigned long code, void *unused) +{ + if (code == SYS_RESTART) + on_each_cpu(kvm_pv_guest_cpu_reboot, NULL, 1); + return NOTIFY_DONE; +} + +static struct notifier_block kvm_pv_reboot_nb = { + .notifier_call = kvm_pv_reboot_notify, +}; + /* * After a PV feature is registered, the host will keep writing to the * registered memory location. If the guest happens to shutdown, this memory