From patchwork Thu Jul 15 13:15:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1505687 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=NG2gZrOR; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4GQZcK0V31z9sX2 for ; Thu, 15 Jul 2021 23:15:29 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236799AbhGONSV (ORCPT ); Thu, 15 Jul 2021 09:18:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35988 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232518AbhGONSV (ORCPT ); Thu, 15 Jul 2021 09:18:21 -0400 Received: from mail-pf1-x42e.google.com (mail-pf1-x42e.google.com [IPv6:2607:f8b0:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1F753C06175F for ; Thu, 15 Jul 2021 06:15:28 -0700 (PDT) Received: by mail-pf1-x42e.google.com with SMTP id y4so5194366pfi.9 for ; Thu, 15 Jul 2021 06:15:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Dg+kECwAuBShufULtVDvhwFWlxHBpA5vuIilKUWCHKw=; b=NG2gZrORvpOx1ukjrrzSAx3vIGo65+AokhbSqsqCW+ATo/Fc3r3a8483nr08C+CKNm T4heEkQjD+3daIkQKXPGMMSFXPdBy/aIfzuBbeKRg2h9bRzXKe1zGJNSWL3Mne/fEEBu HApJ0QcRVqS9vf52yr2VJrTsJChSvAh1wFwRt6BYBN8EoqjEAWfEiCFU7/TkZT6UgCR/ xaAcjH7QATyIohzEZKdM1arlHoh3CKrCiNr32iEf+j/a4khT6qgfp/nHv/IE+ZTdVGc0 FGOx+Y8ryfIlrSpkYm0/1l9SVORCAxxhe9rRPUNc8LyPaRj8sdFV8G8qFdSMuTxFLxGf IEyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Dg+kECwAuBShufULtVDvhwFWlxHBpA5vuIilKUWCHKw=; b=MtTU+tXPPvEl76sDhbtkvA17ZROF2FPRKCy0yXlSrR5AIQpX/RxS4ijCa1nhue7iBk c3zWIz1/7RSn3GzOpRXp3NQ/qobaY+N4HwanttVqeQavh6daALgTSEb+nqQrWyr4/I6O npjL67XwI58+CAtFbQwJAfFloUqwb3sfH9rAKCUDS2pnGpm2Jszmbx2Iw9z6Y82VAWXV LCr7uobsMvsS1dAdnS6viXILvrbIHU4lDUEL+wgMddDJDOqQcqGFJR3ICdh4cAaqrAPY ZgT4U0UnWEsv2TJ4BmNEo+A3b48qk18S+X98F5OE18Feag2h71V2XW9+RwziMIq7+YS5 f+YA== X-Gm-Message-State: AOAM5308/cATW1GEa2/zEK+gU1gw72C4jxDegkFJu5huYl0OFdYKtZny yYoTLg6CUVGk9c9NGlLuTQ1KBly7L59qiw== X-Google-Smtp-Source: ABdhPJwTuWAXRw7sa81YbVR8I3Tp1EmxpLH90UEDYGKD0GbKrm+IKtLnyiBgmLCRrYOaPmHlNwVg6g== X-Received: by 2002:a05:6a00:1713:b029:332:7eca:41a1 with SMTP id h19-20020a056a001713b02903327eca41a1mr4694625pfc.26.1626354927624; Thu, 15 Jul 2021 06:15:27 -0700 (PDT) Received: from bobo.ozlabs.ibm.com (27-33-83-114.tpgi.com.au. [27.33.83.114]) by smtp.gmail.com with ESMTPSA id k6sm4864216pju.8.2021.07.15.06.15.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 06:15:27 -0700 (PDT) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [RFC PATCH 1/6] KVM: PPC: Book3S HV P9: Add unlikely annotation for !mmu_ready Date: Thu, 15 Jul 2021 23:15:13 +1000 Message-Id: <20210715131518.146917-2-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210715131518.146917-1-npiggin@gmail.com> References: <20210715131518.146917-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org The mmu will almost always be ready. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 9d15bbafe333..27a7a856eed1 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -4367,7 +4367,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, vc->runner = vcpu; /* See if the MMU is ready to go */ - if (!kvm->arch.mmu_ready) { + if (unlikely(!kvm->arch.mmu_ready)) { r = kvmhv_setup_mmu(vcpu); if (r) { run->exit_reason = KVM_EXIT_FAIL_ENTRY; From patchwork Thu Jul 15 13:15:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1505688 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=liF8tBgS; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4GQZcN1n7nz9sXM for ; Thu, 15 Jul 2021 23:15:32 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237208AbhGONSY (ORCPT ); Thu, 15 Jul 2021 09:18:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35998 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232518AbhGONSY (ORCPT ); Thu, 15 Jul 2021 09:18:24 -0400 Received: from mail-pj1-x1033.google.com (mail-pj1-x1033.google.com [IPv6:2607:f8b0:4864:20::1033]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6993FC06175F for ; Thu, 15 Jul 2021 06:15:30 -0700 (PDT) Received: by mail-pj1-x1033.google.com with SMTP id g6-20020a17090adac6b029015d1a9a6f1aso5555306pjx.1 for ; Thu, 15 Jul 2021 06:15:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LEyWXV1GBTM3FA5uKVvdVajFCXRipZIvyhNLU9oRCwI=; b=liF8tBgS1TwYNouKOibIOm55eYMwlwz5WHtJvwmlA9QqMkqvlBjY7m9ZC2By0VwSnH b9SHyZS8gMAvMVJxGF9d6tDzKm+pNBgYqUIC7L61QDAWqXY25dlFQ/dt0aas/kiwNqez Y/pVHZ68Zw8VP/ZjctfESm4OcQgzv0looY2LHsZlnqrOTd+nxBqd0lO2yV1ujaqZ1jUG OuriizqRvMTy73vJ7qcvvkb8I9ZL6jrFh6KOaMtE8EBesibhUv4AQYiusZsIZCt3ssje udr19XpM8Q4pk67zTqlDZl2E3b1dpvaQ6l7UW5DVPSzdvDI5Q7VYMPjQDiWKRLZr9i1g Vp6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LEyWXV1GBTM3FA5uKVvdVajFCXRipZIvyhNLU9oRCwI=; b=mf0jXs+IuVS5OsHArGblVBU2TXHprhCTEO40/G40Djp4dNidyqiz9TW4IgwrUab1C7 EksPugCFqyigsbc5XSx+QkefkMIUivY9MnvrOie0nf5RCrtW6L/YWfrRrcjykMX0KxWm Cxi9pzm0JGKhga8Wy6eF8UJkvHJfQ3bqwNBdFqckG1YywxKRNKuIWsvdyn3MeNZwaa7u HoF+4sI7xnndxEr74hl5oaPQ2XhVNdRpuygy0jgcqYjZz3PkTop+nuKKBvMemI2uflOM Fzurn63ko3Iz1Z+TyMJ3iHoVEAVLaVYa27eFabvX97ON3z6oRdZypuq0QdfxNe1lV8QQ ljhA== X-Gm-Message-State: AOAM530wYuG0+mUcno47kGTvJ1jPNHZFOlpTxNqsdvhZfgqmvlEJHRID 9uCl6JGrcaSCmlRyzRPx3djEiUd9GpnFfQ== X-Google-Smtp-Source: ABdhPJxsAieIhcXMVZSeAw12Pqch/PpJ6O1r9RWmvUXhEn6iO5SUhD0tgxz3Wptan3W6tKLNXeweKg== X-Received: by 2002:a17:90a:930e:: with SMTP id p14mr4331970pjo.132.1626354929879; Thu, 15 Jul 2021 06:15:29 -0700 (PDT) Received: from bobo.ozlabs.ibm.com (27-33-83-114.tpgi.com.au. [27.33.83.114]) by smtp.gmail.com with ESMTPSA id k6sm4864216pju.8.2021.07.15.06.15.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 06:15:29 -0700 (PDT) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [RFC PATCH 2/6] KVM: PPC: Book3S HV P9: Avoid cpu_in_guest atomics on entry and exit Date: Thu, 15 Jul 2021 23:15:14 +1000 Message-Id: <20210715131518.146917-3-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210715131518.146917-1-npiggin@gmail.com> References: <20210715131518.146917-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org cpu_in_guest is set to determine if a CPU needs to be IPI'ed to exit the guest and notice the need_tlb_flush bit. This can be implemented as a global per-CPU pointer to the currently running guest instead of per-guest cpumasks, saving 2 atomics per entry/exit. P7/8 doesn't require cpu_in_guest, nor does a nested HV (only the L0 does), so move it to the P9 HV path. Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/kvm_book3s_64.h | 1 - arch/powerpc/include/asm/kvm_host.h | 1 - arch/powerpc/kvm/book3s_hv.c | 38 +++++++++++++----------- 3 files changed, 21 insertions(+), 19 deletions(-) diff --git a/arch/powerpc/include/asm/kvm_book3s_64.h b/arch/powerpc/include/asm/kvm_book3s_64.h index 20ca9b1a2d41..2b442e00fb5d 100644 --- a/arch/powerpc/include/asm/kvm_book3s_64.h +++ b/arch/powerpc/include/asm/kvm_book3s_64.h @@ -43,7 +43,6 @@ struct kvm_nested_guest { struct mutex tlb_lock; /* serialize page faults and tlbies */ struct kvm_nested_guest *next; cpumask_t need_tlb_flush; - cpumask_t cpu_in_guest; short prev_cpu[NR_CPUS]; u8 radix; /* is this nested guest radix */ }; diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h index cd7939eb47ca..45dc92812020 100644 --- a/arch/powerpc/include/asm/kvm_host.h +++ b/arch/powerpc/include/asm/kvm_host.h @@ -288,7 +288,6 @@ struct kvm_arch { u32 online_vcores; atomic_t hpte_mod_interest; cpumask_t need_tlb_flush; - cpumask_t cpu_in_guest; u8 radix; u8 fwnmi_enabled; u8 secure_guest; diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 27a7a856eed1..d809566918de 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -2981,30 +2981,33 @@ static void kvmppc_release_hwthread(int cpu) tpaca->kvm_hstate.kvm_split_mode = NULL; } +static DEFINE_PER_CPU(struct kvm *, cpu_in_guest); + static void radix_flush_cpu(struct kvm *kvm, int cpu, struct kvm_vcpu *vcpu) { struct kvm_nested_guest *nested = vcpu->arch.nested; - cpumask_t *cpu_in_guest; int i; cpu = cpu_first_tlb_thread_sibling(cpu); - if (nested) { + if (nested) cpumask_set_cpu(cpu, &nested->need_tlb_flush); - cpu_in_guest = &nested->cpu_in_guest; - } else { + else cpumask_set_cpu(cpu, &kvm->arch.need_tlb_flush); - cpu_in_guest = &kvm->arch.cpu_in_guest; - } /* - * Make sure setting of bit in need_tlb_flush precedes - * testing of cpu_in_guest bits. The matching barrier on - * the other side is the first smp_mb() in kvmppc_run_core(). + * Make sure setting of bit in need_tlb_flush precedes testing of + * cpu_in_guest. The matching barrier on the other side is hwsync + * when switching to guest MMU mode, which happens between + * cpu_in_guest being set to the guest kvm, and need_tlb_flush bit + * being tested. */ smp_mb(); for (i = cpu; i <= cpu_last_tlb_thread_sibling(cpu); - i += cpu_tlb_thread_sibling_step()) - if (cpumask_test_cpu(i, cpu_in_guest)) + i += cpu_tlb_thread_sibling_step()) { + struct kvm *running = *per_cpu_ptr(&cpu_in_guest, i); + + if (running == kvm) smp_call_function_single(i, do_nothing, NULL, 1); + } } static void do_migrate_away_vcpu(void *arg) @@ -3072,7 +3075,6 @@ static void kvmppc_start_thread(struct kvm_vcpu *vcpu, struct kvmppc_vcore *vc) { int cpu; struct paca_struct *tpaca; - struct kvm *kvm = vc->kvm; cpu = vc->pcpu; if (vcpu) { @@ -3083,7 +3085,6 @@ static void kvmppc_start_thread(struct kvm_vcpu *vcpu, struct kvmppc_vcore *vc) cpu += vcpu->arch.ptid; vcpu->cpu = vc->pcpu; vcpu->arch.thread_cpu = cpu; - cpumask_set_cpu(cpu, &kvm->arch.cpu_in_guest); } tpaca = paca_ptrs[cpu]; tpaca->kvm_hstate.kvm_vcpu = vcpu; @@ -3801,7 +3802,6 @@ static noinline void kvmppc_run_core(struct kvmppc_vcore *vc) kvmppc_release_hwthread(pcpu + i); if (sip && sip->napped[i]) kvmppc_ipi_thread(pcpu + i); - cpumask_clear_cpu(pcpu + i, &vc->kvm->arch.cpu_in_guest); } spin_unlock(&vc->lock); @@ -3968,8 +3968,14 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, } } else { + struct kvm *kvm = vcpu->kvm; + kvmppc_xive_push_vcpu(vcpu); + + __this_cpu_write(cpu_in_guest, kvm); trap = kvmhv_vcpu_entry_p9(vcpu, time_limit, lpcr, tb); + __this_cpu_write(cpu_in_guest, NULL); + if (trap == BOOK3S_INTERRUPT_SYSCALL && !vcpu->arch.nested && !(vcpu->arch.shregs.msr & MSR_PR)) { unsigned long req = kvmppc_get_gpr(vcpu, 3); @@ -3994,7 +4000,7 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, } kvmppc_xive_pull_vcpu(vcpu); - if (kvm_is_radix(vcpu->kvm)) + if (kvm_is_radix(kvm)) vcpu->arch.slb_max = 0; } @@ -4459,8 +4465,6 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, powerpc_local_irq_pmu_restore(flags); - cpumask_clear_cpu(pcpu, &kvm->arch.cpu_in_guest); - preempt_enable(); /* From patchwork Thu Jul 15 13:15:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1505689 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=qtbNk+iz; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4GQZcQ3gc9z9sV8 for ; Thu, 15 Jul 2021 23:15:34 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237330AbhGONS1 (ORCPT ); Thu, 15 Jul 2021 09:18:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36008 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232518AbhGONS0 (ORCPT ); Thu, 15 Jul 2021 09:18:26 -0400 Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F1D2BC06175F for ; Thu, 15 Jul 2021 06:15:32 -0700 (PDT) Received: by mail-pj1-x1035.google.com with SMTP id b5-20020a17090a9905b029016fc06f6c5bso4137187pjp.5 for ; Thu, 15 Jul 2021 06:15:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QqHqtvZb6+Pm+8g3m8ALUWyG/qaVFq846tO0Idu5F9s=; b=qtbNk+iziUvE+kp/nGNUhDo9+yPAmhMFmvRLjq7mMzHpApNtJI9B5r3boU3MpLUwDW Mb03j8pKuIc1ltg5JzwHaygJp5RmchLy3nmdTY1HbXRhXV4dLfNCIxzAKuW9lLiuzNrI 8rxaaL/AHnsueF2vj6PSKpn6ZQL5fY3AD9hu+YeEhBVPXzwHQoODcOnHuVZlva9xtXfy 42KksbaiKAns2d+J+Wf69LDPAA8magU/SGViRMXN8y9ZT309y3vLQtoEwcA2AhFJeVj/ DDsmXSUoJGiZsILT3xIwCbPbbD/M8tiCNIiOFJKAnMGmxSdwNznZQ+lrdG5A6oAaUl5m EdfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QqHqtvZb6+Pm+8g3m8ALUWyG/qaVFq846tO0Idu5F9s=; b=ti/XgmR53Zyh6ak1U9yeTphrsBryR3AVV821CioGf+yH1DND2WMZ4K7gswUt6MWPI9 89lAmPo+xhSad3qSBjEynKYtYhYDDLzf0mDxALJfoqJCmneU7xg3Mc2Zqh0Ush27+uck xRB3GX08/zDTOgvPnKXRlq5kS4HKKdDKdzYE91bInQKMpBTwGDt4rFwlf9lDQpK/gafm tcRoo7koiSnUoDY/9+cXutOV+Wy6EnCKOTrxfBdw1wmtRWRiKyZ7RtLMuFktUuZlolpH 4iJMrNvMWdrvMd8afjnXnihe4IadxNShgJtOe0fDmFtDBj56nELHhPp371uoVBEOLNRQ 0WKA== X-Gm-Message-State: AOAM532K8IxtUkSbtkW3bNiE0Z8QMItMdeMcsAo0STNktn6tzrALVtwF 3Qe8uvpatHKDwRgWQU3drEFW/ADUA9AR8w== X-Google-Smtp-Source: ABdhPJwKZWdjuZHiXzNQBM5xJzxtLJ5MAamBhN+4qRxV6YRcGETbSSIEBca09wtxQcgctdBF/Z/qxQ== X-Received: by 2002:a17:90b:2411:: with SMTP id nr17mr4144241pjb.153.1626354932390; Thu, 15 Jul 2021 06:15:32 -0700 (PDT) Received: from bobo.ozlabs.ibm.com (27-33-83-114.tpgi.com.au. [27.33.83.114]) by smtp.gmail.com with ESMTPSA id k6sm4864216pju.8.2021.07.15.06.15.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 06:15:32 -0700 (PDT) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [RFC PATCH 3/6] KVM: PPC: Book3S HV P9: Remove most of the vcore logic Date: Thu, 15 Jul 2021 23:15:15 +1000 Message-Id: <20210715131518.146917-4-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210715131518.146917-1-npiggin@gmail.com> References: <20210715131518.146917-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org The P9 path always uses one vcpu per vcore, so none of the the vcore, locks, stolen time, blocking logic, shared waitq, etc., is required. Remove most of it. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv.c | 147 ++++++++++++++++++++--------------- 1 file changed, 85 insertions(+), 62 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index d809566918de..2b02cfe3e456 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -281,6 +281,8 @@ static void kvmppc_core_start_stolen(struct kvmppc_vcore *vc, u64 tb) { unsigned long flags; + WARN_ON_ONCE(cpu_has_feature(CPU_FTR_ARCH_300)); + spin_lock_irqsave(&vc->stoltb_lock, flags); vc->preempt_tb = tb; spin_unlock_irqrestore(&vc->stoltb_lock, flags); @@ -290,6 +292,8 @@ static void kvmppc_core_end_stolen(struct kvmppc_vcore *vc, u64 tb) { unsigned long flags; + WARN_ON_ONCE(cpu_has_feature(CPU_FTR_ARCH_300)); + spin_lock_irqsave(&vc->stoltb_lock, flags); if (vc->preempt_tb != TB_NIL) { vc->stolen_tb += tb - vc->preempt_tb; @@ -302,7 +306,12 @@ static void kvmppc_core_vcpu_load_hv(struct kvm_vcpu *vcpu, int cpu) { struct kvmppc_vcore *vc = vcpu->arch.vcore; unsigned long flags; - u64 now = mftb(); + u64 now; + + if (cpu_has_feature(CPU_FTR_ARCH_300)) + return; + + now = mftb(); /* * We can test vc->runner without taking the vcore lock, @@ -326,7 +335,12 @@ static void kvmppc_core_vcpu_put_hv(struct kvm_vcpu *vcpu) { struct kvmppc_vcore *vc = vcpu->arch.vcore; unsigned long flags; - u64 now = mftb(); + u64 now; + + if (cpu_has_feature(CPU_FTR_ARCH_300)) + return; + + now = mftb(); if (vc->runner == vcpu && vc->vcore_state >= VCORE_SLEEPING) kvmppc_core_start_stolen(vc, now); @@ -678,6 +692,8 @@ static u64 vcore_stolen_time(struct kvmppc_vcore *vc, u64 now) u64 p; unsigned long flags; + WARN_ON_ONCE(cpu_has_feature(CPU_FTR_ARCH_300)); + spin_lock_irqsave(&vc->stoltb_lock, flags); p = vc->stolen_tb; if (vc->vcore_state != VCORE_INACTIVE && @@ -700,13 +716,19 @@ static void kvmppc_create_dtl_entry(struct kvm_vcpu *vcpu, dt = vcpu->arch.dtl_ptr; vpa = vcpu->arch.vpa.pinned_addr; now = tb; - core_stolen = vcore_stolen_time(vc, now); - stolen = core_stolen - vcpu->arch.stolen_logged; - vcpu->arch.stolen_logged = core_stolen; - spin_lock_irqsave(&vcpu->arch.tbacct_lock, flags); - stolen += vcpu->arch.busy_stolen; - vcpu->arch.busy_stolen = 0; - spin_unlock_irqrestore(&vcpu->arch.tbacct_lock, flags); + + if (cpu_has_feature(CPU_FTR_ARCH_300)) { + stolen = 0; + } else { + core_stolen = vcore_stolen_time(vc, now); + stolen = core_stolen - vcpu->arch.stolen_logged; + vcpu->arch.stolen_logged = core_stolen; + spin_lock_irqsave(&vcpu->arch.tbacct_lock, flags); + stolen += vcpu->arch.busy_stolen; + vcpu->arch.busy_stolen = 0; + spin_unlock_irqrestore(&vcpu->arch.tbacct_lock, flags); + } + if (!dt || !vpa) return; memset(dt, 0, sizeof(struct dtl_entry)); @@ -903,13 +925,14 @@ static int kvm_arch_vcpu_yield_to(struct kvm_vcpu *target) * mode handler is not called but no other threads are in the * source vcore. */ - - spin_lock(&vcore->lock); - if (target->arch.state == KVMPPC_VCPU_RUNNABLE && - vcore->vcore_state != VCORE_INACTIVE && - vcore->runner) - target = vcore->runner; - spin_unlock(&vcore->lock); + if (!cpu_has_feature(CPU_FTR_ARCH_300)) { + spin_lock(&vcore->lock); + if (target->arch.state == KVMPPC_VCPU_RUNNABLE && + vcore->vcore_state != VCORE_INACTIVE && + vcore->runner) + target = vcore->runner; + spin_unlock(&vcore->lock); + } return kvm_vcpu_yield_to(target); } @@ -3097,13 +3120,6 @@ static void kvmppc_start_thread(struct kvm_vcpu *vcpu, struct kvmppc_vcore *vc) kvmppc_ipi_thread(cpu); } -/* Old path does this in asm */ -static void kvmppc_stop_thread(struct kvm_vcpu *vcpu) -{ - vcpu->cpu = -1; - vcpu->arch.thread_cpu = -1; -} - static void kvmppc_wait_for_nap(int n_threads) { int cpu = smp_processor_id(); @@ -3192,6 +3208,8 @@ static void kvmppc_vcore_preempt(struct kvmppc_vcore *vc) { struct preempted_vcore_list *lp = this_cpu_ptr(&preempted_vcores); + WARN_ON_ONCE(cpu_has_feature(CPU_FTR_ARCH_300)); + vc->vcore_state = VCORE_PREEMPT; vc->pcpu = smp_processor_id(); if (vc->num_threads < threads_per_vcore(vc->kvm)) { @@ -3208,6 +3226,8 @@ static void kvmppc_vcore_end_preempt(struct kvmppc_vcore *vc) { struct preempted_vcore_list *lp; + WARN_ON_ONCE(cpu_has_feature(CPU_FTR_ARCH_300)); + kvmppc_core_end_stolen(vc, mftb()); if (!list_empty(&vc->preempt_list)) { lp = &per_cpu(preempted_vcores, vc->pcpu); @@ -3935,7 +3955,6 @@ static int kvmhv_vcpu_entry_p9_nested(struct kvm_vcpu *vcpu, u64 time_limit, uns static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpcr, u64 *tb) { - struct kvmppc_vcore *vc = vcpu->arch.vcore; u64 next_timer; int trap; @@ -3951,9 +3970,6 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, kvmppc_subcore_enter_guest(); - vc->entry_exit_map = 1; - vc->in_guest = 1; - vcpu_vpa_increment_dispatch(vcpu); if (kvmhv_on_pseries()) { @@ -4006,9 +4022,6 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, vcpu_vpa_increment_dispatch(vcpu); - vc->entry_exit_map = 0x101; - vc->in_guest = 0; - kvmppc_subcore_exit_guest(); return trap; @@ -4074,6 +4087,13 @@ static bool kvmppc_vcpu_woken(struct kvm_vcpu *vcpu) return false; } +static bool kvmppc_vcpu_check_block(struct kvm_vcpu *vcpu) +{ + if (!vcpu->arch.ceded || kvmppc_vcpu_woken(vcpu)) + return true; + return false; +} + /* * Check to see if any of the runnable vcpus on the vcore have pending * exceptions or are no longer ceded @@ -4084,7 +4104,7 @@ static int kvmppc_vcore_check_block(struct kvmppc_vcore *vc) int i; for_each_runnable_thread(i, vcpu, vc) { - if (!vcpu->arch.ceded || kvmppc_vcpu_woken(vcpu)) + if (kvmppc_vcpu_check_block(vcpu)) return 1; } @@ -4101,6 +4121,8 @@ static void kvmppc_vcore_blocked(struct kvmppc_vcore *vc) int do_sleep = 1; u64 block_ns; + WARN_ON_ONCE(cpu_has_feature(CPU_FTR_ARCH_300)); + /* Poll for pending exceptions and ceded state */ cur = start_poll = ktime_get(); if (vc->halt_poll_ns) { @@ -4366,11 +4388,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, vcpu->arch.ceded = 0; vcpu->arch.run_task = current; vcpu->arch.state = KVMPPC_VCPU_RUNNABLE; - vcpu->arch.busy_preempt = TB_NIL; vcpu->arch.last_inst = KVM_INST_FETCH_FAILED; - vc->runnable_threads[0] = vcpu; - vc->n_runnable = 1; - vc->runner = vcpu; /* See if the MMU is ready to go */ if (unlikely(!kvm->arch.mmu_ready)) { @@ -4388,11 +4406,8 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, kvmppc_update_vpas(vcpu); - init_vcore_to_run(vc); - preempt_disable(); pcpu = smp_processor_id(); - vc->pcpu = pcpu; if (kvm_is_radix(kvm)) kvmppc_prepare_radix_vcpu(vcpu, pcpu); @@ -4421,21 +4436,23 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, goto out; } - tb = mftb(); + if (vcpu->arch.timer_running) { + hrtimer_try_to_cancel(&vcpu->arch.dec_timer); + vcpu->arch.timer_running = 0; + } - vcpu->arch.stolen_logged = vcore_stolen_time(vc, tb); - vc->preempt_tb = TB_NIL; + tb = mftb(); - kvmppc_clear_host_core(pcpu); + vcpu->cpu = pcpu; + vcpu->arch.thread_cpu = pcpu; + local_paca->kvm_hstate.kvm_vcpu = vcpu; + local_paca->kvm_hstate.ptid = 0; + local_paca->kvm_hstate.fake_suspend = 0; - local_paca->kvm_hstate.napping = 0; - local_paca->kvm_hstate.kvm_split_mode = NULL; - kvmppc_start_thread(vcpu, vc); + vc->pcpu = pcpu; // for kvmppc_create_dtl_entry kvmppc_create_dtl_entry(vcpu, vc, tb); - trace_kvm_guest_enter(vcpu); - vc->vcore_state = VCORE_RUNNING; - trace_kvmppc_run_core(vc, 0); + trace_kvm_guest_enter(vcpu); guest_enter_irqoff(); @@ -4457,11 +4474,10 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, set_irq_happened(trap); - kvmppc_set_host_core(pcpu); - guest_exit_irqoff(); - kvmppc_stop_thread(vcpu); + vcpu->cpu = -1; + vcpu->arch.thread_cpu = -1; powerpc_local_irq_pmu_restore(flags); @@ -4488,28 +4504,31 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, } vcpu->arch.ret = r; - if (is_kvmppc_resume_guest(r) && vcpu->arch.ceded && - !kvmppc_vcpu_woken(vcpu)) { + if (is_kvmppc_resume_guest(r) && !kvmppc_vcpu_check_block(vcpu)) { kvmppc_set_timer(vcpu); - while (vcpu->arch.ceded && !kvmppc_vcpu_woken(vcpu)) { + + prepare_to_rcuwait(&vcpu->wait); + for (;;) { + set_current_state(TASK_INTERRUPTIBLE); if (signal_pending(current)) { vcpu->stat.signal_exits++; run->exit_reason = KVM_EXIT_INTR; vcpu->arch.ret = -EINTR; break; } - spin_lock(&vc->lock); - kvmppc_vcore_blocked(vc); - spin_unlock(&vc->lock); + + if (kvmppc_vcpu_check_block(vcpu)) + break; + + trace_kvmppc_vcore_blocked(vc, 0); + schedule(); + trace_kvmppc_vcore_blocked(vc, 1); } + finish_rcuwait(&vcpu->wait); } vcpu->arch.ceded = 0; - vc->vcore_state = VCORE_INACTIVE; - trace_kvmppc_run_core(vc, 1); - done: - kvmppc_remove_runnable(vc, vcpu, tb); trace_kvmppc_run_vcpu_exit(vcpu); return vcpu->arch.ret; @@ -4593,7 +4612,8 @@ static int kvmppc_vcpu_run_hv(struct kvm_vcpu *vcpu) kvmppc_save_current_sprs(); - vcpu->arch.waitp = &vcpu->arch.vcore->wait; + if (!cpu_has_feature(CPU_FTR_ARCH_300)) + vcpu->arch.waitp = &vcpu->arch.vcore->wait; vcpu->arch.pgdir = kvm->mm->pgd; vcpu->arch.state = KVMPPC_VCPU_BUSY_IN_HOST; @@ -5055,6 +5075,9 @@ void kvmppc_alloc_host_rm_ops(void) int cpu, core; int size; + if (cpu_has_feature(CPU_FTR_ARCH_300)) + return; + /* Not the first time here ? */ if (kvmppc_host_rm_ops_hv != NULL) return; From patchwork Thu Jul 15 13:15:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1505690 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=PAgd5N/9; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4GQZcS6Kzsz9sXL for ; Thu, 15 Jul 2021 23:15:36 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237394AbhGONS3 (ORCPT ); Thu, 15 Jul 2021 09:18:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232518AbhGONS3 (ORCPT ); Thu, 15 Jul 2021 09:18:29 -0400 Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com [IPv6:2607:f8b0:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3ADC4C06175F for ; Thu, 15 Jul 2021 06:15:35 -0700 (PDT) Received: by mail-pg1-x52d.google.com with SMTP id 70so2870862pgh.2 for ; Thu, 15 Jul 2021 06:15:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lY2yV6h+HiwueBZqXg6zldEZGTIaMkB1Z2ZLqlofoo0=; b=PAgd5N/9dQI1X81Og9aoffrN5ZKpyOEfk1nDRE530JLtivM8+1QGZs7rsyOwPVz4CC Vuo2BkjrUwQ7uJ3niifEulmmJSa45wJ4Uy/qSWJ1o8WwpU0z1wxKysWzsRMwrY8INIF/ tG8EbtyrD2M9V2jmP/UJVP92nXqlbhcxZW+6pKQGyTb37Mend2JYxE8/jTKx6XvOTQE1 cOjXrJdF9qTHYjyEnsFRg3blx6yjdIbNW/1JewLV90vkSO6JM1M5etSL8uoigrjsdaSX bb6fXkDkx5/wuIxDxmX+jGGhU1l/g70OPEWYY+69WC4ffhKJb+VwtJyBZMfh/vemGska csFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lY2yV6h+HiwueBZqXg6zldEZGTIaMkB1Z2ZLqlofoo0=; b=GyAJyj3RmLeqNBji7vOSbLq8cWebZJGCbRKjERkn2iWFJrDFOdWzwGpOi4lF5nhmnq NuAZwrqaC9Z3ErFKRsI+QHZNoPpMxSUTBTv3FyvJGzvIHAQCAyl10odg3DKNBAgQi13V dIJ4fue6afAjlt4jOXkJkb4QdFL5cZ/W/JwtV/kJA0wbMqRDM/n0OCw8Y/OQshYxahkG DOB/SdF7EDAJAbq3lmkcyVw0abjuSWIv6/jN9OGiWhG/purXwht+YDjJlO5ekhOP/9uC n9jrOIB6K2aGgELlQt+miRYm3b2XumqfodDf6aaaEiaeqsX0QDjLvFDU7auL3FRYZszX XvCQ== X-Gm-Message-State: AOAM531BWeO+1ZAnGOyRdHlAeRwbBX4Yv/NlRjMvah3e1CcsDHtWyb9d mUPCVS98Chl4HDgtukm2MetrsDQankLtJw== X-Google-Smtp-Source: ABdhPJweGLpwOhqDXva0BLe6sBSssb8SzCMoxm5+smIxE2O5VcQ4vM6lo95NW2vfXvIaNjlTpS25/A== X-Received: by 2002:a63:ee45:: with SMTP id n5mr4510309pgk.405.1626354934670; Thu, 15 Jul 2021 06:15:34 -0700 (PDT) Received: from bobo.ozlabs.ibm.com (27-33-83-114.tpgi.com.au. [27.33.83.114]) by smtp.gmail.com with ESMTPSA id k6sm4864216pju.8.2021.07.15.06.15.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 06:15:34 -0700 (PDT) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [RFC PATCH 4/6] KVM: PPC: Book3S HV P9: Tidy kvmppc_create_dtl_entry Date: Thu, 15 Jul 2021 23:15:16 +1000 Message-Id: <20210715131518.146917-5-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210715131518.146917-1-npiggin@gmail.com> References: <20210715131518.146917-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org This goes further to removing vcores from the P9 path. Also avoid the memset in favour of explicitly initialising all fields. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv.c | 61 +++++++++++++++++++++--------------- 1 file changed, 35 insertions(+), 26 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 2b02cfe3e456..93ecbc040529 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -703,41 +703,30 @@ static u64 vcore_stolen_time(struct kvmppc_vcore *vc, u64 now) return p; } -static void kvmppc_create_dtl_entry(struct kvm_vcpu *vcpu, - struct kvmppc_vcore *vc, u64 tb) +static void __kvmppc_create_dtl_entry(struct kvm_vcpu *vcpu, + unsigned int pcpu, u64 now, + unsigned long stolen) { struct dtl_entry *dt; struct lppaca *vpa; - unsigned long stolen; - unsigned long core_stolen; - u64 now; - unsigned long flags; dt = vcpu->arch.dtl_ptr; vpa = vcpu->arch.vpa.pinned_addr; - now = tb; - - if (cpu_has_feature(CPU_FTR_ARCH_300)) { - stolen = 0; - } else { - core_stolen = vcore_stolen_time(vc, now); - stolen = core_stolen - vcpu->arch.stolen_logged; - vcpu->arch.stolen_logged = core_stolen; - spin_lock_irqsave(&vcpu->arch.tbacct_lock, flags); - stolen += vcpu->arch.busy_stolen; - vcpu->arch.busy_stolen = 0; - spin_unlock_irqrestore(&vcpu->arch.tbacct_lock, flags); - } if (!dt || !vpa) return; - memset(dt, 0, sizeof(struct dtl_entry)); + dt->dispatch_reason = 7; - dt->processor_id = cpu_to_be16(vc->pcpu + vcpu->arch.ptid); - dt->timebase = cpu_to_be64(now + vc->tb_offset); + dt->preempt_reason = 0; + dt->processor_id = cpu_to_be16(pcpu + vcpu->arch.ptid); dt->enqueue_to_dispatch_time = cpu_to_be32(stolen); + dt->ready_to_enqueue_time = 0; + dt->waiting_to_ready_time = 0; + dt->timebase = cpu_to_be64(now); + dt->fault_addr = 0; dt->srr0 = cpu_to_be64(kvmppc_get_pc(vcpu)); dt->srr1 = cpu_to_be64(vcpu->arch.shregs.msr); + ++dt; if (dt == vcpu->arch.dtl.pinned_end) dt = vcpu->arch.dtl.pinned_addr; @@ -748,6 +737,27 @@ static void kvmppc_create_dtl_entry(struct kvm_vcpu *vcpu, vcpu->arch.dtl.dirty = true; } +static void kvmppc_create_dtl_entry(struct kvm_vcpu *vcpu, + struct kvmppc_vcore *vc) +{ + unsigned long stolen; + unsigned long core_stolen; + u64 now; + unsigned long flags; + + now = mftb(); + + core_stolen = vcore_stolen_time(vc, now); + stolen = core_stolen - vcpu->arch.stolen_logged; + vcpu->arch.stolen_logged = core_stolen; + spin_lock_irqsave(&vcpu->arch.tbacct_lock, flags); + stolen += vcpu->arch.busy_stolen; + vcpu->arch.busy_stolen = 0; + spin_unlock_irqrestore(&vcpu->arch.tbacct_lock, flags); + + __kvmppc_create_dtl_entry(vcpu, vc->pcpu, now + vc->tb_offset, stolen); +} + /* See if there is a doorbell interrupt pending for a vcpu */ static bool kvmppc_doorbell_pending(struct kvm_vcpu *vcpu) { @@ -3722,7 +3732,7 @@ static noinline void kvmppc_run_core(struct kvmppc_vcore *vc) pvc->pcpu = pcpu + thr; for_each_runnable_thread(i, vcpu, pvc) { kvmppc_start_thread(vcpu, pvc); - kvmppc_create_dtl_entry(vcpu, pvc, mftb()); + kvmppc_create_dtl_entry(vcpu, pvc); trace_kvm_guest_enter(vcpu); if (!vcpu->arch.ptid) thr0_done = true; @@ -4272,7 +4282,7 @@ static int kvmppc_run_vcpu(struct kvm_vcpu *vcpu) if ((vc->vcore_state == VCORE_PIGGYBACK || vc->vcore_state == VCORE_RUNNING) && !VCORE_IS_EXITING(vc)) { - kvmppc_create_dtl_entry(vcpu, vc, mftb()); + kvmppc_create_dtl_entry(vcpu, vc); kvmppc_start_thread(vcpu, vc); trace_kvm_guest_enter(vcpu); } else if (vc->vcore_state == VCORE_SLEEPING) { @@ -4449,8 +4459,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, local_paca->kvm_hstate.ptid = 0; local_paca->kvm_hstate.fake_suspend = 0; - vc->pcpu = pcpu; // for kvmppc_create_dtl_entry - kvmppc_create_dtl_entry(vcpu, vc, tb); + __kvmppc_create_dtl_entry(vcpu, pcpu, tb + vc->tb_offset, 0); trace_kvm_guest_enter(vcpu); From patchwork Thu Jul 15 13:15:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1505691 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=Ykp0pKAI; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4GQZcV3NCFz9snk for ; Thu, 15 Jul 2021 23:15:38 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237409AbhGONSb (ORCPT ); Thu, 15 Jul 2021 09:18:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36030 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232518AbhGONSa (ORCPT ); Thu, 15 Jul 2021 09:18:30 -0400 Received: from mail-pf1-x42e.google.com (mail-pf1-x42e.google.com [IPv6:2607:f8b0:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D2B56C06175F for ; Thu, 15 Jul 2021 06:15:37 -0700 (PDT) Received: by mail-pf1-x42e.google.com with SMTP id m83so5240654pfd.0 for ; Thu, 15 Jul 2021 06:15:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=74+e3cPQA8W7A+zw6sguddhanPGUj6P7ehLyiz9gEPQ=; b=Ykp0pKAIEAEhtwgjfIKeSN8ZqJ7fcjMzUPSrrzugejuRzDQbU+pcn3V9RNqwXB7Wa1 FnnWvmE8GBs87AbXpzKr2HiNLGoxnFvqKC7vDiSmq6CofwgOFaGSkvFzpJqZMThiJICS gpuD20FnnM0uMoxqz8NmPY6fdCgwM0hzoaLFVAojY4GraI01s4zdQ+8bxh0U9mFgboMV RTjCvR7xDxmLOZM/OZB2XQMjXA18+e65CSoKxQity0Su+RgoSM0jObCtVCZDlUa1JSL0 VcG8Vozw41T0j2l3WZG4GSe9R9b1Ei07aXFNLIppAismV8Cmnj+WWVGwMC804Vz0aM8b 01JA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=74+e3cPQA8W7A+zw6sguddhanPGUj6P7ehLyiz9gEPQ=; b=Rkx0Nu710PmBnQz9jQtDRJ48KBpPCDaeJlkOC+5JiVYzr/30B3ljw9Ioyj7Uv8kze9 5DAxAHw2Ebym8HXlpXXffvR9acbJsQDWkel3qC2mLJv7k8ylvMDXfVtzUz523HOSpmcd XiN4FO0e8mvu2rmVE3rsYLv7GV5alGb4HKjN6qsFUrLasXbux4dzb2v+zkpGcOLmxwZO b5Pa/fCychnkaw9lSiKdLgMqbWeJVS+GEqY2ayh1RXfJPxHLPiTAWEErIVwxvfWHylgF QRsGCPYp1SRhtKD3hQ0nHVMmL3yvlCnFW5Ee6QFuBsa+jy4rrIWqUiZXovYcKKscsAdF //qQ== X-Gm-Message-State: AOAM5311KrqDhwKXuYwhsB4W6R87/RtWejYh+xtkHmhCuQqm8YFfi3tl fPw5FLRCGEKTKvQtyQwB5W2Y0BoaJUs23w== X-Google-Smtp-Source: ABdhPJxBxY5oJrblOUzwrUPxp2+HgMxZbX9pnfmMnBBiX92eyFM9PX87Ow+j3Ykxg3D7dNecIcNnug== X-Received: by 2002:a65:41c7:: with SMTP id b7mr4633886pgq.81.1626354937290; Thu, 15 Jul 2021 06:15:37 -0700 (PDT) Received: from bobo.ozlabs.ibm.com (27-33-83-114.tpgi.com.au. [27.33.83.114]) by smtp.gmail.com with ESMTPSA id k6sm4864216pju.8.2021.07.15.06.15.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 06:15:37 -0700 (PDT) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [RFC PATCH 5/6] KVM: PPC: Book3S HV P9: Stop using vc->dpdes Date: Thu, 15 Jul 2021 23:15:17 +1000 Message-Id: <20210715131518.146917-6-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210715131518.146917-1-npiggin@gmail.com> References: <20210715131518.146917-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org The P9 path uses vc->dpdes only for msgsndp / SMT emulation. This adds an ordering requirement between vcpu->doorbell_request and vc->dpdes for no real benefit. Use vcpu->doorbell_request directly. XXX: verify msgsndp / DPDES emulation works properly. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv.c | 18 ++++++++++-------- arch/powerpc/kvm/book3s_hv_builtin.c | 2 ++ arch/powerpc/kvm/book3s_hv_p9_entry.c | 14 ++++++++++---- 3 files changed, 22 insertions(+), 12 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 93ecbc040529..957efde59014 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -766,6 +766,8 @@ static bool kvmppc_doorbell_pending(struct kvm_vcpu *vcpu) if (vcpu->arch.doorbell_request) return true; + if (cpu_has_feature(CPU_FTR_ARCH_300)) + return false; /* * Ensure that the read of vcore->dpdes comes after the read * of vcpu->doorbell_request. This barrier matches the @@ -2165,8 +2167,10 @@ static int kvmppc_get_one_reg_hv(struct kvm_vcpu *vcpu, u64 id, * either vcore->dpdes or doorbell_request. * On POWER8, doorbell_request is 0. */ - *val = get_reg_val(id, vcpu->arch.vcore->dpdes | - vcpu->arch.doorbell_request); + if (cpu_has_feature(CPU_FTR_ARCH_300)) + *val = get_reg_val(id, vcpu->arch.doorbell_request); + else + *val = get_reg_val(id, vcpu->arch.vcore->dpdes); break; case KVM_REG_PPC_VTB: *val = get_reg_val(id, vcpu->arch.vcore->vtb); @@ -2403,7 +2407,10 @@ static int kvmppc_set_one_reg_hv(struct kvm_vcpu *vcpu, u64 id, vcpu->arch.pspb = set_reg_val(id, *val); break; case KVM_REG_PPC_DPDES: - vcpu->arch.vcore->dpdes = set_reg_val(id, *val); + if (cpu_has_feature(CPU_FTR_ARCH_300)) + vcpu->arch.doorbell_request = set_reg_val(id, *val) & 1; + else + vcpu->arch.vcore->dpdes = set_reg_val(id, *val); break; case KVM_REG_PPC_VTB: vcpu->arch.vcore->vtb = set_reg_val(id, *val); @@ -4431,11 +4438,6 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, if (!nested) { kvmppc_core_prepare_to_enter(vcpu); - if (vcpu->arch.doorbell_request) { - vc->dpdes = 1; - smp_wmb(); - vcpu->arch.doorbell_request = 0; - } if (test_bit(BOOK3S_IRQPRIO_EXTERNAL, &vcpu->arch.pending_exceptions)) lpcr |= LPCR_MER; diff --git a/arch/powerpc/kvm/book3s_hv_builtin.c b/arch/powerpc/kvm/book3s_hv_builtin.c index be8ef1c5b1bf..9d24f4472b42 100644 --- a/arch/powerpc/kvm/book3s_hv_builtin.c +++ b/arch/powerpc/kvm/book3s_hv_builtin.c @@ -649,6 +649,8 @@ void kvmppc_guest_entry_inject_int(struct kvm_vcpu *vcpu) int ext; unsigned long lpcr; + WARN_ON_ONCE(cpu_has_feature(CPU_FTR_ARCH_300)); + /* Insert EXTERNAL bit into LPCR at the MER bit position */ ext = (vcpu->arch.pending_exceptions >> BOOK3S_IRQPRIO_EXTERNAL) & 1; lpcr = mfspr(SPRN_LPCR); diff --git a/arch/powerpc/kvm/book3s_hv_p9_entry.c b/arch/powerpc/kvm/book3s_hv_p9_entry.c index 82dac23cc678..44150a29f6e0 100644 --- a/arch/powerpc/kvm/book3s_hv_p9_entry.c +++ b/arch/powerpc/kvm/book3s_hv_p9_entry.c @@ -677,6 +677,7 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc unsigned long host_pidr; unsigned long host_dawr1; unsigned long host_dawrx1; + unsigned long dpdes; hdec = time_limit - *tb; if (hdec < 0) @@ -736,8 +737,10 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc if (vc->pcr) mtspr(SPRN_PCR, vc->pcr | PCR_MASK); - if (vc->dpdes) - mtspr(SPRN_DPDES, vc->dpdes); + if (vcpu->arch.doorbell_request) { + vcpu->arch.doorbell_request = 0; + mtspr(SPRN_DPDES, 1); + } if (dawr_enabled()) { if (vcpu->arch.dawr0 != host_dawr0) @@ -968,7 +971,10 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc vcpu->arch.shregs.sprg2 = mfspr(SPRN_SPRG2); vcpu->arch.shregs.sprg3 = mfspr(SPRN_SPRG3); - vc->dpdes = mfspr(SPRN_DPDES); + dpdes = mfspr(SPRN_DPDES); + if (dpdes) + vcpu->arch.doorbell_request = 1; + vc->vtb = mfspr(SPRN_VTB); dec = mfspr(SPRN_DEC); @@ -1030,7 +1036,7 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc } } - if (vc->dpdes) + if (dpdes) mtspr(SPRN_DPDES, 0); if (vc->pcr) mtspr(SPRN_PCR, PCR_MASK); From patchwork Thu Jul 15 13:15:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1505692 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=tjL58WDW; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4GQZcZ188Dz9sXb for ; Thu, 15 Jul 2021 23:15:42 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237413AbhGONSe (ORCPT ); Thu, 15 Jul 2021 09:18:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36040 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232518AbhGONSe (ORCPT ); Thu, 15 Jul 2021 09:18:34 -0400 Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1A6FEC06175F for ; Thu, 15 Jul 2021 06:15:40 -0700 (PDT) Received: by mail-pj1-x102d.google.com with SMTP id g6-20020a17090adac6b029015d1a9a6f1aso5555560pjx.1 for ; Thu, 15 Jul 2021 06:15:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+Nnq2tW9Cjq/DhRE4HH4kRNi6d38rHtNf1hfbltAJ14=; b=tjL58WDWUl+RJU0o0zl4IHc4mDYYKa9FUMKSrioX6ntqLkxRD3TXrc0Win5jf26ns4 O1G7l6XSgCl6FKbcCoKfaVERtu0aLznA5mnj07oTRg499O2Aaoz6LFJVJx/Vrizv3Om5 mVmIjRibIzktvzxsGyvs4bJSn1PF+YsJ98LR7O1aSXsUpXwPGo0mEVLOlvCHuewMdXbW UulxPa2b812Qd8tFMKJUuj50Mv1izAU664jMotO6irjtx67Bg8A2ZFjfE3QZ8Jn9UHb/ sJEJvxftXxuIf+M4I0VALNrqMzZRNBcifiYO6Fuz+kKuwzpdQL/b1bbyPE9eJQZXXUud MXTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+Nnq2tW9Cjq/DhRE4HH4kRNi6d38rHtNf1hfbltAJ14=; b=YumtSfHnKfsLT+zxi9oyq+XaHuVxveA6TGc4qkJ74tGMcbqEhPD/zWYeJyAukNQeQs Xg14MSAqzj6i/SyLiUeS+Q+7Zpfa2m1x8WgjsqItRB3BizlSBFkkKSIYiiGU+RwiUgp3 ZDKmyYTL4FvXWhZi4HXaBkFTT9Jdz9V5LskhLLJrnvdZa4SJ4gdVg4xZv5858hyklOBN aWTV6WWzd/BWNdBdxB46WlVTz/4cNC7F1pTyDe2qoQHs7MEFNfeCCGXAfrNqvnRuQ9dO xf6+xoaE0Tyt/EmuFSqS0YXfnWpQZjIrkUu9vuBO1uqJEzj12jkRdqQb3EhEN9wVUV0F GXqA== X-Gm-Message-State: AOAM530RoR/CFcC1KwTHn8MNuBO2yeVeKwRCkJavtuYQfQ8bJs+fRlra EQ8sWSUvUqA2pWxsH8QME9rw0xhcuND4Aw== X-Google-Smtp-Source: ABdhPJzQJbG9TZ4Mdi0O8WATkk+L5cO1prDMgUyT/yBbImp+Eg5gaEGjZgvprokb3w3MNOPKrJ52NQ== X-Received: by 2002:a17:90a:e611:: with SMTP id j17mr10042010pjy.48.1626354939561; Thu, 15 Jul 2021 06:15:39 -0700 (PDT) Received: from bobo.ozlabs.ibm.com (27-33-83-114.tpgi.com.au. [27.33.83.114]) by smtp.gmail.com with ESMTPSA id k6sm4864216pju.8.2021.07.15.06.15.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jul 2021 06:15:39 -0700 (PDT) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [RFC PATCH 6/6] KVM: PPC: Book3S HV P9: Remove subcore HMI handling Date: Thu, 15 Jul 2021 23:15:18 +1000 Message-Id: <20210715131518.146917-7-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210715131518.146917-1-npiggin@gmail.com> References: <20210715131518.146917-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org On POWER9 and newer, rather than the complex HMI synchronisation and subcore state, have each thread un-apply the guest TB offset before calling into the early HMI handler. This allows the subcore state to be avoided, including subcore enter / exit guest, which includes an expensive divide that shows up slightly in profiles. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv.c | 12 +++++------- arch/powerpc/kvm/book3s_hv_hmi.c | 7 ++++++- arch/powerpc/kvm/book3s_hv_p9_entry.c | 21 ++++++++++++++++++++- arch/powerpc/kvm/book3s_hv_ras.c | 4 ++++ 4 files changed, 35 insertions(+), 9 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 957efde59014..7d5c31537a79 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -3985,8 +3985,6 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, vcpu->arch.ceded = 0; - kvmppc_subcore_enter_guest(); - vcpu_vpa_increment_dispatch(vcpu); if (kvmhv_on_pseries()) { @@ -4039,8 +4037,6 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, vcpu_vpa_increment_dispatch(vcpu); - kvmppc_subcore_exit_guest(); - return trap; } @@ -6022,9 +6018,11 @@ static int kvmppc_book3s_init_hv(void) if (r) return r; - r = kvm_init_subcore_bitmap(); - if (r) - return r; + if (!cpu_has_feature(CPU_FTR_ARCH_300)) { + r = kvm_init_subcore_bitmap(); + if (r) + return r; + } /* * We need a way of accessing the XICS interrupt controller, diff --git a/arch/powerpc/kvm/book3s_hv_hmi.c b/arch/powerpc/kvm/book3s_hv_hmi.c index 9af660476314..1ec50c69678b 100644 --- a/arch/powerpc/kvm/book3s_hv_hmi.c +++ b/arch/powerpc/kvm/book3s_hv_hmi.c @@ -20,10 +20,15 @@ void wait_for_subcore_guest_exit(void) /* * NULL bitmap pointer indicates that KVM module hasn't - * been loaded yet and hence no guests are running. + * been loaded yet and hence no guests are running, or running + * on POWER9 or newer CPU. + * * If no KVM is in use, no need to co-ordinate among threads * as all of them will always be in host and no one is going * to modify TB other than the opal hmi handler. + * + * POWER9 and newer don't need this synchronisation. + * * Hence, just return from here. */ if (!local_paca->sibling_subcore_state) diff --git a/arch/powerpc/kvm/book3s_hv_p9_entry.c b/arch/powerpc/kvm/book3s_hv_p9_entry.c index 44150a29f6e0..3c9e7a500264 100644 --- a/arch/powerpc/kvm/book3s_hv_p9_entry.c +++ b/arch/powerpc/kvm/book3s_hv_p9_entry.c @@ -3,6 +3,7 @@ #include #include #include +#include #include #include #include @@ -900,7 +901,25 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc kvmppc_realmode_machine_check(vcpu); } else if (unlikely(trap == BOOK3S_INTERRUPT_HMI)) { - kvmppc_realmode_hmi_handler(); + /* + * Unapply and clear the offset first. That way, if the TB + * was fine then no harm done, if it is corrupted then the + * HMI resync will bring it back to host mode. This way, we + * don't need to actualy know whether not OPAL resynced the + * timebase. Although it would be cleaner if we could rely + * on that, early POWER9 OPAL did not support the + * OPAL_HANDLE_HMI2 call. + */ + if (vc->tb_offset_applied) { + u64 new_tb = mftb() - vc->tb_offset_applied; + mtspr(SPRN_TBU40, new_tb); + if ((mftb() & 0xffffff) < (new_tb & 0xffffff)) { + new_tb += 0x1000000; + mtspr(SPRN_TBU40, new_tb); + } + vc->tb_offset_applied = 0; + } + hmi_exception_realmode(NULL); } else if (trap == BOOK3S_INTERRUPT_H_EMUL_ASSIST) { vcpu->arch.emul_inst = mfspr(SPRN_HEIR); diff --git a/arch/powerpc/kvm/book3s_hv_ras.c b/arch/powerpc/kvm/book3s_hv_ras.c index d4bca93b79f6..a49ee9bdab67 100644 --- a/arch/powerpc/kvm/book3s_hv_ras.c +++ b/arch/powerpc/kvm/book3s_hv_ras.c @@ -136,6 +136,10 @@ void kvmppc_realmode_machine_check(struct kvm_vcpu *vcpu) vcpu->arch.mce_evt = mce_evt; } +/* + * This subcore HMI handling is all only for pre-POWER9 CPUs. + */ + /* Check if dynamic split is in force and return subcore size accordingly. */ static inline int kvmppc_cur_subcore_size(void) {