| Message ID | 20220129072511.105523-2-npiggin@gmail.com (mailing list archive) |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | KVM: PPC: Book3S PR: Fixes for AIL and SCV | expand |
Nicholas Piggin <npiggin@gmail.com> writes: > PR KVM does not support running with AIL enabled, and SCV does is not > supported with AIL disabled. Fix this by ensuring the SCV facility is > disabled with FSCR while a CPU could be running with AIL=0. > > The PowerNV host supports disabling AIL on a per-CPU basis, so SCV just > needs to be disabled when a vCPU is being run. > > The pSeries machine can only switch AIL on a system-wide basis, so it > must disable SCV support at boot if the configuration can potentially > run a PR KVM guest. > > Also ensure a the FSCR[SCV] bit can not be enabled when emulating > mtFSCR for the guest. > > SCV is not emulated for the PR guest at the moment, this just fixes the > host crashes. > > Alternatives considered and rejected: > - SCV support can not be disabled by PR KVM after boot, because it is > advertised to userspace with HWCAP. > - AIL can not be disabled on a per-CPU basis. At least when running on > pseries it is a per-LPAR setting. > - Support for real-mode SCV vectors will not be added because they are > at 0x17000 so making such a large fixed head space causes immediate > value limits to be exceeded, requiring a lot rework and more code. > - Disabling SCV for any PR KVM possible kernel will cause a slowdown > when not using PR KVM. > - A boot time option to disable SCV to use PR KVM is user-hostile. > - System call instruction emulation for SCV facility unavailable > instructions is too complex and old emulation code was subtly broken > and removed. > > Signed-off-by: Nicholas Piggin <npiggin@gmail.com> > --- > arch/powerpc/kernel/exceptions-64s.S | 4 ++++ > arch/powerpc/kernel/setup_64.c | 28 ++++++++++++++++++++++++++++ > arch/powerpc/kvm/Kconfig | 9 +++++++++ > arch/powerpc/kvm/book3s_pr.c | 20 ++++++++++++++------ > 4 files changed, 55 insertions(+), 6 deletions(-) > > diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S > index 55caeee37c08..b66dd6f775a4 100644 > --- a/arch/powerpc/kernel/exceptions-64s.S > +++ b/arch/powerpc/kernel/exceptions-64s.S > @@ -809,6 +809,10 @@ __start_interrupts: > * - MSR_EE|MSR_RI is clear (no reentrant exceptions) > * - Standard kernel environment is set up (stack, paca, etc) > * > + * KVM: > + * These interrupts do not elevate HV 0->1, so HV is not involved. PR KVM > + * ensures that FSCR[SCV] is disabled whenever it has to force AIL off. > + * > * Call convention: > * > * syscall register convention is in Documentation/powerpc/syscall64-abi.rst > diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c > index be8577ac9397..7f7da641e551 100644 > --- a/arch/powerpc/kernel/setup_64.c > +++ b/arch/powerpc/kernel/setup_64.c > @@ -197,6 +197,34 @@ static void __init configure_exceptions(void) > > /* Under a PAPR hypervisor, we need hypercalls */ > if (firmware_has_feature(FW_FEATURE_SET_MODE)) { > + /* > + * - PR KVM does not support AIL mode interrupts in the host > + * while a PR guest is running. > + * > + * - SCV system call interrupt vectors are only implemented for > + * AIL mode interrupts. > + * > + * - On pseries, AIL mode can only be enabled and disabled > + * system-wide so when a PR VM is created on a pseries host, > + * all CPUs of the host are set to AIL=0 mode. > + * > + * - Therefore host CPUs must not execute scv while a PR VM > + * exists. > + * > + * - SCV support can not be disabled dynamically because the > + * feature is advertised to host userspace. Disabling the > + * facility and emulating it would be possible but is not > + * implemented. > + * > + * - So SCV support is blanket diabled if PR KVM could possibly disabled Reviewed-by: Fabiano Rosas <farosas@linux.ibm.com>
Hi Nicholas,
I love your patch! Yet something to improve:
[auto build test ERROR on powerpc/topic/ppc-kvm]
[also build test ERROR on powerpc/next v5.17-rc2 next-20220202]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Nicholas-Piggin/KVM-PPC-Book3S-PR-Fixes-for-AIL-and-SCV/20220129-152655
base: https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git topic/ppc-kvm
config: powerpc-randconfig-r005-20220131 (https://download.01.org/0day-ci/archive/20220202/202202022141.phJ3zWBF-lkp@intel.com/config)
compiler: powerpc-linux-gcc (GCC) 11.2.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/a751cf4de8f5854152b969afed947b0640ab0c33
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Nicholas-Piggin/KVM-PPC-Book3S-PR-Fixes-for-AIL-and-SCV/20220129-152655
git checkout a751cf4de8f5854152b969afed947b0640ab0c33
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=powerpc SHELL=/bin/bash arch/powerpc/kvm/
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
arch/powerpc/kvm/book3s_pr.c: In function 'kvmppc_core_vcpu_load_pr':
>> arch/powerpc/kvm/book3s_pr.c:146:74: error: 'struct thread_struct' has no member named 'fscr'
146 | if (cpu_has_feature(CPU_FTR_ARCH_300) && (current->thread.fscr & FSCR_SCV))
| ^
arch/powerpc/kvm/book3s_pr.c: In function 'kvmppc_core_vcpu_put_pr':
arch/powerpc/kvm/book3s_pr.c:184:74: error: 'struct thread_struct' has no member named 'fscr'
184 | if (cpu_has_feature(CPU_FTR_ARCH_300) && (current->thread.fscr & FSCR_SCV))
| ^
vim +146 arch/powerpc/kvm/book3s_pr.c
141
142 /* Disable AIL if supported */
143 if (cpu_has_feature(CPU_FTR_HVMODE)) {
144 if (cpu_has_feature(CPU_FTR_ARCH_207S))
145 mtspr(SPRN_LPCR, mfspr(SPRN_LPCR) & ~LPCR_AIL);
> 146 if (cpu_has_feature(CPU_FTR_ARCH_300) && (current->thread.fscr & FSCR_SCV))
147 mtspr(SPRN_FSCR, mfspr(SPRN_FSCR) & ~FSCR_SCV);
148 }
149
150 vcpu->cpu = smp_processor_id();
151 #ifdef CONFIG_PPC_BOOK3S_32
152 current->thread.kvm_shadow_vcpu = vcpu->arch.shadow_vcpu;
153 #endif
154
155 if (kvmppc_is_split_real(vcpu))
156 kvmppc_fixup_split_real(vcpu);
157
158 kvmppc_restore_tm_pr(vcpu);
159 }
160
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S index 55caeee37c08..b66dd6f775a4 100644 --- a/arch/powerpc/kernel/exceptions-64s.S +++ b/arch/powerpc/kernel/exceptions-64s.S @@ -809,6 +809,10 @@ __start_interrupts: * - MSR_EE|MSR_RI is clear (no reentrant exceptions) * - Standard kernel environment is set up (stack, paca, etc) * + * KVM: + * These interrupts do not elevate HV 0->1, so HV is not involved. PR KVM + * ensures that FSCR[SCV] is disabled whenever it has to force AIL off. + * * Call convention: * * syscall register convention is in Documentation/powerpc/syscall64-abi.rst diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c index be8577ac9397..7f7da641e551 100644 --- a/arch/powerpc/kernel/setup_64.c +++ b/arch/powerpc/kernel/setup_64.c @@ -197,6 +197,34 @@ static void __init configure_exceptions(void) /* Under a PAPR hypervisor, we need hypercalls */ if (firmware_has_feature(FW_FEATURE_SET_MODE)) { + /* + * - PR KVM does not support AIL mode interrupts in the host + * while a PR guest is running. + * + * - SCV system call interrupt vectors are only implemented for + * AIL mode interrupts. + * + * - On pseries, AIL mode can only be enabled and disabled + * system-wide so when a PR VM is created on a pseries host, + * all CPUs of the host are set to AIL=0 mode. + * + * - Therefore host CPUs must not execute scv while a PR VM + * exists. + * + * - SCV support can not be disabled dynamically because the + * feature is advertised to host userspace. Disabling the + * facility and emulating it would be possible but is not + * implemented. + * + * - So SCV support is blanket diabled if PR KVM could possibly + * run. That is, PR support compiled in, booting on pseries + * with hash MMU. + */ + if (IS_ENABLED(CONFIG_KVM_BOOK3S_PR_POSSIBLE) && !radix_enabled()) { + init_task.thread.fscr &= ~FSCR_SCV; + cur_cpu_spec->cpu_user_features2 &= ~PPC_FEATURE2_SCV; + } + /* Enable AIL if possible */ if (!pseries_enable_reloc_on_exc()) { init_task.thread.fscr &= ~FSCR_SCV; diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig index 18e58085447c..ddd88179110a 100644 --- a/arch/powerpc/kvm/Kconfig +++ b/arch/powerpc/kvm/Kconfig @@ -112,12 +112,21 @@ config KVM_BOOK3S_64_PR guest in user mode (problem state) and emulating all privileged instructions and registers. + This is only available for hash MMU mode and only supports + guests that use hash MMU mode. + This is not as fast as using hypervisor mode, but works on machines where hypervisor mode is not available or not usable, and can emulate processors that are different from the host processor, including emulating 32-bit processors on a 64-bit host. + Selecting this option will cause the SCV facility to be + disabled when the kernel is booted on the pseries platform in + hash MMU mode (regardless of PR VMs running). When any PR VMs + are running, "AIL" mode is disabled which may slow interrupts + and system calls on the host. + config KVM_BOOK3S_HV_EXIT_TIMING bool "Detailed timing for hypervisor real-mode code" depends on KVM_BOOK3S_HV_POSSIBLE && DEBUG_FS diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c index 34a801c3604a..4d1c84b94b77 100644 --- a/arch/powerpc/kvm/book3s_pr.c +++ b/arch/powerpc/kvm/book3s_pr.c @@ -140,9 +140,12 @@ static void kvmppc_core_vcpu_load_pr(struct kvm_vcpu *vcpu, int cpu) #endif /* Disable AIL if supported */ - if (cpu_has_feature(CPU_FTR_HVMODE) && - cpu_has_feature(CPU_FTR_ARCH_207S)) - mtspr(SPRN_LPCR, mfspr(SPRN_LPCR) & ~LPCR_AIL); + if (cpu_has_feature(CPU_FTR_HVMODE)) { + if (cpu_has_feature(CPU_FTR_ARCH_207S)) + mtspr(SPRN_LPCR, mfspr(SPRN_LPCR) & ~LPCR_AIL); + if (cpu_has_feature(CPU_FTR_ARCH_300) && (current->thread.fscr & FSCR_SCV)) + mtspr(SPRN_FSCR, mfspr(SPRN_FSCR) & ~FSCR_SCV); + } vcpu->cpu = smp_processor_id(); #ifdef CONFIG_PPC_BOOK3S_32 @@ -175,9 +178,12 @@ static void kvmppc_core_vcpu_put_pr(struct kvm_vcpu *vcpu) kvmppc_save_tm_pr(vcpu); /* Enable AIL if supported */ - if (cpu_has_feature(CPU_FTR_HVMODE) && - cpu_has_feature(CPU_FTR_ARCH_207S)) - mtspr(SPRN_LPCR, mfspr(SPRN_LPCR) | LPCR_AIL_3); + if (cpu_has_feature(CPU_FTR_HVMODE)) { + if (cpu_has_feature(CPU_FTR_ARCH_207S)) + mtspr(SPRN_LPCR, mfspr(SPRN_LPCR) | LPCR_AIL_3); + if (cpu_has_feature(CPU_FTR_ARCH_300) && (current->thread.fscr & FSCR_SCV)) + mtspr(SPRN_FSCR, mfspr(SPRN_FSCR) | FSCR_SCV); + } vcpu->cpu = -1; } @@ -1037,6 +1043,8 @@ static int kvmppc_handle_fac(struct kvm_vcpu *vcpu, ulong fac) void kvmppc_set_fscr(struct kvm_vcpu *vcpu, u64 fscr) { + if (fscr & FSCR_SCV) + fscr &= ~FSCR_SCV; /* SCV must not be enabled */ if ((vcpu->arch.fscr & FSCR_TAR) && !(fscr & FSCR_TAR)) { /* TAR got dropped, drop it in shadow too */ kvmppc_giveup_fac(vcpu, FSCR_TAR_LG);
PR KVM does not support running with AIL enabled, and SCV does is not supported with AIL disabled. Fix this by ensuring the SCV facility is disabled with FSCR while a CPU could be running with AIL=0. The PowerNV host supports disabling AIL on a per-CPU basis, so SCV just needs to be disabled when a vCPU is being run. The pSeries machine can only switch AIL on a system-wide basis, so it must disable SCV support at boot if the configuration can potentially run a PR KVM guest. Also ensure a the FSCR[SCV] bit can not be enabled when emulating mtFSCR for the guest. SCV is not emulated for the PR guest at the moment, this just fixes the host crashes. Alternatives considered and rejected: - SCV support can not be disabled by PR KVM after boot, because it is advertised to userspace with HWCAP. - AIL can not be disabled on a per-CPU basis. At least when running on pseries it is a per-LPAR setting. - Support for real-mode SCV vectors will not be added because they are at 0x17000 so making such a large fixed head space causes immediate value limits to be exceeded, requiring a lot rework and more code. - Disabling SCV for any PR KVM possible kernel will cause a slowdown when not using PR KVM. - A boot time option to disable SCV to use PR KVM is user-hostile. - System call instruction emulation for SCV facility unavailable instructions is too complex and old emulation code was subtly broken and removed. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> --- arch/powerpc/kernel/exceptions-64s.S | 4 ++++ arch/powerpc/kernel/setup_64.c | 28 ++++++++++++++++++++++++++++ arch/powerpc/kvm/Kconfig | 9 +++++++++ arch/powerpc/kvm/book3s_pr.c | 20 ++++++++++++++------ 4 files changed, 55 insertions(+), 6 deletions(-)