| Message ID | 20250120164316.31473-13-alexandru.elisei@arm.com |
|---|---|
| State | Handled Elsewhere |
| Headers | show |
| Series | arm/arm64: Add kvmtool to the runner script | expand |
On Mon, Jan 20, 2025 at 04:43:10PM +0000, Alexandru Elisei wrote: > kvmtool, unless told otherwise, will do its best to make sure that a kernel > successfully boots in a virtual machine. Among things like automatically > creating a rootfs, it also adds extra parameters to the kernel command > line. This is actively harmful to kvm-unit-tests, because some tests parse > the kernel command line and they will fail if they encounter the options > added by kvmtool. > > Fortunately for us, kvmtool commit 5613ae26b998 ("Add --nodefaults command > line argument") addded the --nodefaults kvmtool parameter which disables added > all the implicit virtual machine configuration that cannot be disabled by > using other parameters, like modifying the kernel command line. Always use > --nodefaults to allow a test to run. > > kvmtool can be too verbose when running a virtual machine, and this is > controlled with parameters. Add those to the default kvmtool command line > to reduce this verbosity to a minimum. > > Before: > > $ vm run arm/selftest.flat --cpus 2 --mem 256 --params "setup smp=2 mem=256" > Info: # lkvm run -k arm/selftest.flat -m 256 -c 2 --name guest-5035 > Unknown subtest > > EXIT: STATUS=127 > Warning: KVM compatibility warning. > virtio-9p device was not detected. > While you have requested a virtio-9p device, the guest kernel did not initialize it. > Please make sure that the guest kernel was compiled with CONFIG_NET_9P_VIRTIO=y enabled in .config. > Warning: KVM compatibility warning. > virtio-net device was not detected. > While you have requested a virtio-net device, the guest kernel did not initialize it. > Please make sure that the guest kernel was compiled with CONFIG_VIRTIO_NET=y enabled in .config. > Info: KVM session ended normally. > > After: > > $ vm run arm/selftest.flat --nodefaults --network mode=none --loglevel=warning --cpus 2 --mem 256 --params "setup smp=2 mem=256" On riscv I've noticed that with --nodefaults if I don't add parameters with --params then kvmtool segfaults. I have to add --params "" to avoid it. Does that also happen on arm? Anyway, that's something we should fix in kvmtool rather than workaround it here. > PASS: selftest: setup: smp: number of CPUs matches expectation > INFO: selftest: setup: smp: found 2 CPUs > PASS: selftest: setup: mem: memory size matches expectation > INFO: selftest: setup: mem: found 256 MB > SUMMARY: 2 tests > > EXIT: STATUS=1 > > Note that KVMTOOL_DEFAULT_OPTS can be overwritten by an environment > variable with the same name, but it's not documented in the help string for > run_tests.sh. This has been done on purpose, since overwritting > KVMTOOL_DEFAULT_OPTS should only be necessary for debugging or development > purposes. > > Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com> > --- > scripts/runtime.bash | 13 ++++++++++++- > 1 file changed, 12 insertions(+), 1 deletion(-) > > diff --git a/scripts/runtime.bash b/scripts/runtime.bash > index 55d58eef9c7c..abfd1e67b2ef 100644 > --- a/scripts/runtime.bash > +++ b/scripts/runtime.bash > @@ -2,6 +2,17 @@ > : "${MAX_SMP:=$(getconf _NPROCESSORS_ONLN)}" > : "${TIMEOUT:=90s}" > > +# The following parameters are enabled by default when running a test with > +# kvmtool: > +# --nodefaults: suppress VM configuration that cannot be disabled otherwise > +# (like modifying the supplied kernel command line). Tests that > +# use the command line will fail without this parameter. > +# --network mode=none: do not create a network device. kvmtool tries to help the > +# user by automatically create one, and then prints a warning > +# when the VM terminates if the device hasn't been initialized. > +# --loglevel=warning: reduce verbosity > +: "${KVMTOOL_DEFAULT_OPTS:="--nodefaults --network mode=none --loglevel=warning"}" > + > PASS() { echo -ne "\e[32mPASS\e[0m"; } > SKIP() { echo -ne "\e[33mSKIP\e[0m"; } > FAIL() { echo -ne "\e[31mFAIL\e[0m"; } > @@ -103,7 +114,7 @@ function run() > opts="-smp $smp $qemu_opts" > ;; > kvmtool) > - opts="--cpus $smp $kvmtool_opts" > + opts="$KVMTOOL_DEFAULT_OPTS --cpus $smp $kvmtool_opts" > ;; > esac > > -- > 2.47.1 > Otherwise, Reviewed-by: Andrew Jones <andrew.jones@linux.dev>
Hi, On Thu, Jan 23, 2025 at 03:07:18PM +0100, Andrew Jones wrote: > On Mon, Jan 20, 2025 at 04:43:10PM +0000, Alexandru Elisei wrote: > > kvmtool, unless told otherwise, will do its best to make sure that a kernel > > successfully boots in a virtual machine. Among things like automatically > > creating a rootfs, it also adds extra parameters to the kernel command > > line. This is actively harmful to kvm-unit-tests, because some tests parse > > the kernel command line and they will fail if they encounter the options > > added by kvmtool. > > > > Fortunately for us, kvmtool commit 5613ae26b998 ("Add --nodefaults command > > line argument") addded the --nodefaults kvmtool parameter which disables > > added > > > all the implicit virtual machine configuration that cannot be disabled by > > using other parameters, like modifying the kernel command line. Always use > > --nodefaults to allow a test to run. > > > > kvmtool can be too verbose when running a virtual machine, and this is > > controlled with parameters. Add those to the default kvmtool command line > > to reduce this verbosity to a minimum. > > > > Before: > > > > $ vm run arm/selftest.flat --cpus 2 --mem 256 --params "setup smp=2 mem=256" > > Info: # lkvm run -k arm/selftest.flat -m 256 -c 2 --name guest-5035 > > Unknown subtest > > > > EXIT: STATUS=127 > > Warning: KVM compatibility warning. > > virtio-9p device was not detected. > > While you have requested a virtio-9p device, the guest kernel did not initialize it. > > Please make sure that the guest kernel was compiled with CONFIG_NET_9P_VIRTIO=y enabled in .config. > > Warning: KVM compatibility warning. > > virtio-net device was not detected. > > While you have requested a virtio-net device, the guest kernel did not initialize it. > > Please make sure that the guest kernel was compiled with CONFIG_VIRTIO_NET=y enabled in .config. > > Info: KVM session ended normally. > > > > After: > > > > $ vm run arm/selftest.flat --nodefaults --network mode=none --loglevel=warning --cpus 2 --mem 256 --params "setup smp=2 mem=256" > > On riscv I've noticed that with --nodefaults if I don't add parameters > with --params then kvmtool segfaults. I have to add --params "" to > avoid it. Does that also happen on arm? Anyway, that's something we > should fix in kvmtool rather than workaround it here. This should fix it: diff --git a/riscv/fdt.c b/riscv/fdt.c index 85c8f95604f6..f6a702533258 100644 --- a/riscv/fdt.c +++ b/riscv/fdt.c @@ -256,9 +256,10 @@ static int setup_fdt(struct kvm *kvm) if (kvm->cfg.kernel_cmdline) _FDT(fdt_property_string(fdt, "bootargs", kvm->cfg.kernel_cmdline)); - } else + } else if (kvm->cfg.real_cmdline) { _FDT(fdt_property_string(fdt, "bootargs", kvm->cfg.real_cmdline)); + } _FDT(fdt_property_string(fdt, "stdout-path", "serial0")); Looking at the timestamp on the commit, the patch that added --nodefaults came before the patch that added riscv to kvmtool (by about a month). Just in case you want to add a Fixes tag. Thanks, Alex
diff --git a/scripts/runtime.bash b/scripts/runtime.bash index 55d58eef9c7c..abfd1e67b2ef 100644 --- a/scripts/runtime.bash +++ b/scripts/runtime.bash @@ -2,6 +2,17 @@ : "${MAX_SMP:=$(getconf _NPROCESSORS_ONLN)}" : "${TIMEOUT:=90s}" +# The following parameters are enabled by default when running a test with +# kvmtool: +# --nodefaults: suppress VM configuration that cannot be disabled otherwise +# (like modifying the supplied kernel command line). Tests that +# use the command line will fail without this parameter. +# --network mode=none: do not create a network device. kvmtool tries to help the +# user by automatically create one, and then prints a warning +# when the VM terminates if the device hasn't been initialized. +# --loglevel=warning: reduce verbosity +: "${KVMTOOL_DEFAULT_OPTS:="--nodefaults --network mode=none --loglevel=warning"}" + PASS() { echo -ne "\e[32mPASS\e[0m"; } SKIP() { echo -ne "\e[33mSKIP\e[0m"; } FAIL() { echo -ne "\e[31mFAIL\e[0m"; } @@ -103,7 +114,7 @@ function run() opts="-smp $smp $qemu_opts" ;; kvmtool) - opts="--cpus $smp $kvmtool_opts" + opts="$KVMTOOL_DEFAULT_OPTS --cpus $smp $kvmtool_opts" ;; esac
kvmtool, unless told otherwise, will do its best to make sure that a kernel successfully boots in a virtual machine. Among things like automatically creating a rootfs, it also adds extra parameters to the kernel command line. This is actively harmful to kvm-unit-tests, because some tests parse the kernel command line and they will fail if they encounter the options added by kvmtool. Fortunately for us, kvmtool commit 5613ae26b998 ("Add --nodefaults command line argument") addded the --nodefaults kvmtool parameter which disables all the implicit virtual machine configuration that cannot be disabled by using other parameters, like modifying the kernel command line. Always use --nodefaults to allow a test to run. kvmtool can be too verbose when running a virtual machine, and this is controlled with parameters. Add those to the default kvmtool command line to reduce this verbosity to a minimum. Before: $ vm run arm/selftest.flat --cpus 2 --mem 256 --params "setup smp=2 mem=256" Info: # lkvm run -k arm/selftest.flat -m 256 -c 2 --name guest-5035 Unknown subtest EXIT: STATUS=127 Warning: KVM compatibility warning. virtio-9p device was not detected. While you have requested a virtio-9p device, the guest kernel did not initialize it. Please make sure that the guest kernel was compiled with CONFIG_NET_9P_VIRTIO=y enabled in .config. Warning: KVM compatibility warning. virtio-net device was not detected. While you have requested a virtio-net device, the guest kernel did not initialize it. Please make sure that the guest kernel was compiled with CONFIG_VIRTIO_NET=y enabled in .config. Info: KVM session ended normally. After: $ vm run arm/selftest.flat --nodefaults --network mode=none --loglevel=warning --cpus 2 --mem 256 --params "setup smp=2 mem=256" PASS: selftest: setup: smp: number of CPUs matches expectation INFO: selftest: setup: smp: found 2 CPUs PASS: selftest: setup: mem: memory size matches expectation INFO: selftest: setup: mem: found 256 MB SUMMARY: 2 tests EXIT: STATUS=1 Note that KVMTOOL_DEFAULT_OPTS can be overwritten by an environment variable with the same name, but it's not documented in the help string for run_tests.sh. This has been done on purpose, since overwritting KVMTOOL_DEFAULT_OPTS should only be necessary for debugging or development purposes. Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com> --- scripts/runtime.bash | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)