| Message ID | 20231117152855.13382-1-frank.heimes@canonical.com |
|---|---|
| Headers | show |
| Series | Kernel config option missing for s390x PCI passthrough (LP: 2042853) | expand |
On 17.11.23 16:28, frank.heimes@canonical.com wrote: > BugLink: https://bugs.launchpad.net/bugs/2042853 > > SRU Justification: > > [Impact] > > * Today no s390x-specific vfio-pci devices (zPCI) can be passed > from a KVM host to a KVM guest (incl. secure execution guests > in the context of confidential computing). > > * s390x PCI passthrough needs various changes in the s390x kernel zPCI > code (incl. the new s390x-specific Kernel config option > 'CONFIG_VFIO_PCI_ZDEV_KVM') that were introduced with kernel 6.0 > and got backported to 22.04/jammy as part of LP: #1853306. > > * Lunar an newer Ubuntu releases have the code already included from > upstream (incl. the Kernel option 'CONFIG_VFIO_PCI_ZDEV_KVM'), but the > config option is not set, hence zPCI pass-through is still not possible. > > [Fix] > > * To be able to make use of VFIO zPCI pass-through on s390x running newer > Ubuntu releases (especially needed in the context of secure execution) > the (s390x-specific) Kernel config option 'CONFIG_VFIO_PCI_ZDEV_KVM' needs > to be enabled and set to 'y'. > > [Test Case] > > * Hardware used: z14 or greater LPAR, PCI-attached devices > (RoCE VFs, ISM devices, NVMe drive) > > * Setup: Both the kernel and QEMU features are needed for the feature > to function (an upstream QEMU can be used to verify the kernel early), > and the facility is only available on z14 or newer. > When any of those pieces is missing, > the interpretation facility will not be used. > When both the kernel and QEMU features are included in their respective > packages, and running in an LPAR on a z14 or newer machine, > this feature will be enabled automatically. > Existing supported devices should behave as before with no changes > required by an end-user (e.g. no changes to libvirt domain definitions) > -- but will now make use of the interpretation facility. > Additionally, ISM devices will now be eligible for vfio-pci passthrough > (where before QEMU would exit on error if attempting to provide an ISM > device for vfio-pci passthrough, preventing the guest from starting) > > * Testing will include the following scenarios, repeated each for RoCE, > ISM and NVMe: > > 1) Testing of basic device passthrough (create a VM with a vfio-pci > device as part of the libvirt domain definition, passing through > a RoCE VF, an ISM device, or an NVMe drive. Verify that the device > is available in the guest and functioning) > 2) Testing of device hotplug/unplug (create a VM with a vfio-pci device, > virsh detach-device to remove the device from the running guest, > verify the device is removed from the guest, then virsh attach-device > to hotplug the device to the guest again, verify the device functions > in the guest) > 3) Host power off testing: Power off the device from the host, verify > that the device is unplugged from the guest as part of the poweroff > 4) Guest power off testing: Power off the device from within the guest, > verify that the device is unusable in the guest, > power the device back on within the guest and verify that the device > is once again usable. > 5) Guest reboot testing: (create a VM with a vfio-pci device, > verify the device is in working condition, reboot the guest, > verify that the device is still usable after reboot) > > [Regression Potential] > > * The regression potential is moderate, since the code is upstream > for quite a while and already enabled in jammy. > > * The general way on using passthrough has not changed, with this > change (config option) it's now just possible to passthrough > zPCI on top. > > * CCW devices are not affected. > > * And this is s390x-specific anyway, so no other architectures are affected. > > [Other] > > * The enabling of the kernel config option is exactly the same for L, M > and U/N, but I submitted separate patches due to slightly different context > and offsets. > > Frank Heimes (1): > UBUNTU: [Config] enable VFIO zPCI pass-through for s390x > > debian.master/config/annotations | 3 +++ > 1 file changed, 3 insertions(+) > I probably would rather reject for the following reasons: * The config option already exists at least in the M/L annotations * Those should be modified instead of adding duplicate entries * I would expect that if this patch is applied and updateconfigs is run then lines will get moved around. I would rather avoid this when starting to prep a kernel. So please double check and in case this happens submit a new set. Thanks, -Stefan
Okay, I see and agree. Thx for the feedback. Will provide a v2 soon ... On Tue, Nov 21, 2023 at 9:38 AM Stefan Bader <stefan.bader@canonical.com> wrote: > On 17.11.23 16:28, frank.heimes@canonical.com wrote: > > BugLink: https://bugs.launchpad.net/bugs/2042853 > > > > SRU Justification: > > > > [Impact] > > > > * Today no s390x-specific vfio-pci devices (zPCI) can be passed > > from a KVM host to a KVM guest (incl. secure execution guests > > in the context of confidential computing). > > > > * s390x PCI passthrough needs various changes in the s390x kernel zPCI > > code (incl. the new s390x-specific Kernel config option > > 'CONFIG_VFIO_PCI_ZDEV_KVM') that were introduced with kernel 6.0 > > and got backported to 22.04/jammy as part of LP: #1853306. > > > > * Lunar an newer Ubuntu releases have the code already included from > > upstream (incl. the Kernel option 'CONFIG_VFIO_PCI_ZDEV_KVM'), but > the > > config option is not set, hence zPCI pass-through is still not > possible. > > > > [Fix] > > > > * To be able to make use of VFIO zPCI pass-through on s390x running > newer > > Ubuntu releases (especially needed in the context of secure > execution) > > the (s390x-specific) Kernel config option 'CONFIG_VFIO_PCI_ZDEV_KVM' > needs > > to be enabled and set to 'y'. > > > > [Test Case] > > > > * Hardware used: z14 or greater LPAR, PCI-attached devices > > (RoCE VFs, ISM devices, NVMe drive) > > > > * Setup: Both the kernel and QEMU features are needed for the feature > > to function (an upstream QEMU can be used to verify the kernel > early), > > and the facility is only available on z14 or newer. > > When any of those pieces is missing, > > the interpretation facility will not be used. > > When both the kernel and QEMU features are included in their > respective > > packages, and running in an LPAR on a z14 or newer machine, > > this feature will be enabled automatically. > > Existing supported devices should behave as before with no changes > > required by an end-user (e.g. no changes to libvirt domain > definitions) > > -- but will now make use of the interpretation facility. > > Additionally, ISM devices will now be eligible for vfio-pci > passthrough > > (where before QEMU would exit on error if attempting to provide an > ISM > > device for vfio-pci passthrough, preventing the guest from starting) > > > > * Testing will include the following scenarios, repeated each for RoCE, > > ISM and NVMe: > > > > 1) Testing of basic device passthrough (create a VM with a vfio-pci > > device as part of the libvirt domain definition, passing through > > a RoCE VF, an ISM device, or an NVMe drive. Verify that the device > > is available in the guest and functioning) > > 2) Testing of device hotplug/unplug (create a VM with a vfio-pci > device, > > virsh detach-device to remove the device from the running guest, > > verify the device is removed from the guest, then virsh > attach-device > > to hotplug the device to the guest again, verify the device > functions > > in the guest) > > 3) Host power off testing: Power off the device from the host, verify > > that the device is unplugged from the guest as part of the > poweroff > > 4) Guest power off testing: Power off the device from within the > guest, > > verify that the device is unusable in the guest, > > power the device back on within the guest and verify that the > device > > is once again usable. > > 5) Guest reboot testing: (create a VM with a vfio-pci device, > > verify the device is in working condition, reboot the guest, > > verify that the device is still usable after reboot) > > > > [Regression Potential] > > > > * The regression potential is moderate, since the code is upstream > > for quite a while and already enabled in jammy. > > > > * The general way on using passthrough has not changed, with this > > change (config option) it's now just possible to passthrough > > zPCI on top. > > > > * CCW devices are not affected. > > > > * And this is s390x-specific anyway, so no other architectures are > affected. > > > > [Other] > > > > * The enabling of the kernel config option is exactly the same for L, M > > and U/N, but I submitted separate patches due to slightly different > context > > and offsets. > > > > Frank Heimes (1): > > UBUNTU: [Config] enable VFIO zPCI pass-through for s390x > > > > debian.master/config/annotations | 3 +++ > > 1 file changed, 3 insertions(+) > > > I probably would rather reject for the following reasons: > * The config option already exists at least in the M/L annotations > * Those should be modified instead of adding duplicate entries > * I would expect that if this patch is applied and updateconfigs is run > then > lines will get moved around. > I would rather avoid this when starting to prep a kernel. So please > double check and in case this happens submit a new set. > > Thanks, > -Stefan > -- > - Stefan > >
On 17.11.23 16:28, frank.heimes@canonical.com wrote: > BugLink: https://bugs.launchpad.net/bugs/2042853 > > SRU Justification: > > [Impact] > > * Today no s390x-specific vfio-pci devices (zPCI) can be passed > from a KVM host to a KVM guest (incl. secure execution guests > in the context of confidential computing). > > * s390x PCI passthrough needs various changes in the s390x kernel zPCI > code (incl. the new s390x-specific Kernel config option > 'CONFIG_VFIO_PCI_ZDEV_KVM') that were introduced with kernel 6.0 > and got backported to 22.04/jammy as part of LP: #1853306. > > * Lunar an newer Ubuntu releases have the code already included from > upstream (incl. the Kernel option 'CONFIG_VFIO_PCI_ZDEV_KVM'), but the > config option is not set, hence zPCI pass-through is still not possible. > > [Fix] > > * To be able to make use of VFIO zPCI pass-through on s390x running newer > Ubuntu releases (especially needed in the context of secure execution) > the (s390x-specific) Kernel config option 'CONFIG_VFIO_PCI_ZDEV_KVM' needs > to be enabled and set to 'y'. > > [Test Case] > > * Hardware used: z14 or greater LPAR, PCI-attached devices > (RoCE VFs, ISM devices, NVMe drive) > > * Setup: Both the kernel and QEMU features are needed for the feature > to function (an upstream QEMU can be used to verify the kernel early), > and the facility is only available on z14 or newer. > When any of those pieces is missing, > the interpretation facility will not be used. > When both the kernel and QEMU features are included in their respective > packages, and running in an LPAR on a z14 or newer machine, > this feature will be enabled automatically. > Existing supported devices should behave as before with no changes > required by an end-user (e.g. no changes to libvirt domain definitions) > -- but will now make use of the interpretation facility. > Additionally, ISM devices will now be eligible for vfio-pci passthrough > (where before QEMU would exit on error if attempting to provide an ISM > device for vfio-pci passthrough, preventing the guest from starting) > > * Testing will include the following scenarios, repeated each for RoCE, > ISM and NVMe: > > 1) Testing of basic device passthrough (create a VM with a vfio-pci > device as part of the libvirt domain definition, passing through > a RoCE VF, an ISM device, or an NVMe drive. Verify that the device > is available in the guest and functioning) > 2) Testing of device hotplug/unplug (create a VM with a vfio-pci device, > virsh detach-device to remove the device from the running guest, > verify the device is removed from the guest, then virsh attach-device > to hotplug the device to the guest again, verify the device functions > in the guest) > 3) Host power off testing: Power off the device from the host, verify > that the device is unplugged from the guest as part of the poweroff > 4) Guest power off testing: Power off the device from within the guest, > verify that the device is unusable in the guest, > power the device back on within the guest and verify that the device > is once again usable. > 5) Guest reboot testing: (create a VM with a vfio-pci device, > verify the device is in working condition, reboot the guest, > verify that the device is still usable after reboot) > > [Regression Potential] > > * The regression potential is moderate, since the code is upstream > for quite a while and already enabled in jammy. > > * The general way on using passthrough has not changed, with this > change (config option) it's now just possible to passthrough > zPCI on top. > > * CCW devices are not affected. > > * And this is s390x-specific anyway, so no other architectures are affected. > > [Other] > > * The enabling of the kernel config option is exactly the same for L, M > and U/N, but I submitted separate patches due to slightly different context > and offsets. > > Frank Heimes (1): > UBUNTU: [Config] enable VFIO zPCI pass-through for s390x > > debian.master/config/annotations | 3 +++ > 1 file changed, 3 insertions(+) > There should be a v2 coming...
Please reject this upload, since a version 2 was sent: https://lists.ubuntu.com/archives/kernel-team/2023-November/thread.html#147088 On Tue, Nov 21, 2023 at 12:36 PM Frank Heimes <frank.heimes@canonical.com> wrote: > Okay, I see and agree. Thx for the feedback. > > Will provide a v2 soon ... > > > On Tue, Nov 21, 2023 at 9:38 AM Stefan Bader <stefan.bader@canonical.com> > wrote: > >> On 17.11.23 16:28, frank.heimes@canonical.com wrote: >> > BugLink: https://bugs.launchpad.net/bugs/2042853 >> > >> > SRU Justification: >> > >> > [Impact] >> > >> > * Today no s390x-specific vfio-pci devices (zPCI) can be passed >> > from a KVM host to a KVM guest (incl. secure execution guests >> > in the context of confidential computing). >> > >> > * s390x PCI passthrough needs various changes in the s390x kernel zPCI >> > code (incl. the new s390x-specific Kernel config option >> > 'CONFIG_VFIO_PCI_ZDEV_KVM') that were introduced with kernel 6.0 >> > and got backported to 22.04/jammy as part of LP: #1853306. >> > >> > * Lunar an newer Ubuntu releases have the code already included from >> > upstream (incl. the Kernel option 'CONFIG_VFIO_PCI_ZDEV_KVM'), but >> the >> > config option is not set, hence zPCI pass-through is still not >> possible. >> > >> > [Fix] >> > >> > * To be able to make use of VFIO zPCI pass-through on s390x running >> newer >> > Ubuntu releases (especially needed in the context of secure >> execution) >> > the (s390x-specific) Kernel config option >> 'CONFIG_VFIO_PCI_ZDEV_KVM' needs >> > to be enabled and set to 'y'. >> > >> > [Test Case] >> > >> > * Hardware used: z14 or greater LPAR, PCI-attached devices >> > (RoCE VFs, ISM devices, NVMe drive) >> > >> > * Setup: Both the kernel and QEMU features are needed for the feature >> > to function (an upstream QEMU can be used to verify the kernel >> early), >> > and the facility is only available on z14 or newer. >> > When any of those pieces is missing, >> > the interpretation facility will not be used. >> > When both the kernel and QEMU features are included in their >> respective >> > packages, and running in an LPAR on a z14 or newer machine, >> > this feature will be enabled automatically. >> > Existing supported devices should behave as before with no changes >> > required by an end-user (e.g. no changes to libvirt domain >> definitions) >> > -- but will now make use of the interpretation facility. >> > Additionally, ISM devices will now be eligible for vfio-pci >> passthrough >> > (where before QEMU would exit on error if attempting to provide an >> ISM >> > device for vfio-pci passthrough, preventing the guest from starting) >> > >> > * Testing will include the following scenarios, repeated each for >> RoCE, >> > ISM and NVMe: >> > >> > 1) Testing of basic device passthrough (create a VM with a vfio-pci >> > device as part of the libvirt domain definition, passing through >> > a RoCE VF, an ISM device, or an NVMe drive. Verify that the >> device >> > is available in the guest and functioning) >> > 2) Testing of device hotplug/unplug (create a VM with a vfio-pci >> device, >> > virsh detach-device to remove the device from the running guest, >> > verify the device is removed from the guest, then virsh >> attach-device >> > to hotplug the device to the guest again, verify the device >> functions >> > in the guest) >> > 3) Host power off testing: Power off the device from the host, >> verify >> > that the device is unplugged from the guest as part of the >> poweroff >> > 4) Guest power off testing: Power off the device from within the >> guest, >> > verify that the device is unusable in the guest, >> > power the device back on within the guest and verify that the >> device >> > is once again usable. >> > 5) Guest reboot testing: (create a VM with a vfio-pci device, >> > verify the device is in working condition, reboot the guest, >> > verify that the device is still usable after reboot) >> > >> > [Regression Potential] >> > >> > * The regression potential is moderate, since the code is upstream >> > for quite a while and already enabled in jammy. >> > >> > * The general way on using passthrough has not changed, with this >> > change (config option) it's now just possible to passthrough >> > zPCI on top. >> > >> > * CCW devices are not affected. >> > >> > * And this is s390x-specific anyway, so no other architectures are >> affected. >> > >> > [Other] >> > >> > * The enabling of the kernel config option is exactly the same for L, >> M >> > and U/N, but I submitted separate patches due to slightly different >> context >> > and offsets. >> > >> > Frank Heimes (1): >> > UBUNTU: [Config] enable VFIO zPCI pass-through for s390x >> > >> > debian.master/config/annotations | 3 +++ >> > 1 file changed, 3 insertions(+) >> > >> I probably would rather reject for the following reasons: >> * The config option already exists at least in the M/L annotations >> * Those should be modified instead of adding duplicate entries >> * I would expect that if this patch is applied and updateconfigs is run >> then >> lines will get moved around. >> I would rather avoid this when starting to prep a kernel. So please >> double check and in case this happens submit a new set. >> >> Thanks, >> -Stefan >> -- >> - Stefan >> >>
BugLink: https://bugs.launchpad.net/bugs/2042853 SRU Justification: [Impact] * Today no s390x-specific vfio-pci devices (zPCI) can be passed from a KVM host to a KVM guest (incl. secure execution guests in the context of confidential computing). * s390x PCI passthrough needs various changes in the s390x kernel zPCI code (incl. the new s390x-specific Kernel config option 'CONFIG_VFIO_PCI_ZDEV_KVM') that were introduced with kernel 6.0 and got backported to 22.04/jammy as part of LP: #1853306. * Lunar an newer Ubuntu releases have the code already included from upstream (incl. the Kernel option 'CONFIG_VFIO_PCI_ZDEV_KVM'), but the config option is not set, hence zPCI pass-through is still not possible. [Fix] * To be able to make use of VFIO zPCI pass-through on s390x running newer Ubuntu releases (especially needed in the context of secure execution) the (s390x-specific) Kernel config option 'CONFIG_VFIO_PCI_ZDEV_KVM' needs to be enabled and set to 'y'. [Test Case] * Hardware used: z14 or greater LPAR, PCI-attached devices (RoCE VFs, ISM devices, NVMe drive) * Setup: Both the kernel and QEMU features are needed for the feature to function (an upstream QEMU can be used to verify the kernel early), and the facility is only available on z14 or newer. When any of those pieces is missing, the interpretation facility will not be used. When both the kernel and QEMU features are included in their respective packages, and running in an LPAR on a z14 or newer machine, this feature will be enabled automatically. Existing supported devices should behave as before with no changes required by an end-user (e.g. no changes to libvirt domain definitions) -- but will now make use of the interpretation facility. Additionally, ISM devices will now be eligible for vfio-pci passthrough (where before QEMU would exit on error if attempting to provide an ISM device for vfio-pci passthrough, preventing the guest from starting) * Testing will include the following scenarios, repeated each for RoCE, ISM and NVMe: 1) Testing of basic device passthrough (create a VM with a vfio-pci device as part of the libvirt domain definition, passing through a RoCE VF, an ISM device, or an NVMe drive. Verify that the device is available in the guest and functioning) 2) Testing of device hotplug/unplug (create a VM with a vfio-pci device, virsh detach-device to remove the device from the running guest, verify the device is removed from the guest, then virsh attach-device to hotplug the device to the guest again, verify the device functions in the guest) 3) Host power off testing: Power off the device from the host, verify that the device is unplugged from the guest as part of the poweroff 4) Guest power off testing: Power off the device from within the guest, verify that the device is unusable in the guest, power the device back on within the guest and verify that the device is once again usable. 5) Guest reboot testing: (create a VM with a vfio-pci device, verify the device is in working condition, reboot the guest, verify that the device is still usable after reboot) [Regression Potential] * The regression potential is moderate, since the code is upstream for quite a while and already enabled in jammy. * The general way on using passthrough has not changed, with this change (config option) it's now just possible to passthrough zPCI on top. * CCW devices are not affected. * And this is s390x-specific anyway, so no other architectures are affected. [Other] * The enabling of the kernel config option is exactly the same for L, M and U/N, but I submitted separate patches due to slightly different context and offsets. Frank Heimes (1): UBUNTU: [Config] enable VFIO zPCI pass-through for s390x debian.master/config/annotations | 3 +++ 1 file changed, 3 insertions(+)