mbox series

[v10,00/10] virtio-iommu: VFIO integration

Message ID 20201008171558.410886-1-jean-philippe@linaro.org
Headers show
Series virtio-iommu: VFIO integration | expand

Message

Jean-Philippe Brucker Oct. 8, 2020, 5:15 p.m. UTC
This series adds support for VFIO endpoints to virtio-iommu.

Versions 1 to 9 were posted by Bharat Bhushan, but I am taking over for
now since he doesn't have much time to spend on it. Thanks again Bharat
for the work!

Two major changes since [v9]:

* Don't use per-endoint page_size_mask properties. Instead keep a global
  page size for the virtio-iommu device, updated when adding a VFIO
  endpoint. Reject hotplug if the page size is incompatible.

* Try to make the MAP/UNMAP paths more efficient, by keeping track of
  memory region within the endpoint structure.

More testing would be appreciated, since I can only test using a software
model as host at the moment. But it does seem to hold well with PCIe
hotplug/unplug, and pass-through to guest userspace, which are no mean
feat.

Note that one page size combination is not supported: host 64kB guest
4kB cannot work, because the host IOMMU driver will automatically pick
64kB pages which prevents mapping at a smaller granule. Supporting this
case would require introducing a page size negotiation mechanism from
the guest all the way to the host IOMMU driver. Possible, but not
planned at the moment.

[v9] https://lore.kernel.org/qemu-devel/20200323084617.1782-1-bbhushan2@marvell.com/

Bharat Bhushan (7):
  virtio-iommu: Add memory notifiers for map/unmap
  virtio-iommu: Call memory notifiers in attach/detach
  virtio-iommu: Add replay() memory region callback
  virtio-iommu: Add notify_flag_changed() memory region callback
  memory: Add interface to set iommu page size mask
  vfio: Set IOMMU page size as per host supported page size
  virtio-iommu: Set supported page size mask

Jean-Philippe Brucker (3):
  virtio-iommu: Fix virtio_iommu_mr()
  virtio-iommu: Store memory region in endpoint struct
  vfio: Don't issue full 2^64 unmap

 include/exec/memory.h    |  26 +++++
 hw/vfio/common.c         |  19 ++++
 hw/virtio/virtio-iommu.c | 204 ++++++++++++++++++++++++++++++++++++++-
 softmmu/memory.c         |  13 +++
 hw/virtio/trace-events   |   5 +
 5 files changed, 265 insertions(+), 2 deletions(-)

Comments

Eric Auger Oct. 16, 2020, 1:13 p.m. UTC | #1
Hi Jean,

On 10/8/20 7:15 PM, Jean-Philippe Brucker wrote:
> This series adds support for VFIO endpoints to virtio-iommu.
> 
> Versions 1 to 9 were posted by Bharat Bhushan, but I am taking over for
> now since he doesn't have much time to spend on it. Thanks again Bharat
> for the work!
> 
> Two major changes since [v9]:
> 
> * Don't use per-endoint page_size_mask properties. Instead keep a global
>   page size for the virtio-iommu device, updated when adding a VFIO
>   endpoint. Reject hotplug if the page size is incompatible.
> 
> * Try to make the MAP/UNMAP paths more efficient, by keeping track of
>   memory region within the endpoint structure.
> 
> More testing would be appreciated, since I can only test using a software
> model as host at the moment. But it does seem to hold well with PCIe
> hotplug/unplug, and pass-through to guest userspace, which are no mean
> feat.

I tested vhost migration and the following configurations:
host 4K- guest 4K: vhost, vfio, vfio hotplug
host 64K - guest 64K: vhost, vfio, vfio hotplug
host 4K - guest 64K: vhost, vfio, vfio hoplug

All those configs worked for me. I haven't noticed any isse with those.

Thanks

Eric
> 
> Note that one page size combination is not supported: host 64kB guest
> 4kB cannot work, because the host IOMMU driver will automatically pick
> 64kB pages which prevents mapping at a smaller granule. Supporting this
> case would require introducing a page size negotiation mechanism from
> the guest all the way to the host IOMMU driver. Possible, but not
> planned at the moment.
> 
> [v9] https://lore.kernel.org/qemu-devel/20200323084617.1782-1-bbhushan2@marvell.com/
> 
> Bharat Bhushan (7):
>   virtio-iommu: Add memory notifiers for map/unmap
>   virtio-iommu: Call memory notifiers in attach/detach
>   virtio-iommu: Add replay() memory region callback
>   virtio-iommu: Add notify_flag_changed() memory region callback
>   memory: Add interface to set iommu page size mask
>   vfio: Set IOMMU page size as per host supported page size
>   virtio-iommu: Set supported page size mask
> 
> Jean-Philippe Brucker (3):
>   virtio-iommu: Fix virtio_iommu_mr()
>   virtio-iommu: Store memory region in endpoint struct
>   vfio: Don't issue full 2^64 unmap
> 
>  include/exec/memory.h    |  26 +++++
>  hw/vfio/common.c         |  19 ++++
>  hw/virtio/virtio-iommu.c | 204 ++++++++++++++++++++++++++++++++++++++-
>  softmmu/memory.c         |  13 +++
>  hw/virtio/trace-events   |   5 +
>  5 files changed, 265 insertions(+), 2 deletions(-)
>
Michael S. Tsirkin Oct. 30, 2020, 10:27 a.m. UTC | #2
On Thu, Oct 08, 2020 at 07:15:48PM +0200, Jean-Philippe Brucker wrote:
> This series adds support for VFIO endpoints to virtio-iommu.
> 
> Versions 1 to 9 were posted by Bharat Bhushan, but I am taking over for
> now since he doesn't have much time to spend on it. Thanks again Bharat
> for the work!

ok so just minor things left, correct? Do you plan to post v11?

> Two major changes since [v9]:
> 
> * Don't use per-endoint page_size_mask properties. Instead keep a global
>   page size for the virtio-iommu device, updated when adding a VFIO
>   endpoint. Reject hotplug if the page size is incompatible.
> 
> * Try to make the MAP/UNMAP paths more efficient, by keeping track of
>   memory region within the endpoint structure.
> 
> More testing would be appreciated, since I can only test using a software
> model as host at the moment. But it does seem to hold well with PCIe
> hotplug/unplug, and pass-through to guest userspace, which are no mean
> feat.
> 
> Note that one page size combination is not supported: host 64kB guest
> 4kB cannot work, because the host IOMMU driver will automatically pick
> 64kB pages which prevents mapping at a smaller granule. Supporting this
> case would require introducing a page size negotiation mechanism from
> the guest all the way to the host IOMMU driver. Possible, but not
> planned at the moment.
> 
> [v9] https://lore.kernel.org/qemu-devel/20200323084617.1782-1-bbhushan2@marvell.com/
> 
> Bharat Bhushan (7):
>   virtio-iommu: Add memory notifiers for map/unmap
>   virtio-iommu: Call memory notifiers in attach/detach
>   virtio-iommu: Add replay() memory region callback
>   virtio-iommu: Add notify_flag_changed() memory region callback
>   memory: Add interface to set iommu page size mask
>   vfio: Set IOMMU page size as per host supported page size
>   virtio-iommu: Set supported page size mask
> 
> Jean-Philippe Brucker (3):
>   virtio-iommu: Fix virtio_iommu_mr()
>   virtio-iommu: Store memory region in endpoint struct
>   vfio: Don't issue full 2^64 unmap
> 
>  include/exec/memory.h    |  26 +++++
>  hw/vfio/common.c         |  19 ++++
>  hw/virtio/virtio-iommu.c | 204 ++++++++++++++++++++++++++++++++++++++-
>  softmmu/memory.c         |  13 +++
>  hw/virtio/trace-events   |   5 +
>  5 files changed, 265 insertions(+), 2 deletions(-)
> 
> -- 
> 2.28.0
Jean-Philippe Brucker Oct. 30, 2020, 10:48 a.m. UTC | #3
On Fri, Oct 30, 2020 at 06:27:35AM -0400, Michael S. Tsirkin wrote:
> On Thu, Oct 08, 2020 at 07:15:48PM +0200, Jean-Philippe Brucker wrote:
> > This series adds support for VFIO endpoints to virtio-iommu.
> > 
> > Versions 1 to 9 were posted by Bharat Bhushan, but I am taking over for
> > now since he doesn't have much time to spend on it. Thanks again Bharat
> > for the work!
> 
> ok so just minor things left, correct? Do you plan to post v11?

Yes, today or early next week

Thanks,
Jean