mbox series

[RFC,0/6] Enable vdpa net migration with features depending on CVQ

Message ID 20230706191227.835526-1-eperezma@redhat.com
Headers show
Series Enable vdpa net migration with features depending on CVQ | expand

Message

Eugenio Perez Martin July 6, 2023, 7:12 p.m. UTC
At this moment the migration of net features that depends on CVQ is not
possible, as there is no reliable way to restore the device state like mac
address, number of enabled queues, etc to the destination.  This is mainly
caused because the device must only read CVQ, and process all the commands
before resuming the dataplane.

This RFC lift that requirement, sending the VHOST_VDPA_SET_VRING_ENABLE ioctl
for dataplane vqs only after the device has processed all commands.  If this
method is valid or not, or if it must be signalled by the parent driver
somehow, is still under discussion.  In case it is valid, this code allows
testing the vDPA device for it.

Eugenio Pérez (6):
  vdpa: export vhost_vdpa_set_vring_ready
  vdpa: add should_enable op
  vdpa: use virtio_ops->should_enable at vhost_vdpa_set_vrings_ready
  vdpa: add stub vhost_vdpa_should_enable
  vdpa: delay enable of data vqs
  vdpa: remove net cvq migration blocker

 include/hw/virtio/vhost-vdpa.h |  9 +++++++
 hw/virtio/vhost-vdpa.c         | 33 +++++++++++++++++++------
 net/vhost-vdpa.c               | 45 +++++++++++++++++++++++++---------
 hw/virtio/trace-events         |  2 +-
 4 files changed, 68 insertions(+), 21 deletions(-)

Comments

Michael S. Tsirkin July 6, 2023, 8:02 p.m. UTC | #1
On Thu, Jul 06, 2023 at 09:12:21PM +0200, Eugenio Pérez wrote:
> At this moment the migration of net features that depends on CVQ is not
> possible, as there is no reliable way to restore the device state like mac
> address, number of enabled queues, etc to the destination.  This is mainly
> caused because the device must only read CVQ, and process all the commands
> before resuming the dataplane.
> 
> This RFC lift that requirement, sending the VHOST_VDPA_SET_VRING_ENABLE ioctl
> for dataplane vqs only after the device has processed all commands.  If this
> method is valid or not, or if it must be signalled by the parent driver
> somehow, is still under discussion.  In case it is valid, this code allows
> testing the vDPA device for it.

And you plan to add the reset trick too in a future version?

> Eugenio Pérez (6):
>   vdpa: export vhost_vdpa_set_vring_ready
>   vdpa: add should_enable op
>   vdpa: use virtio_ops->should_enable at vhost_vdpa_set_vrings_ready
>   vdpa: add stub vhost_vdpa_should_enable
>   vdpa: delay enable of data vqs
>   vdpa: remove net cvq migration blocker
> 
>  include/hw/virtio/vhost-vdpa.h |  9 +++++++
>  hw/virtio/vhost-vdpa.c         | 33 +++++++++++++++++++------
>  net/vhost-vdpa.c               | 45 +++++++++++++++++++++++++---------
>  hw/virtio/trace-events         |  2 +-
>  4 files changed, 68 insertions(+), 21 deletions(-)
> 
> -- 
> 2.39.3
>
Eugenio Perez Martin July 7, 2023, 6:21 a.m. UTC | #2
On Thu, Jul 6, 2023 at 10:02 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Thu, Jul 06, 2023 at 09:12:21PM +0200, Eugenio Pérez wrote:
> > At this moment the migration of net features that depends on CVQ is not
> > possible, as there is no reliable way to restore the device state like mac
> > address, number of enabled queues, etc to the destination.  This is mainly
> > caused because the device must only read CVQ, and process all the commands
> > before resuming the dataplane.
> >
> > This RFC lift that requirement, sending the VHOST_VDPA_SET_VRING_ENABLE ioctl
> > for dataplane vqs only after the device has processed all commands.  If this
> > method is valid or not, or if it must be signalled by the parent driver
> > somehow, is still under discussion.  In case it is valid, this code allows
> > testing the vDPA device for it.
>
> And you plan to add the reset trick too in a future version?
>

I'll try to come with a version today.

At the current state of vDPA devices, it will just serve to enable
vp_vdpa to be the destination of a migration. But vp_vdpa cannot
migrate the guest again, so it is a dead end. I can try to add
_F_RING_RESET to vdpa_sim if that makes sense.

Anyway, as HW implement _F_RING_RESET, there should be no need to make
more changes to qemu, so we should be good.

> > Eugenio Pérez (6):
> >   vdpa: export vhost_vdpa_set_vring_ready
> >   vdpa: add should_enable op
> >   vdpa: use virtio_ops->should_enable at vhost_vdpa_set_vrings_ready
> >   vdpa: add stub vhost_vdpa_should_enable
> >   vdpa: delay enable of data vqs
> >   vdpa: remove net cvq migration blocker
> >
> >  include/hw/virtio/vhost-vdpa.h |  9 +++++++
> >  hw/virtio/vhost-vdpa.c         | 33 +++++++++++++++++++------
> >  net/vhost-vdpa.c               | 45 +++++++++++++++++++++++++---------
> >  hw/virtio/trace-events         |  2 +-
> >  4 files changed, 68 insertions(+), 21 deletions(-)
> >
> > --
> > 2.39.3
> >
>