mbox series

[RFC,v3,0/8] vDPA support in qemu

Message ID 20200529140620.28759-1-lulu@redhat.com
Headers show
Series vDPA support in qemu | expand

Message

Cindy Lu May 29, 2020, 2:06 p.m. UTC 
vDPA device is a device that uses a datapath which complies with the
virtio specifications with vendor specific control path. vDPA devices
can be both physically located on the hardware or emulated by software.
This RFC introduce the vDPA support in qemu
TODO: 
1) vIOMMU support
2) live migration support

change from v1
separate the patch of introduce vhost_set_vring_ready method
separate the patch of qemu_get_peer
separate the patch  of vhost_set_state
introduce the new macro specific for vDPA in configure
introduce the function to pass the fd from cmdline
introduce the docmation in qemu-options.hx
the other comments form last version 

change from v2
change the work process of vhost set status
introduce vhost_get_device_id
test based on qemu v5.0.0-rc4
the other comments from last version

Cindy Lu (3):
  net: introduce qemu_get_peer
  vhost_net: use the function qemu_get_peer
  vhost-backend: export the vhost backend helper

Jason Wang (3):
  virtio-bus: introduce queue_enabled method
  virtio-pci: implement queue_enabled method
  vhost: introduce vhost_set_vring_ready method

Tiwei Bie (2):
  vhost-vdpa: introduce vhost-vdpa backend
  vhost-vdpa: introduce vhost-vdpa net client

 configure                         |  21 ++
 hw/net/vhost_net-stub.c           |   9 +
 hw/net/vhost_net.c                |  72 +++++-
 hw/virtio/Makefile.objs           |   1 +
 hw/virtio/vhost-backend.c         |  39 +--
 hw/virtio/vhost-vdpa.c            | 399 ++++++++++++++++++++++++++++++
 hw/virtio/vhost.c                 |  37 ++-
 hw/virtio/virtio-pci.c            |  13 +
 hw/virtio/virtio.c                |   6 +
 include/hw/virtio/vhost-backend.h |  38 ++-
 include/hw/virtio/vhost-vdpa.h    |  26 ++
 include/hw/virtio/vhost.h         |   2 +
 include/hw/virtio/virtio-bus.h    |   4 +
 include/net/net.h                 |   1 +
 include/net/vhost-vdpa.h          |  19 ++
 include/net/vhost_net.h           |   3 +-
 net/Makefile.objs                 |   2 +-
 net/clients.h                     |   2 +
 net/net.c                         |   9 +
 net/vhost-vdpa.c                  | 235 ++++++++++++++++++
 qapi/net.json                     |  26 +-
 qemu-options.hx                   |  15 ++
 22 files changed, 951 insertions(+), 28 deletions(-)
 create mode 100644 hw/virtio/vhost-vdpa.c
 create mode 100644 include/hw/virtio/vhost-vdpa.h
 create mode 100644 include/net/vhost-vdpa.h
 create mode 100644 net/vhost-vdpa.c

Comments

Eric Blake May 29, 2020, 2:22 p.m. UTC | #1 
On 5/29/20 9:06 AM, Cindy Lu wrote:
> From: Tiwei Bie <tiwei.bie@intel.com>
> 
> This patch set introduces a new net client type: vhost-vdpa.
> vhost-vdpa net client will set up a vDPA device which is specified
> by a "vhostdev" parameter.
> 
> Co-authored-by: Lingshan Zhu <lingshan.zhu@intel.com>
> Signed-off-by: Cindy Lu <lulu@redhat.com>
> ---

> +static int net_vhost_vdpa_init(NetClientState *peer, const char *device,
> +                               const char *name, const char *vhostdev,
> +                               bool has_fd, char *fd)
> +{

fd is usually an int, not a string.

> +    NetClientState *nc = NULL;
> +    VhostVDPAState *s;
> +    int vdpa_device_fd = -1;
> +    Error *err = NULL;
> +    int ret = 0;
> +    assert(name);
> +
> +    nc = qemu_new_net_client(&net_vhost_vdpa_info, peer, device, name);
> +    snprintf(nc->info_str, sizeof(nc->info_str), "vhost-vdpa");
> +    nc->queue_index = 0;
> +
> +    s = DO_UPCAST(VhostVDPAState, nc, nc);
> +
> +    if (has_fd) {
> +        vdpa_device_fd = monitor_fd_param(cur_mon, fd, &err);
> +    } else{
> +        vdpa_device_fd = open(vhostdev, O_RDWR);
> +    }

Oh, you're trying to use the old way for passing in fds.  The preferred 
way is to use qemu_open(), at which point you can pass in fds via the 
add-fd QMP command, and then pass the string "/dev/fdset/NNN" as 
vhostdev.  Then you don't need a special fd parameter here.

> +++ b/qapi/net.json
> @@ -428,6 +428,27 @@
>       '*vhostforce':    'bool',
>       '*queues':        'int' } }
>   
> +##
> +# @NetdevVhostVDPAOptions:
> +#
> +# Vhost-vdpa network backend
> +#
> +# @vhostdev: name of a vdpa dev path in sysfs
> +#            (default path:/dev/vhost-vdpa-$ID)
> +#
> +# @fd: file descriptor of an already opened vdpa device
> +#
> +# @queues: number of queues to be created for multiqueue vhost-vdpa
> +#          (default: 1)
> +#
> +# Since: 5.1
> +##
> +{ 'struct': 'NetdevVhostVDPAOptions',
> +  'data': {
> +    '*vhostdev':     'str',
> +    '*fd':           'str',
> +    '*queues':       'int' } }

Instead of having vhostdev and fd both be optional (but where the user 
has to specify exactly one of them), you should only have vhostdev be 
mandatory, and rely on the /dev/fdset/NNN string as a way to get 
vhostdev to point to a previously-passed fd.
no-reply@patchew.org May 29, 2020, 8:29 p.m. UTC | #2 
Patchew URL: https://patchew.org/QEMU/20200529140620.28759-1-lulu@redhat.com/



Hi,

This series failed the docker-quick@centos7 build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
=== TEST SCRIPT END ===

  CC      x86_64-softmmu/hw/virtio/vhost-user-fs-pci.o
  CC      x86_64-softmmu/hw/virtio/virtio-iommu.o
/tmp/qemu-test/src/hw/virtio/vhost-vdpa.c: In function 'vhost_vdpa_set_config':
/tmp/qemu-test/src/hw/virtio/vhost-vdpa.c:323:17: error: 'VHOST_VDPA_MAX_CONFIG_SIZE' undeclared (first use in this function)
     if ((size > VHOST_VDPA_MAX_CONFIG_SIZE) || (data == NULL)) {
                 ^
/tmp/qemu-test/src/hw/virtio/vhost-vdpa.c:323:17: note: each undeclared identifier is reported only once for each function it appears in
make[1]: *** [hw/virtio/vhost-vdpa.o] Error 1
make[1]: *** Waiting for unfinished jobs....
  CC      aarch64-softmmu/hw/vfio/common.o
  CC      aarch64-softmmu/hw/vfio/spapr.o
---
  CC      aarch64-softmmu/hw/virtio/vhost-vsock.o
  CC      aarch64-softmmu/hw/virtio/vhost-vsock-pci.o
/tmp/qemu-test/src/hw/virtio/vhost-vdpa.c: In function 'vhost_vdpa_set_config':
/tmp/qemu-test/src/hw/virtio/vhost-vdpa.c:323:17: error: 'VHOST_VDPA_MAX_CONFIG_SIZE' undeclared (first use in this function)
     if ((size > VHOST_VDPA_MAX_CONFIG_SIZE) || (data == NULL)) {
                 ^
/tmp/qemu-test/src/hw/virtio/vhost-vdpa.c:323:17: note: each undeclared identifier is reported only once for each function it appears in
make[1]: *** [hw/virtio/vhost-vdpa.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [x86_64-softmmu/all] Error 2
make: *** Waiting for unfinished jobs....
make: *** [aarch64-softmmu/all] Error 2
Traceback (most recent call last):
  File "./tests/docker/docker.py", line 664, in <module>
    sys.exit(main())
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=c4d0aff9719e4e6986252b1cdad2d78a', '-u', '1001', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-jxkhgbnq/src/docker-src.2020-05-29-16.25.59.22140:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=c4d0aff9719e4e6986252b1cdad2d78a
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-jxkhgbnq/src'
make: *** [docker-run-test-quick@centos7] Error 2

real    3m7.072s
user    0m9.059s


The full log is available at
http://patchew.org/logs/20200529140620.28759-1-lulu@redhat.com/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
no-reply@patchew.org May 29, 2020, 8:33 p.m. UTC | #3 
Patchew URL: https://patchew.org/QEMU/20200529140620.28759-1-lulu@redhat.com/



Hi,

This series failed the asan build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1
=== TEST SCRIPT END ===

  CC      accel/tcg/trace.o
  CC      backends/trace.o

Warning, treated as error:
/tmp/qemu-test/src/docs/../qemu-options.hx:2920:Inline literal start-string without end-string.
  CC      crypto/trace.o
  CC      monitor/trace.o
---
  CC      block/trace.o
  CC      io/trace.o
  CC      nbd/trace.o
make: *** [Makefile:1114: .docs_system_qemu.1_docs_system_qemu-block-drivers.7_docs_system_qemu-cpu-models.7.sentinel.] Error 2
make: *** Deleting file '.docs_system_qemu.1_docs_system_qemu-block-drivers.7_docs_system_qemu-cpu-models.7.sentinel.'
make: *** Waiting for unfinished jobs....

Warning, treated as error:
/tmp/qemu-test/src/docs/../qemu-options.hx:2920:Inline literal start-string without end-string.
make: *** [Makefile:1103: docs/system/index.html] Error 2
Traceback (most recent call last):
  File "./tests/docker/docker.py", line 664, in <module>
    sys.exit(main())
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=97acfdab68d34ed4abd8fdcbff72793c', '-u', '1001', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=x86_64-softmmu', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-l3clgv2k/src/docker-src.2020-05-29-16.30.04.2097:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-debug']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=97acfdab68d34ed4abd8fdcbff72793c
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-l3clgv2k/src'
make: *** [docker-run-test-debug@fedora] Error 2

real    3m32.660s
user    0m8.310s


The full log is available at
http://patchew.org/logs/20200529140620.28759-1-lulu@redhat.com/testing.asan/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
no-reply@patchew.org May 29, 2020, 8:37 p.m. UTC | #4 
Patchew URL: https://patchew.org/QEMU/20200529140620.28759-1-lulu@redhat.com/



Hi,

This series failed the docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#! /bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-mingw@fedora J=14 NETWORK=1
=== TEST SCRIPT END ===

  CC      io/channel-websock.o
  CC      io/channel-util.o

Warning, treated as error:
/tmp/qemu-test/src/docs/../qemu-options.hx:2920:Inline literal start-string without end-string.
  CC      io/dns-resolver.o
  CC      io/net-listener.o
---
  CC      qom/container.o
  CC      qom/qom-qobject.o

Warning, treated as error:
/tmp/qemu-test/src/docs/../qemu-options.hx:2920:Inline literal start-string without end-string.
  CC      qom/object_interfaces.o
  CC      qemu-io.o
---
  CC      iothread.o
  CC      job-qmp.o
  CC      os-win32.o
make: *** [Makefile:1103: docs/system/index.html] Error 2
make: *** Waiting for unfinished jobs....
make: *** [Makefile:1114: .docs_system_qemu.1_docs_system_qemu-block-drivers.7_docs_system_qemu-cpu-models.7.sentinel.] Error 2
make: *** Deleting file '.docs_system_qemu.1_docs_system_qemu-block-drivers.7_docs_system_qemu-cpu-models.7.sentinel.'
Traceback (most recent call last):
  File "./tests/docker/docker.py", line 664, in <module>
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=1b05e0f6710048c78da28e8db7addc87', '-u', '1001', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-9mffj7y4/src/docker-src.2020-05-29-16.35.13.7869:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=1b05e0f6710048c78da28e8db7addc87
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-9mffj7y4/src'
make: *** [docker-run-test-mingw@fedora] Error 2

real    1m50.552s
user    0m8.680s


The full log is available at
http://patchew.org/logs/20200529140620.28759-1-lulu@redhat.com/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Cindy Lu June 1, 2020, 1:41 a.m. UTC | #5 
On Fri, May 29, 2020 at 10:23 PM Eric Blake <eblake@redhat.com> wrote:
>
> On 5/29/20 9:06 AM, Cindy Lu wrote:
> > From: Tiwei Bie <tiwei.bie@intel.com>
> >
> > This patch set introduces a new net client type: vhost-vdpa.
> > vhost-vdpa net client will set up a vDPA device which is specified
> > by a "vhostdev" parameter.
> >
> > Co-authored-by: Lingshan Zhu <lingshan.zhu@intel.com>
> > Signed-off-by: Cindy Lu <lulu@redhat.com>
> > ---
>
> > +static int net_vhost_vdpa_init(NetClientState *peer, const char *device,
> > +                               const char *name, const char *vhostdev,
> > +                               bool has_fd, char *fd)
> > +{
>
> fd is usually an int, not a string.
>
will fix this
> > +    NetClientState *nc = NULL;
> > +    VhostVDPAState *s;
> > +    int vdpa_device_fd = -1;
> > +    Error *err = NULL;
> > +    int ret = 0;
> > +    assert(name);
> > +
> > +    nc = qemu_new_net_client(&net_vhost_vdpa_info, peer, device, name);
> > +    snprintf(nc->info_str, sizeof(nc->info_str), "vhost-vdpa");
> > +    nc->queue_index = 0;
> > +
> > +    s = DO_UPCAST(VhostVDPAState, nc, nc);
> > +
> > +    if (has_fd) {
> > +        vdpa_device_fd = monitor_fd_param(cur_mon, fd, &err);
> > +    } else{
> > +        vdpa_device_fd = open(vhostdev, O_RDWR);
> > +    }
>
> Oh, you're trying to use the old way for passing in fds.  The preferred
> way is to use qemu_open(), at which point you can pass in fds via the
> add-fd QMP command, and then pass the string "/dev/fdset/NNN" as
> vhostdev.  Then you don't need a special fd parameter here.
>
Thanks Eric, I will try this.

> > +++ b/qapi/net.json
> > @@ -428,6 +428,27 @@
> >       '*vhostforce':    'bool',
> >       '*queues':        'int' } }
> >
> > +##
> > +# @NetdevVhostVDPAOptions:
> > +#
> > +# Vhost-vdpa network backend
> > +#
> > +# @vhostdev: name of a vdpa dev path in sysfs
> > +#            (default path:/dev/vhost-vdpa-$ID)
> > +#
> > +# @fd: file descriptor of an already opened vdpa device
> > +#
> > +# @queues: number of queues to be created for multiqueue vhost-vdpa
> > +#          (default: 1)
> > +#
> > +# Since: 5.1
> > +##
> > +{ 'struct': 'NetdevVhostVDPAOptions',
> > +  'data': {
> > +    '*vhostdev':     'str',
> > +    '*fd':           'str',
> > +    '*queues':       'int' } }
>
> Instead of having vhostdev and fd both be optional (but where the user
> has to specify exactly one of them), you should only have vhostdev be
> mandatory, and rely on the /dev/fdset/NNN string as a way to get
> vhostdev to point to a previously-passed fd.
>
will fix this
> --
> Eric Blake, Principal Software Engineer
> Red Hat, Inc.           +1-919-301-3226
> Virtualization:  qemu.org | libvirt.org
>