From patchwork Wed Sep 2 03:25:21 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Wang X-Patchwork-Id: 513276 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4B5251401E7 for ; Wed, 2 Sep 2015 13:27:53 +1000 (AEST) Received: from localhost ([::1]:32957 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZWyiF-0002Nh-Ep for incoming@patchwork.ozlabs.org; Tue, 01 Sep 2015 23:27:51 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:43244) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZWyg7-0007CP-L7 for qemu-devel@nongnu.org; Tue, 01 Sep 2015 23:25:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZWyg5-0006jb-SV for qemu-devel@nongnu.org; Tue, 01 Sep 2015 23:25:39 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57203) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZWyg5-0006hI-IY for qemu-devel@nongnu.org; Tue, 01 Sep 2015 23:25:37 -0400 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (Postfix) with ESMTPS id 48DA4C0EE04F; Wed, 2 Sep 2015 03:25:37 +0000 (UTC) Received: from jason-ThinkPad-T430s.nay.redhat.com (dhcp-14-122.nay.redhat.com [10.66.14.122]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t823PQfk024102; Tue, 1 Sep 2015 23:25:35 -0400 From: Jason Wang To: qemu-devel@nongnu.org Date: Wed, 2 Sep 2015 11:25:21 +0800 Message-Id: <1441164325-14417-5-git-send-email-jasowang@redhat.com> In-Reply-To: <1441164325-14417-1-git-send-email-jasowang@redhat.com> References: <1441164325-14417-1-git-send-email-jasowang@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 209.132.183.28 Cc: cornelia.huck@de.ibm.com, Jason Wang , mst@redhat.com Subject: [Qemu-devel] [PATCH V2 4/8] virtio-pci: fix 1.0 virtqueue migration X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org We don't migrate the followings fields for virtio-pci: uint32_t dfselect; uint32_t gfselect; uint32_t guest_features[2]; struct { uint16_t num; bool enabled; uint32_t desc[2]; uint32_t avail[2]; uint32_t used[2]; } vqs[VIRTIO_QUEUE_MAX]; This will confuse driver if migrating during initialization. Solves this issue by: - introduce transport specific callbacks to load and store extra virtqueue states. - add a new subsection for virtio to migrate transport specific modern device state. - implement pci specific callbacks. - add a new property for virtio-pci for whether or not to migrate extra state. - compat the migration for 2.4 and elder machine types Cc: Michael S. Tsirkin Signed-off-by: Jason Wang --- hw/virtio/virtio-pci.c | 129 +++++++++++++++++++++++++++++++++++++++++ hw/virtio/virtio-pci.h | 20 ++++--- hw/virtio/virtio.c | 57 ++++++++++++++++++ include/hw/compat.h | 6 +- include/hw/virtio/virtio-bus.h | 3 + 5 files changed, 207 insertions(+), 8 deletions(-) diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c index c024161..a96890b 100644 --- a/hw/virtio/virtio-pci.c +++ b/hw/virtio/virtio-pci.c @@ -86,6 +86,129 @@ static void virtio_pci_save_config(DeviceState *d, QEMUFile *f) qemu_put_be16(f, vdev->config_vector); } +static void virtio_pci_load_modern_queue_state(VirtIOPCIQueue *vq, + QEMUFile *f) +{ + vq->num = qemu_get_be16(f); + vq->enabled = qemu_get_be16(f); + vq->desc[0] = qemu_get_be32(f); + vq->desc[1] = qemu_get_be32(f); + vq->avail[0] = qemu_get_be32(f); + vq->avail[1] = qemu_get_be32(f); + vq->used[0] = qemu_get_be32(f); + vq->used[1] = qemu_get_be32(f); +} + +static bool virtio_pci_has_extra_state(DeviceState *d) +{ + VirtIOPCIProxy *proxy = to_virtio_pci_proxy(d); + + return proxy->flags & VIRTIO_PCI_FLAG_MIGRATE_EXTRA; +} + +static int get_virtio_pci_modern_state(QEMUFile *f, void *pv, size_t size) +{ + VirtIOPCIProxy *proxy = pv; + int i; + + proxy->dfselect = qemu_get_be32(f); + proxy->gfselect = qemu_get_be32(f); + proxy->guest_features[0] = qemu_get_be32(f); + proxy->guest_features[1] = qemu_get_be32(f); + for (i = 0; i < VIRTIO_QUEUE_MAX; i++) { + virtio_pci_load_modern_queue_state(&proxy->vqs[i], f); + } + + return 0; +} + +static void virtio_pci_save_modern_queue_state(VirtIOPCIQueue *vq, + QEMUFile *f) +{ + qemu_put_be16(f, vq->num); + qemu_put_be16(f, vq->enabled); + qemu_put_be32(f, vq->desc[0]); + qemu_put_be32(f, vq->desc[1]); + qemu_put_be32(f, vq->avail[0]); + qemu_put_be32(f, vq->avail[1]); + qemu_put_be32(f, vq->used[0]); + qemu_put_be32(f, vq->used[1]); +} + +static void put_virtio_pci_modern_state(QEMUFile *f, void *pv, size_t size) +{ + VirtIOPCIProxy *proxy = pv; + int i; + + qemu_put_be32(f, proxy->dfselect); + qemu_put_be32(f, proxy->gfselect); + qemu_put_be32(f, proxy->guest_features[0]); + qemu_put_be32(f, proxy->guest_features[1]); + for (i = 0; i < VIRTIO_QUEUE_MAX; i++) { + virtio_pci_save_modern_queue_state(&proxy->vqs[i], f); + } +} + +static VMStateInfo vmstate_info_virtio_pci_modern_state = { + .name = "virtqueue_state", + .get = get_virtio_pci_modern_state, + .put = put_virtio_pci_modern_state, +}; + +static bool virtio_pci_modern_state_needed(void *opaque) +{ + VirtIOPCIProxy *proxy = opaque; + + return !(proxy->flags & VIRTIO_PCI_FLAG_DISABLE_MODERN); +} + +static const VMStateDescription vmstate_virtio_pci_modern_state = { + .name = "virtio_pci/modern_state", + .version_id = 1, + .minimum_version_id = 1, + .needed = &virtio_pci_modern_state_needed, + .fields = (VMStateField[]) { + { + .name = "modern_state", + .version_id = 0, + .field_exists = NULL, + .size = 0, + .info = &vmstate_info_virtio_pci_modern_state, + .flags = VMS_SINGLE, + .offset = 0, + }, + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription vmstate_virtio_pci = { + .name = "virtio_pci", + .version_id = 1, + .minimum_version_id = 1, + .minimum_version_id_old = 1, + .fields = (VMStateField[]) { + VMSTATE_END_OF_LIST() + }, + .subsections = (const VMStateDescription*[]) { + &vmstate_virtio_pci_modern_state, + NULL + } +}; + +static void virtio_pci_save_extra_state(DeviceState *d, QEMUFile *f) +{ + VirtIOPCIProxy *proxy = to_virtio_pci_proxy(d); + + vmstate_save_state(f, &vmstate_virtio_pci, proxy, NULL); +} + +static int virtio_pci_load_extra_state(DeviceState *d, QEMUFile *f) +{ + VirtIOPCIProxy *proxy = to_virtio_pci_proxy(d); + + return vmstate_load_state(f, &vmstate_virtio_pci, proxy, 1); +} + static void virtio_pci_save_queue(DeviceState *d, int n, QEMUFile *f) { VirtIOPCIProxy *proxy = to_virtio_pci_proxy(d); @@ -133,6 +256,7 @@ static int virtio_pci_load_queue(DeviceState *d, int n, QEMUFile *f) if (vector != VIRTIO_NO_VECTOR) { return msix_vector_use(&proxy->pci_dev, vector); } + return 0; } @@ -1618,6 +1742,8 @@ static Property virtio_pci_properties[] = { VIRTIO_PCI_FLAG_DISABLE_LEGACY_BIT, false), DEFINE_PROP_BIT("disable-modern", VirtIOPCIProxy, flags, VIRTIO_PCI_FLAG_DISABLE_MODERN_BIT, true), + DEFINE_PROP_BIT("migrate-extra", VirtIOPCIProxy, flags, + VIRTIO_PCI_FLAG_MIGRATE_EXTRA_BIT, true), DEFINE_PROP_END_OF_LIST(), }; @@ -2211,6 +2337,9 @@ static void virtio_pci_bus_class_init(ObjectClass *klass, void *data) k->load_config = virtio_pci_load_config; k->save_queue = virtio_pci_save_queue; k->load_queue = virtio_pci_load_queue; + k->save_extra_state = virtio_pci_save_extra_state; + k->load_extra_state = virtio_pci_load_extra_state; + k->has_extra_state = virtio_pci_has_extra_state; k->query_guest_notifiers = virtio_pci_query_guest_notifiers; k->set_host_notifier = virtio_pci_set_host_notifier; k->set_guest_notifiers = virtio_pci_set_guest_notifiers; diff --git a/hw/virtio/virtio-pci.h b/hw/virtio/virtio-pci.h index b6c442f..d890009 100644 --- a/hw/virtio/virtio-pci.h +++ b/hw/virtio/virtio-pci.h @@ -75,6 +75,10 @@ typedef struct VirtioBusClass VirtioPCIBusClass; #define VIRTIO_PCI_FLAG_DISABLE_LEGACY (1 << VIRTIO_PCI_FLAG_DISABLE_LEGACY_BIT) #define VIRTIO_PCI_FLAG_DISABLE_MODERN (1 << VIRTIO_PCI_FLAG_DISABLE_MODERN_BIT) +/* migrate extra state */ +#define VIRTIO_PCI_FLAG_MIGRATE_EXTRA_BIT 4 +#define VIRTIO_PCI_FLAG_MIGRATE_EXTRA (1 << VIRTIO_PCI_FLAG_MIGRATE_EXTRA_BIT) + typedef struct { MSIMessage msg; int virq; @@ -104,6 +108,14 @@ typedef struct VirtIOPCIRegion { uint32_t type; } VirtIOPCIRegion; +typedef struct VirtIOPCIQueue { + uint16_t num; + bool enabled; + uint32_t desc[2]; + uint32_t avail[2]; + uint32_t used[2]; +} VirtIOPCIQueue; + struct VirtIOPCIProxy { PCIDevice pci_dev; MemoryRegion bar; @@ -124,13 +136,7 @@ struct VirtIOPCIProxy { uint32_t dfselect; uint32_t gfselect; uint32_t guest_features[2]; - struct { - uint16_t num; - bool enabled; - uint32_t desc[2]; - uint32_t avail[2]; - uint32_t used[2]; - } vqs[VIRTIO_QUEUE_MAX]; + VirtIOPCIQueue vqs[VIRTIO_QUEUE_MAX]; bool ioeventfd_disabled; bool ioeventfd_started; diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index 788b556..dcaf022 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -1056,6 +1056,16 @@ static bool virtio_virtqueue_needed(void *opaque) return virtio_host_has_feature(vdev, VIRTIO_F_VERSION_1); } +static bool virtio_extra_state_needed(void *opaque) +{ + VirtIODevice *vdev = opaque; + BusState *qbus = qdev_get_parent_bus(DEVICE(vdev)); + VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); + + return k->has_extra_state && + k->has_extra_state(qbus->parent); +} + static void put_virtqueue_state(QEMUFile *f, void *pv, size_t size) { VirtIODevice *vdev = pv; @@ -1104,6 +1114,52 @@ static const VMStateDescription vmstate_virtio_virtqueues = { } }; +static int get_extra_state(QEMUFile *f, void *pv, size_t size) +{ + VirtIODevice *vdev = pv; + BusState *qbus = qdev_get_parent_bus(DEVICE(vdev)); + VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); + int ret = 0; + + ret = k->load_extra_state(qbus->parent, f); + + return ret; +} + +static void put_extra_state(QEMUFile *f, void *pv, size_t size) +{ + VirtIODevice *vdev = pv; + BusState *qbus = qdev_get_parent_bus(DEVICE(vdev)); + VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); + + k->save_extra_state(qbus->parent, f); +} + +static VMStateInfo vmstate_info_extra_state = { + .name = "virtqueue_extra_state", + .get = get_extra_state, + .put = put_extra_state, +}; + +static const VMStateDescription vmstate_virtio_modern_state = { + .name = "virtio/extra_state", + .version_id = 1, + .minimum_version_id = 1, + .needed = &virtio_extra_state_needed, + .fields = (VMStateField[]) { + { + .name = "modern_state", + .version_id = 0, + .field_exists = NULL, + .size = 0, + .info = &vmstate_info_extra_state, + .flags = VMS_SINGLE, + .offset = 0, + }, + VMSTATE_END_OF_LIST() + } +}; + static const VMStateDescription vmstate_virtio_device_endian = { .name = "virtio/device_endian", .version_id = 1, @@ -1138,6 +1194,7 @@ static const VMStateDescription vmstate_virtio = { &vmstate_virtio_device_endian, &vmstate_virtio_64bit_features, &vmstate_virtio_virtqueues, + &vmstate_virtio_modern_state, NULL } }; diff --git a/include/hw/compat.h b/include/hw/compat.h index 095de5d..47faab0 100644 --- a/include/hw/compat.h +++ b/include/hw/compat.h @@ -2,7 +2,11 @@ #define HW_COMPAT_H #define HW_COMPAT_2_4 \ - /* empty */ + {\ + .driver = "virtio-pci",\ + .property = "migrate-extra",\ + .value = "off",\ + }, #define HW_COMPAT_2_3 \ {\ diff --git a/include/hw/virtio/virtio-bus.h b/include/hw/virtio/virtio-bus.h index 8811415..6c3d4cb 100644 --- a/include/hw/virtio/virtio-bus.h +++ b/include/hw/virtio/virtio-bus.h @@ -44,9 +44,12 @@ typedef struct VirtioBusClass { void (*notify)(DeviceState *d, uint16_t vector); void (*save_config)(DeviceState *d, QEMUFile *f); void (*save_queue)(DeviceState *d, int n, QEMUFile *f); + void (*save_extra_state)(DeviceState *d, QEMUFile *f); int (*load_config)(DeviceState *d, QEMUFile *f); int (*load_queue)(DeviceState *d, int n, QEMUFile *f); int (*load_done)(DeviceState *d, QEMUFile *f); + int (*load_extra_state)(DeviceState *d, QEMUFile *f); + bool (*has_extra_state)(DeviceState *d); bool (*query_guest_notifiers)(DeviceState *d); int (*set_guest_notifiers)(DeviceState *d, int nvqs, bool assign); int (*set_host_notifier)(DeviceState *d, int n, bool assigned);