From patchwork Mon Jul 29 15:36:11 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Hajnoczi X-Patchwork-Id: 262836 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 8BA082C00C1 for ; Tue, 30 Jul 2013 01:39:43 +1000 (EST) Received: from localhost ([::1]:51625 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V3pXx-00033Z-DJ for incoming@patchwork.ozlabs.org; Mon, 29 Jul 2013 11:39:41 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51557) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V3pUx-0006NO-4I for qemu-devel@nongnu.org; Mon, 29 Jul 2013 11:36:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1V3pUq-0002ct-Ja for qemu-devel@nongnu.org; Mon, 29 Jul 2013 11:36:35 -0400 Received: from mx1.redhat.com ([209.132.183.28]:17802) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V3pUq-0002ci-CJ for qemu-devel@nongnu.org; Mon, 29 Jul 2013 11:36:28 -0400 Received: from int-mx01.intmail.prod.int.phx2.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id r6TFaRcN016429 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Mon, 29 Jul 2013 11:36:27 -0400 Received: from localhost (ovpn-112-50.ams2.redhat.com [10.36.112.50]) by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id r6TFaQd0021527; Mon, 29 Jul 2013 11:36:26 -0400 From: Stefan Hajnoczi To: Date: Mon, 29 Jul 2013 17:36:11 +0200 Message-Id: <1375112172-24863-6-git-send-email-stefanha@redhat.com> In-Reply-To: <1375112172-24863-1-git-send-email-stefanha@redhat.com> References: <1375112172-24863-1-git-send-email-stefanha@redhat.com> X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 209.132.183.28 Cc: Anthony Liguori , Stefan Hajnoczi Subject: [Qemu-devel] [PATCH 5/6] dataplane: enable virtio-blk x-data-plane=on live migration X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Although the dataplane thread does not cooperate with dirty memory logging yet it's fairly easy to temporarily disable dataplane during live migration. This way virtio-blk can live migrate when x-data-plane=on. The dataplane thread will restart after migration is cancelled or if the guest resuming virtio-blk operation after migration completes. Reviewed-by: Kevin Wolf Signed-off-by: Stefan Hajnoczi --- hw/block/dataplane/virtio-blk.c | 9 --------- hw/block/virtio-blk.c | 32 ++++++++++++++++++++++++++++++++ include/hw/virtio/virtio-blk.h | 1 + 3 files changed, 33 insertions(+), 9 deletions(-) diff --git a/hw/block/dataplane/virtio-blk.c b/hw/block/dataplane/virtio-blk.c index 2faed43..63c3ffa 100644 --- a/hw/block/dataplane/virtio-blk.c +++ b/hw/block/dataplane/virtio-blk.c @@ -18,7 +18,6 @@ #include "qemu/error-report.h" #include "hw/virtio/dataplane/vring.h" #include "ioq.h" -#include "migration/migration.h" #include "block/block.h" #include "hw/virtio/virtio-blk.h" #include "virtio-blk.h" @@ -69,8 +68,6 @@ struct VirtIOBlockDataPlane { queue */ unsigned int num_reqs; - - Error *migration_blocker; }; /* Raise an interrupt to signal guest, if necessary */ @@ -433,10 +430,6 @@ bool virtio_blk_data_plane_create(VirtIODevice *vdev, VirtIOBlkConf *blk, /* Prevent block operations that conflict with data plane thread */ bdrv_set_in_use(blk->conf.bs, 1); - error_setg(&s->migration_blocker, - "x-data-plane does not support migration"); - migrate_add_blocker(s->migration_blocker); - *dataplane = s; return true; } @@ -448,8 +441,6 @@ void virtio_blk_data_plane_destroy(VirtIOBlockDataPlane *s) } virtio_blk_data_plane_stop(s); - migrate_del_blocker(s->migration_blocker); - error_free(s->migration_blocker); bdrv_set_in_use(s->blk->conf.bs, 0); g_free(s); } diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index cf12469..1237b6a 100644 --- a/hw/block/virtio-blk.c +++ b/hw/block/virtio-blk.c @@ -19,6 +19,7 @@ #include "hw/virtio/virtio-blk.h" #ifdef CONFIG_VIRTIO_BLK_DATA_PLANE # include "dataplane/virtio-blk.h" +# include "migration/migration.h" #endif #include "block/scsi.h" #ifdef __linux__ @@ -628,6 +629,34 @@ void virtio_blk_set_conf(DeviceState *dev, VirtIOBlkConf *blk) memcpy(&(s->blk), blk, sizeof(struct VirtIOBlkConf)); } +#ifdef CONFIG_VIRTIO_BLK_DATA_PLANE +/* Disable dataplane thread during live migration since it does not + * update the dirty memory bitmap yet. + */ +static void virtio_blk_migration_state_changed(Notifier *notifier, void *data) +{ + VirtIOBlock *s = container_of(notifier, VirtIOBlock, + migration_state_notifier); + MigrationState *mig = data; + + if (migration_in_setup(mig)) { + if (!s->dataplane) { + return; + } + virtio_blk_data_plane_destroy(s->dataplane); + s->dataplane = NULL; + } else if (migration_has_finished(mig) || + migration_has_failed(mig)) { + if (s->dataplane) { + return; + } + bdrv_drain_all(); /* complete in-flight non-dataplane requests */ + virtio_blk_data_plane_create(VIRTIO_DEVICE(s), &s->blk, + &s->dataplane); + } +} +#endif /* CONFIG_VIRTIO_BLK_DATA_PLANE */ + static int virtio_blk_device_init(VirtIODevice *vdev) { DeviceState *qdev = DEVICE(vdev); @@ -664,6 +693,8 @@ static int virtio_blk_device_init(VirtIODevice *vdev) virtio_cleanup(vdev); return -1; } + s->migration_state_notifier.notify = virtio_blk_migration_state_changed; + add_migration_state_change_notifier(&s->migration_state_notifier); #endif s->change = qemu_add_vm_change_state_handler(virtio_blk_dma_restart_cb, s); @@ -683,6 +714,7 @@ static int virtio_blk_device_exit(DeviceState *dev) VirtIODevice *vdev = VIRTIO_DEVICE(dev); VirtIOBlock *s = VIRTIO_BLK(dev); #ifdef CONFIG_VIRTIO_BLK_DATA_PLANE + remove_migration_state_change_notifier(&s->migration_state_notifier); virtio_blk_data_plane_destroy(s->dataplane); s->dataplane = NULL; #endif diff --git a/include/hw/virtio/virtio-blk.h b/include/hw/virtio/virtio-blk.h index fc71853..b87cf49 100644 --- a/include/hw/virtio/virtio-blk.h +++ b/include/hw/virtio/virtio-blk.h @@ -125,6 +125,7 @@ typedef struct VirtIOBlock { unsigned short sector_mask; VMChangeStateEntry *change; #ifdef CONFIG_VIRTIO_BLK_DATA_PLANE + Notifier migration_state_notifier; struct VirtIOBlockDataPlane *dataplane; #endif } VirtIOBlock;