From patchwork Fri Jul 26 01:09:41 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Main X-Patchwork-Id: 262027 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 904E02C00F1 for ; Fri, 26 Jul 2013 11:11:35 +1000 (EST) Received: from localhost ([::1]:39569 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V2WZB-0002AW-Ev for incoming@patchwork.ozlabs.org; Thu, 25 Jul 2013 21:11:33 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40939) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V2WXb-0000BW-IT for qemu-devel@nongnu.org; Thu, 25 Jul 2013 21:09:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1V2WXX-0001WH-90 for qemu-devel@nongnu.org; Thu, 25 Jul 2013 21:09:55 -0400 Received: from mx1.redhat.com ([209.132.183.28]:34801) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V2WXW-0001Ts-Vf for qemu-devel@nongnu.org; Thu, 25 Jul 2013 21:09:51 -0400 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id r6Q19o3R011124 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Thu, 25 Jul 2013 21:09:50 -0400 Received: from arnold.mains.com (ovpn-113-23.phx2.redhat.com [10.3.113.23]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id r6Q19kV8000959; Thu, 25 Jul 2013 21:09:48 -0400 From: Ian Main To: qemu-devel@nongnu.org Date: Thu, 25 Jul 2013 18:09:41 -0700 Message-Id: <1374800982-2818-2-git-send-email-imain@redhat.com> In-Reply-To: <1374800982-2818-1-git-send-email-imain@redhat.com> References: <1374800982-2818-1-git-send-email-imain@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 209.132.183.28 Cc: kwolf@redhat.com, famz@redhat.com, rjones@redhat.com, Ian Main , stefanha@redhat.com, pbonzini@redhat.com Subject: [Qemu-devel] [PATCH V7 1/2] Implement sync modes for drive-backup. X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org This patch adds sync-modes to the drive-backup interface and implements the FULL, NONE and TOP modes of synchronization. FULL performs as before copying the entire contents of the drive while preserving the point-in-time using CoW. NONE only copies new writes to the target drive. TOP copies changes to the topmost drive image and preserves the point-in-time using CoW. For sync mode TOP are creating a new target image using the same backing file as the original disk image. Then any new data that has been laid on top of it since creation is copied in the main backup_run() loop. There is an extra check in the 'TOP' case so that we don't bother to copy all the data of the backing file as it already exists in the target. This is where the bdrv_co_is_allocated() is used to determine if the data exists in the topmost layer or below. Also any new data being written is intercepted via the write_notifier hook which ends up calling backup_do_cow() to copy old data out before it gets overwritten. For mode 'NONE' we create the new target image and only copy in the original data from the disk image starting from the time the call was made. This preserves the point in time data by only copying the parts that are *going to change* to the target image. This way we can reconstruct the final image by checking to see if the given block exists in the new target image first, and if it does not, you can get it from the original image. This is basically an optimization allowing you to do point-in-time snapshots with low overhead vs the 'FULL' version. Since there is no old data to copy out the loop in backup_run() for the NONE case just calls qemu_coroutine_yield() which only wakes up after an event (usually cancel in this case). The rest is handled by the before_write notifier which again calls backup_do_cow() to write out the old data so it can be preserved. Signed-off-by: Ian Main --- block/backup.c | 105 +++++++++++++++++++++++++++++++++------------- blockdev.c | 25 +++++++---- include/block/block_int.h | 4 +- qmp-commands.hx | 1 + 4 files changed, 96 insertions(+), 39 deletions(-) diff --git a/block/backup.c b/block/backup.c index 16105d4..6ae8a05 100644 --- a/block/backup.c +++ b/block/backup.c @@ -37,6 +37,7 @@ typedef struct CowRequest { typedef struct BackupBlockJob { BlockJob common; BlockDriverState *target; + MirrorSyncMode sync_mode; RateLimit limit; BlockdevOnError on_source_error; BlockdevOnError on_target_error; @@ -247,40 +248,83 @@ static void coroutine_fn backup_run(void *opaque) bdrv_add_before_write_notifier(bs, &before_write); - for (; start < end; start++) { - bool error_is_read; - - if (block_job_is_cancelled(&job->common)) { - break; + if (job->sync_mode == MIRROR_SYNC_MODE_NONE) { + while (!block_job_is_cancelled(&job->common)) { + /* Yield until the job is cancelled. We just let our before_write + * notify callback service CoW requests. */ + job->common.busy = false; + qemu_coroutine_yield(); + job->common.busy = true; } + } else { + /* Both FULL and TOP SYNC_MODE's require copying.. */ + for (; start < end; start++) { + bool error_is_read; - /* we need to yield so that qemu_aio_flush() returns. - * (without, VM does not reboot) - */ - if (job->common.speed) { - uint64_t delay_ns = ratelimit_calculate_delay( - &job->limit, job->sectors_read); - job->sectors_read = 0; - block_job_sleep_ns(&job->common, rt_clock, delay_ns); - } else { - block_job_sleep_ns(&job->common, rt_clock, 0); - } + if (block_job_is_cancelled(&job->common)) { + break; + } - if (block_job_is_cancelled(&job->common)) { - break; - } + /* we need to yield so that qemu_aio_flush() returns. + * (without, VM does not reboot) + */ + if (job->common.speed) { + uint64_t delay_ns = ratelimit_calculate_delay( + &job->limit, job->sectors_read); + job->sectors_read = 0; + block_job_sleep_ns(&job->common, rt_clock, delay_ns); + } else { + block_job_sleep_ns(&job->common, rt_clock, 0); + } - ret = backup_do_cow(bs, start * BACKUP_SECTORS_PER_CLUSTER, - BACKUP_SECTORS_PER_CLUSTER, &error_is_read); - if (ret < 0) { - /* Depending on error action, fail now or retry cluster */ - BlockErrorAction action = - backup_error_action(job, error_is_read, -ret); - if (action == BDRV_ACTION_REPORT) { + if (block_job_is_cancelled(&job->common)) { break; - } else { - start--; - continue; + } + + if (job->sync_mode == MIRROR_SYNC_MODE_TOP) { + int i, n; + int alloced = 0; + + /* Check to see if these blocks are already in the + * backing file. */ + + for (i = 0; i < BACKUP_SECTORS_PER_CLUSTER;) { + /* bdrv_co_is_allocated() only returns true/false based + * on the first set of sectors it comes accross that + * are are all in the same state. + * For that reason we must verify each sector in the + * backup cluster length. We end up copying more than + * needed but at some point that is always the case. */ + alloced = + bdrv_co_is_allocated(bs, + start * BACKUP_SECTORS_PER_CLUSTER + i, + BACKUP_SECTORS_PER_CLUSTER - i, &n); + i += n; + + if (alloced == 1) { + break; + } + } + + /* If the above loop never found any sectors that are in + * the topmost image, skip this backup. */ + if (alloced == 0) { + continue; + } + } + /* FULL sync mode we copy the whole drive. */ + ret = backup_do_cow(bs, start * BACKUP_SECTORS_PER_CLUSTER, + BACKUP_SECTORS_PER_CLUSTER, &error_is_read); + if (ret < 0) { + /* Depending on error action, fail now or retry cluster */ + BlockErrorAction action = + backup_error_action(job, error_is_read, -ret); + if (action == BDRV_ACTION_REPORT) { + break; + } else { + start--; + continue; + } } } } @@ -300,7 +344,7 @@ static void coroutine_fn backup_run(void *opaque) } void backup_start(BlockDriverState *bs, BlockDriverState *target, - int64_t speed, + int64_t speed, MirrorSyncMode sync_mode, BlockdevOnError on_source_error, BlockdevOnError on_target_error, BlockDriverCompletionFunc *cb, void *opaque, @@ -335,6 +379,7 @@ void backup_start(BlockDriverState *bs, BlockDriverState *target, job->on_source_error = on_source_error; job->on_target_error = on_target_error; job->target = target; + job->sync_mode = sync_mode; job->common.len = len; job->common.co = qemu_coroutine_create(backup_run); qemu_coroutine_enter(job->common.co, job); diff --git a/blockdev.c b/blockdev.c index c5abd65..000dea6 100644 --- a/blockdev.c +++ b/blockdev.c @@ -1430,17 +1430,13 @@ void qmp_drive_backup(const char *device, const char *target, Error **errp) { BlockDriverState *bs; - BlockDriverState *target_bs; + BlockDriverState *target_bs, *source; BlockDriver *drv = NULL; Error *local_err = NULL; int flags; int64_t size; int ret; - if (sync != MIRROR_SYNC_MODE_FULL) { - error_setg(errp, "only sync mode 'full' is currently supported"); - return; - } if (!has_speed) { speed = 0; } @@ -1483,6 +1479,13 @@ void qmp_drive_backup(const char *device, const char *target, flags = bs->open_flags | BDRV_O_RDWR; + /* See if we have a backing HD we can use to create our new image + * on top of. */ + source = bs->backing_hd; + if (!source && sync == MIRROR_SYNC_MODE_TOP) { + sync = MIRROR_SYNC_MODE_FULL; + } + size = bdrv_getlength(bs); if (size < 0) { error_setg_errno(errp, -size, "bdrv_getlength failed"); @@ -1491,8 +1494,14 @@ void qmp_drive_backup(const char *device, const char *target, if (mode != NEW_IMAGE_MODE_EXISTING) { assert(format && drv); - bdrv_img_create(target, format, - NULL, NULL, NULL, size, flags, &local_err, false); + if (sync == MIRROR_SYNC_MODE_TOP) { + bdrv_img_create(target, format, source->filename, + source->drv->format_name, NULL, + size, flags, &local_err, false); + } else { + bdrv_img_create(target, format, NULL, NULL, NULL, + size, flags, &local_err, false); + } } if (error_is_set(&local_err)) { @@ -1508,7 +1517,7 @@ void qmp_drive_backup(const char *device, const char *target, return; } - backup_start(bs, target_bs, speed, on_source_error, on_target_error, + backup_start(bs, target_bs, speed, sync, on_source_error, on_target_error, block_job_cb, bs, &local_err); if (local_err != NULL) { bdrv_delete(target_bs); diff --git a/include/block/block_int.h b/include/block/block_int.h index c6ac871..e45f2a0 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -404,6 +404,7 @@ void mirror_start(BlockDriverState *bs, BlockDriverState *target, * @bs: Block device to operate on. * @target: Block device to write to. * @speed: The maximum speed, in bytes per second, or 0 for unlimited. + * @sync_mode: What parts of the disk image should be copied to the destination. * @on_source_error: The action to take upon error reading from the source. * @on_target_error: The action to take upon error writing to the target. * @cb: Completion function for the job. @@ -413,7 +414,8 @@ void mirror_start(BlockDriverState *bs, BlockDriverState *target, * until the job is cancelled or manually completed. */ void backup_start(BlockDriverState *bs, BlockDriverState *target, - int64_t speed, BlockdevOnError on_source_error, + int64_t speed, MirrorSyncMode sync_mode, + BlockdevOnError on_source_error, BlockdevOnError on_target_error, BlockDriverCompletionFunc *cb, void *opaque, Error **errp); diff --git a/qmp-commands.hx b/qmp-commands.hx index 65a9e26..2e59b0d 100644 --- a/qmp-commands.hx +++ b/qmp-commands.hx @@ -960,6 +960,7 @@ Arguments: Example: -> { "execute": "drive-backup", "arguments": { "device": "drive0", + "sync": "full", "target": "backup.img" } } <- { "return": {} } EQMP