From patchwork Tue Feb 18 08:50:23 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: mrhines@linux.vnet.ibm.com X-Patchwork-Id: 321329 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id BC74E2C00BB for ; Tue, 18 Feb 2014 20:29:08 +1100 (EST) Received: from localhost ([::1]:47205 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WFgTu-0000vL-Sa for incoming@patchwork.ozlabs.org; Tue, 18 Feb 2014 03:56:46 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52172) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WFgOe-0000n5-FP for qemu-devel@nongnu.org; Tue, 18 Feb 2014 03:51:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WFgOV-0001go-H2 for qemu-devel@nongnu.org; Tue, 18 Feb 2014 03:51:20 -0500 Received: from e31.co.us.ibm.com ([32.97.110.149]:52883) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WFgOV-0001gh-8r for qemu-devel@nongnu.org; Tue, 18 Feb 2014 03:51:11 -0500 Received: from /spool/local by e31.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 18 Feb 2014 01:51:10 -0700 Received: from d03dlp01.boulder.ibm.com (9.17.202.177) by e31.co.us.ibm.com (192.168.1.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 18 Feb 2014 01:51:08 -0700 Received: from b03cxnp08026.gho.boulder.ibm.com (b03cxnp08026.gho.boulder.ibm.com [9.17.130.18]) by d03dlp01.boulder.ibm.com (Postfix) with ESMTP id B4D421FF003E for ; Tue, 18 Feb 2014 01:51:07 -0700 (MST) Received: from d03av05.boulder.ibm.com (d03av05.boulder.ibm.com [9.17.195.85]) by b03cxnp08026.gho.boulder.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id s1I8oi1C4850094 for ; Tue, 18 Feb 2014 09:50:44 +0100 Received: from d03av05.boulder.ibm.com (localhost [127.0.0.1]) by d03av05.boulder.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id s1I8p60Y006767 for ; Tue, 18 Feb 2014 01:51:07 -0700 Received: from mrhinesdev.crl.ibm.com ([9.186.105.50]) by d03av05.boulder.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id s1I8oYNC004973; Tue, 18 Feb 2014 01:51:02 -0700 From: mrhines@linux.vnet.ibm.com To: qemu-devel@nongnu.org Date: Tue, 18 Feb 2014 16:50:23 +0800 Message-Id: <1392713429-18201-7-git-send-email-mrhines@linux.vnet.ibm.com> X-Mailer: git-send-email 1.8.1.2 In-Reply-To: <1392713429-18201-1-git-send-email-mrhines@linux.vnet.ibm.com> References: <1392713429-18201-1-git-send-email-mrhines@linux.vnet.ibm.com> X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 14021808-8236-0000-0000-00000754F573 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] X-Received-From: 32.97.110.149 Cc: GILR@il.ibm.com, SADEKJ@il.ibm.com, pbonzini@redhat.com, quintela@redhat.com, EREZH@il.ibm.com, owasserm@redhat.com, junqing.wang@cs2c.com.cn, onom@us.ibm.com, abali@us.ibm.com, isaku.yamahata@gmail.com, gokul@us.ibm.com, dbulkow@gmail.com, hinesmr@cn.ibm.com, BIRAN@il.ibm.com, lig.fnst@cn.fujitsu.com, "Michael R. Hines" Subject: [Qemu-devel] [RFC PATCH v2 06/12] mc: introduce state machine changes for MC X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: "Michael R. Hines" This patch sets up the initial changes to the migration state machine and prototypes to be used by the checkpointing code to interact with the state machine so that we can later handle failure and recovery scenarios. Signed-off-by: Michael R. Hines --- arch_init.c | 29 ++++++++++++++++++++++++----- include/migration/migration.h | 2 ++ migration.c | 37 +++++++++++++++++++++---------------- 3 files changed, 47 insertions(+), 21 deletions(-) diff --git a/arch_init.c b/arch_init.c index db75120..e9d4d9e 100644 --- a/arch_init.c +++ b/arch_init.c @@ -658,13 +658,13 @@ static void ram_migration_cancel(void *opaque) migration_end(); } -static void reset_ram_globals(void) +static void reset_ram_globals(bool reset_bulk_stage) { last_seen_block = NULL; last_sent_block = NULL; last_offset = 0; last_version = ram_list.version; - ram_bulk_stage = true; + ram_bulk_stage = reset_bulk_stage; } #define MAX_WAIT 50 /* ms, half buffered_file limit */ @@ -674,6 +674,15 @@ static int ram_save_setup(QEMUFile *f, void *opaque) RAMBlock *block; int64_t ram_pages = last_ram_offset() >> TARGET_PAGE_BITS; + /* + * RAM stays open during micro-checkpointing for the next transaction. + */ + if (migration_is_mc(migrate_get_current())) { + qemu_mutex_lock_ramlist(); + reset_ram_globals(false); + goto skip_setup; + } + migration_bitmap = bitmap_new(ram_pages); bitmap_set(migration_bitmap, 0, ram_pages); migration_dirty_pages = ram_pages; @@ -710,12 +719,14 @@ static int ram_save_setup(QEMUFile *f, void *opaque) qemu_mutex_lock_iothread(); qemu_mutex_lock_ramlist(); bytes_transferred = 0; - reset_ram_globals(); + reset_ram_globals(true); memory_global_dirty_log_start(); migration_bitmap_sync(); qemu_mutex_unlock_iothread(); +skip_setup: + qemu_put_be64(f, ram_bytes_total() | RAM_SAVE_FLAG_MEM_SIZE); QTAILQ_FOREACH(block, &ram_list.blocks, next) { @@ -744,7 +755,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) qemu_mutex_lock_ramlist(); if (ram_list.version != last_version) { - reset_ram_globals(); + reset_ram_globals(true); } ram_control_before_iterate(f, RAM_CONTROL_ROUND); @@ -825,7 +836,15 @@ static int ram_save_complete(QEMUFile *f, void *opaque) } ram_control_after_iterate(f, RAM_CONTROL_FINISH); - migration_end(); + + /* + * Only cleanup at the end of normal migrations + * or if the MC destination failed and we got an error. + * Otherwise, we are (or will soon be) in MIG_STATE_CHECKPOINTING. + */ + if(!migrate_use_mc() || migration_has_failed(migrate_get_current())) { + migration_end(); + } qemu_mutex_unlock_ramlist(); qemu_put_be64(f, RAM_SAVE_FLAG_EOS); diff --git a/include/migration/migration.h b/include/migration/migration.h index a7c54fe..e876a2c 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -101,7 +101,9 @@ int migrate_fd_close(MigrationState *s); void add_migration_state_change_notifier(Notifier *notify); void remove_migration_state_change_notifier(Notifier *notify); +bool migration_is_active(MigrationState *); bool migration_in_setup(MigrationState *); +bool migration_is_mc(MigrationState *s); bool migration_has_finished(MigrationState *); bool migration_has_failed(MigrationState *); MigrationState *migrate_get_current(void); diff --git a/migration.c b/migration.c index 25add6f..f42dae4 100644 --- a/migration.c +++ b/migration.c @@ -36,16 +36,6 @@ do { } while (0) #endif -enum { - MIG_STATE_ERROR = -1, - MIG_STATE_NONE, - MIG_STATE_SETUP, - MIG_STATE_CANCELLING, - MIG_STATE_CANCELLED, - MIG_STATE_ACTIVE, - MIG_STATE_COMPLETED, -}; - #define MAX_THROTTLE (32 << 20) /* Migration speed throttling */ /* Amount of time to allocate to each "chunk" of bandwidth-throttled @@ -273,7 +263,7 @@ void qmp_migrate_set_capabilities(MigrationCapabilityStatusList *params, MigrationState *s = migrate_get_current(); MigrationCapabilityStatusList *cap; - if (s->state == MIG_STATE_ACTIVE || s->state == MIG_STATE_SETUP) { + if (migration_is_active(s)) { error_set(errp, QERR_MIGRATION_ACTIVE); return; } @@ -285,7 +275,13 @@ void qmp_migrate_set_capabilities(MigrationCapabilityStatusList *params, /* shared migration helpers */ -static void migrate_set_state(MigrationState *s, int old_state, int new_state) +bool migration_is_active(MigrationState *s) +{ + return (s->state == MIG_STATE_ACTIVE) || s->state == MIG_STATE_SETUP + || s->state == MIG_STATE_CHECKPOINTING; +} + +void migrate_set_state(MigrationState *s, int old_state, int new_state) { if (atomic_cmpxchg(&s->state, old_state, new_state) == new_state) { trace_migrate_set_state(new_state); @@ -309,7 +305,7 @@ static void migrate_fd_cleanup(void *opaque) s->file = NULL; } - assert(s->state != MIG_STATE_ACTIVE); + assert(!migration_is_active(s)); if (s->state != MIG_STATE_COMPLETED) { qemu_savevm_state_cancel(); @@ -356,7 +352,12 @@ void remove_migration_state_change_notifier(Notifier *notify) bool migration_in_setup(MigrationState *s) { - return s->state == MIG_STATE_SETUP; + return s->state == MIG_STATE_SETUP; +} + +bool migration_is_mc(MigrationState *s) +{ + return s->state == MIG_STATE_CHECKPOINTING; } bool migration_has_finished(MigrationState *s) @@ -419,7 +420,8 @@ void qmp_migrate(const char *uri, bool has_blk, bool blk, params.shared = has_inc && inc; if (s->state == MIG_STATE_ACTIVE || s->state == MIG_STATE_SETUP || - s->state == MIG_STATE_CANCELLING) { + s->state == MIG_STATE_CANCELLING + || s->state == MIG_STATE_CHECKPOINTING) { error_set(errp, QERR_MIGRATION_ACTIVE); return; } @@ -624,7 +626,10 @@ static void *migration_thread(void *opaque) } if (!qemu_file_get_error(s->file)) { - migrate_set_state(s, MIG_STATE_ACTIVE, MIG_STATE_COMPLETED); + if (!migrate_use_mc()) { + migrate_set_state(s, + MIG_STATE_ACTIVE, MIG_STATE_COMPLETED); + } break; } }