From patchwork Mon Oct 21 01:14:15 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: mrhines@linux.vnet.ibm.com X-Patchwork-Id: 285056 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id F035B2C012B for ; Mon, 21 Oct 2013 12:15:54 +1100 (EST) Received: from localhost ([::1]:37891 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VY464-00047k-Ni for incoming@patchwork.ozlabs.org; Sun, 20 Oct 2013 21:15:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50491) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VY45C-0003p4-Jx for qemu-devel@nongnu.org; Sun, 20 Oct 2013 21:15:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VY453-00046g-Bi for qemu-devel@nongnu.org; Sun, 20 Oct 2013 21:14:58 -0400 Received: from e9.ny.us.ibm.com ([32.97.182.139]:36108) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VY453-00046c-7I for qemu-devel@nongnu.org; Sun, 20 Oct 2013 21:14:49 -0400 Received: from /spool/local by e9.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Sun, 20 Oct 2013 21:14:49 -0400 Received: from d01dlp01.pok.ibm.com (9.56.250.166) by e9.ny.us.ibm.com (192.168.1.109) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Sun, 20 Oct 2013 21:14:46 -0400 Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by d01dlp01.pok.ibm.com (Postfix) with ESMTP id 66DF438C8045 for ; Sun, 20 Oct 2013 21:14:44 -0400 (EDT) Received: from d03av06.boulder.ibm.com (d03av06.boulder.ibm.com [9.17.195.245]) by b01cxnp23034.gho.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id r9L1Eikf9109798 for ; Mon, 21 Oct 2013 01:14:45 GMT Received: from d03av06.boulder.ibm.com (loopback [127.0.0.1]) by d03av06.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id r9L1HvpZ000594 for ; Sun, 20 Oct 2013 19:17:57 -0600 Received: from mahler.ibm.com ([9.80.101.39]) by d03av06.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id r9L1HikS032531; Sun, 20 Oct 2013 19:17:55 -0600 From: mrhines@linux.vnet.ibm.com To: qemu-devel@nongnu.org Date: Mon, 21 Oct 2013 01:14:15 +0000 Message-Id: <1382318062-6288-6-git-send-email-mrhines@linux.vnet.ibm.com> X-Mailer: git-send-email 1.8.1.2 In-Reply-To: <1382318062-6288-1-git-send-email-mrhines@linux.vnet.ibm.com> References: <1382318062-6288-1-git-send-email-mrhines@linux.vnet.ibm.com> X-TM-AS-MML: No X-Content-Scanned: Fidelis XPS MAILER x-cbid: 13102101-7182-0000-0000-000008D2169D X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] X-Received-From: 32.97.182.139 Cc: aliguori@us.ibm.com, quintela@redhat.com, owasserm@redhat.com, onom@us.ibm.com, abali@us.ibm.com, mrhines@us.ibm.com, gokul@us.ibm.com, pbonzini@redhat.com Subject: [Qemu-devel] [RFC PATCH v1: 05/12] migration: support custom page loading X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: "Michael R. Hines" Just as RDMA has custom routines for saving memory, this provides us with custom routines for loading memory. Micro-checkpointing needs this support in order to be able to handle loading of the latest checkpoint into memory as they are received from the network. Signed-off-by: Michael R. Hines --- arch_init.c | 17 ++++++++++++----- include/migration/migration.h | 12 ++++++++++-- include/migration/qemu-file.h | 16 ++++++++++++++-- savevm.c | 27 ++++++++++++++++++++++++--- 4 files changed, 60 insertions(+), 12 deletions(-) diff --git a/arch_init.c b/arch_init.c index b139512..9cf7d18 100644 --- a/arch_init.c +++ b/arch_init.c @@ -684,7 +684,7 @@ static int ram_save_block(QEMUFile *f, bool last_stage) /* In doubt sent page as normal */ bytes_sent = -1; ret = ram_control_save_page(f, block->offset, - offset, TARGET_PAGE_SIZE, &bytes_sent); + block->host, offset, TARGET_PAGE_SIZE, &bytes_sent); if (ret != RAM_SAVE_CONTROL_NOT_SUPP) { if (ret != RAM_SAVE_CONTROL_DELAYED) { @@ -712,9 +712,11 @@ static int ram_save_block(QEMUFile *f, bool last_stage) /* XBZRLE overflow or normal page */ if (bytes_sent == -1) { bytes_sent = save_block_hdr(f, block, offset, cont, RAM_SAVE_FLAG_PAGE); - qemu_put_buffer_async(f, p, TARGET_PAGE_SIZE); - bytes_sent += TARGET_PAGE_SIZE; - acct_info.norm_pages++; + if (ret != RAM_SAVE_CONTROL_DELAYED) { + qemu_put_buffer_async(f, p, TARGET_PAGE_SIZE); + bytes_sent += TARGET_PAGE_SIZE; + acct_info.norm_pages++; + } } /* if page is unmodified, continue to the next */ @@ -1133,13 +1135,18 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id) ram_handle_compressed(host, ch, TARGET_PAGE_SIZE); } else if (flags & RAM_SAVE_FLAG_PAGE) { void *host; + int r; host = host_from_stream_offset(f, addr, flags); if (!host) { return -EINVAL; } - qemu_get_buffer(f, host, TARGET_PAGE_SIZE); + r = ram_control_load_page(f, host, TARGET_PAGE_SIZE); + + if (r == RAM_LOAD_CONTROL_NOT_SUPP) { + qemu_get_buffer(f, host, TARGET_PAGE_SIZE); + } } else if (flags & RAM_SAVE_FLAG_XBZRLE) { void *host = host_from_stream_offset(f, addr, flags); if (!host) { diff --git a/include/migration/migration.h b/include/migration/migration.h index 3ad06c5..ac1b438 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -169,9 +169,17 @@ void ram_control_load_hook(QEMUFile *f, uint64_t flags); #define RAM_SAVE_CONTROL_NOT_SUPP -1000 #define RAM_SAVE_CONTROL_DELAYED -2000 +#define RAM_LOAD_CONTROL_NOT_SUPP -3000 +#define RAM_LOAD_CONTROL_DELAYED -4000 -size_t ram_control_save_page(QEMUFile *f, ram_addr_t block_offset, - ram_addr_t offset, size_t size, +#define RDMA_CONTROL_VERSION_CURRENT 1 + +int ram_control_save_page(QEMUFile *f, ram_addr_t block_offset, + uint8_t *host_addr, + ram_addr_t offset, long size, int *bytes_sent); +int ram_control_load_page(QEMUFile *f, + void *host_addr, + long size); #endif diff --git a/include/migration/qemu-file.h b/include/migration/qemu-file.h index 0f757fb..d396b40 100644 --- a/include/migration/qemu-file.h +++ b/include/migration/qemu-file.h @@ -76,12 +76,22 @@ typedef int (QEMURamHookFunc)(QEMUFile *f, void *opaque, uint64_t flags); * This function allows override of where the RAM page * is saved (such as RDMA, for example.) */ -typedef size_t (QEMURamSaveFunc)(QEMUFile *f, void *opaque, +typedef int (QEMURamSaveFunc)(QEMUFile *f, void *opaque, ram_addr_t block_offset, + uint8_t *host_addr, ram_addr_t offset, - size_t size, + long size, int *bytes_sent); +/* + * This function allows override of where the RAM page + * is saved (such as RDMA, for example.) + */ +typedef int (QEMURamLoadFunc)(QEMUFile *f, + void *opaque, + void *host_addr, + long size); + typedef struct QEMUFileOps { QEMUFilePutBufferFunc *put_buffer; QEMUFileGetBufferFunc *get_buffer; @@ -92,12 +102,14 @@ typedef struct QEMUFileOps { QEMURamHookFunc *after_ram_iterate; QEMURamHookFunc *hook_ram_load; QEMURamSaveFunc *save_page; + QEMURamLoadFunc *load_page; } QEMUFileOps; QEMUFile *qemu_fopen_ops(void *opaque, const QEMUFileOps *ops); QEMUFile *qemu_fopen(const char *filename, const char *mode); QEMUFile *qemu_fdopen(int fd, const char *mode); QEMUFile *qemu_fopen_socket(int fd, const char *mode); +QEMUFile *qemu_fopen_mc(void *opaque, const char *mode); QEMUFile *qemu_popen_cmd(const char *command, const char *mode); int qemu_get_fd(QEMUFile *f); int qemu_fclose(QEMUFile *f); diff --git a/savevm.c b/savevm.c index 2f631d4..05f8a05 100644 --- a/savevm.c +++ b/savevm.c @@ -661,14 +661,17 @@ void ram_control_load_hook(QEMUFile *f, uint64_t flags) } } -size_t ram_control_save_page(QEMUFile *f, ram_addr_t block_offset, - ram_addr_t offset, size_t size, int *bytes_sent) +int ram_control_save_page(QEMUFile *f, ram_addr_t block_offset, + uint8_t *host_addr, + ram_addr_t offset, long size, int *bytes_sent) { if (f->ops->save_page) { int ret = f->ops->save_page(f, f->opaque, block_offset, + host_addr, offset, size, bytes_sent); - if (ret != RAM_SAVE_CONTROL_DELAYED) { + if (ret != RAM_SAVE_CONTROL_DELAYED + && ret != RAM_SAVE_CONTROL_NOT_SUPP) { if (bytes_sent && *bytes_sent > 0) { qemu_update_position(f, *bytes_sent); } else if (ret < 0) { @@ -682,6 +685,24 @@ size_t ram_control_save_page(QEMUFile *f, ram_addr_t block_offset, return RAM_SAVE_CONTROL_NOT_SUPP; } +int ram_control_load_page(QEMUFile *f, void *host_addr, long size) +{ + if (f->ops->load_page) { + int ret = f->ops->load_page(f, f->opaque, host_addr, size); + + if (ret != RAM_LOAD_CONTROL_DELAYED + && ret != RAM_LOAD_CONTROL_NOT_SUPP) { + if (ret < 0) { + qemu_file_set_error(f, ret); + } + } + + return ret; + } + + return RAM_LOAD_CONTROL_NOT_SUPP; +} + static void qemu_fill_buffer(QEMUFile *f) { int len;