From patchwork Wed Apr 10 13:13:59 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: mrhines@linux.vnet.ibm.com X-Patchwork-Id: 235389 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id C43912C00BB for ; Wed, 10 Apr 2013 23:14:47 +1000 (EST) Received: from localhost ([::1]:39711 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UPurN-0002kH-Vq for incoming@patchwork.ozlabs.org; Wed, 10 Apr 2013 09:14:45 -0400 Received: from eggs.gnu.org ([208.118.235.92]:52941) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UPuqr-0002Ri-LM for qemu-devel@nongnu.org; Wed, 10 Apr 2013 09:14:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UPuqj-0001Oo-4K for qemu-devel@nongnu.org; Wed, 10 Apr 2013 09:14:13 -0400 Received: from e7.ny.us.ibm.com ([32.97.182.137]:47829) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UPuqi-0001N4-RH for qemu-devel@nongnu.org; Wed, 10 Apr 2013 09:14:04 -0400 Received: from /spool/local by e7.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 10 Apr 2013 09:14:04 -0400 Received: from d01dlp02.pok.ibm.com (9.56.250.167) by e7.ny.us.ibm.com (192.168.1.107) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 10 Apr 2013 09:14:02 -0400 Received: from d01relay04.pok.ibm.com (d01relay04.pok.ibm.com [9.56.227.236]) by d01dlp02.pok.ibm.com (Postfix) with ESMTP id 16AD16E803C for ; Wed, 10 Apr 2013 09:13:58 -0400 (EDT) Received: from d01av05.pok.ibm.com (d01av05.pok.ibm.com [9.56.224.195]) by d01relay04.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id r3ADE0mR280000 for ; Wed, 10 Apr 2013 09:14:00 -0400 Received: from d01av05.pok.ibm.com (loopback [127.0.0.1]) by d01av05.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id r3ADDxoG003841 for ; Wed, 10 Apr 2013 09:13:59 -0400 Received: from [172.16.1.222] (klinux.watson.ibm.com [9.2.208.21]) by d01av05.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id r3ADDxTf003826; Wed, 10 Apr 2013 09:13:59 -0400 Message-ID: <51656597.1060601@linux.vnet.ibm.com> Date: Wed, 10 Apr 2013 09:13:59 -0400 From: "Michael R. Hines" User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130106 Thunderbird/17.0.2 MIME-Version: 1.0 To: Orit Wasserman References: <1365568180-19593-1-git-send-email-mrhines@linux.vnet.ibm.com> <1365568180-19593-5-git-send-email-mrhines@linux.vnet.ibm.com> <51655F78.5040500@redhat.com> In-Reply-To: <51655F78.5040500@redhat.com> X-TM-AS-MML: No X-Content-Scanned: Fidelis XPS MAILER x-cbid: 13041013-5806-0000-0000-000020AB6CDE X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] X-Received-From: 32.97.182.137 Cc: aliguori@us.ibm.com, mst@redhat.com, qemu-devel@nongnu.org, abali@us.ibm.com, mrhines@us.ibm.com, gokul@us.ibm.com, pbonzini@redhat.com Subject: Re: [Qemu-devel] [RFC PATCH RDMA support v6: 4/7] Introduce two new capabilities X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org On 04/10/2013 08:47 AM, Orit Wasserman wrote: > On 04/10/2013 07:29 AM, mrhines@linux.vnet.ibm.com wrote: >> From: "Michael R. Hines" >> >> RDMA performs very slowly with zero-page checking. >> Without the ability to disable it, RDMA throughput and >> latency promises and high performance links cannot be >> fully realized. >> >> On the other hand, dynamic page registration support is also >> included in the RDMA protocol. This second capability also >> cannot be fully realized without the ability to enable zero >> page scanning. >> >> So, we have two new capabilities which work together: >> >> 1. migrate_set_capability check_for_zero on|off (default on) > This is not a capability but a migration parameter, the command > should be named migrate_check_for_zero (on|off) and you will need to add a flag > to MigrationParams (and it should be in a separate patch as Paolo already requested) I'm happy to convert to using a command, but is this not consistent with XBZRLE the capability? The XBZRLE capability does not affect the destination either - but it is implemented as a capability. Am I misundertanding? - Michael >> 2. migrate_set_capability chunk_register_destination on|off (default off) > This is indeed a capability as it effect both source and destination Acknowledged. void qemu_put_buffer_async(QEMUFile *f, const uint8_t *buf, int size); +void qemu_file_set_error(QEMUFile *f, int ret); + +void qemu_rdma_cleanup(void *opaque); +int qemu_rdma_close(void *opaque); +int qemu_rdma_get_fd(void *opaque); +int qemu_rdma_get_buffer(void *opaque, uint8_t *buf, int64_t pos, int size); +int qemu_rdma_put_buffer(void *opaque, const uint8_t *buf, + int64_t pos, int size); +bool qemu_file_mode_is_not_valid(const char * mode); + +extern const QEMUFileOps rdma_read_ops; +extern const QEMUFileOps rdma_write_ops; static inline void qemu_put_ubyte(QEMUFile *f, unsigned int v) { diff --git a/migration.c b/migration.c index 3b4b467..875cee3 100644 --- a/migration.c +++ b/migration.c @@ -66,6 +66,7 @@ MigrationState *migrate_get_current(void) .state = MIG_STATE_SETUP, .bandwidth_limit = MAX_THROTTLE, .xbzrle_cache_size = DEFAULT_MIGRATE_CACHE_SIZE, + .enabled_capabilities[MIGRATION_CAPABILITY_CHECK_FOR_ZERO] = true, }; return ¤t_migration; @@ -77,6 +78,10 @@ void qemu_start_incoming_migration(const char *uri, Error **errp) if (strstart(uri, "tcp:", &p)) tcp_start_incoming_migration(p, errp); +#ifdef CONFIG_RDMA + else if (strstart(uri, "rdma:", &p)) + rdma_start_incoming_migration(p, errp); +#endif #if !defined(WIN32) else if (strstart(uri, "exec:", &p)) exec_start_incoming_migration(p, errp); @@ -120,8 +125,10 @@ void process_incoming_migration(QEMUFile *f) Coroutine *co = qemu_coroutine_create(process_incoming_migration_co); int fd = qemu_get_fd(f); - assert(fd != -1); - qemu_set_nonblock(fd); + if(fd != -2) { /* rdma returns -2 */ + assert(fd != -1); + qemu_set_nonblock(fd); + } qemu_coroutine_enter(co, f); } @@ -405,6 +412,10 @@ void qmp_migrate(const char *uri, bool has_blk, bool blk, if (strstart(uri, "tcp:", &p)) { tcp_start_outgoing_migration(s, p, &local_err); +#ifdef CONFIG_RDMA + } else if (strstart(uri, "rdma:", &p)) { + rdma_start_outgoing_migration(s, p, &local_err); +#endif #if !defined(WIN32) } else if (strstart(uri, "exec:", &p)) { exec_start_outgoing_migration(s, p, &local_err); @@ -474,6 +485,24 @@ void qmp_migrate_set_downtime(double value, Error **errp) max_downtime = (uint64_t)value; } +bool migrate_chunk_register_destination(void) +{ + MigrationState *s; + + s = migrate_get_current(); + + return s->enabled_capabilities[MIGRATION_CAPABILITY_CHUNK_REGISTER_DESTINATION]; +} + +bool migrate_check_for_zero(void) +{ + MigrationState *s; + + s = migrate_get_current(); + + return s->enabled_capabilities[MIGRATION_CAPABILITY_CHECK_FOR_ZERO]; +} + int migrate_use_xbzrle(void) { MigrationState *s; diff --git a/qapi-schema.json b/qapi-schema.json index db542f6..7ebcf99 100644 --- a/qapi-schema.json +++ b/qapi-schema.json @@ -602,7 +602,7 @@ # Since: 1.2 ## { 'enum': 'MigrationCapability', - 'data': ['xbzrle'] } + 'data': ['xbzrle', 'check_for_zero', 'chunk_register_destination'] } ## # @MigrationCapabilityStatus