From patchwork Thu Jun 27 22:44:53 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: mrhines@linux.vnet.ibm.com X-Patchwork-Id: 255237 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 491902C02EA for ; Fri, 28 Jun 2013 08:46:24 +1000 (EST) Received: from localhost ([::1]:56630 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UsKxK-0006Ja-6d for incoming@patchwork.ozlabs.org; Thu, 27 Jun 2013 18:46:22 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41554) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UsKwa-0006FK-NF for qemu-devel@nongnu.org; Thu, 27 Jun 2013 18:45:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UsKwY-0002vx-5W for qemu-devel@nongnu.org; Thu, 27 Jun 2013 18:45:36 -0400 Received: from e9.ny.us.ibm.com ([32.97.182.139]:53760) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UsKwY-0002vc-0k for qemu-devel@nongnu.org; Thu, 27 Jun 2013 18:45:34 -0400 Received: from /spool/local by e9.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 27 Jun 2013 18:45:33 -0400 Received: from d01dlp02.pok.ibm.com (9.56.250.167) by e9.ny.us.ibm.com (192.168.1.109) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 27 Jun 2013 18:45:31 -0400 Received: from d01relay06.pok.ibm.com (d01relay06.pok.ibm.com [9.56.227.116]) by d01dlp02.pok.ibm.com (Postfix) with ESMTP id A32D86E803A for ; Thu, 27 Jun 2013 18:45:26 -0400 (EDT) Received: from d01av05.pok.ibm.com (d01av05.pok.ibm.com [9.56.224.195]) by d01relay06.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id r5RMj0uV55508994 for ; Thu, 27 Jun 2013 18:45:00 -0400 Received: from d01av05.pok.ibm.com (loopback [127.0.0.1]) by d01av05.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id r5RMj0xx010256 for ; Thu, 27 Jun 2013 18:45:00 -0400 Received: from mrhinesdev.klabtestbed.com (klinux.watson.ibm.com [9.2.208.21]) by d01av05.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id r5RMj0Wl010198; Thu, 27 Jun 2013 18:45:00 -0400 From: mrhines@linux.vnet.ibm.com To: qemu-devel@nongnu.org Date: Thu, 27 Jun 2013 18:44:53 -0400 Message-Id: <1372373098-5877-2-git-send-email-mrhines@linux.vnet.ibm.com> X-Mailer: git-send-email 1.7.10.4 In-Reply-To: <1372373098-5877-1-git-send-email-mrhines@linux.vnet.ibm.com> References: <1372373098-5877-1-git-send-email-mrhines@linux.vnet.ibm.com> X-TM-AS-MML: No x-cbid: 13062722-7182-0000-0000-0000078F64ED X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] X-Received-From: 32.97.182.139 Cc: aliguori@us.ibm.com, quintela@redhat.com, knoel@redhat.com, owasserm@redhat.com, abali@us.ibm.com, mrhines@us.ibm.com, gokul@us.ibm.com, pbonzini@redhat.com, chegu_vinod@hp.com Subject: [Qemu-devel] [PATCH 1/6] rdma: update documentation to reflect new unpin support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: "Michael R. Hines" As requested, the protocol now includes memory unpinning support. This has been implemented in a non-optimized manner, in such a way that one could devise an LRU or other workload-specific information on top of the basic mechanism to influence the way unpinning happens during runtime. The feature is not yet user-facing, and is thus can only be enable at compile-time. Signed-off-by: Michael R. Hines --- docs/rdma.txt | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/docs/rdma.txt b/docs/rdma.txt index 45a4b1d..c842da4 100644 --- a/docs/rdma.txt +++ b/docs/rdma.txt @@ -204,15 +204,17 @@ observations on the maximum future benefit of simultaneous page registrations. The 'type' field has 10 different command values: 1. Unused - 2. Error (sent to the source during bad things) - 3. Ready (control-channel is available) - 4. QEMU File (for sending non-live device state) - 5. RAM Blocks request (used right after connection setup) - 6. RAM Blocks result (used right after connection setup) - 7. Compress page (zap zero page and skip registration) - 8. Register request (dynamic chunk registration) - 9. Register result ('rkey' to be used by sender) - 10. Register finished (registration for current iteration finished) + 2. Error (sent to the source during bad things) + 3. Ready (control-channel is available) + 4. QEMU File (for sending non-live device state) + 5. RAM Blocks request (used right after connection setup) + 6. RAM Blocks result (used right after connection setup) + 7. Compress page (zap zero page and skip registration) + 8. Register request (dynamic chunk registration) + 9. Register result ('rkey' to be used by sender) + 10. Register finished (registration for current iteration finished) + 11. Unregister request (unpin previously registered memory) + 12. Unregister finished (confirmation that unpin completed) A single control message, as hinted above, can contain within the data portion an array of many commands of the same type. If there is more than @@ -413,3 +415,6 @@ TODO: the use of KSM and ballooning while using RDMA. 4. Also, some form of balloon-device usage tracking would also help alleviate some issues. +5. Use LRU or workload-specific information to provide more + fine-grained direction of UNREGISTER requests for unpinning + memory in an overcommitted environment.