From patchwork Fri Jan 26 15:59:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Henrique Barboza X-Patchwork-Id: 866502 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zSly20HTlz9t2c for ; Sat, 27 Jan 2018 04:19:22 +1100 (AEDT) Received: from localhost ([::1]:40198 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ef7eq-0006YS-4l for incoming@patchwork.ozlabs.org; Fri, 26 Jan 2018 12:19:20 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51388) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ef7eH-0006Xd-Gg for qemu-devel@nongnu.org; Fri, 26 Jan 2018 12:18:46 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ef7eC-0006VC-Dv for qemu-devel@nongnu.org; Fri, 26 Jan 2018 12:18:45 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:38922 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ef7eC-0006Uh-8G for qemu-devel@nongnu.org; Fri, 26 Jan 2018 12:18:40 -0500 Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w0QFx7hJ123934 for ; Fri, 26 Jan 2018 10:59:57 -0500 Received: from e17.ny.us.ibm.com (e17.ny.us.ibm.com [129.33.205.207]) by mx0b-001b2d01.pphosted.com with ESMTP id 2fr5cvw82h-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Fri, 26 Jan 2018 10:59:56 -0500 Received: from localhost by e17.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 26 Jan 2018 10:59:56 -0500 Received: from b01cxnp22035.gho.pok.ibm.com (9.57.198.25) by e17.ny.us.ibm.com (146.89.104.204) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Fri, 26 Jan 2018 10:59:54 -0500 Received: from b01ledav005.gho.pok.ibm.com (b01ledav005.gho.pok.ibm.com [9.57.199.110]) by b01cxnp22035.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w0QFxr2648234496; Fri, 26 Jan 2018 15:59:53 GMT Received: from b01ledav005.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4C830AE03C; Fri, 26 Jan 2018 11:01:09 -0500 (EST) Received: from localhost.localdomain (unknown [9.80.210.41]) by b01ledav005.gho.pok.ibm.com (Postfix) with ESMTP id 4D203AE03B; Fri, 26 Jan 2018 11:01:08 -0500 (EST) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Date: Fri, 26 Jan 2018 13:59:40 -0200 X-Mailer: git-send-email 2.14.3 X-TM-AS-GCONF: 00 x-cbid: 18012615-0040-0000-0000-000003E9A067 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00008432; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000248; SDB=6.00980685; UDB=6.00497164; IPR=6.00759986; BA=6.00005797; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00019226; XFM=3.00000015; UTC=2018-01-26 15:59:54 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18012615-0041-0000-0000-000007DF08FF Message-Id: <20180126155940.26994-1-danielhb@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2018-01-26_09:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1801260210 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 Subject: [Qemu-devel] [PATCH v2 1/1] migration/savevm.c: set MAX_VM_CMD_PACKAGED_SIZE to 1ul << 32 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Henrique Barboza , dgilbert@redhat.com, quintela@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" MAX_VM_CMD_PACKAGED_SIZE is a constant used in qemu_savevm_send_packaged and loadvm_handle_cmd_packaged to determine whether a package is too big to be sent or received. qemu_savevm_send_packaged is called inside postcopy_start (migration/migration.c) to send the MigrationState in a single blob to the destination, using the MIG_CMD_PACKAGED subcommand, which will read it up using loadvm_handle_cmd_packaged. If the blob is larger than MAX_VM_CMD_PACKAGED_SIZE, an error is thrown and the postcopy migration is aborted. Both MAX_VM_CMD_PACKAGED_SIZE and MIG_CMD_PACKAGED were introduced by commit 11cf1d984b ("MIG_CMD_PACKAGED: Send a packaged chunk ..."). The constant has its original value of 1ul << 24 (16MB). The current MAX_VM_CMD_PACKAGED_SIZE value is not enough to support postcopy migration of bigger pseries guests. The blob size for a postcopy migration of a pseries guest with the following setup: qemu-system-ppc64 --nographic -vga none -machine pseries,accel=kvm -m 64G \ -smp 1,maxcpus=32 -device virtio-blk-pci,drive=rootdisk \ -drive file=f27.qcow2,if=none,cache=none,format=qcow2,id=rootdisk \ -netdev user,id=u1 -net nic,netdev=u1 Goes around 12MB. Bumping the RAM to 128G makes the blob sizes goes to 20MB. With 256G the blob goes to 37MB - more than twice the current maximum size. At this moment the pseries machine can handle guests with up to 1TB of RAM, making this postcopy blob goes to 128MB of size approximately. Following the discussions made in [1], there is a need to understand what devices are aggressively consuming the blob in that manner and see if that can be mitigated. Until then, we can set MAX_VM_CMD_PACKAGED_SIZE to the maximum value allowed. Since the size is a 32 bit int variable, we can set it as 1ul << 32, giving a maximum blob size of 4G that is enough to support postcopy migration of 32TB RAM guests given the above constraints. [1] https://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg06313.html Signed-off-by: Daniel Henrique Barboza Reported-by: Balamuruhan S Reviewed-by: Juan Quintela --- migration/savevm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/savevm.c b/migration/savevm.c index b7908f62be..cff679fff8 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -81,7 +81,7 @@ enum qemu_vm_cmd { MIG_CMD_MAX }; -#define MAX_VM_CMD_PACKAGED_SIZE (1ul << 24) +#define MAX_VM_CMD_PACKAGED_SIZE (1ul << 32) static struct mig_cmd_args { ssize_t len; /* -1 = variable */ const char *name;