From patchwork Thu Oct 26 20:08:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 1855974 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=GlIcAV57; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SGcLw5x08z23jc for ; Fri, 27 Oct 2023 07:08:56 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qw6eo-00047T-JT; Thu, 26 Oct 2023 16:08:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qw6en-000473-H0 for qemu-devel@nongnu.org; Thu, 26 Oct 2023 16:08:41 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qw6ek-0008S7-OA for qemu-devel@nongnu.org; Thu, 26 Oct 2023 16:08:41 -0400 Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 39QJsVAs008639; Thu, 26 Oct 2023 20:08:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id; s=corp-2023-03-30; bh=Id7o2gxtwrhMgbJ56TJfp4F8gdrOfMwZsRTL7YNUTDw=; b=GlIcAV57IZX0vaVca9adkgqTIERxWZ+E7ccc+a3JIXBooHTDcC30dxc0V9bK8rBASKOo qPoeLD5Ewk9eWexELf5tB1PavQwvsko6KPDejHmNlAnwAiWNa3VPG3aJnnKl25A7u3Ur IbZlFgBSYtLZ6gBmfkumUoipIipV0p3OingKho/FuBytZNyD21LBxKDUToz8oeEdUK// 5zsr+QvYAGY7kJG7ymSizm8YZ61Alcqlw+cZcDfSAb53G46MTmvHquf6bY0AamIy5Cfg JtmMoI/pSOjTx9hcMp3bDdfr+H1MACmnJh6O9mk9Cw/L8sAtkiYL4uNW0vp3ZjXErqfg tQ== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3tywtt04k0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 26 Oct 2023 20:08:35 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 39QImm7W009159; Thu, 26 Oct 2023 20:08:34 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3tywqhtym2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 26 Oct 2023 20:08:34 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 39QK8YLl036361; Thu, 26 Oct 2023 20:08:34 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3tywqhtykn-1; Thu, 26 Oct 2023 20:08:34 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Juan Quintela , Peter Xu , Fabiano Rosas , Leonardo Bras , "Daniel P. Berrange" , Steve Sistare Subject: [PATCH V2 0/6] Live Update reboot mode Date: Thu, 26 Oct 2023 13:08:27 -0700 Message-Id: <1698350913-415926-1-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.987,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-10-26_19,2023-10-26_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 suspectscore=0 adultscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2310240000 definitions=main-2310260174 X-Proofpoint-GUID: 6AEtm3dTwKTv3g1JFXovjTA0H2ucgxBQ X-Proofpoint-ORIG-GUID: 6AEtm3dTwKTv3g1JFXovjTA0H2ucgxBQ Received-SPF: pass client-ip=205.220.165.32; envelope-from=steven.sistare@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Add a mode migration parameter that can be used to select alternate migration algorithms. The default mode is normal, representing the current migration algorithm, and does not need to be explicitly set. Provide the cpr-reboot (CheckPoint and Restart) migration mode for live update, which saves state to a file. This allows one to quit qemu, reboot to an updated kernel, install an updated version of qemu, and resume via the migrate-incoming command. The caller must specify a migration URI that writes to and reads from a file, and must set the mode parameter before invoking the migrate or migrate-incoming commands. Unlike normal mode, the use of certain local storage options does not block cpr-reboot mode, but the caller must not modify guest block devices between the quit and restart. To avoid saving guest RAM to the state file, the memory backend must be shared, and the @x-ignore-shared migration capability must be set. Guest RAM must be non-volatile across reboot, which can be achieved by backing it with a dax device, or /dev/shm PKRAM as proposed in https://lore.kernel.org/lkml/1617140178-8773-1-git-send-email-anthony.yznaga@oracle.com but this is not enforced. The restarted qemu arguments must match those used to initially start qemu, plus the -incoming option. This patch series contains minimal functionality. Future patches will enhance reboot mode by preserving vfio devices for suspended guests. They will also add a new mode for updating qemu using the exec system call, which will keep vfio devices and certain character devices alive. Here is an example of updating the host kernel using reboot mode. window 1 | window 2 | # qemu-system-$arch -monitor stdio | mem-path=/dev/dax0.0 ... | QEMU 8.1.50 monitor - type 'help' for more info | (qemu) info status | VM status: running | | # yum update kernel-uek (qemu) migrate_set_capability x-ignore-shared on| (qemu) migrate_set_parameter mode cpr-reboot | (qemu) migrate -d file:vm.state | (qemu) info status | VM status: paused (postmigrate) | (qemu) quit | | # systemctl kexec | kexec_core: Starting new kernel | ... | | # qemu-system-$arch -monitor stdio | mem-path=/dev/dax0.0 -incoming defer ... | QEMU 8.1.50 monitor - type 'help' for more info | (qemu) info status | VM status: paused (inmigrate) | (qemu) migrate_set_capability x-ignore-shared on| (qemu) migrate_set_parameter mode cpr-reboot | (qemu) migrate_incoming file:vm.state | (qemu) info status | VM status: running | Changes in V2: - moved "migration mode" code and comments to more appropriate places - clarified the behavior of non-shared-memory backends - split blocker patches and reverted some blockers - added a test Steve Sistare (6): migration: mode parameter migration: per-mode blockers cpr: relax blockdev migration blockers cpr: relax vhost migration blockers cpr: reboot mode tests/qtest: migration: add reboot mode test block/parallels.c | 2 +- block/qcow.c | 2 +- block/vdi.c | 2 +- block/vhdx.c | 2 +- block/vmdk.c | 2 +- block/vpc.c | 2 +- block/vvfat.c | 2 +- hw/core/qdev-properties-system.c | 14 ++++++ hw/scsi/vhost-scsi.c | 2 +- hw/virtio/vhost.c | 2 +- include/hw/qdev-properties-system.h | 4 ++ include/migration/blocker.h | 44 +++++++++++++++-- include/migration/misc.h | 1 + migration/migration-hmp-cmds.c | 9 ++++ migration/migration.c | 95 ++++++++++++++++++++++++++++++++----- migration/options.c | 21 ++++++++ migration/options.h | 1 + qapi/migration.json | 40 ++++++++++++++-- stubs/migr-blocker.c | 10 ++++ tests/qtest/migration-test.c | 27 +++++++++++ 20 files changed, 255 insertions(+), 29 deletions(-)