From patchwork Mon Aug 28 15:18:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 1826821 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=Nvidia.com header.i=@Nvidia.com header.a=rsa-sha256 header.s=selector2 header.b=CuA2lEtt; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RZDql5xK3z1yZs for ; Tue, 29 Aug 2023 01:24:19 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qae67-0007Id-P8; Mon, 28 Aug 2023 11:24:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qae60-0007H0-8E for qemu-devel@nongnu.org; Mon, 28 Aug 2023 11:24:04 -0400 Received: from mail-dm6nam12on2077.outbound.protection.outlook.com ([40.107.243.77] helo=NAM12-DM6-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qae5w-0007rp-Ho for qemu-devel@nongnu.org; Mon, 28 Aug 2023 11:24:04 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=g3B7hmjuPeR/zbbodQ8wogQb/WBNd7iYzuWg/QkvOZhiVZYXVLi1GAokK6gOI3yRtbepWWWS1e2UlMxbbNYVYIEWAanaWmKCSFxxQUIHz3IKniwdw5auaFe9gYVRmWgmYTcz7iiqq/l2bhZW7G8O/V+h+5O92Sh9QilGaYRZMeUsEPt6l/NtnH++BpN5QLltdL5MO900ioUEBE1GtwgcgUv3hx4fUXb4aLXu15UOda5S1Tra/LwjK57vaMazxK/39yoB1E2FT9w0OCahHJx6d8xDKq6S4byLTsx5KfauhMmrzYpCDikDIKJK0DmSMBhu1p8Klle0gLvkYjVNDNOBDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3HcdPQkHk1jWmoUgP28yTld8g7ZvkZY6IvCwyw/irso=; b=LssD9zuWROZEvV9SW/ZxjRCC6qCVroD9IMeYUA8hCT50hbUAUsFz194S25E2cv7hFfL/FEk7u7XjfgP7Iz4nThQlgKFnuWy5uJVp2ZvFGiK2Ukx61avUFzusy/FcsPioNJXWjVOuG7Yt/FCjKuC/YCUeH9dw5Xv7RCVVBtFheJwB/3ZQfxvzfNy2d6e53pRAXx9B32XgVxHdP1SPqYs7Ekwl/Pnp5RF+o0Gxrj4nhF8EMHW1nmIgYCDvwg+PwFUFZZOu+SU3RDCZ1O27syadL3G8tXjEIDUkn/NlueqfRU1/GYsu6MdYl68NdDB3WEMVbiFcFa/3HK+PJPcA12YphA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3HcdPQkHk1jWmoUgP28yTld8g7ZvkZY6IvCwyw/irso=; b=CuA2lEttUcQ8Td9XfZIl3ezjgFJMwJFdxJ8ztqi8e9UcZjkNqFdcN+KjAxFNnvcUy0748CUOjlFxfzFcF5C2qF012Pcc6ud7ehYW+JJ6qsu5+gOHFpt+eQpNFLWUvJfU0DkeyqvcOgVN4YeE0DVTLftV36xCbnpas3oIqio+4HNL8vTWvFDudCf5KSb1jdOjg3Pi/6wW4Ov8rqE6q5P62QOlnJ1CdLOSMqwzNb4/fuw/lhak6L3GvHKatjknaHxmzzrzoBruSo1nrUi2kgwzZxN0DuHFhhroVzsv4oK6vbr44ZcnSa475LB8/WnDVJ5sKdXu22bbl6QZdjRjZ04D/w== Received: from DS7PR06CA0021.namprd06.prod.outlook.com (2603:10b6:8:2a::23) by SJ2PR12MB9211.namprd12.prod.outlook.com (2603:10b6:a03:55e::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.34; Mon, 28 Aug 2023 15:18:56 +0000 Received: from CY4PEPF0000EE3A.namprd03.prod.outlook.com (2603:10b6:8:2a:cafe::5d) by DS7PR06CA0021.outlook.office365.com (2603:10b6:8:2a::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.35 via Frontend Transport; Mon, 28 Aug 2023 15:18:56 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CY4PEPF0000EE3A.mail.protection.outlook.com (10.167.242.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.16 via Frontend Transport; Mon, 28 Aug 2023 15:18:55 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Mon, 28 Aug 2023 08:18:48 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Mon, 28 Aug 2023 08:18:47 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.37 via Frontend Transport; Mon, 28 Aug 2023 08:18:45 -0700 From: Avihai Horon To: CC: Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_G?= =?utf-8?q?oater?= , Juan Quintela , Peter Xu , Leonardo Bras , Yanghang Liu , Avihai Horon Subject: [PATCH 1/6] migration: Add migration prefix to functions in target.c Date: Mon, 28 Aug 2023 18:18:37 +0300 Message-ID: <20230828151842.11303-2-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20230828151842.11303-1-avihaih@nvidia.com> References: <20230828151842.11303-1-avihaih@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE3A:EE_|SJ2PR12MB9211:EE_ X-MS-Office365-Filtering-Correlation-Id: efb84bc1-c231-4683-75d6-08dba7da1880 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2WVGeWWWg62djozyLSGVgObUGoXt2aYVLCbgJ3UHMBBFLyUZgbMDQtSyNvyRp0cgxvEesYuOB2nCG8W34Fmeva+IF2gY9DbbjKyhkBlEs8H9Nqx2itU3Q3p7e4wdC7tUM5/dEbcZPRLf0nGc5mqqZguxgFYEexwFUuWSt4cidNdIHT+9FG+8wWCWy9JRLKz8fkJgEuKEsdijn0LQGK6fZ5UQIDvvYc8/xYH2hcj/Vy/9vwh/s/hJZ+zvpjXP87RyrBv9ubP0illbWKDQEXc4FNG3dN0im/xNgNGhniAFoNrK6PzlcqaXKjNCJTxi7KKTJI6Bfw1SSdzAyPIeX/npxMlonfsAmQ++5YtSMuHMIHjoJeyl5bk2ab+wcQMt8tx5zesPlugzuwJrynSgekosyY9oVTkHssiZZjJ7hDK5nbazpe3xKh1T7K4RuzsKmMEM60Pl50iDOatK9bLfT99ThYU2+Ya9Bw4L1YYJ+olkeAgUWmtoKbDbXAcSUzox7OZ2uBgsOIXkbpx9TpE4VxBWYhJ1Qj6xoe3/e2Lzg4uZ3r4ZgWN8R1f7jrLMbU4cB+5IsdA/EQY5+lhcmzFsT29ooMZW4krvo8d1ZPgtQe4rn9F3Mb5bxp5qxxuayBSr6eglV0rkY+bLQr3pFNN/1pBhOD6PRxO2kGdGsu8IiuZHFgCkw+Bk1uPNE2kmrb+33shzfqGYgE7K+LRaXGhUpXUNcRNu6kNCJ1S6M22VJvmM5c9yjfVz1V3Wi9CyF8tYmk57 X-Forefront-Antispam-Report: CIP:216.228.118.233; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge2.nvidia.com; CAT:NONE; SFS:(13230031)(4636009)(136003)(376002)(346002)(39860400002)(396003)(186009)(1800799009)(82310400011)(451199024)(36840700001)(46966006)(40470700004)(36860700001)(40460700003)(41300700001)(356005)(7636003)(82740400003)(6666004)(86362001)(83380400001)(478600001)(47076005)(2616005)(426003)(26005)(336012)(107886003)(1076003)(7696005)(40480700001)(70206006)(70586007)(36756003)(54906003)(6916009)(2906002)(316002)(5660300002)(8676002)(8936002)(4326008); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Aug 2023 15:18:55.7900 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: efb84bc1-c231-4683-75d6-08dba7da1880 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.233]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EE3A.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB9211 Received-SPF: softfail client-ip=40.107.243.77; envelope-from=avihaih@nvidia.com; helo=NAM12-DM6-obe.outbound.protection.outlook.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URG_BIZ=0.573 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The functions in target.c are not static, yet they don't have a proper migration prefix. Add such prefix. Signed-off-by: Avihai Horon Reviewed-by: Cédric Le Goater --- migration/migration.h | 4 ++-- migration/migration.c | 6 +++--- migration/savevm.c | 2 +- migration/target.c | 8 ++++---- 4 files changed, 10 insertions(+), 10 deletions(-) diff --git a/migration/migration.h b/migration/migration.h index 6eea18db36..c5695de214 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -512,8 +512,8 @@ void migration_consume_urgent_request(void); bool migration_rate_limit(void); void migration_cancel(const Error *error); -void populate_vfio_info(MigrationInfo *info); -void reset_vfio_bytes_transferred(void); +void migration_populate_vfio_info(MigrationInfo *info); +void migration_reset_vfio_bytes_transferred(void); void postcopy_temp_page_reset(PostcopyTmpPage *tmp_page); #endif diff --git a/migration/migration.c b/migration/migration.c index 5528acb65e..92866a8f49 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1039,7 +1039,7 @@ static void fill_source_migration_info(MigrationInfo *info) populate_time_info(info, s); populate_ram_info(info, s); populate_disk_info(info); - populate_vfio_info(info); + migration_populate_vfio_info(info); break; case MIGRATION_STATUS_COLO: info->has_status = true; @@ -1048,7 +1048,7 @@ static void fill_source_migration_info(MigrationInfo *info) case MIGRATION_STATUS_COMPLETED: populate_time_info(info, s); populate_ram_info(info, s); - populate_vfio_info(info); + migration_populate_vfio_info(info); break; case MIGRATION_STATUS_FAILED: info->has_status = true; @@ -1641,7 +1641,7 @@ static bool migrate_prepare(MigrationState *s, bool blk, bool blk_inc, */ memset(&mig_stats, 0, sizeof(mig_stats)); memset(&compression_counters, 0, sizeof(compression_counters)); - reset_vfio_bytes_transferred(); + migration_reset_vfio_bytes_transferred(); return true; } diff --git a/migration/savevm.c b/migration/savevm.c index a2cb8855e2..5bf8b59a7d 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1622,7 +1622,7 @@ static int qemu_savevm_state(QEMUFile *f, Error **errp) migrate_init(ms); memset(&mig_stats, 0, sizeof(mig_stats)); memset(&compression_counters, 0, sizeof(compression_counters)); - reset_vfio_bytes_transferred(); + migration_reset_vfio_bytes_transferred(); ms->to_dst_file = f; qemu_mutex_unlock_iothread(); diff --git a/migration/target.c b/migration/target.c index f39c9a8d88..a6ffa9a5ce 100644 --- a/migration/target.c +++ b/migration/target.c @@ -15,7 +15,7 @@ #endif #ifdef CONFIG_VFIO -void populate_vfio_info(MigrationInfo *info) +void migration_populate_vfio_info(MigrationInfo *info) { if (vfio_mig_active()) { info->vfio = g_malloc0(sizeof(*info->vfio)); @@ -23,16 +23,16 @@ void populate_vfio_info(MigrationInfo *info) } } -void reset_vfio_bytes_transferred(void) +void migration_reset_vfio_bytes_transferred(void) { vfio_reset_bytes_transferred(); } #else -void populate_vfio_info(MigrationInfo *info) +void migration_populate_vfio_info(MigrationInfo *info) { } -void reset_vfio_bytes_transferred(void) +void migration_reset_vfio_bytes_transferred(void) { } #endif From patchwork Mon Aug 28 15:18:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 1826804 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=Nvidia.com header.i=@Nvidia.com header.a=rsa-sha256 header.s=selector2 header.b=OEOaWHcv; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RZDlC16LFz1yZs for ; Tue, 29 Aug 2023 01:20:23 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qae1N-0002By-0g; Mon, 28 Aug 2023 11:19:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qae1A-00025e-Q3 for qemu-devel@nongnu.org; Mon, 28 Aug 2023 11:19:06 -0400 Received: from mail-dm6nam10on2060a.outbound.protection.outlook.com ([2a01:111:f400:7e88::60a] helo=NAM10-DM6-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qae18-0006v6-8N for qemu-devel@nongnu.org; Mon, 28 Aug 2023 11:19:04 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=b/KwiBxRWxGiKDADlUHG0Aymq+JKws8qv3GBZuZBH9HI2JlnKpLz35tOG+ykOeeu6dnSR9hlEWeSea5DvYqTZl//zp8KaVnXwSKVsLe5y8JAr7VqubfO3xQGgmXZcsMT9K/nyRWNyI9yUHTUcKP4v4MLKfMBXOud6sp+yzKtYuEskRAJbHPIdbAsNOcNwxarlgMC6glrPa0/2XRVATlSfwkiq0bf7+yhjy6XmTvYucjTlqmmIqHtYQGsQF6fNMAr3WNbd2pBzlPPnqF/AbEnTjXC9fNX+vMjfs7F1h9N4sEDbh7VcvXdjshS9URzDSZLikZm5LgqkmG1jn6w/m/dLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tZfdw4/Q5yuFIGY0xGxMOXoPWdrRx8tuok7uG7FFrnY=; b=gOps7YGbVVdgaZFP3BE5YtCApFfWQfi/0ft9ikmvs8UuWbPsoj+3LyWbGWJurV3JEbLM70Bq4yHJjDGKpu43I71SK/Qveev2eYjZWCVHymOzgFDk8G6TTikQAXwuqO46YHQWQ8yQug/iD3XIDTXoguB58qVVRuq67VjMSqq1L6M8XDdSmsbk+5/Us+n09DPE1S+5yXbnjJ6II+U/lIUc1IhpsDa6K5TmDmVUx1wGXI+eAfvky0+PAwNUP5CNSHNLGax+AmTtEZL/roSf/xNADw2NZRnJlG18F35/NfQurC2X0kVcPp4eRhKzR+AOvM/hSO0nSHSjFSR//sZzJxR7Cg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tZfdw4/Q5yuFIGY0xGxMOXoPWdrRx8tuok7uG7FFrnY=; b=OEOaWHcvY4tDVWWdCyVilVHORX8uMZ+6CNH6yliaKCxvZk8MHC7OEB6El3+DJrlrAQuWx13dFprQYyrsifIxn1reR2+/uw2C/f1NyPyS5bK3f7G5LwHo9y5t5my9P37fQZex336mXUdLV6JQIga198Vkgnf/H9hsBLosU6QPXswckg21oPIvvWvEXJMlhXEYzG53th8yBcB3OZxj1GHz7TsQphHzzn7rLK3dVhcHyIRqJNgwxm1+YX8h4mdi6z4wlXgmDf4u7NDeEzka7bq/OA/l/KKO431suzss9a4IMhF/5FN3Q3S5+Unh1OFPdXWXlb7lYXVRLxV6eFfFf5wFbg== Received: from CY8PR10CA0047.namprd10.prod.outlook.com (2603:10b6:930:4b::25) by LV3PR12MB9266.namprd12.prod.outlook.com (2603:10b6:408:21b::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.34; Mon, 28 Aug 2023 15:18:59 +0000 Received: from CY4PEPF0000EE3B.namprd03.prod.outlook.com (2603:10b6:930:4b:cafe::7b) by CY8PR10CA0047.outlook.office365.com (2603:10b6:930:4b::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.34 via Frontend Transport; Mon, 28 Aug 2023 15:18:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CY4PEPF0000EE3B.mail.protection.outlook.com (10.167.242.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.17 via Frontend Transport; Mon, 28 Aug 2023 15:18:58 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Mon, 28 Aug 2023 08:18:50 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail201.nvidia.com (10.126.190.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Mon, 28 Aug 2023 08:18:50 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.37 via Frontend Transport; Mon, 28 Aug 2023 08:18:48 -0700 From: Avihai Horon To: CC: Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_G?= =?utf-8?q?oater?= , Juan Quintela , Peter Xu , Leonardo Bras , Yanghang Liu , Avihai Horon Subject: [PATCH 2/6] vfio/migration: Fail adding device with enable-migration=on and existing blocker Date: Mon, 28 Aug 2023 18:18:38 +0300 Message-ID: <20230828151842.11303-3-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20230828151842.11303-1-avihaih@nvidia.com> References: <20230828151842.11303-1-avihaih@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE3B:EE_|LV3PR12MB9266:EE_ X-MS-Office365-Filtering-Correlation-Id: ea4fe5ac-b258-4c78-dbf2-08dba7da1a5f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: MbduWWFaoHPfhYJE0Dla1VM6YXd1Kh4PNnS3ZRnu7YDXB0XCyrvMPnUQmYd4SZSVWPUgCkIehthsieirIKEAiZOtaT+2aGQ6PLb6JM8yGaGBHqqJo64FEQxyejbwZrydivkxJqb4Cm9c9O1vrr5PKLTfMTa3ByF4g8cSShL9HhzaZDRcO3ABpy0GSXyy2Kuz2sWW2Zo7VQgJeeh+5F3azQu3TbFRc2s2GCNcFcDJ3In9ksojvz2IOhU7o0BqwXK+kmypLZf1AUwCwaY8FSN1s3BSnCLc3whjD3fZnWuCTFpG8VYhnsdY6zi5zQuz8TnvjFapoetaqwOpleOxKZ0+ZO3B5KLbpWYLSokq+boFyJvogZs3vAWrGcsBIZfsQQA9nxD05ukFavZI0Q1ItMa/1syJkdawSsbi+pqKOpHWefN/IqQp7DPmXflxnSIc5qic9F4W00HrN+iOClxAb2OmIbQsEHMxVl34dUBjIZQhXxWReNsPBJK3SEpYTMLs9J9MqLkOIBUXL+Nd7Suz/ij3QZOyHo8+FHMj1mCb/n+ctmMYN2v7y/WXh0pe8FncAQABj1cstyR/IKnu5uV13EmcXYGltALS/za6BN73N+pJRAVDbpQE2bFodTaWzcdo3HVKcb1nP25VEltJKcd07PY3hGNeeRSL+NSw9B9lU739AiosTP5rsIx83dY2Ukta+WOWXaFwW8fYSxzdOW60Kr3nPyv2AkrB42anhUQrFnOsJUk= X-Forefront-Antispam-Report: CIP:216.228.118.233; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge2.nvidia.com; CAT:NONE; SFS:(13230031)(4636009)(346002)(396003)(136003)(39860400002)(376002)(451199024)(1800799009)(82310400011)(186009)(36840700001)(46966006)(40470700004)(83380400001)(478600001)(7636003)(356005)(82740400003)(26005)(426003)(336012)(47076005)(36860700001)(107886003)(1076003)(2616005)(40480700001)(7696005)(6666004)(86362001)(2906002)(316002)(4326008)(6916009)(70206006)(8676002)(70586007)(41300700001)(8936002)(54906003)(36756003)(5660300002)(40460700003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Aug 2023 15:18:58.9318 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ea4fe5ac-b258-4c78-dbf2-08dba7da1a5f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.233]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EE3B.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR12MB9266 Received-SPF: softfail client-ip=2a01:111:f400:7e88::60a; envelope-from=avihaih@nvidia.com; helo=NAM10-DM6-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org If a device with enable-migration=on is added and it causes a migration blocker, adding the device should fail with a proper error. This is not the case with multiple device migration blocker when the blocker already exists. If the blocker already exists and a device with enable-migration=on is added which causes a migration blocker, adding the device will succeed. Fix it by failing adding the device in such case. Fixes: 8bbcb64a71d8 ("vfio/migration: Make VFIO migration non-experimental") Signed-off-by: Avihai Horon Reviewed-by: Cédric Le Goater --- hw/vfio/common.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 8a8d074e18..237101d038 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -394,8 +394,7 @@ int vfio_block_multiple_devices_migration(VFIODevice *vbasedev, Error **errp) { int ret; - if (multiple_devices_migration_blocker || - vfio_multiple_devices_migration_is_supported()) { + if (vfio_multiple_devices_migration_is_supported()) { return 0; } @@ -405,6 +404,10 @@ int vfio_block_multiple_devices_migration(VFIODevice *vbasedev, Error **errp) return -EINVAL; } + if (multiple_devices_migration_blocker) { + return 0; + } + error_setg(&multiple_devices_migration_blocker, "Multiple VFIO devices migration is supported only if all of " "them support P2P migration"); From patchwork Mon Aug 28 15:18:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 1826802 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=Nvidia.com header.i=@Nvidia.com header.a=rsa-sha256 header.s=selector2 header.b=mlP6KwbX; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RZDkl119wz26jK for ; Tue, 29 Aug 2023 01:19:59 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qae1R-0002RL-ET; Mon, 28 Aug 2023 11:19:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qae1G-00029q-1B for qemu-devel@nongnu.org; Mon, 28 Aug 2023 11:19:10 -0400 Received: from mail-bn8nam11on20601.outbound.protection.outlook.com ([2a01:111:f400:7eae::601] helo=NAM11-BN8-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qae1D-0006vZ-G4 for qemu-devel@nongnu.org; Mon, 28 Aug 2023 11:19:09 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lmWdtcqfHvI311InJ06ZJcsPwrakmV5nCNSC+CItnkkmUZDmR1Jf4u94Wvmi1YuDtadOor4cq+6ZZfZbXCMUlHbnnUiMiPheVE6KFl81PCad21HJzbhyh5IqxPOJ7uGMHI28xTDetDr5THM1GfmpDrjiAGPAleRAMWDjLi05tnMmfNyg5mhUmmFmd5ZlomxtY/mqdys8oQv4g85C6TQt/CXUJv8O8vNJR4N9alpzmmht/FAbiz710xAQM12LjjlXcK/iSyl0gMxdfhVfYhJXQ8cAXd/NiWxiO7+FTTHKuB/tVWgwu6xftCUNrVNEFCHPpzTIq4kgrOYmoK29mDY9Kg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ku1KexpIv4gEfzcN4VFmXzotJNNzhBJ7AosgGlJ/DfY=; b=afA1MkGvsyo3WlkD/TerKCYzLpIjaWdmqk1U8wxu5zeac58idq7NBeubyps6+iaxfj8dwDynWhY6dWqFVxuUOCDuoE1C/o+rc1MvcgMwiGvYhRJlp9IBzrZkwTmYthUfcmmIIuchsQ3q/Il80dwZMeY7jlT/7ofuXJ15hbPmPn5EgokiiwwvfOBJf/dDLe4uzIpYb2RTffHdd0BsZjP70r3yYO0A2xq9MDZ8PnhBXWvtwsmZnhpHLqfxiKsZB0SziSW6VWLgsu2yBOkPUMlVhDFRndql0BK4RVwJG3eM2/aNoaXCGi5kRQfPxy/kOFLNoPG7i8LzJFb6/oCaqut2xw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ku1KexpIv4gEfzcN4VFmXzotJNNzhBJ7AosgGlJ/DfY=; b=mlP6KwbXbePa0BR2E6ebeo8ALmLMsAKm0IXK7xnchjuzXhUgcMKZrwXn3pGpmQeEJlRqSyo7gTs668YXqHtM8xYajAYJtvrAKI2akqv3HiyCb1wQyzto0eYYsn/i3UhOwE3UMivSEyifjdX3iqyfBxyxxgTQImdy932elChvXtL2lIPZOgRFvAgMbJ1BuSljoJokJHUGglAtkYKlZ7FdgxEDUHDX6rp83enSoYvTOg8KoORpr9hbGQk0oTKto/EOtRZYeXLw2R51FK1SS1+acWfBwHSO9Aw13aoqQp2BiNBRqkGW6GZhGhh9aOjwSv7oeW0XchiifcVfEz0aDhBtwQ== Received: from BYAPR11CA0092.namprd11.prod.outlook.com (2603:10b6:a03:f4::33) by PH7PR12MB7939.namprd12.prod.outlook.com (2603:10b6:510:278::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.34; Mon, 28 Aug 2023 15:19:02 +0000 Received: from MWH0EPF000989E8.namprd02.prod.outlook.com (2603:10b6:a03:f4:cafe::d5) by BYAPR11CA0092.outlook.office365.com (2603:10b6:a03:f4::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.34 via Frontend Transport; Mon, 28 Aug 2023 15:19:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by MWH0EPF000989E8.mail.protection.outlook.com (10.167.241.135) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.17 via Frontend Transport; Mon, 28 Aug 2023 15:19:02 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Mon, 28 Aug 2023 08:18:53 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Mon, 28 Aug 2023 08:18:52 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.37 via Frontend Transport; Mon, 28 Aug 2023 08:18:50 -0700 From: Avihai Horon To: CC: Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_G?= =?utf-8?q?oater?= , Juan Quintela , Peter Xu , Leonardo Bras , Yanghang Liu , Avihai Horon Subject: [PATCH 3/6] vfio/migration: Add vfio_migratable_devices_num() Date: Mon, 28 Aug 2023 18:18:39 +0300 Message-ID: <20230828151842.11303-4-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20230828151842.11303-1-avihaih@nvidia.com> References: <20230828151842.11303-1-avihaih@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000989E8:EE_|PH7PR12MB7939:EE_ X-MS-Office365-Filtering-Correlation-Id: c7e4f00d-cdb5-4510-7426-08dba7da1c5e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: jdpH4Ah/18fWy7C1E+3ChpJ2xxjV/gtBJnPJwbBsLJ9hzOfgdxr1bjfW41009xvgwjpX72JOZ+lFyI9PC9Js2qg+5yi3Nl+yQwlcjBJRwBLeoT6VwInJYM6XNVRvPFwIuNNOZ2S3/aZ7bNpioYTZ0yv9TuS81rLHS0rEgPzmkuxFmw/sqYig/X6sqctnJmWWGB34KMb8BauHWjYTiM4SZlqf6uPUWBd3Hkrs/FQaz8XzhxDgv2eoZJtS8NV4QpMzuHUMsbtHxsa15ec/auvJHlyZCr1w8REyuUqPUWjTYh2OQluydB6Sk0XLhlgjlrOUOaBSgGBNjPPBbnjGUtqp4H6SW7D0HtHTTX779zje13NKV2AFRpBLIRSNQR+9CSyyQ2eO+O80LkJdWmUyi2folJUemHSlrXOSow2zeXeMhjamx6EL7afhK69d8HJyALaYNW/kmkPvV0gGPATnp1+yPSM1GX8Pc2dm+4ax4ZXIAe/aiCvL5axl/Z/PVEyxkAhsjKCcdISE6SivI93P0YVZW052MarOjvxZsptzFB44cLOcmqhX5haPD7anyPtzQmjIxO6+0LI/aK3SgLNtFU62zRLLJffDKgOEZBYoiNp8FUE0sKS8hHlR4jCtsLNW6xLPdlGUJ+PFwdteNhVpN/p1nWwe2NVMyT4YwkIIKaBpwB2F4RSzWd34q2ccyggXFDGhupfMnnSLx3dtBqFsfeN4MjlsNdrbcolJ4cvvuranrtY= X-Forefront-Antispam-Report: CIP:216.228.118.232; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge1.nvidia.com; CAT:NONE; SFS:(13230031)(4636009)(39860400002)(376002)(136003)(396003)(346002)(186009)(82310400011)(451199024)(1800799009)(40470700004)(46966006)(36840700001)(7696005)(40460700003)(6666004)(36756003)(40480700001)(86362001)(7636003)(82740400003)(356005)(36860700001)(47076005)(107886003)(2616005)(83380400001)(2906002)(426003)(336012)(26005)(1076003)(478600001)(70586007)(41300700001)(70206006)(8676002)(8936002)(5660300002)(316002)(54906003)(6916009)(4326008); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Aug 2023 15:19:02.2624 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c7e4f00d-cdb5-4510-7426-08dba7da1c5e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.232]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000989E8.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7939 Received-SPF: softfail client-ip=2a01:111:f400:7eae::601; envelope-from=avihaih@nvidia.com; helo=NAM11-BN8-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Add vfio_migratable_devices_num() function, which returns the number of VFIO devices that are using VFIO migration, and use it in vfio_multiple_devices_migration_is_supported(). This is done in preparation for next patches which will block VFIO migration with postcopy migration or background snapshot, as they are not compatible together. Signed-off-by: Avihai Horon Reviewed-by: Cédric Le Goater --- hw/vfio/common.c | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 237101d038..57a76feab1 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -363,6 +363,23 @@ bool vfio_mig_active(void) static Error *multiple_devices_migration_blocker; +static unsigned int vfio_migratable_devices_num(void) +{ + VFIOGroup *group; + VFIODevice *vbasedev; + unsigned int device_num = 0; + + QLIST_FOREACH(group, &vfio_group_list, next) { + QLIST_FOREACH(vbasedev, &group->device_list, next) { + if (vbasedev->migration) { + device_num++; + } + } + } + + return device_num; +} + /* * Multiple devices migration is allowed only if all devices support P2P * migration. Single device migration is allowed regardless of P2P migration @@ -372,14 +389,11 @@ static bool vfio_multiple_devices_migration_is_supported(void) { VFIOGroup *group; VFIODevice *vbasedev; - unsigned int device_num = 0; bool all_support_p2p = true; QLIST_FOREACH(group, &vfio_group_list, next) { QLIST_FOREACH(vbasedev, &group->device_list, next) { if (vbasedev->migration) { - device_num++; - if (!(vbasedev->migration->mig_flags & VFIO_MIGRATION_P2P)) { all_support_p2p = false; } @@ -387,7 +401,7 @@ static bool vfio_multiple_devices_migration_is_supported(void) } } - return all_support_p2p || device_num <= 1; + return all_support_p2p || vfio_migratable_devices_num() <= 1; } int vfio_block_multiple_devices_migration(VFIODevice *vbasedev, Error **errp) From patchwork Mon Aug 28 15:18:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 1826805 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=Nvidia.com header.i=@Nvidia.com header.a=rsa-sha256 header.s=selector2 header.b=QbtWsBYo; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RZDlH32zZz1yZs for ; Tue, 29 Aug 2023 01:20:27 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qae1N-0002HE-Md; Mon, 28 Aug 2023 11:19:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qae1H-0002Aq-Ul for qemu-devel@nongnu.org; Mon, 28 Aug 2023 11:19:12 -0400 Received: from mail-dm3nam02on20615.outbound.protection.outlook.com ([2a01:111:f400:7e83::615] helo=NAM02-DM3-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qae1E-0006va-FI for qemu-devel@nongnu.org; Mon, 28 Aug 2023 11:19:11 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ID+2CTLjjIfWGhYcxr1dfy9MCmzFD873ryQ1vvRgVBZGU4jb/7gI5sCw47WMU0tavKUJjASVC5/fl07xdZ6lCfz/zwLv1fF4M5pSFAV5fKogaLWWzJK23/4ePqkemfUrvMkLkuQABm2uHMzIFBlCkXMmUgYerUZUMQr4N5n4yH6z6vD49zbUYVZQHhe9jSM44ZaQ4jvd/9hQHxCJbTTXz4coi5rdQ4dLPuHG/VrBLbaApxu0DLcidkgkUdPIt8+Jzehxso91wyqHMFRTZ+d+XVH1hyn9twYPJE9sIC8hPdKDWcYIK/IXP7kl5UPM5jVD0p2nMCZKI1x0qoNSrCUckw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Kq11ZFvI+aodT+I4vYkl5Shj8KeJH+7llckm6cpZyTo=; b=Aoaq2MUHUqgymaz28helnCRBYYyDld0+4t58CpewDz809r7oyuwL1rrCzkDUr93C2ZYvPxHWm7coXprEVlQ5aoZAIjDTZgiigDHcUIoforg1pX3XmsQAy3gxsSyoo9aMQYMJu16Xy8rmA2/StnVJOUj3IplQhCig6QQNLFDC6xFZk9L6lP2c5xK+P6cY5Z9rICJjtiWXkAv9uMDbA9b/hKx+jVSmMK8C6aJXL1VTHZfeS6fK9wN63J6FrOoPmhL6RvHCumN+J14g9/EYFcrnYt4Co+DDb+qtxfq7twbM7/f5qgciwtqRtgJcnUdGUxdVuB55PHykMA9Udcvg4aJULw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Kq11ZFvI+aodT+I4vYkl5Shj8KeJH+7llckm6cpZyTo=; b=QbtWsBYoujLXx/SDm/0r5BVH/+4O3Yf5syNkALch5b8R57QCROF51/F/4TPzR0skpldpG4UXdCmJtnGPjwu9SKT5S7gX+6GSZ9FOMylXiAoBhKLUzXrDFWmrO4Y37clgkDatfKph41aHmfl/EOJqMMtAOp0vO92TpI10xEFQg0ZA5dj/iEaR9KI7ssP5+8rU3GAuqsexFK9muNESKwbZH2L6F0qjGBwCtD3XWxcec7t5X3l4NfWpmupGVzdk1d9p+ZsJFCtf/I4SPK7C2cMt0SJCWXcQ/ccg8t3QDQs8ecFmiXiO/pn6mQk3PK6AfwNNi20AHHialPUde3ezvqr0rQ== Received: from CY8PR10CA0035.namprd10.prod.outlook.com (2603:10b6:930:4b::20) by DM6PR12MB4107.namprd12.prod.outlook.com (2603:10b6:5:218::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.34; Mon, 28 Aug 2023 15:19:04 +0000 Received: from CY4PEPF0000EE3B.namprd03.prod.outlook.com (2603:10b6:930:4b:cafe::1c) by CY8PR10CA0035.outlook.office365.com (2603:10b6:930:4b::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.35 via Frontend Transport; Mon, 28 Aug 2023 15:19:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CY4PEPF0000EE3B.mail.protection.outlook.com (10.167.242.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.17 via Frontend Transport; Mon, 28 Aug 2023 15:19:04 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Mon, 28 Aug 2023 08:18:55 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Mon, 28 Aug 2023 08:18:55 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.37 via Frontend Transport; Mon, 28 Aug 2023 08:18:53 -0700 From: Avihai Horon To: CC: Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_G?= =?utf-8?q?oater?= , Juan Quintela , Peter Xu , Leonardo Bras , Yanghang Liu , Avihai Horon Subject: [PATCH 4/6] vfio/migration: Change vfio_mig_active() semantics Date: Mon, 28 Aug 2023 18:18:40 +0300 Message-ID: <20230828151842.11303-5-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20230828151842.11303-1-avihaih@nvidia.com> References: <20230828151842.11303-1-avihaih@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE3B:EE_|DM6PR12MB4107:EE_ X-MS-Office365-Filtering-Correlation-Id: 33556242-a02c-4536-3ed1-08dba7da1d8f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: v9OGkYid/LkDIQooZi6x0GyFn2qqfDg8EWL0ylaK4YIxEeKmqDHbyydOvcVaiP1YTcv1ioaa3JrKZzgwvTWgYwMAxfRFyQhILoFPfUzVW4P/1IwS0KA1BVFGnNtFLqz5bi5/5Otb3GCPE8skiD+dAhr2WU3Rmzaztv7yNGJuv0s6VqN7+BiZ33RJDwU9XA4aDEVYjbTIkhGnmKmoOFey65EfvLwShjRKirZ0csG1dp5bbH4iAuGQbaO6+R5KI4IP2vUKgquob6e6rGpCns3p7SNnUV66yr/RWHKmC5I2tOTD2Ig8FemkXrrUNWBKxsspv32eiJAU23Ew7MaJcurROe7C7vTc0acTwyi3cUc4Q216+HLjskmNX8rJQP8Ff/xVqfBv9x85wuf5WUNmaq7PRz7M3ZZjT7aegtuS3TqbXWw6Thrd/52Abgo/K8SC0G7CbnrJeOopl9x6ezOt78yIP9Wp5zqtYeomqwRsBtS2S+LegbjqV3E2gcbL67k8/IqdXYU7Kna37gVJK31ejgXRNpllGsoite6EScrfPgviq68eB9hVqL8fNfv581ZzJEgTuTm77oKGSeYGrNjOGIw0xyxLOd6AtbdRAOWsCT3aMfDM+gNSpC8cP0tniTltUia7zEiT2aWe+iFApLybB2j5Cnm06ned1h1t4vbC6fmJ9M6fU84UgxHRF2oHLFb1qjLmsaTjUJYy0PFsbyUe0oBb4akJXgxZFQvICqN5wqdhuAn/E4YWXtsZF9Sfji06wyEF X-Forefront-Antispam-Report: CIP:216.228.118.233; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge2.nvidia.com; CAT:NONE; SFS:(13230031)(4636009)(136003)(376002)(346002)(39860400002)(396003)(186009)(1800799009)(82310400011)(451199024)(36840700001)(46966006)(40470700004)(36860700001)(40460700003)(41300700001)(356005)(7636003)(82740400003)(6666004)(86362001)(83380400001)(478600001)(47076005)(2616005)(426003)(26005)(336012)(107886003)(1076003)(7696005)(40480700001)(70206006)(70586007)(36756003)(54906003)(6916009)(2906002)(316002)(5660300002)(8676002)(8936002)(4326008); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Aug 2023 15:19:04.2755 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 33556242-a02c-4536-3ed1-08dba7da1d8f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.233]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EE3B.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4107 Received-SPF: softfail client-ip=2a01:111:f400:7e83::615; envelope-from=avihaih@nvidia.com; helo=NAM02-DM3-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org vfio_mig_active() is used by migration_populate_vfio_info() to populate VFIO migration info when it is active. Currently, VFIO migration is considered active if there are VFIO devices and none of them has a migration blocker. Change that and consider VFIO migration to be active if there is a VFIO device that is using VFIO migration, regardless of whether a device has migration blocker or not. This is done in preparation for next patches which will block VFIO migration with postcopy migration or background snapshot, as they are not compatible together. It will allow adding a migration blocker for such cases even if the VFIO device already has a blocker. Note that migration_populate_vfio_info() still behaves correctly, as if there is a VFIO device with migration blocker, migration can't be started and thus migration_populate_vfio_info() will never be called. Signed-off-by: Avihai Horon --- hw/vfio/common.c | 24 +++++------------------- 1 file changed, 5 insertions(+), 19 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 57a76feab1..373f6e5932 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -342,25 +342,6 @@ static int vfio_bitmap_alloc(VFIOBitmap *vbmap, hwaddr size) static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, uint64_t size, ram_addr_t ram_addr); -bool vfio_mig_active(void) -{ - VFIOGroup *group; - VFIODevice *vbasedev; - - if (QLIST_EMPTY(&vfio_group_list)) { - return false; - } - - QLIST_FOREACH(group, &vfio_group_list, next) { - QLIST_FOREACH(vbasedev, &group->device_list, next) { - if (vbasedev->migration_blocker) { - return false; - } - } - } - return true; -} - static Error *multiple_devices_migration_blocker; static unsigned int vfio_migratable_devices_num(void) @@ -446,6 +427,11 @@ void vfio_unblock_multiple_devices_migration(void) multiple_devices_migration_blocker = NULL; } +bool vfio_mig_active(void) +{ + return vfio_migratable_devices_num(); +} + bool vfio_viommu_preset(VFIODevice *vbasedev) { return vbasedev->group->container->space->as != &address_space_memory; From patchwork Mon Aug 28 15:18:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 1826801 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=Nvidia.com header.i=@Nvidia.com header.a=rsa-sha256 header.s=selector2 header.b=tYm+96Ta; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RZDkk3twzz20b9 for ; Tue, 29 Aug 2023 01:19:58 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qae1O-0002Jn-R2; Mon, 28 Aug 2023 11:19:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qae1J-0002Bb-BK for qemu-devel@nongnu.org; Mon, 28 Aug 2023 11:19:13 -0400 Received: from mail-dm6nam10on20629.outbound.protection.outlook.com ([2a01:111:f400:7e88::629] helo=NAM10-DM6-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qae1G-0006vm-JL for qemu-devel@nongnu.org; Mon, 28 Aug 2023 11:19:13 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mw1mQ76OC7abUdTbEK7CJ1Qa0zc2RVRTv1BBXOhUWU9pibWnRAUXTBzn42JUm/2k+GZemx2TVCGQVRZnZA4RwVbe0CpknnKSiRoKexQomjBTj+uD4Nq8ZqSLefa9iTFOhZTvteiOlsUvH4RkHrtgqXRdvtaP1MPodfhHw68XTH4x7pKeqD8VWmGbJc9tnM0BnrfZtiJcoEvVYRw/i7LfgZkWZnI+ooFRmJadV5YWPkwuNTTsiMzIeyp89NTUZ2qBJpGIaQFhGCAcgaCy0VyMaGkjUvK5zA+rzeVD1WwafOv0l1/+jdRT5ScVpajhqmege2J70TG2hKRcXVzEwrQbQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AjFATC057FdvyFfLmeCrAsmdDhG+uHoBwHw9POt5f88=; b=S1TRPX7NVYln2+h1GHyjmguQF3F3uY9tIX0osVG0aB7ZBYskV+2KpodllqFDuRveVusZ749QNK9BkLhIgIuC/g6ZnSKnxM6fzluCPc//xsZk4ww0e2xtwpZxtcaO1+m1xQ4NlUV29DPhEcYkJWmw+fWYMvgb/97l9UWtZAYOfDa6Yr20YcPen/cnTQO5VG15+b7xe63PFpUff8LbjHnC9RxaxHD5WEojEF21Wc7tNpKVnd51WyhTDKp4gTUBbQY57X9h/6mQHjdWLK1Tg/iu/xuHi6gLzxXS4wbLsUc/MZApq0bdGez5UlFJ32vMX3V8tdpJ+O6HGKymnWjmVjQ1vQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AjFATC057FdvyFfLmeCrAsmdDhG+uHoBwHw9POt5f88=; b=tYm+96TaTBMga1Fi4goIxom7/CSecmjP5rHPT81bNhFw+aS6ZNfP8ReOceGhLObYagc3Vmza8bLgcNsCZ/zfl5IsieigkJxdqRavO2tLgC4x/znpETYoAUmBRGKl8BRGQ88DimNZHWz67V5UfmyMYvZ9ib5s3EkJ8CLu9ag50xeoGAsaueRzj/pYpW7aUZgqLpvIjflnDUgwQFaDPbdUZijMKKTe3YFGqcogCVyI+8IN0vzFZBtkc1H4nq3e9xPW9ig4GfryIkyzfahQUE13gRAzMqaflWqJNf9G4xUFnCyLvnsuiLrv/jxinJiLP/xWPjnOkfKSZYnjQl+P6Uh3Ew== Received: from MW4PR04CA0280.namprd04.prod.outlook.com (2603:10b6:303:89::15) by IA1PR12MB6092.namprd12.prod.outlook.com (2603:10b6:208:3ec::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.35; Mon, 28 Aug 2023 15:19:07 +0000 Received: from MWH0EPF000989E7.namprd02.prod.outlook.com (2603:10b6:303:89:cafe::79) by MW4PR04CA0280.outlook.office365.com (2603:10b6:303:89::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.35 via Frontend Transport; Mon, 28 Aug 2023 15:19:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by MWH0EPF000989E7.mail.protection.outlook.com (10.167.241.134) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.17 via Frontend Transport; Mon, 28 Aug 2023 15:19:06 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Mon, 28 Aug 2023 08:18:58 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Mon, 28 Aug 2023 08:18:57 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.37 via Frontend Transport; Mon, 28 Aug 2023 08:18:56 -0700 From: Avihai Horon To: CC: Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_G?= =?utf-8?q?oater?= , Juan Quintela , Peter Xu , Leonardo Bras , Yanghang Liu , Avihai Horon Subject: [PATCH 5/6] vfio/migration: Block VFIO migration with postcopy migration Date: Mon, 28 Aug 2023 18:18:41 +0300 Message-ID: <20230828151842.11303-6-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20230828151842.11303-1-avihaih@nvidia.com> References: <20230828151842.11303-1-avihaih@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000989E7:EE_|IA1PR12MB6092:EE_ X-MS-Office365-Filtering-Correlation-Id: 14e56f21-6e79-4099-d6dd-08dba7da1f00 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: jpwPijatn17bvbyl5CjG9A6sURQC59EV8HTePJx8vr7Kqy6xfsr6Ww8Q0OQvpZUe+EIM/YWbTyQPupXKeXHWSCIZ6nqv+yl7FjHZmCu2PQ/5t+O6VB+pNroW2MTi6WJCkh3fBNwqdVuZTr2xq5iuBpfr5ONvHEY3TwqbTO/cFicijXsTv7POtm34Lx+CmPzoHpXcm8ILMNIj7J1Ls5uxdM5BdUm/d+Qi7muXqNzndbBm2/INs+AbmsR6sqEDjUuY9uUq/h6Reo9uxCttxrcWpaeD9iBOrNP1XtfEZhxvRcGzJ43YDX+Z6EidwKncgvnQe6irI1CDorsBok9qZkKRGvq5RIZ6GWPrJdMQnkvZVVExiDyXiKWEVZdZQLy1qQjBprnNL+i+/sJ+LIiZvpQf2hfmIylAqf6i8/4qFCIuco8+KLAy0uSBrLxhZAEM8TGY5QLI/Smj3io52LuObwmzJfSHBjwfzC5KevMh59BSgNO9LXA0AWsTqiJF3YBWC6L7cbPwehhSmOCSlrVG6P0rqWhQH9V9uyJRICntRbG4c8RaUf8zJw5i9ll3FXhu1SizGeVkA78DGxdM6C+fiBmsTKxCEnPBi8BImqHrl30g46keqZ7pThuvYq1BRgXCd35MNJvzI519ks5PAPweOcKY2jkuhS9gJUYS59tTRbHXo/AL4TEI0u8ZhnLFFD5PIkYVnhrXCR70KwA9vnj/q+cpyCDKbOQl6ICBw+NDK0G6H+ruxMZWKJTsCFGs9hTbIO5f X-Forefront-Antispam-Report: CIP:216.228.118.232; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge1.nvidia.com; CAT:NONE; SFS:(13230031)(4636009)(136003)(376002)(346002)(39860400002)(396003)(451199024)(1800799009)(186009)(82310400011)(36840700001)(40470700004)(46966006)(82740400003)(7696005)(6666004)(36756003)(40460700003)(86362001)(7636003)(356005)(40480700001)(36860700001)(47076005)(107886003)(2616005)(1076003)(336012)(2906002)(426003)(26005)(83380400001)(478600001)(70586007)(70206006)(4326008)(8676002)(8936002)(5660300002)(41300700001)(6916009)(54906003)(316002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Aug 2023 15:19:06.7133 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 14e56f21-6e79-4099-d6dd-08dba7da1f00 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.232]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000989E7.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6092 Received-SPF: softfail client-ip=2a01:111:f400:7e88::629; envelope-from=avihaih@nvidia.com; helo=NAM10-DM6-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org VFIO migration is not compatible with postcopy migration. A VFIO device in the destination can't handle page faults for pages that have not been sent yet. Doing such migration will cause the VM to crash in the destination: qemu-system-x86_64: VFIO_MAP_DMA failed: Bad address qemu-system-x86_64: vfio_dma_map(0x55a28c7659d0, 0xc0000, 0xb000, 0x7f1b11a00000) = -14 (Bad address) qemu: hardware error: vfio: DMA mapping failed, unable to continue To prevent this and to be explicit about supported features, block VFIO migration with postcopy migration: Fail setting postcopy capability if a VFIO device is present, and add a migration blocker if a VFIO device is added when postcopy capability is on. Reported-by: Yanghang Liu Signed-off-by: Avihai Horon --- include/hw/vfio/vfio-common.h | 2 ++ migration/migration.h | 2 ++ hw/vfio/common.c | 43 +++++++++++++++++++++++++++++++++++ hw/vfio/migration.c | 6 +++++ migration/options.c | 19 ++++++++++++++++ migration/target.c | 19 ++++++++++++++++ 6 files changed, 91 insertions(+) diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index e9b8954595..c0b58f2bb7 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -227,6 +227,8 @@ extern VFIOGroupList vfio_group_list; bool vfio_mig_active(void); int vfio_block_multiple_devices_migration(VFIODevice *vbasedev, Error **errp); void vfio_unblock_multiple_devices_migration(void); +int vfio_block_postcopy_migration(VFIODevice *vbasedev, Error **errp); +void vfio_unblock_postcopy_migration(void); bool vfio_viommu_preset(VFIODevice *vbasedev); int64_t vfio_mig_bytes_transferred(void); void vfio_reset_bytes_transferred(void); diff --git a/migration/migration.h b/migration/migration.h index c5695de214..21a6423408 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -514,6 +514,8 @@ void migration_cancel(const Error *error); void migration_populate_vfio_info(MigrationInfo *info); void migration_reset_vfio_bytes_transferred(void); +bool migration_vfio_mig_active(void); +void migration_vfio_unblock_postcopy_migration(void); void postcopy_temp_page_reset(PostcopyTmpPage *tmp_page); #endif diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 373f6e5932..7461194b2b 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -40,6 +40,7 @@ #include "trace.h" #include "qapi/error.h" #include "migration/migration.h" +#include "migration/options.h" #include "migration/misc.h" #include "migration/blocker.h" #include "migration/qemu-file.h" @@ -343,6 +344,7 @@ static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, uint64_t size, ram_addr_t ram_addr); static Error *multiple_devices_migration_blocker; +static Error *postcopy_migration_blocker; static unsigned int vfio_migratable_devices_num(void) { @@ -427,6 +429,47 @@ void vfio_unblock_multiple_devices_migration(void) multiple_devices_migration_blocker = NULL; } +int vfio_block_postcopy_migration(VFIODevice *vbasedev, Error **errp) +{ + int ret; + + if (!migrate_postcopy_ram()) { + return 0; + } + + if (vbasedev->enable_migration == ON_OFF_AUTO_ON) { + error_setg(errp, + "VFIO migration is not compatible with postcopy migration"); + return -EINVAL; + } + + if (postcopy_migration_blocker) { + return 0; + } + + error_setg(&postcopy_migration_blocker, + "VFIO migration is not compatible with postcopy migration"); + ret = migrate_add_blocker(postcopy_migration_blocker, errp); + if (ret < 0) { + error_free(postcopy_migration_blocker); + postcopy_migration_blocker = NULL; + } + + return ret; +} + +void vfio_unblock_postcopy_migration(void) +{ + if (!postcopy_migration_blocker || + (vfio_migratable_devices_num() && migrate_postcopy_ram())) { + return; + } + + migrate_del_blocker(postcopy_migration_blocker); + error_free(postcopy_migration_blocker); + postcopy_migration_blocker = NULL; +} + bool vfio_mig_active(void) { return vfio_migratable_devices_num(); diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 71855468fe..76406e9ae9 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -856,6 +856,7 @@ static void vfio_migration_deinit(VFIODevice *vbasedev) unregister_savevm(VMSTATE_IF(vbasedev->dev), "vfio", vbasedev); vfio_migration_free(vbasedev); vfio_unblock_multiple_devices_migration(); + vfio_unblock_postcopy_migration(); } static int vfio_block_migration(VFIODevice *vbasedev, Error *err, Error **errp) @@ -939,6 +940,11 @@ bool vfio_migration_realize(VFIODevice *vbasedev, Error **errp) goto out_deinit; } + ret = vfio_block_postcopy_migration(vbasedev, errp); + if (ret) { + goto out_deinit; + } + if (vfio_viommu_preset(vbasedev)) { error_setg(&err, "%s: Migration is currently not supported " "with vIOMMU enabled", vbasedev->name); diff --git a/migration/options.c b/migration/options.c index 1d1e1321b0..e201053563 100644 --- a/migration/options.c +++ b/migration/options.c @@ -499,6 +499,11 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) error_setg(errp, "Postcopy is not yet compatible with multifd"); return false; } + + if (migration_vfio_mig_active()) { + error_setg(errp, "Postcopy is not compatible with VFIO migration"); + return false; + } } if (new_caps[MIGRATION_CAPABILITY_BACKGROUND_SNAPSHOT]) { @@ -612,6 +617,16 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) return true; } +/* + * Devices might have added migration blockers based on migration capabilities + * values when those devices were added. Remove such blockers according to new + * changes in migration capabilities. + */ +static void migration_caps_remove_blockers(void) +{ + migration_vfio_unblock_postcopy_migration(); +} + bool migrate_cap_set(int cap, bool value, Error **errp) { MigrationState *s = migrate_get_current(); @@ -629,6 +644,8 @@ bool migrate_cap_set(int cap, bool value, Error **errp) return false; } s->capabilities[cap] = value; + migration_caps_remove_blockers(); + return true; } @@ -678,6 +695,8 @@ void qmp_migrate_set_capabilities(MigrationCapabilityStatusList *params, for (cap = params; cap; cap = cap->next) { s->capabilities[cap->value->capability] = cap->value->state; } + + migration_caps_remove_blockers(); } /* parameters */ diff --git a/migration/target.c b/migration/target.c index a6ffa9a5ce..690ecb4dd5 100644 --- a/migration/target.c +++ b/migration/target.c @@ -27,6 +27,16 @@ void migration_reset_vfio_bytes_transferred(void) { vfio_reset_bytes_transferred(); } + +bool migration_vfio_mig_active(void) +{ + return vfio_mig_active(); +} + +void migration_vfio_unblock_postcopy_migration(void) +{ + vfio_unblock_postcopy_migration(); +} #else void migration_populate_vfio_info(MigrationInfo *info) { @@ -35,4 +45,13 @@ void migration_populate_vfio_info(MigrationInfo *info) void migration_reset_vfio_bytes_transferred(void) { } + +bool migration_vfio_mig_active(void) +{ + return false; +} + +void migration_vfio_unblock_postcopy_migration() +{ +} #endif From patchwork Mon Aug 28 15:18:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 1826800 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=Nvidia.com header.i=@Nvidia.com header.a=rsa-sha256 header.s=selector2 header.b=mBRZqn+D; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RZDkk2mPvz1yfv for ; Tue, 29 Aug 2023 01:19:58 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qae1Q-0002Mt-2U; Mon, 28 Aug 2023 11:19:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qae1K-0002C2-Fk for qemu-devel@nongnu.org; Mon, 28 Aug 2023 11:19:16 -0400 Received: from mail-mw2nam10on20615.outbound.protection.outlook.com ([2a01:111:f400:7e89::615] helo=NAM10-MW2-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qae1H-0006vw-If for qemu-devel@nongnu.org; Mon, 28 Aug 2023 11:19:14 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ae7EWa8UTLxIy8B/sJVcUDp6WKrqkmxdtcXPNRA7lSHR92ALUh7HRVDNBRGn2jtWcuIwTrDKd5r+/Cu3D/YeyK+McXXaZgMyAilKEe/OhhuBg5vKPGPn9IM1nbpmP9NRhG8IRSOOzM66v+u9G6vDjDBFDiNYD45txbXE+Pq80Izw9nW6c+g729hwcVROO0+xI20cAtZAjFgZGZtXw3PVtPnyd1zps5Bq3paUJC4ls/tA5zSag3ukJ3FYu1zGk2kTnfDTdAGKFFojJpcUmZIUTIWkHkyOi3iCcmDdchUuDTzEXirXgH5a2TPJT1S7e4dtj1tjaS7XKA5Q+g9RD7VVsw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=iYG6rTlMxRTsyWmZW/8xtKd76dgZyAeYF8kh0nulYzQ=; b=TkuxLwmkWnUNFpz38EEKf0i8CpIKw7hhSvjxUVz5wVwibJZxlQEgcZLSkybWwkz/DCqkza0Dz4IT38NBxTL65QxhWA5c5Moqp3EJyfoxzmjchu7QwV5FgwixORndrgJnFmGoO4PLfxq63yRJbMptmEapGAoYYvE87Zvkhdc7LQQSw0IrjPpkCVa/uA7Fa6bqsAnJc30j9sY+LkG+kUr0h7XIvFz2u1E3/I2fe4ST0kBDCC3RsIo+SZUDXQZYSM822718Jwis52kFAsKNiB+RPaAJeh4ckNZaYX/C+Sgq40m7li+2x3516MjIsQGfTCZAJokitC122dtJg/8LCkjMgQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iYG6rTlMxRTsyWmZW/8xtKd76dgZyAeYF8kh0nulYzQ=; b=mBRZqn+DW2nK/x/zVI8xRBc+nJcgg1oJ1pzgFA+s6OesvYtPHxOJFxy99FX590i1GxXiNoI/sXcMQOTtYtecPiWrcAHiFubxczL2xoD40Ah9vM3D/6kFaEy3l+fKS29hCZuinihERpRX6uns3ZlH67IreoUrf0FF0GMkXopR7Gqbak/s2/+mT653NtT6vr25a02rB0xrWs4CxVEOvRUch7Ji2cvmz9HXWBxlFWD0Nn+1kffb05vEkZ/uwqjBNjw+a2lD1RBqf1bsEh2KwWitBSEusOFDgUviVo3qCaAB7VlmkQdkP/KJ7PMzyoPLn54GKJ5AzTPtXkbkc614DGhjHQ== Received: from DM5PR07CA0074.namprd07.prod.outlook.com (2603:10b6:4:ad::39) by SJ1PR12MB6051.namprd12.prod.outlook.com (2603:10b6:a03:48a::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.34; Mon, 28 Aug 2023 15:19:07 +0000 Received: from CY4PEPF0000EE3D.namprd03.prod.outlook.com (2603:10b6:4:ad:cafe::9c) by DM5PR07CA0074.outlook.office365.com (2603:10b6:4:ad::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.35 via Frontend Transport; Mon, 28 Aug 2023 15:19:07 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CY4PEPF0000EE3D.mail.protection.outlook.com (10.167.242.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.17 via Frontend Transport; Mon, 28 Aug 2023 15:19:06 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Mon, 28 Aug 2023 08:19:00 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Mon, 28 Aug 2023 08:19:00 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.37 via Frontend Transport; Mon, 28 Aug 2023 08:18:58 -0700 From: Avihai Horon To: CC: Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_G?= =?utf-8?q?oater?= , Juan Quintela , Peter Xu , Leonardo Bras , Yanghang Liu , Avihai Horon Subject: [PATCH 6/6] vfio/migration: Block VFIO migration with background snapshot Date: Mon, 28 Aug 2023 18:18:42 +0300 Message-ID: <20230828151842.11303-7-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20230828151842.11303-1-avihaih@nvidia.com> References: <20230828151842.11303-1-avihaih@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE3D:EE_|SJ1PR12MB6051:EE_ X-MS-Office365-Filtering-Correlation-Id: 18aabe2d-11b7-44eb-b70d-08dba7da1f2e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yS3tu09VvA3A1tJnsovhGfJnVYTtyTdX3p5y4LC23ePq0r4+0xlpF4sjG5BpmOukIsVjUXmrcDjMd0kNLovxJWazCbP/Rg1yoe1vUVV783dIw/ahaqxIf/8w5RdStjxP3vzs7nZ8RyqBcXRSS685A3i5BuH5kgK9vNobE49PAG89UOrK37mxeA8AdleWWwU+bULoLGyZyKrH8weMkh0pCpNucj+4d2D4mKC/sJVqKJZRJ3+AWyEV108kAjev8ast2pS6W8FAuLClXOWnNZdcpVxn/UIxWQx5o7qAdHE+xExsyuTzwfKqrZ9HxyTU96xrY9jwDpB7gUUVwQhuXifoa6tbPAVJRmyQekOR/hU4vYY7SLiMZXzinzjosku0eO0FnCEfM0hZvJDVWeGqL4uE3yR1thNPptPSjuGSXTfXXjY7ysFO9vUfRqhQAB8pSA9LEA7Z+pxUjwQ7in2X3INYqu7hgA5YvU/NdTuoOwSV2NavSv0Fm+PVZuKGEp9dz0M9nZbEK1SKpTebLaeOGIs9JWS+k+6ZEzsX6LU6mjaLB2o02q4cD2Kuf3Q6ckT25MWH9jucEj6rwI0jbOijx1ZzRtOBN3jLH0XLnaakk0ePuoW0VkpL+80tWtzoRJhh7LuldhPU/U+UindOk1Ow24cqZBxlWSgnchtMz1WDFXxt+lDcAsTZafRki7EgWloGlShoM2v2h6HpGzl63NFvM9Jgiv4Bs9U5/k4ISrhX6ezu+E26tQ0defFYvDRQPR3rWrGU X-Forefront-Antispam-Report: CIP:216.228.118.233; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge2.nvidia.com; CAT:NONE; SFS:(13230031)(4636009)(346002)(396003)(136003)(39860400002)(376002)(451199024)(1800799009)(82310400011)(186009)(36840700001)(46966006)(40470700004)(83380400001)(478600001)(7636003)(356005)(82740400003)(26005)(426003)(336012)(47076005)(36860700001)(107886003)(1076003)(2616005)(40480700001)(7696005)(6666004)(86362001)(2906002)(316002)(4326008)(6916009)(70206006)(8676002)(70586007)(41300700001)(8936002)(54906003)(36756003)(5660300002)(40460700003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Aug 2023 15:19:06.9984 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 18aabe2d-11b7-44eb-b70d-08dba7da1f2e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.233]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EE3D.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR12MB6051 Received-SPF: softfail client-ip=2a01:111:f400:7e89::615; envelope-from=avihaih@nvidia.com; helo=NAM10-MW2-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Background snapshot allows creating a snapshot of the VM while it's running and keeping it small by not including dirty RAM pages. The way it works is by first stopping the VM, saving the non-iterable devices' state and then starting the VM and saving the RAM while write protecting it with UFFD. The resulting snapshot represents the VM state at snapshot start. VFIO migration is not compatible with background snapshot. First of all, VFIO device state is not even saved in background snapshot because only non-iterable device state is saved. But even if it was saved, after starting the VM, a VFIO device could dirty pages without it being detected by UFFD write protection. This would corrupt the snapshot, as the RAM in it would not represent the RAM at snapshot start. To prevent this and to be explicit about supported features, block VFIO migration with background snapshot: Fail setting background snapshot capability if a VFIO device is present, and add a migration blocker if a VFIO device is added when background snapshot capability is on. Signed-off-by: Avihai Horon --- include/hw/vfio/vfio-common.h | 2 ++ migration/migration.h | 1 + hw/vfio/common.c | 42 +++++++++++++++++++++++++++++++++++ hw/vfio/migration.c | 6 +++++ migration/options.c | 7 ++++++ migration/target.c | 9 ++++++++ 6 files changed, 67 insertions(+) diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index c0b58f2bb7..bb94f320f1 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -229,6 +229,8 @@ int vfio_block_multiple_devices_migration(VFIODevice *vbasedev, Error **errp); void vfio_unblock_multiple_devices_migration(void); int vfio_block_postcopy_migration(VFIODevice *vbasedev, Error **errp); void vfio_unblock_postcopy_migration(void); +int vfio_block_background_snapshot(VFIODevice *vbasedev, Error **errp); +void vfio_unblock_background_snapshot(void); bool vfio_viommu_preset(VFIODevice *vbasedev); int64_t vfio_mig_bytes_transferred(void); void vfio_reset_bytes_transferred(void); diff --git a/migration/migration.h b/migration/migration.h index 21a6423408..3077ed430b 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -516,6 +516,7 @@ void migration_populate_vfio_info(MigrationInfo *info); void migration_reset_vfio_bytes_transferred(void); bool migration_vfio_mig_active(void); void migration_vfio_unblock_postcopy_migration(void); +void migration_vfio_unblock_background_snapshot(void); void postcopy_temp_page_reset(PostcopyTmpPage *tmp_page); #endif diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 7461194b2b..4f6bc40cc0 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -345,6 +345,7 @@ static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, static Error *multiple_devices_migration_blocker; static Error *postcopy_migration_blocker; +static Error *background_snapshot_blocker; static unsigned int vfio_migratable_devices_num(void) { @@ -470,6 +471,47 @@ void vfio_unblock_postcopy_migration(void) postcopy_migration_blocker = NULL; } +int vfio_block_background_snapshot(VFIODevice *vbasedev, Error **errp) +{ + int ret; + + if (!migrate_background_snapshot()) { + return 0; + } + + if (vbasedev->enable_migration == ON_OFF_AUTO_ON) { + error_setg(errp, + "VFIO migration is not compatible with background snapshot"); + return -EINVAL; + } + + if (background_snapshot_blocker) { + return 0; + } + + error_setg(&background_snapshot_blocker, + "VFIO migration is not compatible with background snapshot"); + ret = migrate_add_blocker(background_snapshot_blocker, errp); + if (ret < 0) { + error_free(background_snapshot_blocker); + background_snapshot_blocker = NULL; + } + + return ret; +} + +void vfio_unblock_background_snapshot(void) +{ + if (!background_snapshot_blocker || + (vfio_migratable_devices_num() && migrate_background_snapshot())) { + return; + } + + migrate_del_blocker(background_snapshot_blocker); + error_free(background_snapshot_blocker); + background_snapshot_blocker = NULL; +} + bool vfio_mig_active(void) { return vfio_migratable_devices_num(); diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 76406e9ae9..adf98ac8e3 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -857,6 +857,7 @@ static void vfio_migration_deinit(VFIODevice *vbasedev) vfio_migration_free(vbasedev); vfio_unblock_multiple_devices_migration(); vfio_unblock_postcopy_migration(); + vfio_unblock_background_snapshot(); } static int vfio_block_migration(VFIODevice *vbasedev, Error *err, Error **errp) @@ -945,6 +946,11 @@ bool vfio_migration_realize(VFIODevice *vbasedev, Error **errp) goto out_deinit; } + ret = vfio_block_background_snapshot(vbasedev, errp); + if (ret) { + goto out_deinit; + } + if (vfio_viommu_preset(vbasedev)) { error_setg(&err, "%s: Migration is currently not supported " "with vIOMMU enabled", vbasedev->name); diff --git a/migration/options.c b/migration/options.c index e201053563..2e13363de6 100644 --- a/migration/options.c +++ b/migration/options.c @@ -537,6 +537,12 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) return false; } } + + if (migration_vfio_mig_active()) { + error_setg(errp, "Background-snapshot is not compatible with VFIO " + "migration"); + return false; + } } #ifdef CONFIG_LINUX @@ -625,6 +631,7 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) static void migration_caps_remove_blockers(void) { migration_vfio_unblock_postcopy_migration(); + migration_vfio_unblock_background_snapshot(); } bool migrate_cap_set(int cap, bool value, Error **errp) diff --git a/migration/target.c b/migration/target.c index 690ecb4dd5..c2be0b39db 100644 --- a/migration/target.c +++ b/migration/target.c @@ -37,6 +37,11 @@ void migration_vfio_unblock_postcopy_migration(void) { vfio_unblock_postcopy_migration(); } + +void migration_vfio_unblock_background_snapshot(void) +{ + vfio_unblock_background_snapshot(); +} #else void migration_populate_vfio_info(MigrationInfo *info) { @@ -54,4 +59,8 @@ bool migration_vfio_mig_active(void) void migration_vfio_unblock_postcopy_migration() { } + +void migration_vfio_unblock_background_snapshot(void) +{ +} #endif