From patchwork Mon Oct 23 20:35:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854014 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=GY0XD0a7; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=k9ilP6I/; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn8G3G9Tz23k5 for ; Tue, 24 Oct 2023 07:38:22 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1ez-0002j5-Ex; Mon, 23 Oct 2023 16:36:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1ex-0002ie-By for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:23 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1es-0001mk-95 for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:20 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 4697521B24; Mon, 23 Oct 2023 20:36:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093376; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2SO3JnEnMhOQJ8CPQPJIKFKAyEl5rYqk1sj5JESuE5w=; b=GY0XD0a7RJWyIJ0I+FIxKmLZwVuSoYtbzOyz7e/Dgf0DvGpXwOyczDtCsohPj6PE8BCiVb sw4t6cRqdYtH6++UzbzaNnxbNQnYR7TN9bR9kcHpbx3OJdTl+Ak6zazG21FLIMHB+lYsLE l2iXy6hEy970pqjSv4wlw5JvcgNCSMg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093376; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2SO3JnEnMhOQJ8CPQPJIKFKAyEl5rYqk1sj5JESuE5w=; b=k9ilP6I/mT0tqPOVR/LcoPNoO3u4wzyG/FbHUN2jmoZP3n2xRpTmXNP13xrDXb6Fz5in7/ 4wn6LhDwHR7B2XCg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 81A7B132FD; Mon, 23 Oct 2023 20:36:13 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 6HdREj3ZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:36:13 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana , Steve Sistare , Thomas Huth , Laurent Vivier , Paolo Bonzini Subject: [PATCH v2 01/29] tests/qtest: migration events Date: Mon, 23 Oct 2023 17:35:40 -0300 Message-Id: <20231023203608.26370-2-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out1.suse.de; none X-Spam-Score: -2.10 X-Spamd-Result: default: False [-2.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[11]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] Received-SPF: pass client-ip=195.135.220.28; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Steve Sistare Define a state object to capture events seen by migration tests, to allow more events to be captured in a subsequent patch, and simplify event checking in wait_for_migration_pass. No functional change. Signed-off-by: Steve Sistare Reviewed-by: Daniel P. Berrangé --- tests/qtest/migration-helpers.c | 24 ++++------- tests/qtest/migration-helpers.h | 8 ++-- tests/qtest/migration-test.c | 74 +++++++++++++++------------------ 3 files changed, 46 insertions(+), 60 deletions(-) diff --git a/tests/qtest/migration-helpers.c b/tests/qtest/migration-helpers.c index 24fb7b3525..fd3b94efa2 100644 --- a/tests/qtest/migration-helpers.c +++ b/tests/qtest/migration-helpers.c @@ -24,26 +24,16 @@ */ #define MIGRATION_STATUS_WAIT_TIMEOUT 120 -bool migrate_watch_for_stop(QTestState *who, const char *name, - QDict *event, void *opaque) -{ - bool *seen = opaque; - - if (g_str_equal(name, "STOP")) { - *seen = true; - return true; - } - - return false; -} - -bool migrate_watch_for_resume(QTestState *who, const char *name, +bool migrate_watch_for_events(QTestState *who, const char *name, QDict *event, void *opaque) { - bool *seen = opaque; + QTestMigrationState *state = opaque; - if (g_str_equal(name, "RESUME")) { - *seen = true; + if (g_str_equal(name, "STOP")) { + state->stop_seen = true; + return true; + } else if (g_str_equal(name, "RESUME")) { + state->resume_seen = true; return true; } diff --git a/tests/qtest/migration-helpers.h b/tests/qtest/migration-helpers.h index e31dc85cc7..c1d4c84995 100644 --- a/tests/qtest/migration-helpers.h +++ b/tests/qtest/migration-helpers.h @@ -15,9 +15,11 @@ #include "libqtest.h" -bool migrate_watch_for_stop(QTestState *who, const char *name, - QDict *event, void *opaque); -bool migrate_watch_for_resume(QTestState *who, const char *name, +typedef struct QTestMigrationState { + bool stop_seen, resume_seen; +} QTestMigrationState; + +bool migrate_watch_for_events(QTestState *who, const char *name, QDict *event, void *opaque); G_GNUC_PRINTF(3, 4) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 35e0ded9d7..0425d1d527 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -43,8 +43,8 @@ unsigned start_address; unsigned end_address; static bool uffd_feature_thread_id; -static bool got_src_stop; -static bool got_dst_resume; +static QTestMigrationState src_state; +static QTestMigrationState dst_state; /* * An initial 3 MB offset is used as that corresponds @@ -230,6 +230,13 @@ static void wait_for_serial(const char *side) } while (true); } +static void wait_for_stop(QTestState *who, QTestMigrationState *state) +{ + if (!state->stop_seen) { + qtest_qmp_eventwait(who, "STOP"); + } +} + /* * It's tricky to use qemu's migration event capability with qtest, * events suddenly appearing confuse the qmp()/hmp() responses. @@ -277,21 +284,19 @@ static void read_blocktime(QTestState *who) qobject_unref(rsp_return); } +/* + * Wait for two changes in the migration pass count, but bail if we stop. + */ static void wait_for_migration_pass(QTestState *who) { - uint64_t initial_pass = get_migration_pass(who); - uint64_t pass; + uint64_t pass, prev_pass = 0, changes = 0; - /* Wait for the 1st sync */ - while (!got_src_stop && !initial_pass) { - usleep(1000); - initial_pass = get_migration_pass(who); - } - - do { + while (changes < 2 && !src_state.stop_seen) { usleep(1000); pass = get_migration_pass(who); - } while (pass == initial_pass && !got_src_stop); + changes += (pass != prev_pass); + prev_pass = pass; + } } static void check_guests_ram(QTestState *who) @@ -617,10 +622,7 @@ static void migrate_postcopy_start(QTestState *from, QTestState *to) { qtest_qmp_assert_success(from, "{ 'execute': 'migrate-start-postcopy' }"); - if (!got_src_stop) { - qtest_qmp_eventwait(from, "STOP"); - } - + wait_for_stop(from, &src_state); qtest_qmp_eventwait(to, "RESUME"); } @@ -755,8 +757,9 @@ static int test_migrate_start(QTestState **from, QTestState **to, } } - got_src_stop = false; - got_dst_resume = false; + dst_state = (QTestMigrationState) { }; + src_state = (QTestMigrationState) { }; + if (strcmp(arch, "i386") == 0 || strcmp(arch, "x86_64") == 0) { memory_size = "150M"; @@ -847,8 +850,8 @@ static int test_migrate_start(QTestState **from, QTestState **to, if (!args->only_target) { *from = qtest_init_with_env(QEMU_ENV_SRC, cmd_source); qtest_qmp_set_event_callback(*from, - migrate_watch_for_stop, - &got_src_stop); + migrate_watch_for_events, + &src_state); } cmd_target = g_strdup_printf("-accel kvm%s -accel tcg " @@ -868,8 +871,8 @@ static int test_migrate_start(QTestState **from, QTestState **to, ignore_stderr); *to = qtest_init_with_env(QEMU_ENV_DST, cmd_target); qtest_qmp_set_event_callback(*to, - migrate_watch_for_resume, - &got_dst_resume); + migrate_watch_for_events, + &dst_state); /* * Remove shmem file immediately to avoid memory leak in test failed case. @@ -1619,9 +1622,7 @@ static void test_precopy_common(MigrateCommon *args) */ if (args->result == MIG_TEST_SUCCEED) { qtest_qmp_assert_success(from, "{ 'execute' : 'stop'}"); - if (!got_src_stop) { - qtest_qmp_eventwait(from, "STOP"); - } + wait_for_stop(from, &src_state); migrate_ensure_converge(from); } } @@ -1667,9 +1668,8 @@ static void test_precopy_common(MigrateCommon *args) */ wait_for_migration_complete(from); - if (!got_src_stop) { - qtest_qmp_eventwait(from, "STOP"); - } + wait_for_stop(from, &src_state); + } else { wait_for_migration_complete(from); /* @@ -1682,7 +1682,7 @@ static void test_precopy_common(MigrateCommon *args) qtest_qmp_assert_success(to, "{ 'execute' : 'cont'}"); } - if (!got_dst_resume) { + if (!dst_state.resume_seen) { qtest_qmp_eventwait(to, "RESUME"); } @@ -1723,9 +1723,7 @@ static void test_file_common(MigrateCommon *args, bool stop_src) if (stop_src) { qtest_qmp_assert_success(from, "{ 'execute' : 'stop'}"); - if (!got_src_stop) { - qtest_qmp_eventwait(from, "STOP"); - } + wait_for_stop(from, &src_state); } if (args->result == MIG_TEST_QMP_ERROR) { @@ -1747,7 +1745,7 @@ static void test_file_common(MigrateCommon *args, bool stop_src) qtest_qmp_assert_success(to, "{ 'execute' : 'cont'}"); } - if (!got_dst_resume) { + if (!dst_state.resume_seen) { qtest_qmp_eventwait(to, "RESUME"); } @@ -1868,9 +1866,7 @@ static void test_ignore_shared(void) migrate_wait_for_dirty_mem(from, to); - if (!got_src_stop) { - qtest_qmp_eventwait(from, "STOP"); - } + wait_for_stop(from, &src_state); qtest_qmp_eventwait(to, "RESUME"); @@ -2380,7 +2376,7 @@ static void test_migrate_auto_converge(void) break; } usleep(20); - g_assert_false(got_src_stop); + g_assert_false(src_state.stop_seen); } while (true); /* The first percentage of throttling should be at least init_pct */ g_assert_cmpint(percentage, >=, init_pct); @@ -2719,9 +2715,7 @@ static void test_multifd_tcp_cancel(void) migrate_ensure_converge(from); - if (!got_src_stop) { - qtest_qmp_eventwait(from, "STOP"); - } + wait_for_stop(from, &src_state); qtest_qmp_eventwait(to2, "RESUME"); wait_for_serial("dest_serial"); From patchwork Mon Oct 23 20:35:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854025 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=0KqVa5dR; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=qQ7pE7mM; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn9v0vtSz23jl for ; Tue, 24 Oct 2023 07:39:47 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1f2-0002k1-SB; Mon, 23 Oct 2023 16:36:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1ey-0002it-Uw for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:25 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1ew-0001mz-MU for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:24 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 41BE721B27; Mon, 23 Oct 2023 20:36:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Z6/HK5rLwaWV5PTzUS7sMhXDaBGHK8AaYqDlOVmMg0o=; b=0KqVa5dRR77WS8DeG989MiY1FL8PIT+QBSGOtxtc+/Ap1b4BpsfoonFTU4FPaBeN8PIB7o RTLd7L5RDH8GOQ6ZpLAMWqJG7wS7Qo37zpD6ztci8cGuvIj7QLLH+sqK33NC7z4FL7zJpm HZer3XqkfBwVgR0fW1q2RxV0LgtMjCE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Z6/HK5rLwaWV5PTzUS7sMhXDaBGHK8AaYqDlOVmMg0o=; b=qQ7pE7mM7mhoj9VxyZjGukEoLql1kLGoQxiKY1zXdcTA9tzy66TXfq0JnJHtUkYxNHe02f 23qwSV2ybt9MiCCA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id B5AEB132FD; Mon, 23 Oct 2023 20:36:16 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id CD3/H0DZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:36:16 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana , Thomas Huth , Laurent Vivier , Paolo Bonzini Subject: [PATCH v2 02/29] tests/qtest: Move QTestMigrationState to libqtest Date: Mon, 23 Oct 2023 17:35:41 -0300 Message-Id: <20231023203608.26370-3-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out1.suse.de; none X-Spam-Score: -2.10 X-Spamd-Result: default: False [-2.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[10]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] Received-SPF: pass client-ip=195.135.220.28; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Move the QTestMigrationState into QTestState so we don't have to pass it around to the wait_for_* helpers anymore. Since QTestState is private to libqtest.c, move the migration state struct to libqtest.h and add a getter. Signed-off-by: Fabiano Rosas --- tests/qtest/libqtest.c | 14 ++++++++++ tests/qtest/libqtest.h | 23 ++++++++++++++++ tests/qtest/migration-helpers.c | 18 +++++++++++++ tests/qtest/migration-helpers.h | 8 +++--- tests/qtest/migration-test.c | 47 +++++++++------------------------ 5 files changed, 72 insertions(+), 38 deletions(-) diff --git a/tests/qtest/libqtest.c b/tests/qtest/libqtest.c index f33a210861..f7e85486dc 100644 --- a/tests/qtest/libqtest.c +++ b/tests/qtest/libqtest.c @@ -87,6 +87,7 @@ struct QTestState GList *pending_events; QTestQMPEventCallback eventCB; void *eventData; + QTestMigrationState *migration_state; }; static GHookList abrt_hooks; @@ -500,6 +501,8 @@ static QTestState *qtest_init_internal(const char *qemu_bin, s->irq_level[i] = false; } + s->migration_state = g_new0(QTestMigrationState, 1); + /* * Stopping QEMU for debugging is not supported on Windows. * @@ -601,6 +604,7 @@ void qtest_quit(QTestState *s) close(s->fd); close(s->qmp_fd); g_string_free(s->rx, true); + g_free(s->migration_state); for (GList *it = s->pending_events; it != NULL; it = it->next) { qobject_unref((QDict *)it->data); @@ -854,6 +858,11 @@ void qtest_qmp_set_event_callback(QTestState *s, s->eventData = opaque; } +void qtest_qmp_set_migration_callback(QTestState *s, QTestQMPEventCallback cb) +{ + qtest_qmp_set_event_callback(s, cb, s->migration_state); +} + QDict *qtest_qmp_event_ref(QTestState *s, const char *event) { while (s->pending_events) { @@ -1906,3 +1915,8 @@ bool mkimg(const char *file, const char *fmt, unsigned size_mb) return ret && !err; } + +QTestMigrationState *qtest_migration_state(QTestState *s) +{ + return s->migration_state; +} diff --git a/tests/qtest/libqtest.h b/tests/qtest/libqtest.h index 6e3d3525bf..0421a1da24 100644 --- a/tests/qtest/libqtest.h +++ b/tests/qtest/libqtest.h @@ -23,6 +23,20 @@ typedef struct QTestState QTestState; +struct QTestMigrationState { + bool stop_seen; + bool resume_seen; +}; +typedef struct QTestMigrationState QTestMigrationState; + +/** + * qtest_migration_state: + * @s: #QTestState instance to operate on. + * + * Returns: #QTestMigrationState instance. + */ +QTestMigrationState *qtest_migration_state(QTestState *s); + /** * qtest_initf: * @fmt: Format for creating other arguments to pass to QEMU, formatted @@ -288,6 +302,15 @@ typedef bool (*QTestQMPEventCallback)(QTestState *s, const char *name, void qtest_qmp_set_event_callback(QTestState *s, QTestQMPEventCallback cb, void *opaque); +/** + * qtest_qmp_set_migration_callback: + * @s: #QTestSTate instance to operate on + * @cb: callback to invoke for events + * + * Like qtest_qmp_set_event_callback, but includes migration state events + */ +void qtest_qmp_set_migration_callback(QTestState *s, QTestQMPEventCallback cb); + /** * qtest_qmp_eventwait: * @s: #QTestState instance to operate on. diff --git a/tests/qtest/migration-helpers.c b/tests/qtest/migration-helpers.c index fd3b94efa2..cffa525c81 100644 --- a/tests/qtest/migration-helpers.c +++ b/tests/qtest/migration-helpers.c @@ -92,6 +92,24 @@ void migrate_set_capability(QTestState *who, const char *capability, capability, value); } +void wait_for_stop(QTestState *who) +{ + QTestMigrationState *state = qtest_migration_state(who); + + if (!state->stop_seen) { + qtest_qmp_eventwait(who, "STOP"); + } +} + +void wait_for_resume(QTestState *who) +{ + QTestMigrationState *state = qtest_migration_state(who); + + if (!state->resume_seen) { + qtest_qmp_eventwait(who, "RESUME"); + } +} + void migrate_incoming_qmp(QTestState *to, const char *uri, const char *fmt, ...) { va_list ap; diff --git a/tests/qtest/migration-helpers.h b/tests/qtest/migration-helpers.h index c1d4c84995..7297f1ff2c 100644 --- a/tests/qtest/migration-helpers.h +++ b/tests/qtest/migration-helpers.h @@ -15,13 +15,13 @@ #include "libqtest.h" -typedef struct QTestMigrationState { - bool stop_seen, resume_seen; -} QTestMigrationState; - bool migrate_watch_for_events(QTestState *who, const char *name, QDict *event, void *opaque); + +void wait_for_stop(QTestState *who); +void wait_for_resume(QTestState *who); + G_GNUC_PRINTF(3, 4) void migrate_qmp(QTestState *who, const char *uri, const char *fmt, ...); diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 0425d1d527..88e611e98f 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -43,8 +43,6 @@ unsigned start_address; unsigned end_address; static bool uffd_feature_thread_id; -static QTestMigrationState src_state; -static QTestMigrationState dst_state; /* * An initial 3 MB offset is used as that corresponds @@ -230,13 +228,6 @@ static void wait_for_serial(const char *side) } while (true); } -static void wait_for_stop(QTestState *who, QTestMigrationState *state) -{ - if (!state->stop_seen) { - qtest_qmp_eventwait(who, "STOP"); - } -} - /* * It's tricky to use qemu's migration event capability with qtest, * events suddenly appearing confuse the qmp()/hmp() responses. @@ -290,8 +281,9 @@ static void read_blocktime(QTestState *who) static void wait_for_migration_pass(QTestState *who) { uint64_t pass, prev_pass = 0, changes = 0; + QTestMigrationState *state = qtest_migration_state(who); - while (changes < 2 && !src_state.stop_seen) { + while (changes < 2 && !state->stop_seen) { usleep(1000); pass = get_migration_pass(who); changes += (pass != prev_pass); @@ -622,7 +614,7 @@ static void migrate_postcopy_start(QTestState *from, QTestState *to) { qtest_qmp_assert_success(from, "{ 'execute': 'migrate-start-postcopy' }"); - wait_for_stop(from, &src_state); + wait_for_stop(from); qtest_qmp_eventwait(to, "RESUME"); } @@ -757,9 +749,6 @@ static int test_migrate_start(QTestState **from, QTestState **to, } } - dst_state = (QTestMigrationState) { }; - src_state = (QTestMigrationState) { }; - if (strcmp(arch, "i386") == 0 || strcmp(arch, "x86_64") == 0) { memory_size = "150M"; @@ -849,9 +838,7 @@ static int test_migrate_start(QTestState **from, QTestState **to, ignore_stderr); if (!args->only_target) { *from = qtest_init_with_env(QEMU_ENV_SRC, cmd_source); - qtest_qmp_set_event_callback(*from, - migrate_watch_for_events, - &src_state); + qtest_qmp_set_migration_callback(*from, migrate_watch_for_events); } cmd_target = g_strdup_printf("-accel kvm%s -accel tcg " @@ -870,9 +857,7 @@ static int test_migrate_start(QTestState **from, QTestState **to, args->opts_target ? args->opts_target : "", ignore_stderr); *to = qtest_init_with_env(QEMU_ENV_DST, cmd_target); - qtest_qmp_set_event_callback(*to, - migrate_watch_for_events, - &dst_state); + qtest_qmp_set_migration_callback(*to, migrate_watch_for_events); /* * Remove shmem file immediately to avoid memory leak in test failed case. @@ -1622,7 +1607,7 @@ static void test_precopy_common(MigrateCommon *args) */ if (args->result == MIG_TEST_SUCCEED) { qtest_qmp_assert_success(from, "{ 'execute' : 'stop'}"); - wait_for_stop(from, &src_state); + wait_for_stop(from); migrate_ensure_converge(from); } } @@ -1668,7 +1653,7 @@ static void test_precopy_common(MigrateCommon *args) */ wait_for_migration_complete(from); - wait_for_stop(from, &src_state); + wait_for_stop(from); } else { wait_for_migration_complete(from); @@ -1682,10 +1667,7 @@ static void test_precopy_common(MigrateCommon *args) qtest_qmp_assert_success(to, "{ 'execute' : 'cont'}"); } - if (!dst_state.resume_seen) { - qtest_qmp_eventwait(to, "RESUME"); - } - + wait_for_resume(to); wait_for_serial("dest_serial"); } @@ -1723,7 +1705,7 @@ static void test_file_common(MigrateCommon *args, bool stop_src) if (stop_src) { qtest_qmp_assert_success(from, "{ 'execute' : 'stop'}"); - wait_for_stop(from, &src_state); + wait_for_stop(from); } if (args->result == MIG_TEST_QMP_ERROR) { @@ -1745,10 +1727,7 @@ static void test_file_common(MigrateCommon *args, bool stop_src) qtest_qmp_assert_success(to, "{ 'execute' : 'cont'}"); } - if (!dst_state.resume_seen) { - qtest_qmp_eventwait(to, "RESUME"); - } - + wait_for_resume(to); wait_for_serial("dest_serial"); finish: @@ -1866,7 +1845,7 @@ static void test_ignore_shared(void) migrate_wait_for_dirty_mem(from, to); - wait_for_stop(from, &src_state); + wait_for_stop(from); qtest_qmp_eventwait(to, "RESUME"); @@ -2376,7 +2355,7 @@ static void test_migrate_auto_converge(void) break; } usleep(20); - g_assert_false(src_state.stop_seen); + g_assert_false(qtest_migration_state(from)->stop_seen); } while (true); /* The first percentage of throttling should be at least init_pct */ g_assert_cmpint(percentage, >=, init_pct); @@ -2715,7 +2694,7 @@ static void test_multifd_tcp_cancel(void) migrate_ensure_converge(from); - wait_for_stop(from, &src_state); + wait_for_stop(from); qtest_qmp_eventwait(to2, "RESUME"); wait_for_serial("dest_serial"); From patchwork Mon Oct 23 20:35:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854018 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=OGI0tyyY; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=cXmrf+YI; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn8x5BWhz23jl for ; Tue, 24 Oct 2023 07:38:57 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1f0-0002jg-NJ; Mon, 23 Oct 2023 16:36:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1ez-0002j0-7Y for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:25 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1ex-0001n5-Dx for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:24 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 3BF8A21B28; Mon, 23 Oct 2023 20:36:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093382; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/acRSj9Gns6BPDskwHG2W/5R6DDTgO/bxnL85AIt0Po=; b=OGI0tyyYVOZBeQ6qc4tQDzRfOJ3NCtzTO5SNmX7lCy3VSQNQ9LcZTXJFTCI+ySgP7RAZbZ MYVbrt52DT4nMHdiQM2eJUZEV1oaoaSkmXhj/QsnkMXWvX2nSZEaLhyLHmSFd918oBbdOr y587dCHYhvR2T7UPMA3z4nD9hQGKk64= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093382; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/acRSj9Gns6BPDskwHG2W/5R6DDTgO/bxnL85AIt0Po=; b=cXmrf+YIRbsKSjTwgISRQoPkotDWyUatbYiovwiOoX7HWhmLC4g86di1kDOwpPITO6pjje K9MgLkMCbgdAZ6Cw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id AC3EF132FD; Mon, 23 Oct 2023 20:36:19 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id kPWxHUPZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:36:19 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana , Thomas Huth , Laurent Vivier , Paolo Bonzini Subject: [PATCH v2 03/29] tests/qtest: Allow waiting for migration events Date: Mon, 23 Oct 2023 17:35:42 -0300 Message-Id: <20231023203608.26370-4-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out1.suse.de; none X-Spam-Score: -2.10 X-Spamd-Result: default: False [-2.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[10]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] Received-SPF: pass client-ip=195.135.220.28; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Add support for waiting for a migration state change event to happen. This can help disambiguate between runstate changes that happen during VM lifecycle. Specifically, the next couple of patches want to know whether STOP events happened at the migration start or end. Add the "setup" and "active" migration states for that purpose. Signed-off-by: Fabiano Rosas --- tests/qtest/libqtest.h | 2 + tests/qtest/migration-helpers.c | 66 ++++++++++++++++++++++++++++----- tests/qtest/migration-helpers.h | 2 + 3 files changed, 61 insertions(+), 9 deletions(-) diff --git a/tests/qtest/libqtest.h b/tests/qtest/libqtest.h index 0421a1da24..67fc2ae487 100644 --- a/tests/qtest/libqtest.h +++ b/tests/qtest/libqtest.h @@ -26,6 +26,8 @@ typedef struct QTestState QTestState; struct QTestMigrationState { bool stop_seen; bool resume_seen; + bool setup_seen; + bool active_seen; }; typedef struct QTestMigrationState QTestMigrationState; diff --git a/tests/qtest/migration-helpers.c b/tests/qtest/migration-helpers.c index cffa525c81..a3beff8b57 100644 --- a/tests/qtest/migration-helpers.c +++ b/tests/qtest/migration-helpers.c @@ -34,6 +34,22 @@ bool migrate_watch_for_events(QTestState *who, const char *name, return true; } else if (g_str_equal(name, "RESUME")) { state->resume_seen = true; + return true; + } else if (g_str_equal(name, "MIGRATION")) { + QDict *data; + g_assert(qdict_haskey(event, "data")); + + data = qdict_get_qdict(event, "data"); + g_assert(qdict_haskey(data, "status")); + + if (g_str_equal(qdict_get_str(data, "status"), "setup")) { + state->setup_seen = true; + } else if (g_str_equal(qdict_get_str(data, "status"), "active")) { + state->active_seen = true; + } else { + return false; + } + return true; } @@ -110,10 +126,49 @@ void wait_for_resume(QTestState *who) } } +static void wait_for_migration_state(QTestState *who, const char* state) +{ + QDict *rsp, *data; + + for (;;) { + rsp = qtest_qmp_eventwait_ref(who, "MIGRATION"); + g_assert(qdict_haskey(rsp, "data")); + + data = qdict_get_qdict(rsp, "data"); + g_assert(qdict_haskey(data, "status")); + + if (g_str_equal(qdict_get_str(data, "status"), state)) { + break; + } + qobject_unref(rsp); + } + + qobject_unref(rsp); + return; +} + +void wait_for_setup(QTestState *who) +{ + QTestMigrationState *state = qtest_migration_state(who); + + if (!state->setup_seen) { + wait_for_migration_state(who, "setup"); + } +} + +void wait_for_active(QTestState *who) +{ + QTestMigrationState *state = qtest_migration_state(who); + + if (!state->active_seen) { + wait_for_migration_state(who, "active"); + } +} + void migrate_incoming_qmp(QTestState *to, const char *uri, const char *fmt, ...) { va_list ap; - QDict *args, *rsp, *data; + QDict *args, *rsp; va_start(ap, fmt); args = qdict_from_vjsonf_nofail(fmt, ap); @@ -129,14 +184,7 @@ void migrate_incoming_qmp(QTestState *to, const char *uri, const char *fmt, ...) g_assert(qdict_haskey(rsp, "return")); qobject_unref(rsp); - rsp = qtest_qmp_eventwait_ref(to, "MIGRATION"); - g_assert(qdict_haskey(rsp, "data")); - - data = qdict_get_qdict(rsp, "data"); - g_assert(qdict_haskey(data, "status")); - g_assert_cmpstr(qdict_get_str(data, "status"), ==, "setup"); - - qobject_unref(rsp); + wait_for_setup(to); } /* diff --git a/tests/qtest/migration-helpers.h b/tests/qtest/migration-helpers.h index 7297f1ff2c..11a93dd48d 100644 --- a/tests/qtest/migration-helpers.h +++ b/tests/qtest/migration-helpers.h @@ -21,6 +21,8 @@ bool migrate_watch_for_events(QTestState *who, const char *name, void wait_for_stop(QTestState *who); void wait_for_resume(QTestState *who); +void wait_for_setup(QTestState *who); +void wait_for_active(QTestState *who); G_GNUC_PRINTF(3, 4) void migrate_qmp(QTestState *who, const char *uri, const char *fmt, ...); From patchwork Mon Oct 23 20:35:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854013 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=v09lQ3QK; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=/RLLFmPk; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn8G0nCQz23jl for ; Tue, 24 Oct 2023 07:38:22 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1f5-0002kX-FE; Mon, 23 Oct 2023 16:36:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1f3-0002k8-D7 for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:29 -0400 Received: from smtp-out2.suse.de ([2001:67c:2178:6::1d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1f1-0001nQ-Of for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:29 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id A4BA21FE38; Mon, 23 Oct 2023 20:36:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093384; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=a0u7duS6dyvaT8TiOJ6V1+0DT2Vtx+lGHJt60GutAQM=; b=v09lQ3QKPYhGMFrC+CQKUHaEWLi/i6IXsWuNoOpSlFIlTMnE5Ct0ApbgoYTe8oEPYt4kMw 3FwM4dQZyW5r7GMmN2KAwWGQwq/Z0p6sSl8Z1Hc/Q0oO/Ldli4p6NMIGA1Lo2cq2vmvqhn daOheRQfC5ZkCeHO427Sc5x+00ooAGA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093384; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=a0u7duS6dyvaT8TiOJ6V1+0DT2Vtx+lGHJt60GutAQM=; b=/RLLFmPkYaBJutNtM3dPFQ4N8ehpxZs1lGqsv0ZPRS9xw2jJA4FT7G727ePI/I47+KfnvI aLtUr1M1SVAHZfBA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id ACC85132FD; Mon, 23 Oct 2023 20:36:22 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id IEFjHUbZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:36:22 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana Subject: [PATCH v2 04/29] migration: Return the saved state from global_state_store Date: Mon, 23 Oct 2023 17:35:43 -0300 Message-Id: <20231023203608.26370-5-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out2.suse.de; none X-Spam-Score: -0.48 X-Spamd-Result: default: False [-0.48 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[7]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-1.38)[90.76%] Received-SPF: pass client-ip=2001:67c:2178:6::1d; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_FILL_THIS_FORM_SHORT=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org There is a pattern of calling runstate_get() to store the current runstate and calling global_state_store() to save the current runstate for migration. Since global_state_store() also calls runstate_get(), make it return the runstate instead. Signed-off-by: Fabiano Rosas Reviewed-by: Daniel P. Berrangé --- include/migration/global_state.h | 2 +- migration/global_state.c | 7 +++++-- migration/migration.c | 6 ++---- 3 files changed, 8 insertions(+), 7 deletions(-) diff --git a/include/migration/global_state.h b/include/migration/global_state.h index d7c2cd3216..e268dc1f18 100644 --- a/include/migration/global_state.h +++ b/include/migration/global_state.h @@ -16,7 +16,7 @@ #include "qapi/qapi-types-run-state.h" void register_global_state(void); -void global_state_store(void); +RunState global_state_store(void); void global_state_store_running(void); bool global_state_received(void); RunState global_state_get_runstate(void); diff --git a/migration/global_state.c b/migration/global_state.c index 4e2a9d8ec0..d094af6198 100644 --- a/migration/global_state.c +++ b/migration/global_state.c @@ -37,9 +37,12 @@ static void global_state_do_store(RunState state) state_str, '\0'); } -void global_state_store(void) +RunState global_state_store(void) { - global_state_do_store(runstate_get()); + RunState r = runstate_get(); + + global_state_do_store(r); + return r; } void global_state_store_running(void) diff --git a/migration/migration.c b/migration/migration.c index 67547eb6a1..0c23117369 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2344,8 +2344,7 @@ static int migration_completion_precopy(MigrationState *s, s->downtime_start = qemu_clock_get_ms(QEMU_CLOCK_REALTIME); qemu_system_wakeup_request(QEMU_WAKEUP_REASON_OTHER, NULL); - s->vm_old_state = runstate_get(); - global_state_store(); + s->vm_old_state = global_state_store(); ret = vm_stop_force_state(RUN_STATE_FINISH_MIGRATE); trace_migration_completion_vm_stop(ret); @@ -3201,9 +3200,8 @@ static void *bg_migration_thread(void *opaque) * transition in vm_stop_force_state() we need to wakeup it up. */ qemu_system_wakeup_request(QEMU_WAKEUP_REASON_OTHER, NULL); - s->vm_old_state = runstate_get(); + s->vm_old_state = global_state_store(); - global_state_store(); /* Forcibly stop VM before saving state of vCPUs and devices */ if (vm_stop_force_state(RUN_STATE_PAUSED)) { goto fail; From patchwork Mon Oct 23 20:35:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854005 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=F64ij8sS; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=5I9hAkaI; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn7P6Z2tz23jl for ; Tue, 24 Oct 2023 07:37:37 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1f9-0002lQ-To; Mon, 23 Oct 2023 16:36:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1f4-0002kO-O7 for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:30 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1f2-0001nb-57 for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:30 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id F072321B2B; Mon, 23 Oct 2023 20:36:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093386; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MN/9Rl5hTxRBM5kTsHsF/qHtTFcfjGc7CfXTqshdHOY=; b=F64ij8sSYsOWIbJCp/iSVGv2odXZJTtGNWysGfu2rQjweehtFH9GAKdXECK2PgzRew8g1x CzDjS9eexiHxQxioYDLI5MNIYGhEo0e1dvnIXi8pIWUXhTKNBYO2zGHRSsGFsriH9+VfIe 662mlxtU1R5s/nmlHaTzfgp2ecQqO0g= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093386; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MN/9Rl5hTxRBM5kTsHsF/qHtTFcfjGc7CfXTqshdHOY=; b=5I9hAkaIp1Q15cvRiwVa1AdtHf3A0xNp8btIqpi6caH4HdBMaNRIBNAKdiC+c6sbSgIoUD zYBc8zH2OWrtVdCg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 16523132FD; Mon, 23 Oct 2023 20:36:24 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id EOywNEjZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:36:24 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana Subject: [PATCH v2 05/29] migration: Introduce global_state_store_once Date: Mon, 23 Oct 2023 17:35:44 -0300 Message-Id: <20231023203608.26370-6-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out1.suse.de; none X-Spam-Score: -2.10 X-Spamd-Result: default: False [-2.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[7]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] Received-SPF: pass client-ip=195.135.220.28; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org There are some situations during migration when we want to change the runstate of the VM, but don't actually want the new runstate to be put on the wire to be restored on the destination VM. In those cases, the pattern is to use global_state_store() to save the state for migration before changing it. One scenario where this happens is when switching the source VM into the FINISH_MIGRATE state. This state only makes sense on the source VM. Another situation is when pausing the source VM prior to migration completion. We are about to introduce a third scenario when the whole migration should be performed with a paused VM. In this case we will want to save the VM runstate at the very start of the migration and that state will be the one restored on the destination regardless of all the runstate changes that happen in between. To achieve that we need to make sure that the other two calls to global_state_store() do not overwrite the state that is to be migrated. Introduce a version of global_state_store() that only saves the state if no other state has already been saved. Signed-off-by: Fabiano Rosas --- include/migration/global_state.h | 1 + migration/global_state.c | 13 +++++++++++++ migration/migration.c | 6 +++--- 3 files changed, 17 insertions(+), 3 deletions(-) diff --git a/include/migration/global_state.h b/include/migration/global_state.h index e268dc1f18..9d3624100e 100644 --- a/include/migration/global_state.h +++ b/include/migration/global_state.h @@ -17,6 +17,7 @@ void register_global_state(void); RunState global_state_store(void); +RunState global_state_store_once(void); void global_state_store_running(void); bool global_state_received(void); RunState global_state_get_runstate(void); diff --git a/migration/global_state.c b/migration/global_state.c index d094af6198..beb00039d9 100644 --- a/migration/global_state.c +++ b/migration/global_state.c @@ -45,6 +45,19 @@ RunState global_state_store(void) return r; } +RunState global_state_store_once(void) +{ + int r; + char *runstate = (char *)global_state.runstate; + + r = qapi_enum_parse(&RunState_lookup, runstate, -1, NULL); + if (r < 0) { + return global_state_store(); + } + + return r; +} + void global_state_store_running(void) { global_state_do_store(RUN_STATE_RUNNING); diff --git a/migration/migration.c b/migration/migration.c index 0c23117369..a6efbd837a 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2144,7 +2144,7 @@ static int postcopy_start(MigrationState *ms, Error **errp) trace_postcopy_start_set_run(); qemu_system_wakeup_request(QEMU_WAKEUP_REASON_OTHER, NULL); - global_state_store(); + global_state_store_once(); ret = vm_stop_force_state(RUN_STATE_FINISH_MIGRATE); if (ret < 0) { goto fail; @@ -2344,7 +2344,7 @@ static int migration_completion_precopy(MigrationState *s, s->downtime_start = qemu_clock_get_ms(QEMU_CLOCK_REALTIME); qemu_system_wakeup_request(QEMU_WAKEUP_REASON_OTHER, NULL); - s->vm_old_state = global_state_store(); + s->vm_old_state = global_state_store_once(); ret = vm_stop_force_state(RUN_STATE_FINISH_MIGRATE); trace_migration_completion_vm_stop(ret); @@ -3200,7 +3200,7 @@ static void *bg_migration_thread(void *opaque) * transition in vm_stop_force_state() we need to wakeup it up. */ qemu_system_wakeup_request(QEMU_WAKEUP_REASON_OTHER, NULL); - s->vm_old_state = global_state_store(); + s->vm_old_state = global_state_store_once(); /* Forcibly stop VM before saving state of vCPUs and devices */ if (vm_stop_force_state(RUN_STATE_PAUSED)) { From patchwork Mon Oct 23 20:35:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854030 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=i98Hf1xC; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=ZG3VbFGQ; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDnBt3Dsnz202k for ; Tue, 24 Oct 2023 07:40:38 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1f8-0002l7-Ha; Mon, 23 Oct 2023 16:36:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1f6-0002kh-OU for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:32 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1f5-0001nr-1n for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:32 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 85DB621B29; Mon, 23 Oct 2023 20:36:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093389; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FpnhhvWkT5fjFh1pW+rIsNyZU3Z0zam/MWzdccJKKlc=; b=i98Hf1xC6hCPwH0njFqVXLuur93QAmp+QzWeLBWeaUrnMhpnNMrhSPfsZzaaTgPPkJ213x mw85cVsygJzUlp8eR/8H/j0azLkMDKmR2ctFoukqP9Z8rjDBNeVsszmXlYOIQdAzzb8ssE f4BlRu5gTpExFWYieWlweqsjgmIjzuA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093389; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FpnhhvWkT5fjFh1pW+rIsNyZU3Z0zam/MWzdccJKKlc=; b=ZG3VbFGQPapwfMp1w4BP1u32HxVzmVN+f/37KienqW1bT4LP1DKzaxTOVt8b3wUJkrebyk TDKj7HIXYdDrTVBg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 67A6F132FD; Mon, 23 Oct 2023 20:36:27 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 4PQODUvZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:36:27 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana , Eric Blake Subject: [PATCH v2 06/29] migration: Add auto-pause capability Date: Mon, 23 Oct 2023 17:35:45 -0300 Message-Id: <20231023203608.26370-7-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out1.suse.de; none X-Spam-Score: -2.10 X-Spamd-Result: default: False [-2.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[8]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] Received-SPF: pass client-ip=195.135.220.28; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Add a capability that allows the management layer to delegate to QEMU the decision of whether to pause a VM and perform a non-live migration. Depending on the type of migration being performed, this could bring performance benefits. Note that the capability is enabled by default but at this moment no migration scheme is making use of it. Signed-off-by: Fabiano Rosas --- migration/migration.c | 19 +++++++++++++++++++ migration/options.c | 9 +++++++++ migration/options.h | 1 + qapi/migration.json | 6 +++++- 4 files changed, 34 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index a6efbd837a..8b0c3b0911 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -124,6 +124,20 @@ migration_channels_and_uri_compatible(const char *uri, Error **errp) return true; } +static bool migration_should_pause(const char *uri) +{ + if (!migrate_auto_pause()) { + return false; + } + + /* + * Return true for migration schemes that benefit from a nonlive + * migration. + */ + + return false; +} + static gint page_request_addr_cmp(gconstpointer ap, gconstpointer bp) { uintptr_t a = (uintptr_t) ap, b = (uintptr_t) bp; @@ -1724,6 +1738,11 @@ void qmp_migrate(const char *uri, bool has_blk, bool blk, } } + if (migration_should_pause(uri)) { + global_state_store(); + vm_stop_force_state(RUN_STATE_PAUSED); + } + if (strstart(uri, "tcp:", &p) || strstart(uri, "unix:", NULL) || strstart(uri, "vsock:", NULL)) { diff --git a/migration/options.c b/migration/options.c index 42fb818956..c3def757fe 100644 --- a/migration/options.c +++ b/migration/options.c @@ -200,6 +200,8 @@ Property migration_properties[] = { DEFINE_PROP_MIG_CAP("x-switchover-ack", MIGRATION_CAPABILITY_SWITCHOVER_ACK), DEFINE_PROP_MIG_CAP("x-dirty-limit", MIGRATION_CAPABILITY_DIRTY_LIMIT), + DEFINE_PROP_BOOL("x-auto-pause", MigrationState, + capabilities[MIGRATION_CAPABILITY_AUTO_PAUSE], true), DEFINE_PROP_END_OF_LIST(), }; @@ -210,6 +212,13 @@ bool migrate_auto_converge(void) return s->capabilities[MIGRATION_CAPABILITY_AUTO_CONVERGE]; } +bool migrate_auto_pause(void) +{ + MigrationState *s = migrate_get_current(); + + return s->capabilities[MIGRATION_CAPABILITY_AUTO_PAUSE]; +} + bool migrate_background_snapshot(void) { MigrationState *s = migrate_get_current(); diff --git a/migration/options.h b/migration/options.h index 237f2d6b4a..d1ba5c9de7 100644 --- a/migration/options.h +++ b/migration/options.h @@ -24,6 +24,7 @@ extern Property migration_properties[]; /* capabilities */ bool migrate_auto_converge(void); +bool migrate_auto_pause(void); bool migrate_background_snapshot(void); bool migrate_block(void); bool migrate_colo(void); diff --git a/qapi/migration.json b/qapi/migration.json index db3df12d6c..74f12adc0e 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -523,6 +523,10 @@ # and can result in more stable read performance. Requires KVM # with accelerator property "dirty-ring-size" set. (Since 8.1) # +# @auto-pause: If enabled, allows QEMU to decide whether to pause the +# VM before migration for an optimal migration performance. +# Enabled by default. (since 8.1) +# # Features: # # @unstable: Members @x-colo and @x-ignore-shared are experimental. @@ -539,7 +543,7 @@ { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] }, 'validate-uuid', 'background-snapshot', 'zero-copy-send', 'postcopy-preempt', 'switchover-ack', - 'dirty-limit'] } + 'dirty-limit', 'auto-pause'] } ## # @MigrationCapabilityStatus: From patchwork Mon Oct 23 20:35:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854008 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=ybKbt+cQ; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=CLJopk4h; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn7k0jfGz23k5 for ; Tue, 24 Oct 2023 07:37:53 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1fH-0002ml-IW; Mon, 23 Oct 2023 16:36:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1fA-0002li-Dk for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:37 -0400 Received: from smtp-out1.suse.de ([2001:67c:2178:6::1c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1f6-0001oD-VX for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:35 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id E367521B28; Mon, 23 Oct 2023 20:36:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093391; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=628rLYMqqtImPvYBgo5lycDzIA+WSGFa/8NoW//PIb0=; b=ybKbt+cQy2U4wjaQZ5zHEG7jNbkFl5Xr2J/W95d2TJWsPhjwiAqhA/+iWxkj0vh2CO+jPJ ToDdFh5x7x+43Sq99nIWy3OduQZXyz32HPGcsmF6tSlFl1Zvrg/Wobyro8Mee8njNExqnD VsmF2mVLa4pgs0i2tLHQ0NvJzG9hCO0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093391; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=628rLYMqqtImPvYBgo5lycDzIA+WSGFa/8NoW//PIb0=; b=CLJopk4hIeARUxGwv1wxjZBMEXRvX+DylhnD/AKiFz+OumZ4Z/rNTjdnb2InqONRudtIm6 JQoTtFFjBYYgQeDw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id EF60113A82; Mon, 23 Oct 2023 20:36:29 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id eIfhLU3ZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:36:29 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana Subject: [PATCH v2 07/29] migration: Run "file:" migration with a stopped VM Date: Mon, 23 Oct 2023 17:35:46 -0300 Message-Id: <20231023203608.26370-8-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out1.suse.de; none X-Spam-Score: -1.53 X-Spamd-Result: default: False [-1.53 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[7]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-2.43)[97.41%] Received-SPF: pass client-ip=2001:67c:2178:6::1c; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The file migration is asynchronous, so it benefits from being done with a stopped VM. Allow the file migration to take benefit of the auto-pause capability. Signed-off-by: Fabiano Rosas --- migration/migration.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/migration/migration.c b/migration/migration.c index 8b0c3b0911..692fbc5ad6 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -135,6 +135,10 @@ static bool migration_should_pause(const char *uri) * migration. */ + if (strstart(uri, "file:", NULL)) { + return true; + } + return false; } From patchwork Mon Oct 23 20:35:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854027 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=o4EBRf86; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=D7oVELd/; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDnBC0rkJz23jl for ; Tue, 24 Oct 2023 07:40:03 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1fI-0002mu-Fo; Mon, 23 Oct 2023 16:36:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1fB-0002lp-RN for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:39 -0400 Received: from smtp-out2.suse.de ([2001:67c:2178:6::1d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1fA-0001oV-78 for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:37 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id DD7A21FE33; Mon, 23 Oct 2023 20:36:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093394; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LIpQJ0V6For/p0/u5J+CsEBzz8Un4GWJz64HkeebkKs=; b=o4EBRf86ozj+VvNIymWVjB2jwv8hIPvKf7T4vU0VrrZM6tYftSVuWU1LNYhSfJWBm3bZDV qK6iPc8Jd5Ih21zONqswP42HHJRS9qIuTUi8mhzUPlKv8X8sxe7rFJ7Dk+qt4Jg3CNtX88 0DliPjywzBJmSM9CKFgIQ+ltFMfAYoE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093394; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LIpQJ0V6For/p0/u5J+CsEBzz8Un4GWJz64HkeebkKs=; b=D7oVELd/r7KORX0g/y40N2/p7bvEpH2mWI7MzGD38kwxOmodpwcsueJnB3XYkrgCJLQYnu Iq4QLiYfikck2CDw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 59B46132FD; Mon, 23 Oct 2023 20:36:32 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id UE0xCVDZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:36:32 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana , Thomas Huth , Laurent Vivier , Paolo Bonzini Subject: [PATCH v2 08/29] tests/qtest: File migration auto-pause tests Date: Mon, 23 Oct 2023 17:35:47 -0300 Message-Id: <20231023203608.26370-9-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out2.suse.de; none X-Spam-Score: 0.90 X-Spamd-Result: default: False [0.90 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[10]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-0.00)[43.90%] Received-SPF: pass client-ip=2001:67c:2178:6::1d; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Adapt the file migration tests to take into account the auto-pause feature. The test currently has a flag 'stop_src' that is used to know if the test itself should stop the VM. Add a new flag 'auto_pause' to enable QEMU to stop the VM instead.. The two in combination allow us to migrate a already stopped VM and check that it is still stopped on the destination (auto-pause in effect restoring the original state). By adding a more precise tracking of migration state changes, we can also make sure that auto-pause is actually stopping the VM right after qmp_migrate(), as opposed to the vm_stop() that happens at migration_complete(). When resuming the destination a similar situation occurs, we use 'stop_src' to have a stopped VM and check that the destination does not get a "resume" event. Signed-off-by: Fabiano Rosas --- tests/qtest/migration-test.c | 41 ++++++++++++++++++++++++++++++------ 1 file changed, 35 insertions(+), 6 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 88e611e98f..06a7dd3c0a 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -1679,7 +1679,7 @@ finish: test_migrate_end(from, to, args->result == MIG_TEST_SUCCEED); } -static void test_file_common(MigrateCommon *args, bool stop_src) +static void test_file_common(MigrateCommon *args, bool stop_src, bool auto_pause) { QTestState *from, *to; void *data_hook = NULL; @@ -1689,6 +1689,13 @@ static void test_file_common(MigrateCommon *args, bool stop_src) return; } + migrate_set_capability(from, "events", true); + migrate_set_capability(to, "events", true); + + if (!auto_pause) { + migrate_set_capability(from, "auto-pause", false); + } + /* * File migration is never live. We can keep the source VM running * during migration, but the destination will not be running @@ -1712,8 +1719,24 @@ static void test_file_common(MigrateCommon *args, bool stop_src) migrate_qmp_fail(from, connect_uri, "{}"); goto finish; } - migrate_qmp(from, connect_uri, "{}"); + + wait_for_setup(from); + + /* auto-pause stops the VM right after setup */ + if (auto_pause && !stop_src) { + wait_for_stop(from); + } + + wait_for_active(from); + + /* + * If the VM is not already stop by the test or auto-pause, + * migration completion will stop it. + */ + if (!stop_src && !auto_pause) { + wait_for_stop(from); + } wait_for_migration_complete(from); /* @@ -1721,9 +1744,15 @@ static void test_file_common(MigrateCommon *args, bool stop_src) * destination. */ migrate_incoming_qmp(to, connect_uri, "{}"); + wait_for_active(to); wait_for_migration_complete(to); - if (stop_src) { + if (stop_src || auto_pause) { + /* + * The VM has been paused on source by either the test or + * auto-pause, re-start on destination to make sure it won't + * crash. + */ qtest_qmp_assert_success(to, "{ 'execute' : 'cont'}"); } @@ -1940,7 +1969,7 @@ static void test_precopy_file(void) .listen_uri = "defer", }; - test_file_common(&args, true); + test_file_common(&args, true, true); } static void file_offset_finish_hook(QTestState *from, QTestState *to, @@ -1984,7 +2013,7 @@ static void test_precopy_file_offset(void) .finish_hook = file_offset_finish_hook, }; - test_file_common(&args, false); + test_file_common(&args, false, true); } static void test_precopy_file_offset_bad(void) @@ -1998,7 +2027,7 @@ static void test_precopy_file_offset_bad(void) .result = MIG_TEST_QMP_ERROR, }; - test_file_common(&args, false); + test_file_common(&args, false, false); } static void test_precopy_tcp_plain(void) From patchwork Mon Oct 23 20:35:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854007 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=nKY1OXK7; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=sP2Yx5vx; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn7k0tkDz23kh for ; Tue, 24 Oct 2023 07:37:53 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1fJ-0002nC-5j; Mon, 23 Oct 2023 16:36:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1fE-0002mH-HZ for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:40 -0400 Received: from smtp-out2.suse.de ([2001:67c:2178:6::1d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1fC-0001ol-QE for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:40 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 7B6001FE37; Mon, 23 Oct 2023 20:36:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093397; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HANNei1RIw7D6aPro8oCqEXXgEDARrXt4Bkj7ntOXS4=; b=nKY1OXK7R3CThwUe2oehbJiIaqGaH7z11ueB2xb4gyGey/hAZeZ3rUzoF8fjbt1b7XYrLH 9JTea3TWwyc+zFpz57ugs0+YMMORc3CCF+sJ2RbK3y6wL0Pll4fYmT688iNjoO62vTrUzn JoXVh0JBkP3sWRN4rFOL2nqgt4s68ms= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093397; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HANNei1RIw7D6aPro8oCqEXXgEDARrXt4Bkj7ntOXS4=; b=sP2Yx5vxBHaTRZE1oTYSK0FRfAgic5Wv3KI7m7RnJEunjKnQQf+WTUJth9EFHkvYCHbUQX 1fMQlEBZJ35FjpBA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 5833E132FD; Mon, 23 Oct 2023 20:36:35 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 2O1eCFPZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:36:35 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana , Nikolay Borisov Subject: [PATCH v2 09/29] io: add and implement QIO_CHANNEL_FEATURE_SEEKABLE for channel file Date: Mon, 23 Oct 2023 17:35:48 -0300 Message-Id: <20231023203608.26370-10-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out2.suse.de; none X-Spam-Score: -4.21 X-Spamd-Result: default: False [-4.21 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[8]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-1.11)[88.29%] Received-SPF: pass client-ip=2001:67c:2178:6::1d; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Nikolay Borisov Add a generic QIOChannel feature SEEKABLE which would be used by the qemu_file* apis. For the time being this will be only implemented for file channels. Signed-off-by: Nikolay Borisov Signed-off-by: Fabiano Rosas Reviewed-by: Daniel P. Berrangé --- include/io/channel.h | 1 + io/channel-file.c | 8 ++++++++ 2 files changed, 9 insertions(+) diff --git a/include/io/channel.h b/include/io/channel.h index 5f9dbaab65..fcb19fd672 100644 --- a/include/io/channel.h +++ b/include/io/channel.h @@ -44,6 +44,7 @@ enum QIOChannelFeature { QIO_CHANNEL_FEATURE_LISTEN, QIO_CHANNEL_FEATURE_WRITE_ZERO_COPY, QIO_CHANNEL_FEATURE_READ_MSG_PEEK, + QIO_CHANNEL_FEATURE_SEEKABLE, }; diff --git a/io/channel-file.c b/io/channel-file.c index 4a12c61886..f91bf6db1c 100644 --- a/io/channel-file.c +++ b/io/channel-file.c @@ -36,6 +36,10 @@ qio_channel_file_new_fd(int fd) ioc->fd = fd; + if (lseek(fd, 0, SEEK_CUR) != (off_t)-1) { + qio_channel_set_feature(QIO_CHANNEL(ioc), QIO_CHANNEL_FEATURE_SEEKABLE); + } + trace_qio_channel_file_new_fd(ioc, fd); return ioc; @@ -60,6 +64,10 @@ qio_channel_file_new_path(const char *path, return NULL; } + if (lseek(ioc->fd, 0, SEEK_CUR) != (off_t)-1) { + qio_channel_set_feature(QIO_CHANNEL(ioc), QIO_CHANNEL_FEATURE_SEEKABLE); + } + trace_qio_channel_file_new_path(ioc, path, flags, mode, ioc->fd); return ioc; From patchwork Mon Oct 23 20:35:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854031 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=uDBO9epv; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=Zy8mzVQn; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDnCB6QQzz202k for ; Tue, 24 Oct 2023 07:40:54 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1fN-0002np-9a; Mon, 23 Oct 2023 16:36:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1fJ-0002nR-G3 for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:45 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1fG-0001ou-8s for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:45 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 1524F1FE33; Mon, 23 Oct 2023 20:36:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093400; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KNFo/1srXoDhurqZlCkYHwr2qvPHJ1okh+0mJbH2U5k=; b=uDBO9epv9Q0xJn83ithrN+TqwxLiIn+riZDY9h9ET3aJxumpqBLNLpeWsrpglYR2p8DZrw uRO6oY7r8S87OqFqLDCwHDjCpEEAMKwAdwHqxY2+RnWxsaTCvc9tgqPn7ZnLxJPecmUaus n0rGXugXH9bdaoGSD1Xzbz4kd+mrskE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093400; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KNFo/1srXoDhurqZlCkYHwr2qvPHJ1okh+0mJbH2U5k=; b=Zy8mzVQn3opbPI9s6MKMv6Zt1VdyuFElvFavOIm51rX1x+WzWAV7OQub7Fgq4enturK7Rg WAlEMk1TxYrx0KBw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id E7354132FD; Mon, 23 Oct 2023 20:36:37 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id MF/MK1XZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:36:37 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana , Nikolay Borisov Subject: [PATCH v2 10/29] io: Add generic pwritev/preadv interface Date: Mon, 23 Oct 2023 17:35:49 -0300 Message-Id: <20231023203608.26370-11-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out2.suse.de; none X-Spam-Score: -2.10 X-Spamd-Result: default: False [-2.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[8]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] Received-SPF: pass client-ip=195.135.220.29; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Nikolay Borisov Introduce basic pwritev/preadv support in the generic channel layer. Specific implementation will follow for the file channel as this is required in order to support migration streams with fixed location of each ram page. Signed-off-by: Nikolay Borisov Signed-off-by: Fabiano Rosas --- include/io/channel.h | 82 ++++++++++++++++++++++++++++++++++++++++++++ io/channel.c | 58 +++++++++++++++++++++++++++++++ 2 files changed, 140 insertions(+) diff --git a/include/io/channel.h b/include/io/channel.h index fcb19fd672..a8181d576a 100644 --- a/include/io/channel.h +++ b/include/io/channel.h @@ -131,6 +131,16 @@ struct QIOChannelClass { Error **errp); /* Optional callbacks */ + ssize_t (*io_pwritev)(QIOChannel *ioc, + const struct iovec *iov, + size_t niov, + off_t offset, + Error **errp); + ssize_t (*io_preadv)(QIOChannel *ioc, + const struct iovec *iov, + size_t niov, + off_t offset, + Error **errp); int (*io_shutdown)(QIOChannel *ioc, QIOChannelShutdown how, Error **errp); @@ -529,6 +539,78 @@ void qio_channel_set_follow_coroutine_ctx(QIOChannel *ioc, bool enabled); int qio_channel_close(QIOChannel *ioc, Error **errp); +/** + * qio_channel_pwritev_full + * @ioc: the channel object + * @iov: the array of memory regions to write data from + * @niov: the length of the @iov array + * @offset: offset in the channel where writes should begin + * @errp: pointer to a NULL-initialized error object + * + * Not all implementations will support this facility, so may report + * an error. To avoid errors, the caller may check for the feature + * flag QIO_CHANNEL_FEATURE_SEEKABLE prior to calling this method. + * + * Behaves as qio_channel_writev_full, apart from not supporting + * sending of file handles as well as beginning the write at the + * passed @offset + * + */ +ssize_t qio_channel_pwritev_full(QIOChannel *ioc, const struct iovec *iov, + size_t niov, off_t offset, Error **errp); + +/** + * qio_channel_pwritev + * @ioc: the channel object + * @buf: the memory region to write data into + * @buflen: the number of bytes to @buf + * @offset: offset in the channel where writes should begin + * @errp: pointer to a NULL-initialized error object + * + * Not all implementations will support this facility, so may report + * an error. To avoid errors, the caller may check for the feature + * flag QIO_CHANNEL_FEATURE_SEEKABLE prior to calling this method. + * + */ +ssize_t qio_channel_pwritev(QIOChannel *ioc, char *buf, size_t buflen, + off_t offset, Error **errp); + +/** + * qio_channel_preadv_full + * @ioc: the channel object + * @iov: the array of memory regions to read data into + * @niov: the length of the @iov array + * @offset: offset in the channel where writes should begin + * @errp: pointer to a NULL-initialized error object + * + * Not all implementations will support this facility, so may report + * an error. To avoid errors, the caller may check for the feature + * flag QIO_CHANNEL_FEATURE_SEEKABLE prior to calling this method. + * + * Behaves as qio_channel_readv_full, apart from not supporting + * receiving of file handles as well as beginning the read at the + * passed @offset + * + */ +ssize_t qio_channel_preadv_full(QIOChannel *ioc, const struct iovec *iov, + size_t niov, off_t offset, Error **errp); + +/** + * qio_channel_preadv + * @ioc: the channel object + * @buf: the memory region to write data into + * @buflen: the number of bytes to @buf + * @offset: offset in the channel where writes should begin + * @errp: pointer to a NULL-initialized error object + * + * Not all implementations will support this facility, so may report + * an error. To avoid errors, the caller may check for the feature + * flag QIO_CHANNEL_FEATURE_SEEKABLE prior to calling this method. + * + */ +ssize_t qio_channel_preadv(QIOChannel *ioc, char *buf, size_t buflen, + off_t offset, Error **errp); + /** * qio_channel_shutdown: * @ioc: the channel object diff --git a/io/channel.c b/io/channel.c index 86c5834510..770d61ea00 100644 --- a/io/channel.c +++ b/io/channel.c @@ -454,6 +454,64 @@ GSource *qio_channel_add_watch_source(QIOChannel *ioc, } +ssize_t qio_channel_pwritev_full(QIOChannel *ioc, const struct iovec *iov, + size_t niov, off_t offset, Error **errp) +{ + QIOChannelClass *klass = QIO_CHANNEL_GET_CLASS(ioc); + + if (!klass->io_pwritev) { + error_setg(errp, "Channel does not support pwritev"); + return -1; + } + + if (!qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_SEEKABLE)) { + error_setg_errno(errp, EINVAL, "Requested channel is not seekable"); + return -1; + } + + return klass->io_pwritev(ioc, iov, niov, offset, errp); +} + +ssize_t qio_channel_pwritev(QIOChannel *ioc, char *buf, size_t buflen, + off_t offset, Error **errp) +{ + struct iovec iov = { + .iov_base = buf, + .iov_len = buflen + }; + + return qio_channel_pwritev_full(ioc, &iov, 1, offset, errp); +} + +ssize_t qio_channel_preadv_full(QIOChannel *ioc, const struct iovec *iov, + size_t niov, off_t offset, Error **errp) +{ + QIOChannelClass *klass = QIO_CHANNEL_GET_CLASS(ioc); + + if (!klass->io_preadv) { + error_setg(errp, "Channel does not support preadv"); + return -1; + } + + if (!qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_SEEKABLE)) { + error_setg_errno(errp, EINVAL, "Requested channel is not seekable"); + return -1; + } + + return klass->io_preadv(ioc, iov, niov, offset, errp); +} + +ssize_t qio_channel_preadv(QIOChannel *ioc, char *buf, size_t buflen, + off_t offset, Error **errp) +{ + struct iovec iov = { + .iov_base = buf, + .iov_len = buflen + }; + + return qio_channel_preadv_full(ioc, &iov, 1, offset, errp); +} + int qio_channel_shutdown(QIOChannel *ioc, QIOChannelShutdown how, Error **errp) From patchwork Mon Oct 23 20:35:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854024 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=Ci4MiACh; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=+PNXiJy8; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn9K2ZCKz23jl for ; Tue, 24 Oct 2023 07:39:17 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1fN-0002oA-Rf; Mon, 23 Oct 2023 16:36:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1fK-0002nc-Ml for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:47 -0400 Received: from smtp-out2.suse.de ([2001:67c:2178:6::1d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1fH-0001pB-VE for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:46 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id A50F31FE38; Mon, 23 Oct 2023 20:36:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093402; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=q8BvrNP6oGgMhVNXyB1l2O+22/tN8WAR3oFhZePALc8=; b=Ci4MiAChIFoDrxAzpghrK8sNTVMzYBpE2R4UZtV9OY2JIxnlHv6fiNezo4ZUr0ba8kqUo2 tjTONJuKimOyCdibvjLio0BzHO3k0zWni6PwbhMN1tfD1uL1ql0ZZxBsAKurhx2M1t2Jla mwPLY0amM5wxf6CseEJWkdDkUuWxmdY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093402; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=q8BvrNP6oGgMhVNXyB1l2O+22/tN8WAR3oFhZePALc8=; b=+PNXiJy8vLnWj+r4W+bcFHusQeoj9c5vaF8fOihlGMwkx+egZ1imIhxMaPWEg33gtmcFqf pv8CVluDHgBVttAQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 7EA69132FD; Mon, 23 Oct 2023 20:36:40 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id QHcsEljZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:36:40 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana , Nikolay Borisov Subject: [PATCH v2 11/29] io: implement io_pwritev/preadv for QIOChannelFile Date: Mon, 23 Oct 2023 17:35:50 -0300 Message-Id: <20231023203608.26370-12-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out2.suse.de; none X-Spam-Score: -6.02 X-Spamd-Result: default: False [-6.02 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[8]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-2.92)[99.65%] Received-SPF: pass client-ip=2001:67c:2178:6::1d; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Nikolay Borisov The upcoming 'fixed-ram' feature will require qemu to write data to (and restore from) specific offsets of the migration file. Add a minimal implementation of pwritev/preadv and expose them via the io_pwritev and io_preadv interfaces. Signed-off-by: Nikolay Borisov Signed-off-by: Fabiano Rosas Reviewed-by: Daniel P. Berrangé --- io/channel-file.c | 52 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/io/channel-file.c b/io/channel-file.c index f91bf6db1c..731a2fbc68 100644 --- a/io/channel-file.c +++ b/io/channel-file.c @@ -146,6 +146,56 @@ static ssize_t qio_channel_file_writev(QIOChannel *ioc, return ret; } +static ssize_t qio_channel_file_preadv(QIOChannel *ioc, + const struct iovec *iov, + size_t niov, + off_t offset, + Error **errp) +{ + QIOChannelFile *fioc = QIO_CHANNEL_FILE(ioc); + ssize_t ret; + + retry: + ret = preadv(fioc->fd, iov, niov, offset); + if (ret < 0) { + if (errno == EAGAIN) { + return QIO_CHANNEL_ERR_BLOCK; + } + if (errno == EINTR) { + goto retry; + } + + error_setg_errno(errp, errno, "Unable to read from file"); + return -1; + } + + return ret; +} + +static ssize_t qio_channel_file_pwritev(QIOChannel *ioc, + const struct iovec *iov, + size_t niov, + off_t offset, + Error **errp) +{ + QIOChannelFile *fioc = QIO_CHANNEL_FILE(ioc); + ssize_t ret; + + retry: + ret = pwritev(fioc->fd, iov, niov, offset); + if (ret <= 0) { + if (errno == EAGAIN) { + return QIO_CHANNEL_ERR_BLOCK; + } + if (errno == EINTR) { + goto retry; + } + error_setg_errno(errp, errno, "Unable to write to file"); + return -1; + } + return ret; +} + static int qio_channel_file_set_blocking(QIOChannel *ioc, bool enabled, Error **errp) @@ -231,6 +281,8 @@ static void qio_channel_file_class_init(ObjectClass *klass, ioc_klass->io_writev = qio_channel_file_writev; ioc_klass->io_readv = qio_channel_file_readv; ioc_klass->io_set_blocking = qio_channel_file_set_blocking; + ioc_klass->io_pwritev = qio_channel_file_pwritev; + ioc_klass->io_preadv = qio_channel_file_preadv; ioc_klass->io_seek = qio_channel_file_seek; ioc_klass->io_close = qio_channel_file_close; ioc_klass->io_create_watch = qio_channel_file_create_watch; From patchwork Mon Oct 23 20:35:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854009 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=zaRq/9zF; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=v5ESw46v; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn7k0fTSz23jl for ; Tue, 24 Oct 2023 07:37:52 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1fP-0002oi-Ej; Mon, 23 Oct 2023 16:36:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1fN-0002o8-J2 for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:49 -0400 Received: from smtp-out2.suse.de ([2001:67c:2178:6::1d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1fK-0001pT-Fu for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:49 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 480D41FE35; Mon, 23 Oct 2023 20:36:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093405; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9zTk8RuUB8J5Jl5LJdYeolkcSZcE9CG74FoPNVMi374=; b=zaRq/9zF99cFIiNuo9ubSyXpCC3czzReFbih7irtKLH0N2hIsASWcbSRzhi/r0MXhJCuCr btHREqV0O40nnk2YyrBHNvEnUkbUcK3u1xS2nvu+NQCdVzfjUqHQC4HVYLichIQutN3LO3 6eIMD4ZL9jv2h8l0yPJTTGNz/xFjP9E= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093405; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9zTk8RuUB8J5Jl5LJdYeolkcSZcE9CG74FoPNVMi374=; b=v5ESw46vayZuqQIlwKVz+teQlYFBTjbFQ1Z0IRzCsCxRdbyi78q3IyJo4L8H/r4joKCp5j wcdXx7BaQpDMNfDQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 1D2BF132FD; Mon, 23 Oct 2023 20:36:42 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id UCI5NlrZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:36:42 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana , Nikolay Borisov Subject: [PATCH v2 12/29] migration/qemu-file: add utility methods for working with seekable channels Date: Mon, 23 Oct 2023 17:35:51 -0300 Message-Id: <20231023203608.26370-13-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out2.suse.de; none X-Spam-Score: -2.10 X-Spamd-Result: default: False [-2.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[8]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] Received-SPF: pass client-ip=2001:67c:2178:6::1d; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Nikolay Borisov Add utility methods that will be needed when implementing 'fixed-ram' migration capability. qemu_file_is_seekable qemu_put_buffer_at qemu_get_buffer_at qemu_set_offset qemu_get_offset Signed-off-by: Nikolay Borisov Signed-off-by: Fabiano Rosas Reviewed-by: Daniel P. Berrangé --- fixed total_transferred accounting restructured to use qio_channel_file_preadv instead of the _full variant --- include/migration/qemu-file-types.h | 2 + migration/qemu-file.c | 80 +++++++++++++++++++++++++++++ migration/qemu-file.h | 4 ++ 3 files changed, 86 insertions(+) diff --git a/include/migration/qemu-file-types.h b/include/migration/qemu-file-types.h index 9ba163f333..adec5abc07 100644 --- a/include/migration/qemu-file-types.h +++ b/include/migration/qemu-file-types.h @@ -50,6 +50,8 @@ unsigned int qemu_get_be16(QEMUFile *f); unsigned int qemu_get_be32(QEMUFile *f); uint64_t qemu_get_be64(QEMUFile *f); +bool qemu_file_is_seekable(QEMUFile *f); + static inline void qemu_put_be64s(QEMUFile *f, const uint64_t *pv) { qemu_put_be64(f, *pv); diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 3fb25148d1..c8f635aa2b 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -33,6 +33,7 @@ #include "options.h" #include "qapi/error.h" #include "rdma.h" +#include "io/channel-file.h" #define IO_BUF_SIZE 32768 #define MAX_IOV_SIZE MIN_CONST(IOV_MAX, 64) @@ -258,6 +259,10 @@ static void qemu_iovec_release_ram(QEMUFile *f) memset(f->may_free, 0, sizeof(f->may_free)); } +bool qemu_file_is_seekable(QEMUFile *f) +{ + return qio_channel_has_feature(f->ioc, QIO_CHANNEL_FEATURE_SEEKABLE); +} /** * Flushes QEMUFile buffer @@ -460,6 +465,81 @@ void qemu_put_buffer(QEMUFile *f, const uint8_t *buf, size_t size) } } +void qemu_put_buffer_at(QEMUFile *f, const uint8_t *buf, size_t buflen, off_t pos) +{ + Error *err = NULL; + + if (f->last_error) { + return; + } + + qemu_fflush(f); + qio_channel_pwritev(f->ioc, (char *)buf, buflen, pos, &err); + + if (err) { + qemu_file_set_error_obj(f, -EIO, err); + } else { + f->total_transferred += buflen; + } + + return; +} + + +size_t qemu_get_buffer_at(QEMUFile *f, const uint8_t *buf, size_t buflen, off_t pos) +{ + Error *err = NULL; + ssize_t ret; + + if (f->last_error) { + return 0; + } + + ret = qio_channel_preadv(f->ioc, (char *)buf, buflen, pos, &err); + if (ret == -1 || err) { + goto error; + } + + return (size_t)ret; + + error: + qemu_file_set_error_obj(f, -EIO, err); + return 0; +} + +void qemu_set_offset(QEMUFile *f, off_t off, int whence) +{ + Error *err = NULL; + off_t ret; + + qemu_fflush(f); + + if (!qemu_file_is_writable(f)) { + f->buf_index = 0; + f->buf_size = 0; + } + + ret = qio_channel_io_seek(f->ioc, off, whence, &err); + if (ret == (off_t)-1) { + qemu_file_set_error_obj(f, -EIO, err); + } +} + +off_t qemu_get_offset(QEMUFile *f) +{ + Error *err = NULL; + off_t ret; + + qemu_fflush(f); + + ret = qio_channel_io_seek(f->ioc, 0, SEEK_CUR, &err); + if (ret == (off_t)-1) { + qemu_file_set_error_obj(f, -EIO, err); + } + return ret; +} + + void qemu_put_byte(QEMUFile *f, int v) { if (f->last_error) { diff --git a/migration/qemu-file.h b/migration/qemu-file.h index a29c37b0d0..fef4a35a6a 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -93,6 +93,10 @@ QEMUFile *qemu_file_get_return_path(QEMUFile *f); void qemu_fflush(QEMUFile *f); void qemu_file_set_blocking(QEMUFile *f, bool block); int qemu_file_get_to_fd(QEMUFile *f, int fd, size_t size); +void qemu_set_offset(QEMUFile *f, off_t off, int whence); +off_t qemu_get_offset(QEMUFile *f); +void qemu_put_buffer_at(QEMUFile *f, const uint8_t *buf, size_t buflen, off_t pos); +size_t qemu_get_buffer_at(QEMUFile *f, const uint8_t *buf, size_t buflen, off_t pos); QIOChannel *qemu_file_get_ioc(QEMUFile *file); From patchwork Mon Oct 23 20:35:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854032 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=bcw9NZsx; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=/8mGSE9p; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDnCN0kjCz202k for ; Tue, 24 Oct 2023 07:41:04 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1fT-0002vf-12; Mon, 23 Oct 2023 16:36:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1fO-0002oN-Mz for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:50 -0400 Received: from smtp-out2.suse.de ([2001:67c:2178:6::1d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1fN-0001pf-8a for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:50 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id A35121FE32; Mon, 23 Oct 2023 20:36:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093407; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pAVFxiDFeacmkca8s9WK4yCViaCO1YWTERbARIPhDI8=; b=bcw9NZsxz04Ko6jGlR+OsyJDzlb65Wn0FPZj544/m6tWMMSDwFF+mPM7ZZpub4zcNzYR6X iK5SLHZevF7DgsPiMd5A64q1i0O0rzIWhWpFV53xg7OyxIruuuTfYb7mmCJ8oaol1bMNEe P2lAqWeE6IhwzP07yJUpYgM5E1vfVoE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093407; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pAVFxiDFeacmkca8s9WK4yCViaCO1YWTERbARIPhDI8=; b=/8mGSE9pY596x5sn2ezfveEezartCd1kqhl6vLkFTRen8aUWR2JnnjK4ppRXw4NTgVU9qq 8/F3Xc00lhqxqOBw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id B0DE2132FD; Mon, 23 Oct 2023 20:36:45 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 4PjLHl3ZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:36:45 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana Subject: [PATCH v2 13/29] migration: fixed-ram: Add URI compatibility check Date: Mon, 23 Oct 2023 17:35:52 -0300 Message-Id: <20231023203608.26370-14-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out2.suse.de; none X-Spam-Score: -1.30 X-Spamd-Result: default: False [-1.30 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[7]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-2.20)[96.17%] Received-SPF: pass client-ip=2001:67c:2178:6::1d; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The fixed-ram migration format needs a channel that supports seeking to be able to write each page to an arbitrary offset in the migration stream. Signed-off-by: Fabiano Rosas Reviewed-by: Daniel P. Berrangé --- migration/migration.c | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 692fbc5ad6..cabb3ad3a5 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -106,22 +106,40 @@ static bool migration_needs_multiple_sockets(void) return migrate_multifd() || migrate_postcopy_preempt(); } +static bool migration_needs_seekable_channel(void) +{ + return migrate_fixed_ram(); +} + static bool uri_supports_multi_channels(const char *uri) { return strstart(uri, "tcp:", NULL) || strstart(uri, "unix:", NULL) || strstart(uri, "vsock:", NULL); } +static bool uri_supports_seeking(const char *uri) +{ + return strstart(uri, "file:", NULL); +} + static bool migration_channels_and_uri_compatible(const char *uri, Error **errp) { + bool compatible = true; + + if (migration_needs_seekable_channel() && + !uri_supports_seeking(uri)) { + error_setg(errp, "Migration requires seekable transport (e.g. file)"); + compatible = false; + } + if (migration_needs_multiple_sockets() && !uri_supports_multi_channels(uri)) { error_setg(errp, "Migration requires multi-channel URIs (e.g. tcp)"); - return false; + compatible = false; } - return true; + return compatible; } static bool migration_should_pause(const char *uri) From patchwork Mon Oct 23 20:35:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854003 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=STAJKGlh; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=EaG+CeKD; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn6z4gVMz23jl for ; Tue, 24 Oct 2023 07:37:15 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1fU-00037h-Ba; Mon, 23 Oct 2023 16:36:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1fS-00030F-FM for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:54 -0400 Received: from smtp-out2.suse.de ([2001:67c:2178:6::1d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1fP-0001pq-94 for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:53 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 36D091FE3B; Mon, 23 Oct 2023 20:36:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093410; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4G/cHUxn6iGZmUsTZYq/RXjW4IQMnPKPDGRmRoZnArQ=; b=STAJKGlh90ilMmpdhhabSXVA1VdV6Gep5p1zVmIHSnYyimfffyxuHBQQuIvu2E4I9lkrml zIWPGqGd8LnqHQg1oNJgC29RlZU7XHgFtTbsTeQNrA9L+FJ1aDc1bDEAL9TakR40z+6Aa0 zrjMsj7HVxblMy5c9fGldVXKaZRhU3k= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093410; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4G/cHUxn6iGZmUsTZYq/RXjW4IQMnPKPDGRmRoZnArQ=; b=EaG+CeKDVXLHxCyka8Z4lzz6tWjNR3O05CqQcx5Bu4yiZfTmdIzEjBGnoQhEW9HEIrk8B1 sW+DtZdsaBo6UnBw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 1C926132FD; Mon, 23 Oct 2023 20:36:47 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id OAoUNl/ZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:36:47 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana , Eric Blake Subject: [PATCH v2 14/29] migration/ram: Introduce 'fixed-ram' migration capability Date: Mon, 23 Oct 2023 17:35:53 -0300 Message-Id: <20231023203608.26370-15-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out2.suse.de; none X-Spam-Score: -2.10 X-Spamd-Result: default: False [-2.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[8]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] Received-SPF: pass client-ip=2001:67c:2178:6::1d; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Add a new migration capability 'fixed-ram'. The core of the feature is to ensure that each ram page has a specific offset in the resulting migration stream. The reason why we'd want such behavior are two fold: - When doing a 'fixed-ram' migration the resulting file will have a bounded size, since pages which are dirtied multiple times will always go to a fixed location in the file, rather than constantly being added to a sequential stream. This eliminates cases where a vm with, say, 1G of ram can result in a migration file that's 10s of GBs, provided that the workload constantly redirties memory. - It paves the way to implement DIRECT_IO-enabled save/restore of the migration stream as the pages are ensured to be written at aligned offsets. For now, enabling the capability has no effect. The next couple of patches implement the core funcionality. Signed-off-by: Fabiano Rosas --- docs/devel/migration.rst | 14 ++++++++++++++ migration/options.c | 37 +++++++++++++++++++++++++++++++++++++ migration/options.h | 1 + migration/savevm.c | 1 + qapi/migration.json | 5 ++++- 5 files changed, 57 insertions(+), 1 deletion(-) diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst index c3e1400c0c..6f898b5dbd 100644 --- a/docs/devel/migration.rst +++ b/docs/devel/migration.rst @@ -566,6 +566,20 @@ Others (especially either older devices or system devices which for some reason don't have a bus concept) make use of the ``instance id`` for otherwise identically named devices. +Fixed-ram format +---------------- + +When the ``fixed-ram`` capability is enabled, a slightly different +stream format is used for the RAM section. Instead of having a +sequential stream of pages that follow the RAMBlock headers, the dirty +pages for a RAMBlock follow its header. This ensures that each RAM +page has a fixed offset in the resulting migration stream. + +The ``fixed-ram`` capaility can be enabled in both source and +destination with: + + ``migrate_set_capability fixed-ram on`` + Return path ----------- diff --git a/migration/options.c b/migration/options.c index c3def757fe..2622d8c483 100644 --- a/migration/options.c +++ b/migration/options.c @@ -202,6 +202,7 @@ Property migration_properties[] = { DEFINE_PROP_MIG_CAP("x-dirty-limit", MIGRATION_CAPABILITY_DIRTY_LIMIT), DEFINE_PROP_BOOL("x-auto-pause", MigrationState, capabilities[MIGRATION_CAPABILITY_AUTO_PAUSE], true), + DEFINE_PROP_MIG_CAP("x-fixed-ram", MIGRATION_CAPABILITY_FIXED_RAM), DEFINE_PROP_END_OF_LIST(), }; @@ -268,6 +269,16 @@ bool migrate_events(void) return s->capabilities[MIGRATION_CAPABILITY_EVENTS]; } +bool migrate_fixed_ram(void) +{ +/* + MigrationState *s = migrate_get_current(); + + return s->capabilities[MIGRATION_CAPABILITY_FIXED_RAM]; +*/ + return false; +} + bool migrate_ignore_shared(void) { MigrationState *s = migrate_get_current(); @@ -627,6 +638,32 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) } } + if (new_caps[MIGRATION_CAPABILITY_FIXED_RAM]) { + if (new_caps[MIGRATION_CAPABILITY_MULTIFD]) { + error_setg(errp, + "Fixed-ram migration is incompatible with multifd"); + return false; + } + + if (new_caps[MIGRATION_CAPABILITY_XBZRLE]) { + error_setg(errp, + "Fixed-ram migration is incompatible with xbzrle"); + return false; + } + + if (new_caps[MIGRATION_CAPABILITY_COMPRESS]) { + error_setg(errp, + "Fixed-ram migration is incompatible with compression"); + return false; + } + + if (new_caps[MIGRATION_CAPABILITY_POSTCOPY_RAM]) { + error_setg(errp, + "Fixed-ram migration is incompatible with postcopy ram"); + return false; + } + } + return true; } diff --git a/migration/options.h b/migration/options.h index d1ba5c9de7..2a9e0e9e13 100644 --- a/migration/options.h +++ b/migration/options.h @@ -32,6 +32,7 @@ bool migrate_compress(void); bool migrate_dirty_bitmaps(void); bool migrate_dirty_limit(void); bool migrate_events(void); +bool migrate_fixed_ram(void); bool migrate_ignore_shared(void); bool migrate_late_block_activate(void); bool migrate_multifd(void); diff --git a/migration/savevm.c b/migration/savevm.c index 8622f229e5..54e084122a 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -243,6 +243,7 @@ static bool should_validate_capability(int capability) /* Validate only new capabilities to keep compatibility. */ switch (capability) { case MIGRATION_CAPABILITY_X_IGNORE_SHARED: + case MIGRATION_CAPABILITY_FIXED_RAM: return true; default: return false; diff --git a/qapi/migration.json b/qapi/migration.json index 74f12adc0e..1317dd32ab 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -527,6 +527,9 @@ # VM before migration for an optimal migration performance. # Enabled by default. (since 8.1) # +# @fixed-ram: Migrate using fixed offsets for each RAM page. Requires +# a seekable transport such as a file. (since 8.1) +# # Features: # # @unstable: Members @x-colo and @x-ignore-shared are experimental. @@ -543,7 +546,7 @@ { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] }, 'validate-uuid', 'background-snapshot', 'zero-copy-send', 'postcopy-preempt', 'switchover-ack', - 'dirty-limit', 'auto-pause'] } + 'dirty-limit', 'auto-pause', 'fixed-ram'] } ## # @MigrationCapabilityStatus: From patchwork Mon Oct 23 20:35:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854010 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=f7aF1wFh; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=rVDcB+bU; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn810fGlz23jl for ; Tue, 24 Oct 2023 07:38:09 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1fZ-0003H0-4W; Mon, 23 Oct 2023 16:37:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1fV-0003CT-96 for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:57 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1fT-0001qF-3W for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:36:56 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 7BEC61FE3C; Mon, 23 Oct 2023 20:36:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093413; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/jivMw68Fg3qDh8bc8bKUmBAFfcKozzLzYpUxkRQMH4=; b=f7aF1wFhRjkedWZj5cEam73J58GixIGgT0WKT/h1jfShQrgpRaQWjDYLZq/iamWGcCoD0c /RkVDP0/rpgJU0b9a9ykQF/K5Fke2yenmsWWNkAmgAxeZMS0LutEK3J+BpkB42G97BKu+4 kXWnM1c6s/o41QhKFOiX6/SEqapbzV4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093413; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/jivMw68Fg3qDh8bc8bKUmBAFfcKozzLzYpUxkRQMH4=; b=rVDcB+bUYw5E+W/bPIqR3/04YmuzGgdH2816DWgQO5qEGZx+LKvzRAvQGudFuCu+mW/EC5 z+ZzMKga5xWq7VAw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id A45FD132FD; Mon, 23 Oct 2023 20:36:50 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id KB3WG2LZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:36:50 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana , Nikolay Borisov , Paolo Bonzini , David Hildenbrand , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PATCH v2 15/29] migration/ram: Add support for 'fixed-ram' outgoing migration Date: Mon, 23 Oct 2023 17:35:54 -0300 Message-Id: <20231023203608.26370-16-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out2.suse.de; none X-Spam-Score: -2.10 X-Spamd-Result: default: False [-2.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[11]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] Received-SPF: pass client-ip=195.135.220.29; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Nikolay Borisov Implement the outgoing migration side for the 'fixed-ram' capability. A bitmap is introduced to track which pages have been written in the migration file. Pages are written at a fixed location for every ramblock. Zero pages are ignored as they'd be zero in the destination migration as well. The migration stream is altered to put the dirty pages for a ramblock after its header instead of having a sequential stream of pages that follow the ramblock headers. Since all pages have a fixed location, RAM_SAVE_FLAG_EOS is no longer generated on every migration iteration. Without fixed-ram (current): ramblock 1 header|ramblock 2 header|...|RAM_SAVE_FLAG_EOS|stream of pages (iter 1)|RAM_SAVE_FLAG_EOS|stream of pages (iter 2)|... With fixed-ram (new): ramblock 1 header|ramblock 1 fixed-ram header|ramblock 1 pages (fixed offsets)|ramblock 2 header|ramblock 2 fixed-ram header|ramblock 2 pages (fixed offsets)|...|RAM_SAVE_FLAG_EOS where: - ramblock header: the generic information for a ramblock, such as idstr, used_len, etc. - ramblock fixed-ram header: the new information added by this feature: bitmap of pages written, bitmap size and offset of pages in the migration file. Signed-off-by: Nikolay Borisov Signed-off-by: Fabiano Rosas --- include/exec/ramblock.h | 8 ++++ migration/options.c | 3 -- migration/ram.c | 98 ++++++++++++++++++++++++++++++++++++----- 3 files changed, 96 insertions(+), 13 deletions(-) diff --git a/include/exec/ramblock.h b/include/exec/ramblock.h index 69c6a53902..e0e3f16852 100644 --- a/include/exec/ramblock.h +++ b/include/exec/ramblock.h @@ -44,6 +44,14 @@ struct RAMBlock { size_t page_size; /* dirty bitmap used during migration */ unsigned long *bmap; + /* shadow dirty bitmap used when migrating to a file */ + unsigned long *shadow_bmap; + /* + * offset in the file pages belonging to this ramblock are saved, + * used only during migration to a file. + */ + off_t bitmap_offset; + uint64_t pages_offset; /* bitmap of already received pages in postcopy */ unsigned long *receivedmap; diff --git a/migration/options.c b/migration/options.c index 2622d8c483..9f693d909f 100644 --- a/migration/options.c +++ b/migration/options.c @@ -271,12 +271,9 @@ bool migrate_events(void) bool migrate_fixed_ram(void) { -/* MigrationState *s = migrate_get_current(); return s->capabilities[MIGRATION_CAPABILITY_FIXED_RAM]; -*/ - return false; } bool migrate_ignore_shared(void) diff --git a/migration/ram.c b/migration/ram.c index 92769902bb..152a03604f 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1157,12 +1157,18 @@ static int save_zero_page(RAMState *rs, PageSearchStatus *pss, return 0; } + stat64_add(&mig_stats.zero_pages, 1); + + if (migrate_fixed_ram()) { + /* zero pages are not transferred with fixed-ram */ + clear_bit(offset >> TARGET_PAGE_BITS, pss->block->shadow_bmap); + return 1; + } + len += save_page_header(pss, file, pss->block, offset | RAM_SAVE_FLAG_ZERO); qemu_put_byte(file, 0); len += 1; ram_release_page(pss->block->idstr, offset); - - stat64_add(&mig_stats.zero_pages, 1); ram_transferred_add(len); /* @@ -1220,14 +1226,20 @@ static int save_normal_page(PageSearchStatus *pss, RAMBlock *block, { QEMUFile *file = pss->pss_channel; - ram_transferred_add(save_page_header(pss, pss->pss_channel, block, - offset | RAM_SAVE_FLAG_PAGE)); - if (async) { - qemu_put_buffer_async(file, buf, TARGET_PAGE_SIZE, - migrate_release_ram() && - migration_in_postcopy()); + if (migrate_fixed_ram()) { + qemu_put_buffer_at(file, buf, TARGET_PAGE_SIZE, + block->pages_offset + offset); + set_bit(offset >> TARGET_PAGE_BITS, block->shadow_bmap); } else { - qemu_put_buffer(file, buf, TARGET_PAGE_SIZE); + ram_transferred_add(save_page_header(pss, pss->pss_channel, block, + offset | RAM_SAVE_FLAG_PAGE)); + if (async) { + qemu_put_buffer_async(file, buf, TARGET_PAGE_SIZE, + migrate_release_ram() && + migration_in_postcopy()); + } else { + qemu_put_buffer(file, buf, TARGET_PAGE_SIZE); + } } ram_transferred_add(TARGET_PAGE_SIZE); stat64_add(&mig_stats.normal_pages, 1); @@ -2475,6 +2487,8 @@ static void ram_save_cleanup(void *opaque) block->clear_bmap = NULL; g_free(block->bmap); block->bmap = NULL; + g_free(block->shadow_bmap); + block->shadow_bmap = NULL; } xbzrle_cleanup(); @@ -2842,6 +2856,7 @@ static void ram_list_init_bitmaps(void) */ block->bmap = bitmap_new(pages); bitmap_set(block->bmap, 0, pages); + block->shadow_bmap = bitmap_new(block->used_length >> TARGET_PAGE_BITS); block->clear_bmap_shift = shift; block->clear_bmap = bitmap_new(clear_bmap_size(pages, shift)); } @@ -2979,6 +2994,44 @@ void qemu_guest_free_page_hint(void *addr, size_t len) } } +#define FIXED_RAM_HDR_VERSION 1 +struct FixedRamHeader { + uint32_t version; + uint64_t page_size; + uint64_t bitmap_offset; + uint64_t pages_offset; + /* end of v1 */ +} QEMU_PACKED; + +static void fixed_ram_insert_header(QEMUFile *file, RAMBlock *block) +{ + g_autofree struct FixedRamHeader *header; + size_t header_size, bitmap_size; + long num_pages; + + header = g_new0(struct FixedRamHeader, 1); + header_size = sizeof(struct FixedRamHeader); + + num_pages = block->used_length >> TARGET_PAGE_BITS; + bitmap_size = BITS_TO_LONGS(num_pages) * sizeof(unsigned long); + + /* + * Save the file offsets of where the bitmap and the pages should + * go as they are written at the end of migration and during the + * iterative phase, respectively. + */ + block->bitmap_offset = qemu_get_offset(file) + header_size; + block->pages_offset = ROUND_UP(block->bitmap_offset + + bitmap_size, 0x100000); + + header->version = cpu_to_be32(FIXED_RAM_HDR_VERSION); + header->page_size = cpu_to_be64(TARGET_PAGE_SIZE); + header->bitmap_offset = cpu_to_be64(block->bitmap_offset); + header->pages_offset = cpu_to_be64(block->pages_offset); + + qemu_put_buffer(file, (uint8_t *) header, header_size); +} + /* * Each of ram_save_setup, ram_save_iterate and ram_save_complete has * long-running RCU critical section. When rcu-reclaims in the code @@ -3028,6 +3081,12 @@ static int ram_save_setup(QEMUFile *f, void *opaque) if (migrate_ignore_shared()) { qemu_put_be64(f, block->mr->addr); } + + if (migrate_fixed_ram()) { + fixed_ram_insert_header(f, block); + /* prepare offset for next ramblock */ + qemu_set_offset(f, block->pages_offset + block->used_length, SEEK_SET); + } } } @@ -3061,6 +3120,20 @@ static int ram_save_setup(QEMUFile *f, void *opaque) return 0; } +static void ram_save_shadow_bmap(QEMUFile *f) +{ + RAMBlock *block; + + RAMBLOCK_FOREACH_MIGRATABLE(block) { + long num_pages = block->used_length >> TARGET_PAGE_BITS; + long bitmap_size = BITS_TO_LONGS(num_pages) * sizeof(unsigned long); + qemu_put_buffer_at(f, (uint8_t *)block->shadow_bmap, bitmap_size, + block->bitmap_offset); + /* to catch any thread late sending pages */ + block->shadow_bmap = NULL; + } +} + /** * ram_save_iterate: iterative stage for migration * @@ -3179,7 +3252,6 @@ out: qemu_put_be64(f, RAM_SAVE_FLAG_EOS); qemu_fflush(f); ram_transferred_add(8); - ret = qemu_file_get_error(f); } if (ret < 0) { @@ -3256,7 +3328,13 @@ static int ram_save_complete(QEMUFile *f, void *opaque) if (migrate_multifd() && !migrate_multifd_flush_after_each_section()) { qemu_put_be64(f, RAM_SAVE_FLAG_MULTIFD_FLUSH); } + + if (migrate_fixed_ram()) { + ram_save_shadow_bmap(f); + } + qemu_put_be64(f, RAM_SAVE_FLAG_EOS); + qemu_fflush(f); return 0; From patchwork Mon Oct 23 20:35:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854026 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=PPjmGxyR; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=70PGgajt; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn9v4fmHz23k5 for ; Tue, 24 Oct 2023 07:39:47 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1fc-0003PJ-6J; Mon, 23 Oct 2023 16:37:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1fZ-0003Ik-E4 for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:01 -0400 Received: from smtp-out2.suse.de ([2001:67c:2178:6::1d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1fX-0001r9-NM for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:01 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 10E361FE3B; Mon, 23 Oct 2023 20:36:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093416; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zj1w64ezVZU0bavfntnDEAPLXAKVFP1yUxizTzb9PgE=; b=PPjmGxyRfPitH0sZ4iawF9yATbWrtqVLNeYh8JQi4K/iW6CIoSuTraghtlCYYXhN7DtLqT tjLvPWJtpRNKwWNvXMLhBx5sllzl1LPkYRk5DKrFxZyoZZ0Dw4B+pynA0so9Ci6pLilW/x wCpb+bMQY9Mu3Ez8rfIDBHT9ZV4OtGY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093416; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zj1w64ezVZU0bavfntnDEAPLXAKVFP1yUxizTzb9PgE=; b=70PGgajtGYydJ3eUsakGkdNwVMlyrr/Il7LTz0yQFaQKkrnq7Xt0H3bcYiBaaMlLbkBxuU Bd49e+BDUTyzXAAg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id E4AA6132FD; Mon, 23 Oct 2023 20:36:53 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id YFZkK2XZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:36:53 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana , Nikolay Borisov Subject: [PATCH v2 16/29] migration/ram: Add support for 'fixed-ram' migration restore Date: Mon, 23 Oct 2023 17:35:55 -0300 Message-Id: <20231023203608.26370-17-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out2.suse.de; none X-Spam-Score: -0.57 X-Spamd-Result: default: False [-0.57 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[8]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-1.47)[91.47%] Received-SPF: pass client-ip=2001:67c:2178:6::1d; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Nikolay Borisov Add the necessary code to parse the format changes for the 'fixed-ram' capability. One of the more notable changes in behavior is that in the 'fixed-ram' case ram pages are restored in one go rather than constantly looping through the migration stream. Signed-off-by: Nikolay Borisov Signed-off-by: Fabiano Rosas --- (farosas) reused more of the common code by making the fixed-ram function take only one ramblock and calling it from inside parse_ramblock. --- migration/ram.c | 93 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 93 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 152a03604f..cea6971ab2 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3032,6 +3032,32 @@ static void fixed_ram_insert_header(QEMUFile *file, RAMBlock *block) qemu_put_buffer(file, (uint8_t *) header, header_size); } +static int fixed_ram_read_header(QEMUFile *file, struct FixedRamHeader *header) +{ + size_t ret, header_size = sizeof(struct FixedRamHeader); + + ret = qemu_get_buffer(file, (uint8_t *)header, header_size); + if (ret != header_size) { + return -1; + } + + /* migration stream is big-endian */ + be32_to_cpus(&header->version); + + if (header->version > FIXED_RAM_HDR_VERSION) { + error_report("Migration fixed-ram capability version mismatch (expected %d, got %d)", + FIXED_RAM_HDR_VERSION, header->version); + return -1; + } + + be64_to_cpus(&header->page_size); + be64_to_cpus(&header->bitmap_offset); + be64_to_cpus(&header->pages_offset); + + + return 0; +} + /* * Each of ram_save_setup, ram_save_iterate and ram_save_complete has * long-running RCU critical section. When rcu-reclaims in the code @@ -3932,6 +3958,68 @@ void colo_flush_ram_cache(void) trace_colo_flush_ram_cache_end(); } +static void read_ramblock_fixed_ram(QEMUFile *f, RAMBlock *block, + long num_pages, unsigned long *bitmap) +{ + unsigned long set_bit_idx, clear_bit_idx; + unsigned long len; + ram_addr_t offset; + void *host; + size_t read, completed, read_len; + + for (set_bit_idx = find_first_bit(bitmap, num_pages); + set_bit_idx < num_pages; + set_bit_idx = find_next_bit(bitmap, num_pages, clear_bit_idx + 1)) { + + clear_bit_idx = find_next_zero_bit(bitmap, num_pages, set_bit_idx + 1); + + len = TARGET_PAGE_SIZE * (clear_bit_idx - set_bit_idx); + offset = set_bit_idx << TARGET_PAGE_BITS; + + for (read = 0, completed = 0; completed < len; offset += read) { + host = host_from_ram_block_offset(block, offset); + read_len = MIN(len, TARGET_PAGE_SIZE); + + read = qemu_get_buffer_at(f, host, read_len, + block->pages_offset + offset); + completed += read; + } + } +} + +static int parse_ramblock_fixed_ram(QEMUFile *f, RAMBlock *block, ram_addr_t length) +{ + g_autofree unsigned long *bitmap = NULL; + struct FixedRamHeader header; + size_t bitmap_size; + long num_pages; + int ret = 0; + + ret = fixed_ram_read_header(f, &header); + if (ret < 0) { + error_report("Error reading fixed-ram header"); + return -EINVAL; + } + + block->pages_offset = header.pages_offset; + num_pages = length / header.page_size; + bitmap_size = BITS_TO_LONGS(num_pages) * sizeof(unsigned long); + + bitmap = g_malloc0(bitmap_size); + if (qemu_get_buffer_at(f, (uint8_t *)bitmap, bitmap_size, + header.bitmap_offset) != bitmap_size) { + error_report("Error parsing dirty bitmap"); + return -EINVAL; + } + + read_ramblock_fixed_ram(f, block, num_pages, bitmap); + + /* Skip pages array */ + qemu_set_offset(f, block->pages_offset + length, SEEK_SET); + + return ret; +} + static int parse_ramblock(QEMUFile *f, RAMBlock *block, ram_addr_t length) { int ret = 0; @@ -3940,6 +4028,10 @@ static int parse_ramblock(QEMUFile *f, RAMBlock *block, ram_addr_t length) assert(block); + if (migrate_fixed_ram()) { + return parse_ramblock_fixed_ram(f, block, length); + } + if (!qemu_ram_is_migratable(block)) { error_report("block %s should not be migrated !", block->idstr); return -EINVAL; @@ -4142,6 +4234,7 @@ static int ram_load_precopy(QEMUFile *f) migrate_multifd_flush_after_each_section()) { multifd_recv_sync_main(); } + break; case RAM_SAVE_FLAG_HOOK: ret = rdma_registration_handle(f); From patchwork Mon Oct 23 20:35:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854020 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=wUBHmhdq; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=VnwnjOR2; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn9749Vjz23jl for ; Tue, 24 Oct 2023 07:39:07 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1fg-0003UI-AW; Mon, 23 Oct 2023 16:37:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1fa-0003Jz-Kk for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:02 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1fY-0001s7-Rh for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:02 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 43E4E218B8; Mon, 23 Oct 2023 20:36:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093419; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GBwPCpPCLYJ+5kl+1gYe5y916EWi0+RLe1CZozNG5iM=; b=wUBHmhdqsTNzLvr2tlJ3kGnG7JTInp2lfgk4wfvx44De6sXd/X1EPhHmhyUqrUyaWFLXsD OoEwBouNl8C4IBU/umQd+A5aoPuDhq5FpNerQuXQE5uxqjSMbjjnI6DBqilqfe+rverGhb bJhi9ftxdZW7uwYLCSS71ULDrWe7ZHI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093419; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GBwPCpPCLYJ+5kl+1gYe5y916EWi0+RLe1CZozNG5iM=; b=VnwnjOR2e25MjXyEaDWzY4lD3uPAr1vLoopW4Dfwf2MDGO31wGAjrfieBrcgDRHUU8ISQ+ p/X5CNyuIwuBaXDg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 78361132FD; Mon, 23 Oct 2023 20:36:56 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id EIEIEWjZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:36:56 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana , Nikolay Borisov , Thomas Huth , Laurent Vivier , Paolo Bonzini Subject: [PATCH v2 17/29] tests/qtest: migration-test: Add tests for fixed-ram file-based migration Date: Mon, 23 Oct 2023 17:35:56 -0300 Message-Id: <20231023203608.26370-18-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out1.suse.de; none X-Spam-Score: -1.19 X-Spamd-Result: default: False [-1.19 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[11]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-2.09)[95.56%] Received-SPF: pass client-ip=195.135.220.28; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Nikolay Borisov Add basic tests for 'fixed-ram' migration. Signed-off-by: Nikolay Borisov Signed-off-by: Fabiano Rosas --- tests/qtest/migration-test.c | 39 ++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 06a7dd3c0a..bd4e866e67 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -2030,6 +2030,40 @@ static void test_precopy_file_offset_bad(void) test_file_common(&args, false, false); } +static void *migrate_fixed_ram_start(QTestState *from, QTestState *to) +{ + migrate_set_capability(from, "fixed-ram", true); + migrate_set_capability(to, "fixed-ram", true); + + return NULL; +} + +static void test_precopy_file_fixed_ram_live(void) +{ + g_autofree char *uri = g_strdup_printf("file:%s/%s", tmpfs, + FILE_TEST_FILENAME); + MigrateCommon args = { + .connect_uri = uri, + .listen_uri = "defer", + .start_hook = migrate_fixed_ram_start, + }; + + test_file_common(&args, false, false); +} + +static void test_precopy_file_fixed_ram(void) +{ + g_autofree char *uri = g_strdup_printf("file:%s/%s", tmpfs, + FILE_TEST_FILENAME); + MigrateCommon args = { + .connect_uri = uri, + .listen_uri = "defer", + .start_hook = migrate_fixed_ram_start, + }; + + test_file_common(&args, false, true); +} + static void test_precopy_tcp_plain(void) { MigrateCommon args = { @@ -3098,6 +3132,11 @@ int main(int argc, char **argv) qtest_add_func("/migration/precopy/file/offset/bad", test_precopy_file_offset_bad); + qtest_add_func("/migration/precopy/file/fixed-ram", + test_precopy_file_fixed_ram); + qtest_add_func("/migration/precopy/file/fixed-ram/live", + test_precopy_file_fixed_ram_live); + #ifdef CONFIG_GNUTLS qtest_add_func("/migration/precopy/unix/tls/psk", test_precopy_unix_tls_psk); From patchwork Mon Oct 23 20:35:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854012 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=QJ5eHmEa; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=QsOXVVO7; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn8D3X9Dz23jl for ; Tue, 24 Oct 2023 07:38:20 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1fn-0003ch-Fp; Mon, 23 Oct 2023 16:37:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1fe-0003Tj-8l for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:06 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1fc-0001sK-13 for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:05 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 9A3361FD80; Mon, 23 Oct 2023 20:37:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093421; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+Ab7YP+vDIwkVLvT/SbH5dSZ5HGHeC0d4KTNOwymzv4=; b=QJ5eHmEaVvsREXFdaAOvx8AX8fHJTeR5bQ76u23ymHbBDhbsADvCc92kwX2/74G+PIWIvw pw1hmeaYx7IbtL1Sw1LT5cEEeIg+gFhmCJ4BDwm8fyVHO9svQ81OYvedEJQzuinqJmi1t9 9balRzUkapCBvR/A0ZjYTu8+j8P61y4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093421; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+Ab7YP+vDIwkVLvT/SbH5dSZ5HGHeC0d4KTNOwymzv4=; b=QsOXVVO7Fiag6yRwyFSoa/xwwzu/XCMalvpfWsFUzz5wFRsby6rcfJJPTrVgD1fSoq8Hjw 1krqetJQyhR0j8Bg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id B064B13A94; Mon, 23 Oct 2023 20:36:59 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id YIaqHmvZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:36:59 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana Subject: [PATCH v2 18/29] migration/multifd: Allow multifd without packets Date: Mon, 23 Oct 2023 17:35:57 -0300 Message-Id: <20231023203608.26370-19-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out2.suse.de; none X-Spam-Score: -2.10 X-Spamd-Result: default: False [-2.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[7]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] Received-SPF: pass client-ip=195.135.220.29; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org For the upcoming support to the new 'fixed-ram' migration stream format, we cannot use multifd packets because each write into the ramblock section in the migration file is expected to contain only the guest pages. They are written at their respective offsets relative to the ramblock section header. There is no space for the packet information and the expected gains from the new approach come partly from being able to write the pages sequentially without extraneous data in between. The new format also doesn't need the packets and all necessary information can be taken from the standard migration headers with some (future) changes to multifd code. Use the presence of the fixed-ram capability to decide whether to send packets. For now this has no effect as fixed-ram cannot yet be enabled with multifd. Signed-off-by: Fabiano Rosas --- migration/multifd.c | 119 +++++++++++++++++++++++++++----------------- migration/options.c | 5 ++ migration/options.h | 1 + 3 files changed, 80 insertions(+), 45 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index e2a45c667a..b912060b32 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -655,18 +655,22 @@ static void *multifd_send_thread(void *opaque) Error *local_err = NULL; int ret = 0; bool use_zero_copy_send = migrate_zero_copy_send(); + bool use_packets = migrate_multifd_packets(); thread = migration_threads_add(p->name, qemu_get_thread_id()); trace_multifd_send_thread_start(p->id); rcu_register_thread(); - if (multifd_send_initial_packet(p, &local_err) < 0) { - ret = -1; - goto out; + if (use_packets) { + if (multifd_send_initial_packet(p, &local_err) < 0) { + ret = -1; + goto out; + } + + /* initial packet */ + p->num_packets = 1; } - /* initial packet */ - p->num_packets = 1; while (true) { qemu_sem_post(&multifd_send_state->channels_ready); @@ -678,11 +682,10 @@ static void *multifd_send_thread(void *opaque) qemu_mutex_lock(&p->mutex); if (p->pending_job) { - uint64_t packet_num = p->packet_num; uint32_t flags; p->normal_num = 0; - if (use_zero_copy_send) { + if (!use_packets || use_zero_copy_send) { p->iovs_num = 0; } else { p->iovs_num = 1; @@ -700,16 +703,20 @@ static void *multifd_send_thread(void *opaque) break; } } - multifd_send_fill_packet(p); + + if (use_packets) { + multifd_send_fill_packet(p); + p->num_packets++; + } + flags = p->flags; p->flags = 0; - p->num_packets++; p->total_normal_pages += p->normal_num; p->pages->num = 0; p->pages->block = NULL; qemu_mutex_unlock(&p->mutex); - trace_multifd_send(p->id, packet_num, p->normal_num, flags, + trace_multifd_send(p->id, p->packet_num, p->normal_num, flags, p->next_packet_size); if (use_zero_copy_send) { @@ -719,7 +726,7 @@ static void *multifd_send_thread(void *opaque) if (ret != 0) { break; } - } else { + } else if (use_packets) { /* Send header using the same writev call */ p->iov[0].iov_len = p->packet_len; p->iov[0].iov_base = p->packet; @@ -907,6 +914,7 @@ int multifd_save_setup(Error **errp) { int thread_count; uint32_t page_count = MULTIFD_PACKET_SIZE / qemu_target_page_size(); + bool use_packets = migrate_multifd_packets(); uint8_t i; if (!migrate_multifd()) { @@ -931,14 +939,20 @@ int multifd_save_setup(Error **errp) p->pending_job = 0; p->id = i; p->pages = multifd_pages_init(page_count); - p->packet_len = sizeof(MultiFDPacket_t) - + sizeof(uint64_t) * page_count; - p->packet = g_malloc0(p->packet_len); - p->packet->magic = cpu_to_be32(MULTIFD_MAGIC); - p->packet->version = cpu_to_be32(MULTIFD_VERSION); + + if (use_packets) { + p->packet_len = sizeof(MultiFDPacket_t) + + sizeof(uint64_t) * page_count; + p->packet = g_malloc0(p->packet_len); + p->packet->magic = cpu_to_be32(MULTIFD_MAGIC); + p->packet->version = cpu_to_be32(MULTIFD_VERSION); + + /* We need one extra place for the packet header */ + p->iov = g_new0(struct iovec, page_count + 1); + } else { + p->iov = g_new0(struct iovec, page_count); + } p->name = g_strdup_printf("multifdsend_%d", i); - /* We need one extra place for the packet header */ - p->iov = g_new0(struct iovec, page_count + 1); p->normal = g_new0(ram_addr_t, page_count); p->page_size = qemu_target_page_size(); p->page_count = page_count; @@ -1070,7 +1084,7 @@ void multifd_recv_sync_main(void) { int i; - if (!migrate_multifd()) { + if (!migrate_multifd() || !migrate_multifd_packets()) { return; } for (i = 0; i < migrate_multifd_channels(); i++) { @@ -1097,6 +1111,7 @@ static void *multifd_recv_thread(void *opaque) { MultiFDRecvParams *p = opaque; Error *local_err = NULL; + bool use_packets = migrate_multifd_packets(); int ret; trace_multifd_recv_thread_start(p->id); @@ -1109,17 +1124,20 @@ static void *multifd_recv_thread(void *opaque) break; } - ret = qio_channel_read_all_eof(p->c, (void *)p->packet, - p->packet_len, &local_err); - if (ret == 0 || ret == -1) { /* 0: EOF -1: Error */ - break; - } + if (use_packets) { + ret = qio_channel_read_all_eof(p->c, (void *)p->packet, + p->packet_len, &local_err); + if (ret == 0 || ret == -1) { /* 0: EOF -1: Error */ + break; + } - qemu_mutex_lock(&p->mutex); - ret = multifd_recv_unfill_packet(p, &local_err); - if (ret) { - qemu_mutex_unlock(&p->mutex); - break; + qemu_mutex_lock(&p->mutex); + ret = multifd_recv_unfill_packet(p, &local_err); + if (ret) { + qemu_mutex_unlock(&p->mutex); + break; + } + p->num_packets++; } flags = p->flags; @@ -1127,7 +1145,7 @@ static void *multifd_recv_thread(void *opaque) p->flags &= ~MULTIFD_FLAG_SYNC; trace_multifd_recv(p->id, p->packet_num, p->normal_num, flags, p->next_packet_size); - p->num_packets++; + p->total_normal_pages += p->normal_num; qemu_mutex_unlock(&p->mutex); @@ -1162,6 +1180,7 @@ int multifd_load_setup(Error **errp) { int thread_count; uint32_t page_count = MULTIFD_PACKET_SIZE / qemu_target_page_size(); + bool use_packets = migrate_multifd_packets(); uint8_t i; /* @@ -1186,9 +1205,12 @@ int multifd_load_setup(Error **errp) qemu_sem_init(&p->sem_sync, 0); p->quit = false; p->id = i; - p->packet_len = sizeof(MultiFDPacket_t) - + sizeof(uint64_t) * page_count; - p->packet = g_malloc0(p->packet_len); + + if (use_packets) { + p->packet_len = sizeof(MultiFDPacket_t) + + sizeof(uint64_t) * page_count; + p->packet = g_malloc0(p->packet_len); + } p->name = g_strdup_printf("multifdrecv_%d", i); p->iov = g_new0(struct iovec, page_count); p->normal = g_new0(ram_addr_t, page_count); @@ -1234,18 +1256,26 @@ void multifd_recv_new_channel(QIOChannel *ioc, Error **errp) { MultiFDRecvParams *p; Error *local_err = NULL; - int id; + bool use_packets = migrate_multifd_packets(); + int id, num_packets = 0; - id = multifd_recv_initial_packet(ioc, &local_err); - if (id < 0) { - multifd_recv_terminate_threads(local_err); - error_propagate_prepend(errp, local_err, - "failed to receive packet" - " via multifd channel %d: ", - qatomic_read(&multifd_recv_state->count)); - return; + if (use_packets) { + id = multifd_recv_initial_packet(ioc, &local_err); + if (id < 0) { + multifd_recv_terminate_threads(local_err); + error_propagate_prepend(errp, local_err, + "failed to receive packet" + " via multifd channel %d: ", + qatomic_read(&multifd_recv_state->count)); + return; + } + trace_multifd_recv_new_channel(id); + + /* initial packet */ + num_packets = 1; + } else { + id = 0; } - trace_multifd_recv_new_channel(id); p = &multifd_recv_state->params[id]; if (p->c != NULL) { @@ -1256,9 +1286,8 @@ void multifd_recv_new_channel(QIOChannel *ioc, Error **errp) return; } p->c = ioc; + p->num_packets = num_packets; object_ref(OBJECT(ioc)); - /* initial packet */ - p->num_packets = 1; p->running = true; qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p, diff --git a/migration/options.c b/migration/options.c index 9f693d909f..bb7a2bbe06 100644 --- a/migration/options.c +++ b/migration/options.c @@ -390,6 +390,11 @@ bool migrate_multifd_flush_after_each_section(void) return s->multifd_flush_after_each_section; } +bool migrate_multifd_packets(void) +{ + return !migrate_fixed_ram(); +} + bool migrate_postcopy(void) { return migrate_postcopy_ram() || migrate_dirty_bitmaps(); diff --git a/migration/options.h b/migration/options.h index 2a9e0e9e13..4a3e7e36a8 100644 --- a/migration/options.h +++ b/migration/options.h @@ -57,6 +57,7 @@ bool migrate_zero_copy_send(void); */ bool migrate_multifd_flush_after_each_section(void); +bool migrate_multifd_packets(void); bool migrate_postcopy(void); bool migrate_rdma(void); bool migrate_tls(void); From patchwork Mon Oct 23 20:35:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854006 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=M92azYZI; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=3ZV7pGJp; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn7Q6V51z23k5 for ; Tue, 24 Oct 2023 07:37:38 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1fq-0003eT-IF; Mon, 23 Oct 2023 16:37:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1ff-0003Ui-Uz for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:08 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1fd-0001sm-8l for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:07 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id F336A218B8; Mon, 23 Oct 2023 20:37:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093424; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GXco7ODBLFbdpBvI36wDc465nJCANDAxYLcH8E4hR+E=; b=M92azYZIrGKmV++COkNvBeogGyz9wUxfrzseteh2HtQBJSkfMA6fKt1Nx1tR8jeXqqplpz AEZCO+LHNZKXJpFoIPaC5JTnZkrJpq/bgcVv//+02Wbm1NWJ3NFUGTCzVzegDQ9cZzWUCE I2v2vYPLJV57A+rq5CtOnI49AYIYajQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093424; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GXco7ODBLFbdpBvI36wDc465nJCANDAxYLcH8E4hR+E=; b=3ZV7pGJpwFf8MXHnQnbGhUDK8HmL2TM4370ljTRcoa96yW/B72yvD+HzLfvjT7rHB/uCci sfzzBfotyriErrAQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 141E4132FD; Mon, 23 Oct 2023 20:37:01 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id aH41NG3ZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:37:01 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana Subject: [PATCH v2 19/29] migration/multifd: Add outgoing QIOChannelFile support Date: Mon, 23 Oct 2023 17:35:58 -0300 Message-Id: <20231023203608.26370-20-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out1.suse.de; none X-Spam-Score: -2.10 X-Spamd-Result: default: False [-2.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[7]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] Received-SPF: pass client-ip=195.135.220.28; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Allow multifd to open file-backed channels. This will be used when enabling the fixed-ram migration stream format which expects a seekable transport. The QIOChannel read and write methods will use the preadv/pwritev versions which don't update the file offset at each call so we can reuse the fd without re-opening for every channel. Note that this is just setup code and multifd cannot yet make use of the file channels. Signed-off-by: Fabiano Rosas --- migration/file.c | 64 +++++++++++++++++++++++++++++++++++++++++-- migration/file.h | 10 +++++-- migration/migration.c | 2 +- migration/multifd.c | 14 ++++++++-- migration/options.c | 7 +++++ migration/options.h | 1 + 6 files changed, 90 insertions(+), 8 deletions(-) diff --git a/migration/file.c b/migration/file.c index cf5b1bf365..93b9b7bf5d 100644 --- a/migration/file.c +++ b/migration/file.c @@ -17,6 +17,12 @@ #define OFFSET_OPTION ",offset=" +static struct FileOutgoingArgs { + char *fname; + int flags; + int mode; +} outgoing_args; + /* Remove the offset option from @filespec and return it in @offsetp. */ static int file_parse_offset(char *filespec, uint64_t *offsetp, Error **errp) @@ -36,13 +42,62 @@ static int file_parse_offset(char *filespec, uint64_t *offsetp, Error **errp) return 0; } +static void qio_channel_file_connect_worker(QIOTask *task, gpointer opaque) +{ + /* noop */ +} + +static void file_migration_cancel(Error *errp) +{ + MigrationState *s; + + s = migrate_get_current(); + + migrate_set_state(&s->state, MIGRATION_STATUS_SETUP, + MIGRATION_STATUS_FAILED); + migration_cancel(errp); +} + +int file_send_channel_destroy(QIOChannel *ioc) +{ + if (ioc) { + qio_channel_close(ioc, NULL); + object_unref(OBJECT(ioc)); + } + g_free(outgoing_args.fname); + outgoing_args.fname = NULL; + + return 0; +} + +void file_send_channel_create(QIOTaskFunc f, void *data) +{ + QIOChannelFile *ioc; + QIOTask *task; + Error *errp = NULL; + + ioc = qio_channel_file_new_path(outgoing_args.fname, + outgoing_args.flags, + outgoing_args.mode, &errp); + if (!ioc) { + file_migration_cancel(errp); + return; + } + + task = qio_task_new(OBJECT(ioc), f, (gpointer)data, NULL); + qio_task_run_in_thread(task, qio_channel_file_connect_worker, + (gpointer)data, NULL, NULL); +} + void file_start_outgoing_migration(MigrationState *s, const char *filespec, Error **errp) { - g_autofree char *filename = g_strdup(filespec); g_autoptr(QIOChannelFile) fioc = NULL; + g_autofree char *filename = g_strdup(filespec); uint64_t offset = 0; QIOChannel *ioc; + int flags = O_CREAT | O_TRUNC | O_WRONLY; + mode_t mode = 0660; trace_migration_file_outgoing(filename); @@ -50,12 +105,15 @@ void file_start_outgoing_migration(MigrationState *s, const char *filespec, return; } - fioc = qio_channel_file_new_path(filename, O_CREAT | O_WRONLY | O_TRUNC, - 0600, errp); + fioc = qio_channel_file_new_path(filename, flags, mode, errp); if (!fioc) { return; } + outgoing_args.fname = g_strdup(filename); + outgoing_args.flags = flags; + outgoing_args.mode = mode; + ioc = QIO_CHANNEL(fioc); if (offset && qio_channel_io_seek(ioc, offset, SEEK_SET, errp) < 0) { return; diff --git a/migration/file.h b/migration/file.h index 90fa4849e0..10148233c5 100644 --- a/migration/file.h +++ b/migration/file.h @@ -7,8 +7,14 @@ #ifndef QEMU_MIGRATION_FILE_H #define QEMU_MIGRATION_FILE_H -void file_start_incoming_migration(const char *filename, Error **errp); -void file_start_outgoing_migration(MigrationState *s, const char *filename, +#include "io/task.h" +#include "channel.h" + +void file_start_incoming_migration(const char *filespec, Error **errp); + +void file_start_outgoing_migration(MigrationState *s, const char *filespec, Error **errp); +void file_send_channel_create(QIOTaskFunc f, void *data); +int file_send_channel_destroy(QIOChannel *ioc); #endif diff --git a/migration/migration.c b/migration/migration.c index cabb3ad3a5..ba806cea55 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -114,7 +114,7 @@ static bool migration_needs_seekable_channel(void) static bool uri_supports_multi_channels(const char *uri) { return strstart(uri, "tcp:", NULL) || strstart(uri, "unix:", NULL) || - strstart(uri, "vsock:", NULL); + strstart(uri, "vsock:", NULL) || strstart(uri, "file:", NULL); } static bool uri_supports_seeking(const char *uri) diff --git a/migration/multifd.c b/migration/multifd.c index b912060b32..75a17ea8ab 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -17,6 +17,7 @@ #include "exec/ramblock.h" #include "qemu/error-report.h" #include "qapi/error.h" +#include "file.h" #include "ram.h" #include "migration.h" #include "migration-stats.h" @@ -28,6 +29,7 @@ #include "threadinfo.h" #include "options.h" #include "qemu/yank.h" +#include "io/channel-file.h" #include "io/channel-socket.h" #include "yank_functions.h" @@ -512,7 +514,11 @@ static void multifd_send_terminate_threads(Error *err) static int multifd_send_channel_destroy(QIOChannel *send) { - return socket_send_channel_destroy(send); + if (migrate_to_file()) { + return file_send_channel_destroy(send); + } else { + return socket_send_channel_destroy(send); + } } void multifd_save_cleanup(void) @@ -907,7 +913,11 @@ static void multifd_new_send_channel_async(QIOTask *task, gpointer opaque) static void multifd_new_send_channel_create(gpointer opaque) { - socket_send_channel_create(multifd_new_send_channel_async, opaque); + if (migrate_to_file()) { + file_send_channel_create(multifd_new_send_channel_async, opaque); + } else { + socket_send_channel_create(multifd_new_send_channel_async, opaque); + } } int multifd_save_setup(Error **errp) diff --git a/migration/options.c b/migration/options.c index bb7a2bbe06..469d5d4c50 100644 --- a/migration/options.c +++ b/migration/options.c @@ -414,6 +414,13 @@ bool migrate_tls(void) return s->parameters.tls_creds && *s->parameters.tls_creds; } +bool migrate_to_file(void) +{ + MigrationState *s = migrate_get_current(); + + return qemu_file_is_seekable(s->to_dst_file); +} + typedef enum WriteTrackingSupport { WT_SUPPORT_UNKNOWN = 0, WT_SUPPORT_ABSENT, diff --git a/migration/options.h b/migration/options.h index 4a3e7e36a8..01bba5b928 100644 --- a/migration/options.h +++ b/migration/options.h @@ -61,6 +61,7 @@ bool migrate_multifd_packets(void); bool migrate_postcopy(void); bool migrate_rdma(void); bool migrate_tls(void); +bool migrate_to_file(void); /* capabilities helpers */ From patchwork Mon Oct 23 20:35:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854017 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=dHjUTrmr; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=OHL4cQy3; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn8r6QM5z23jl for ; Tue, 24 Oct 2023 07:38:52 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1g1-0003po-1V; Mon, 23 Oct 2023 16:37:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1fk-0003Yn-SV for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:14 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1ff-0001t1-OH for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:10 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 57F411FE33; Mon, 23 Oct 2023 20:37:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093426; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LxXCrbtkZnuGnuLEOFYoGSN8ud46u43gByhDryWps9k=; b=dHjUTrmrJDeH+d8Zf1KDQSMq7rySrZYGsUMD//vVhFny7mXNCS34PLDTRRwOUX2zd+aqM0 M/scemS/j3orEXQ6PgTzIqjX4vXKmoz89tU9YAqxod1SanSkTwal/KZmSRAgvsprIhKJqY ZC6uLNDzkD7f8ctQDC+JeTHIVDPlN1Q= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093426; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LxXCrbtkZnuGnuLEOFYoGSN8ud46u43gByhDryWps9k=; b=OHL4cQy3V4zpKgcwf50PluyOxsQeXfoZpKnccXK3R+CH1xLdU3ZSAgCmjTa7ASn9vml9/G 6EfFcNRqHIGBa5Cw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 6B785132FD; Mon, 23 Oct 2023 20:37:04 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id SCjNDXDZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:37:04 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana Subject: [PATCH v2 20/29] migration/multifd: Add incoming QIOChannelFile support Date: Mon, 23 Oct 2023 17:35:59 -0300 Message-Id: <20231023203608.26370-21-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out2.suse.de; none X-Spam-Score: -2.10 X-Spamd-Result: default: False [-2.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[7]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] Received-SPF: pass client-ip=195.135.220.29; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org On the receiving side we don't need to differentiate between main channel and threads, so whichever channel is defined first gets to be the main one. And since there are no packets, use the atomic channel count to index into the params array. Signed-off-by: Fabiano Rosas --- migration/file.c | 39 +++++++++++++++++++++++++++++---------- migration/migration.c | 2 ++ migration/multifd.c | 7 ++++++- migration/multifd.h | 1 + 4 files changed, 38 insertions(+), 11 deletions(-) diff --git a/migration/file.c b/migration/file.c index 93b9b7bf5d..ad75225f43 100644 --- a/migration/file.c +++ b/migration/file.c @@ -6,13 +6,15 @@ */ #include "qemu/osdep.h" -#include "qemu/cutils.h" #include "qapi/error.h" +#include "qemu/cutils.h" +#include "qemu/error-report.h" #include "channel.h" #include "file.h" #include "migration.h" #include "io/channel-file.h" #include "io/channel-util.h" +#include "options.h" #include "trace.h" #define OFFSET_OPTION ",offset=" @@ -136,7 +138,8 @@ void file_start_incoming_migration(const char *filespec, Error **errp) g_autofree char *filename = g_strdup(filespec); QIOChannelFile *fioc = NULL; uint64_t offset = 0; - QIOChannel *ioc; + int channels = 1; + int i = 0, fd; trace_migration_file_incoming(filename); @@ -146,16 +149,32 @@ void file_start_incoming_migration(const char *filespec, Error **errp) fioc = qio_channel_file_new_path(filename, O_RDONLY, 0, errp); if (!fioc) { - return; + goto out; + } + + if (migrate_multifd()) { + channels += migrate_multifd_channels(); } - ioc = QIO_CHANNEL(fioc); - if (offset && qio_channel_io_seek(ioc, offset, SEEK_SET, errp) < 0) { + fd = fioc->fd; + + do { + QIOChannel *ioc = QIO_CHANNEL(fioc); + + if (offset && qio_channel_io_seek(ioc, offset, SEEK_SET, errp) < 0) { + return; + } + + qio_channel_set_name(ioc, "migration-file-incoming"); + qio_channel_add_watch_full(ioc, G_IO_IN, + file_accept_incoming_migration, + NULL, NULL, + g_main_context_get_thread_default()); + } while (++i < channels && (fioc = qio_channel_file_new_fd(fd))); + +out: + if (!fioc) { + error_report("Error creating migration incoming channel"); return; } - qio_channel_set_name(QIO_CHANNEL(ioc), "migration-file-incoming"); - qio_channel_add_watch_full(ioc, G_IO_IN, - file_accept_incoming_migration, - NULL, NULL, - g_main_context_get_thread_default()); } diff --git a/migration/migration.c b/migration/migration.c index ba806cea55..5fa726f6d4 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -756,6 +756,8 @@ void migration_ioc_process_incoming(QIOChannel *ioc, Error **errp) } default_channel = (channel_magic == cpu_to_be32(QEMU_VM_FILE_MAGIC)); + } else if (migrate_multifd() && migrate_fixed_ram()) { + default_channel = multifd_recv_first_channel(); } else { default_channel = !mis->from_src_file; } diff --git a/migration/multifd.c b/migration/multifd.c index 75a17ea8ab..ad51210f13 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -1242,6 +1242,11 @@ int multifd_load_setup(Error **errp) return 0; } +bool multifd_recv_first_channel(void) +{ + return !multifd_recv_state; +} + bool multifd_recv_all_channels_created(void) { int thread_count = migrate_multifd_channels(); @@ -1284,7 +1289,7 @@ void multifd_recv_new_channel(QIOChannel *ioc, Error **errp) /* initial packet */ num_packets = 1; } else { - id = 0; + id = qatomic_read(&multifd_recv_state->count); } p = &multifd_recv_state->params[id]; diff --git a/migration/multifd.h b/migration/multifd.h index a835643b48..a112ec7ac6 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -18,6 +18,7 @@ void multifd_save_cleanup(void); int multifd_load_setup(Error **errp); void multifd_load_cleanup(void); void multifd_load_shutdown(void); +bool multifd_recv_first_channel(void); bool multifd_recv_all_channels_created(void); void multifd_recv_new_channel(QIOChannel *ioc, Error **errp); void multifd_recv_sync_main(void); From patchwork Mon Oct 23 20:36:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854016 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=GU+qB/vm; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=csYAX9Zw; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn8P2VvZz23k5 for ; Tue, 24 Oct 2023 07:38:29 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1g2-00046m-Id; Mon, 23 Oct 2023 16:37:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1fq-0003jl-SG for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:20 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1fk-0001tM-Jn for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:18 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id C2B741FE35; Mon, 23 Oct 2023 20:37:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093428; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rGDXXkd9mpTgbPLkflQJtoKien+TPU2p5r3bG/wfUw4=; b=GU+qB/vmoWKYvn5lnjxBrDq5lWme+Dfl3iavBlbOSgfzyfx4W5J2kbuXFyg1gbgSdNJq39 NHpKprGUVMkjM8G4vvadK3vYjbW2XhizyQRXas66wY7qbXhmlaAoAy1oyUz/wm7D0XK4tu TFLWQLaCt43NNJLLpDzs3ol3H9gGrDA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093428; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rGDXXkd9mpTgbPLkflQJtoKien+TPU2p5r3bG/wfUw4=; b=csYAX9ZwppK2sQza+S4VQQp9z/GGuct/YPbhbKsMWnbZ9uVyxA8O6sdKIo9qZykUsmP3A7 /4YquCwuklx8k8AQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id C0642132FD; Mon, 23 Oct 2023 20:37:06 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id GGBHInLZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:37:06 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana Subject: [PATCH v2 21/29] migration/multifd: Add pages to the receiving side Date: Mon, 23 Oct 2023 17:36:00 -0300 Message-Id: <20231023203608.26370-22-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out2.suse.de; none X-Spam-Score: -2.10 X-Spamd-Result: default: False [-2.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[7]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] Received-SPF: pass client-ip=195.135.220.29; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Currently multifd does not need to have knowledge of pages on the receiving side because all the information needed is within the packets that come in the stream. We're about to add support to fixed-ram migration, which cannot use packets because it expects the ramblock section in the migration file to contain only the guest pages data. Add a pointer to MultiFDPages in the multifd_recv_state and use the pages similarly to what we already do on the sending side. The pages are used to transfer data between the ram migration code in the main migration thread and the multifd receiving threads. Signed-off-by: Fabiano Rosas --- migration/multifd.c | 107 ++++++++++++++++++++++++++++++++++++++++++++ migration/multifd.h | 13 +++++- 2 files changed, 119 insertions(+), 1 deletion(-) diff --git a/migration/multifd.c b/migration/multifd.c index ad51210f13..20e8635740 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -992,6 +992,8 @@ int multifd_save_setup(Error **errp) struct { MultiFDRecvParams *params; + /* array of pages to receive */ + MultiFDPages_t *pages; /* number of created threads */ int count; /* syncs main thread and channels */ @@ -1002,6 +1004,75 @@ struct { MultiFDMethods *ops; } *multifd_recv_state; +static int multifd_recv_pages(QEMUFile *f) +{ + int i; + static int next_recv_channel; + MultiFDRecvParams *p = NULL; + MultiFDPages_t *pages = multifd_recv_state->pages; + + /* + * next_channel can remain from a previous migration that was + * using more channels, so ensure it doesn't overflow if the + * limit is lower now. + */ + next_recv_channel %= migrate_multifd_channels(); + for (i = next_recv_channel;; i = (i + 1) % migrate_multifd_channels()) { + p = &multifd_recv_state->params[i]; + + qemu_mutex_lock(&p->mutex); + if (p->quit) { + error_report("%s: channel %d has already quit!", __func__, i); + qemu_mutex_unlock(&p->mutex); + return -1; + } + if (!p->pending_job) { + p->pending_job++; + next_recv_channel = (i + 1) % migrate_multifd_channels(); + break; + } + qemu_mutex_unlock(&p->mutex); + } + + multifd_recv_state->pages = p->pages; + p->pages = pages; + qemu_mutex_unlock(&p->mutex); + qemu_sem_post(&p->sem); + + return 1; +} + +int multifd_recv_queue_page(QEMUFile *f, RAMBlock *block, ram_addr_t offset) +{ + MultiFDPages_t *pages = multifd_recv_state->pages; + bool changed = false; + + if (!pages->block) { + pages->block = block; + } + + if (pages->block == block) { + pages->offset[pages->num] = offset; + pages->num++; + + if (pages->num < pages->allocated) { + return 1; + } + } else { + changed = true; + } + + if (multifd_recv_pages(f) < 0) { + return -1; + } + + if (changed) { + return multifd_recv_queue_page(f, block, offset); + } + + return 1; +} + static void multifd_recv_terminate_threads(Error *err) { int i; @@ -1023,6 +1094,7 @@ static void multifd_recv_terminate_threads(Error *err) qemu_mutex_lock(&p->mutex); p->quit = true; + qemu_sem_post(&p->sem); /* * We could arrive here for two reasons: * - normal quit, i.e. everything went fine, just finished @@ -1072,8 +1144,11 @@ void multifd_load_cleanup(void) p->c = NULL; qemu_mutex_destroy(&p->mutex); qemu_sem_destroy(&p->sem_sync); + qemu_sem_destroy(&p->sem); g_free(p->name); p->name = NULL; + multifd_pages_clear(p->pages); + p->pages = NULL; p->packet_len = 0; g_free(p->packet); p->packet = NULL; @@ -1086,6 +1161,8 @@ void multifd_load_cleanup(void) qemu_sem_destroy(&multifd_recv_state->sem_sync); g_free(multifd_recv_state->params); multifd_recv_state->params = NULL; + multifd_pages_clear(multifd_recv_state->pages); + multifd_recv_state->pages = NULL; g_free(multifd_recv_state); multifd_recv_state = NULL; } @@ -1148,6 +1225,25 @@ static void *multifd_recv_thread(void *opaque) break; } p->num_packets++; + } else { + /* + * No packets, so we need to wait for the vmstate code to + * queue pages. + */ + qemu_sem_wait(&p->sem); + qemu_mutex_lock(&p->mutex); + if (!p->pending_job) { + qemu_mutex_unlock(&p->mutex); + break; + } + + for (int i = 0; i < p->pages->num; i++) { + p->normal[p->normal_num] = p->pages->offset[i]; + p->normal_num++; + } + + p->pages->num = 0; + p->host = p->pages->block->host; } flags = p->flags; @@ -1170,6 +1266,13 @@ static void *multifd_recv_thread(void *opaque) qemu_sem_post(&multifd_recv_state->sem_sync); qemu_sem_wait(&p->sem_sync); } + + if (!use_packets) { + qemu_mutex_lock(&p->mutex); + p->pending_job--; + p->pages->block = NULL; + qemu_mutex_unlock(&p->mutex); + } } if (local_err) { @@ -1204,6 +1307,7 @@ int multifd_load_setup(Error **errp) thread_count = migrate_multifd_channels(); multifd_recv_state = g_malloc0(sizeof(*multifd_recv_state)); multifd_recv_state->params = g_new0(MultiFDRecvParams, thread_count); + multifd_recv_state->pages = multifd_pages_init(page_count); qatomic_set(&multifd_recv_state->count, 0); qemu_sem_init(&multifd_recv_state->sem_sync, 0); multifd_recv_state->ops = multifd_ops[migrate_multifd_compression()]; @@ -1213,8 +1317,11 @@ int multifd_load_setup(Error **errp) qemu_mutex_init(&p->mutex); qemu_sem_init(&p->sem_sync, 0); + qemu_sem_init(&p->sem, 0); p->quit = false; + p->pending_job = 0; p->id = i; + p->pages = multifd_pages_init(page_count); if (use_packets) { p->packet_len = sizeof(MultiFDPacket_t) diff --git a/migration/multifd.h b/migration/multifd.h index a112ec7ac6..b571b1e4a2 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -24,6 +24,7 @@ void multifd_recv_new_channel(QIOChannel *ioc, Error **errp); void multifd_recv_sync_main(void); int multifd_send_sync_main(QEMUFile *f); int multifd_queue_page(QEMUFile *f, RAMBlock *block, ram_addr_t offset); +int multifd_recv_queue_page(QEMUFile *f, RAMBlock *block, ram_addr_t offset); /* Multifd Compression flags */ #define MULTIFD_FLAG_SYNC (1 << 0) @@ -153,9 +154,13 @@ typedef struct { uint32_t page_size; /* number of pages in a full packet */ uint32_t page_count; + /* multifd flags for receiving ram */ + int read_flags; /* syncs main thread and channels */ QemuSemaphore sem_sync; + /* sem where to wait for more work */ + QemuSemaphore sem; /* this mutex protects the following parameters */ QemuMutex mutex; @@ -167,6 +172,13 @@ typedef struct { uint32_t flags; /* global number of generated multifd packets */ uint64_t packet_num; + int pending_job; + /* array of pages to sent. + * The owner of 'pages' depends of 'pending_job' value: + * pending_job == 0 -> migration_thread can use it. + * pending_job != 0 -> multifd_channel can use it. + */ + MultiFDPages_t *pages; /* thread local variables. No locking required */ @@ -210,4 +222,3 @@ typedef struct { void multifd_register_ops(int method, MultiFDMethods *ops); #endif - From patchwork Mon Oct 23 20:36:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854023 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=y8VvrYl3; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=0yBW5+qH; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn9J15fnz23jl for ; Tue, 24 Oct 2023 07:39:16 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1g1-00040V-UD; Mon, 23 Oct 2023 16:37:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1fq-0003iS-OG for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:20 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1fk-0001tS-K8 for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:16 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 16A3621B25; Mon, 23 Oct 2023 20:37:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093431; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=91V6Vpi0EQg1XNaw6w4iigg8MpuDSYcENbFQ9e3PdNQ=; b=y8VvrYl34JVgIqeccYF0hf/fRSMxhoHtklftnfqbEG9vooDdamixRYGMU2RZ4H4BLw5b3i 7VZFvTvCSn0mL2HLsQcoM+1o3Zjp0eo1luiLvX4o4S8eyvyJtMbfsgnhXnfKyqr9cQKWZy j3Ns6IQtzbtXdRiVbHE37mMRa6FM5g8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093431; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=91V6Vpi0EQg1XNaw6w4iigg8MpuDSYcENbFQ9e3PdNQ=; b=0yBW5+qHEkO5VbeBNY8DQABmvDNeKFfYUKthL+Nz6/2pEKTlGlbtW/JPFMIFPU/6boUN4k sS4+OOumnmS+PGBw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 2B830132FD; Mon, 23 Oct 2023 20:37:08 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id KKTHOXTZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:37:08 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana Subject: [PATCH v2 22/29] io: Add a pwritev/preadv version that takes a discontiguous iovec Date: Mon, 23 Oct 2023 17:36:01 -0300 Message-Id: <20231023203608.26370-23-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out1.suse.de; none X-Spam-Score: -2.10 X-Spamd-Result: default: False [-2.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[7]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] Received-SPF: pass client-ip=195.135.220.28; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org For the upcoming support to fixed-ram migration with multifd, we need to be able to accept an iovec array with non-contiguous data. Add a pwritev and preadv version that splits the array into contiguous segments before writing. With that we can have the ram code continue to add pages in any order and the multifd code continue to send large arrays for reading and writing. Signed-off-by: Fabiano Rosas --- Since iovs can be non contiguous, we'd need a separate array on the side to carry an extra file offset for each of them, so I'm relying on the fact that iovs are all within a same host page and passing in an encoded offset that takes the host page into account. --- include/io/channel.h | 50 +++++++++++++++++++++++++++ io/channel.c | 82 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 132 insertions(+) diff --git a/include/io/channel.h b/include/io/channel.h index a8181d576a..51a99fb9f6 100644 --- a/include/io/channel.h +++ b/include/io/channel.h @@ -33,8 +33,10 @@ OBJECT_DECLARE_TYPE(QIOChannel, QIOChannelClass, #define QIO_CHANNEL_ERR_BLOCK -2 #define QIO_CHANNEL_WRITE_FLAG_ZERO_COPY 0x1 +#define QIO_CHANNEL_WRITE_FLAG_WITH_OFFSET 0x2 #define QIO_CHANNEL_READ_FLAG_MSG_PEEK 0x1 +#define QIO_CHANNEL_READ_FLAG_WITH_OFFSET 0x2 typedef enum QIOChannelFeature QIOChannelFeature; @@ -559,6 +561,30 @@ int qio_channel_close(QIOChannel *ioc, ssize_t qio_channel_pwritev_full(QIOChannel *ioc, const struct iovec *iov, size_t niov, off_t offset, Error **errp); +/** + * qio_channel_write_full_all: + * @ioc: the channel object + * @iov: the array of memory regions to write data from + * @niov: the length of the @iov array + * @offset: the iovec offset in the file where to write the data + * @fds: an array of file handles to send + * @nfds: number of file handles in @fds + * @flags: write flags (QIO_CHANNEL_WRITE_FLAG_*) + * @errp: pointer to a NULL-initialized error object + * + * + * Selects between a writev or pwritev channel writer function. + * + * If QIO_CHANNEL_WRITE_FLAG_OFFSET is passed in flags, pwritev is + * used and @offset is expected to be a meaningful value, @fds and + * @nfds are ignored; otherwise uses writev and @offset is ignored. + * + * Returns: 0 if all bytes were written, or -1 on error + */ +int qio_channel_write_full_all(QIOChannel *ioc, const struct iovec *iov, + size_t niov, off_t offset, int *fds, size_t nfds, + int flags, Error **errp); + /** * qio_channel_pwritev * @ioc: the channel object @@ -595,6 +621,30 @@ ssize_t qio_channel_pwritev(QIOChannel *ioc, char *buf, size_t buflen, ssize_t qio_channel_preadv_full(QIOChannel *ioc, const struct iovec *iov, size_t niov, off_t offset, Error **errp); +/** + * qio_channel_read_full_all: + * @ioc: the channel object + * @iov: the array of memory regions to read data to + * @niov: the length of the @iov array + * @offset: the iovec offset in the file from where to read the data + * @fds: an array of file handles to send + * @nfds: number of file handles in @fds + * @flags: read flags (QIO_CHANNEL_READ_FLAG_*) + * @errp: pointer to a NULL-initialized error object + * + * + * Selects between a readv or preadv channel reader function. + * + * If QIO_CHANNEL_READ_FLAG_OFFSET is passed in flags, preadv is + * used and @offset is expected to be a meaningful value, @fds and + * @nfds are ignored; otherwise uses readv and @offset is ignored. + * + * Returns: 0 if all bytes were read, or -1 on error + */ +int qio_channel_read_full_all(QIOChannel *ioc, const struct iovec *iov, + size_t niov, off_t offset, + int flags, Error **errp); + /** * qio_channel_preadv * @ioc: the channel object diff --git a/io/channel.c b/io/channel.c index 770d61ea00..648b68451d 100644 --- a/io/channel.c +++ b/io/channel.c @@ -472,6 +472,76 @@ ssize_t qio_channel_pwritev_full(QIOChannel *ioc, const struct iovec *iov, return klass->io_pwritev(ioc, iov, niov, offset, errp); } +static int qio_channel_preadv_pwritev_contiguous(QIOChannel *ioc, + const struct iovec *iov, + size_t niov, off_t offset, + bool is_write, Error **errp) +{ + ssize_t ret; + int i, slice_idx, slice_num; + uint64_t base, next, file_offset; + size_t len; + + slice_idx = 0; + slice_num = 1; + + /* + * If the iov array doesn't have contiguous elements, we need to + * split it in slices because we only have one (file) 'offset' for + * the whole iov. Do this here so callers don't need to break the + * iov array themselves. + */ + for (i = 0; i < niov; i++, slice_num++) { + base = (uint64_t) iov[i].iov_base; + + if (i != niov - 1) { + len = iov[i].iov_len; + next = (uint64_t) iov[i + 1].iov_base; + + if (base + len == next) { + continue; + } + } + + /* + * Use the offset of the first element of the segment that + * we're sending. + */ + file_offset = offset + (uint64_t) iov[slice_idx].iov_base; + + if (is_write) { + ret = qio_channel_pwritev_full(ioc, &iov[slice_idx], slice_num, + file_offset, errp); + } else { + ret = qio_channel_preadv_full(ioc, &iov[slice_idx], slice_num, + file_offset, errp); + } + + if (ret < 0) { + break; + } + + slice_idx += slice_num; + slice_num = 0; + } + + return (ret < 0) ? -1 : 0; +} + +int qio_channel_write_full_all(QIOChannel *ioc, + const struct iovec *iov, + size_t niov, off_t offset, + int *fds, size_t nfds, + int flags, Error **errp) +{ + if (flags & QIO_CHANNEL_WRITE_FLAG_WITH_OFFSET) { + return qio_channel_preadv_pwritev_contiguous(ioc, iov, niov, + offset, true, errp); + } + + return qio_channel_writev_full_all(ioc, iov, niov, NULL, 0, flags, errp); +} + ssize_t qio_channel_pwritev(QIOChannel *ioc, char *buf, size_t buflen, off_t offset, Error **errp) { @@ -501,6 +571,18 @@ ssize_t qio_channel_preadv_full(QIOChannel *ioc, const struct iovec *iov, return klass->io_preadv(ioc, iov, niov, offset, errp); } +int qio_channel_read_full_all(QIOChannel *ioc, const struct iovec *iov, + size_t niov, off_t offset, + int flags, Error **errp) +{ + if (flags & QIO_CHANNEL_READ_FLAG_WITH_OFFSET) { + return qio_channel_preadv_pwritev_contiguous(ioc, iov, niov, + offset, false, errp); + } + + return qio_channel_readv_full_all(ioc, iov, niov, NULL, NULL, errp); +} + ssize_t qio_channel_preadv(QIOChannel *ioc, char *buf, size_t buflen, off_t offset, Error **errp) { From patchwork Mon Oct 23 20:36:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854029 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=LJTWQq5F; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=dUQGJL3E; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDnBV1FFsz23jn for ; Tue, 24 Oct 2023 07:40:18 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1g3-0004H0-S4; Mon, 23 Oct 2023 16:37:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1fq-0003jL-R8 for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:20 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1fm-0001ti-Jh for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:18 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 692C71FE37; Mon, 23 Oct 2023 20:37:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093433; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fioivwaIQ4HIvSYijeTJFczB0DjmhMjP2qC1xqQx5q8=; b=LJTWQq5FBOdS7sXlx2j7/YzrQ9taZWUVehZ9BzC1Ll6gHtpM1ukBz4pejYpo24B+reiccP v2QQQF/+rchuspAYPfjF9dTWC/uZKQZDSeA470K12zXS9xcJhzUE6SQxaDGplGA9OAjTkr qWqrDxtrnLwrnOjYD5RMfqHwOspnKts= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093433; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fioivwaIQ4HIvSYijeTJFczB0DjmhMjP2qC1xqQx5q8=; b=dUQGJL3EX4xHXUS1Tf018mBXjG92DxNRe0iHwuddztJYv1HCVd1uRXKEcJVmbmcO6Bfmcc EyBpQK/1vuQzBWBQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 7E1AC132FD; Mon, 23 Oct 2023 20:37:11 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id +G9EEnfZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:37:11 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana Subject: [PATCH v2 23/29] migration/ram: Add a wrapper for fixed-ram shadow bitmap Date: Mon, 23 Oct 2023 17:36:02 -0300 Message-Id: <20231023203608.26370-24-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out2.suse.de; none X-Spam-Score: -1.90 X-Spamd-Result: default: False [-1.90 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[7]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-2.80)[99.16%] Received-SPF: pass client-ip=195.135.220.29; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org We'll need to set the shadow_bmap bits from outside ram.c soon and TARGET_PAGE_BITS is poisoned, so add a wrapper to it. Signed-off-by: Fabiano Rosas --- migration/ram.c | 5 +++++ migration/ram.h | 1 + 2 files changed, 6 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index cea6971ab2..8e34c1b597 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3160,6 +3160,11 @@ static void ram_save_shadow_bmap(QEMUFile *f) } } +void ramblock_set_shadow_bmap_atomic(RAMBlock *block, ram_addr_t offset) +{ + set_bit_atomic(offset >> TARGET_PAGE_BITS, block->shadow_bmap); +} + /** * ram_save_iterate: iterative stage for migration * diff --git a/migration/ram.h b/migration/ram.h index 145c915ca7..1acadffb06 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -75,6 +75,7 @@ int ram_dirty_bitmap_reload(MigrationState *s, RAMBlock *rb); bool ramblock_page_is_discarded(RAMBlock *rb, ram_addr_t start); void postcopy_preempt_shutdown_file(MigrationState *s); void *postcopy_preempt_thread(void *opaque); +void ramblock_set_shadow_bmap_atomic(RAMBlock *block, ram_addr_t offset); /* ram cache */ int colo_init_ram_cache(void); From patchwork Mon Oct 23 20:36:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854021 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=NmuacHj9; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=WAfaqhxf; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn9G4FB3z23jl for ; Tue, 24 Oct 2023 07:39:14 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1g2-00040g-3R; Mon, 23 Oct 2023 16:37:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1fs-0003mF-TF for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:22 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1fq-0001tu-Hy for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:19 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id BA65B218B8; Mon, 23 Oct 2023 20:37:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093435; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=U8REzouWVMUSyxt0F5ihPhQ2s5ZaLga2GdJROF81qn0=; b=NmuacHj91YLceXLktn8Gog37dnISdr2RZ7L4YjdhgWS3TJj7Hz0Pp17ZA56qeeANdfdaKz RrsVXAq7/UH5Ku4e1Nw+yWY5fP705KR1gylEMZXO26oKdkM86uBbIdjh6K0iotiCzEC/yL SpT7Jb0dRM7GAQNt+UbhNTzCBHBlnqM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093435; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=U8REzouWVMUSyxt0F5ihPhQ2s5ZaLga2GdJROF81qn0=; b=WAfaqhxflCfBR81DChLVw2lEnsNIK9DdouVhNk1oAS/UVSLGFuwm7saxDtF7ZwzObOlAYl Xm71EJmkhUONo/AA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id CDDED132FD; Mon, 23 Oct 2023 20:37:13 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id cJHYJXnZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:37:13 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana Subject: [PATCH v2 24/29] migration/ram: Ignore multifd flush when doing fixed-ram migration Date: Mon, 23 Oct 2023 17:36:03 -0300 Message-Id: <20231023203608.26370-25-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out1.suse.de; none X-Spam-Score: -1.75 X-Spamd-Result: default: False [-1.75 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[7]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-2.65)[98.44%] Received-SPF: pass client-ip=195.135.220.28; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Some functionalities of multifd are incompatible with the 'fixed-ram' migration format. The MULTIFD_FLUSH flag in particular is not used because in fixed-ram there is no sinchronicity between migration source and destination so there is not need for a sync packet. In fact, fixed-ram disables packets in multifd as a whole. Make sure RAM_SAVE_FLAG_MULTIFD_FLUSH is never emitted when fixed-ram is enabled. Signed-off-by: Fabiano Rosas --- migration/ram.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index 8e34c1b597..3497ed186a 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1386,7 +1386,7 @@ static int find_dirty_block(RAMState *rs, PageSearchStatus *pss) pss->page = 0; pss->block = QLIST_NEXT_RCU(pss->block, next); if (!pss->block) { - if (migrate_multifd() && + if (!migrate_fixed_ram() && migrate_multifd() && !migrate_multifd_flush_after_each_section()) { QEMUFile *f = rs->pss[RAM_CHANNEL_PRECOPY].pss_channel; int ret = multifd_send_sync_main(f); From patchwork Mon Oct 23 20:36:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854011 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=d7geVfel; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=VqmQUIMc; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn886jWxz23jl for ; Tue, 24 Oct 2023 07:38:16 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1g4-0004Lq-ON; Mon, 23 Oct 2023 16:37:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1fu-0003nq-TB for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:24 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1fr-0001u4-AX for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:21 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 2ACEE21B28; Mon, 23 Oct 2023 20:37:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093438; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=B8nEthtwB2qEKS5Hv45AtaqNIBTuRIA7SDrRGv3fpRE=; b=d7geVfelSn1pQUZ6lzI6MQAQS/HXhbdK7mo3tRk0vq/LEXOn1bwMkBUrWslSoDsAacxveY +BqZkKttKjZwTFDecxYBNrn2HlMNSoChn96np6UwLbFR1ZbSRziEZI2E8jv8zBdYLvxGgp 4TZZpH8HIdsZmMAXnP7DQOnFec39/pg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093438; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=B8nEthtwB2qEKS5Hv45AtaqNIBTuRIA7SDrRGv3fpRE=; b=VqmQUIMcBxYxaTnK+zQQczkmNuhpi7jDy0goqZ3PwJeM2+JnPsfNDzggCJ9/nZE9iZXeWG mJYNznKROzwh09CQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 355A4132FD; Mon, 23 Oct 2023 20:37:16 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id yI8vAHzZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:37:16 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana Subject: [PATCH v2 25/29] migration/multifd: Support outgoing fixed-ram stream format Date: Mon, 23 Oct 2023 17:36:04 -0300 Message-Id: <20231023203608.26370-26-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out1.suse.de; none X-Spam-Score: -2.10 X-Spamd-Result: default: False [-2.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[7]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] Received-SPF: pass client-ip=195.135.220.28; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The new fixed-ram stream format uses a file transport and puts ram pages in the migration file at their respective offsets and can be done in parallel by using the pwritev system call which takes iovecs and an offset. Add support to enabling the new format along with multifd to make use of the threading and page handling already in place. This requires multifd to stop sending headers and leaving the stream format to the fixed-ram code. When it comes time to write the data, we need to call a version of qio_channel_write that can take an offset. Usage on HMP is: (qemu) stop (qemu) migrate_set_capability multifd on (qemu) migrate_set_capability fixed-ram on (qemu) migrate_set_parameter max-bandwidth 0 (qemu) migrate_set_parameter multifd-channels 8 (qemu) migrate file:migfile Signed-off-by: Fabiano Rosas --- include/qemu/bitops.h | 13 ++++++++++ migration/multifd.c | 55 +++++++++++++++++++++++++++++++++++++++++-- migration/options.c | 6 ----- migration/ram.c | 2 +- 4 files changed, 67 insertions(+), 9 deletions(-) diff --git a/include/qemu/bitops.h b/include/qemu/bitops.h index cb3526d1f4..2c0a2fe751 100644 --- a/include/qemu/bitops.h +++ b/include/qemu/bitops.h @@ -67,6 +67,19 @@ static inline void clear_bit(long nr, unsigned long *addr) *p &= ~mask; } +/** + * clear_bit_atomic - Clears a bit in memory atomically + * @nr: Bit to clear + * @addr: Address to start counting from + */ +static inline void clear_bit_atomic(long nr, unsigned long *addr) +{ + unsigned long mask = BIT_MASK(nr); + unsigned long *p = addr + BIT_WORD(nr); + + return qatomic_and(p, ~mask); +} + /** * change_bit - Toggle a bit in memory * @nr: Bit to change diff --git a/migration/multifd.c b/migration/multifd.c index 20e8635740..3f95a41ee9 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -260,6 +260,19 @@ static void multifd_pages_clear(MultiFDPages_t *pages) g_free(pages); } +static void multifd_set_file_bitmap(MultiFDSendParams *p) +{ + MultiFDPages_t *pages = p->pages; + + if (!pages->block) { + return; + } + + for (int i = 0; i < p->normal_num; i++) { + ramblock_set_shadow_bmap_atomic(pages->block, pages->offset[i]); + } +} + static void multifd_send_fill_packet(MultiFDSendParams *p) { MultiFDPacket_t *packet = p->packet; @@ -606,6 +619,29 @@ int multifd_send_sync_main(QEMUFile *f) } } + if (!migrate_multifd_packets()) { + /* + * There's no sync packet to send. Just make sure the sending + * above has finished. + */ + for (i = 0; i < migrate_multifd_channels(); i++) { + qemu_sem_wait(&multifd_send_state->channels_ready); + } + + /* sanity check and release the channels */ + for (i = 0; i < migrate_multifd_channels(); i++) { + MultiFDSendParams *p = &multifd_send_state->params[i]; + + qemu_mutex_lock(&p->mutex); + assert(!p->pending_job || p->quit); + qemu_mutex_unlock(&p->mutex); + + qemu_sem_post(&p->sem); + } + + return 0; + } + /* * When using zero-copy, it's necessary to flush the pages before any of * the pages can be sent again, so we'll make sure the new version of the @@ -689,6 +725,8 @@ static void *multifd_send_thread(void *opaque) if (p->pending_job) { uint32_t flags; + uint64_t write_base; + p->normal_num = 0; if (!use_packets || use_zero_copy_send) { @@ -713,6 +751,16 @@ static void *multifd_send_thread(void *opaque) if (use_packets) { multifd_send_fill_packet(p); p->num_packets++; + write_base = 0; + } else { + multifd_set_file_bitmap(p); + + /* + * If we subtract the host page now, we don't need to + * pass it into qio_channel_write_full_all() below. + */ + write_base = p->pages->block->pages_offset - + (uint64_t)p->pages->block->host; } flags = p->flags; @@ -738,8 +786,9 @@ static void *multifd_send_thread(void *opaque) p->iov[0].iov_base = p->packet; } - ret = qio_channel_writev_full_all(p->c, p->iov, p->iovs_num, NULL, - 0, p->write_flags, &local_err); + ret = qio_channel_write_full_all(p->c, p->iov, p->iovs_num, + write_base, NULL, 0, + p->write_flags, &local_err); if (ret != 0) { break; } @@ -969,6 +1018,8 @@ int multifd_save_setup(Error **errp) if (migrate_zero_copy_send()) { p->write_flags = QIO_CHANNEL_WRITE_FLAG_ZERO_COPY; + } else if (!use_packets) { + p->write_flags |= QIO_CHANNEL_WRITE_FLAG_WITH_OFFSET; } else { p->write_flags = 0; } diff --git a/migration/options.c b/migration/options.c index 469d5d4c50..2193d69e71 100644 --- a/migration/options.c +++ b/migration/options.c @@ -648,12 +648,6 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) } if (new_caps[MIGRATION_CAPABILITY_FIXED_RAM]) { - if (new_caps[MIGRATION_CAPABILITY_MULTIFD]) { - error_setg(errp, - "Fixed-ram migration is incompatible with multifd"); - return false; - } - if (new_caps[MIGRATION_CAPABILITY_XBZRLE]) { error_setg(errp, "Fixed-ram migration is incompatible with xbzrle"); diff --git a/migration/ram.c b/migration/ram.c index 3497ed186a..5c67e30e55 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1161,7 +1161,7 @@ static int save_zero_page(RAMState *rs, PageSearchStatus *pss, if (migrate_fixed_ram()) { /* zero pages are not transferred with fixed-ram */ - clear_bit(offset >> TARGET_PAGE_BITS, pss->block->shadow_bmap); + clear_bit_atomic(offset >> TARGET_PAGE_BITS, pss->block->shadow_bmap); return 1; } From patchwork Mon Oct 23 20:36:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854019 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=2GL0D2nf; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=6NlwyTIE; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn8y4r60z23k5 for ; Tue, 24 Oct 2023 07:38:58 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1gL-0004ie-9E; Mon, 23 Oct 2023 16:37:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1fx-0003pU-4j for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:27 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1fu-0001uS-AX for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:23 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 8CF5121B0E; Mon, 23 Oct 2023 20:37:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093440; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TvFX52MvFl8uE+na5ThDGfRCvVD8CmpSVUukn+S2dHg=; b=2GL0D2nfhhR4UjIKmI/FMj7I/2aRQYP1miSwkJb7wpet3nnBc7r1CgH3kpdwZ+Vqi+9Hk6 fGgniuHYt3mEnUx06I/+BrBg5A8Fuu3cwQyhagl08bQPNbOxfOTXAK1z7IVIP8/qUBWyla FTeriPHK8kNWxtrgxRn+A/L3qe475+E= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093440; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TvFX52MvFl8uE+na5ThDGfRCvVD8CmpSVUukn+S2dHg=; b=6NlwyTIEvTTPfSiLg4sivI7+uAhCQaGMV6EQbU5QRC1eRYRzoTllFfRzEjcMs2TycQO79V MFkqXOFIJJ01vrCg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 97FE4132FD; Mon, 23 Oct 2023 20:37:18 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id uMy4F37ZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:37:18 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana Subject: [PATCH v2 26/29] migration/multifd: Support incoming fixed-ram stream format Date: Mon, 23 Oct 2023 17:36:05 -0300 Message-Id: <20231023203608.26370-27-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out1.suse.de; none X-Spam-Score: -2.10 X-Spamd-Result: default: False [-2.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[7]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] Received-SPF: pass client-ip=195.135.220.28; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org For the incoming fixed-ram migration we need to read the ramblock headers, get the pages bitmap and send the host address of each non-zero page to the multifd channel thread for writing. To read from the migration file we need a preadv function that can read into the iovs in segments of contiguous pages because (as in the writing case) the file offset applies to the entire iovec. Usage on HMP is: (qemu) migrate_set_capability multifd on (qemu) migrate_set_capability fixed-ram on (qemu) migrate_set_parameter max-bandwidth 0 (qemu) migrate_set_parameter multifd-channels 8 (qemu) migrate_incoming file:migfile (qemu) info status (qemu) c Signed-off-by: Fabiano Rosas --- migration/multifd.c | 13 ++++++++++++- migration/ram.c | 9 +++++++-- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 3f95a41ee9..3b6053ae5a 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -142,6 +142,7 @@ static void nocomp_recv_cleanup(MultiFDRecvParams *p) static int nocomp_recv_pages(MultiFDRecvParams *p, Error **errp) { uint32_t flags = p->flags & MULTIFD_FLAG_COMPRESSION_MASK; + uint64_t read_base = 0; if (flags != MULTIFD_FLAG_NOCOMP) { error_setg(errp, "multifd %u: flags received %x flags expected %x", @@ -152,7 +153,13 @@ static int nocomp_recv_pages(MultiFDRecvParams *p, Error **errp) p->iov[i].iov_base = p->host + p->normal[i]; p->iov[i].iov_len = p->page_size; } - return qio_channel_readv_all(p->c, p->iov, p->normal_num, errp); + + if (migrate_fixed_ram()) { + read_base = p->pages->block->pages_offset - (uint64_t) p->host; + } + + return qio_channel_read_full_all(p->c, p->iov, p->normal_num, read_base, + p->read_flags, errp); } static MultiFDMethods multifd_nocomp_ops = { @@ -1225,6 +1232,7 @@ void multifd_recv_sync_main(void) if (!migrate_multifd() || !migrate_multifd_packets()) { return; } + for (i = 0; i < migrate_multifd_channels(); i++) { MultiFDRecvParams *p = &multifd_recv_state->params[i]; @@ -1257,6 +1265,7 @@ static void *multifd_recv_thread(void *opaque) while (true) { uint32_t flags; + p->normal_num = 0; if (p->quit) { break; @@ -1378,6 +1387,8 @@ int multifd_load_setup(Error **errp) p->packet_len = sizeof(MultiFDPacket_t) + sizeof(uint64_t) * page_count; p->packet = g_malloc0(p->packet_len); + } else { + p->read_flags |= QIO_CHANNEL_READ_FLAG_WITH_OFFSET; } p->name = g_strdup_printf("multifdrecv_%d", i); p->iov = g_new0(struct iovec, page_count); diff --git a/migration/ram.c b/migration/ram.c index 5c67e30e55..9a5ee4767b 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3985,8 +3985,13 @@ static void read_ramblock_fixed_ram(QEMUFile *f, RAMBlock *block, host = host_from_ram_block_offset(block, offset); read_len = MIN(len, TARGET_PAGE_SIZE); - read = qemu_get_buffer_at(f, host, read_len, - block->pages_offset + offset); + if (migrate_multifd()) { + multifd_recv_queue_page(f, block, offset); + read = read_len; + } else { + read = qemu_get_buffer_at(f, host, read_len, + block->pages_offset + offset); + } completed += read; } } From patchwork Mon Oct 23 20:36:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854022 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=B81RE54w; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=awMxd1mO; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn9H2B3zz23k5 for ; Tue, 24 Oct 2023 07:39:15 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1g4-0004L2-IT; Mon, 23 Oct 2023 16:37:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1g1-0003t8-0t for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:29 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1fw-0001uh-Ta for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:26 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 8C1EF218B8; Mon, 23 Oct 2023 20:37:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093443; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jbj+lfNoYZuvrFmM1+xCWownS6WVnQj3JPJd+xN/RQs=; b=B81RE54w0pIj3NBJAW59hPi8F+caSRgoNBCLPvxmG6hwepR1RLfJzMh6DP4b8BWQR/cNFm J+lVKWBzRu+bwDRTq1ZSU1JeH7vKfjBfUHDXzVKNlJ3czHGTSd4G+Xfy/qnNKqsYrWN8nv T98fYVLlq3j9hZAouF7mDpxfjZUqKms= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093443; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jbj+lfNoYZuvrFmM1+xCWownS6WVnQj3JPJd+xN/RQs=; b=awMxd1mOIt/o8Y3A3WZd7dEkCfypYiCL5EQwQrIocM77zr5yYu3uJQIlxSIjjHaZ8xOhi5 z+LyCKsifTwDT6DQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id EA65B132FD; Mon, 23 Oct 2023 20:37:20 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id uJy/LIDZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:37:20 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana , Thomas Huth , Laurent Vivier , Paolo Bonzini Subject: [PATCH v2 27/29] tests/qtest: Add a multifd + fixed-ram migration test Date: Mon, 23 Oct 2023 17:36:06 -0300 Message-Id: <20231023203608.26370-28-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out1.suse.de; none X-Spam-Score: 0.30 X-Spamd-Result: default: False [0.30 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[10]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-0.60)[81.75%] Received-SPF: pass client-ip=195.135.220.28; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Signed-off-by: Fabiano Rosas --- tests/qtest/migration-test.c | 45 ++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index bd4e866e67..c74c911283 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -2064,6 +2064,46 @@ static void test_precopy_file_fixed_ram(void) test_file_common(&args, false, true); } +static void *migrate_multifd_fixed_ram_start(QTestState *from, QTestState *to) +{ + migrate_fixed_ram_start(from, to); + + migrate_set_parameter_int(from, "multifd-channels", 4); + migrate_set_parameter_int(to, "multifd-channels", 4); + + migrate_set_capability(from, "multifd", true); + migrate_set_capability(to, "multifd", true); + + return NULL; +} + +static void test_multifd_file_fixed_ram_live(void) +{ + g_autofree char *uri = g_strdup_printf("file:%s/%s", tmpfs, + FILE_TEST_FILENAME); + MigrateCommon args = { + .connect_uri = uri, + .listen_uri = "defer", + .start_hook = migrate_multifd_fixed_ram_start, + }; + + test_file_common(&args, false, false); +} + +static void test_multifd_file_fixed_ram(void) +{ + g_autofree char *uri = g_strdup_printf("file:%s/%s", tmpfs, + FILE_TEST_FILENAME); + MigrateCommon args = { + .connect_uri = uri, + .listen_uri = "defer", + .start_hook = migrate_multifd_fixed_ram_start, + }; + + test_file_common(&args, false, true); +} + + static void test_precopy_tcp_plain(void) { MigrateCommon args = { @@ -3137,6 +3177,11 @@ int main(int argc, char **argv) qtest_add_func("/migration/precopy/file/fixed-ram/live", test_precopy_file_fixed_ram_live); + qtest_add_func("/migration/multifd/file/fixed-ram", + test_multifd_file_fixed_ram); + qtest_add_func("/migration/multifd/file/fixed-ram/live", + test_multifd_file_fixed_ram_live); + #ifdef CONFIG_GNUTLS qtest_add_func("/migration/precopy/unix/tls/psk", test_precopy_unix_tls_psk); From patchwork Mon Oct 23 20:36:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854028 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=me8vNr6M; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=2gcc/fNs; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDnBT2s5Lz202k for ; Tue, 24 Oct 2023 07:40:17 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1gL-0004hQ-51; Mon, 23 Oct 2023 16:37:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1g3-0004IL-Vd for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:31 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1g0-0001v2-PA for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:31 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 40DC61FE32; Mon, 23 Oct 2023 20:37:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093446; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vn5D4xMrx6L+YOzdic/8lubsQz9ayHBY+jnjMPqEryM=; b=me8vNr6MxTr0LuxKNH/7TYLYcOZiJMP03HBqzimnqqHEnSUfbPE2hsRzazt0jlBH7wbjI5 mMi/TSwjlJAmchYcVl7GloUJQ86St0et3c2YdEA7MglPVunC1r+Y0jtCwBCxQ53a7XqCdJ HSRis9j/bLj2jIFzngVO6uW0EyJQyXk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093446; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vn5D4xMrx6L+YOzdic/8lubsQz9ayHBY+jnjMPqEryM=; b=2gcc/fNsj0de8CuKrsn7OC7gVoXf6PUVvYywQXynBApPbG3ShT/KJMTtCWvZZkU7uMS5lX bp+62G1Ux6P0zbBA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 1DF0C132FD; Mon, 23 Oct 2023 20:37:23 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 8IF/NoPZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:37:23 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana , Eric Blake Subject: [PATCH v2 28/29] migration: Add direct-io parameter Date: Mon, 23 Oct 2023 17:36:07 -0300 Message-Id: <20231023203608.26370-29-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out2.suse.de; none X-Spam-Score: -2.10 X-Spamd-Result: default: False [-2.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[8]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] Received-SPF: pass client-ip=195.135.220.29; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Add the direct-io migration parameter that tells the migration code to use O_DIRECT when opening the migration stream file whenever possible. This is currently only used for the secondary channels of fixed-ram migration, which can guarantee that writes are page aligned. However the parameter could be made to affect other types of file-based migrations in the future. Signed-off-by: Fabiano Rosas --- include/qemu/osdep.h | 2 ++ migration/file.c | 15 ++++++++++++--- migration/migration-hmp-cmds.c | 10 ++++++++++ migration/options.c | 30 ++++++++++++++++++++++++++++++ migration/options.h | 1 + qapi/migration.json | 17 ++++++++++++++--- util/osdep.c | 9 +++++++++ 7 files changed, 78 insertions(+), 6 deletions(-) diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h index 475a1c62ff..ea5d29ab9b 100644 --- a/include/qemu/osdep.h +++ b/include/qemu/osdep.h @@ -597,6 +597,8 @@ int qemu_lock_fd_test(int fd, int64_t start, int64_t len, bool exclusive); bool qemu_has_ofd_lock(void); #endif +bool qemu_has_direct_io(void); + #if defined(__HAIKU__) && defined(__i386__) #define FMT_pid "%ld" #elif defined(WIN64) diff --git a/migration/file.c b/migration/file.c index ad75225f43..3d3c58ecad 100644 --- a/migration/file.c +++ b/migration/file.c @@ -11,9 +11,9 @@ #include "qemu/error-report.h" #include "channel.h" #include "file.h" -#include "migration.h" #include "io/channel-file.h" #include "io/channel-util.h" +#include "migration.h" #include "options.h" #include "trace.h" @@ -77,9 +77,18 @@ void file_send_channel_create(QIOTaskFunc f, void *data) QIOChannelFile *ioc; QIOTask *task; Error *errp = NULL; + int flags = outgoing_args.flags; - ioc = qio_channel_file_new_path(outgoing_args.fname, - outgoing_args.flags, + if (migrate_direct_io() && qemu_has_direct_io()) { + /* + * Enable O_DIRECT for the secondary channels. These are used + * for sending ram pages and writes should be guaranteed to be + * aligned to at least page size. + */ + flags |= O_DIRECT; + } + + ioc = qio_channel_file_new_path(outgoing_args.fname, flags, outgoing_args.mode, &errp); if (!ioc) { file_migration_cancel(errp); diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index a82597f18e..eab5ac3588 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -387,6 +387,12 @@ void hmp_info_migrate_parameters(Monitor *mon, const QDict *qdict) monitor_printf(mon, "%s: %" PRIu64 " MB/s\n", MigrationParameter_str(MIGRATION_PARAMETER_VCPU_DIRTY_LIMIT), params->vcpu_dirty_limit); + + if (params->has_direct_io) { + monitor_printf(mon, "%s: %s\n", + MigrationParameter_str(MIGRATION_PARAMETER_DIRECT_IO), + params->direct_io ? "on" : "off"); + } } qapi_free_MigrationParameters(params); @@ -661,6 +667,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict) p->has_vcpu_dirty_limit = true; visit_type_size(v, param, &p->vcpu_dirty_limit, &err); break; + case MIGRATION_PARAMETER_DIRECT_IO: + p->has_direct_io = true; + visit_type_bool(v, param, &p->direct_io, &err); + break; default: assert(0); } diff --git a/migration/options.c b/migration/options.c index 2193d69e71..6d0e3c26ae 100644 --- a/migration/options.c +++ b/migration/options.c @@ -817,6 +817,22 @@ int migrate_decompress_threads(void) return s->parameters.decompress_threads; } +bool migrate_direct_io(void) +{ + MigrationState *s = migrate_get_current(); + + /* For now O_DIRECT is only supported with fixed-ram */ + if (!s->capabilities[MIGRATION_CAPABILITY_FIXED_RAM]) { + return false; + } + + if (s->parameters.has_direct_io) { + return s->parameters.direct_io; + } + + return false; +} + uint64_t migrate_downtime_limit(void) { MigrationState *s = migrate_get_current(); @@ -1025,6 +1041,11 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp) params->has_vcpu_dirty_limit = true; params->vcpu_dirty_limit = s->parameters.vcpu_dirty_limit; + if (s->parameters.has_direct_io) { + params->has_direct_io = true; + params->direct_io = s->parameters.direct_io; + } + return params; } @@ -1059,6 +1080,7 @@ void migrate_params_init(MigrationParameters *params) params->has_announce_step = true; params->has_x_vcpu_dirty_limit_period = true; params->has_vcpu_dirty_limit = true; + params->has_direct_io = qemu_has_direct_io(); } /* @@ -1356,6 +1378,10 @@ static void migrate_params_test_apply(MigrateSetParameters *params, if (params->has_vcpu_dirty_limit) { dest->vcpu_dirty_limit = params->vcpu_dirty_limit; } + + if (params->has_direct_io) { + dest->direct_io = params->direct_io; + } } static void migrate_params_apply(MigrateSetParameters *params, Error **errp) @@ -1486,6 +1512,10 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp) if (params->has_vcpu_dirty_limit) { s->parameters.vcpu_dirty_limit = params->vcpu_dirty_limit; } + + if (params->has_direct_io) { + s->parameters.direct_io = params->direct_io; + } } void qmp_migrate_set_parameters(MigrateSetParameters *params, Error **errp) diff --git a/migration/options.h b/migration/options.h index 01bba5b928..280f86bed1 100644 --- a/migration/options.h +++ b/migration/options.h @@ -82,6 +82,7 @@ uint8_t migrate_cpu_throttle_increment(void); uint8_t migrate_cpu_throttle_initial(void); bool migrate_cpu_throttle_tailslow(void); int migrate_decompress_threads(void); +bool migrate_direct_io(void); uint64_t migrate_downtime_limit(void); uint8_t migrate_max_cpu_throttle(void); uint64_t migrate_max_bandwidth(void); diff --git a/qapi/migration.json b/qapi/migration.json index 1317dd32ab..3eb9e2c9b5 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -840,6 +840,9 @@ # @vcpu-dirty-limit: Dirtyrate limit (MB/s) during live migration. # Defaults to 1. (Since 8.1) # +# @direct-io: Open migration files with O_DIRECT when possible. Not +# all migration transports support this. (since 8.1) +# # Features: # # @unstable: Members @x-checkpoint-delay and @x-vcpu-dirty-limit-period @@ -864,7 +867,7 @@ 'multifd-zlib-level', 'multifd-zstd-level', 'block-bitmap-mapping', { 'name': 'x-vcpu-dirty-limit-period', 'features': ['unstable'] }, - 'vcpu-dirty-limit'] } + 'vcpu-dirty-limit', 'direct-io'] } ## # @MigrateSetParameters: @@ -1016,6 +1019,9 @@ # @vcpu-dirty-limit: Dirtyrate limit (MB/s) during live migration. # Defaults to 1. (Since 8.1) # +# @direct-io: Open migration files with O_DIRECT when possible. Not +# all migration transports support this. (since 8.1) +# # Features: # # @unstable: Members @x-checkpoint-delay and @x-vcpu-dirty-limit-period @@ -1058,7 +1064,8 @@ '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ], '*x-vcpu-dirty-limit-period': { 'type': 'uint64', 'features': [ 'unstable' ] }, - '*vcpu-dirty-limit': 'uint64'} } + '*vcpu-dirty-limit': 'uint64', + '*direct-io': 'bool' } } ## # @migrate-set-parameters: @@ -1230,6 +1237,9 @@ # @vcpu-dirty-limit: Dirtyrate limit (MB/s) during live migration. # Defaults to 1. (Since 8.1) # +# @direct-io: Open migration files with O_DIRECT when possible. Not +# all migration transports support this. (since 8.1) +# # Features: # # @unstable: Members @x-checkpoint-delay and @x-vcpu-dirty-limit-period @@ -1269,7 +1279,8 @@ '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ], '*x-vcpu-dirty-limit-period': { 'type': 'uint64', 'features': [ 'unstable' ] }, - '*vcpu-dirty-limit': 'uint64'} } + '*vcpu-dirty-limit': 'uint64', + '*direct-io': 'bool' } } ## # @query-migrate-parameters: diff --git a/util/osdep.c b/util/osdep.c index e996c4744a..d0227a60ab 100644 --- a/util/osdep.c +++ b/util/osdep.c @@ -277,6 +277,15 @@ int qemu_lock_fd_test(int fd, int64_t start, int64_t len, bool exclusive) } #endif +bool qemu_has_direct_io(void) +{ +#ifdef O_DIRECT + return true; +#else + return false; +#endif +} + static int qemu_open_cloexec(const char *name, int flags, mode_t mode) { int ret; From patchwork Mon Oct 23 20:36:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1854015 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=MNSW9cDa; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=f2zlPQ6B; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDn8N4mQwz23jl for ; Tue, 24 Oct 2023 07:38:28 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qv1gP-000514-Pq; Mon, 23 Oct 2023 16:37:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv1g7-0004RT-86 for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:42 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qv1g3-0001vK-F3 for qemu-devel@nongnu.org; Mon, 23 Oct 2023 16:37:33 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 4A5491FD80; Mon, 23 Oct 2023 20:37:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698093449; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VxjcXnlpfVciijB/64SFMFUdVR3bR4Q58jhz8LrltoE=; b=MNSW9cDaJgOiinEQn1fB459epuu3KWfey9fEXvIdTcIOH+w7cKrLWsiHqjkg1Rrjly2UIU kOWKpWYOCV1kToqktIh5eg2oRAOlq0Lcc9BKn1QzyEVfrZ2+U5WHW/TO24OjfXS237Ef6M U47LbRFvBOOT1o1A2FIAILzu+3dm85w= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698093449; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VxjcXnlpfVciijB/64SFMFUdVR3bR4Q58jhz8LrltoE=; b=f2zlPQ6BvW1MXuoGutfE/ma3Gy0WEwVyGki+AuRXOdaP5XAd9GruMUQ+URhHDUwMrVpjR/ oEXW+E6fuJsb5MBA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id AAEE7132FD; Mon, 23 Oct 2023 20:37:26 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id gLcjHYbZNmV1JQAAMHmgww (envelope-from ); Mon, 23 Oct 2023 20:37:26 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Juan Quintela , Peter Xu , Leonardo Bras , Claudio Fontana , Thomas Huth , Laurent Vivier , Paolo Bonzini Subject: [PATCH v2 29/29] tests/qtest: Add a test for migration with direct-io and multifd Date: Mon, 23 Oct 2023 17:36:08 -0300 Message-Id: <20231023203608.26370-30-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231023203608.26370-1-farosas@suse.de> References: <20231023203608.26370-1-farosas@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out2.suse.de; none X-Spam-Score: -1.49 X-Spamd-Result: default: False [-1.49 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_SEVEN(0.00)[10]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-2.39)[97.21%] Received-SPF: pass client-ip=195.135.220.29; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Signed-off-by: Fabiano Rosas --- tests/qtest/migration-test.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index c74c911283..30e70c0e4e 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -2077,6 +2077,16 @@ static void *migrate_multifd_fixed_ram_start(QTestState *from, QTestState *to) return NULL; } +static void *migrate_multifd_fixed_ram_dio_start(QTestState *from, QTestState *to) +{ + migrate_multifd_fixed_ram_start(from, to); + + migrate_set_parameter_bool(from, "direct-io", true); + migrate_set_parameter_bool(to, "direct-io", true); + + return NULL; +} + static void test_multifd_file_fixed_ram_live(void) { g_autofree char *uri = g_strdup_printf("file:%s/%s", tmpfs, @@ -2103,6 +2113,18 @@ static void test_multifd_file_fixed_ram(void) test_file_common(&args, false, true); } +static void test_multifd_file_fixed_ram_dio(void) +{ + g_autofree char *uri = g_strdup_printf("file:%s/%s", tmpfs, + FILE_TEST_FILENAME); + MigrateCommon args = { + .connect_uri = uri, + .listen_uri = "defer", + .start_hook = migrate_multifd_fixed_ram_dio_start, + }; + + test_file_common(&args, false, true); +} static void test_precopy_tcp_plain(void) { @@ -3182,6 +3204,9 @@ int main(int argc, char **argv) qtest_add_func("/migration/multifd/file/fixed-ram/live", test_multifd_file_fixed_ram_live); + qtest_add_func("/migration/multifd/file/fixed-ram/dio", + test_multifd_file_fixed_ram_dio); + #ifdef CONFIG_GNUTLS qtest_add_func("/migration/precopy/unix/tls/psk", test_precopy_unix_tls_psk);