From patchwork Wed Oct 4 12:49:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1843344 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=i/E2/cri; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4S0vkw31bXz1yng for ; Wed, 4 Oct 2023 23:53:44 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qo1Jh-0004Iq-0v; Wed, 04 Oct 2023 08:49:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qo1Jf-0004Hm-Ky for qemu-devel@nongnu.org; Wed, 04 Oct 2023 08:49:27 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qo1Jd-0006li-QC for qemu-devel@nongnu.org; Wed, 04 Oct 2023 08:49:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696423765; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=e9MU41DA6JfnvYM4B2zisrscwrZtuuap9bac0VUCHHY=; b=i/E2/cri5Cslmnsf/s9pCgwG5Z+Anvg95qUTout6m8K0pDx15gNo3n51F/PFaqk571hIG/ oIK2cgtDyWYcIjNzmL+DEhVj5C3HsS9T7feXAkMH9zLUaxIN1XJDm8MYAt3toQh4LkHv9l dk/NBJdcisKtxw2ILVboSWw5B9B86GU= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-176-fgnqabkjOGWUu0_n7yZQhQ-1; Wed, 04 Oct 2023 08:49:17 -0400 X-MC-Unique: fgnqabkjOGWUu0_n7yZQhQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EB35A1C0433F; Wed, 4 Oct 2023 12:49:16 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.90]) by smtp.corp.redhat.com (Postfix) with ESMTP id BFE30C15BB8; Wed, 4 Oct 2023 12:49:15 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Li Zhijian , Juan Quintela , Peter Xu , Fabiano Rosas , Leonardo Bras Subject: [PATCH v2 01/10] migration: Create migrate_rdma() Date: Wed, 4 Oct 2023 14:49:04 +0200 Message-ID: <20231004124913.16360-2-quintela@redhat.com> In-Reply-To: <20231004124913.16360-1-quintela@redhat.com> References: <20231004124913.16360-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Helper to say if we are doing a migration over rdma. Reviewed-by: Peter Xu Signed-off-by: Juan Quintela --- migration/migration.h | 2 ++ migration/options.h | 1 + migration/migration.c | 1 + migration/options.c | 7 +++++++ migration/rdma.c | 4 +++- 5 files changed, 14 insertions(+), 1 deletion(-) diff --git a/migration/migration.h b/migration/migration.h index 972597f4de..4543e36081 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -463,6 +463,8 @@ struct MigrationState { * switchover has been received. */ bool switchover_acked; + /* Is this a rdma migration */ + bool rdma_migration; }; void migrate_set_state(int *state, int old_state, int new_state); diff --git a/migration/options.h b/migration/options.h index 045e2a41a2..a26fd1680b 100644 --- a/migration/options.h +++ b/migration/options.h @@ -56,6 +56,7 @@ bool migrate_zero_copy_send(void); bool migrate_multifd_flush_after_each_section(void); bool migrate_postcopy(void); +bool migrate_rdma(void); bool migrate_tls(void); /* capabilities helpers */ diff --git a/migration/migration.c b/migration/migration.c index 585d3c8f55..1cb8898e6b 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1439,6 +1439,7 @@ int migrate_init(MigrationState *s, Error **errp) s->iteration_initial_bytes = 0; s->threshold_size = 0; s->switchover_acked = false; + s->rdma_migration = false; /* * set mig_stats compression_counters memory to zero for a * new migration diff --git a/migration/options.c b/migration/options.c index 1d1e1321b0..0107d63b9b 100644 --- a/migration/options.c +++ b/migration/options.c @@ -376,6 +376,13 @@ bool migrate_postcopy(void) return migrate_postcopy_ram() || migrate_dirty_bitmaps(); } +bool migrate_rdma(void) +{ + MigrationState *s = migrate_get_current(); + + return s->rdma_migration; +} + bool migrate_tls(void) { MigrationState *s = migrate_get_current(); diff --git a/migration/rdma.c b/migration/rdma.c index cd5e1afe60..6219227915 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -4126,6 +4126,7 @@ void rdma_start_incoming_migration(const char *host_port, Error **errp) int ret; RDMAContext *rdma; Error *local_err = NULL; + MigrationState *s = migrate_get_current(); trace_rdma_start_incoming_migration(); @@ -4156,7 +4157,7 @@ void rdma_start_incoming_migration(const char *host_port, Error **errp) } trace_rdma_start_incoming_migration_after_rdma_listen(); - + s->rdma_migration = true; qemu_set_fd_handler(rdma->channel->fd, rdma_accept_incoming_migration, NULL, (void *)(intptr_t)rdma); return; @@ -4233,6 +4234,7 @@ void rdma_start_outgoing_migration(void *opaque, trace_rdma_start_outgoing_migration_after_rdma_connect(); s->to_dst_file = rdma_new_output(rdma); + s->rdma_migration = true; migrate_fd_connect(s, NULL); return; return_path_err: From patchwork Wed Oct 4 12:49:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1843340 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4S0vkf1cLTz1yng for ; Wed, 4 Oct 2023 23:53:30 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qo1Jh-0004JA-7i; Wed, 04 Oct 2023 08:49:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qo1Jf-0004HK-3r for qemu-devel@nongnu.org; Wed, 04 Oct 2023 08:49:27 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qo1Jd-0006ku-5a for qemu-devel@nongnu.org; Wed, 04 Oct 2023 08:49:26 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-650-LKQwVBWtNgGGzqbizK45-A-1; Wed, 04 Oct 2023 08:49:18 -0400 X-MC-Unique: LKQwVBWtNgGGzqbizK45-A-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 62D5781D783; Wed, 4 Oct 2023 12:49:18 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.90]) by smtp.corp.redhat.com (Postfix) with ESMTP id 39B8BC154CB; Wed, 4 Oct 2023 12:49:17 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Li Zhijian , Juan Quintela , Peter Xu , Fabiano Rosas , Leonardo Bras Subject: [PATCH v2 02/10] migration/rdma: Unfold ram_control_before_iterate() Date: Wed, 4 Oct 2023 14:49:05 +0200 Message-ID: <20231004124913.16360-3-quintela@redhat.com> In-Reply-To: <20231004124913.16360-1-quintela@redhat.com> References: <20231004124913.16360-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Once there: - Remove unused data parameter - unfold it in its callers. - change all callers to call qemu_rdma_registration_start() - We need to call QIO_CHANNEL_RDMA() after we check for migrate_rdma() Reviewed-by: Peter Xu Signed-off-by: Juan Quintela Reviewed-by: Fabiano Rosas --- initilazize rioc after checknig that rdma is enabled. Signed-off-by: Juan Quintela --- migration/qemu-file.h | 2 -- migration/rdma.h | 7 +++++++ migration/qemu-file.c | 13 +------------ migration/ram.c | 16 +++++++++++++--- migration/rdma.c | 12 ++++-------- 5 files changed, 25 insertions(+), 25 deletions(-) diff --git a/migration/qemu-file.h b/migration/qemu-file.h index 03e718c264..d6a370c569 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -55,7 +55,6 @@ typedef int (QEMURamSaveFunc)(QEMUFile *f, size_t size); typedef struct QEMUFileHooks { - QEMURamHookFunc *before_ram_iterate; QEMURamHookFunc *after_ram_iterate; QEMURamHookFunc *hook_ram_load; QEMURamSaveFunc *save_page; @@ -127,7 +126,6 @@ void qemu_fflush(QEMUFile *f); void qemu_file_set_blocking(QEMUFile *f, bool block); int qemu_file_get_to_fd(QEMUFile *f, int fd, size_t size); -void ram_control_before_iterate(QEMUFile *f, uint64_t flags); void ram_control_after_iterate(QEMUFile *f, uint64_t flags); void ram_control_load_hook(QEMUFile *f, uint64_t flags, void *data); diff --git a/migration/rdma.h b/migration/rdma.h index de2ba09dc5..670c67a8cb 100644 --- a/migration/rdma.h +++ b/migration/rdma.h @@ -22,4 +22,11 @@ void rdma_start_outgoing_migration(void *opaque, const char *host_port, void rdma_start_incoming_migration(const char *host_port, Error **errp); + +#ifdef CONFIG_RDMA +int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags); +#else +static inline +int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags) { return 0; } +#endif #endif diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 5e8207dae4..1a8170421f 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -32,6 +32,7 @@ #include "trace.h" #include "options.h" #include "qapi/error.h" +#include "rdma.h" #define IO_BUF_SIZE 32768 #define MAX_IOV_SIZE MIN_CONST(IOV_MAX, 64) @@ -288,18 +289,6 @@ void qemu_fflush(QEMUFile *f) f->iovcnt = 0; } -void ram_control_before_iterate(QEMUFile *f, uint64_t flags) -{ - int ret = 0; - - if (f->hooks && f->hooks->before_ram_iterate) { - ret = f->hooks->before_ram_iterate(f, flags, NULL); - if (ret < 0) { - qemu_file_set_error(f, ret); - } - } -} - void ram_control_after_iterate(QEMUFile *f, uint64_t flags) { int ret = 0; diff --git a/migration/ram.c b/migration/ram.c index e4bfd39f08..407760b3a2 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -59,6 +59,7 @@ #include "qemu/iov.h" #include "multifd.h" #include "sysemu/runstate.h" +#include "rdma.h" #include "options.h" #include "sysemu/dirtylimit.h" #include "sysemu/kvm.h" @@ -3054,7 +3055,10 @@ static int ram_save_setup(QEMUFile *f, void *opaque) } } - ram_control_before_iterate(f, RAM_CONTROL_SETUP); + ret = qemu_rdma_registration_start(f, RAM_CONTROL_SETUP); + if (ret < 0) { + qemu_file_set_error(f, ret); + } ram_control_after_iterate(f, RAM_CONTROL_SETUP); migration_ops = g_malloc0(sizeof(MigrationOps)); @@ -3114,7 +3118,10 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) /* Read version before ram_list.blocks */ smp_rmb(); - ram_control_before_iterate(f, RAM_CONTROL_ROUND); + ret = qemu_rdma_registration_start(f, RAM_CONTROL_ROUND); + if (ret < 0) { + qemu_file_set_error(f, ret); + } t0 = qemu_clock_get_ns(QEMU_CLOCK_REALTIME); i = 0; @@ -3219,7 +3226,10 @@ static int ram_save_complete(QEMUFile *f, void *opaque) migration_bitmap_sync_precopy(rs, true); } - ram_control_before_iterate(f, RAM_CONTROL_FINISH); + ret = qemu_rdma_registration_start(f, RAM_CONTROL_FINISH); + if (ret < 0) { + qemu_file_set_error(f, ret); + } /* try transferring iterative blocks of memory */ diff --git a/migration/rdma.c b/migration/rdma.c index 6219227915..a93d8d705d 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3868,18 +3868,15 @@ static int rdma_load_hook(QEMUFile *f, uint64_t flags, void *data) } } -static int qemu_rdma_registration_start(QEMUFile *f, - uint64_t flags, void *data) +int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags) { - QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(qemu_file_get_ioc(f)); - RDMAContext *rdma; - - if (migration_in_postcopy()) { + if (!migrate_rdma () || migration_in_postcopy()) { return 0; } + QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(qemu_file_get_ioc(f)); RCU_READ_LOCK_GUARD(); - rdma = qatomic_rcu_read(&rioc->rdmaout); + RDMAContext *rdma = qatomic_rcu_read(&rioc->rdmaout); if (!rdma) { return -EIO; } @@ -4012,7 +4009,6 @@ static const QEMUFileHooks rdma_read_hooks = { }; static const QEMUFileHooks rdma_write_hooks = { - .before_ram_iterate = qemu_rdma_registration_start, .after_ram_iterate = qemu_rdma_registration_stop, .save_page = qemu_rdma_save_page, }; From patchwork Wed Oct 4 12:49:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1843337 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=hyGfvLL4; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4S0vgL0mQnz1yng for ; Wed, 4 Oct 2023 23:50:38 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qo1Jh-0004JE-8h; Wed, 04 Oct 2023 08:49:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qo1Jf-0004Hq-Mi for qemu-devel@nongnu.org; Wed, 04 Oct 2023 08:49:27 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qo1Jd-0006kZ-H0 for qemu-devel@nongnu.org; Wed, 04 Oct 2023 08:49:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696423763; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bqkYDICcUKQeEAJNHNLdGwxaS8sV7bbJx68o/+F+X6M=; b=hyGfvLL4tVClkRVJU8ITDWigJ58kqNmF0lUfQw5eE9wRYmhqGCRa9Dei0o6xe2FZNYMDhT sW6M9LLoyBxMCqEFlQgdaVo5EfBUkmCRGo7/CPTcPh7NxueMXa8cnSIrpbu9FXSX0xqC2F aqKjTMX0/atY93Bl5DH83+7UbaTPc5s= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-392-CKQZZP80NxCpkMrcQu6y8Q-1; Wed, 04 Oct 2023 08:49:20 -0400 X-MC-Unique: CKQZZP80NxCpkMrcQu6y8Q-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CC3A31C0419B; Wed, 4 Oct 2023 12:49:19 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.90]) by smtp.corp.redhat.com (Postfix) with ESMTP id A433FC15BB8; Wed, 4 Oct 2023 12:49:18 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Li Zhijian , Juan Quintela , Peter Xu , Fabiano Rosas , Leonardo Bras Subject: [PATCH v2 03/10] migration/rdma: Unfold ram_control_after_iterate() Date: Wed, 4 Oct 2023 14:49:06 +0200 Message-ID: <20231004124913.16360-4-quintela@redhat.com> In-Reply-To: <20231004124913.16360-1-quintela@redhat.com> References: <20231004124913.16360-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Once there: - Remove unused data parameter - unfold it in its callers - change all callers to call qemu_rdma_registration_stop() - We need to call QIO_CHANNEL_RDMA() after we check for migrate_rdma() Reviewed-by: Peter Xu Signed-off-by: Juan Quintela --- This function has goto's. So I don't change the place where we declare variables, althought I think that it is correct, just don't start that discussion. --- migration/qemu-file.h | 2 -- migration/rdma.h | 3 +++ migration/qemu-file.c | 12 ------------ migration/ram.c | 17 ++++++++++++++--- migration/rdma.c | 9 ++++----- 5 files changed, 21 insertions(+), 22 deletions(-) diff --git a/migration/qemu-file.h b/migration/qemu-file.h index d6a370c569..35e671a01e 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -55,7 +55,6 @@ typedef int (QEMURamSaveFunc)(QEMUFile *f, size_t size); typedef struct QEMUFileHooks { - QEMURamHookFunc *after_ram_iterate; QEMURamHookFunc *hook_ram_load; QEMURamSaveFunc *save_page; } QEMUFileHooks; @@ -126,7 +125,6 @@ void qemu_fflush(QEMUFile *f); void qemu_file_set_blocking(QEMUFile *f, bool block); int qemu_file_get_to_fd(QEMUFile *f, int fd, size_t size); -void ram_control_after_iterate(QEMUFile *f, uint64_t flags); void ram_control_load_hook(QEMUFile *f, uint64_t flags, void *data); /* Whenever this is found in the data stream, the flags diff --git a/migration/rdma.h b/migration/rdma.h index 670c67a8cb..c13b94c782 100644 --- a/migration/rdma.h +++ b/migration/rdma.h @@ -25,8 +25,11 @@ void rdma_start_incoming_migration(const char *host_port, Error **errp); #ifdef CONFIG_RDMA int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags); +int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags); #else static inline int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags) { return 0; } +static inline +int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags) { return 0; } #endif #endif diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 1a8170421f..35c5c9109b 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -289,18 +289,6 @@ void qemu_fflush(QEMUFile *f) f->iovcnt = 0; } -void ram_control_after_iterate(QEMUFile *f, uint64_t flags) -{ - int ret = 0; - - if (f->hooks && f->hooks->after_ram_iterate) { - ret = f->hooks->after_ram_iterate(f, flags, NULL); - if (ret < 0) { - qemu_file_set_error(f, ret); - } - } -} - void ram_control_load_hook(QEMUFile *f, uint64_t flags, void *data) { if (f->hooks && f->hooks->hook_ram_load) { diff --git a/migration/ram.c b/migration/ram.c index 407760b3a2..7bbdb69501 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3059,7 +3059,11 @@ static int ram_save_setup(QEMUFile *f, void *opaque) if (ret < 0) { qemu_file_set_error(f, ret); } - ram_control_after_iterate(f, RAM_CONTROL_SETUP); + + ret = qemu_rdma_registration_stop(f, RAM_CONTROL_SETUP); + if (ret < 0) { + qemu_file_set_error(f, ret); + } migration_ops = g_malloc0(sizeof(MigrationOps)); migration_ops->ram_save_target_page = ram_save_target_page_legacy; @@ -3178,7 +3182,10 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) * Must occur before EOS (or any QEMUFile operation) * because of RDMA protocol. */ - ram_control_after_iterate(f, RAM_CONTROL_ROUND); + ret = qemu_rdma_registration_stop(f, RAM_CONTROL_ROUND); + if (ret < 0) { + qemu_file_set_error(f, ret); + } out: if (ret >= 0 @@ -3251,7 +3258,11 @@ static int ram_save_complete(QEMUFile *f, void *opaque) qemu_mutex_unlock(&rs->bitmap_mutex); ram_flush_compressed_data(rs); - ram_control_after_iterate(f, RAM_CONTROL_FINISH); + + int ret = qemu_rdma_registration_stop(f, RAM_CONTROL_FINISH); + if (ret < 0) { + qemu_file_set_error(f, ret); + } } if (ret < 0) { diff --git a/migration/rdma.c b/migration/rdma.c index a93d8d705d..0eff308577 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3894,19 +3894,19 @@ int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags) * Inform dest that dynamic registrations are done for now. * First, flush writes, if any. */ -static int qemu_rdma_registration_stop(QEMUFile *f, - uint64_t flags, void *data) +int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags) { - QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(qemu_file_get_ioc(f)); + QIOChannelRDMA *rioc; RDMAContext *rdma; RDMAControlHeader head = { .len = 0, .repeat = 1 }; int ret = 0; - if (migration_in_postcopy()) { + if (!migrate_rdma() || migration_in_postcopy()) { return 0; } RCU_READ_LOCK_GUARD(); + rioc = QIO_CHANNEL_RDMA(qemu_file_get_ioc(f)); rdma = qatomic_rcu_read(&rioc->rdmaout); if (!rdma) { return -EIO; @@ -4009,7 +4009,6 @@ static const QEMUFileHooks rdma_read_hooks = { }; static const QEMUFileHooks rdma_write_hooks = { - .after_ram_iterate = qemu_rdma_registration_stop, .save_page = qemu_rdma_save_page, }; From patchwork Wed Oct 4 12:49:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1843341 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=RmVkc6jC; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4S0vkj3qxrz1yng for ; Wed, 4 Oct 2023 23:53:33 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qo1Jj-0004MP-8o; Wed, 04 Oct 2023 08:49:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qo1Jf-0004He-Gv for qemu-devel@nongnu.org; Wed, 04 Oct 2023 08:49:27 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qo1Jd-0006lS-Dl for qemu-devel@nongnu.org; Wed, 04 Oct 2023 08:49:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696423764; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=i9Q7zBvhm5DMMFW68+/DUJZtZWVnU2s37cpMWFPBBqE=; b=RmVkc6jCHBfwnDk0uvbvM//rCh6mU1dlCDc302Xr+o6sae22wObJPscQrnbX8m/QpuABPR tnzEoXPz3qOEaxBu/+9a72t6T/Ord1hNFk6pqvqMkQt3zsTM5diVV83Ebb6eneArS8l08D soRYhmnbV5ehLoenbMe12VaqQe6SRt0= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-515-qraykl6VPTmeMomOe7hxgg-1; Wed, 04 Oct 2023 08:49:21 -0400 X-MC-Unique: qraykl6VPTmeMomOe7hxgg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 22EC028237C7; Wed, 4 Oct 2023 12:49:21 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.90]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1ABE1C15BB8; Wed, 4 Oct 2023 12:49:19 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Li Zhijian , Juan Quintela , Peter Xu , Fabiano Rosas , Leonardo Bras Subject: [PATCH v2 04/10] migration/rdma: Remove all uses of RAM_CONTROL_HOOK Date: Wed, 4 Oct 2023 14:49:07 +0200 Message-ID: <20231004124913.16360-5-quintela@redhat.com> In-Reply-To: <20231004124913.16360-1-quintela@redhat.com> References: <20231004124913.16360-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Instead of going trhough ram_control_load_hook(), call qemu_rdma_registration_handle() directly. Reviewed-by: Peter Xu Signed-off-by: Juan Quintela --- migration/qemu-file.h | 1 - migration/rdma.h | 3 +++ migration/ram.c | 5 ++++- migration/rdma.c | 12 +++++++----- 4 files changed, 14 insertions(+), 7 deletions(-) diff --git a/migration/qemu-file.h b/migration/qemu-file.h index 35e671a01e..14ff0d9cc4 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -41,7 +41,6 @@ typedef int (QEMURamHookFunc)(QEMUFile *f, uint64_t flags, void *data); */ #define RAM_CONTROL_SETUP 0 #define RAM_CONTROL_ROUND 1 -#define RAM_CONTROL_HOOK 2 #define RAM_CONTROL_FINISH 3 #define RAM_CONTROL_BLOCK_REG 4 diff --git a/migration/rdma.h b/migration/rdma.h index c13b94c782..8bd277efb9 100644 --- a/migration/rdma.h +++ b/migration/rdma.h @@ -24,10 +24,13 @@ void rdma_start_incoming_migration(const char *host_port, Error **errp); #ifdef CONFIG_RDMA +int qemu_rdma_registration_handle(QEMUFile *f); int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags); int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags); #else static inline +int qemu_rdma_registration_handle(QEMUFile *f) { return 0; } +static inline int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags) { return 0; } static inline int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags) { return 0; } diff --git a/migration/ram.c b/migration/ram.c index 7bbdb69501..b3abcad1b4 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -4064,7 +4064,10 @@ static int ram_load_precopy(QEMUFile *f) } break; case RAM_SAVE_FLAG_HOOK: - ram_control_load_hook(f, RAM_CONTROL_HOOK, NULL); + ret = qemu_rdma_registration_handle(f); + if (ret < 0) { + qemu_file_set_error(f, ret); + } break; default: error_report("Unknown combination of migration flags: 0x%x", flags); diff --git a/migration/rdma.c b/migration/rdma.c index 0eff308577..6208f1f818 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3535,7 +3535,7 @@ static int dest_ram_sort_func(const void *a, const void *b) * * Keep doing this until the source tells us to stop. */ -static int qemu_rdma_registration_handle(QEMUFile *f) +int qemu_rdma_registration_handle(QEMUFile *f) { RDMAControlHeader reg_resp = { .len = sizeof(RDMARegisterResult), .type = RDMA_CONTROL_REGISTER_RESULT, @@ -3547,7 +3547,7 @@ static int qemu_rdma_registration_handle(QEMUFile *f) }; RDMAControlHeader blocks = { .type = RDMA_CONTROL_RAM_BLOCKS_RESULT, .repeat = 1 }; - QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(qemu_file_get_ioc(f)); + QIOChannelRDMA *rioc; RDMAContext *rdma; RDMALocalBlocks *local; RDMAControlHeader head; @@ -3562,7 +3562,12 @@ static int qemu_rdma_registration_handle(QEMUFile *f) int count = 0; int i = 0; + if (!migrate_rdma()) { + return 0; + } + RCU_READ_LOCK_GUARD(); + rioc = QIO_CHANNEL_RDMA(qemu_file_get_ioc(f)); rdma = qatomic_rcu_read(&rioc->rdmain); if (!rdma) { @@ -3859,9 +3864,6 @@ static int rdma_load_hook(QEMUFile *f, uint64_t flags, void *data) case RAM_CONTROL_BLOCK_REG: return rdma_block_notification_handle(f, data); - case RAM_CONTROL_HOOK: - return qemu_rdma_registration_handle(f); - default: /* Shouldn't be called with any other values */ abort(); From patchwork Wed Oct 4 12:49:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1843345 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=QVJ658b3; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4S0vky6Dscz1yng for ; Wed, 4 Oct 2023 23:53:46 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qo1Ji-0004JO-UL; Wed, 04 Oct 2023 08:49:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qo1Jg-0004In-PF for qemu-devel@nongnu.org; Wed, 04 Oct 2023 08:49:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qo1Jd-0006kw-W5 for qemu-devel@nongnu.org; Wed, 04 Oct 2023 08:49:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696423764; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8xS+kII0jvSdTKfFZqzKi2cAJZEjKFgKVl7CcotjwCA=; b=QVJ658b3lMZKK+JCy3KmoA7q+kI/bwZ+YfyJp+RVY+Jzt7U3NNCFh27JpImWnKBevbnF4h Hczh21WFj5GrEWFUXCAStdDEvvtd8KQbnIuwQKA9NXENFeRYyJEbsQvmXnpv83IJ6R55kN Tn069AVhbBgIO7XKYkkrSu8YXe1zAVY= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-26-fU1-IfcmPvSyqHqJBtfaoQ-1; Wed, 04 Oct 2023 08:49:22 -0400 X-MC-Unique: fU1-IfcmPvSyqHqJBtfaoQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 750D83801EF0; Wed, 4 Oct 2023 12:49:22 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.90]) by smtp.corp.redhat.com (Postfix) with ESMTP id 62506C15BB8; Wed, 4 Oct 2023 12:49:21 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Li Zhijian , Juan Quintela , Peter Xu , Fabiano Rosas , Leonardo Bras Subject: [PATCH v2 05/10] migration/rdma: Unfold hook_ram_load() Date: Wed, 4 Oct 2023 14:49:08 +0200 Message-ID: <20231004124913.16360-6-quintela@redhat.com> In-Reply-To: <20231004124913.16360-1-quintela@redhat.com> References: <20231004124913.16360-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org There is only one flag called with: RAM_CONTROL_BLOCK_REG. Reviewed-by: Peter Xu Signed-off-by: Juan Quintela --- migration/qemu-file.h | 11 ----------- migration/rdma.h | 3 +++ migration/qemu-file.c | 10 ---------- migration/ram.c | 6 ++++-- migration/rdma.c | 34 +++++++++++----------------------- 5 files changed, 18 insertions(+), 46 deletions(-) diff --git a/migration/qemu-file.h b/migration/qemu-file.h index 14ff0d9cc4..80c30631dc 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -29,20 +29,12 @@ #include "exec/cpu-common.h" #include "io/channel.h" -/* - * This function provides hooks around different - * stages of RAM migration. - * 'data' is call specific data associated with the 'flags' value - */ -typedef int (QEMURamHookFunc)(QEMUFile *f, uint64_t flags, void *data); - /* * Constants used by ram_control_* hooks */ #define RAM_CONTROL_SETUP 0 #define RAM_CONTROL_ROUND 1 #define RAM_CONTROL_FINISH 3 -#define RAM_CONTROL_BLOCK_REG 4 /* * This function allows override of where the RAM page @@ -54,7 +46,6 @@ typedef int (QEMURamSaveFunc)(QEMUFile *f, size_t size); typedef struct QEMUFileHooks { - QEMURamHookFunc *hook_ram_load; QEMURamSaveFunc *save_page; } QEMUFileHooks; @@ -124,8 +115,6 @@ void qemu_fflush(QEMUFile *f); void qemu_file_set_blocking(QEMUFile *f, bool block); int qemu_file_get_to_fd(QEMUFile *f, int fd, size_t size); -void ram_control_load_hook(QEMUFile *f, uint64_t flags, void *data); - /* Whenever this is found in the data stream, the flags * will be passed to ram_control_load_hook in the incoming-migration * side. This lets before_ram_iterate/after_ram_iterate add diff --git a/migration/rdma.h b/migration/rdma.h index 8bd277efb9..8df8b4089a 100644 --- a/migration/rdma.h +++ b/migration/rdma.h @@ -27,6 +27,7 @@ void rdma_start_incoming_migration(const char *host_port, Error **errp); int qemu_rdma_registration_handle(QEMUFile *f); int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags); int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags); +int rdma_block_notification_handle(QEMUFile *f, const char *name); #else static inline int qemu_rdma_registration_handle(QEMUFile *f) { return 0; } @@ -34,5 +35,7 @@ static inline int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags) { return 0; } static inline int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags) { return 0; } +static inline +int rdma_block_notification_handle(QEMUFile *f, const char *name) { return 0; } #endif #endif diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 35c5c9109b..a15d1f6635 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -289,16 +289,6 @@ void qemu_fflush(QEMUFile *f) f->iovcnt = 0; } -void ram_control_load_hook(QEMUFile *f, uint64_t flags, void *data) -{ - if (f->hooks && f->hooks->hook_ram_load) { - int ret = f->hooks->hook_ram_load(f, flags, data); - if (ret < 0) { - qemu_file_set_error(f, ret); - } - } -} - int ram_control_save_page(QEMUFile *f, ram_addr_t block_offset, ram_addr_t offset, size_t size) { diff --git a/migration/ram.c b/migration/ram.c index b3abcad1b4..656e1dd96c 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -4015,8 +4015,10 @@ static int ram_load_precopy(QEMUFile *f) ret = -EINVAL; } } - ram_control_load_hook(f, RAM_CONTROL_BLOCK_REG, - block->idstr); + ret = rdma_block_notification_handle(f, block->idstr); + if (ret < 0) { + qemu_file_set_error(f, ret); + } } else { error_report("Unknown ramblock \"%s\", cannot " "accept migration", id); diff --git a/migration/rdma.c b/migration/rdma.c index 6208f1f818..65514999a3 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3817,22 +3817,23 @@ out: } /* Destination: - * Called via a ram_control_load_hook during the initial RAM load section which - * lists the RAMBlocks by name. This lets us know the order of the RAMBlocks - * on the source. - * We've already built our local RAMBlock list, but not yet sent the list to - * the source. + * Called during the initial RAM load section which lists the + * RAMBlocks by name. This lets us know the order of the RAMBlocks on + * the source. We've already built our local RAMBlock list, but not + * yet sent the list to the source. */ -static int -rdma_block_notification_handle(QEMUFile *f, const char *name) +int rdma_block_notification_handle(QEMUFile *f, const char *name) { - RDMAContext *rdma; - QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(qemu_file_get_ioc(f)); int curr; int found = -1; + if (!migrate_rdma()) { + return 0; + } + RCU_READ_LOCK_GUARD(); - rdma = qatomic_rcu_read(&rioc->rdmain); + QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(qemu_file_get_ioc(f)); + RDMAContext *rdma = qatomic_rcu_read(&rioc->rdmain); if (!rdma) { return -EIO; @@ -3858,18 +3859,6 @@ rdma_block_notification_handle(QEMUFile *f, const char *name) return 0; } -static int rdma_load_hook(QEMUFile *f, uint64_t flags, void *data) -{ - switch (flags) { - case RAM_CONTROL_BLOCK_REG: - return rdma_block_notification_handle(f, data); - - default: - /* Shouldn't be called with any other values */ - abort(); - } -} - int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags) { if (!migrate_rdma () || migration_in_postcopy()) { @@ -4007,7 +3996,6 @@ err: } static const QEMUFileHooks rdma_read_hooks = { - .hook_ram_load = rdma_load_hook, }; static const QEMUFileHooks rdma_write_hooks = { From patchwork Wed Oct 4 12:49:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1843335 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=BWikptjQ; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4S0vfR5Ycgz1yph for ; Wed, 4 Oct 2023 23:49:51 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qo1Jk-0004NY-2y; Wed, 04 Oct 2023 08:49:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qo1Jh-0004JM-Sn for qemu-devel@nongnu.org; Wed, 04 Oct 2023 08:49:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qo1Jf-0006mV-VW for qemu-devel@nongnu.org; Wed, 04 Oct 2023 08:49:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696423767; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=g8Pjmasy6+GRIBw3Fg1/G8cwMzWt0Q224997T/Po1Sc=; b=BWikptjQqq5awUS3k9iQFB8q4cFYWCoh9RWXlEEDXIGeFg+SthJmzzgGDDni8G7dzwDqmr hJS40mZaGsNzZoW6Ki0q9S8KzYwMK7RnyQNIHKH6+OPkjxvMBzl7Bq1uEhF46/4i9FBRPk SBdlfmgmug3PaKfSv/Z2bP5ag6YCpNc= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-386-iSrzpD2MOsK3mFYdkh_Fqw-1; Wed, 04 Oct 2023 08:49:24 -0400 X-MC-Unique: iSrzpD2MOsK3mFYdkh_Fqw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E00ED1C04199; Wed, 4 Oct 2023 12:49:23 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.90]) by smtp.corp.redhat.com (Postfix) with ESMTP id B5843C15BB8; Wed, 4 Oct 2023 12:49:22 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Li Zhijian , Juan Quintela , Peter Xu , Fabiano Rosas , Leonardo Bras Subject: [PATCH v2 06/10] migration/rdma: Create rdma_control_save_page() Date: Wed, 4 Oct 2023 14:49:09 +0200 Message-ID: <20231004124913.16360-7-quintela@redhat.com> In-Reply-To: <20231004124913.16360-1-quintela@redhat.com> References: <20231004124913.16360-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The only user of ram_control_save_page() and save_page() hook was rdma. Just move the function to rdma.c, rename it to rdma_control_save_page(). Reviewed-by: Peter Xu Signed-off-by: Juan Quintela --- migration/qemu-file.h | 12 ------------ migration/rdma.h | 10 ++++++++++ migration/qemu-file.c | 20 -------------------- migration/ram.c | 4 ++-- migration/rdma.c | 20 +++++++++++++++++++- 5 files changed, 31 insertions(+), 35 deletions(-) diff --git a/migration/qemu-file.h b/migration/qemu-file.h index 80c30631dc..60510a2819 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -36,17 +36,7 @@ #define RAM_CONTROL_ROUND 1 #define RAM_CONTROL_FINISH 3 -/* - * This function allows override of where the RAM page - * is saved (such as RDMA, for example.) - */ -typedef int (QEMURamSaveFunc)(QEMUFile *f, - ram_addr_t block_offset, - ram_addr_t offset, - size_t size); - typedef struct QEMUFileHooks { - QEMURamSaveFunc *save_page; } QEMUFileHooks; QEMUFile *qemu_file_new_input(QIOChannel *ioc); @@ -125,8 +115,6 @@ int qemu_file_get_to_fd(QEMUFile *f, int fd, size_t size); #define RAM_SAVE_CONTROL_NOT_SUPP -1000 #define RAM_SAVE_CONTROL_DELAYED -2000 -int ram_control_save_page(QEMUFile *f, ram_addr_t block_offset, - ram_addr_t offset, size_t size); QIOChannel *qemu_file_get_ioc(QEMUFile *file); #endif diff --git a/migration/rdma.h b/migration/rdma.h index 8df8b4089a..09a16c1e3c 100644 --- a/migration/rdma.h +++ b/migration/rdma.h @@ -17,6 +17,8 @@ #ifndef QEMU_MIGRATION_RDMA_H #define QEMU_MIGRATION_RDMA_H +#include "exec/memory.h" + void rdma_start_outgoing_migration(void *opaque, const char *host_port, Error **errp); @@ -28,6 +30,8 @@ int qemu_rdma_registration_handle(QEMUFile *f); int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags); int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags); int rdma_block_notification_handle(QEMUFile *f, const char *name); +int rdma_control_save_page(QEMUFile *f, ram_addr_t block_offset, + ram_addr_t offset, size_t size); #else static inline int qemu_rdma_registration_handle(QEMUFile *f) { return 0; } @@ -37,5 +41,11 @@ static inline int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags) { return 0; } static inline int rdma_block_notification_handle(QEMUFile *f, const char *name) { return 0; } +static inline +int rdma_control_save_page(QEMUFile *f, ram_addr_t block_offset, + ram_addr_t offset, size_t size) +{ + return RAM_SAVE_CONTROL_NOT_SUPP; +} #endif #endif diff --git a/migration/qemu-file.c b/migration/qemu-file.c index a15d1f6635..e3514740fb 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -289,26 +289,6 @@ void qemu_fflush(QEMUFile *f) f->iovcnt = 0; } -int ram_control_save_page(QEMUFile *f, ram_addr_t block_offset, - ram_addr_t offset, size_t size) -{ - if (f->hooks && f->hooks->save_page) { - int ret = f->hooks->save_page(f, block_offset, offset, size); - /* - * RAM_SAVE_CONTROL_* are negative values - */ - if (ret != RAM_SAVE_CONTROL_DELAYED && - ret != RAM_SAVE_CONTROL_NOT_SUPP) { - if (ret < 0) { - qemu_file_set_error(f, ret); - } - } - return ret; - } - - return RAM_SAVE_CONTROL_NOT_SUPP; -} - /* * Attempt to fill the buffer from the underlying file * Returns the number of bytes read, or negative value for an error. diff --git a/migration/ram.c b/migration/ram.c index 656e1dd96c..e5d443daf2 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1189,8 +1189,8 @@ static bool control_save_page(PageSearchStatus *pss, RAMBlock *block, { int ret; - ret = ram_control_save_page(pss->pss_channel, block->offset, offset, - TARGET_PAGE_SIZE); + ret = rdma_control_save_page(pss->pss_channel, block->offset, offset, + TARGET_PAGE_SIZE); if (ret == RAM_SAVE_CONTROL_NOT_SUPP) { return false; } diff --git a/migration/rdma.c b/migration/rdma.c index 65514999a3..7f10bfb81b 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3324,6 +3324,25 @@ err: return ret; } +int rdma_control_save_page(QEMUFile *f, ram_addr_t block_offset, + ram_addr_t offset, size_t size) +{ + if (!migrate_rdma()) { + return RAM_SAVE_CONTROL_NOT_SUPP; + } + + int ret = qemu_rdma_save_page(f, block_offset, offset, size); + + if (ret != RAM_SAVE_CONTROL_DELAYED && + ret != RAM_SAVE_CONTROL_NOT_SUPP) { + if (ret < 0) { + qemu_file_set_error(f, ret); + } + } + return ret; +} + + static void rdma_accept_incoming_migration(void *opaque); static void rdma_cm_poll_handler(void *opaque) @@ -3999,7 +4018,6 @@ static const QEMUFileHooks rdma_read_hooks = { }; static const QEMUFileHooks rdma_write_hooks = { - .save_page = qemu_rdma_save_page, }; From patchwork Wed Oct 4 12:49:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1843336 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Dc5DGYJv; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4S0vfk5yGbz1ypx for ; Wed, 4 Oct 2023 23:50:06 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qo1Jm-0004On-MG; Wed, 04 Oct 2023 08:49:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qo1Jk-0004NP-TJ for qemu-devel@nongnu.org; Wed, 04 Oct 2023 08:49:33 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qo1Jh-0006nt-7d for qemu-devel@nongnu.org; Wed, 04 Oct 2023 08:49:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696423768; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=f0oy7NU7qNwaSv6DKXuDG0U9gzTLoRBDG+LgTwRNog4=; b=Dc5DGYJvRyFwEK4DMRmeP8Kw+uk33mP0/w0oM9TOk79SctNt+tGtXnL5LEn86+r75wKi3R or3NmOcTS9/S8TrVna7KtVPgCuaQmQDMGIoHALO4atLSWUovGtBanFUHOy2zoymF4Brwnn Dw+4OJVc44u0fedZswwplloetHlhYLM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-342-Ibt-sZJTPr2VeYUyUstoKA-1; Wed, 04 Oct 2023 08:49:25 -0400 X-MC-Unique: Ibt-sZJTPr2VeYUyUstoKA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 360678007A4; Wed, 4 Oct 2023 12:49:25 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.90]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2DA22C154CC; Wed, 4 Oct 2023 12:49:24 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Li Zhijian , Juan Quintela , Peter Xu , Fabiano Rosas , Leonardo Bras Subject: [PATCH v2 07/10] qemu-file: Remove QEMUFileHooks Date: Wed, 4 Oct 2023 14:49:10 +0200 Message-ID: <20231004124913.16360-8-quintela@redhat.com> In-Reply-To: <20231004124913.16360-1-quintela@redhat.com> References: <20231004124913.16360-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The only user was rdma, and its use is gone. Reviewed-by: Peter Xu Signed-off-by: Juan Quintela --- migration/qemu-file.h | 4 ---- migration/qemu-file.c | 6 ------ migration/rdma.c | 9 --------- 3 files changed, 19 deletions(-) diff --git a/migration/qemu-file.h b/migration/qemu-file.h index 60510a2819..0b22d8335f 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -36,12 +36,8 @@ #define RAM_CONTROL_ROUND 1 #define RAM_CONTROL_FINISH 3 -typedef struct QEMUFileHooks { -} QEMUFileHooks; - QEMUFile *qemu_file_new_input(QIOChannel *ioc); QEMUFile *qemu_file_new_output(QIOChannel *ioc); -void qemu_file_set_hooks(QEMUFile *f, const QEMUFileHooks *hooks); int qemu_fclose(QEMUFile *f); /* diff --git a/migration/qemu-file.c b/migration/qemu-file.c index e3514740fb..352bb320a2 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -38,7 +38,6 @@ #define MAX_IOV_SIZE MIN_CONST(IOV_MAX, 64) struct QEMUFile { - const QEMUFileHooks *hooks; QIOChannel *ioc; bool is_writable; @@ -133,11 +132,6 @@ QEMUFile *qemu_file_new_input(QIOChannel *ioc) return qemu_file_new_impl(ioc, false); } -void qemu_file_set_hooks(QEMUFile *f, const QEMUFileHooks *hooks) -{ - f->hooks = hooks; -} - /* * Get last error for stream f with optional Error* * diff --git a/migration/rdma.c b/migration/rdma.c index 7f10bfb81b..bcb6de14e7 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -4014,13 +4014,6 @@ err: return ret; } -static const QEMUFileHooks rdma_read_hooks = { -}; - -static const QEMUFileHooks rdma_write_hooks = { -}; - - static void qio_channel_rdma_finalize(Object *obj) { QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(obj); @@ -4072,7 +4065,6 @@ static QEMUFile *rdma_new_input(RDMAContext *rdma) rioc->file = qemu_file_new_input(QIO_CHANNEL(rioc)); rioc->rdmain = rdma; rioc->rdmaout = rdma->return_path; - qemu_file_set_hooks(rioc->file, &rdma_read_hooks); return rioc->file; } @@ -4084,7 +4076,6 @@ static QEMUFile *rdma_new_output(RDMAContext *rdma) rioc->file = qemu_file_new_output(QIO_CHANNEL(rioc)); rioc->rdmaout = rdma; rioc->rdmain = rdma->return_path; - qemu_file_set_hooks(rioc->file, &rdma_write_hooks); return rioc->file; } From patchwork Wed Oct 4 12:49:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1843342 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ZQKeyPLh; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4S0vkj6g5Yz1ypx for ; Wed, 4 Oct 2023 23:53:33 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qo1Jo-0004QV-Mf; Wed, 04 Oct 2023 08:49:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qo1Jm-0004Om-IW for qemu-devel@nongnu.org; Wed, 04 Oct 2023 08:49:34 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qo1Jk-0006pU-UI for qemu-devel@nongnu.org; Wed, 04 Oct 2023 08:49:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696423772; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NAww9ifZU6DIVsqPpQv+zvXrEcr7jvwrYo1gmKjhqcM=; b=ZQKeyPLhzopehcyNn1QeG2TLZ3La+Mv5IiAl8VBFdnz2yJG1JvtN33fu2eJVhGyEts1FmT QhekUrZkAwJ3lT/xTsu66Osj6G7A4Io1/O4T+ayVVbmdc15vPxMAQU0hmrcyC+3/tltI+t 8vO2W7Ssd1OqYjvzKe55rez86m6rS8E= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-663-9plFeFxzM0aYfHDXS__XWg-1; Wed, 04 Oct 2023 08:49:26 -0400 X-MC-Unique: 9plFeFxzM0aYfHDXS__XWg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7EF24101A550; Wed, 4 Oct 2023 12:49:26 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.90]) by smtp.corp.redhat.com (Postfix) with ESMTP id 78214C15BB8; Wed, 4 Oct 2023 12:49:25 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Li Zhijian , Juan Quintela , Peter Xu , Fabiano Rosas , Leonardo Bras Subject: [PATCH v2 08/10] migration/rdma: Move rdma constants from qemu-file.h to rdma.h Date: Wed, 4 Oct 2023 14:49:11 +0200 Message-ID: <20231004124913.16360-9-quintela@redhat.com> In-Reply-To: <20231004124913.16360-1-quintela@redhat.com> References: <20231004124913.16360-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Reviewed-by: Peter Xu Signed-off-by: Juan Quintela --- migration/qemu-file.h | 17 ----------------- migration/rdma.h | 16 ++++++++++++++++ migration/ram.c | 2 +- 3 files changed, 17 insertions(+), 18 deletions(-) diff --git a/migration/qemu-file.h b/migration/qemu-file.h index 0b22d8335f..a29c37b0d0 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -29,13 +29,6 @@ #include "exec/cpu-common.h" #include "io/channel.h" -/* - * Constants used by ram_control_* hooks - */ -#define RAM_CONTROL_SETUP 0 -#define RAM_CONTROL_ROUND 1 -#define RAM_CONTROL_FINISH 3 - QEMUFile *qemu_file_new_input(QIOChannel *ioc); QEMUFile *qemu_file_new_output(QIOChannel *ioc); int qemu_fclose(QEMUFile *f); @@ -101,16 +94,6 @@ void qemu_fflush(QEMUFile *f); void qemu_file_set_blocking(QEMUFile *f, bool block); int qemu_file_get_to_fd(QEMUFile *f, int fd, size_t size); -/* Whenever this is found in the data stream, the flags - * will be passed to ram_control_load_hook in the incoming-migration - * side. This lets before_ram_iterate/after_ram_iterate add - * transport-specific sections to the RAM migration data. - */ -#define RAM_SAVE_FLAG_HOOK 0x80 - -#define RAM_SAVE_CONTROL_NOT_SUPP -1000 -#define RAM_SAVE_CONTROL_DELAYED -2000 - QIOChannel *qemu_file_get_ioc(QEMUFile *file); #endif diff --git a/migration/rdma.h b/migration/rdma.h index 09a16c1e3c..1ff3718a76 100644 --- a/migration/rdma.h +++ b/migration/rdma.h @@ -24,6 +24,22 @@ void rdma_start_outgoing_migration(void *opaque, const char *host_port, void rdma_start_incoming_migration(const char *host_port, Error **errp); +/* + * Constants used by rdma return codes + */ +#define RAM_CONTROL_SETUP 0 +#define RAM_CONTROL_ROUND 1 +#define RAM_CONTROL_FINISH 3 + +/* + * Whenever this is found in the data stream, the flags + * will be passed to rdma functions in the incoming-migration + * side. + */ +#define RAM_SAVE_FLAG_HOOK 0x80 + +#define RAM_SAVE_CONTROL_NOT_SUPP -1000 +#define RAM_SAVE_CONTROL_DELAYED -2000 #ifdef CONFIG_RDMA int qemu_rdma_registration_handle(QEMUFile *f); diff --git a/migration/ram.c b/migration/ram.c index e5d443daf2..f3bf7b9185 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -89,7 +89,7 @@ #define RAM_SAVE_FLAG_EOS 0x10 #define RAM_SAVE_FLAG_CONTINUE 0x20 #define RAM_SAVE_FLAG_XBZRLE 0x40 -/* 0x80 is reserved in qemu-file.h for RAM_SAVE_FLAG_HOOK */ +/* 0x80 is reserved in rdma.h for RAM_SAVE_FLAG_HOOK */ #define RAM_SAVE_FLAG_COMPRESS_PAGE 0x100 #define RAM_SAVE_FLAG_MULTIFD_FLUSH 0x200 /* We can't use any flag that is bigger than 0x200 */ From patchwork Wed Oct 4 12:49:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1843343 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=EswDKGdI; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4S0vkn1Lfwz1yng for ; Wed, 4 Oct 2023 23:53:37 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qo1Jp-0004Qi-AY; Wed, 04 Oct 2023 08:49:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qo1Jn-0004PD-Ai for qemu-devel@nongnu.org; Wed, 04 Oct 2023 08:49:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qo1Jl-0006pa-4L for qemu-devel@nongnu.org; Wed, 04 Oct 2023 08:49:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696423772; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9wRTD5giybEw68V1PDmeHnoeErlYmhxTKuys4OCt95M=; b=EswDKGdIPzyQxnQVig2tHDvjjL8BsG0T3pQ+klnaM58cvESth9F/ZMV9mPSzfY9K8whcvw sfePMDd157lOi8+2Np6RbRvgPzlVwY5m8lbtUkqY0paWY89C9SGA69yeRBjGiHxjkfu0kN oJFCwdGJRxg6X08WQ9EeJaoLjXguNmU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-116-LNHBBVFOP0WThU0zbOi1rw-1; Wed, 04 Oct 2023 08:49:28 -0400 X-MC-Unique: LNHBBVFOP0WThU0zbOi1rw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EADDB81B13F; Wed, 4 Oct 2023 12:49:27 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.90]) by smtp.corp.redhat.com (Postfix) with ESMTP id C2196C15BB8; Wed, 4 Oct 2023 12:49:26 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Li Zhijian , Juan Quintela , Peter Xu , Fabiano Rosas , Leonardo Bras Subject: [PATCH v2 09/10] migration/rdma: Remove qemu_ prefix from exported functions Date: Wed, 4 Oct 2023 14:49:12 +0200 Message-ID: <20231004124913.16360-10-quintela@redhat.com> In-Reply-To: <20231004124913.16360-1-quintela@redhat.com> References: <20231004124913.16360-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Functions are long enough even without this. Reviewed-by: Peter Xu Signed-off-by: Juan Quintela --- migration/rdma.h | 12 ++++++------ migration/ram.c | 14 +++++++------- migration/rdma.c | 40 +++++++++++++++++++--------------------- migration/trace-events | 28 ++++++++++++++-------------- 4 files changed, 46 insertions(+), 48 deletions(-) diff --git a/migration/rdma.h b/migration/rdma.h index 1ff3718a76..30b15b4466 100644 --- a/migration/rdma.h +++ b/migration/rdma.h @@ -42,19 +42,19 @@ void rdma_start_incoming_migration(const char *host_port, Error **errp); #define RAM_SAVE_CONTROL_DELAYED -2000 #ifdef CONFIG_RDMA -int qemu_rdma_registration_handle(QEMUFile *f); -int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags); -int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags); +int rdma_registration_handle(QEMUFile *f); +int rdma_registration_start(QEMUFile *f, uint64_t flags); +int rdma_registration_stop(QEMUFile *f, uint64_t flags); int rdma_block_notification_handle(QEMUFile *f, const char *name); int rdma_control_save_page(QEMUFile *f, ram_addr_t block_offset, ram_addr_t offset, size_t size); #else static inline -int qemu_rdma_registration_handle(QEMUFile *f) { return 0; } +int rdma_registration_handle(QEMUFile *f) { return 0; } static inline -int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags) { return 0; } +int rdma_registration_start(QEMUFile *f, uint64_t flags) { return 0; } static inline -int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags) { return 0; } +int rdma_registration_stop(QEMUFile *f, uint64_t flags) { return 0; } static inline int rdma_block_notification_handle(QEMUFile *f, const char *name) { return 0; } static inline diff --git a/migration/ram.c b/migration/ram.c index f3bf7b9185..abfc57a58c 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3055,12 +3055,12 @@ static int ram_save_setup(QEMUFile *f, void *opaque) } } - ret = qemu_rdma_registration_start(f, RAM_CONTROL_SETUP); + ret = rdma_registration_start(f, RAM_CONTROL_SETUP); if (ret < 0) { qemu_file_set_error(f, ret); } - ret = qemu_rdma_registration_stop(f, RAM_CONTROL_SETUP); + ret = rdma_registration_stop(f, RAM_CONTROL_SETUP); if (ret < 0) { qemu_file_set_error(f, ret); } @@ -3122,7 +3122,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) /* Read version before ram_list.blocks */ smp_rmb(); - ret = qemu_rdma_registration_start(f, RAM_CONTROL_ROUND); + ret = rdma_registration_start(f, RAM_CONTROL_ROUND); if (ret < 0) { qemu_file_set_error(f, ret); } @@ -3182,7 +3182,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) * Must occur before EOS (or any QEMUFile operation) * because of RDMA protocol. */ - ret = qemu_rdma_registration_stop(f, RAM_CONTROL_ROUND); + ret = rdma_registration_stop(f, RAM_CONTROL_ROUND); if (ret < 0) { qemu_file_set_error(f, ret); } @@ -3233,7 +3233,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque) migration_bitmap_sync_precopy(rs, true); } - ret = qemu_rdma_registration_start(f, RAM_CONTROL_FINISH); + ret = rdma_registration_start(f, RAM_CONTROL_FINISH); if (ret < 0) { qemu_file_set_error(f, ret); } @@ -3259,7 +3259,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque) ram_flush_compressed_data(rs); - int ret = qemu_rdma_registration_stop(f, RAM_CONTROL_FINISH); + int ret = rdma_registration_stop(f, RAM_CONTROL_FINISH); if (ret < 0) { qemu_file_set_error(f, ret); } @@ -4066,7 +4066,7 @@ static int ram_load_precopy(QEMUFile *f) } break; case RAM_SAVE_FLAG_HOOK: - ret = qemu_rdma_registration_handle(f); + ret = rdma_registration_handle(f); if (ret < 0) { qemu_file_set_error(f, ret); } diff --git a/migration/rdma.c b/migration/rdma.c index bcb6de14e7..c73e610416 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3554,7 +3554,7 @@ static int dest_ram_sort_func(const void *a, const void *b) * * Keep doing this until the source tells us to stop. */ -int qemu_rdma_registration_handle(QEMUFile *f) +int rdma_registration_handle(QEMUFile *f) { RDMAControlHeader reg_resp = { .len = sizeof(RDMARegisterResult), .type = RDMA_CONTROL_REGISTER_RESULT, @@ -3597,7 +3597,7 @@ int qemu_rdma_registration_handle(QEMUFile *f) local = &rdma->local_ram_blocks; do { - trace_qemu_rdma_registration_handle_wait(); + trace_rdma_registration_handle_wait(); ret = qemu_rdma_exchange_recv(rdma, &head, RDMA_CONTROL_NONE); @@ -3617,9 +3617,9 @@ int qemu_rdma_registration_handle(QEMUFile *f) comp = (RDMACompress *) rdma->wr_data[idx].control_curr; network_to_compress(comp); - trace_qemu_rdma_registration_handle_compress(comp->length, - comp->block_idx, - comp->offset); + trace_rdma_registration_handle_compress(comp->length, + comp->block_idx, + comp->offset); if (comp->block_idx >= rdma->local_ram_blocks.nb_blocks) { error_report("rdma: 'compress' bad block index %u (vs %d)", (unsigned int)comp->block_idx, @@ -3636,11 +3636,11 @@ int qemu_rdma_registration_handle(QEMUFile *f) break; case RDMA_CONTROL_REGISTER_FINISHED: - trace_qemu_rdma_registration_handle_finished(); + trace_rdma_registration_handle_finished(); goto out; case RDMA_CONTROL_RAM_BLOCKS_REQUEST: - trace_qemu_rdma_registration_handle_ram_blocks(); + trace_rdma_registration_handle_ram_blocks(); /* Sort our local RAM Block list so it's the same as the source, * we can do this since we've filled in a src_index in the list @@ -3680,7 +3680,7 @@ int qemu_rdma_registration_handle(QEMUFile *f) rdma->dest_blocks[i].length = local->block[i].length; dest_block_to_network(&rdma->dest_blocks[i]); - trace_qemu_rdma_registration_handle_ram_blocks_loop( + trace_rdma_registration_handle_ram_blocks_loop( local->block[i].block_name, local->block[i].offset, local->block[i].length, @@ -3702,7 +3702,7 @@ int qemu_rdma_registration_handle(QEMUFile *f) break; case RDMA_CONTROL_REGISTER_REQUEST: - trace_qemu_rdma_registration_handle_register(head.repeat); + trace_rdma_registration_handle_register(head.repeat); reg_resp.repeat = head.repeat; registers = (RDMARegister *) rdma->wr_data[idx].control_curr; @@ -3716,7 +3716,7 @@ int qemu_rdma_registration_handle(QEMUFile *f) reg_result = &results[count]; - trace_qemu_rdma_registration_handle_register_loop(count, + trace_rdma_registration_handle_register_loop(count, reg->current_index, reg->key.current_addr, reg->chunks); if (reg->current_index >= rdma->local_ram_blocks.nb_blocks) { @@ -3768,8 +3768,7 @@ int qemu_rdma_registration_handle(QEMUFile *f) reg_result->host_addr = (uintptr_t)block->local_host_addr; - trace_qemu_rdma_registration_handle_register_rkey( - reg_result->rkey); + trace_rdma_registration_handle_register_rkey(reg_result->rkey); result_to_network(reg_result); } @@ -3783,7 +3782,7 @@ int qemu_rdma_registration_handle(QEMUFile *f) } break; case RDMA_CONTROL_UNREGISTER_REQUEST: - trace_qemu_rdma_registration_handle_unregister(head.repeat); + trace_rdma_registration_handle_unregister(head.repeat); unreg_resp.repeat = head.repeat; registers = (RDMARegister *) rdma->wr_data[idx].control_curr; @@ -3791,7 +3790,7 @@ int qemu_rdma_registration_handle(QEMUFile *f) reg = ®isters[count]; network_to_register(reg); - trace_qemu_rdma_registration_handle_unregister_loop(count, + trace_rdma_registration_handle_unregister_loop(count, reg->current_index, reg->key.chunk); block = &(rdma->local_ram_blocks.block[reg->current_index]); @@ -3807,8 +3806,7 @@ int qemu_rdma_registration_handle(QEMUFile *f) rdma->total_registrations--; - trace_qemu_rdma_registration_handle_unregister_success( - reg->key.chunk); + trace_rdma_registration_handle_unregister_success(reg->key.chunk); } ret = qemu_rdma_post_send_control(rdma, NULL, &unreg_resp); @@ -3878,7 +3876,7 @@ int rdma_block_notification_handle(QEMUFile *f, const char *name) return 0; } -int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags) +int rdma_registration_start(QEMUFile *f, uint64_t flags) { if (!migrate_rdma () || migration_in_postcopy()) { return 0; @@ -3893,7 +3891,7 @@ int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags) CHECK_ERROR_STATE(); - trace_qemu_rdma_registration_start(flags); + trace_rdma_registration_start(flags); qemu_put_be64(f, RAM_SAVE_FLAG_HOOK); qemu_fflush(f); @@ -3904,7 +3902,7 @@ int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags) * Inform dest that dynamic registrations are done for now. * First, flush writes, if any. */ -int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags) +int rdma_registration_stop(QEMUFile *f, uint64_t flags) { QIOChannelRDMA *rioc; RDMAContext *rdma; @@ -3937,7 +3935,7 @@ int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags) int reg_result_idx, i, nb_dest_blocks; head.type = RDMA_CONTROL_RAM_BLOCKS_REQUEST; - trace_qemu_rdma_registration_stop_ram(); + trace_rdma_registration_stop_ram(); /* * Make sure that we parallelize the pinning on both sides. @@ -3999,7 +3997,7 @@ int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags) } } - trace_qemu_rdma_registration_stop(flags); + trace_rdma_registration_stop(flags); head.type = RDMA_CONTROL_REGISTER_FINISHED; ret = qemu_rdma_exchange_send(rdma, &head, NULL, NULL, NULL, NULL); diff --git a/migration/trace-events b/migration/trace-events index 002abe3a4e..3f86575f41 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -229,20 +229,6 @@ qemu_rdma_post_send_control(const char *desc) "CONTROL: sending %s.." qemu_rdma_register_and_get_keys(uint64_t len, void *start) "Registering %" PRIu64 " bytes @ %p" qemu_rdma_register_odp_mr(const char *name) "Try to register On-Demand Paging memory region: %s" qemu_rdma_advise_mr(const char *name, uint32_t len, uint64_t addr, const char *res) "Try to advise block %s prefetch at %" PRIu32 "@0x%" PRIx64 ": %s" -qemu_rdma_registration_handle_compress(int64_t length, int index, int64_t offset) "Zapping zero chunk: %" PRId64 " bytes, index %d, offset %" PRId64 -qemu_rdma_registration_handle_finished(void) "" -qemu_rdma_registration_handle_ram_blocks(void) "" -qemu_rdma_registration_handle_ram_blocks_loop(const char *name, uint64_t offset, uint64_t length, void *local_host_addr, unsigned int src_index) "%s: @0x%" PRIx64 "/%" PRIu64 " host:@%p src_index: %u" -qemu_rdma_registration_handle_register(int requests) "%d requests" -qemu_rdma_registration_handle_register_loop(int req, int index, uint64_t addr, uint64_t chunks) "Registration request (%d): index %d, current_addr %" PRIu64 " chunks: %" PRIu64 -qemu_rdma_registration_handle_register_rkey(int rkey) "0x%x" -qemu_rdma_registration_handle_unregister(int requests) "%d requests" -qemu_rdma_registration_handle_unregister_loop(int count, int index, uint64_t chunk) "Unregistration request (%d): index %d, chunk %" PRIu64 -qemu_rdma_registration_handle_unregister_success(uint64_t chunk) "%" PRIu64 -qemu_rdma_registration_handle_wait(void) "" -qemu_rdma_registration_start(uint64_t flags) "%" PRIu64 -qemu_rdma_registration_stop(uint64_t flags) "%" PRIu64 -qemu_rdma_registration_stop_ram(void) "" qemu_rdma_resolve_host_trying(const char *host, const char *ip) "Trying %s => %s" qemu_rdma_signal_unregister_append(uint64_t chunk, int pos) "Appending unregister chunk %" PRIu64 " at position %d" qemu_rdma_signal_unregister_already(uint64_t chunk) "Unregister chunk %" PRIu64 " already in queue" @@ -261,6 +247,20 @@ qemu_rdma_write_one_zero(uint64_t chunk, int len, int index, int64_t offset) "En rdma_add_block(const char *block_name, int block, uint64_t addr, uint64_t offset, uint64_t len, uint64_t end, uint64_t bits, int chunks) "Added Block: '%s':%d, addr: %" PRIu64 ", offset: %" PRIu64 " length: %" PRIu64 " end: %" PRIu64 " bits %" PRIu64 " chunks %d" rdma_block_notification_handle(const char *name, int index) "%s at %d" rdma_delete_block(void *block, uint64_t addr, uint64_t offset, uint64_t len, uint64_t end, uint64_t bits, int chunks) "Deleted Block: %p, addr: %" PRIu64 ", offset: %" PRIu64 " length: %" PRIu64 " end: %" PRIu64 " bits %" PRIu64 " chunks %d" +rdma_registration_handle_compress(int64_t length, int index, int64_t offset) "Zapping zero chunk: %" PRId64 " bytes, index %d, offset %" PRId64 +rdma_registration_handle_finished(void) "" +rdma_registration_handle_ram_blocks(void) "" +rdma_registration_handle_ram_blocks_loop(const char *name, uint64_t offset, uint64_t length, void *local_host_addr, unsigned int src_index) "%s: @0x%" PRIx64 "/%" PRIu64 " host:@%p src_index: %u" +rdma_registration_handle_register(int requests) "%d requests" +rdma_registration_handle_register_loop(int req, int index, uint64_t addr, uint64_t chunks) "Registration request (%d): index %d, current_addr %" PRIu64 " chunks: %" PRIu64 +rdma_registration_handle_register_rkey(int rkey) "0x%x" +rdma_registration_handle_unregister(int requests) "%d requests" +rdma_registration_handle_unregister_loop(int count, int index, uint64_t chunk) "Unregistration request (%d): index %d, chunk %" PRIu64 +rdma_registration_handle_unregister_success(uint64_t chunk) "%" PRIu64 +rdma_registration_handle_wait(void) "" +rdma_registration_start(uint64_t flags) "%" PRIu64 +rdma_registration_stop(uint64_t flags) "%" PRIu64 +rdma_registration_stop_ram(void) "" rdma_start_incoming_migration(void) "" rdma_start_incoming_migration_after_dest_init(void) "" rdma_start_incoming_migration_after_rdma_listen(void) "" From patchwork Wed Oct 4 12:49:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1843339 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=eufpir9N; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4S0vj66MzMz1yng for ; Wed, 4 Oct 2023 23:52:10 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qo1K1-0004YO-Ou; Wed, 04 Oct 2023 08:49:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qo1K0-0004YB-6G for qemu-devel@nongnu.org; Wed, 04 Oct 2023 08:49:48 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qo1Jv-0006qP-Jk for qemu-devel@nongnu.org; Wed, 04 Oct 2023 08:49:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696423782; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=M5eb0yuI4vnTN3ZsTIkFqXV6GouG2yDxVPAgnQXw9hA=; b=eufpir9NDxnX38eWFvZNLQolf00if2pyyfy2Wa6Sy9v1eqp+cz44XzzcIyLeFfzT5W81EG dIXXCS0e//iV26XZe/Mgxc1ckjK75/6x9br/D5PYy60Yd5zzGmYvJrxsIYwTl+8nY/cS7k jSSmPWpdI0W3/hblljzeCyYZ9fTgbhA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-475-t_udXBFSOG6eq9gYba68aA-1; Wed, 04 Oct 2023 08:49:29 -0400 X-MC-Unique: t_udXBFSOG6eq9gYba68aA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3FC32101A590; Wed, 4 Oct 2023 12:49:29 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.90]) by smtp.corp.redhat.com (Postfix) with ESMTP id 38E91C15BB8; Wed, 4 Oct 2023 12:49:28 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Li Zhijian , Juan Quintela , Peter Xu , Fabiano Rosas , Leonardo Bras Subject: [PATCH v2 10/10] migration/rdma: Check sooner if we are in postcopy for save_page() Date: Wed, 4 Oct 2023 14:49:13 +0200 Message-ID: <20231004124913.16360-11-quintela@redhat.com> In-Reply-To: <20231004124913.16360-1-quintela@redhat.com> References: <20231004124913.16360-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Reviewed-by: Peter Xu Signed-off-by: Juan Quintela --- migration/rdma.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index c73e610416..62293ab1d6 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3253,10 +3253,6 @@ static int qemu_rdma_save_page(QEMUFile *f, ram_addr_t block_offset, RDMAContext *rdma; int ret; - if (migration_in_postcopy()) { - return RAM_SAVE_CONTROL_NOT_SUPP; - } - RCU_READ_LOCK_GUARD(); rdma = qatomic_rcu_read(&rioc->rdmaout); @@ -3327,7 +3323,7 @@ err: int rdma_control_save_page(QEMUFile *f, ram_addr_t block_offset, ram_addr_t offset, size_t size) { - if (!migrate_rdma()) { + if (!migrate_rdma() || migration_in_postcopy()) { return RAM_SAVE_CONTROL_NOT_SUPP; }