From patchwork Fri Nov 10 20:02:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1862565 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=c0+KaYSB; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=DWgoLPsw; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SRqXr1Kr0z1yRZ for ; Sat, 11 Nov 2023 07:04:28 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r1XiR-00021r-Jk; Fri, 10 Nov 2023 15:02:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r1XiM-0001yK-0n for qemu-devel@nongnu.org; Fri, 10 Nov 2023 15:02:52 -0500 Received: from smtp-out2.suse.de ([195.135.220.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1r1XiK-0004cS-Bm for qemu-devel@nongnu.org; Fri, 10 Nov 2023 15:02:49 -0500 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 0197C1F8C0; Fri, 10 Nov 2023 20:02:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1699646567; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tmwd7iHU0NMBiDqCHDKHLq00DyPOwq8NnlmxuEjsLMc=; b=c0+KaYSBTbP0WIPIdh8/fSjGJONOiF92amDY0rFZNXMrS0dQT5lb/Yj7Q4EfZvWKl8bZGW f/MjBhXAHb2gUXOTJAg2TYjt9XgtBCBrFugbSzORVUQyI9tgf0/TOxwkZGfWb/MsshHMoP PHkxfi/TIqpwZSDSrNOh7SksaOVy2C8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1699646567; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tmwd7iHU0NMBiDqCHDKHLq00DyPOwq8NnlmxuEjsLMc=; b=DWgoLPswpP577C0T1kGrAMwC4spNHL62cOEtHeSm5aVQeYFODcr5VUTWbLgsnyBeylskGM w54tCF+EZdR/naCw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 74481138FC; Fri, 10 Nov 2023 20:02:45 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id ACsBEGWMTmWiaQAAMHmgww (envelope-from ); Fri, 10 Nov 2023 20:02:45 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Juan Quintela , Peter Xu , Leonardo Bras , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [RFC PATCH v2 1/4] migration/multifd: Stop setting p->ioc before connecting Date: Fri, 10 Nov 2023 17:02:38 -0300 Message-Id: <20231110200241.20679-2-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231110200241.20679-1-farosas@suse.de> References: <20231110200241.20679-1-farosas@suse.de> MIME-Version: 1.0 Received-SPF: pass client-ip=195.135.220.29; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org This is being shadowed but the assignments at multifd_channel_connect() and multifd_tls_channel_connect() . Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu Reviewed-by: Juan Quintela --- migration/multifd.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index ec58c58082..409460684f 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -883,8 +883,7 @@ static void multifd_new_send_channel_async(QIOTask *task, gpointer opaque) trace_multifd_new_send_channel_async(p->id); if (!qio_task_propagate_error(task, &local_err)) { - p->c = ioc; - qio_channel_set_delay(p->c, false); + qio_channel_set_delay(ioc, false); p->running = true; if (multifd_channel_connect(p, ioc, &local_err)) { return; From patchwork Fri Nov 10 20:02:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1862564 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=MnZV9Kop; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=cjarW7Dp; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SRqXq0Y0gz1yRV for ; Sat, 11 Nov 2023 07:04:27 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r1XiR-00021c-FX; Fri, 10 Nov 2023 15:02:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r1XiP-0001zI-GV for qemu-devel@nongnu.org; Fri, 10 Nov 2023 15:02:53 -0500 Received: from smtp-out1.suse.de ([2001:67c:2178:6::1c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1r1XiN-0004db-U4 for qemu-devel@nongnu.org; Fri, 10 Nov 2023 15:02:53 -0500 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id E3E4D219B2; Fri, 10 Nov 2023 20:02:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1699646568; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1XDw7h+MPjEb3Fw95R1zEtbmr3eWUmHUulSWsjufGZg=; b=MnZV9KopVQQ8ipOudvugtx6AXDetVFvHUliIaAczRz9fMuXcSy6xFLWsVEqDXm2j0lxKfn 7eL+SgeelUSbxiliguUovU0IePsfL6BahSi3oHTxJQzliohJeiWMQaBLh74zowVvEgdU4e WivfZDdYEyfsj4FxTUe9cYXP8aBobGw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1699646568; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1XDw7h+MPjEb3Fw95R1zEtbmr3eWUmHUulSWsjufGZg=; b=cjarW7DpZ5pbL1kPd1h/uNQvcxpvuZJynoNkdW35SQ1nk/hsDCKVQXhbbmujbfGYndsAVm qgfYgfvfN95kSFDw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 6D801138FC; Fri, 10 Nov 2023 20:02:47 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id wKlGDmeMTmWiaQAAMHmgww (envelope-from ); Fri, 10 Nov 2023 20:02:47 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Juan Quintela , Peter Xu , Leonardo Bras , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [RFC PATCH v2 2/4] migration/multifd: Join the TLS thread Date: Fri, 10 Nov 2023 17:02:39 -0300 Message-Id: <20231110200241.20679-3-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231110200241.20679-1-farosas@suse.de> References: <20231110200241.20679-1-farosas@suse.de> MIME-Version: 1.0 Received-SPF: pass client-ip=2001:67c:2178:6::1c; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org We're currently leaking the resources of the TLS thread by not joining it and also overwriting the p->thread pointer altogether. Signed-off-by: Fabiano Rosas --- migration/multifd.c | 10 +++++++++- migration/multifd.h | 1 + 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/migration/multifd.c b/migration/multifd.c index 409460684f..d632dbc095 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -525,6 +525,10 @@ void multifd_save_cleanup(void) for (i = 0; i < migrate_multifd_channels(); i++) { MultiFDSendParams *p = &multifd_send_state->params[i]; + if (p->tls_thread) { + qemu_thread_join(p->tls_thread); + } + if (p->running) { qemu_thread_join(&p->thread); } @@ -552,6 +556,8 @@ void multifd_save_cleanup(void) p->iov = NULL; g_free(p->normal); p->normal = NULL; + g_free(p->tls_thread); + p->tls_thread = NULL; multifd_send_state->ops->send_cleanup(p, &local_err); if (local_err) { migrate_set_error(migrate_get_current(), local_err); @@ -826,7 +832,9 @@ static bool multifd_tls_channel_connect(MultiFDSendParams *p, trace_multifd_tls_outgoing_handshake_start(ioc, tioc, hostname); qio_channel_set_name(QIO_CHANNEL(tioc), "multifd-tls-outgoing"); p->c = QIO_CHANNEL(tioc); - qemu_thread_create(&p->thread, "multifd-tls-handshake-worker", + + p->tls_thread = g_new0(QemuThread, 1); + qemu_thread_create(p->tls_thread, "multifd-tls-handshake-worker", multifd_tls_handshake_thread, p, QEMU_THREAD_JOINABLE); return true; diff --git a/migration/multifd.h b/migration/multifd.h index a835643b48..4ff78e9863 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -75,6 +75,7 @@ typedef struct { char *name; /* channel thread id */ QemuThread thread; + QemuThread *tls_thread; /* communication channel */ QIOChannel *c; /* is the yank function registered */ From patchwork Fri Nov 10 20:02:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1862566 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=1kuBlZ9Y; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=v7gdYNxx; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SRqXs5424z1yRV for ; Sat, 11 Nov 2023 07:04:29 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r1XiR-00021X-0n; Fri, 10 Nov 2023 15:02:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r1XiQ-00021L-Ec for qemu-devel@nongnu.org; Fri, 10 Nov 2023 15:02:54 -0500 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1r1XiO-0004dx-Hb for qemu-devel@nongnu.org; Fri, 10 Nov 2023 15:02:54 -0500 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id CAA99219B5; Fri, 10 Nov 2023 20:02:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1699646570; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2aI0UloZRongbOYJrsT9CxfT8577Scr/Q6hhR4hYmDg=; b=1kuBlZ9Y9HzkyVNUFyEYxG7+ycQLcHFmeNMuGnOvyKrZDU6kk0psmPR19C9Nh4GKXU7SHY +LDouoiRvaqjOaYn5uUZ4kbaqdVFXi8H2gAKrLyq81DL5rqyy0iW8Wcn7l3Nhg5xYOQ2lW ESiJBjG8jsAS9ykNgnrcoQs+2w6J+EA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1699646570; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2aI0UloZRongbOYJrsT9CxfT8577Scr/Q6hhR4hYmDg=; b=v7gdYNxxqKVpED7XNDTbesahpiaH4VcqvHkXKCKVNVaNfv6JYTDVITa1S5ZuYvNgrOAmX4 pVbLQoH733OKijCQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 596F3138FC; Fri, 10 Nov 2023 20:02:49 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 0Ml6CWmMTmWiaQAAMHmgww (envelope-from ); Fri, 10 Nov 2023 20:02:49 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Juan Quintela , Peter Xu , Leonardo Bras , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [RFC PATCH v2 3/4] migration/multifd: Remove p->running Date: Fri, 10 Nov 2023 17:02:40 -0300 Message-Id: <20231110200241.20679-4-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231110200241.20679-1-farosas@suse.de> References: <20231110200241.20679-1-farosas@suse.de> MIME-Version: 1.0 Received-SPF: pass client-ip=195.135.220.28; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org We currently only need p->running to avoid calling qemu_thread_join() on a non existent thread if the thread has never been created. We could turn the QemuThread into a pointer and check for NULL instead and get rid of the p->running flag. Testing the pointer directly is more precise and less prone to misuse. Signed-off-by: Fabiano Rosas --- migration/multifd.c | 41 ++++++++++++++++++++--------------------- migration/multifd.h | 8 ++------ 2 files changed, 22 insertions(+), 27 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index d632dbc095..639505dd10 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -529,8 +529,8 @@ void multifd_save_cleanup(void) qemu_thread_join(p->tls_thread); } - if (p->running) { - qemu_thread_join(&p->thread); + if (p->thread) { + qemu_thread_join(p->thread); } } for (i = 0; i < migrate_multifd_channels(); i++) { @@ -558,6 +558,8 @@ void multifd_save_cleanup(void) p->normal = NULL; g_free(p->tls_thread); p->tls_thread = NULL; + g_free(p->thread); + p->thread = NULL; multifd_send_state->ops->send_cleanup(p, &local_err); if (local_err) { migrate_set_error(migrate_get_current(), local_err); @@ -762,10 +764,6 @@ out: error_free(local_err); } - qemu_mutex_lock(&p->mutex); - p->running = false; - qemu_mutex_unlock(&p->mutex); - rcu_unregister_thread(); migration_threads_remove(thread); trace_multifd_send_thread_end(p->id, p->num_packets, p->total_normal_pages); @@ -860,7 +858,9 @@ static bool multifd_channel_connect(MultiFDSendParams *p, migration_ioc_register_yank(ioc); p->registered_yank = true; p->c = ioc; - qemu_thread_create(&p->thread, p->name, multifd_send_thread, p, + + p->thread = g_new0(QemuThread, 1); + qemu_thread_create(p->thread, p->name, multifd_send_thread, p, QEMU_THREAD_JOINABLE); } return true; @@ -892,7 +892,6 @@ static void multifd_new_send_channel_async(QIOTask *task, gpointer opaque) trace_multifd_new_send_channel_async(p->id); if (!qio_task_propagate_error(task, &local_err)) { qio_channel_set_delay(ioc, false); - p->running = true; if (multifd_channel_connect(p, ioc, &local_err)) { return; } @@ -1034,15 +1033,15 @@ void multifd_load_cleanup(void) for (i = 0; i < migrate_multifd_channels(); i++) { MultiFDRecvParams *p = &multifd_recv_state->params[i]; - if (p->running) { - /* - * multifd_recv_thread may hung at MULTIFD_FLAG_SYNC handle code, - * however try to wakeup it without harm in cleanup phase. - */ - qemu_sem_post(&p->sem_sync); - } + /* + * multifd_recv_thread may hung at MULTIFD_FLAG_SYNC handle code, + * however try to wakeup it without harm in cleanup phase. + */ + qemu_sem_post(&p->sem_sync); - qemu_thread_join(&p->thread); + if (p->thread) { + qemu_thread_join(p->thread); + } } for (i = 0; i < migrate_multifd_channels(); i++) { MultiFDRecvParams *p = &multifd_recv_state->params[i]; @@ -1061,6 +1060,8 @@ void multifd_load_cleanup(void) p->iov = NULL; g_free(p->normal); p->normal = NULL; + g_free(p->thread); + p->thread = NULL; multifd_recv_state->ops->recv_cleanup(p); } qemu_sem_destroy(&multifd_recv_state->sem_sync); @@ -1152,9 +1153,6 @@ static void *multifd_recv_thread(void *opaque) multifd_recv_terminate_threads(local_err); error_free(local_err); } - qemu_mutex_lock(&p->mutex); - p->running = false; - qemu_mutex_unlock(&p->mutex); rcu_unregister_thread(); trace_multifd_recv_thread_end(p->id, p->num_packets, p->total_normal_pages); @@ -1198,6 +1196,7 @@ int multifd_load_setup(Error **errp) p->normal = g_new0(ram_addr_t, page_count); p->page_count = page_count; p->page_size = qemu_target_page_size(); + p->thread = g_new0(QemuThread, 1); } for (i = 0; i < thread_count; i++) { @@ -1264,8 +1263,8 @@ void multifd_recv_new_channel(QIOChannel *ioc, Error **errp) /* initial packet */ p->num_packets = 1; - p->running = true; - qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p, + p->thread = g_new0(QemuThread, 1); + qemu_thread_create(p->thread, p->name, multifd_recv_thread, p, QEMU_THREAD_JOINABLE); qatomic_inc(&multifd_recv_state->count); } diff --git a/migration/multifd.h b/migration/multifd.h index 4ff78e9863..d21edaeaae 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -74,7 +74,7 @@ typedef struct { /* channel thread name */ char *name; /* channel thread id */ - QemuThread thread; + QemuThread *thread; QemuThread *tls_thread; /* communication channel */ QIOChannel *c; @@ -96,8 +96,6 @@ typedef struct { /* this mutex protects the following parameters */ QemuMutex mutex; - /* is this channel thread running */ - bool running; /* should this thread finish */ bool quit; /* multifd flags for each packet */ @@ -144,7 +142,7 @@ typedef struct { /* channel thread name */ char *name; /* channel thread id */ - QemuThread thread; + QemuThread *thread; /* communication channel */ QIOChannel *c; /* packet allocated len */ @@ -159,8 +157,6 @@ typedef struct { /* this mutex protects the following parameters */ QemuMutex mutex; - /* is this channel thread running */ - bool running; /* should this thread finish */ bool quit; /* multifd flags for each packet */ From patchwork Fri Nov 10 20:02:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 1862562 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=suse.de header.i=@suse.de header.a=rsa-sha256 header.s=susede2_rsa header.b=DnLRJA7l; dkim=pass header.d=suse.de header.i=@suse.de header.a=ed25519-sha256 header.s=susede2_ed25519 header.b=Pg4P5eoW; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SRqXH2Rk5z1yRV for ; Sat, 11 Nov 2023 07:03:59 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r1XiS-00022g-Q3; Fri, 10 Nov 2023 15:02:56 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r1XiR-00021g-GR for qemu-devel@nongnu.org; Fri, 10 Nov 2023 15:02:55 -0500 Received: from smtp-out1.suse.de ([2001:67c:2178:6::1c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1r1XiP-0004eF-Q0 for qemu-devel@nongnu.org; Fri, 10 Nov 2023 15:02:55 -0500 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id B1B90219B9; Fri, 10 Nov 2023 20:02:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1699646572; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nZpA8N+9GDsiJN8mA/lqa+vyWaYNZiYwdvx4LS9shsI=; b=DnLRJA7lJeYRAGRCryl/YINUyslowVAtl8W9iS5WV1257n5LHQv3NU2Nieyfo44PjIOC6t 4+aGZTGmKXIUsw54khfOyHqV6a/95SAb6MZDPxX7c5R2dIFHnaRjhTAMgmKQrB2XO1qJM7 59i/E0wAKmwSFyMFHosRDrxWNmz8PqQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1699646572; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nZpA8N+9GDsiJN8mA/lqa+vyWaYNZiYwdvx4LS9shsI=; b=Pg4P5eoWEbOtkBObHmxf8upiOhi3lpE87/ehNkPNXFNBYxRiKjmsu3Av8WpJOQ5RUyHE1B Txbmb2EbOA66XPBA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 40BFD138FC; Fri, 10 Nov 2023 20:02:51 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 6D2LA2uMTmWiaQAAMHmgww (envelope-from ); Fri, 10 Nov 2023 20:02:51 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Juan Quintela , Peter Xu , Leonardo Bras , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [RFC PATCH v2 4/4] migration/multifd: Move semaphore release into main thread Date: Fri, 10 Nov 2023 17:02:41 -0300 Message-Id: <20231110200241.20679-5-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20231110200241.20679-1-farosas@suse.de> References: <20231110200241.20679-1-farosas@suse.de> MIME-Version: 1.0 Received-SPF: pass client-ip=2001:67c:2178:6::1c; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org We cannot operate on the multifd semaphores outside of the multifd channel thread because multifd_save_cleanup() can run in parallel and attempt to destroy the mutexes, which causes an assert. Looking at the places where we use the semaphores aside from the migration thread, there's only the TLS handshake thread and the initial multifd_channel_connect() in the main thread. These are places where creating the multifd channel cannot fail, so releasing the semaphores at these places only serves the purpose of avoiding a deadlock when an error happens before the channel(s) have started, but after the migration thread has already called multifd_send_sync_main(). Instead of attempting to release the semaphores at those places, move the release into multifd_save_cleanup(). This puts the semaphore usage in the same thread that does the cleanup, eliminating the race. Move the call to multifd_save_cleanup() before joining the migration thread so we release the semaphores before and avoid deadlock. Signed-off-by: Fabiano Rosas --- migration/migration.c | 4 +++- migration/multifd.c | 33 +++++++++++++++++---------------- 2 files changed, 20 insertions(+), 17 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 28a34c9068..5a07f5318d 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1293,6 +1293,9 @@ static void migrate_fd_cleanup(MigrationState *s) QEMUFile *tmp; trace_migrate_fd_cleanup(); + + multifd_save_cleanup(); + qemu_mutex_unlock_iothread(); if (s->migration_thread_running) { qemu_thread_join(&s->thread); @@ -1300,7 +1303,6 @@ static void migrate_fd_cleanup(MigrationState *s) } qemu_mutex_lock_iothread(); - multifd_save_cleanup(); qemu_mutex_lock(&s->qemu_file_lock); tmp = s->to_dst_file; s->to_dst_file = NULL; diff --git a/migration/multifd.c b/migration/multifd.c index 639505dd10..a66be2d4e4 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -531,6 +531,15 @@ void multifd_save_cleanup(void) if (p->thread) { qemu_thread_join(p->thread); + } else { + /* + * The migration thread might be waiting on these. It is + * the responsibility of the channel thread to release + * them, unless it has never been created, then do it + * here. + */ + qemu_sem_post(&p->sem_sync); + qemu_sem_post(&multifd_send_state->channels_ready); } } for (i = 0; i < migrate_multifd_channels(); i++) { @@ -784,20 +793,15 @@ static void multifd_tls_outgoing_handshake(QIOTask *task, if (!qio_task_propagate_error(task, &err)) { trace_multifd_tls_outgoing_handshake_complete(ioc); - if (multifd_channel_connect(p, ioc, &err)) { - return; - } + multifd_channel_connect(p, ioc, &error_abort); + } else { + /* + * The multifd client could already be waiting to queue data, + * so let it know that we didn't even start. + */ + p->quit = true; + trace_multifd_tls_outgoing_handshake_error(ioc, error_get_pretty(err)); } - - trace_multifd_tls_outgoing_handshake_error(ioc, error_get_pretty(err)); - - /* - * Error happen, mark multifd_send_thread status as 'quit' although it - * is not created, and then tell who pay attention to me. - */ - p->quit = true; - qemu_sem_post(&multifd_send_state->channels_ready); - qemu_sem_post(&p->sem_sync); } static void *multifd_tls_handshake_thread(void *opaque) @@ -870,9 +874,6 @@ static void multifd_new_send_channel_cleanup(MultiFDSendParams *p, QIOChannel *ioc, Error *err) { migrate_set_error(migrate_get_current(), err); - /* Error happen, we need to tell who pay attention to me */ - qemu_sem_post(&multifd_send_state->channels_ready); - qemu_sem_post(&p->sem_sync); /* * Although multifd_send_thread is not created, but main migration * thread need to judge whether it is running, so we need to mark