From patchwork Thu Oct 19 11:07:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1851569 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=foxdTknu; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SB4hB0tcMz20cx for ; Thu, 19 Oct 2023 22:08:10 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtQsX-0001b3-Gu; Thu, 19 Oct 2023 07:07:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsK-0001Sx-0j for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsG-0000ds-1p for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1697713650; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s0BIRYjUYDy1atH1tMuQopw9uVCAw1+EC8yKtTqEUFM=; b=foxdTknuhFqcnZ7Eyikc84zTUebmENeQJ5atXCeqizmiBeYlIExZLvdJsIVdKQyat1HImk IkcrstBanDPGS6Vp7dCetZ1s2rnpE0xq520pJxWdiq2Fm9xvtvMGqsrOUJ9pGP5VblvOQm J2DRBdOBAxVR/ZYrWFCSbAXRsjWRBSQ= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-640-JRQ5zd1RPmyNciEJWkSkDQ-1; Thu, 19 Oct 2023 07:07:28 -0400 X-MC-Unique: JRQ5zd1RPmyNciEJWkSkDQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 966DE10201E5; Thu, 19 Oct 2023 11:07:28 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id B42C725C8; Thu, 19 Oct 2023 11:07:27 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Leonardo Bras , Peter Xu , Juan Quintela Subject: [PATCH v2 01/11] migration: Give one error if trying to set MULTIFD and XBZRLE Date: Thu, 19 Oct 2023 13:07:14 +0200 Message-ID: <20231019110724.15324-2-quintela@redhat.com> In-Reply-To: <20231019110724.15324-1-quintela@redhat.com> References: <20231019110724.15324-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Signed-off-by: Juan Quintela Reviewed-by: Fabiano Rosas --- migration/options.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/migration/options.c b/migration/options.c index 42fb818956..b8c3c3218d 100644 --- a/migration/options.c +++ b/migration/options.c @@ -618,6 +618,13 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) } } + if (new_caps[MIGRATION_CAPABILITY_MULTIFD]) { + if (new_caps[MIGRATION_CAPABILITY_XBZRLE]) { + error_setg(errp, "Multifd is not compatible with xbzrle"); + return false; + } + } + return true; } From patchwork Thu Oct 19 11:07:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1851570 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=hQo9mDkq; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SB4hR5Rgkz20cx for ; Thu, 19 Oct 2023 22:08:23 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtQsY-0001j1-Au; Thu, 19 Oct 2023 07:07:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsN-0001TL-Jz for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsG-0000dw-EZ for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1697713651; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=akHeKG8JX+TQnhjrZyHBBORskgZE4lxHIWZJHAHEYkk=; b=hQo9mDkqANh29B08Y90ET2Ux7lUKIdLqn4YWEYzPhlZ+jk7EETTpSZBmojkALGNOuqCLah OkeuyI9EwDXgl1h8UAOAfsoyn/IM/uqv+7mBeTHD2fLjznH3O2hz6p/jmi208mjgeFZFpF ZfE0oaJU/ktwvk8sKIKhjyrDSf3u8k8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-455-OJUx1Q19Ni2C3Gr6dCejlA-1; Thu, 19 Oct 2023 07:07:30 -0400 X-MC-Unique: OJUx1Q19Ni2C3Gr6dCejlA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B7E7087A9F7; Thu, 19 Oct 2023 11:07:29 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id D6EBF25C8; Thu, 19 Oct 2023 11:07:28 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Leonardo Bras , Peter Xu , Juan Quintela Subject: [PATCH v2 02/11] migration: Give one error if trying to set COMPRESSION and XBZRLE Date: Thu, 19 Oct 2023 13:07:15 +0200 Message-ID: <20231019110724.15324-3-quintela@redhat.com> In-Reply-To: <20231019110724.15324-1-quintela@redhat.com> References: <20231019110724.15324-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Signed-off-by: Juan Quintela Reviewed-by: Fabiano Rosas --- migration/options.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/migration/options.c b/migration/options.c index b8c3c3218d..37fa1cfe74 100644 --- a/migration/options.c +++ b/migration/options.c @@ -625,6 +625,13 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) } } + if (new_caps[MIGRATION_CAPABILITY_COMPRESS]) { + if (new_caps[MIGRATION_CAPABILITY_XBZRLE]) { + error_setg(errp, "Compression is not compatible with xbzrle"); + return false; + } + } + return true; } From patchwork Thu Oct 19 11:07:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1851614 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=KXdSJS2s; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SB60l3yNyz20cx for ; Thu, 19 Oct 2023 23:07:35 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtQsZ-0001oQ-Go; Thu, 19 Oct 2023 07:07:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsO-0001TX-Qn for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:43 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsJ-0000e5-1X for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1697713652; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=99Xo9O3yZngmIzx0R6UKHpA7F+L8C7a6yZ1nNGOIyMo=; b=KXdSJS2sNcUYWhpgpsnJKfGX7yjw4anK2a7NzEsKuVT51Z91mMuRgJqx+jk6vU2d9mHROO oGQrpZy2IYbWLBM2DnyazwH9uvFljy9D9YnnEWZYxvVhYZU1oZmQYZVwEZKwTUJuZ/ZcNb loR1dTOoQi9UvOaNSoH1r6ZLh2O6OzY= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-522-_tRzXdtOO6y5O-lS8WPtlA-1; Thu, 19 Oct 2023 07:07:31 -0400 X-MC-Unique: _tRzXdtOO6y5O-lS8WPtlA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DCB6E1C06521; Thu, 19 Oct 2023 11:07:30 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0536525C8; Thu, 19 Oct 2023 11:07:29 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Leonardo Bras , Peter Xu , Juan Quintela Subject: [PATCH v2 03/11] migration: Remove save_page_use_compression() Date: Thu, 19 Oct 2023 13:07:16 +0200 Message-ID: <20231019110724.15324-4-quintela@redhat.com> In-Reply-To: <20231019110724.15324-1-quintela@redhat.com> References: <20231019110724.15324-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org After previous patch, we disable the posiblity that we use compression together with xbzrle. So we can use directly migrate_compress(). Once there, now we don't need the rs parameter, so remove it. Signed-off-by: Juan Quintela Reviewed-by: Fabiano Rosas --- migration/ram.c | 34 +++++++--------------------------- 1 file changed, 7 insertions(+), 27 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 16c30a9d7a..42b704ac40 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1291,8 +1291,6 @@ static int ram_save_multifd_page(QEMUFile *file, RAMBlock *block, return 1; } -static bool save_page_use_compression(RAMState *rs); - static int send_queued_data(CompressParam *param) { PageSearchStatus *pss = &ram_state->pss[RAM_CHANNEL_PRECOPY]; @@ -1329,9 +1327,9 @@ static int send_queued_data(CompressParam *param) return len; } -static void ram_flush_compressed_data(RAMState *rs) +static void ram_flush_compressed_data(void) { - if (!save_page_use_compression(rs)) { + if (!migrate_compress()) { return; } @@ -1393,7 +1391,7 @@ static int find_dirty_block(RAMState *rs, PageSearchStatus *pss) * Also If xbzrle is on, stop using the data compression at this * point. In theory, xbzrle can do better than compression. */ - ram_flush_compressed_data(rs); + ram_flush_compressed_data(); /* Hit the end of the list */ pss->block = QLIST_FIRST_RCU(&ram_list.blocks); @@ -2042,24 +2040,6 @@ int ram_save_queue_pages(const char *rbname, ram_addr_t start, ram_addr_t len) return 0; } -static bool save_page_use_compression(RAMState *rs) -{ - if (!migrate_compress()) { - return false; - } - - /* - * If xbzrle is enabled (e.g., after first round of migration), stop - * using the data compression. In theory, xbzrle can do better than - * compression. - */ - if (rs->xbzrle_started) { - return false; - } - - return true; -} - /* * try to compress the page before posting it out, return true if the page * has been properly handled by compression, otherwise needs other @@ -2068,7 +2048,7 @@ static bool save_page_use_compression(RAMState *rs) static bool save_compress_page(RAMState *rs, PageSearchStatus *pss, ram_addr_t offset) { - if (!save_page_use_compression(rs)) { + if (!migrate_compress()) { return false; } @@ -2083,7 +2063,7 @@ static bool save_compress_page(RAMState *rs, PageSearchStatus *pss, * much CPU resource. */ if (pss->block != pss->last_sent_block) { - ram_flush_compressed_data(rs); + ram_flush_compressed_data(); return false; } @@ -3135,7 +3115,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) * page is sent in one chunk. */ if (migrate_postcopy_ram()) { - ram_flush_compressed_data(rs); + ram_flush_compressed_data(); } /* @@ -3236,7 +3216,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque) } qemu_mutex_unlock(&rs->bitmap_mutex); - ram_flush_compressed_data(rs); + ram_flush_compressed_data(); int ret = rdma_registration_stop(f, RAM_CONTROL_FINISH); if (ret < 0) { From patchwork Thu Oct 19 11:07:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1851590 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=KuDpV9u3; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SB5P65cp0z20Zj for ; Thu, 19 Oct 2023 22:40:10 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtQse-0001v3-H5; Thu, 19 Oct 2023 07:07:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsc-0001sD-73 for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:55 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsa-0000iN-KW for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1697713671; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NtlaEy+a5gWfE0TGKB2llmczW7hoEesHFsw/QDYHiY4=; b=KuDpV9u3bEfQMBxoIXd71YPCoKheuh7NFLDUVRrcSc1WrgNffDqf8W2ZXn2HFagRCPN9QO p3mETtgvSV9g9JfGfFDtkEHbyyGQhsgk4L4MOpYTB/Ce91LC8xUXsF85vFvJ8vyxaHb1ht uITngx/fcgin+G5A3sfU/hgSrXgxkTk= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-101-Xi0H6dUHOiiTmNIDYYrxbg-1; Thu, 19 Oct 2023 07:07:32 -0400 X-MC-Unique: Xi0H6dUHOiiTmNIDYYrxbg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 430571C06510; Thu, 19 Oct 2023 11:07:32 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id 281BA25C8; Thu, 19 Oct 2023 11:07:31 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Leonardo Bras , Peter Xu , Juan Quintela , Lukas Straub Subject: [PATCH v2 04/11] migration: Make compress_data_with_multithreads return bool Date: Thu, 19 Oct 2023 13:07:17 +0200 Message-ID: <20231019110724.15324-5-quintela@redhat.com> In-Reply-To: <20231019110724.15324-1-quintela@redhat.com> References: <20231019110724.15324-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub Reviewed-by: Fabiano Rosas --- migration/ram-compress.h | 4 ++-- migration/ram-compress.c | 17 ++++++++++------- migration/ram.c | 3 +-- 3 files changed, 13 insertions(+), 11 deletions(-) diff --git a/migration/ram-compress.h b/migration/ram-compress.h index e55d3b50bd..b228640092 100644 --- a/migration/ram-compress.h +++ b/migration/ram-compress.h @@ -60,8 +60,8 @@ void compress_threads_save_cleanup(void); int compress_threads_save_setup(void); void flush_compressed_data(int (send_queued_data(CompressParam *))); -int compress_page_with_multi_thread(RAMBlock *block, ram_addr_t offset, - int (send_queued_data(CompressParam *))); +bool compress_page_with_multi_thread(RAMBlock *block, ram_addr_t offset, + int (send_queued_data(CompressParam *))); int wait_for_decompress_done(void); void compress_threads_load_cleanup(void); diff --git a/migration/ram-compress.c b/migration/ram-compress.c index d037dfe6cf..ef03d60a6d 100644 --- a/migration/ram-compress.c +++ b/migration/ram-compress.c @@ -260,10 +260,13 @@ static inline void set_compress_params(CompressParam *param, RAMBlock *block, param->trigger = true; } -int compress_page_with_multi_thread(RAMBlock *block, ram_addr_t offset, - int (send_queued_data(CompressParam *))) +/* + * Return true when it compress a page + */ +bool compress_page_with_multi_thread(RAMBlock *block, ram_addr_t offset, + int (send_queued_data(CompressParam *))) { - int thread_count, pages = -1; + int thread_count; bool wait = migrate_compress_wait_thread(); thread_count = migrate_compress_threads(); @@ -281,8 +284,8 @@ retry: qemu_cond_signal(¶m->cond); qemu_mutex_unlock(¶m->mutex); - pages = 1; - break; + qemu_mutex_unlock(&comp_done_lock); + return true; } } @@ -290,13 +293,13 @@ retry: * wait for the free thread if the user specifies 'compress-wait-thread', * otherwise we will post the page out in the main thread as normal page. */ - if (pages < 0 && wait) { + if (wait) { qemu_cond_wait(&comp_done_cond, &comp_done_lock); goto retry; } qemu_mutex_unlock(&comp_done_lock); - return pages; + return false; } /* return the size after decompression, or negative value on error */ diff --git a/migration/ram.c b/migration/ram.c index 42b704ac40..de7878c46e 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2067,8 +2067,7 @@ static bool save_compress_page(RAMState *rs, PageSearchStatus *pss, return false; } - if (compress_page_with_multi_thread(pss->block, offset, - send_queued_data) > 0) { + if (compress_page_with_multi_thread(pss->block, offset, send_queued_data)) { return true; } From patchwork Thu Oct 19 11:07:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1851615 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=WjsdOcoO; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SB66x2w55z20Zj for ; Thu, 19 Oct 2023 23:12:57 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtQsZ-0001p0-Pu; Thu, 19 Oct 2023 07:07:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsU-0001WH-Ob for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:47 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsN-0000eg-8g for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1697713657; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=L1OZ4CA3EfoO0cVtF38vD3HoFk/OaklAIFwd5ScwIB4=; b=WjsdOcoOziOjnzoFpuJmHIEzLZUi27vwvNekozgkgpyaim8DtiO/R8H51vM19e1Uj9/YS0 7kZA2sunK/2gi0QQXQEzPoCxAFuOIFY0SeaYd/86HlhMYk2AtWxw/FVxhHcxG34a1ANIse JC1Iy5pYjK006UmTBSpONwj/1tHFgfQ= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-25-lFe9YGwXPiqXqrEINALx_w-1; Thu, 19 Oct 2023 07:07:33 -0400 X-MC-Unique: lFe9YGwXPiqXqrEINALx_w-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 84EFB869ECF; Thu, 19 Oct 2023 11:07:33 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id 834808B9; Thu, 19 Oct 2023 11:07:32 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Leonardo Bras , Peter Xu , Juan Quintela , Lukas Straub Subject: [PATCH v2 05/11] migration: Simplify compress_page_with_multithread() Date: Thu, 19 Oct 2023 13:07:18 +0200 Message-ID: <20231019110724.15324-6-quintela@redhat.com> In-Reply-To: <20231019110724.15324-1-quintela@redhat.com> References: <20231019110724.15324-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Move the goto to a while true. Reviewed-by: Lukas Straub Signed-off-by: Juan Quintela Reviewed-by: Fabiano Rosas --- migration/ram-compress.c | 48 ++++++++++++++++++++-------------------- 1 file changed, 24 insertions(+), 24 deletions(-) diff --git a/migration/ram-compress.c b/migration/ram-compress.c index ef03d60a6d..a991b15b7a 100644 --- a/migration/ram-compress.c +++ b/migration/ram-compress.c @@ -271,35 +271,35 @@ bool compress_page_with_multi_thread(RAMBlock *block, ram_addr_t offset, thread_count = migrate_compress_threads(); qemu_mutex_lock(&comp_done_lock); -retry: - for (int i = 0; i < thread_count; i++) { - if (comp_param[i].done) { - CompressParam *param = &comp_param[i]; - qemu_mutex_lock(¶m->mutex); - param->done = false; - send_queued_data(param); - assert(qemu_file_buffer_empty(param->file)); - compress_reset_result(param); - set_compress_params(param, block, offset); - qemu_cond_signal(¶m->cond); - qemu_mutex_unlock(¶m->mutex); + while (true) { + for (int i = 0; i < thread_count; i++) { + if (comp_param[i].done) { + CompressParam *param = &comp_param[i]; + qemu_mutex_lock(¶m->mutex); + param->done = false; + send_queued_data(param); + assert(qemu_file_buffer_empty(param->file)); + compress_reset_result(param); + set_compress_params(param, block, offset); + + qemu_cond_signal(¶m->cond); + qemu_mutex_unlock(¶m->mutex); + qemu_mutex_unlock(&comp_done_lock); + return true; + } + } + if (!wait) { qemu_mutex_unlock(&comp_done_lock); - return true; + return false; } - } - - /* - * wait for the free thread if the user specifies 'compress-wait-thread', - * otherwise we will post the page out in the main thread as normal page. - */ - if (wait) { + /* + * wait for a free thread if the user specifies + * 'compress-wait-thread', otherwise we will post the page out + * in the main thread as normal page. + */ qemu_cond_wait(&comp_done_cond, &comp_done_lock); - goto retry; } - qemu_mutex_unlock(&comp_done_lock); - - return false; } /* return the size after decompression, or negative value on error */ From patchwork Thu Oct 19 11:07:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1851617 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=JHeLOKf2; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SB6DK321lz20cx for ; Thu, 19 Oct 2023 23:17:37 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtQsd-0001t2-GZ; Thu, 19 Oct 2023 07:07:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsV-0001Z7-HM for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:47 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsQ-0000ey-DW for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1697713659; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PLlIdJkdIqQe1aXcLmosz1ae+dtpSUozbxpric+R6yo=; b=JHeLOKf2UEYP4XkXUq+E3C2pXifowsrH4ynAYWrZDFrdB8uPLiA7lYvTyDw0dvSNcNVheq ABOIn2n9/7LUaN+lEYP5b5VJBC2WR44+D3YjzcRRLKajroqtbf79ysLFpA0SdqZBzpKM0g dZ6fxvJClG0+ux9ycLn9TReZpvrw9GE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-646-67rZLPJFOmKKhW9lli-KNQ-1; Thu, 19 Oct 2023 07:07:35 -0400 X-MC-Unique: 67rZLPJFOmKKhW9lli-KNQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C71A4857A91; Thu, 19 Oct 2023 11:07:34 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id C45AD25C8; Thu, 19 Oct 2023 11:07:33 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Leonardo Bras , Peter Xu , Juan Quintela , Lukas Straub Subject: [PATCH v2 06/11] migration: Move busy++ to migrate_with_multithread Date: Thu, 19 Oct 2023 13:07:19 +0200 Message-ID: <20231019110724.15324-7-quintela@redhat.com> In-Reply-To: <20231019110724.15324-1-quintela@redhat.com> References: <20231019110724.15324-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org And now we can simplify save_compress_page(). Reviewed-by: Lukas Straub Signed-off-by: Juan Quintela Reviewed-by: Fabiano Rosas --- migration/ram-compress.c | 1 + migration/ram.c | 8 ++------ 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/migration/ram-compress.c b/migration/ram-compress.c index a991b15b7a..f56e1f8e69 100644 --- a/migration/ram-compress.c +++ b/migration/ram-compress.c @@ -291,6 +291,7 @@ bool compress_page_with_multi_thread(RAMBlock *block, ram_addr_t offset, } if (!wait) { qemu_mutex_unlock(&comp_done_lock); + compression_counters.busy++; return false; } /* diff --git a/migration/ram.c b/migration/ram.c index de7878c46e..a3c5fcc549 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2067,12 +2067,8 @@ static bool save_compress_page(RAMState *rs, PageSearchStatus *pss, return false; } - if (compress_page_with_multi_thread(pss->block, offset, send_queued_data)) { - return true; - } - - compression_counters.busy++; - return false; + return compress_page_with_multi_thread(pss->block, offset, + send_queued_data); } /** From patchwork Thu Oct 19 11:07:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1851589 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=NFv8gxfx; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SB5Lk5zwGz20cx for ; Thu, 19 Oct 2023 22:38:06 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtQsx-0002Zx-Ab; Thu, 19 Oct 2023 07:08:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsf-0001vG-EF for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:57 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsd-0000ij-7v for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1697713674; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uIqDoW9WVr1jmeZoP05oTbHh5fAuu+Ousk16Y6f/NmU=; b=NFv8gxfxADYBHoGin8Y04FXAq7JUYCnNRU2rhMF39qI/ZltMMQBD+O1b4LM2DdB26Sv6Hv SZ4zlbZ5kvxllGiUzZ6jK+m60WerIXIk8FO35ILcZzVqs2ZpuuewLS7OVg9N5mn5uRLplJ 94J3fRaH/UdLUA02P4OvDenCN1jXiYM= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-665-khxqajb4MMWanr7yZ9tzgQ-1; Thu, 19 Oct 2023 07:07:36 -0400 X-MC-Unique: khxqajb4MMWanr7yZ9tzgQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1B2B63804500; Thu, 19 Oct 2023 11:07:36 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id 15A6225C8; Thu, 19 Oct 2023 11:07:34 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Leonardo Bras , Peter Xu , Juan Quintela , Lukas Straub Subject: [PATCH v2 07/11] migration: Create compress_update_rates() Date: Thu, 19 Oct 2023 13:07:20 +0200 Message-ID: <20231019110724.15324-8-quintela@redhat.com> In-Reply-To: <20231019110724.15324-1-quintela@redhat.com> References: <20231019110724.15324-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org So we can move more compression_counters stuff to ram-compress.c. Create compression_counters struct to add the stuff that was on MigrationState. Reviewed-by: Lukas Straub Signed-off-by: Juan Quintela Reviewed-by: Fabiano Rosas --- migration/ram-compress.h | 1 + migration/ram.h | 1 - migration/ram-compress.c | 42 +++++++++++++++++++++++++++++++++++++++- migration/ram.c | 29 +-------------------------- 4 files changed, 43 insertions(+), 30 deletions(-) diff --git a/migration/ram-compress.h b/migration/ram-compress.h index b228640092..76dacd3ec7 100644 --- a/migration/ram-compress.h +++ b/migration/ram-compress.h @@ -71,5 +71,6 @@ void decompress_data_with_multi_threads(QEMUFile *f, void *host, int len); void populate_compress(MigrationInfo *info); uint64_t ram_compressed_pages(void); void update_compress_thread_counts(const CompressParam *param, int bytes_xmit); +void compress_update_rates(uint64_t page_count); #endif diff --git a/migration/ram.h b/migration/ram.h index 145c915ca7..9cf8f58e97 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -34,7 +34,6 @@ #include "io/channel.h" extern XBZRLECacheStats xbzrle_counters; -extern CompressionStats compression_counters; /* Should be holding either ram_list.mutex, or the RCU lock. */ #define RAMBLOCK_FOREACH_NOT_IGNORED(block) \ diff --git a/migration/ram-compress.c b/migration/ram-compress.c index f56e1f8e69..af42cab0fe 100644 --- a/migration/ram-compress.c +++ b/migration/ram-compress.c @@ -41,7 +41,20 @@ #include "ram.h" #include "migration-stats.h" -CompressionStats compression_counters; +static struct { + int64_t pages; + int64_t busy; + double busy_rate; + int64_t compressed_size; + double compression_rate; + /* compression statistics since the beginning of the period */ + /* amount of count that no free thread to compress data */ + uint64_t compress_thread_busy_prev; + /* amount bytes after compression */ + uint64_t compressed_size_prev; + /* amount of compressed pages */ + uint64_t compress_pages_prev; +} compression_counters; static CompressParam *comp_param; static QemuThread *compress_threads; @@ -518,3 +531,30 @@ void update_compress_thread_counts(const CompressParam *param, int bytes_xmit) compression_counters.pages++; } +void compress_update_rates(uint64_t page_count) +{ + if (!migrate_compress()) { + return; + } + compression_counters.busy_rate = (double)(compression_counters.busy - + compression_counters.compress_thread_busy_prev) / page_count; + compression_counters.compress_thread_busy_prev = + compression_counters.busy; + + double compressed_size = compression_counters.compressed_size - + compression_counters.compressed_size_prev; + if (compressed_size) { + double uncompressed_size = (compression_counters.pages - + compression_counters.compress_pages_prev) * + qemu_target_page_size(); + + /* Compression-Ratio = Uncompressed-size / Compressed-size */ + compression_counters.compression_rate = + uncompressed_size / compressed_size; + + compression_counters.compress_pages_prev = + compression_counters.pages; + compression_counters.compressed_size_prev = + compression_counters.compressed_size; + } +} diff --git a/migration/ram.c b/migration/ram.c index a3c5fcc549..bfb2f02351 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -369,13 +369,6 @@ struct RAMState { bool xbzrle_started; /* Are we on the last stage of migration */ bool last_stage; - /* compression statistics since the beginning of the period */ - /* amount of count that no free thread to compress data */ - uint64_t compress_thread_busy_prev; - /* amount bytes after compression */ - uint64_t compressed_size_prev; - /* amount of compressed pages */ - uint64_t compress_pages_prev; /* total handled target pages at the beginning of period */ uint64_t target_page_count_prev; @@ -945,7 +938,6 @@ uint64_t ram_get_total_transferred_pages(void) static void migration_update_rates(RAMState *rs, int64_t end_time) { uint64_t page_count = rs->target_page_count - rs->target_page_count_prev; - double compressed_size; /* calculate period counters */ stat64_set(&mig_stats.dirty_pages_rate, @@ -973,26 +965,7 @@ static void migration_update_rates(RAMState *rs, int64_t end_time) rs->xbzrle_pages_prev = xbzrle_counters.pages; rs->xbzrle_bytes_prev = xbzrle_counters.bytes; } - - if (migrate_compress()) { - compression_counters.busy_rate = (double)(compression_counters.busy - - rs->compress_thread_busy_prev) / page_count; - rs->compress_thread_busy_prev = compression_counters.busy; - - compressed_size = compression_counters.compressed_size - - rs->compressed_size_prev; - if (compressed_size) { - double uncompressed_size = (compression_counters.pages - - rs->compress_pages_prev) * TARGET_PAGE_SIZE; - - /* Compression-Ratio = Uncompressed-size / Compressed-size */ - compression_counters.compression_rate = - uncompressed_size / compressed_size; - - rs->compress_pages_prev = compression_counters.pages; - rs->compressed_size_prev = compression_counters.compressed_size; - } - } + compress_update_rates(page_count); } /* From patchwork Thu Oct 19 11:07:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1851609 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=VVjxuCLR; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SB5hX3qLqz20cx for ; Thu, 19 Oct 2023 22:53:32 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtQsh-0001vW-FX; Thu, 19 Oct 2023 07:07:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsV-0001Zz-Rd for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:48 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsS-0000er-SI for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1697713659; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zYMJDbOgdjA3oi9QzUh16drfKdEhythpDflUwlvEkPs=; b=VVjxuCLRREbZfY6n6MF4aNreOFGOVh/BGFX/vBEuJq3uwRwqItB+l0GXvM0b4NCb7oZUvf 27QtM6GtuPdwyhZ5+fyiQvf/TTFE8Q7Jhiv6fx+W7ed4Hk91b/KWLpyHKalDoNVKf86nag Lhz7vaTuDLflo/uBLac52ajuG1P7v6Y= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-28-seRrDuapOu-oGQ6doNvq0g-1; Thu, 19 Oct 2023 07:07:37 -0400 X-MC-Unique: seRrDuapOu-oGQ6doNvq0g-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3DB37185A790; Thu, 19 Oct 2023 11:07:37 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5C3F125C8; Thu, 19 Oct 2023 11:07:36 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Leonardo Bras , Peter Xu , Juan Quintela Subject: [PATCH v2 08/11] migration: Export send_queued_data() Date: Thu, 19 Oct 2023 13:07:21 +0200 Message-ID: <20231019110724.15324-9-quintela@redhat.com> In-Reply-To: <20231019110724.15324-1-quintela@redhat.com> References: <20231019110724.15324-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org This function is only used for compression. So we rename it as compress_send_queued_data(). We put it on ram-compress.h because we are moving it later to ram-compress.c. Signed-off-by: Juan Quintela Reviewed-by: Fabiano Rosas --- migration/ram-compress.h | 1 + migration/ram.c | 6 +++--- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/migration/ram-compress.h b/migration/ram-compress.h index 76dacd3ec7..636281ed97 100644 --- a/migration/ram-compress.h +++ b/migration/ram-compress.h @@ -72,5 +72,6 @@ void populate_compress(MigrationInfo *info); uint64_t ram_compressed_pages(void); void update_compress_thread_counts(const CompressParam *param, int bytes_xmit); void compress_update_rates(uint64_t page_count); +int compress_send_queued_data(CompressParam *param); #endif diff --git a/migration/ram.c b/migration/ram.c index bfb2f02351..d8b38ed999 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1264,7 +1264,7 @@ static int ram_save_multifd_page(QEMUFile *file, RAMBlock *block, return 1; } -static int send_queued_data(CompressParam *param) +int compress_send_queued_data(CompressParam *param) { PageSearchStatus *pss = &ram_state->pss[RAM_CHANNEL_PRECOPY]; MigrationState *ms = migrate_get_current(); @@ -1306,7 +1306,7 @@ static void ram_flush_compressed_data(void) return; } - flush_compressed_data(send_queued_data); + flush_compressed_data(compress_send_queued_data); } #define PAGE_ALL_CLEAN 0 @@ -2041,7 +2041,7 @@ static bool save_compress_page(RAMState *rs, PageSearchStatus *pss, } return compress_page_with_multi_thread(pss->block, offset, - send_queued_data); + compress_send_queued_data); } /** From patchwork Thu Oct 19 11:07:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1851608 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=DzdCZPod; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SB5fd52Dgz20cx for ; Thu, 19 Oct 2023 22:51:53 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtQsh-0001vK-G5; Thu, 19 Oct 2023 07:07:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsY-0001ky-DK for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsW-0000gT-CZ for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1697713667; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1S0MdEPDopjZwdIjA/scTxF1B0hg8WBNV2ShFi+1GsU=; b=DzdCZPodi7q9sWV2w0pe579a4jM3jGYZDjTpv/1PBEOtesCdFdKO8+dCxidVLNroxSQ79Z MEoImKohKd4HNWyYWR2h1GdvYTpRWtoE1gi7RAFSPbWmmmj6NF5gPPgUiSBAUTo/oYsmOY HyrQei0Q6dORIoOTzAtKfpBU7mo7CeA= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-660-jkJGzg5tOieOIQYyT6e_-g-1; Thu, 19 Oct 2023 07:07:38 -0400 X-MC-Unique: jkJGzg5tOieOIQYyT6e_-g-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 605713C0F44F; Thu, 19 Oct 2023 11:07:38 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7EEE38BE; Thu, 19 Oct 2023 11:07:37 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Leonardo Bras , Peter Xu , Juan Quintela Subject: [PATCH v2 09/11] migration: Move ram_flush_compressed_data() to ram-compress.c Date: Thu, 19 Oct 2023 13:07:22 +0200 Message-ID: <20231019110724.15324-10-quintela@redhat.com> In-Reply-To: <20231019110724.15324-1-quintela@redhat.com> References: <20231019110724.15324-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org As we export it, rename it compress_flush_data(). Signed-off-by: Juan Quintela Reviewed-by: Fabiano Rosas --- migration/ram-compress.h | 1 + migration/ram-compress.c | 9 +++++++++ migration/ram.c | 17 ++++------------- 3 files changed, 14 insertions(+), 13 deletions(-) diff --git a/migration/ram-compress.h b/migration/ram-compress.h index 636281ed97..7ba01e2882 100644 --- a/migration/ram-compress.h +++ b/migration/ram-compress.h @@ -73,5 +73,6 @@ uint64_t ram_compressed_pages(void); void update_compress_thread_counts(const CompressParam *param, int bytes_xmit); void compress_update_rates(uint64_t page_count); int compress_send_queued_data(CompressParam *param); +void compress_flush_data(void); #endif diff --git a/migration/ram-compress.c b/migration/ram-compress.c index af42cab0fe..1443a1cb45 100644 --- a/migration/ram-compress.c +++ b/migration/ram-compress.c @@ -558,3 +558,12 @@ void compress_update_rates(uint64_t page_count) compression_counters.compressed_size; } } + +void compress_flush_data(void) +{ + if (!migrate_compress()) { + return; + } + + flush_compressed_data(compress_send_queued_data); +} diff --git a/migration/ram.c b/migration/ram.c index d8b38ed999..4a07406c9a 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1300,15 +1300,6 @@ int compress_send_queued_data(CompressParam *param) return len; } -static void ram_flush_compressed_data(void) -{ - if (!migrate_compress()) { - return; - } - - flush_compressed_data(compress_send_queued_data); -} - #define PAGE_ALL_CLEAN 0 #define PAGE_TRY_AGAIN 1 #define PAGE_DIRTY_FOUND 2 @@ -1364,7 +1355,7 @@ static int find_dirty_block(RAMState *rs, PageSearchStatus *pss) * Also If xbzrle is on, stop using the data compression at this * point. In theory, xbzrle can do better than compression. */ - ram_flush_compressed_data(); + compress_flush_data(); /* Hit the end of the list */ pss->block = QLIST_FIRST_RCU(&ram_list.blocks); @@ -2036,7 +2027,7 @@ static bool save_compress_page(RAMState *rs, PageSearchStatus *pss, * much CPU resource. */ if (pss->block != pss->last_sent_block) { - ram_flush_compressed_data(); + compress_flush_data(); return false; } @@ -3083,7 +3074,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) * page is sent in one chunk. */ if (migrate_postcopy_ram()) { - ram_flush_compressed_data(); + compress_flush_data(); } /* @@ -3184,7 +3175,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque) } qemu_mutex_unlock(&rs->bitmap_mutex); - ram_flush_compressed_data(); + compress_flush_data(); int ret = rdma_registration_stop(f, RAM_CONTROL_FINISH); if (ret < 0) { From patchwork Thu Oct 19 11:07:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1851618 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=iiq33NLT; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SB6Sj3Q7Gz20Zj for ; Thu, 19 Oct 2023 23:28:21 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtQsq-0002U5-Lr; Thu, 19 Oct 2023 07:08:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsW-0001aE-P4 for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:48 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsU-0000fJ-H9 for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1697713661; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YemAaB2cQOJsEm2Kf4m5fvxeQysh+VHHJl/C/sQRj98=; b=iiq33NLTtKKxuPdIjWqzDuMyxjWwHAySnkkNWVU0lrcr7bji79nXZA29mjqmHARRupgnOk 7u/n6Jw3/Q0vYIEmyIx6dDpPYz5NO8qoEm1Zfsf5WanH7+a5OXmnHXd7yrSWUS4CWa5++J LJVYuFcnQdwNxCYG+HtdKL8gtwPvnFY= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-132-CkLIubWnPeKSW4iMbt9nlA-1; Thu, 19 Oct 2023 07:07:39 -0400 X-MC-Unique: CkLIubWnPeKSW4iMbt9nlA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 82A0B3C0F44F; Thu, 19 Oct 2023 11:07:39 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id A145925C8; Thu, 19 Oct 2023 11:07:38 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Leonardo Bras , Peter Xu , Juan Quintela Subject: [PATCH v2 10/11] migration: Merge flush_compressed_data() and compress_flush_data() Date: Thu, 19 Oct 2023 13:07:23 +0200 Message-ID: <20231019110724.15324-11-quintela@redhat.com> In-Reply-To: <20231019110724.15324-1-quintela@redhat.com> References: <20231019110724.15324-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Signed-off-by: Juan Quintela Reviewed-by: Fabiano Rosas --- migration/ram-compress.h | 1 - migration/ram-compress.c | 17 ++++++----------- 2 files changed, 6 insertions(+), 12 deletions(-) diff --git a/migration/ram-compress.h b/migration/ram-compress.h index 7ba01e2882..e222887fb7 100644 --- a/migration/ram-compress.h +++ b/migration/ram-compress.h @@ -59,7 +59,6 @@ typedef struct CompressParam CompressParam; void compress_threads_save_cleanup(void); int compress_threads_save_setup(void); -void flush_compressed_data(int (send_queued_data(CompressParam *))); bool compress_page_with_multi_thread(RAMBlock *block, ram_addr_t offset, int (send_queued_data(CompressParam *))); diff --git a/migration/ram-compress.c b/migration/ram-compress.c index 1443a1cb45..036e44085b 100644 --- a/migration/ram-compress.c +++ b/migration/ram-compress.c @@ -241,10 +241,14 @@ static inline void compress_reset_result(CompressParam *param) param->offset = 0; } -void flush_compressed_data(int (send_queued_data(CompressParam *))) +void compress_flush_data(void) { int thread_count = migrate_compress_threads(); + if (!migrate_compress()) { + return; + } + qemu_mutex_lock(&comp_done_lock); for (int i = 0; i < thread_count; i++) { while (!comp_param[i].done) { @@ -257,7 +261,7 @@ void flush_compressed_data(int (send_queued_data(CompressParam *))) qemu_mutex_lock(&comp_param[i].mutex); if (!comp_param[i].quit) { CompressParam *param = &comp_param[i]; - send_queued_data(param); + compress_send_queued_data(param); assert(qemu_file_buffer_empty(param->file)); compress_reset_result(param); } @@ -558,12 +562,3 @@ void compress_update_rates(uint64_t page_count) compression_counters.compressed_size; } } - -void compress_flush_data(void) -{ - if (!migrate_compress()) { - return; - } - - flush_compressed_data(compress_send_queued_data); -} From patchwork Thu Oct 19 11:07:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1851591 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=NvcmJvbN; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SB5PW0lD4z20Zj for ; Thu, 19 Oct 2023 22:40:31 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtQse-0001v2-H4; Thu, 19 Oct 2023 07:07:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsW-0001aD-Mr for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:48 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtQsU-0000fP-3j for qemu-devel@nongnu.org; Thu, 19 Oct 2023 07:07:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1697713662; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8wfCxek9OoPColiW0oHwTCqU4sKgH5HqEBvo1BgtFVs=; b=NvcmJvbNTIrK3pIvd/QamS8kK6p9UdKQzWG4A3l7NWwsGsLk1SqZrgmsr6CxPX7XxzTNGE FddxKK3IwqGcM0xiQ1OysGxOniZPt8hkyxgeOAzBjewIlFzOM4qagcZcSTSQ9yOqCqt/vr JuhAvoyb+zwfnlF2Lcb8MSglyb61yRg= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-475-K7uulQQpMq-F3ldfLegzAA-1; Thu, 19 Oct 2023 07:07:41 -0400 X-MC-Unique: K7uulQQpMq-F3ldfLegzAA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A4BF588B770; Thu, 19 Oct 2023 11:07:40 +0000 (UTC) Received: from secure.mitica (unknown [10.39.194.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id C444F25C8; Thu, 19 Oct 2023 11:07:39 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Fabiano Rosas , Leonardo Bras , Peter Xu , Juan Quintela Subject: [PATCH v2 11/11] migration: Rename ram_compressed_pages() to compress_ram_pages() Date: Thu, 19 Oct 2023 13:07:24 +0200 Message-ID: <20231019110724.15324-12-quintela@redhat.com> In-Reply-To: <20231019110724.15324-1-quintela@redhat.com> References: <20231019110724.15324-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org We are moving to have all functions exported from ram-compress.c to start with compress_. Signed-off-by: Juan Quintela Reviewed-by: Fabiano Rosas --- migration/ram-compress.h | 2 +- migration/ram-compress.c | 2 +- migration/ram.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/migration/ram-compress.h b/migration/ram-compress.h index e222887fb7..0d89a2f55e 100644 --- a/migration/ram-compress.h +++ b/migration/ram-compress.h @@ -68,7 +68,7 @@ int compress_threads_load_setup(QEMUFile *f); void decompress_data_with_multi_threads(QEMUFile *f, void *host, int len); void populate_compress(MigrationInfo *info); -uint64_t ram_compressed_pages(void); +uint64_t compress_ram_pages(void); void update_compress_thread_counts(const CompressParam *param, int bytes_xmit); void compress_update_rates(uint64_t page_count); int compress_send_queued_data(CompressParam *param); diff --git a/migration/ram-compress.c b/migration/ram-compress.c index 036e44085b..fa4388f6a6 100644 --- a/migration/ram-compress.c +++ b/migration/ram-compress.c @@ -516,7 +516,7 @@ void populate_compress(MigrationInfo *info) info->compression->compression_rate = compression_counters.compression_rate; } -uint64_t ram_compressed_pages(void) +uint64_t compress_ram_pages(void) { return compression_counters.pages; } diff --git a/migration/ram.c b/migration/ram.c index 4a07406c9a..e8a1527d82 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -932,7 +932,7 @@ uint64_t ram_get_total_transferred_pages(void) { return stat64_get(&mig_stats.normal_pages) + stat64_get(&mig_stats.zero_pages) + - ram_compressed_pages() + xbzrle_counters.pages; + compress_ram_pages() + xbzrle_counters.pages; } static void migration_update_rates(RAMState *rs, int64_t end_time)