From patchwork Tue Aug 7 09:12:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 954415 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="uRxng30Z"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41l8276JZZz9rxx for ; Tue, 7 Aug 2018 19:13:19 +1000 (AEST) Received: from localhost ([::1]:38008 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmy3J-000377-G2 for incoming@patchwork.ozlabs.org; Tue, 07 Aug 2018 05:13:17 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48441) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmy2j-00035Z-Kq for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:12:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fmy2h-00054E-UJ for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:12:41 -0400 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]:37552) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fmy2h-00053y-LR for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:12:39 -0400 Received: by mail-pf1-x42d.google.com with SMTP id a26-v6so8278942pfo.4 for ; Tue, 07 Aug 2018 02:12:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=RQRkGEZOty9XAAuLnX2ufrbtBd41H//NQcO3kT+pIy0=; b=uRxng30ZzByi1Nj8qjSvPylKM8xOujMTZWbDZjdxvHbXGGWcd01PEXt8pS62Zhkvc4 xA7XBY28h9FrJatU/RrtZvQNPSHc4nPmSDJX+pTG0OmooGnJdtoZuPHxUdHtpNVDg3ni 2dBAQnkaIddmNC9TlSJ6cSGQvdKr5JCHjmO6GyywczSFtPrbrt80t4fvtYuOy/G8fCoO Wyd6JiKjBtUcSWB6iwfjivTDcRq0yUACJ971SFAGRRrEHNAy1aeXg1Gv0ZiayyHK0KN9 1ZwsboVmUKkQpfLIFF1gKxUuTO8Ol0nQao83CPxZsr/6fuQAD/WGkZtlOGHczY6zbrEW IZeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=RQRkGEZOty9XAAuLnX2ufrbtBd41H//NQcO3kT+pIy0=; b=H2UN1qG9/gTTpO6pLQtMkbD6coCpM47RfUrbjZs8LmeWSHuc1xtsy92kx4wOeTzghp 2v3U+a1TLfgyJOpJTVAjqhtCgVTu45P8EmVDz9zQFoHMYwFm498jZq5gD1eo/nzWglBM qfx/raTMOT8Xlhzfzuxo7dJqgDkm/eJt646KbIqRzkyhCgXlN/villsqv6oUcWayH+iT e/BD1Y4GsQRMnz2kSc4hwIcknPFWJ1E9y9eRHycyHc0H0kgPRHo0AkmPvur0LtbkbCaq b3+Rf+EOKJ3EvBj2KZfRIAFOmJt5llC4IM987qnvVdilj4wRP/vz7J4zW7N4OAQM9YKC ILUg== X-Gm-Message-State: AOUpUlErNmBox2NXGA8wS/lNHGw1MnipSGwnKXPY5pw6qbTfAiwikabV cSwP74bN1HMa3+K9CUSpOek= X-Google-Smtp-Source: AAOMgpdwX0IghLV4XqWR+hzh9vZtMWLvR9hpgx2uH7jQIeWaBZ+ocljKbpBwlQX4E7oKPhzpGfdDEQ== X-Received: by 2002:a63:ff4d:: with SMTP id s13-v6mr17982470pgk.150.1533633158804; Tue, 07 Aug 2018 02:12:38 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.37]) by smtp.gmail.com with ESMTPSA id z4-v6sm2159645pfl.11.2018.08.07.02.12.35 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Aug 2018 02:12:38 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 7 Aug 2018 17:12:00 +0800 Message-Id: <20180807091209.13531-2-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180807091209.13531-1-xiaoguangrong@tencent.com> References: <20180807091209.13531-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::42d Subject: [Qemu-devel] [PATCH v3 01/10] migration: do not wait for free thread X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong Instead of putting the main thread to sleep state to wait for free compression thread, we can directly post it out as normal page that reduces the latency and uses CPUs more efficiently A parameter, compress-wait-thread, is introduced, it can be enabled if the user really wants the old behavior Signed-off-by: Xiao Guangrong Reviewed-by: Peter Xu --- hmp.c | 8 ++++++++ migration/migration.c | 21 +++++++++++++++++++++ migration/migration.h | 1 + migration/ram.c | 45 ++++++++++++++++++++++++++------------------- qapi/migration.json | 23 ++++++++++++++++++----- 5 files changed, 74 insertions(+), 24 deletions(-) diff --git a/hmp.c b/hmp.c index 2aafb50e8e..47d36e3ccf 100644 --- a/hmp.c +++ b/hmp.c @@ -327,6 +327,10 @@ void hmp_info_migrate_parameters(Monitor *mon, const QDict *qdict) monitor_printf(mon, "%s: %u\n", MigrationParameter_str(MIGRATION_PARAMETER_COMPRESS_THREADS), params->compress_threads); + assert(params->has_compress_wait_thread); + monitor_printf(mon, "%s: %s\n", + MigrationParameter_str(MIGRATION_PARAMETER_COMPRESS_WAIT_THREAD), + params->compress_wait_thread ? "on" : "off"); assert(params->has_decompress_threads); monitor_printf(mon, "%s: %u\n", MigrationParameter_str(MIGRATION_PARAMETER_DECOMPRESS_THREADS), @@ -1623,6 +1627,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict) p->has_compress_threads = true; visit_type_int(v, param, &p->compress_threads, &err); break; + case MIGRATION_PARAMETER_COMPRESS_WAIT_THREAD: + p->has_compress_wait_thread = true; + visit_type_bool(v, param, &p->compress_wait_thread, &err); + break; case MIGRATION_PARAMETER_DECOMPRESS_THREADS: p->has_decompress_threads = true; visit_type_int(v, param, &p->decompress_threads, &err); diff --git a/migration/migration.c b/migration/migration.c index b7d9854bda..2ccaadc03d 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -671,6 +671,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp) params->compress_level = s->parameters.compress_level; params->has_compress_threads = true; params->compress_threads = s->parameters.compress_threads; + params->has_compress_wait_thread = true; + params->compress_wait_thread = s->parameters.compress_wait_thread; params->has_decompress_threads = true; params->decompress_threads = s->parameters.decompress_threads; params->has_cpu_throttle_initial = true; @@ -1061,6 +1063,10 @@ static void migrate_params_test_apply(MigrateSetParameters *params, dest->compress_threads = params->compress_threads; } + if (params->has_compress_wait_thread) { + dest->compress_wait_thread = params->compress_wait_thread; + } + if (params->has_decompress_threads) { dest->decompress_threads = params->decompress_threads; } @@ -1126,6 +1132,10 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp) s->parameters.compress_threads = params->compress_threads; } + if (params->has_compress_wait_thread) { + s->parameters.compress_wait_thread = params->compress_wait_thread; + } + if (params->has_decompress_threads) { s->parameters.decompress_threads = params->decompress_threads; } @@ -1871,6 +1881,15 @@ int migrate_compress_threads(void) return s->parameters.compress_threads; } +int migrate_compress_wait_thread(void) +{ + MigrationState *s; + + s = migrate_get_current(); + + return s->parameters.compress_wait_thread; +} + int migrate_decompress_threads(void) { MigrationState *s; @@ -3131,6 +3150,8 @@ static Property migration_properties[] = { DEFINE_PROP_UINT8("x-compress-threads", MigrationState, parameters.compress_threads, DEFAULT_MIGRATE_COMPRESS_THREAD_COUNT), + DEFINE_PROP_BOOL("x-compress-wait-thread", MigrationState, + parameters.compress_wait_thread, true), DEFINE_PROP_UINT8("x-decompress-threads", MigrationState, parameters.decompress_threads, DEFAULT_MIGRATE_DECOMPRESS_THREAD_COUNT), diff --git a/migration/migration.h b/migration/migration.h index 64a7b33735..a46b9e6c8d 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -271,6 +271,7 @@ bool migrate_use_return_path(void); bool migrate_use_compression(void); int migrate_compress_level(void); int migrate_compress_threads(void); +int migrate_compress_wait_thread(void); int migrate_decompress_threads(void); bool migrate_use_events(void); bool migrate_postcopy_blocktime(void); diff --git a/migration/ram.c b/migration/ram.c index 24dea2730c..ae9e83c2b6 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1889,30 +1889,34 @@ static int compress_page_with_multi_thread(RAMState *rs, RAMBlock *block, ram_addr_t offset) { int idx, thread_count, bytes_xmit = -1, pages = -1; + bool wait = migrate_compress_wait_thread(); thread_count = migrate_compress_threads(); qemu_mutex_lock(&comp_done_lock); - while (true) { - for (idx = 0; idx < thread_count; idx++) { - if (comp_param[idx].done) { - comp_param[idx].done = false; - bytes_xmit = qemu_put_qemu_file(rs->f, comp_param[idx].file); - qemu_mutex_lock(&comp_param[idx].mutex); - set_compress_params(&comp_param[idx], block, offset); - qemu_cond_signal(&comp_param[idx].cond); - qemu_mutex_unlock(&comp_param[idx].mutex); - pages = 1; - ram_counters.normal++; - ram_counters.transferred += bytes_xmit; - break; - } - } - if (pages > 0) { +retry: + for (idx = 0; idx < thread_count; idx++) { + if (comp_param[idx].done) { + comp_param[idx].done = false; + bytes_xmit = qemu_put_qemu_file(rs->f, comp_param[idx].file); + qemu_mutex_lock(&comp_param[idx].mutex); + set_compress_params(&comp_param[idx], block, offset); + qemu_cond_signal(&comp_param[idx].cond); + qemu_mutex_unlock(&comp_param[idx].mutex); + pages = 1; + ram_counters.normal++; + ram_counters.transferred += bytes_xmit; break; - } else { - qemu_cond_wait(&comp_done_cond, &comp_done_lock); } } + + /* + * wait for the free thread if the user specifies 'compress-wait-thread', + * otherwise we will post the page out in the main thread as normal page. + */ + if (pages < 0 && wait) { + qemu_cond_wait(&comp_done_cond, &comp_done_lock); + goto retry; + } qemu_mutex_unlock(&comp_done_lock); return pages; @@ -2226,7 +2230,10 @@ static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss, * CPU resource. */ if (block == rs->last_sent_block && save_page_use_compression(rs)) { - return compress_page_with_multi_thread(rs, block, offset); + res = compress_page_with_multi_thread(rs, block, offset); + if (res > 0) { + return res; + } } else if (migrate_use_multifd()) { return ram_save_multifd_page(rs, block, offset); } diff --git a/qapi/migration.json b/qapi/migration.json index 186e8a7303..e6991fcbd2 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -462,6 +462,11 @@ # @compress-threads: Set compression thread count to be used in live migration, # the compression thread count is an integer between 1 and 255. # +# @compress-wait-thread: Wait if no thread is free to compress the memory page +# if it's enabled, otherwise, the page will be posted out immediately +# in the main thread without compression. It's true on default. +# (Since: 3.1) +# # @decompress-threads: Set decompression thread count to be used in live # migration, the decompression thread count is an integer between 1 # and 255. Usually, decompression is at least 4 times as fast as @@ -526,11 +531,11 @@ # Since: 2.4 ## { 'enum': 'MigrationParameter', - 'data': ['compress-level', 'compress-threads', 'decompress-threads', - 'cpu-throttle-initial', 'cpu-throttle-increment', - 'tls-creds', 'tls-hostname', 'max-bandwidth', - 'downtime-limit', 'x-checkpoint-delay', 'block-incremental', - 'x-multifd-channels', 'x-multifd-page-count', + 'data': ['compress-level', 'compress-threads', 'compress-wait-thread', + 'decompress-threads', 'cpu-throttle-initial', + 'cpu-throttle-increment', 'tls-creds', 'tls-hostname', + 'max-bandwidth', 'downtime-limit', 'x-checkpoint-delay', + 'block-incremental', 'x-multifd-channels', 'x-multifd-page-count', 'xbzrle-cache-size', 'max-postcopy-bandwidth' ] } ## @@ -540,6 +545,9 @@ # # @compress-threads: compression thread count # +# @compress-wait-thread: Wait if no thread is free to compress the memory page +# (Since: 3.1) +# # @decompress-threads: decompression thread count # # @cpu-throttle-initial: Initial percentage of time guest cpus are @@ -610,6 +618,7 @@ { 'struct': 'MigrateSetParameters', 'data': { '*compress-level': 'int', '*compress-threads': 'int', + '*compress-wait-thread': 'bool', '*decompress-threads': 'int', '*cpu-throttle-initial': 'int', '*cpu-throttle-increment': 'int', @@ -649,6 +658,9 @@ # # @compress-threads: compression thread count # +# @compress-wait-thread: Wait if no thread is free to compress the memory page +# (Since: 3.1) +# # @decompress-threads: decompression thread count # # @cpu-throttle-initial: Initial percentage of time guest cpus are @@ -714,6 +726,7 @@ { 'struct': 'MigrationParameters', 'data': { '*compress-level': 'uint8', '*compress-threads': 'uint8', + '*compress-wait-thread': 'bool', '*decompress-threads': 'uint8', '*cpu-throttle-initial': 'uint8', '*cpu-throttle-increment': 'uint8', From patchwork Tue Aug 7 09:12:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 954418 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Exw7H4EN"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41l85R1GgNz9rxx for ; Tue, 7 Aug 2018 19:16:11 +1000 (AEST) Received: from localhost ([::1]:38020 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmy64-0005n5-MF for incoming@patchwork.ozlabs.org; Tue, 07 Aug 2018 05:16:08 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48453) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmy2m-00037C-Qx for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:12:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fmy2l-00055F-Tc for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:12:44 -0400 Received: from mail-pf1-x42b.google.com ([2607:f8b0:4864:20::42b]:44766) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fmy2l-000555-Lq for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:12:43 -0400 Received: by mail-pf1-x42b.google.com with SMTP id k21-v6so8262453pff.11 for ; Tue, 07 Aug 2018 02:12:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=3zt+Khuobr5RUdqrSMIsTzX+YCNaFtVf0EUqgmfZh9Y=; b=Exw7H4ENaSU2hFN1EhNGRbEzsZ23VhzzX1aOgpP52zfDIX70ABc6LP/1PT6n5c2XuU +DFtTdgKN7bBiypFca7qYJaUIYy1tL8nVeC1nqe81vlIiZHbTN+hwoVgPLqt12pNcQpx LwMc6WBZeKBBuPV0f44LBbA9SXj02nTECT/S+h2D4DFzpIasc7WwMN8W9nK+1CRnHwKT oqjFrG/UUZzrLr2dwxbv6Jj4FfKFop1eJ2O4pan04ykAncG06+TuZD11WBGOk76X+Wk1 DaXGP3z/duwpBfRsHL2RKA4wY/tMQDAfWGB2exr4GKxBnw3DJf8nts8XPctrdl3V60xz qaBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=3zt+Khuobr5RUdqrSMIsTzX+YCNaFtVf0EUqgmfZh9Y=; b=fMRAxPe4Gqd18PEnO1bN6EqSiKdek0W3CviMy9x+k0nD1rLE2isKX8BH6ZqjlYbGCA ax/2bjCjDdPprBbfPZyyD992bAjwnW48IxmWmg5Hob+mkdbUJsokkOmFNBVV4kFGDtIw fTEQ0sabCsGKILniebF7h9x9S+Cj5zK7wu3A+gyqxdNQQKydwfsk3Vw8cTmr5VJoXaQO 9m5Fxenw4DuqHAdi+fu+hsQp72WSNd84ODqCk1wb0PLLp8AftvsxEHJY60p0a4h4ooki gw0pvyzqxXquGECN6f4RjosPheFF6NF2xQJ8g9IhlfH9enU/XtW26NQbWUJs/OwuMCcL sxDg== X-Gm-Message-State: AOUpUlG9RNLUFDKJtSjvn5VokZVLWn4vB/IU68cSM9VZlxVQRtr80hWC EPl8awcHTmlrZm1R4pBMVNA= X-Google-Smtp-Source: AAOMgpcrmQLNM+Kw+ncIPnKjkVvJvAVlmglvJ1XT4ohLh7Jj6g03JnMWSSxbIpKl8D5sHgmmSvpseg== X-Received: by 2002:a63:4663:: with SMTP id v35-v6mr17516657pgk.178.1533633162951; Tue, 07 Aug 2018 02:12:42 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.37]) by smtp.gmail.com with ESMTPSA id z4-v6sm2159645pfl.11.2018.08.07.02.12.39 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Aug 2018 02:12:42 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 7 Aug 2018 17:12:01 +0800 Message-Id: <20180807091209.13531-3-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180807091209.13531-1-xiaoguangrong@tencent.com> References: <20180807091209.13531-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::42b Subject: [Qemu-devel] [PATCH v3 02/10] migration: fix counting normal page for compression X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong The compressed page is not normal page Reviewed-by: Peter Xu Signed-off-by: Xiao Guangrong --- migration/ram.c | 1 - 1 file changed, 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index ae9e83c2b6..d631b9a6fe 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1903,7 +1903,6 @@ retry: qemu_cond_signal(&comp_param[idx].cond); qemu_mutex_unlock(&comp_param[idx].mutex); pages = 1; - ram_counters.normal++; ram_counters.transferred += bytes_xmit; break; } From patchwork Tue Aug 7 09:12:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 954419 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="VdAadeq4"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41l85h4lMnz9rxx for ; Tue, 7 Aug 2018 19:16:24 +1000 (AEST) Received: from localhost ([::1]:38025 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmy6I-00061m-8u for incoming@patchwork.ozlabs.org; Tue, 07 Aug 2018 05:16:22 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48471) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmy2t-0003CO-8Z for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:12:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fmy2p-00056Q-WE for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:12:51 -0400 Received: from mail-pg1-x541.google.com ([2607:f8b0:4864:20::541]:37341) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fmy2p-00056G-PZ for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:12:47 -0400 Received: by mail-pg1-x541.google.com with SMTP id n7-v6so7556650pgq.4 for ; Tue, 07 Aug 2018 02:12:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=NsmhccRCp5gWMJ0cmbdhkw17Dbew1msB4Oy/3ymK/ko=; b=VdAadeq4a+hsKW8NyZ0bPVrTumkfiG/Kr3mchOmTJX21881YoJcQix1DtazH75t+pp vcPZoAeOBXh38F332znUMh4k5ACldc4d/FNPPljiCD3t+x0Eu4qfY6fLvUXMzoJGnUZI QE8n0UmaMyJbMtHUpS/ho9mcn+6tssw4lbzlQmsFTeN/S0Nmev8Tk5LM+6aCL7KsgUJB Bk9zV+q8PSy/BsnVmxG/Pa8iwhJm6GU3UsqnUXPinjZ3VKU/hvX1M4KtDY59I5/MPd2c eGaDIAOm91gir3dhXIQUBo9Xdy+B1RxnXTbXxaZnO/n62ZSrlFUBpsk767vM/jsVLJ8r mbyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=NsmhccRCp5gWMJ0cmbdhkw17Dbew1msB4Oy/3ymK/ko=; b=Lfp2/IKCYa1yscFb49k9dS2HG2TtvzPLnkLMKZfFGB8Ym43WeYwOP9VpsmeG4EyYrj 9JhBYb3Ni4OseRONuSPYd5GRtV/iljKzXYsItu2bnEoIyxEJm8PrwyBD+/WSAttUmQJV yxECIZYDSvvEduFq9Jsbox0ci2/leferi6AE8LqsZruC/q/rdUpo21Lxfq3JvnUr42gD Ow9VutYIb85p8cjWBtkHRlJN8oHK7xDlGEZvb2FSKlf8btPAH9qDe0oGTPb+yxyZtH0g kPFhR0JKun1fKofDN/CpabbwYfZdMIXEMijmFp3Ahna9RCUi9syh4+tdWE+sq44cRJWS ywlA== X-Gm-Message-State: AOUpUlGwjiExNlr+9K2XGV9zzvZVhGYXzNkzPSAw97uMgPh5ockHNmks JuhDrerJUpXAlz0G5JbgsZw= X-Google-Smtp-Source: AAOMgpcrn/iPQWL41RfvsVBEryGl4xwf+a+P9u77MDSjGneBykAgRUdJYWvHtgcAWXEDI5EAegXXNQ== X-Received: by 2002:a65:6455:: with SMTP id s21-v6mr17352098pgv.394.1533633166956; Tue, 07 Aug 2018 02:12:46 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.37]) by smtp.gmail.com with ESMTPSA id z4-v6sm2159645pfl.11.2018.08.07.02.12.43 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Aug 2018 02:12:46 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 7 Aug 2018 17:12:02 +0800 Message-Id: <20180807091209.13531-4-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180807091209.13531-1-xiaoguangrong@tencent.com> References: <20180807091209.13531-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::541 Subject: [Qemu-devel] [PATCH v3 03/10] migration: introduce save_zero_page_to_file X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong It will be used by the compression threads Reviewed-by: Peter Xu Signed-off-by: Xiao Guangrong --- migration/ram.c | 40 ++++++++++++++++++++++++++++++---------- 1 file changed, 30 insertions(+), 10 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index d631b9a6fe..49ace30614 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1667,27 +1667,47 @@ static void migration_bitmap_sync(RAMState *rs) /** * save_zero_page: send the zero page to the stream * - * Returns the number of pages written. + * Returns the size of data written to the file, 0 means the page is not + * a zero page * * @rs: current RAM state + * @file: the file where the data is saved * @block: block that contains the page we want to send * @offset: offset inside the block for the page */ -static int save_zero_page(RAMState *rs, RAMBlock *block, ram_addr_t offset) +static int save_zero_page_to_file(RAMState *rs, QEMUFile *file, + RAMBlock *block, ram_addr_t offset) { uint8_t *p = block->host + offset; - int pages = -1; + int len = 0; if (is_zero_range(p, TARGET_PAGE_SIZE)) { - ram_counters.duplicate++; - ram_counters.transferred += - save_page_header(rs, rs->f, block, offset | RAM_SAVE_FLAG_ZERO); - qemu_put_byte(rs->f, 0); - ram_counters.transferred += 1; - pages = 1; + len += save_page_header(rs, file, block, offset | RAM_SAVE_FLAG_ZERO); + qemu_put_byte(file, 0); + len += 1; } + return len; +} - return pages; +/** + * save_zero_page: send the zero page to the stream + * + * Returns the number of pages written. + * + * @rs: current RAM state + * @block: block that contains the page we want to send + * @offset: offset inside the block for the page + */ +static int save_zero_page(RAMState *rs, RAMBlock *block, ram_addr_t offset) +{ + int len = save_zero_page_to_file(rs, rs->f, block, offset); + + if (len) { + ram_counters.duplicate++; + ram_counters.transferred += len; + return 1; + } + return -1; } static void ram_release_pages(const char *rbname, uint64_t offset, int pages) From patchwork Tue Aug 7 09:12:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 954421 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="kGzKSUAU"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41l88Z5c1sz9rxx for ; Tue, 7 Aug 2018 19:18:54 +1000 (AEST) Received: from localhost ([::1]:38043 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmy8i-0007o1-C4 for incoming@patchwork.ozlabs.org; Tue, 07 Aug 2018 05:18:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48483) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmy2u-0003DG-Vg for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:12:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fmy2t-00057V-U1 for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:12:52 -0400 Received: from mail-pf1-x42b.google.com ([2607:f8b0:4864:20::42b]:41001) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fmy2t-00057M-NW for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:12:51 -0400 Received: by mail-pf1-x42b.google.com with SMTP id y10-v6so8279000pfn.8 for ; Tue, 07 Aug 2018 02:12:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=PgGybqDLCJbEJ+qjtJtsZofD+YTrV5tTXrhVisMfhNg=; b=kGzKSUAUCol2lDdu6U4hQb9DlkJ9+vQ5XVRxZpO+4iZ/fGZoHUUhdJctVnNxOdEefc qvkECWQygJCVRalwCpx4qAWwRkE1lZNdrMcWZxBTwsQpEXmrI/l2cv7ICSjDEqBjcfd1 nV4gLDJXkLR4jGj9Bqcq0s11vdee5dsPgQptRfJCT3q5aImsGG+DS6CXW3MrFkONLBdT a8NLMwxu9JfEhVwIJYAlWfIbiRx6ElohOKmzAMHeA71HTzI1xKq4nxasZjhabCCAM4hM WxJHAZ0jG9PW6gySQ0v2QmoPXhGurJglS+vIa4zuT470V3CLU70BRFi+sXmjIeqoLcCx yqTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=PgGybqDLCJbEJ+qjtJtsZofD+YTrV5tTXrhVisMfhNg=; b=OTvGTJzgL8VPqEb8SMB7RfTYWcFGLD+AnMM1M9s/hPFbYUlCkbCkcAhYeqxmL2MrKD T1aLqL21e3YY3Iu8cEoWVTTC5PZqZN3LnU++SyMc3FPLGZDJ2uj3Wt6ivUoS5MWXAoVj m+V5CQhXKxoU7OkhPC3b4kd+RjNYbdIzguHrvqI/zO4/ii4Xp2OCRpywz+0bauBymKs1 Uw5KqJvg5CDmzsjX7LPnlUCwsmiFV5Gg2II3srWPWGof3wE0e7dZJf9Xc72EukMRon4R uBjHxTTa2hbFnCszT/lp0FF1xhQc3q1QXQHvWHItkSNzILSGJnUEGNCSR3qkWk3p+RKV yqUg== X-Gm-Message-State: AOUpUlGb91EMgkPapaC2kXYvmjVknPCAQsS0qX1dLZTMh0rVnm3KJKvp 5mXzX+/H99aKVkBXCBsxmno= X-Google-Smtp-Source: AAOMgpe+My3pheKlqaW64cvs3NW4Vbtr9B6OYUrivCbaJ7mftJqfNCsJQCaPc8Jv25jDK81kauJRkA== X-Received: by 2002:a63:fb57:: with SMTP id w23-v6mr17942523pgj.441.1533633171030; Tue, 07 Aug 2018 02:12:51 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.37]) by smtp.gmail.com with ESMTPSA id z4-v6sm2159645pfl.11.2018.08.07.02.12.47 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Aug 2018 02:12:50 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 7 Aug 2018 17:12:03 +0800 Message-Id: <20180807091209.13531-5-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180807091209.13531-1-xiaoguangrong@tencent.com> References: <20180807091209.13531-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::42b Subject: [Qemu-devel] [PATCH v3 04/10] migration: drop the return value of do_compress_ram_page X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong It is not used and cleans the code up a little Reviewed-by: Peter Xu Signed-off-by: Xiao Guangrong --- migration/ram.c | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 49ace30614..e463de4f69 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -381,8 +381,8 @@ static QemuThread *decompress_threads; static QemuMutex decomp_done_lock; static QemuCond decomp_done_cond; -static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, - ram_addr_t offset, uint8_t *source_buf); +static void do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, + ram_addr_t offset, uint8_t *source_buf); static void *do_data_compress(void *opaque) { @@ -1842,15 +1842,14 @@ static int ram_save_multifd_page(RAMState *rs, RAMBlock *block, return 1; } -static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, - ram_addr_t offset, uint8_t *source_buf) +static void do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, + ram_addr_t offset, uint8_t *source_buf) { RAMState *rs = ram_state; - int bytes_sent, blen; uint8_t *p = block->host + (offset & TARGET_PAGE_MASK); + int ret; - bytes_sent = save_page_header(rs, f, block, offset | - RAM_SAVE_FLAG_COMPRESS_PAGE); + save_page_header(rs, f, block, offset | RAM_SAVE_FLAG_COMPRESS_PAGE); /* * copy it to a internal buffer to avoid it being modified by VM @@ -1858,17 +1857,14 @@ static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, * decompression */ memcpy(source_buf, p, TARGET_PAGE_SIZE); - blen = qemu_put_compression_data(f, stream, source_buf, TARGET_PAGE_SIZE); - if (blen < 0) { - bytes_sent = 0; - qemu_file_set_error(migrate_get_current()->to_dst_file, blen); + ret = qemu_put_compression_data(f, stream, source_buf, TARGET_PAGE_SIZE); + if (ret < 0) { + qemu_file_set_error(migrate_get_current()->to_dst_file, ret); error_report("compressed data failed!"); - } else { - bytes_sent += blen; - ram_release_pages(block->idstr, offset & TARGET_PAGE_MASK, 1); + return; } - return bytes_sent; + ram_release_pages(block->idstr, offset & TARGET_PAGE_MASK, 1); } static void flush_compressed_data(RAMState *rs) From patchwork Tue Aug 7 09:12:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 954416 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="erwLDtIc"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41l82f45MTz9rxx for ; Tue, 7 Aug 2018 19:13:46 +1000 (AEST) Received: from localhost ([::1]:38009 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmy3k-0003JJ-3Z for incoming@patchwork.ozlabs.org; Tue, 07 Aug 2018 05:13:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48508) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmy2z-0003Gj-Iu for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:13:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fmy2x-00058h-Vj for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:12:57 -0400 Received: from mail-pl0-x234.google.com ([2607:f8b0:400e:c01::234]:34187) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fmy2x-00058S-Md for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:12:55 -0400 Received: by mail-pl0-x234.google.com with SMTP id f6-v6so6863005plo.1 for ; Tue, 07 Aug 2018 02:12:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=SyYM5d+cxagMdam5gM9cfiGpIGmcp+5oxXbimk7kdsA=; b=erwLDtIcesHcfOTbdFDVXXcDEve1q1xCd5eZ1ra3oCI20liUY6FDkoT2XVW4Lb7FlP RnhrhLOlWROqbg3V4PUiRpG+YeHEUVrDjqIXMLF+MNi+xcQPAimGqHWwdFWK/77+BlqL SUuPmdRf2j0lCHE/Y0rY5FH1ZBytGCX2ATGljdillrysmeGZk/NO6P4v19KX+7PNsYpf ZDIRe+gT5swLsbeFZ+Wps88+OIUlBCxjLIRQeEmp92sEd1Aax919Am8O5iXFrfyDqByq uD5jIPNW6GK/4ih96qDfOmRK3XMQ1X/k+o30JSgeazpo+VsdRk2Nax0gszD+cj1ryUWI TdZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=SyYM5d+cxagMdam5gM9cfiGpIGmcp+5oxXbimk7kdsA=; b=b6l+K3TtiBzLkS6a46nSenzvzrbyYaxTN6gcXudkEy1ykuuJwd9RevQ8jPKILIbEVf wt/0BwiGZ/oCeNH+YT2ms7t1wBrC+1XSgGj/EueYtC+zbs8rSxkbDd5c5blRZ6rIgccF f2Nn/BVyoWIcdKKMdX0RiPAxsF/dhCGjGEnhB1kfY0sD0fcdxiCWBMx8o+xd/flCQkQ3 hRLmgJOOeWCid+84+YEu6TkcbBFe9chAfvI5pkd7cxVIdXRsFrPe15mBnYgdsRtwFMHn x6bCDuC/kA/NFRfjp3U0K8l2iAJBAfWwIqsqfAI5S0uzQeuqgFDx7/MRAzyu5TAjn765 U9CA== X-Gm-Message-State: AOUpUlHnLplif/12PcsikCJuw7Wo88NwVMGRqo+xyUM5TaadSNxSY6mY T9Cu/JJbgRRQp2m+pZV9VOA= X-Google-Smtp-Source: AAOMgpfeDPTll+CvIXiPXPBmlSKTGmKR3klyngKc2zFr0BYZ0vYQ/1M1pWFdqv6fpDCW2m+2yvfCJg== X-Received: by 2002:a17:902:27a8:: with SMTP id d37-v6mr17377764plb.290.1533633174907; Tue, 07 Aug 2018 02:12:54 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.37]) by smtp.gmail.com with ESMTPSA id z4-v6sm2159645pfl.11.2018.08.07.02.12.51 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Aug 2018 02:12:54 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 7 Aug 2018 17:12:04 +0800 Message-Id: <20180807091209.13531-6-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180807091209.13531-1-xiaoguangrong@tencent.com> References: <20180807091209.13531-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::234 Subject: [Qemu-devel] [PATCH v3 05/10] migration: move handle of zero page to the thread X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong Detecting zero page is not a light work, moving it to the thread to speed the main thread up, btw, handling ram_release_pages() for the zero page is moved to the thread as well Signed-off-by: Xiao Guangrong Reviewed-by: Peter Xu --- migration/ram.c | 96 +++++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 70 insertions(+), 26 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index e463de4f69..d804d01aae 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -340,6 +340,7 @@ typedef struct PageSearchStatus PageSearchStatus; struct CompressParam { bool done; bool quit; + bool zero_page; QEMUFile *file; QemuMutex mutex; QemuCond cond; @@ -381,7 +382,7 @@ static QemuThread *decompress_threads; static QemuMutex decomp_done_lock; static QemuCond decomp_done_cond; -static void do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, +static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, ram_addr_t offset, uint8_t *source_buf); static void *do_data_compress(void *opaque) @@ -389,6 +390,7 @@ static void *do_data_compress(void *opaque) CompressParam *param = opaque; RAMBlock *block; ram_addr_t offset; + bool zero_page; qemu_mutex_lock(¶m->mutex); while (!param->quit) { @@ -398,11 +400,12 @@ static void *do_data_compress(void *opaque) param->block = NULL; qemu_mutex_unlock(¶m->mutex); - do_compress_ram_page(param->file, ¶m->stream, block, offset, - param->originbuf); + zero_page = do_compress_ram_page(param->file, ¶m->stream, + block, offset, param->originbuf); qemu_mutex_lock(&comp_done_lock); param->done = true; + param->zero_page = zero_page; qemu_cond_signal(&comp_done_cond); qemu_mutex_unlock(&comp_done_lock); @@ -1842,13 +1845,19 @@ static int ram_save_multifd_page(RAMState *rs, RAMBlock *block, return 1; } -static void do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, +static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, ram_addr_t offset, uint8_t *source_buf) { RAMState *rs = ram_state; uint8_t *p = block->host + (offset & TARGET_PAGE_MASK); + bool zero_page = false; int ret; + if (save_zero_page_to_file(rs, f, block, offset)) { + zero_page = true; + goto exit; + } + save_page_header(rs, f, block, offset | RAM_SAVE_FLAG_COMPRESS_PAGE); /* @@ -1861,10 +1870,21 @@ static void do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, if (ret < 0) { qemu_file_set_error(migrate_get_current()->to_dst_file, ret); error_report("compressed data failed!"); - return; + return false; } +exit: ram_release_pages(block->idstr, offset & TARGET_PAGE_MASK, 1); + return zero_page; +} + +static void +update_compress_thread_counts(const CompressParam *param, int bytes_xmit) +{ + if (param->zero_page) { + ram_counters.duplicate++; + } + ram_counters.transferred += bytes_xmit; } static void flush_compressed_data(RAMState *rs) @@ -1888,7 +1908,12 @@ static void flush_compressed_data(RAMState *rs) qemu_mutex_lock(&comp_param[idx].mutex); if (!comp_param[idx].quit) { len = qemu_put_qemu_file(rs->f, comp_param[idx].file); - ram_counters.transferred += len; + /* + * it's safe to fetch zero_page without holding comp_done_lock + * as there is no further request submitted to the thread, + * i.e, the thread should be waiting for a request at this point. + */ + update_compress_thread_counts(&comp_param[idx], len); } qemu_mutex_unlock(&comp_param[idx].mutex); } @@ -1919,7 +1944,7 @@ retry: qemu_cond_signal(&comp_param[idx].cond); qemu_mutex_unlock(&comp_param[idx].mutex); pages = 1; - ram_counters.transferred += bytes_xmit; + update_compress_thread_counts(&comp_param[idx], bytes_xmit); break; } } @@ -2193,6 +2218,39 @@ static bool save_page_use_compression(RAMState *rs) return false; } +/* + * try to compress the page before posting it out, return true if the page + * has been properly handled by compression, otherwise needs other + * paths to handle it + */ +static bool save_compress_page(RAMState *rs, RAMBlock *block, ram_addr_t offset) +{ + if (!save_page_use_compression(rs)) { + return false; + } + + /* + * When starting the process of a new block, the first page of + * the block should be sent out before other pages in the same + * block, and all the pages in last block should have been sent + * out, keeping this order is important, because the 'cont' flag + * is used to avoid resending the block name. + * + * We post the fist page as normal page as compression will take + * much CPU resource. + */ + if (block != rs->last_sent_block) { + flush_compressed_data(rs); + return false; + } + + if (compress_page_with_multi_thread(rs, block, offset) > 0) { + return true; + } + + return false; +} + /** * ram_save_target_page: save one target page * @@ -2213,15 +2271,8 @@ static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss, return res; } - /* - * When starting the process of a new block, the first page of - * the block should be sent out before other pages in the same - * block, and all the pages in last block should have been sent - * out, keeping this order is important, because the 'cont' flag - * is used to avoid resending the block name. - */ - if (block != rs->last_sent_block && save_page_use_compression(rs)) { - flush_compressed_data(rs); + if (save_compress_page(rs, block, offset)) { + return 1; } res = save_zero_page(rs, block, offset); @@ -2239,17 +2290,10 @@ static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss, } /* - * Make sure the first page is sent out before other pages. - * - * we post it as normal page as compression will take much - * CPU resource. + * do not use multifd for compression as the first page in the new + * block should be posted out before sending the compressed page */ - if (block == rs->last_sent_block && save_page_use_compression(rs)) { - res = compress_page_with_multi_thread(rs, block, offset); - if (res > 0) { - return res; - } - } else if (migrate_use_multifd()) { + if (!save_page_use_compression(rs) && migrate_use_multifd()) { return ram_save_multifd_page(rs, block, offset); } From patchwork Tue Aug 7 09:12:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 954425 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="D66JwiKx"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41l8CF4J0cz9rxx for ; Tue, 7 Aug 2018 19:21:13 +1000 (AEST) Received: from localhost ([::1]:38057 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmyAx-0001hI-7s for incoming@patchwork.ozlabs.org; Tue, 07 Aug 2018 05:21:11 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48529) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmy32-0003Hc-V3 for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:13:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fmy31-00059s-Mn for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:13:00 -0400 Received: from mail-pl0-x244.google.com ([2607:f8b0:400e:c01::244]:39881) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fmy31-00059Y-F6 for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:12:59 -0400 Received: by mail-pl0-x244.google.com with SMTP id w14-v6so1273406plp.6 for ; Tue, 07 Aug 2018 02:12:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=9aVx/9MXIX8chFdQ0nNGdumPNvIXauKoEQdqfl233Qw=; b=D66JwiKxPbOHM+l+HIaPUG71Jmp8lx7gbvddX/owjF9HDjU2CIv+UGoyG6D28C/jRp pupggEPtbKm6Ev5TQkNmeJhFJOebDl3wPA8G217S3KpHZkezNuz1YFcpg5cPZyc5p4Yg WrCNYWRSaA4LFU4vMrToZMdagSUFOkayuwX6ZdpLM/jDSVJDKDdapSVOTTvvElUPpzRt crI7IVy6UnsP08A3l5lLg//ZXmaVuu6gw3g7WFYQERKCKu2I3FQBoUKu8JY4F79RJfbc vc6FkfluxMiyKOK3ARokMnsVD9HBFPhQ8jRG1Yx4Y25hK9Z7CzNjXRyJzvwVJNPl8r7J CEiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=9aVx/9MXIX8chFdQ0nNGdumPNvIXauKoEQdqfl233Qw=; b=X4GKuuq9QYFWpy+sDDjM8aBDlB2bbevaJUTy91a713b5XVKuwNpaT45bCKQ7dyoVGR numJRC1jlsysrhnvtC0a0Z4e7qYR+8INP71hDXLoZq6HkmdxMpAiQPnT0PofYaP7fsnM kllQUtUAB3EAjCy3IDtMrCBhNa459RHrE3d9G1Z5ebKFDEi8AQCXibEe5p/q1rxlqllp RmHzNPHv3U57ep6bBPwjbMeSnU3ic261zAuvUjNmfx1moqq5xElPnKaFtL+XykYNJlOr +fEmMVIglUmi4y1Wa2n2SPhOl05Bke9BNnVToGOeQxklRaB/IgBxbDnxnUt7oFa7cz/0 TGlg== X-Gm-Message-State: AOUpUlGlTCBRjW+9o1gxFeV+sUqdFjj4ivCYlDKQ3LC6K9tUOMW62MBB 0NgxsaJ7wD0vXy09xm4NATQwQXZF X-Google-Smtp-Source: AAOMgpcYryFkcGrdLRww1vEazllSMKHCkLpyH2ua5ICy4PRTiTbWqnOkeNDdSsPK6AwwbD9OeTsQCg== X-Received: by 2002:a17:902:8d91:: with SMTP id v17-v6mr17339480plo.9.1533633178727; Tue, 07 Aug 2018 02:12:58 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.37]) by smtp.gmail.com with ESMTPSA id z4-v6sm2159645pfl.11.2018.08.07.02.12.55 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Aug 2018 02:12:58 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 7 Aug 2018 17:12:05 +0800 Message-Id: <20180807091209.13531-7-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180807091209.13531-1-xiaoguangrong@tencent.com> References: <20180807091209.13531-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::244 Subject: [Qemu-devel] [PATCH v3 06/10] migration: hold the lock only if it is really needed X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong Try to hold src_page_req_mutex only if the queue is not empty Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Peter Xu Signed-off-by: Xiao Guangrong --- include/qemu/queue.h | 1 + migration/ram.c | 4 ++++ 2 files changed, 5 insertions(+) diff --git a/include/qemu/queue.h b/include/qemu/queue.h index 59fd1203a1..ac418efc43 100644 --- a/include/qemu/queue.h +++ b/include/qemu/queue.h @@ -341,6 +341,7 @@ struct { \ /* * Simple queue access methods. */ +#define QSIMPLEQ_EMPTY_ATOMIC(head) (atomic_read(&((head)->sqh_first)) == NULL) #define QSIMPLEQ_EMPTY(head) ((head)->sqh_first == NULL) #define QSIMPLEQ_FIRST(head) ((head)->sqh_first) #define QSIMPLEQ_NEXT(elm, field) ((elm)->field.sqe_next) diff --git a/migration/ram.c b/migration/ram.c index d804d01aae..99ecf9b315 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2026,6 +2026,10 @@ static RAMBlock *unqueue_page(RAMState *rs, ram_addr_t *offset) { RAMBlock *block = NULL; + if (QSIMPLEQ_EMPTY_ATOMIC(&rs->src_page_requests)) { + return NULL; + } + qemu_mutex_lock(&rs->src_page_req_mutex); if (!QSIMPLEQ_EMPTY(&rs->src_page_requests)) { struct RAMSrcPageRequest *entry = From patchwork Tue Aug 7 09:12:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 954417 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="L3XH1/V3"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41l82l2skJz9rxx for ; Tue, 7 Aug 2018 19:13:51 +1000 (AEST) Received: from localhost ([::1]:38010 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmy3o-0003N1-Sn for incoming@patchwork.ozlabs.org; Tue, 07 Aug 2018 05:13:48 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48554) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmy36-0003KW-MY for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:13:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fmy35-0005Ay-MS for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:13:04 -0400 Received: from mail-pl0-x242.google.com ([2607:f8b0:400e:c01::242]:43405) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fmy35-0005Al-FU for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:13:03 -0400 Received: by mail-pl0-x242.google.com with SMTP id x6-v6so6846602plv.10 for ; Tue, 07 Aug 2018 02:13:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=76tOyzDvpxGSakJHlQHhrtxXZY1tzQpvddPPuUF4ip0=; b=L3XH1/V3Es7+EMlTxYDHiSCvgHKOWPAycSHPtnfRxQy4z0Z7hWRj1Y9pPz5Yj6L3fs V2mSeDWHP9c6kBI+2QLrSfMdPoM0pNC4bBDyY5wAlp2vmzRGB/Ur3hkg/k06p7/ahX9a Ih3xSMD7aOcl/XOgR57YF6rk3KUixGVGUI7npaGmQrEJoayIOkPUTeTNp1Ba+IqVufWt 9MT1kPsiJp+mp2fr+9tsvh+Aat28fvNeppPvyY9AquJkJiG1OVyEyPYKWeQY+gY2qiLA 2+2ui/T2ccs4nZ5iVXAqhr5LM0iSHirdQL/TVCwrdSJUO2Qsx4Vbd9DR5jZuHJftkx/y IFwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=76tOyzDvpxGSakJHlQHhrtxXZY1tzQpvddPPuUF4ip0=; b=FbKadrPLiOjGr7xkaB2ditSTC+NAY/ZPqqmztxdcaV/oLY61JtQ4DY5b8ZpfM25Drb GS/aBJL44hnx08ZxAa94dJcj1VI+CckrHkg6UcNarkXZs7mTrVzEax8ZJLST4VcIXZ3X +XVFjgW2sUTHMv2HCyKHuicqLzSiw11iJW3/TckXB++Zz3PyVpMYbQ9f1en9fU2X3T2x vJgImUQv7hccIscmVmC2Kube6bFMXK1FAGy9TXYBLCAFNUMHXXmCLBwwdXKHAeuVaVvl wiMps5lBLCbOU+c+nqx71kgDyAfdcrSVmb9UEEAu6KrDUt7oNDRnmdq57x0tZgeMNenU Y14w== X-Gm-Message-State: AOUpUlGObDXCZrtaoXLYsL0J6R0Xp3hXu3QvMPdVuvxNMJwFgScmDDFi zwHTWs008TXTj6f8Ou+A9Jc= X-Google-Smtp-Source: AAOMgpd8K/2MS+uzOpqbpCHBaz/3O8ujDk75E3kgluPzez3jJr/LI1qzm9VDt/DFQyy0cb5sPJs5zw== X-Received: by 2002:a17:902:1a2:: with SMTP id b31-v6mr16760276plb.279.1533633182658; Tue, 07 Aug 2018 02:13:02 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.37]) by smtp.gmail.com with ESMTPSA id z4-v6sm2159645pfl.11.2018.08.07.02.12.58 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Aug 2018 02:13:02 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 7 Aug 2018 17:12:06 +0800 Message-Id: <20180807091209.13531-8-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180807091209.13531-1-xiaoguangrong@tencent.com> References: <20180807091209.13531-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::242 Subject: [Qemu-devel] [PATCH v3 07/10] migration: do not flush_compressed_data at the end of each iteration X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong flush_compressed_data() needs to wait all compression threads to finish their work, after that all threads are free until the migration feeds new request to them, reducing its call can improve the throughput and use CPU resource more effectively We do not need to flush all threads at the end of iteration, the data can be kept locally until the memory block is changed or memory migration starts over in that case we will meet a dirtied page which may still exists in compression threads's ring Signed-off-by: Xiao Guangrong --- migration/ram.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index 99ecf9b315..55966bc2c1 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -306,6 +306,8 @@ struct RAMState { uint64_t iterations; /* number of dirty bits in the bitmap */ uint64_t migration_dirty_pages; + /* last dirty_sync_count we have seen */ + uint64_t dirty_sync_count_prev; /* protects modification of the bitmap */ QemuMutex bitmap_mutex; /* The RAMBlock used in the last src_page_requests */ @@ -3173,6 +3175,17 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) ram_control_before_iterate(f, RAM_CONTROL_ROUND); + /* + * if memory migration starts over, we will meet a dirtied page which + * may still exists in compression threads's ring, so we should flush + * the compressed data to make sure the new page is not overwritten by + * the old one in the destination. + */ + if (ram_counters.dirty_sync_count != rs->dirty_sync_count_prev) { + rs->dirty_sync_count_prev = ram_counters.dirty_sync_count; + flush_compressed_data(rs); + } + t0 = qemu_clock_get_ns(QEMU_CLOCK_REALTIME); i = 0; while ((ret = qemu_file_rate_limit(f)) == 0 || @@ -3205,7 +3218,6 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) } i++; } - flush_compressed_data(rs); rcu_read_unlock(); /* From patchwork Tue Aug 7 09:12:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 954427 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="C+re795D"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41l8FV1LRZz9rxx for ; Tue, 7 Aug 2018 19:23:10 +1000 (AEST) Received: from localhost ([::1]:38066 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmyCp-000322-Ks for incoming@patchwork.ozlabs.org; Tue, 07 Aug 2018 05:23:07 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48576) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmy3C-0003Q6-Oq for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:13:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fmy39-0005Bk-Fd for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:13:10 -0400 Received: from mail-pg1-x544.google.com ([2607:f8b0:4864:20::544]:35842) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fmy39-0005Ba-8x for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:13:07 -0400 Received: by mail-pg1-x544.google.com with SMTP id h12-v6so7556892pgs.3 for ; Tue, 07 Aug 2018 02:13:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=jZSF6mo3a1QZs+hmAPVe+cRD2slhVUOS13Dutb4XB6E=; b=C+re795DD6qniafMAuLie4Bl0wImNO/6BNFccdfQdvHsHwQwnTaEadennIUEQ7q0f8 s6kaqrliirgg/lRdM8Ouviq3q20hks9w2C06cFemNDK0foTgeYPnuhUlgyV/bLHY55ke 5cR4iUYa3KnnQwpoZqaBttLTSBGXazrMHdYUYIUVTS5zWwS4xye1cJGft/BK/U/VYIlL 4G+x9Y1khdtTaAlJA5zbsJifkars5N1xn2c9G7RNyjc8BQRCBx1YX1ymg1EZMuWcVTzD a72ymg3+iFHveJO0Zkv/br1H3Xui2MeMDWJdarayIeRjZxc3i3SKO1/AIjIE+7p+YpsH zRSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=jZSF6mo3a1QZs+hmAPVe+cRD2slhVUOS13Dutb4XB6E=; b=TXMymzgmCeJe5XSJb1qGdxiAufIXqgbNFNaPXZhepc7yFuNYYGoa9Z4ETaW+qmYias DKynQmuHgT0nFOIBXNj8RWvJUlBqsIW/ePVPZRkfEHfPqWNGKe2wyphqyBF3Smfy+5+w XSudk5if7YeWZdaW9E72uo0edHF23ky+yTkJ3jNyKvrzwc7VgeSs2kerLEzM4fJz/2AP 1+ubf5d77vfLokMcTketfyWyANPTO/Pkzh7d5NV2drMaeJxG4EumH69N6V2iQVvNutsY JWjV6cAfX6c394YTSlYmPQ5LNKIyOrvQdvDj1E5u8oFHMg6f8RVF2PwEceHvlvg2w8Fi 31fA== X-Gm-Message-State: AOUpUlEBvyUQXNpK6dhecZ1EQ+xDegOOhi6hR3VNI8PCMteOHlRn1Jz9 qGsavggbcKdl72cWfKR7yHo= X-Google-Smtp-Source: AAOMgpfQx5+hsfvfSmYM9TuETrv2wkItB4NCHsnv1qLpqPwb8lUHsqK6qspq1sLspYSIpgoMKjut9w== X-Received: by 2002:a63:f919:: with SMTP id h25-v6mr17371541pgi.401.1533633186515; Tue, 07 Aug 2018 02:13:06 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.37]) by smtp.gmail.com with ESMTPSA id z4-v6sm2159645pfl.11.2018.08.07.02.13.02 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Aug 2018 02:13:06 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 7 Aug 2018 17:12:07 +0800 Message-Id: <20180807091209.13531-9-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180807091209.13531-1-xiaoguangrong@tencent.com> References: <20180807091209.13531-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::544 Subject: [Qemu-devel] [PATCH v3 08/10] migration: handle the error condition properly X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong ram_find_and_save_block() can return negative if any error hanppens, however, it is completely ignored in current code Signed-off-by: Xiao Guangrong --- migration/ram.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 55966bc2c1..09be01dca2 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2367,7 +2367,8 @@ static int ram_save_host_page(RAMState *rs, PageSearchStatus *pss, * * Called within an RCU critical section. * - * Returns the number of pages written where zero means no dirty pages + * Returns the number of pages written where zero means no dirty pages, + * or negative on error * * @rs: current RAM state * @last_stage: if we are at the completion stage @@ -3202,6 +3203,12 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) done = 1; break; } + + if (pages < 0) { + qemu_file_set_error(f, pages); + break; + } + rs->iterations++; /* we want to check in the 1st loop, just in case it was the 1st time @@ -3243,7 +3250,7 @@ out: /** * ram_save_complete: function called to send the remaining amount of ram * - * Returns zero to indicate success + * Returns zero to indicate success or negative on error * * Called with iothread lock * @@ -3254,6 +3261,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque) { RAMState **temp = opaque; RAMState *rs = *temp; + int ret = 0; rcu_read_lock(); @@ -3274,6 +3282,10 @@ static int ram_save_complete(QEMUFile *f, void *opaque) if (pages == 0) { break; } + if (pages < 0) { + ret = pages; + break; + } } flush_compressed_data(rs); @@ -3285,7 +3297,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque) qemu_put_be64(f, RAM_SAVE_FLAG_EOS); qemu_fflush(f); - return 0; + return ret; } static void ram_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, From patchwork Tue Aug 7 09:12:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 954422 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="QFp1QJRD"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41l88r6GH9z9rxx for ; Tue, 7 Aug 2018 19:19:08 +1000 (AEST) Received: from localhost ([::1]:38045 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmy8w-0007uM-EM for incoming@patchwork.ozlabs.org; Tue, 07 Aug 2018 05:19:06 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48588) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmy3E-0003Rf-Dn for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:13:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fmy3D-0005Cl-Cs for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:13:12 -0400 Received: from mail-pl0-x244.google.com ([2607:f8b0:400e:c01::244]:42158) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fmy3D-0005Cb-62 for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:13:11 -0400 Received: by mail-pl0-x244.google.com with SMTP id g6-v6so3295886plq.9 for ; Tue, 07 Aug 2018 02:13:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=64IrWQDQ1LSSbayQ3BFrpDYaoZU6o3Q5eD775BnwU4c=; b=QFp1QJRD8h/DSJUSOkIpb7xW3ZzD8Pyt3KO1e0BLE3/bVgzRGLOE14Xeiz6VzUL9Ne z1Htsw2w9kQnImNMreXRy/+xbKbLLzoYin0yMllpHsfXqq/rGXpcOXkmfhJh1s869WS4 grayqI/xWSKmZYRPey9+M7SD95TD+yb02z/JFvkdQ90svPmCfjJ6kZ4CPMkzl+71JC5u lMUCe3XOTcxSHI6H8ydeMTYqZcwzBjpTXfIGY4M7JkfXx0ODV50lFE5f7Y0BHu+1AFcb QY98/We2acJ26B/PxMJcMHUJWcXlionQXkKCwksmYhJeT7yV9XjlLeFvHY4DXNekyxLS bE7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=64IrWQDQ1LSSbayQ3BFrpDYaoZU6o3Q5eD775BnwU4c=; b=PkOiIVD9nfz5i/R9UnsQke36exFY4Ro3r7dlQ2yZCDp0+W/k3RALViBsyS1uaDOilJ Qs5tLvTfKq2QslzHN8DUfJusui4zSOOEtcLLy/z3uPJtHaHiJ6cT+6zFxWDM5OM+f0Yw R+KuTtJVz3ArWhnUgxsN0HaO04P4eqcR4elgV7/U72eHg+DG2Q82rWTfQLinZ6lQcPoX BsvI86WgRoilgx9qE0lLpH6vYHRTiETfETppWNTHCaFo+3JPQ5ZyD7ZmC+as972V4Ozw YjO4t641uOFO1fsXdgE3cesoOCmtdMa/tKHg8BNJAaOp5TtwPx0G+O6JaDBnivVRUXeB dkfw== X-Gm-Message-State: AOUpUlEo7VLjmZVwAAO1pg5l63OkmSJpJ5VDyqi4cuwmw+olQZC34pr7 709K2ZmlwPyOW5YNPfiQi1Q= X-Google-Smtp-Source: AAOMgpddQ3dYn2GuVwXOh8UAT2gc47dy87w/lhh6lToWDP2LPYL9FZcwWLv3r+xxOz7h5N61pOvU9Q== X-Received: by 2002:a17:902:6b46:: with SMTP id g6-v6mr17488145plt.251.1533633190426; Tue, 07 Aug 2018 02:13:10 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.37]) by smtp.gmail.com with ESMTPSA id z4-v6sm2159645pfl.11.2018.08.07.02.13.06 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Aug 2018 02:13:09 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 7 Aug 2018 17:12:08 +0800 Message-Id: <20180807091209.13531-10-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180807091209.13531-1-xiaoguangrong@tencent.com> References: <20180807091209.13531-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::244 Subject: [Qemu-devel] [PATCH v3 09/10] migration: fix calculating xbzrle_counters.cache_miss_rate X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong As Peter pointed out: | - xbzrle_counters.cache_miss is done in save_xbzrle_page(), so it's | per-guest-page granularity | | - RAMState.iterations is done for each ram_find_and_save_block(), so | it's per-host-page granularity | | An example is that when we migrate a 2M huge page in the guest, we | will only increase the RAMState.iterations by 1 (since | ram_find_and_save_block() will be called once), but we might increase | xbzrle_counters.cache_miss for 2M/4K=512 times (we'll call | save_xbzrle_page() that many times) if all the pages got cache miss. | Then IMHO the cache miss rate will be 512/1=51200% (while it should | actually be just 100% cache miss). And he also suggested as xbzrle_counters.cache_miss_rate is the only user of rs->iterations we can adapt it to count guest page numbers After that, rename 'iterations' to 'handle_pages' to better reflect its meaning Suggested-by: Peter Xu Signed-off-by: Xiao Guangrong Reviewed-by: Peter Xu --- migration/ram.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 09be01dca2..bd7c18d1f9 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -300,10 +300,10 @@ struct RAMState { uint64_t num_dirty_pages_period; /* xbzrle misses since the beginning of the period */ uint64_t xbzrle_cache_miss_prev; - /* number of iterations at the beginning of period */ - uint64_t iterations_prev; - /* Iterations since start */ - uint64_t iterations; + /* total handled pages at the beginning of period */ + uint64_t handle_pages_prev; + /* total handled pages since start */ + uint64_t handle_pages; /* number of dirty bits in the bitmap */ uint64_t migration_dirty_pages; /* last dirty_sync_count we have seen */ @@ -1587,19 +1587,19 @@ uint64_t ram_pagesize_summary(void) static void migration_update_rates(RAMState *rs, int64_t end_time) { - uint64_t iter_count = rs->iterations - rs->iterations_prev; + uint64_t page_count = rs->handle_pages - rs->handle_pages_prev; /* calculate period counters */ ram_counters.dirty_pages_rate = rs->num_dirty_pages_period * 1000 / (end_time - rs->time_last_bitmap_sync); - if (!iter_count) { + if (!page_count) { return; } if (migrate_use_xbzrle()) { xbzrle_counters.cache_miss_rate = (double)(xbzrle_counters.cache_miss - - rs->xbzrle_cache_miss_prev) / iter_count; + rs->xbzrle_cache_miss_prev) / page_count; rs->xbzrle_cache_miss_prev = xbzrle_counters.cache_miss; } } @@ -1657,7 +1657,7 @@ static void migration_bitmap_sync(RAMState *rs) migration_update_rates(rs, end_time); - rs->iterations_prev = rs->iterations; + rs->handle_pages_prev = rs->handle_pages; /* reset period counters */ rs->time_last_bitmap_sync = end_time; @@ -3209,7 +3209,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) break; } - rs->iterations++; + rs->handle_pages += pages; /* we want to check in the 1st loop, just in case it was the 1st time and we had to sync the dirty bitmap. From patchwork Tue Aug 7 09:12:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 954426 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="s/nrOhg/"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41l8CS06jzz9rxx for ; Tue, 7 Aug 2018 19:21:24 +1000 (AEST) Received: from localhost ([::1]:38059 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmyB7-0001nq-Jq for incoming@patchwork.ozlabs.org; Tue, 07 Aug 2018 05:21:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48608) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmy3I-0003Vy-QF for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:13:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fmy3H-0005Di-Dv for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:13:16 -0400 Received: from mail-pf1-x442.google.com ([2607:f8b0:4864:20::442]:37808) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fmy3H-0005DW-4m for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:13:15 -0400 Received: by mail-pf1-x442.google.com with SMTP id a26-v6so8279785pfo.4 for ; Tue, 07 Aug 2018 02:13:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ZhatVTBN/1SlmZUKm+fn4hlZQqccFIDYuJ7ByOMlGLk=; b=s/nrOhg/3u01F8JUuU7qxKqPzCP5S9z7s90EIxXyfKaohajK2WZ7YbjMAhvIc6RX6W 9bj/0q3KMlHtzS59dK2GmMfj0QoPcIKxmXHwE57fn5dLswbD8XLnrdD84p07SMXpc/Js 5CARA40LLbpbYtq6b9utJiSUcEVz2GOyVt69tztMNr/dvM1ejoy14gG5W4YIkE74UWKd UjTHYyYGKtTsaynvn0zmU0/HIpmQxWNABrumTEohb/ffJ0aYGGcEWCtxtCS+kKtGSsmR 6BS20iej17ro9EQheVth0kstijS9i7cn8sOn401pOBQQ/OZyZllqge6H6tctDGR8rAIA aPIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ZhatVTBN/1SlmZUKm+fn4hlZQqccFIDYuJ7ByOMlGLk=; b=rn9glPHbjlxkGYPPSTJBhEpzfM6WXeFBmnUqb7zZmbctie2ad8s1v9V3R2c9rqjT4n RSvDWnAA46nE3vUZPvWEMAOvC/4GPkrZI0BFd6dXQOZqBOk0+01nE61sIS1Ojcdi22N4 t/CckmEfMDfeMzrHHQlh6hULP7XUv2R2z2mdknfucHaSfEcfsC0XfGSl7br732UFpvg9 Q444latUpvjqfGr97zFMnYsBmvQpXlGQZ/VNVBSCDFtzjdIeR2J4cKvQl0Vtf+Z9hi2S 61Cq6IW9ZXD/uIq4TodQ/npWZ6f7jXSoMV88hJG8EJERzrYA/32u9vZN/cS2BITIPYFw Wcuw== X-Gm-Message-State: AOUpUlFeZSTthtQNYByK9QCHxKaJf16Vy82ZbHFEBnOF8jbLi23TaUtf ImbW8SBB8Qi6jL+be0OLmJQ= X-Google-Smtp-Source: AAOMgpdCMU9ZLAp56FAqht2lAm0mtULvnN79RFwm9qB2QGM+hOMHBUYJeZzPIn0oHZOX8pFp16B6ng== X-Received: by 2002:a63:a5c:: with SMTP id z28-v6mr17694727pgk.209.1533633194334; Tue, 07 Aug 2018 02:13:14 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.37]) by smtp.gmail.com with ESMTPSA id z4-v6sm2159645pfl.11.2018.08.07.02.13.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Aug 2018 02:13:13 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 7 Aug 2018 17:12:09 +0800 Message-Id: <20180807091209.13531-11-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180807091209.13531-1-xiaoguangrong@tencent.com> References: <20180807091209.13531-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::442 Subject: [Qemu-devel] [PATCH v3 10/10] migration: show the statistics of compression X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong Currently, it includes: pages: amount of pages compressed and transferred to the target VM busy: amount of count that no free thread to compress data busy-rate: rate of thread busy compressed-size: amount of bytes after compression compression-rate: rate of compressed size Signed-off-by: Xiao Guangrong --- hmp.c | 13 +++++++++++++ migration/migration.c | 12 ++++++++++++ migration/ram.c | 41 ++++++++++++++++++++++++++++++++++++++++- migration/ram.h | 1 + qapi/migration.json | 26 +++++++++++++++++++++++++- 5 files changed, 91 insertions(+), 2 deletions(-) diff --git a/hmp.c b/hmp.c index 47d36e3ccf..e76e45e672 100644 --- a/hmp.c +++ b/hmp.c @@ -271,6 +271,19 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict) info->xbzrle_cache->overflow); } + if (info->has_compression) { + monitor_printf(mon, "compression pages: %" PRIu64 " pages\n", + info->compression->pages); + monitor_printf(mon, "compression busy: %" PRIu64 "\n", + info->compression->busy); + monitor_printf(mon, "compression busy rate: %0.2f\n", + info->compression->busy_rate); + monitor_printf(mon, "compressed size: %" PRIu64 "\n", + info->compression->compressed_size); + monitor_printf(mon, "compression rate: %0.2f\n", + info->compression->compression_rate); + } + if (info->has_cpu_throttle_percentage) { monitor_printf(mon, "cpu throttle percentage: %" PRIu64 "\n", info->cpu_throttle_percentage); diff --git a/migration/migration.c b/migration/migration.c index 2ccaadc03d..4da0a20275 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -754,6 +754,18 @@ static void populate_ram_info(MigrationInfo *info, MigrationState *s) info->xbzrle_cache->overflow = xbzrle_counters.overflow; } + if (migrate_use_compression()) { + info->has_compression = true; + info->compression = g_malloc0(sizeof(*info->compression)); + info->compression->pages = compression_counters.pages; + info->compression->busy = compression_counters.busy; + info->compression->busy_rate = compression_counters.busy_rate; + info->compression->compressed_size = + compression_counters.compressed_size; + info->compression->compression_rate = + compression_counters.compression_rate; + } + if (cpu_throttle_active()) { info->has_cpu_throttle_percentage = true; info->cpu_throttle_percentage = cpu_throttle_get_percentage(); diff --git a/migration/ram.c b/migration/ram.c index bd7c18d1f9..d1cb453e53 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -301,6 +301,15 @@ struct RAMState { /* xbzrle misses since the beginning of the period */ uint64_t xbzrle_cache_miss_prev; /* total handled pages at the beginning of period */ + + /* compression statistics since the beginning of the period */ + /* amount of count that no free thread to compress data */ + uint64_t compress_thread_busy_prev; + /* amount bytes after compression */ + uint64_t compressed_size_prev; + /* amount of compressed pages */ + uint64_t compress_pages_prev; + uint64_t handle_pages_prev; /* total handled pages since start */ uint64_t handle_pages; @@ -339,6 +348,8 @@ struct PageSearchStatus { }; typedef struct PageSearchStatus PageSearchStatus; +CompressionStats compression_counters; + struct CompressParam { bool done; bool quit; @@ -1588,6 +1599,7 @@ uint64_t ram_pagesize_summary(void) static void migration_update_rates(RAMState *rs, int64_t end_time) { uint64_t page_count = rs->handle_pages - rs->handle_pages_prev; + double compressed_size; /* calculate period counters */ ram_counters.dirty_pages_rate = rs->num_dirty_pages_period * 1000 @@ -1602,6 +1614,26 @@ static void migration_update_rates(RAMState *rs, int64_t end_time) rs->xbzrle_cache_miss_prev) / page_count; rs->xbzrle_cache_miss_prev = xbzrle_counters.cache_miss; } + + if (migrate_use_compression()) { + compression_counters.busy_rate = (double)(compression_counters.busy - + rs->compress_thread_busy_prev) / page_count; + rs->compress_thread_busy_prev = compression_counters.busy; + + compressed_size = compression_counters.compressed_size - + rs->compressed_size_prev; + if (compressed_size) { + double uncompressed_size = (compression_counters.pages - + rs->compress_pages_prev) * TARGET_PAGE_SIZE; + + /* Compression-Ratio = Uncompressed-size / Compressed-size */ + compression_counters.compression_rate = + uncompressed_size / compressed_size; + + rs->compress_pages_prev = compression_counters.pages; + rs->compressed_size_prev = compression_counters.compressed_size; + } + } } static void migration_bitmap_sync(RAMState *rs) @@ -1883,10 +1915,16 @@ exit: static void update_compress_thread_counts(const CompressParam *param, int bytes_xmit) { + ram_counters.transferred += bytes_xmit; + if (param->zero_page) { ram_counters.duplicate++; + return; } - ram_counters.transferred += bytes_xmit; + + /* 8 means a header with RAM_SAVE_FLAG_CONTINUE. */ + compression_counters.compressed_size += bytes_xmit - 8; + compression_counters.pages++; } static void flush_compressed_data(RAMState *rs) @@ -2254,6 +2292,7 @@ static bool save_compress_page(RAMState *rs, RAMBlock *block, ram_addr_t offset) return true; } + compression_counters.busy++; return false; } diff --git a/migration/ram.h b/migration/ram.h index 457bf54b8c..a139066846 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -36,6 +36,7 @@ extern MigrationStats ram_counters; extern XBZRLECacheStats xbzrle_counters; +extern CompressionStats compression_counters; int xbzrle_cache_resize(int64_t new_size, Error **errp); uint64_t ram_bytes_remaining(void); diff --git a/qapi/migration.json b/qapi/migration.json index e6991fcbd2..69e1510429 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -75,6 +75,27 @@ 'cache-miss': 'int', 'cache-miss-rate': 'number', 'overflow': 'int' } } +## +# @CompressionStats: +# +# Detailed migration compression statistics +# +# @pages: amount of pages compressed and transferred to the target VM +# +# @busy: count of times that no free thread was available to compress data +# +# @busy-rate: rate of thread busy +# +# @compressed-size: amount of bytes after compression +# +# @compression-rate: rate of compressed size +# +# Since: 3.1 +## +{ 'struct': 'CompressionStats', + 'data': {'pages': 'int', 'busy': 'int', 'busy-rate': 'number', + 'compressed-size': 'int', 'compression-rate': 'number' } } + ## # @MigrationStatus: # @@ -172,6 +193,8 @@ # only present when the postcopy-blocktime migration capability # is enabled. (Since 3.0) # +# @compression: migration compression statistics, only returned if compression +# feature is on and status is 'active' or 'completed' (Since 3.1) # # Since: 0.14.0 ## @@ -186,7 +209,8 @@ '*cpu-throttle-percentage': 'int', '*error-desc': 'str', '*postcopy-blocktime' : 'uint32', - '*postcopy-vcpu-blocktime': ['uint32']} } + '*postcopy-vcpu-blocktime': ['uint32'], + '*compression': 'CompressionStats'} } ## # @query-migrate: