From patchwork Tue Aug 7 09:12:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 954426 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="s/nrOhg/"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41l8CS06jzz9rxx for ; Tue, 7 Aug 2018 19:21:24 +1000 (AEST) Received: from localhost ([::1]:38059 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmyB7-0001nq-Jq for incoming@patchwork.ozlabs.org; Tue, 07 Aug 2018 05:21:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48608) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmy3I-0003Vy-QF for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:13:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fmy3H-0005Di-Dv for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:13:16 -0400 Received: from mail-pf1-x442.google.com ([2607:f8b0:4864:20::442]:37808) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fmy3H-0005DW-4m for qemu-devel@nongnu.org; Tue, 07 Aug 2018 05:13:15 -0400 Received: by mail-pf1-x442.google.com with SMTP id a26-v6so8279785pfo.4 for ; Tue, 07 Aug 2018 02:13:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ZhatVTBN/1SlmZUKm+fn4hlZQqccFIDYuJ7ByOMlGLk=; b=s/nrOhg/3u01F8JUuU7qxKqPzCP5S9z7s90EIxXyfKaohajK2WZ7YbjMAhvIc6RX6W 9bj/0q3KMlHtzS59dK2GmMfj0QoPcIKxmXHwE57fn5dLswbD8XLnrdD84p07SMXpc/Js 5CARA40LLbpbYtq6b9utJiSUcEVz2GOyVt69tztMNr/dvM1ejoy14gG5W4YIkE74UWKd UjTHYyYGKtTsaynvn0zmU0/HIpmQxWNABrumTEohb/ffJ0aYGGcEWCtxtCS+kKtGSsmR 6BS20iej17ro9EQheVth0kstijS9i7cn8sOn401pOBQQ/OZyZllqge6H6tctDGR8rAIA aPIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ZhatVTBN/1SlmZUKm+fn4hlZQqccFIDYuJ7ByOMlGLk=; b=rn9glPHbjlxkGYPPSTJBhEpzfM6WXeFBmnUqb7zZmbctie2ad8s1v9V3R2c9rqjT4n RSvDWnAA46nE3vUZPvWEMAOvC/4GPkrZI0BFd6dXQOZqBOk0+01nE61sIS1Ojcdi22N4 t/CckmEfMDfeMzrHHQlh6hULP7XUv2R2z2mdknfucHaSfEcfsC0XfGSl7br732UFpvg9 Q444latUpvjqfGr97zFMnYsBmvQpXlGQZ/VNVBSCDFtzjdIeR2J4cKvQl0Vtf+Z9hi2S 61Cq6IW9ZXD/uIq4TodQ/npWZ6f7jXSoMV88hJG8EJERzrYA/32u9vZN/cS2BITIPYFw Wcuw== X-Gm-Message-State: AOUpUlFeZSTthtQNYByK9QCHxKaJf16Vy82ZbHFEBnOF8jbLi23TaUtf ImbW8SBB8Qi6jL+be0OLmJQ= X-Google-Smtp-Source: AAOMgpdCMU9ZLAp56FAqht2lAm0mtULvnN79RFwm9qB2QGM+hOMHBUYJeZzPIn0oHZOX8pFp16B6ng== X-Received: by 2002:a63:a5c:: with SMTP id z28-v6mr17694727pgk.209.1533633194334; Tue, 07 Aug 2018 02:13:14 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.37]) by smtp.gmail.com with ESMTPSA id z4-v6sm2159645pfl.11.2018.08.07.02.13.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Aug 2018 02:13:13 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 7 Aug 2018 17:12:09 +0800 Message-Id: <20180807091209.13531-11-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180807091209.13531-1-xiaoguangrong@tencent.com> References: <20180807091209.13531-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::442 Subject: [Qemu-devel] [PATCH v3 10/10] migration: show the statistics of compression X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong Currently, it includes: pages: amount of pages compressed and transferred to the target VM busy: amount of count that no free thread to compress data busy-rate: rate of thread busy compressed-size: amount of bytes after compression compression-rate: rate of compressed size Signed-off-by: Xiao Guangrong --- hmp.c | 13 +++++++++++++ migration/migration.c | 12 ++++++++++++ migration/ram.c | 41 ++++++++++++++++++++++++++++++++++++++++- migration/ram.h | 1 + qapi/migration.json | 26 +++++++++++++++++++++++++- 5 files changed, 91 insertions(+), 2 deletions(-) diff --git a/hmp.c b/hmp.c index 47d36e3ccf..e76e45e672 100644 --- a/hmp.c +++ b/hmp.c @@ -271,6 +271,19 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict) info->xbzrle_cache->overflow); } + if (info->has_compression) { + monitor_printf(mon, "compression pages: %" PRIu64 " pages\n", + info->compression->pages); + monitor_printf(mon, "compression busy: %" PRIu64 "\n", + info->compression->busy); + monitor_printf(mon, "compression busy rate: %0.2f\n", + info->compression->busy_rate); + monitor_printf(mon, "compressed size: %" PRIu64 "\n", + info->compression->compressed_size); + monitor_printf(mon, "compression rate: %0.2f\n", + info->compression->compression_rate); + } + if (info->has_cpu_throttle_percentage) { monitor_printf(mon, "cpu throttle percentage: %" PRIu64 "\n", info->cpu_throttle_percentage); diff --git a/migration/migration.c b/migration/migration.c index 2ccaadc03d..4da0a20275 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -754,6 +754,18 @@ static void populate_ram_info(MigrationInfo *info, MigrationState *s) info->xbzrle_cache->overflow = xbzrle_counters.overflow; } + if (migrate_use_compression()) { + info->has_compression = true; + info->compression = g_malloc0(sizeof(*info->compression)); + info->compression->pages = compression_counters.pages; + info->compression->busy = compression_counters.busy; + info->compression->busy_rate = compression_counters.busy_rate; + info->compression->compressed_size = + compression_counters.compressed_size; + info->compression->compression_rate = + compression_counters.compression_rate; + } + if (cpu_throttle_active()) { info->has_cpu_throttle_percentage = true; info->cpu_throttle_percentage = cpu_throttle_get_percentage(); diff --git a/migration/ram.c b/migration/ram.c index bd7c18d1f9..d1cb453e53 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -301,6 +301,15 @@ struct RAMState { /* xbzrle misses since the beginning of the period */ uint64_t xbzrle_cache_miss_prev; /* total handled pages at the beginning of period */ + + /* compression statistics since the beginning of the period */ + /* amount of count that no free thread to compress data */ + uint64_t compress_thread_busy_prev; + /* amount bytes after compression */ + uint64_t compressed_size_prev; + /* amount of compressed pages */ + uint64_t compress_pages_prev; + uint64_t handle_pages_prev; /* total handled pages since start */ uint64_t handle_pages; @@ -339,6 +348,8 @@ struct PageSearchStatus { }; typedef struct PageSearchStatus PageSearchStatus; +CompressionStats compression_counters; + struct CompressParam { bool done; bool quit; @@ -1588,6 +1599,7 @@ uint64_t ram_pagesize_summary(void) static void migration_update_rates(RAMState *rs, int64_t end_time) { uint64_t page_count = rs->handle_pages - rs->handle_pages_prev; + double compressed_size; /* calculate period counters */ ram_counters.dirty_pages_rate = rs->num_dirty_pages_period * 1000 @@ -1602,6 +1614,26 @@ static void migration_update_rates(RAMState *rs, int64_t end_time) rs->xbzrle_cache_miss_prev) / page_count; rs->xbzrle_cache_miss_prev = xbzrle_counters.cache_miss; } + + if (migrate_use_compression()) { + compression_counters.busy_rate = (double)(compression_counters.busy - + rs->compress_thread_busy_prev) / page_count; + rs->compress_thread_busy_prev = compression_counters.busy; + + compressed_size = compression_counters.compressed_size - + rs->compressed_size_prev; + if (compressed_size) { + double uncompressed_size = (compression_counters.pages - + rs->compress_pages_prev) * TARGET_PAGE_SIZE; + + /* Compression-Ratio = Uncompressed-size / Compressed-size */ + compression_counters.compression_rate = + uncompressed_size / compressed_size; + + rs->compress_pages_prev = compression_counters.pages; + rs->compressed_size_prev = compression_counters.compressed_size; + } + } } static void migration_bitmap_sync(RAMState *rs) @@ -1883,10 +1915,16 @@ exit: static void update_compress_thread_counts(const CompressParam *param, int bytes_xmit) { + ram_counters.transferred += bytes_xmit; + if (param->zero_page) { ram_counters.duplicate++; + return; } - ram_counters.transferred += bytes_xmit; + + /* 8 means a header with RAM_SAVE_FLAG_CONTINUE. */ + compression_counters.compressed_size += bytes_xmit - 8; + compression_counters.pages++; } static void flush_compressed_data(RAMState *rs) @@ -2254,6 +2292,7 @@ static bool save_compress_page(RAMState *rs, RAMBlock *block, ram_addr_t offset) return true; } + compression_counters.busy++; return false; } diff --git a/migration/ram.h b/migration/ram.h index 457bf54b8c..a139066846 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -36,6 +36,7 @@ extern MigrationStats ram_counters; extern XBZRLECacheStats xbzrle_counters; +extern CompressionStats compression_counters; int xbzrle_cache_resize(int64_t new_size, Error **errp); uint64_t ram_bytes_remaining(void); diff --git a/qapi/migration.json b/qapi/migration.json index e6991fcbd2..69e1510429 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -75,6 +75,27 @@ 'cache-miss': 'int', 'cache-miss-rate': 'number', 'overflow': 'int' } } +## +# @CompressionStats: +# +# Detailed migration compression statistics +# +# @pages: amount of pages compressed and transferred to the target VM +# +# @busy: count of times that no free thread was available to compress data +# +# @busy-rate: rate of thread busy +# +# @compressed-size: amount of bytes after compression +# +# @compression-rate: rate of compressed size +# +# Since: 3.1 +## +{ 'struct': 'CompressionStats', + 'data': {'pages': 'int', 'busy': 'int', 'busy-rate': 'number', + 'compressed-size': 'int', 'compression-rate': 'number' } } + ## # @MigrationStatus: # @@ -172,6 +193,8 @@ # only present when the postcopy-blocktime migration capability # is enabled. (Since 3.0) # +# @compression: migration compression statistics, only returned if compression +# feature is on and status is 'active' or 'completed' (Since 3.1) # # Since: 0.14.0 ## @@ -186,7 +209,8 @@ '*cpu-throttle-percentage': 'int', '*error-desc': 'str', '*postcopy-blocktime' : 'uint32', - '*postcopy-vcpu-blocktime': ['uint32']} } + '*postcopy-vcpu-blocktime': ['uint32'], + '*compression': 'CompressionStats'} } ## # @query-migrate: