From patchwork Mon Sep 3 09:26:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 965319 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="ZHAWZMkm"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 423l475P1mz9s3C for ; Mon, 3 Sep 2018 19:27:35 +1000 (AEST) Received: from localhost ([::1]:44249 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fwl8v-000781-B9 for incoming@patchwork.ozlabs.org; Mon, 03 Sep 2018 05:27:33 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39676) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fwl8M-00075U-Ji for qemu-devel@nongnu.org; Mon, 03 Sep 2018 05:26:59 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fwl8L-00042A-HS for qemu-devel@nongnu.org; Mon, 03 Sep 2018 05:26:58 -0400 Received: from mail-pl1-x641.google.com ([2607:f8b0:4864:20::641]:33930) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fwl8L-00041v-As for qemu-devel@nongnu.org; Mon, 03 Sep 2018 05:26:57 -0400 Received: by mail-pl1-x641.google.com with SMTP id f6-v6so8400555plo.1 for ; Mon, 03 Sep 2018 02:26:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=i7/Q4JsMLeh0YBIjVpnpDZhWPuGe41yrvKr/wJ5inLI=; b=ZHAWZMkmQ+X1gSpJHiJJdNHcNmSjwuhCe4kYLn2jYqPZvGQoYeNTr6Wga9bF96nf3i JbxKOaxtOsefmml6c23/bamlOWt0d6ecSxaa5ZhGIuf3gPYxqNn5GGZsRVbyt3Vn3+AM +irwxvZpqM8MlxQt3DS3FMkcPaDa/noMisySLE5zE6qym/0GzbsvZpk1AgwqCinTP4Jr efjtM53C3vNsxrhVakbUdXhv0fL7WVT5tbWIKWt0dG0O5/YMRKo/k11ATeU6p9rhqg5S yulM22Qb8kZgwt0wY/hmoI9ePVY021Wp1ekE23BP9Fua44eHE6SLEPnF3+7HGYQCpxR2 RCvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=i7/Q4JsMLeh0YBIjVpnpDZhWPuGe41yrvKr/wJ5inLI=; b=UKM9VkP+RHiXPa8WHHQVgiSOjF9XG8qq/6d7AVs+Q4MG9IQyeDGZ4PVAH64H/GOHag NgsHZp5kbA/mF9OJTWWf/vd64r9drWKEoftBM23X/2XC9s4JlyLAUth9JsMMVLkP1nS8 SYIuNH5fnJJ+hM6ecrIltC5aGvNhQMsN2MYiTjO/zJbFmHgIBmdaxWdqwVitqeG72cM4 vAfJBeO8GXTty7fba1QAiKsKytA3R7CaMf45PrlwNPP3fvco7OKAVRTzgbzq+0boiLut l04v/EFaCwn2gAd2KcwXCdy2c81aae6uMk5VM6JtashZEmw52BcUCSm3PFabR0Ym584x o9NA== X-Gm-Message-State: APzg51AbOy1q893z4NcmzVy3Pu1MxCgaXzqCjeuShPtgW1TCRelaBeNu +7dGA9iZRBJ+LQVK0E9MwIg= X-Google-Smtp-Source: ANB0VdYACEq+NN/sX1vsrzYjv1ITfYyVQCK2T23L/l+ol3pcWi2cF7JgGYywoUzu+Lhfuj0rko7lUg== X-Received: by 2002:a17:902:820a:: with SMTP id x10-v6mr27467039pln.261.1535966816510; Mon, 03 Sep 2018 02:26:56 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.40]) by smtp.gmail.com with ESMTPSA id 3-v6sm33543102pfq.10.2018.09.03.02.26.52 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 03 Sep 2018 02:26:56 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Mon, 3 Sep 2018 17:26:41 +0800 Message-Id: <20180903092644.25812-2-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180903092644.25812-1-xiaoguangrong@tencent.com> References: <20180903092644.25812-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::641 Subject: [Qemu-devel] [PATCH v5 1/4] migration: do not flush_compressed_data at the end of each iteration X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, quintela@redhat.com, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong flush_compressed_data() needs to wait all compression threads to finish their work, after that all threads are free until the migration feeds new request to them, reducing its call can improve the throughput and use CPU resource more effectively We do not need to flush all threads at the end of iteration, the data can be kept locally until the memory block is changed or memory migration starts over in that case we will meet a dirtied page which may still exists in compression threads's ring Signed-off-by: Xiao Guangrong --- migration/ram.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index 79c89425a3..2ad07b5e15 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -307,6 +307,8 @@ struct RAMState { uint64_t iterations; /* number of dirty bits in the bitmap */ uint64_t migration_dirty_pages; + /* last dirty_sync_count we have seen */ + uint64_t dirty_sync_count; /* protects modification of the bitmap */ QemuMutex bitmap_mutex; /* The RAMBlock used in the last src_page_requests */ @@ -3180,6 +3182,17 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) ram_control_before_iterate(f, RAM_CONTROL_ROUND); + /* + * if memory migration starts over, we will meet a dirtied page which + * may still exists in compression threads's ring, so we should flush + * the compressed data to make sure the new page is not overwritten by + * the old one in the destination. + */ + if (ram_counters.dirty_sync_count != rs->dirty_sync_count) { + rs->dirty_sync_count = ram_counters.dirty_sync_count; + flush_compressed_data(rs); + } + t0 = qemu_clock_get_ns(QEMU_CLOCK_REALTIME); i = 0; while ((ret = qemu_file_rate_limit(f)) == 0 || @@ -3212,7 +3225,6 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) } i++; } - flush_compressed_data(rs); rcu_read_unlock(); /* From patchwork Mon Sep 3 09:26:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 965320 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="eLBi4gZg"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 423l6V5pcxz9s55 for ; Mon, 3 Sep 2018 19:29:38 +1000 (AEST) Received: from localhost ([::1]:44254 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fwlAu-0001y5-Fg for incoming@patchwork.ozlabs.org; Mon, 03 Sep 2018 05:29:36 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39711) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fwl8Q-00078W-IG for qemu-devel@nongnu.org; Mon, 03 Sep 2018 05:27:03 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fwl8P-00044i-J0 for qemu-devel@nongnu.org; Mon, 03 Sep 2018 05:27:02 -0400 Received: from mail-pf1-x442.google.com ([2607:f8b0:4864:20::442]:40829) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fwl8P-00044Z-CX for qemu-devel@nongnu.org; Mon, 03 Sep 2018 05:27:01 -0400 Received: by mail-pf1-x442.google.com with SMTP id s13-v6so8451122pfi.7 for ; Mon, 03 Sep 2018 02:27:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=E4iFB29osfxzkuTgHYrtMGQRw1k89sq8DvMO5tpJsPg=; b=eLBi4gZghBvQFff6eAktAHL9PJnxtCf12EUPm55jltnERw+J5SRXnpTxZ2yYgfW4Rh eyXGempVvbZXm17gLBYdH0ABC9qzWSsEGE6wzpvfUoElfpA2KFPESMhdy/7EHEQ3biGr oFkZciqYsIQ5+LyV0RcvyaRfeCZeX1ZORQ5duLiTMdjU0WN5ParXfseIoKGishv9ZJBt 7OHDiraEYhc/AiQ/CtsqrqKKMavDQqE4Q8t0zdAVp5Bz1mvVr627pDrfyTzvHoaq40iB OhrntoRGFC2tZPgws/Xg4dfLj0VTQSGV7scJZASbnzEMq/ZQlNqWsuvZGbog5yaIhxV4 20xQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=E4iFB29osfxzkuTgHYrtMGQRw1k89sq8DvMO5tpJsPg=; b=If7kxiEC/zqBs30yK4yPV1Bqes4XTbicc/9x2JsbM0LXfCCL3NrOuCLtCmc0qFwTHl XDIAgEQbAdlUfpccRagPnmeCx9mnhZ78aGeKIT2uHWZdoAwFn3RzXlRK9VXO2h7951qN VpkzhAeEERTyhu4KWU7hxY5tHkrOUAjrAgCEitwbthBXg7UyYo6d3vi2Tst6EVZugAM6 E+F+hbMEejfRxKrFzhgS38xlWXyV6osuOhA26tEZY3V5E9/P6OWsF2s1CHTPVzlefn/I eT4c+xjtOO1hwonFUOoP8MDWeKPmv/0ivY+3ALux8OksPPfIkCeVB5nrtbSbD8tkhAuO eK7A== X-Gm-Message-State: APzg51CKgB9GRgjvYZEwgx9CF3hgiaUQidPGXryT9PPweowIMFxUW3Os ggdm6/m1Wp3TbvOpKZR+2n4= X-Google-Smtp-Source: ANB0VdaHG69sRDefiHAaTDacoUQyG4GBHlWH2QwEpmH3xwzr5f1OxZluHwbPvwv+8xECQxGBKd6uIw== X-Received: by 2002:a63:6ce:: with SMTP id 197-v6mr25018713pgg.338.1535966820521; Mon, 03 Sep 2018 02:27:00 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.40]) by smtp.gmail.com with ESMTPSA id 3-v6sm33543102pfq.10.2018.09.03.02.26.56 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 03 Sep 2018 02:27:00 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Mon, 3 Sep 2018 17:26:42 +0800 Message-Id: <20180903092644.25812-3-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180903092644.25812-1-xiaoguangrong@tencent.com> References: <20180903092644.25812-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::442 Subject: [Qemu-devel] [PATCH v5 2/4] migration: fix calculating xbzrle_counters.cache_miss_rate X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, quintela@redhat.com, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong As Peter pointed out: | - xbzrle_counters.cache_miss is done in save_xbzrle_page(), so it's | per-guest-page granularity | | - RAMState.iterations is done for each ram_find_and_save_block(), so | it's per-host-page granularity | | An example is that when we migrate a 2M huge page in the guest, we | will only increase the RAMState.iterations by 1 (since | ram_find_and_save_block() will be called once), but we might increase | xbzrle_counters.cache_miss for 2M/4K=512 times (we'll call | save_xbzrle_page() that many times) if all the pages got cache miss. | Then IMHO the cache miss rate will be 512/1=51200% (while it should | actually be just 100% cache miss). And he also suggested as xbzrle_counters.cache_miss_rate is the only user of rs->iterations we can adapt it to count target guest page numbers After that, rename 'iterations' to 'target_page_count' to better reflect its meaning Suggested-by: Peter Xu Reviewed-by: Peter Xu Signed-off-by: Xiao Guangrong Reviewed-by: Juan Quintela --- migration/ram.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 2ad07b5e15..25af797c0a 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -301,10 +301,10 @@ struct RAMState { uint64_t num_dirty_pages_period; /* xbzrle misses since the beginning of the period */ uint64_t xbzrle_cache_miss_prev; - /* number of iterations at the beginning of period */ - uint64_t iterations_prev; - /* Iterations since start */ - uint64_t iterations; + /* total handled target pages at the beginning of period */ + uint64_t target_page_count_prev; + /* total handled target pages since start */ + uint64_t target_page_count; /* number of dirty bits in the bitmap */ uint64_t migration_dirty_pages; /* last dirty_sync_count we have seen */ @@ -1594,19 +1594,19 @@ uint64_t ram_pagesize_summary(void) static void migration_update_rates(RAMState *rs, int64_t end_time) { - uint64_t iter_count = rs->iterations - rs->iterations_prev; + uint64_t page_count = rs->target_page_count - rs->target_page_count_prev; /* calculate period counters */ ram_counters.dirty_pages_rate = rs->num_dirty_pages_period * 1000 / (end_time - rs->time_last_bitmap_sync); - if (!iter_count) { + if (!page_count) { return; } if (migrate_use_xbzrle()) { xbzrle_counters.cache_miss_rate = (double)(xbzrle_counters.cache_miss - - rs->xbzrle_cache_miss_prev) / iter_count; + rs->xbzrle_cache_miss_prev) / page_count; rs->xbzrle_cache_miss_prev = xbzrle_counters.cache_miss; } } @@ -1664,7 +1664,7 @@ static void migration_bitmap_sync(RAMState *rs) migration_update_rates(rs, end_time); - rs->iterations_prev = rs->iterations; + rs->target_page_count_prev = rs->target_page_count; /* reset period counters */ rs->time_last_bitmap_sync = end_time; @@ -3209,7 +3209,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) done = 1; break; } - rs->iterations++; + rs->target_page_count += pages; /* we want to check in the 1st loop, just in case it was the 1st time and we had to sync the dirty bitmap. From patchwork Mon Sep 3 09:26:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 965322 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="pERdZ6gD"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 423l994PwGz9s3C for ; Mon, 3 Sep 2018 19:31:56 +1000 (AEST) Received: from localhost ([::1]:44265 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fwlD6-0003Df-RY for incoming@patchwork.ozlabs.org; Mon, 03 Sep 2018 05:31:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39729) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fwl8a-0007FO-Aw for qemu-devel@nongnu.org; Mon, 03 Sep 2018 05:27:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fwl8T-00045h-Qa for qemu-devel@nongnu.org; Mon, 03 Sep 2018 05:27:12 -0400 Received: from mail-pl1-x641.google.com ([2607:f8b0:4864:20::641]:38298) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fwl8T-00045X-GL for qemu-devel@nongnu.org; Mon, 03 Sep 2018 05:27:05 -0400 Received: by mail-pl1-x641.google.com with SMTP id u11-v6so8390341plq.5 for ; Mon, 03 Sep 2018 02:27:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=MNAh+EqNClvUdc9lNcsHUfio0Zd2vG6xOUvo3bJifck=; b=pERdZ6gDu4olAvJHg2yNtEWiNEb+MptLk+b7LWqi9FxRDW5hDlfLrmHyQvarmyy6lY aGz5JD9U64420q3/ZYMjKD7yk6R3AioEpmq6vDM5Ob5MuD/DiICWlQ4qj4uP1u8W21Bu paRSZ0laCON/Xi5mcjGRGCOygVx9jD6V7wanfZsUbwMrWqVR8l6owqJG9lqzW6AvIb1r HaoaTcpdd8bfXwrjmniJq6sIfjk4n9ZRvW6tzevhelTQsmRfVfo45ZsdIs0Ug0T+ZwLr fsNnF8hWBI6jRwOyyU/2MHx1idGy7duoQ1EdqB5Bvks01ujsdTaZOpavguYBCsWsamSP klFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=MNAh+EqNClvUdc9lNcsHUfio0Zd2vG6xOUvo3bJifck=; b=iAtQ8WgGBJrHjTOIrapL/oYPXzlbo5OGBuQBtWIJArKLx5Gg/nc4SNwL7YhiMaWaRY A23+kX6zN+E85Ko3FrGQ7rNC31vBoLUKwUxafxApamHEX0MtuCY9wy+E+06eYmJP08MQ eJeakSFQR+XwSmZpSUTEmU2UoiA5izm24XiQ+uecP9kAGvPmrH4abGvnEprEnKy5EM+X 5WW6HMAI/89wADuq+g4EMIqj4QKz+jMeKMBfFvARLUoNmhWYeFBnvttJpF6YuBWlK45A om8Cw4AQGN3IxVUAAWNzesiw1LhoE+x/BnVTrXoSESGZH16X8DdoivKdayd1+8hplHIp Hadw== X-Gm-Message-State: APzg51C4CDJeZqK6cI1Xa/bQwezyTusrinh0pRMYbv3LQ4A9XqTBGbbP HByHRjTD6c1KNRxnR7AgSS5lNIQh X-Google-Smtp-Source: ANB0VdZAdXzX4G4FbX1gdYqBfLXlpOYmmCAjbwcB2qef/SNfivIUWv1IJPUL+OmR4FzqMsMEuePZQQ== X-Received: by 2002:a17:902:7447:: with SMTP id e7-v6mr3158486plt.186.1535966824699; Mon, 03 Sep 2018 02:27:04 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.40]) by smtp.gmail.com with ESMTPSA id 3-v6sm33543102pfq.10.2018.09.03.02.27.00 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 03 Sep 2018 02:27:04 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Mon, 3 Sep 2018 17:26:43 +0800 Message-Id: <20180903092644.25812-4-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180903092644.25812-1-xiaoguangrong@tencent.com> References: <20180903092644.25812-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::641 Subject: [Qemu-devel] [PATCH v5 3/4] migration: show the statistics of compression X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, quintela@redhat.com, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong Currently, it includes: pages: amount of pages compressed and transferred to the target VM busy: amount of count that no free thread to compress data busy-rate: rate of thread busy compressed-size: amount of bytes after compression compression-rate: rate of compressed size Reviewed-by: Peter Xu Signed-off-by: Xiao Guangrong Reviewed-by: Juan Quintela --- hmp.c | 13 +++++++++++++ migration/migration.c | 12 ++++++++++++ migration/ram.c | 41 ++++++++++++++++++++++++++++++++++++++++- migration/ram.h | 1 + qapi/migration.json | 26 +++++++++++++++++++++++++- 5 files changed, 91 insertions(+), 2 deletions(-) diff --git a/hmp.c b/hmp.c index 4975fa56b0..f57b23d889 100644 --- a/hmp.c +++ b/hmp.c @@ -271,6 +271,19 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict) info->xbzrle_cache->overflow); } + if (info->has_compression) { + monitor_printf(mon, "compression pages: %" PRIu64 " pages\n", + info->compression->pages); + monitor_printf(mon, "compression busy: %" PRIu64 "\n", + info->compression->busy); + monitor_printf(mon, "compression busy rate: %0.2f\n", + info->compression->busy_rate); + monitor_printf(mon, "compressed size: %" PRIu64 "\n", + info->compression->compressed_size); + monitor_printf(mon, "compression rate: %0.2f\n", + info->compression->compression_rate); + } + if (info->has_cpu_throttle_percentage) { monitor_printf(mon, "cpu throttle percentage: %" PRIu64 "\n", info->cpu_throttle_percentage); diff --git a/migration/migration.c b/migration/migration.c index 4b316ec343..f1d662f928 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -758,6 +758,18 @@ static void populate_ram_info(MigrationInfo *info, MigrationState *s) info->xbzrle_cache->overflow = xbzrle_counters.overflow; } + if (migrate_use_compression()) { + info->has_compression = true; + info->compression = g_malloc0(sizeof(*info->compression)); + info->compression->pages = compression_counters.pages; + info->compression->busy = compression_counters.busy; + info->compression->busy_rate = compression_counters.busy_rate; + info->compression->compressed_size = + compression_counters.compressed_size; + info->compression->compression_rate = + compression_counters.compression_rate; + } + if (cpu_throttle_active()) { info->has_cpu_throttle_percentage = true; info->cpu_throttle_percentage = cpu_throttle_get_percentage(); diff --git a/migration/ram.c b/migration/ram.c index 25af797c0a..0e8c3ca33d 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -301,6 +301,15 @@ struct RAMState { uint64_t num_dirty_pages_period; /* xbzrle misses since the beginning of the period */ uint64_t xbzrle_cache_miss_prev; + + /* compression statistics since the beginning of the period */ + /* amount of count that no free thread to compress data */ + uint64_t compress_thread_busy_prev; + /* amount bytes after compression */ + uint64_t compressed_size_prev; + /* amount of compressed pages */ + uint64_t compress_pages_prev; + /* total handled target pages at the beginning of period */ uint64_t target_page_count_prev; /* total handled target pages since start */ @@ -340,6 +349,8 @@ struct PageSearchStatus { }; typedef struct PageSearchStatus PageSearchStatus; +CompressionStats compression_counters; + struct CompressParam { bool done; bool quit; @@ -1595,6 +1606,7 @@ uint64_t ram_pagesize_summary(void) static void migration_update_rates(RAMState *rs, int64_t end_time) { uint64_t page_count = rs->target_page_count - rs->target_page_count_prev; + double compressed_size; /* calculate period counters */ ram_counters.dirty_pages_rate = rs->num_dirty_pages_period * 1000 @@ -1609,6 +1621,26 @@ static void migration_update_rates(RAMState *rs, int64_t end_time) rs->xbzrle_cache_miss_prev) / page_count; rs->xbzrle_cache_miss_prev = xbzrle_counters.cache_miss; } + + if (migrate_use_compression()) { + compression_counters.busy_rate = (double)(compression_counters.busy - + rs->compress_thread_busy_prev) / page_count; + rs->compress_thread_busy_prev = compression_counters.busy; + + compressed_size = compression_counters.compressed_size - + rs->compressed_size_prev; + if (compressed_size) { + double uncompressed_size = (compression_counters.pages - + rs->compress_pages_prev) * TARGET_PAGE_SIZE; + + /* Compression-Ratio = Uncompressed-size / Compressed-size */ + compression_counters.compression_rate = + uncompressed_size / compressed_size; + + rs->compress_pages_prev = compression_counters.pages; + rs->compressed_size_prev = compression_counters.compressed_size; + } + } } static void migration_bitmap_sync(RAMState *rs) @@ -1890,10 +1922,16 @@ exit: static void update_compress_thread_counts(const CompressParam *param, int bytes_xmit) { + ram_counters.transferred += bytes_xmit; + if (param->zero_page) { ram_counters.duplicate++; + return; } - ram_counters.transferred += bytes_xmit; + + /* 8 means a header with RAM_SAVE_FLAG_CONTINUE. */ + compression_counters.compressed_size += bytes_xmit - 8; + compression_counters.pages++; } static void flush_compressed_data(RAMState *rs) @@ -2261,6 +2299,7 @@ static bool save_compress_page(RAMState *rs, RAMBlock *block, ram_addr_t offset) return true; } + compression_counters.busy++; return false; } diff --git a/migration/ram.h b/migration/ram.h index 457bf54b8c..a139066846 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -36,6 +36,7 @@ extern MigrationStats ram_counters; extern XBZRLECacheStats xbzrle_counters; +extern CompressionStats compression_counters; int xbzrle_cache_resize(int64_t new_size, Error **errp); uint64_t ram_bytes_remaining(void); diff --git a/qapi/migration.json b/qapi/migration.json index f62d3f9a4b..6e8c21258a 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -75,6 +75,27 @@ 'cache-miss': 'int', 'cache-miss-rate': 'number', 'overflow': 'int' } } +## +# @CompressionStats: +# +# Detailed migration compression statistics +# +# @pages: amount of pages compressed and transferred to the target VM +# +# @busy: count of times that no free thread was available to compress data +# +# @busy-rate: rate of thread busy +# +# @compressed-size: amount of bytes after compression +# +# @compression-rate: rate of compressed size +# +# Since: 3.1 +## +{ 'struct': 'CompressionStats', + 'data': {'pages': 'int', 'busy': 'int', 'busy-rate': 'number', + 'compressed-size': 'int', 'compression-rate': 'number' } } + ## # @MigrationStatus: # @@ -172,6 +193,8 @@ # only present when the postcopy-blocktime migration capability # is enabled. (Since 3.0) # +# @compression: migration compression statistics, only returned if compression +# feature is on and status is 'active' or 'completed' (Since 3.1) # # Since: 0.14.0 ## @@ -186,7 +209,8 @@ '*cpu-throttle-percentage': 'int', '*error-desc': 'str', '*postcopy-blocktime' : 'uint32', - '*postcopy-vcpu-blocktime': ['uint32']} } + '*postcopy-vcpu-blocktime': ['uint32'], + '*compression': 'CompressionStats'} } ## # @query-migrate: From patchwork Mon Sep 3 09:26:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 965321 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="NNRIsmVL"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 423l6c23RCz9s3C for ; Mon, 3 Sep 2018 19:29:44 +1000 (AEST) Received: from localhost ([::1]:44256 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fwlAz-000209-UR for incoming@patchwork.ozlabs.org; Mon, 03 Sep 2018 05:29:41 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39740) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fwl8c-0007PH-Es for qemu-devel@nongnu.org; Mon, 03 Sep 2018 05:27:15 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fwl8a-00046m-DD for qemu-devel@nongnu.org; Mon, 03 Sep 2018 05:27:14 -0400 Received: from mail-pg1-x542.google.com ([2607:f8b0:4864:20::542]:43450) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fwl8Y-00046D-F8 for qemu-devel@nongnu.org; Mon, 03 Sep 2018 05:27:12 -0400 Received: by mail-pg1-x542.google.com with SMTP id v66-v6so8338298pgb.10 for ; Mon, 03 Sep 2018 02:27:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=4v1KhMeqsfctLzh5wAQ3nkZnkCI9ZncjoRGmCxofBzo=; b=NNRIsmVLsJwkQb06jVyGP7uEc4dNijylB8CtVSF/8hlNrRIIvq17CCCo7YfPaP1/Dg Z/AQT/P/TZQPzx6aju7vI4LrTMiqoQ0/4IesPxBD4I3fOTyGwR2GVTGXMVf0Cyc1Ek7o 4u/9pS4ZEqsUm3zBnqAjNug5dIk1L38qBaGZN5/rB0jPnoVYQfvooEKxfYQVovz9dF1I 4N8rqMCkKo8tbWwf4CjZE7fnIt9Ug/bzYeyEsCaIgkoP3lU0kxr7eO/TWyof2xTOkgE0 UUhdJW5B3NtPDm3bWPl74Yt+t6XvvVJXUyDRcS0KUn/49ki+jDzvr0OcJ14Tiq4XssEy AOnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=4v1KhMeqsfctLzh5wAQ3nkZnkCI9ZncjoRGmCxofBzo=; b=lR+z7a7oN1CPRa5Gkn0bRCE+1Ulwny/ocEyOhKRhsB9rfKWKBmda2xJtaHVEA0M23B r3Fl5JGFQK+B9tKBshFpt3FzM+9Ok0yD8pXD0v1B1azpQveRuhmTJgmVAOhXhgs2FPfU kgfrb0VbFGLLb5RtgG9eLWSNpKqlcTL4jy8vOpu/UtjSc0LW/2gs7R9teE9c5ihT1Myv 0/KGqiOqYv7kDPdiVDMQLEAWrlenGbJWkrWO6JcLBP5exkEJy8ItMWKmkrFKpVo/DpfC j9anrJIrRZ4/Ce6KsLvWLQDhSgYD2b8+Y/WUC68ttPUt0nNllYuW2H895vNecHazLw/C bbGg== X-Gm-Message-State: APzg51BZa56OBfkvRh7yOLwVEzM8bjB1wxmFmLcVKR+VPuzCAPp7LpJ5 QcvZHgmu8YUMxHca0vs1r+Y= X-Google-Smtp-Source: ANB0VdboP1++JJ2smUf5KqZD0+43Yl7nel/LrN/cnGmCz8u0PQyy0vM17PhwPuME1iTEBAbcLn506A== X-Received: by 2002:a62:7d8d:: with SMTP id y135-v6mr28095270pfc.259.1535966828784; Mon, 03 Sep 2018 02:27:08 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.40]) by smtp.gmail.com with ESMTPSA id 3-v6sm33543102pfq.10.2018.09.03.02.27.04 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 03 Sep 2018 02:27:08 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Mon, 3 Sep 2018 17:26:44 +0800 Message-Id: <20180903092644.25812-5-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180903092644.25812-1-xiaoguangrong@tencent.com> References: <20180903092644.25812-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::542 Subject: [Qemu-devel] [PATCH v5 4/4] migration: handle the error condition properly X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, quintela@redhat.com, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong ram_find_and_save_block() can return negative if any error hanppens, however, it is completely ignored in current code Signed-off-by: Xiao Guangrong Reviewed-by: Juan Quintela --- migration/ram.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 0e8c3ca33d..cb402ca6c8 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2413,7 +2413,8 @@ static int ram_save_host_page(RAMState *rs, PageSearchStatus *pss, * * Called within an RCU critical section. * - * Returns the number of pages written where zero means no dirty pages + * Returns the number of pages written where zero means no dirty pages, + * or negative on error * * @rs: current RAM state * @last_stage: if we are at the completion stage @@ -3248,6 +3249,12 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) done = 1; break; } + + if (pages < 0) { + qemu_file_set_error(f, pages); + break; + } + rs->target_page_count += pages; /* we want to check in the 1st loop, just in case it was the 1st time @@ -3289,7 +3296,7 @@ out: /** * ram_save_complete: function called to send the remaining amount of ram * - * Returns zero to indicate success + * Returns zero to indicate success or negative on error * * Called with iothread lock * @@ -3300,6 +3307,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque) { RAMState **temp = opaque; RAMState *rs = *temp; + int ret = 0; rcu_read_lock(); @@ -3320,6 +3328,10 @@ static int ram_save_complete(QEMUFile *f, void *opaque) if (pages == 0) { break; } + if (pages < 0) { + ret = pages; + break; + } } flush_compressed_data(rs); @@ -3331,7 +3343,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque) qemu_put_be64(f, RAM_SAVE_FLAG_EOS); qemu_fflush(f); - return 0; + return ret; } static void ram_save_pending(QEMUFile *f, void *opaque, uint64_t max_size,