From patchwork Thu Jul 19 12:15:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 946234 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="pHk8Bf81"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41WY0R23F4z9rxs for ; Thu, 19 Jul 2018 22:16:39 +1000 (AEST) Received: from localhost ([::1]:42066 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fg7rD-00021s-UT for incoming@patchwork.ozlabs.org; Thu, 19 Jul 2018 08:16:31 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34913) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fg7qT-00020L-8b for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:15:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fg7qQ-0005Ke-1s for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:15:45 -0400 Received: from mail-pg1-x542.google.com ([2607:f8b0:4864:20::542]:36997) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fg7qP-0005KN-Sq for qemu-devel@nongnu.org; Thu, 19 Jul 2018 08:15:41 -0400 Received: by mail-pg1-x542.google.com with SMTP id n7-v6so3589057pgq.4 for ; Thu, 19 Jul 2018 05:15:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=B8TJczg08DYT0t+Q74ntvoEGY6SxUSVfWDW/1HNUO44=; b=pHk8Bf81+E/+i1uSYEj8J7y6/liATOw4J+wSCIK2D72AE1e9bobZrGahaYSNjRJ0/4 vpDoqWrAZNjQf9vdDV65QcOkujZyUqu3s0qfC6nspMo4fBnGiLvwbQ2+dDZPrQezPTYL MWTZT7wzd9XkLykcC0NdHkwHBSo00NseVw17cCYba5P0zP6uCQcVf7fBdLZLKGQqxFGL HRH70IQe81J9GZCG0B+/r4Jif+BMOZJQYU4vDzq/4Sfx7QuiH2laM87tvoX3qerGgTS6 xungowqtkmvQyNQnsFbXmAmCV5oIp1J5iDQk24hD1mkrodnHVZ6oGuvdr8GWDG+o5bOu z8Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=B8TJczg08DYT0t+Q74ntvoEGY6SxUSVfWDW/1HNUO44=; b=jOEZ5jrtIZa326OKIFS/uk0Jd/K90lzqbA/7PW1ehfidPntJcqUjohHcqp92Q+wfhj 4sQVDooiNqZ9OCXNYB4z6b2fYVPNqv3oRzY3gv5dCDBORpXcW2fC7at24Us2jHIWNxwb HK8hchP3i5i2I0oCbs8q9lzrLW0tryGtk0/uqJDuh/n/cYXS9T4avgL2+fDoEm4SRWui nkeoP3vnI+CwaLKOWsB5qg4AVNNBgELLg1oXdR80sYXkQQZ/RZZbuFxFEJreCntaPoK0 ABhYvh5UH/Rd5XusGiffbZ4/bTJbS2RRAaSjDekgiQDXqifEOUJcJJE/Ywwwg7QCq0iA ZlBg== X-Gm-Message-State: AOUpUlF13F+05hjhRyaDC9f7wiR4fRffFNTbvgGbtIYSkujYcUUORqgS P3vnY0tAOTmMhDyIon673KU= X-Google-Smtp-Source: AAOMgpc2Z808l1N7NLy8qOCisgTI6Q2pSpVAFUj4wp/U+j5/m4S+ryz/teWcGn4RowZa1fC5jC/odg== X-Received: by 2002:a62:e18:: with SMTP id w24-v6mr9389911pfi.145.1532002540880; Thu, 19 Jul 2018 05:15:40 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.37]) by smtp.gmail.com with ESMTPSA id h190-v6sm21270456pge.85.2018.07.19.05.15.37 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 19 Jul 2018 05:15:40 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Thu, 19 Jul 2018 20:15:12 +0800 Message-Id: <20180719121520.30026-1-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.4 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::542 Subject: [Qemu-devel] [PATCH v2 0/8] migration: compression optimization X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong Thanks to Peter's suggestion, i split the long series (1) and this is the first part. I am not sure if Dave is happy to @reduced-size, will change immediately if it's objected. :) Changelog in v2: 1) introduce a parameter to make the main thread wait for free thread thread to compress the data as current behavior based on the comments from Dave and Peter 2) description improvement for compression statistics, thanks for Eric's improvement 3) move the handle of zero page to the threads 4) fix the issue that the new dirtied page would be overwritten at the destination when we drop flush_compressed_data at the end of iteration which is pointed out by Dave (1) https://www.spinics.net/lists/kvm/msg169774.html Xiao Guangrong (8): migration: do not wait for free thread migration: fix counting normal page for compression migration: show the statistics of compression migration: introduce save_zero_page_to_file migration: drop the return value of do_compress_ram_page migration: move handle of zero page to the thread migration: hold the lock only if it is really needed migration: do not flush_compressed_data at the end of each iteration hmp.c | 21 +++++ include/qemu/queue.h | 1 + migration/migration.c | 32 +++++++ migration/migration.h | 1 + migration/ram.c | 244 +++++++++++++++++++++++++++++++++++++------------- migration/ram.h | 1 + qapi/migration.json | 49 ++++++++-- 7 files changed, 281 insertions(+), 68 deletions(-)