From patchwork Tue Mar 27 09:10:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 892029 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="eoN1Ck9X"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40B2Dh0pXDz9s08 for ; Wed, 28 Mar 2018 20:11:16 +1100 (AEDT) Received: from localhost ([::1]:38152 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f176v-0000mM-GP for incoming@patchwork.ozlabs.org; Wed, 28 Mar 2018 05:11:13 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56621) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f176C-0000kr-JT for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f176B-0004w1-MA for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:28 -0400 Received: from mail-pg0-x244.google.com ([2607:f8b0:400e:c05::244]:35875) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1f176B-0004vb-AK for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:27 -0400 Received: by mail-pg0-x244.google.com with SMTP id 201so714525pgg.3 for ; Wed, 28 Mar 2018 02:10:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=TL1XWhIt+sETAFduxXn4nL0YeK0YLzCPZxJw286oiUg=; b=eoN1Ck9XMmxWPUpfGeNrYdJbnYwiF0+2GycSflA1NyHlURopUJhE7orbNDpDQjF4Wc B50IAIndhiB8WKEiwmbrrzUrlLA3vQxFOkUGlAvie9BOc63zAUdZ+2oW2lIFhqtQM96F vtK6squJBjOh7Jvs9MSx6N2xkpBxsWADbWKj9MtJRGSYunSFO6G1pydvdjEONVEutlx7 SlSZbcP02WjPq3hlnFgdllpl1N2vFVrK7aDOV8g8RgolntwIw+BM0JNxS7z3NYYo4g7R 7VsXz+gzgg9t1SG8jsdwUjMMM0LEnxFZ8YVl1L5t2mfUEAAtaW69GfnCKSAvtkeWbas/ 0Upw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=TL1XWhIt+sETAFduxXn4nL0YeK0YLzCPZxJw286oiUg=; b=c0JkJaYA9S7zdJg9biFzVbWLd/8+9vUEPGbVmu/vxXI0o5JMof997KdYknVlO3MkE1 FIzKtfb33BORTjko2k39CuXNOYRVLslr8odtXODxaq1cBBHYjsawgWomgOKETdA+hMgo PPYsblsd0z64QkydaYgOyucIcnMsvqAR+gLOmEBayAaiL7HoAEebvOWp4bQE+1DBZasC kXMpsFoh6Y8DCYmTZXyEp7ky8W4wvbVSkGt+/np8dOXl+ZtAd+p7PIJWVSUMgIf6pTw6 ccqXZfbidTKar760KkELnLPanYr0EoUFMOaeM711ai0xF8gT9cizPiQMDO2XQDtWpNAP ofnA== X-Gm-Message-State: AElRT7Ef7Pp4y8BWf97yAB54S+vdZ44+KV1vKycXd6by1DBjxurYNhhv DaqEo7kZx26edsnGmqGjDo8= X-Google-Smtp-Source: AIpwx4/XAroKcoWqFf+tJ+h/MPP5W2h4lw0W+/qd8TqK6uFYtn3iGkom5/PPudB6Ft/BNhhJCChn/Q== X-Received: by 10.98.174.23 with SMTP id q23mr2311480pff.103.1522228226272; Wed, 28 Mar 2018 02:10:26 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.38]) by smtp.gmail.com with ESMTPSA id q9sm5590648pgs.89.2018.03.28.02.10.23 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 28 Mar 2018 02:10:25 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 27 Mar 2018 17:10:34 +0800 Message-Id: <20180327091043.30220-2-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180327091043.30220-1-xiaoguangrong@tencent.com> References: <20180327091043.30220-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::244 Subject: [Qemu-devel] [PATCH v2 01/10] migration: stop compressing page in migration thread X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong As compression is a heavy work, do not do it in migration thread, instead, we post it out as a normal page Reviewed-by: Wei Wang Reviewed-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Xiao Guangrong --- migration/ram.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 0e90efa092..409c847a76 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1137,7 +1137,7 @@ static int ram_save_compressed_page(RAMState *rs, PageSearchStatus *pss, int pages = -1; uint64_t bytes_xmit = 0; uint8_t *p; - int ret, blen; + int ret; RAMBlock *block = pss->block; ram_addr_t offset = pss->page << TARGET_PAGE_BITS; @@ -1167,23 +1167,23 @@ static int ram_save_compressed_page(RAMState *rs, PageSearchStatus *pss, if (block != rs->last_sent_block) { flush_compressed_data(rs); pages = save_zero_page(rs, block, offset); - if (pages == -1) { - /* Make sure the first page is sent out before other pages */ - bytes_xmit = save_page_header(rs, rs->f, block, offset | - RAM_SAVE_FLAG_COMPRESS_PAGE); - blen = qemu_put_compression_data(rs->f, p, TARGET_PAGE_SIZE, - migrate_compress_level()); - if (blen > 0) { - ram_counters.transferred += bytes_xmit + blen; - ram_counters.normal++; - pages = 1; - } else { - qemu_file_set_error(rs->f, blen); - error_report("compressed data failed!"); - } - } if (pages > 0) { ram_release_pages(block->idstr, offset, pages); + } else { + /* + * Make sure the first page is sent out before other pages. + * + * we post it as normal page as compression will take much + * CPU resource. + */ + ram_counters.transferred += save_page_header(rs, rs->f, block, + offset | RAM_SAVE_FLAG_PAGE); + qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE, + migrate_release_ram() & + migration_in_postcopy()); + ram_counters.transferred += TARGET_PAGE_SIZE; + ram_counters.normal++; + pages = 1; } } else { pages = save_zero_page(rs, block, offset); From patchwork Tue Mar 27 09:10:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 892035 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="GSNSfvTc"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40B2JT0rB9z9s0b for ; Wed, 28 Mar 2018 20:14:33 +1100 (AEDT) Received: from localhost ([::1]:38167 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f17A6-0003hm-J6 for incoming@patchwork.ozlabs.org; Wed, 28 Mar 2018 05:14:30 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56643) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f176J-0000qS-L5 for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f176E-0004xK-On for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:35 -0400 Received: from mail-pf0-x241.google.com ([2607:f8b0:400e:c00::241]:37918) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1f176E-0004x2-FY for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:30 -0400 Received: by mail-pf0-x241.google.com with SMTP id y69so738821pfb.5 for ; Wed, 28 Mar 2018 02:10:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1pBhsXeblwLzjcdPYvSpffqx6J9BP8PioUGzSV4JxuM=; b=GSNSfvTchwQDO3Jtt9Qi+uPiToYBn2J0oXLnS6xvXTfCfr/kdXlo9Vx23mYsstfuBN QiT5nUvJF8SsRh7oD1qJjzVj9BPqRAozFkGNvxQqGFw0mYFnBOEgcaM5j/wSVBMdtozP veEscZ0dqfmBJ5K+qBHQOxwMZDAdoEcl2Vwb3nrSbbw2L8Lhtsak1pYEpgzEP5sU8LhU FFeFjVjrjehlJ6v4/nbKgjpCUDkHyqziLKGFvSpsQUxI2JmnHdACEYgt8A2RLyrT3Gh9 FUhijKtE5ZuvXYY2gl6ZzHRgdPr8VKeRWbtxp965nHr8gyLLjcP9WPgazAwIbze9EHbt QzNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1pBhsXeblwLzjcdPYvSpffqx6J9BP8PioUGzSV4JxuM=; b=l8uwaYjV9fIIEoG/PEvbW7IFRyv0ljxNFGSb7M+kGi+bnQCDN8mHC8q8HF5tFTNfaR 1E6Bij/G89hWze03FKEfAEykNhqn7I7lunHLiDR0Zt5bMcONcvuRkTUaSkL+ysYTWRc0 /sXRu/phao8OKlDCyRmJOK8ySwCURzV+XtEq9Uov5jssmBz3zeMJXsaCUf9UZ3OBIKo1 BMs9U1ys6wdEp1+alhfHFW4FfbyEH89qypMgwldzxFfAuyaxKibbJQPAA0yYnzpKhC1I dAEYjWCqcPiMOyDNKh8sdF2kzLtQ1FVIBDbA1PAUNJkYtjDWHlJl0Y5q1i+m1RWfDZ0K aymA== X-Gm-Message-State: AElRT7FmG3zGQmByZsYz1NN+OqtzDUMuidpup3n2T3xgNIdhZ416muY8 /J1AhFr1SRGL1MOS/qdkh1EIbg== X-Google-Smtp-Source: AIpwx48WvbYvg4kQOg9akRY8WhyvtoZ1ii79A9JAadLti6yNW0HGQfPqBTlQiA84LLxAJkQJbLR/Tg== X-Received: by 10.98.180.13 with SMTP id h13mr2271570pfn.139.1522228229578; Wed, 28 Mar 2018 02:10:29 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.38]) by smtp.gmail.com with ESMTPSA id q9sm5590648pgs.89.2018.03.28.02.10.26 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 28 Mar 2018 02:10:29 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 27 Mar 2018 17:10:35 +0800 Message-Id: <20180327091043.30220-3-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180327091043.30220-1-xiaoguangrong@tencent.com> References: <20180327091043.30220-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::241 Subject: [Qemu-devel] [PATCH v2 02/10] migration: stop compression to allocate and free memory frequently X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong Current code uses compress2() to compress memory which manages memory internally, that causes huge memory is allocated and freed very frequently More worse, frequently returning memory to kernel will flush TLBs and trigger invalidation callbacks on mmu-notification which interacts with KVM MMU, that dramatically reduce the performance of VM So, we maintain the memory by ourselves and reuse it for each compression Reviewed-by: Jiang Biao Signed-off-by: Xiao Guangrong --- migration/qemu-file.c | 39 ++++++++++++++++++++++++++++++++------- migration/qemu-file.h | 6 ++++-- migration/ram.c | 41 +++++++++++++++++++++++++++++++++-------- 3 files changed, 69 insertions(+), 17 deletions(-) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index e85f501f86..e924cc23c5 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -658,8 +658,32 @@ uint64_t qemu_get_be64(QEMUFile *f) return v; } -/* Compress size bytes of data start at p with specific compression - * level and store the compressed data to the buffer of f. +/* return the size after compression, or negative value on error */ +static int qemu_compress_data(z_stream *stream, uint8_t *dest, size_t dest_len, + const uint8_t *source, size_t source_len) +{ + int err; + + err = deflateReset(stream); + if (err != Z_OK) { + return -1; + } + + stream->avail_in = source_len; + stream->next_in = (uint8_t *)source; + stream->avail_out = dest_len; + stream->next_out = dest; + + err = deflate(stream, Z_FINISH); + if (err != Z_STREAM_END) { + return -1; + } + + return stream->next_out - dest; +} + +/* Compress size bytes of data start at p and store the compressed + * data to the buffer of f. * * When f is not writable, return -1 if f has no space to save the * compressed data. @@ -667,9 +691,8 @@ uint64_t qemu_get_be64(QEMUFile *f) * do fflush first, if f still has no space to save the compressed * data, return -1. */ - -ssize_t qemu_put_compression_data(QEMUFile *f, const uint8_t *p, size_t size, - int level) +ssize_t qemu_put_compression_data(QEMUFile *f, z_stream *stream, + const uint8_t *p, size_t size) { ssize_t blen = IO_BUF_SIZE - f->buf_index - sizeof(int32_t); @@ -683,8 +706,10 @@ ssize_t qemu_put_compression_data(QEMUFile *f, const uint8_t *p, size_t size, return -1; } } - if (compress2(f->buf + f->buf_index + sizeof(int32_t), (uLongf *)&blen, - (Bytef *)p, size, level) != Z_OK) { + + blen = qemu_compress_data(stream, f->buf + f->buf_index + sizeof(int32_t), + blen, p, size); + if (blen < 0) { error_report("Compress Failed!"); return 0; } diff --git a/migration/qemu-file.h b/migration/qemu-file.h index f4f356ab12..2ccfcfb2a8 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -25,6 +25,8 @@ #ifndef MIGRATION_QEMU_FILE_H #define MIGRATION_QEMU_FILE_H +#include + /* Read a chunk of data from a file at the given position. The pos argument * can be ignored if the file is only be used for streaming. The number of * bytes actually read should be returned. @@ -132,8 +134,8 @@ bool qemu_file_is_writable(QEMUFile *f); size_t qemu_peek_buffer(QEMUFile *f, uint8_t **buf, size_t size, size_t offset); size_t qemu_get_buffer_in_place(QEMUFile *f, uint8_t **buf, size_t size); -ssize_t qemu_put_compression_data(QEMUFile *f, const uint8_t *p, size_t size, - int level); +ssize_t qemu_put_compression_data(QEMUFile *f, z_stream *stream, + const uint8_t *p, size_t size); int qemu_put_qemu_file(QEMUFile *f_des, QEMUFile *f_src); /* diff --git a/migration/ram.c b/migration/ram.c index 409c847a76..e043a192e1 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -269,6 +269,7 @@ struct CompressParam { QemuCond cond; RAMBlock *block; ram_addr_t offset; + z_stream stream; }; typedef struct CompressParam CompressParam; @@ -299,7 +300,7 @@ static QemuThread *decompress_threads; static QemuMutex decomp_done_lock; static QemuCond decomp_done_cond; -static int do_compress_ram_page(QEMUFile *f, RAMBlock *block, +static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, ram_addr_t offset); static void *do_data_compress(void *opaque) @@ -316,7 +317,7 @@ static void *do_data_compress(void *opaque) param->block = NULL; qemu_mutex_unlock(¶m->mutex); - do_compress_ram_page(param->file, block, offset); + do_compress_ram_page(param->file, ¶m->stream, block, offset); qemu_mutex_lock(&comp_done_lock); param->done = true; @@ -357,10 +358,20 @@ static void compress_threads_save_cleanup(void) terminate_compression_threads(); thread_count = migrate_compress_threads(); for (i = 0; i < thread_count; i++) { + /* + * stream.opaque can be used to store private data, we use it + * as a indicator which shows if the thread is properly init'd + * or not + */ + if (!comp_param[i].stream.opaque) { + break; + } qemu_thread_join(compress_threads + i); qemu_fclose(comp_param[i].file); qemu_mutex_destroy(&comp_param[i].mutex); qemu_cond_destroy(&comp_param[i].cond); + deflateEnd(&comp_param[i].stream); + comp_param[i].stream.opaque = NULL; } qemu_mutex_destroy(&comp_done_lock); qemu_cond_destroy(&comp_done_cond); @@ -370,12 +381,12 @@ static void compress_threads_save_cleanup(void) comp_param = NULL; } -static void compress_threads_save_setup(void) +static int compress_threads_save_setup(void) { int i, thread_count; if (!migrate_use_compression()) { - return; + return 0; } thread_count = migrate_compress_threads(); compress_threads = g_new0(QemuThread, thread_count); @@ -383,6 +394,12 @@ static void compress_threads_save_setup(void) qemu_cond_init(&comp_done_cond); qemu_mutex_init(&comp_done_lock); for (i = 0; i < thread_count; i++) { + if (deflateInit(&comp_param[i].stream, + migrate_compress_level()) != Z_OK) { + goto exit; + } + comp_param[i].stream.opaque = &comp_param[i]; + /* comp_param[i].file is just used as a dummy buffer to save data, * set its ops to empty. */ @@ -395,6 +412,11 @@ static void compress_threads_save_setup(void) do_data_compress, comp_param + i, QEMU_THREAD_JOINABLE); } + return 0; + +exit: + compress_threads_save_cleanup(); + return -1; } /* Multiple fd's */ @@ -1031,7 +1053,7 @@ static int ram_save_page(RAMState *rs, PageSearchStatus *pss, bool last_stage) return pages; } -static int do_compress_ram_page(QEMUFile *f, RAMBlock *block, +static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, ram_addr_t offset) { RAMState *rs = ram_state; @@ -1040,8 +1062,7 @@ static int do_compress_ram_page(QEMUFile *f, RAMBlock *block, bytes_sent = save_page_header(rs, f, block, offset | RAM_SAVE_FLAG_COMPRESS_PAGE); - blen = qemu_put_compression_data(f, p, TARGET_PAGE_SIZE, - migrate_compress_level()); + blen = qemu_put_compression_data(f, stream, p, TARGET_PAGE_SIZE); if (blen < 0) { bytes_sent = 0; qemu_file_set_error(migrate_get_current()->to_dst_file, blen); @@ -2214,9 +2235,14 @@ static int ram_save_setup(QEMUFile *f, void *opaque) RAMState **rsp = opaque; RAMBlock *block; + if (compress_threads_save_setup()) { + return -1; + } + /* migration has already setup the bitmap, reuse it. */ if (!migration_in_colo_state()) { if (ram_init_all(rsp) != 0) { + compress_threads_save_cleanup(); return -1; } } @@ -2236,7 +2262,6 @@ static int ram_save_setup(QEMUFile *f, void *opaque) } rcu_read_unlock(); - compress_threads_save_setup(); ram_control_before_iterate(f, RAM_CONTROL_SETUP); ram_control_after_iterate(f, RAM_CONTROL_SETUP); From patchwork Tue Mar 27 09:10:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 892030 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="mBY8rEM9"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40B2Dw34Lwz9s0b for ; Wed, 28 Mar 2018 20:11:28 +1100 (AEDT) Received: from localhost ([::1]:38154 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f1777-0000t0-Q3 for incoming@patchwork.ozlabs.org; Wed, 28 Mar 2018 05:11:25 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56646) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f176K-0000rF-LJ for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f176H-0004yP-Vj for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:36 -0400 Received: from mail-pf0-x243.google.com ([2607:f8b0:400e:c00::243]:45875) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1f176H-0004yA-Md for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:33 -0400 Received: by mail-pf0-x243.google.com with SMTP id l27so730936pfk.12 for ; Wed, 28 Mar 2018 02:10:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=WZz3mziASCDosDIFuMNrw4HmHLseGAEClkhPtP4ywK0=; b=mBY8rEM9R++0xUsCYTCa5b0glqtnt0h/dbPNm0RrefA5omIgT+8QvtWIH8OD0gPXW3 SvyxQpfzuVRCNjcDYG8oweqn0TlIJFJ3q4R3zOTF5peD/GRkQanU9nWUkuWftI3i1jTy p05IDvl+yPQGxqoGFFzITECbIePmxm2KrEDN77fVEURnBTcUpCprxMs5kQ5HqDC1uI31 DE6VhvkECzOUZQUK6S+K7pX9hGqxX/fbsap2SD5Hnhl8ocyVoaaL7P2jwyQlRwu9iqzx QtldUURHc4HzjyuKH2IIdPEyGfUaa7Yp2qH56GYRxmZ23TwcEPz+NVXCbK6uDTBJ49S+ qSHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=WZz3mziASCDosDIFuMNrw4HmHLseGAEClkhPtP4ywK0=; b=tSar46Jwpp63rNQMnD6cE55Yr6WzuvOljMnbrun4HBwrzLaqd/Dpghxj52iwFFvLzi 4JS+hOaiDGU9XeDuHljyzVNAr6sL+kA91gM4Ui9Det9OYvnmv+yiUMPMm2pHdVXoZjNZ e3qfijU8CzS/tbQgYlEmiGuFMK24RKzpNjiCJFgMNEszqdhT0xwP/0wttXoOVOhPlU5G pIOWP6hIdAB5il2QHa+hzHIiM/ZwrYWejAqySRnTZWHu1dkveCZEDaL7ZqsDAY24Wy9A GLUtLMZd5TXBEhxLe4NyqjTSXiPaSahdEG3Wg3Umerle6qZqONfv7GpdVefLuqgzmYlz 73Bg== X-Gm-Message-State: AElRT7FfLc9DUg9fzmsuPUr/dt/PsiW7toH1aYK4MrkT5Jq9lIIEpU8E KY5GA5i7VvhUR01NSzzJfWU= X-Google-Smtp-Source: AIpwx49jTVkP1Ep+Qc21md4hSDuongCtBMMGvm81cliiWCGBmylyg43ROHtJqziCTRS/fTUXVpc3dg== X-Received: by 10.98.93.149 with SMTP id n21mr2318913pfj.222.1522228232815; Wed, 28 Mar 2018 02:10:32 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.38]) by smtp.gmail.com with ESMTPSA id q9sm5590648pgs.89.2018.03.28.02.10.29 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 28 Mar 2018 02:10:32 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 27 Mar 2018 17:10:36 +0800 Message-Id: <20180327091043.30220-4-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180327091043.30220-1-xiaoguangrong@tencent.com> References: <20180327091043.30220-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::243 Subject: [Qemu-devel] [PATCH v2 03/10] migration: stop decompression to allocate and free memory frequently X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong Current code uses uncompress() to decompress memory which manages memory internally, that causes huge memory is allocated and freed very frequently, more worse, frequently returning memory to kernel will flush TLBs So, we maintain the memory by ourselves and reuse it for each decompression Reviewed-by: Jiang Biao Signed-off-by: Xiao Guangrong --- migration/ram.c | 110 ++++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 80 insertions(+), 30 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index e043a192e1..6b699650ca 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -281,6 +281,7 @@ struct DecompressParam { void *des; uint8_t *compbuf; int len; + z_stream stream; }; typedef struct DecompressParam DecompressParam; @@ -2526,6 +2527,31 @@ void ram_handle_compressed(void *host, uint8_t ch, uint64_t size) } } +/* return the size after decompression, or negative value on error */ +static int +qemu_uncompress_data(z_stream *stream, uint8_t *dest, size_t dest_len, + const uint8_t *source, size_t source_len) +{ + int err; + + err = inflateReset(stream); + if (err != Z_OK) { + return -1; + } + + stream->avail_in = source_len; + stream->next_in = (uint8_t *)source; + stream->avail_out = dest_len; + stream->next_out = dest; + + err = inflate(stream, Z_NO_FLUSH); + if (err != Z_STREAM_END) { + return -1; + } + + return stream->total_out; +} + static void *do_data_decompress(void *opaque) { DecompressParam *param = opaque; @@ -2542,13 +2568,13 @@ static void *do_data_decompress(void *opaque) qemu_mutex_unlock(¶m->mutex); pagesize = TARGET_PAGE_SIZE; - /* uncompress() will return failed in some case, especially - * when the page is dirted when doing the compression, it's - * not a problem because the dirty page will be retransferred + /* qemu_uncompress_data() will return failed in some case, + * especially when the page is dirtied when doing the compression, + * it's not a problem because the dirty page will be retransferred * and uncompress() won't break the data in other pages. */ - uncompress((Bytef *)des, &pagesize, - (const Bytef *)param->compbuf, len); + qemu_uncompress_data(¶m->stream, des, pagesize, param->compbuf, + len); qemu_mutex_lock(&decomp_done_lock); param->done = true; @@ -2583,30 +2609,6 @@ static void wait_for_decompress_done(void) qemu_mutex_unlock(&decomp_done_lock); } -static void compress_threads_load_setup(void) -{ - int i, thread_count; - - if (!migrate_use_compression()) { - return; - } - thread_count = migrate_decompress_threads(); - decompress_threads = g_new0(QemuThread, thread_count); - decomp_param = g_new0(DecompressParam, thread_count); - qemu_mutex_init(&decomp_done_lock); - qemu_cond_init(&decomp_done_cond); - for (i = 0; i < thread_count; i++) { - qemu_mutex_init(&decomp_param[i].mutex); - qemu_cond_init(&decomp_param[i].cond); - decomp_param[i].compbuf = g_malloc0(compressBound(TARGET_PAGE_SIZE)); - decomp_param[i].done = true; - decomp_param[i].quit = false; - qemu_thread_create(decompress_threads + i, "decompress", - do_data_decompress, decomp_param + i, - QEMU_THREAD_JOINABLE); - } -} - static void compress_threads_load_cleanup(void) { int i, thread_count; @@ -2616,16 +2618,27 @@ static void compress_threads_load_cleanup(void) } thread_count = migrate_decompress_threads(); for (i = 0; i < thread_count; i++) { + /* see the comments in compress_threads_save_cleanup() */ + if (!decomp_param[i].stream.opaque) { + break; + } + qemu_mutex_lock(&decomp_param[i].mutex); decomp_param[i].quit = true; qemu_cond_signal(&decomp_param[i].cond); qemu_mutex_unlock(&decomp_param[i].mutex); } for (i = 0; i < thread_count; i++) { + if (!decomp_param[i].stream.opaque) { + break; + } + qemu_thread_join(decompress_threads + i); qemu_mutex_destroy(&decomp_param[i].mutex); qemu_cond_destroy(&decomp_param[i].cond); g_free(decomp_param[i].compbuf); + inflateEnd(&decomp_param[i].stream); + decomp_param[i].stream.opaque = NULL; } g_free(decompress_threads); g_free(decomp_param); @@ -2633,6 +2646,40 @@ static void compress_threads_load_cleanup(void) decomp_param = NULL; } +static int compress_threads_load_setup(void) +{ + int i, thread_count; + + if (!migrate_use_compression()) { + return 0; + } + + thread_count = migrate_decompress_threads(); + decompress_threads = g_new0(QemuThread, thread_count); + decomp_param = g_new0(DecompressParam, thread_count); + qemu_mutex_init(&decomp_done_lock); + qemu_cond_init(&decomp_done_cond); + for (i = 0; i < thread_count; i++) { + if (inflateInit(&decomp_param[i].stream) != Z_OK) { + goto exit; + } + decomp_param[i].stream.opaque = &decomp_param[i]; + + qemu_mutex_init(&decomp_param[i].mutex); + qemu_cond_init(&decomp_param[i].cond); + decomp_param[i].compbuf = g_malloc0(compressBound(TARGET_PAGE_SIZE)); + decomp_param[i].done = true; + decomp_param[i].quit = false; + qemu_thread_create(decompress_threads + i, "decompress", + do_data_decompress, decomp_param + i, + QEMU_THREAD_JOINABLE); + } + return 0; +exit: + compress_threads_load_cleanup(); + return -1; +} + static void decompress_data_with_multi_threads(QEMUFile *f, void *host, int len) { @@ -2672,8 +2719,11 @@ static void decompress_data_with_multi_threads(QEMUFile *f, */ static int ram_load_setup(QEMUFile *f, void *opaque) { + if (compress_threads_load_setup()) { + return -1; + } + xbzrle_load_setup(); - compress_threads_load_setup(); ramblock_recv_map_init(); return 0; } From patchwork Tue Mar 27 09:10:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 892041 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="ZAAD0BPT"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40B2Nb1vLLz9s0m for ; Wed, 28 Mar 2018 20:18:06 +1100 (AEDT) Received: from localhost ([::1]:38184 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f17DX-0006P9-JU for incoming@patchwork.ozlabs.org; Wed, 28 Mar 2018 05:18:03 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56664) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f176M-0000sj-JX for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f176L-0004zo-61 for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:38 -0400 Received: from mail-pl0-x244.google.com ([2607:f8b0:400e:c01::244]:44892) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1f176K-0004zO-U4 for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:37 -0400 Received: by mail-pl0-x244.google.com with SMTP id 9-v6so1196446ple.11 for ; Wed, 28 Mar 2018 02:10:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=IadA7RXf8Bc1iPVuEx3SNmtkcRJ7JNpOk6YbIEwuGgY=; b=ZAAD0BPT6uqEGXKOtNyn1lrNkODhmdB7MeXdPjF1wuRf4vyR0Q2lpxTsOL1a/ExI6b x9EVbLGC6WqruN3wsKXQBzHdYCvlVIf2IqwH17IQfdwQ6VtoB7JkpYaSbPASbq60rG/N xbiqO30B/aipUxXJOjnjnGz0Mdgq9iZDmRlJKYRf3EuYEHRMUs0ByGHxkSjH7fvVIq4A AyRtbFuggC4BzZqJcOeYzxsRz+t7m4evMxkKsIC83BvVWRq8TgKUEQ0CqUQf69k+ASR9 XWrCQC+uCA8I1LY+WjOSUVK1H4gSzZijShJbojjHvTr9S2IfWL2Lc8I9IEwLvP3iypKe U6SA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=IadA7RXf8Bc1iPVuEx3SNmtkcRJ7JNpOk6YbIEwuGgY=; b=J7sn/w4ssfQDvhljbsG5YjP/bxl9McnhSDTZ1D7s9YGQ7iAIRteFcrtfC01bvyoW3f hG/nh0WBd3TEGLgeFtMJsfJHTJBoclMy5uhW4zs8/4kxw2rFGcYSr9ShZvg9UATd8V/s tN05eAfVFvMzUmwtRkcg0dOIeAdY3fAJvjRdPfNG1mcVK/8nG0AvwBmNJo9U9ScIWpxH iBLciwL7Xo6jnCdt5+p3XrNRwK/dXZnSHFrj5D6wju4bDi1kPj/iVlFUU9+k01dJZXV4 KYszvbveD1wnqxEkiLiQG4/myx8xkoqmPMjtqU+oe2TDmFFW+elV0kEwJufp+h3BG8ol 4IaQ== X-Gm-Message-State: AElRT7HxVLi/DpGF31GV+VZrb6I7VYYTNxIZeD9XMiqhBAds5lqpN4Tn Z5hM/7CuhrZi4PGAYaMIAz4= X-Google-Smtp-Source: AIpwx48gggc5V4EdtW6uR0gdtRl4LYvhFkk9C5kyekQODYNwOv2rewOJovLigUvqEihhyo6+FMn89A== X-Received: by 2002:a17:902:b7cc:: with SMTP id v12-v6mr2977255plz.237.1522228236078; Wed, 28 Mar 2018 02:10:36 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.38]) by smtp.gmail.com with ESMTPSA id q9sm5590648pgs.89.2018.03.28.02.10.33 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 28 Mar 2018 02:10:35 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 27 Mar 2018 17:10:37 +0800 Message-Id: <20180327091043.30220-5-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180327091043.30220-1-xiaoguangrong@tencent.com> References: <20180327091043.30220-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::244 Subject: [Qemu-devel] [PATCH v2 04/10] migration: detect compression and decompression errors X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong Currently the page being compressed is allowed to be updated by the VM on the source QEMU, correspondingly the destination QEMU just ignores the decompression error. However, we completely miss the chance to catch real errors, then the VM is corrupted silently To make the migration more robuster, we copy the page to a buffer first to avoid it being written by VM, then detect and handle the errors of both compression and decompression errors properly Signed-off-by: Xiao Guangrong --- migration/qemu-file.c | 4 ++-- migration/ram.c | 55 +++++++++++++++++++++++++++++++++++---------------- 2 files changed, 40 insertions(+), 19 deletions(-) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index e924cc23c5..a7614e8c28 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -710,9 +710,9 @@ ssize_t qemu_put_compression_data(QEMUFile *f, z_stream *stream, blen = qemu_compress_data(stream, f->buf + f->buf_index + sizeof(int32_t), blen, p, size); if (blen < 0) { - error_report("Compress Failed!"); - return 0; + return -1; } + qemu_put_be32(f, blen); if (f->ops->writev_buffer) { add_to_iovec(f, f->buf + f->buf_index, blen, false); diff --git a/migration/ram.c b/migration/ram.c index 6b699650ca..e85191c1cb 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -269,7 +269,10 @@ struct CompressParam { QemuCond cond; RAMBlock *block; ram_addr_t offset; + + /* internally used fields */ z_stream stream; + uint8_t *originbuf; }; typedef struct CompressParam CompressParam; @@ -278,6 +281,7 @@ struct DecompressParam { bool quit; QemuMutex mutex; QemuCond cond; + QEMUFile *file; void *des; uint8_t *compbuf; int len; @@ -302,7 +306,7 @@ static QemuMutex decomp_done_lock; static QemuCond decomp_done_cond; static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, - ram_addr_t offset); + ram_addr_t offset, uint8_t *source_buf); static void *do_data_compress(void *opaque) { @@ -318,7 +322,8 @@ static void *do_data_compress(void *opaque) param->block = NULL; qemu_mutex_unlock(¶m->mutex); - do_compress_ram_page(param->file, ¶m->stream, block, offset); + do_compress_ram_page(param->file, ¶m->stream, block, offset, + param->originbuf); qemu_mutex_lock(&comp_done_lock); param->done = true; @@ -372,6 +377,7 @@ static void compress_threads_save_cleanup(void) qemu_mutex_destroy(&comp_param[i].mutex); qemu_cond_destroy(&comp_param[i].cond); deflateEnd(&comp_param[i].stream); + g_free(comp_param[i].originbuf); comp_param[i].stream.opaque = NULL; } qemu_mutex_destroy(&comp_done_lock); @@ -395,8 +401,14 @@ static int compress_threads_save_setup(void) qemu_cond_init(&comp_done_cond); qemu_mutex_init(&comp_done_lock); for (i = 0; i < thread_count; i++) { + comp_param[i].originbuf = g_try_malloc(TARGET_PAGE_SIZE); + if (!comp_param[i].originbuf) { + goto exit; + } + if (deflateInit(&comp_param[i].stream, migrate_compress_level()) != Z_OK) { + g_free(comp_param[i].originbuf); goto exit; } comp_param[i].stream.opaque = &comp_param[i]; @@ -1055,7 +1067,7 @@ static int ram_save_page(RAMState *rs, PageSearchStatus *pss, bool last_stage) } static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, - ram_addr_t offset) + ram_addr_t offset, uint8_t *source_buf) { RAMState *rs = ram_state; int bytes_sent, blen; @@ -1063,7 +1075,14 @@ static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, bytes_sent = save_page_header(rs, f, block, offset | RAM_SAVE_FLAG_COMPRESS_PAGE); - blen = qemu_put_compression_data(f, stream, p, TARGET_PAGE_SIZE); + + /* + * copy it to a internal buffer to avoid it being modified by VM + * so that we can catch up the error during compression and + * decompression + */ + memcpy(source_buf, p, TARGET_PAGE_SIZE); + blen = qemu_put_compression_data(f, stream, source_buf, TARGET_PAGE_SIZE); if (blen < 0) { bytes_sent = 0; qemu_file_set_error(migrate_get_current()->to_dst_file, blen); @@ -2557,7 +2576,7 @@ static void *do_data_decompress(void *opaque) DecompressParam *param = opaque; unsigned long pagesize; uint8_t *des; - int len; + int len, ret; qemu_mutex_lock(¶m->mutex); while (!param->quit) { @@ -2568,13 +2587,13 @@ static void *do_data_decompress(void *opaque) qemu_mutex_unlock(¶m->mutex); pagesize = TARGET_PAGE_SIZE; - /* qemu_uncompress_data() will return failed in some case, - * especially when the page is dirtied when doing the compression, - * it's not a problem because the dirty page will be retransferred - * and uncompress() won't break the data in other pages. - */ - qemu_uncompress_data(¶m->stream, des, pagesize, param->compbuf, - len); + + ret = qemu_uncompress_data(¶m->stream, des, pagesize, + param->compbuf, len); + if (ret < 0) { + error_report("decompress data failed"); + qemu_file_set_error(param->file, ret); + } qemu_mutex_lock(&decomp_done_lock); param->done = true; @@ -2591,12 +2610,12 @@ static void *do_data_decompress(void *opaque) return NULL; } -static void wait_for_decompress_done(void) +static int wait_for_decompress_done(QEMUFile *f) { int idx, thread_count; if (!migrate_use_compression()) { - return; + return 0; } thread_count = migrate_decompress_threads(); @@ -2607,6 +2626,7 @@ static void wait_for_decompress_done(void) } } qemu_mutex_unlock(&decomp_done_lock); + return qemu_file_get_error(f); } static void compress_threads_load_cleanup(void) @@ -2646,7 +2666,7 @@ static void compress_threads_load_cleanup(void) decomp_param = NULL; } -static int compress_threads_load_setup(void) +static int compress_threads_load_setup(QEMUFile *f) { int i, thread_count; @@ -2665,6 +2685,7 @@ static int compress_threads_load_setup(void) } decomp_param[i].stream.opaque = &decomp_param[i]; + decomp_param[i].file = f; qemu_mutex_init(&decomp_param[i].mutex); qemu_cond_init(&decomp_param[i].cond); decomp_param[i].compbuf = g_malloc0(compressBound(TARGET_PAGE_SIZE)); @@ -2719,7 +2740,7 @@ static void decompress_data_with_multi_threads(QEMUFile *f, */ static int ram_load_setup(QEMUFile *f, void *opaque) { - if (compress_threads_load_setup()) { + if (compress_threads_load_setup(f)) { return -1; } @@ -3074,7 +3095,7 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id) } } - wait_for_decompress_done(); + ret |= wait_for_decompress_done(f); rcu_read_unlock(); trace_ram_load_complete(ret, seq_iter); return ret; From patchwork Tue Mar 27 09:10:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 892040 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="dQuQX8zT"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40B2MY3BmVz9s0b for ; Wed, 28 Mar 2018 20:17:13 +1100 (AEDT) Received: from localhost ([::1]:38183 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f17Cg-0005qL-UL for incoming@patchwork.ozlabs.org; Wed, 28 Mar 2018 05:17:10 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56685) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f176P-0000w5-Vo for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f176O-00053R-Cz for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:41 -0400 Received: from mail-pg0-x232.google.com ([2607:f8b0:400e:c05::232]:41949) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1f176O-00051e-55 for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:40 -0400 Received: by mail-pg0-x232.google.com with SMTP id t10so707091pgv.8 for ; Wed, 28 Mar 2018 02:10:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=sFu4QSTOV4tjzMqswvpkzJdod0DV1g8C2XOfU7z++68=; b=dQuQX8zT08hHGELQImGx2uWAh8IYSRVvM6hEZliQCCUmwZSoVAMiyV9H4lUaNOkJCr GZ6r5DmDlgMd9koZHWGEHQzVWp2Labze6vvNUrp3cQteg6iv3Ml9VpWzrAOCLc2gioJb qTwFrahq37SpLFk7RBKl5hfP0weEXm0asaTd07AD7QXXh9Wm1CrSntfIYIKmwtN72yP9 eDVL1+cbO/rf0p0pnbgSjJyizvY8HfN5+I3xJ4INOkYb4PbaDq0IZrOSFvhO+fxhptPI f1PIaZyoodM7oIyunhOUVDSPuEugShuerp+s2NYthLlkRPelq2bE9qRWtu/4iXL+G+fr i2kQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=sFu4QSTOV4tjzMqswvpkzJdod0DV1g8C2XOfU7z++68=; b=AY6fHLMkfLilV64DukD3LC4m/dqQYmjm79kny7xMPk510pwyceGVVa4jVtTYOf6KO9 0JjqOY6lZDQbZksg9w59Y//Oq6alrLw0dbdl3nyqw0MSet2YIAfcrUAs2f5KJD9PgbYj j6yAgK5dFVZWiwSgHPWBaoTLNT1+I5YnrmWWn2mU199CuRLb0jkMCpxcx7BrCM2yY6WS VYLKkRlRkN3q5+osR0rndC3ZXMPvygtpJoeoj8/cNrd8M21vpYyExPjUgKehU5eHyoq8 8h5JcTdROIjwzfejF1tC0sxuqNgWnyxNATa2RFTJ8OZxl1XfvPgC7y+cU+qSOFgX0ar8 tPXA== X-Gm-Message-State: AElRT7FC7dDS88wGt7CYXCyjQA/4V5BeHxFRdW7ENT2NXKlSrZzjc974 UjMCCuqPT9qXR8z1aBrAWrg= X-Google-Smtp-Source: AIpwx49Pk/4GJXEyFFlz3aBmb4Lgws7Pc8xRwopRMQD1NfU2R8ByYaP49JPBW53w04oPfuPU136Z0g== X-Received: by 10.99.123.71 with SMTP id k7mr2052468pgn.134.1522228239279; Wed, 28 Mar 2018 02:10:39 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.38]) by smtp.gmail.com with ESMTPSA id q9sm5590648pgs.89.2018.03.28.02.10.36 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 28 Mar 2018 02:10:38 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 27 Mar 2018 17:10:38 +0800 Message-Id: <20180327091043.30220-6-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180327091043.30220-1-xiaoguangrong@tencent.com> References: <20180327091043.30220-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::232 Subject: [Qemu-devel] [PATCH v2 05/10] migration: introduce control_save_page() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong Abstract the common function control_save_page() to cleanup the code, no logic is changed Reviewed-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Xiao Guangrong --- migration/ram.c | 174 +++++++++++++++++++++++++++++--------------------------- 1 file changed, 89 insertions(+), 85 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index e85191c1cb..fc1fde7bb7 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -976,6 +976,44 @@ static void ram_release_pages(const char *rbname, uint64_t offset, int pages) ram_discard_range(rbname, offset, pages << TARGET_PAGE_BITS); } +/* + * @pages: the number of pages written by the control path, + * < 0 - error + * > 0 - number of pages written + * + * Return true if the pages has been saved, otherwise false is returned. + */ +static bool control_save_page(RAMState *rs, RAMBlock *block, ram_addr_t offset, + int *pages) +{ + uint64_t bytes_xmit = 0; + int ret; + + *pages = -1; + ret = ram_control_save_page(rs->f, block->offset, offset, TARGET_PAGE_SIZE, + &bytes_xmit); + if (ret == RAM_SAVE_CONTROL_NOT_SUPP) { + return false; + } + + if (bytes_xmit) { + ram_counters.transferred += bytes_xmit; + *pages = 1; + } + + if (ret == RAM_SAVE_CONTROL_DELAYED) { + return true; + } + + if (bytes_xmit > 0) { + ram_counters.normal++; + } else if (bytes_xmit == 0) { + ram_counters.duplicate++; + } + + return true; +} + /** * ram_save_page: send the given page to the stream * @@ -992,56 +1030,36 @@ static void ram_release_pages(const char *rbname, uint64_t offset, int pages) static int ram_save_page(RAMState *rs, PageSearchStatus *pss, bool last_stage) { int pages = -1; - uint64_t bytes_xmit; - ram_addr_t current_addr; uint8_t *p; - int ret; bool send_async = true; RAMBlock *block = pss->block; ram_addr_t offset = pss->page << TARGET_PAGE_BITS; + ram_addr_t current_addr = block->offset + offset; p = block->host + offset; trace_ram_save_page(block->idstr, (uint64_t)offset, p); - /* In doubt sent page as normal */ - bytes_xmit = 0; - ret = ram_control_save_page(rs->f, block->offset, - offset, TARGET_PAGE_SIZE, &bytes_xmit); - if (bytes_xmit) { - ram_counters.transferred += bytes_xmit; - pages = 1; + if (control_save_page(rs, block, offset, &pages)) { + return pages; } XBZRLE_cache_lock(); - - current_addr = block->offset + offset; - - if (ret != RAM_SAVE_CONTROL_NOT_SUPP) { - if (ret != RAM_SAVE_CONTROL_DELAYED) { - if (bytes_xmit > 0) { - ram_counters.normal++; - } else if (bytes_xmit == 0) { - ram_counters.duplicate++; - } - } - } else { - pages = save_zero_page(rs, block, offset); - if (pages > 0) { - /* Must let xbzrle know, otherwise a previous (now 0'd) cached - * page would be stale + pages = save_zero_page(rs, block, offset); + if (pages > 0) { + /* Must let xbzrle know, otherwise a previous (now 0'd) cached + * page would be stale + */ + xbzrle_cache_zero_page(rs, current_addr); + ram_release_pages(block->idstr, offset, pages); + } else if (!rs->ram_bulk_stage && + !migration_in_postcopy() && migrate_use_xbzrle()) { + pages = save_xbzrle_page(rs, &p, current_addr, block, + offset, last_stage); + if (!last_stage) { + /* Can't send this cached data async, since the cache page + * might get updated before it gets to the wire */ - xbzrle_cache_zero_page(rs, current_addr); - ram_release_pages(block->idstr, offset, pages); - } else if (!rs->ram_bulk_stage && - !migration_in_postcopy() && migrate_use_xbzrle()) { - pages = save_xbzrle_page(rs, &p, current_addr, block, - offset, last_stage); - if (!last_stage) { - /* Can't send this cached data async, since the cache page - * might get updated before it gets to the wire - */ - send_async = false; - } + send_async = false; } } @@ -1176,63 +1194,49 @@ static int ram_save_compressed_page(RAMState *rs, PageSearchStatus *pss, bool last_stage) { int pages = -1; - uint64_t bytes_xmit = 0; uint8_t *p; - int ret; RAMBlock *block = pss->block; ram_addr_t offset = pss->page << TARGET_PAGE_BITS; p = block->host + offset; - ret = ram_control_save_page(rs->f, block->offset, - offset, TARGET_PAGE_SIZE, &bytes_xmit); - if (bytes_xmit) { - ram_counters.transferred += bytes_xmit; - pages = 1; + if (control_save_page(rs, block, offset, &pages)) { + return pages; } - if (ret != RAM_SAVE_CONTROL_NOT_SUPP) { - if (ret != RAM_SAVE_CONTROL_DELAYED) { - if (bytes_xmit > 0) { - ram_counters.normal++; - } else if (bytes_xmit == 0) { - ram_counters.duplicate++; - } + + /* When starting the process of a new block, the first page of + * the block should be sent out before other pages in the same + * block, and all the pages in last block should have been sent + * out, keeping this order is important, because the 'cont' flag + * is used to avoid resending the block name. + */ + if (block != rs->last_sent_block) { + flush_compressed_data(rs); + pages = save_zero_page(rs, block, offset); + if (pages > 0) { + ram_release_pages(block->idstr, offset, pages); + } else { + /* + * Make sure the first page is sent out before other pages. + * + * we post it as normal page as compression will take much + * CPU resource. + */ + ram_counters.transferred += save_page_header(rs, rs->f, block, + offset | RAM_SAVE_FLAG_PAGE); + qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE, + migrate_release_ram() & + migration_in_postcopy()); + ram_counters.transferred += TARGET_PAGE_SIZE; + ram_counters.normal++; + pages = 1; } } else { - /* When starting the process of a new block, the first page of - * the block should be sent out before other pages in the same - * block, and all the pages in last block should have been sent - * out, keeping this order is important, because the 'cont' flag - * is used to avoid resending the block name. - */ - if (block != rs->last_sent_block) { - flush_compressed_data(rs); - pages = save_zero_page(rs, block, offset); - if (pages > 0) { - ram_release_pages(block->idstr, offset, pages); - } else { - /* - * Make sure the first page is sent out before other pages. - * - * we post it as normal page as compression will take much - * CPU resource. - */ - ram_counters.transferred += save_page_header(rs, rs->f, block, - offset | RAM_SAVE_FLAG_PAGE); - qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE, - migrate_release_ram() & - migration_in_postcopy()); - ram_counters.transferred += TARGET_PAGE_SIZE; - ram_counters.normal++; - pages = 1; - } + pages = save_zero_page(rs, block, offset); + if (pages == -1) { + pages = compress_page_with_multi_thread(rs, block, offset); } else { - pages = save_zero_page(rs, block, offset); - if (pages == -1) { - pages = compress_page_with_multi_thread(rs, block, offset); - } else { - ram_release_pages(block->idstr, offset, pages); - } + ram_release_pages(block->idstr, offset, pages); } } From patchwork Tue Mar 27 09:10:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 892034 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="ZU6gUdfi"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40B2JK4CQtz9s0b for ; Wed, 28 Mar 2018 20:14:25 +1100 (AEDT) Received: from localhost ([::1]:38165 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f179z-0003LF-35 for incoming@patchwork.ozlabs.org; Wed, 28 Mar 2018 05:14:23 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56705) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f176S-0000yn-OP for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f176R-00057q-8Z for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:44 -0400 Received: from mail-pg0-x244.google.com ([2607:f8b0:400e:c05::244]:46097) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1f176R-00057d-3B for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:43 -0400 Received: by mail-pg0-x244.google.com with SMTP id t12so701897pgp.13 for ; Wed, 28 Mar 2018 02:10:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=BowyGQOXuV8DY4UpAHOoLnlrcBFl68/F7Px7ZtSi3+A=; b=ZU6gUdfiHTnnn5dahmpe5CKuHTsbdy6K2KwDscYuIjHYchNvzORiRGk1wJuxtHmgi/ dz4fm8+hi5gbqr8miLSeuW+A+IYqOB06RfcgdMD9ntsVrt7t5ZuDcfkjWhVgRHlsEdeM t39G9/5HY5Y1KEk/ERqKvV4aCNvf006CtLEJ/3mh9kyCKH6fw3ujWkV4a1QEzRbM5Rpt IC6LJ5HbDt1N7q6gPNASli+lncXqnW+Odau5cuX7oFxCheQRuepAdmLvEIN96Cp2cy/x cEdFIGSs8x6FpZRE9+vt2FNjvJW1h+EWqv7lO4VIlesGRToRPsa4NxjbvIfVBbGvK46W H94A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=BowyGQOXuV8DY4UpAHOoLnlrcBFl68/F7Px7ZtSi3+A=; b=GsJ0wwf1IJqRumZplHAY0QB+w7QpvQn2mFQrivuCag8CNZ3HrcoT0U7n8EvMMXQu7/ 8Ytz2CcDVA5cemxFOM35CLdW9elHOs1qndtrCuWHqMGY6PIvslSNzKWAieuD/qGkOIdF TdUf5Yha/2R4LryMtLTnlJKt+1IbLZRrzFBaQ9PdNpfN2qwS6/DJz/3eod/KJu2eyN4/ x3HcbS46aAi5RFy1YpRZ5Z4syHIqqwHYiTWhR0//A1mvj6bzvz9BVMRJBU/C+iIj0okK AnvV/CZvqXM7FrJonmIsQGPyawoPC0tNthKpzMDJZkFjNHrTMwZWRyaMViSGkJfa8dH9 vh1Q== X-Gm-Message-State: AElRT7Hiiar4+QgvL400v5rx5wtainImBNbijrsrqtFE11s3QU33nZfe KWHt4yOdG5xLCcs9UPowVCc= X-Google-Smtp-Source: AIpwx4/u8OvAJ8cjQqNgXoptL0HnnJ2eK+MFFx5G65OCsVOrz9knDwVPX8/q5Y+G1h0MeCTQB7q4oQ== X-Received: by 10.101.82.12 with SMTP id o12mr1983638pgp.126.1522228242384; Wed, 28 Mar 2018 02:10:42 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.38]) by smtp.gmail.com with ESMTPSA id q9sm5590648pgs.89.2018.03.28.02.10.39 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 28 Mar 2018 02:10:41 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 27 Mar 2018 17:10:39 +0800 Message-Id: <20180327091043.30220-7-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180327091043.30220-1-xiaoguangrong@tencent.com> References: <20180327091043.30220-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::244 Subject: [Qemu-devel] [PATCH v2 06/10] migration: move some code ram_save_host_page X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong Move some code from ram_save_target_page() to ram_save_host_page() to make it be more readable for latter patches that dramatically clean ram_save_target_page() up Signed-off-by: Xiao Guangrong Reviewed-by: Peter Xu --- migration/ram.c | 43 +++++++++++++++++++------------------------ 1 file changed, 19 insertions(+), 24 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index fc1fde7bb7..104d3d3e9f 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1485,38 +1485,23 @@ err: * Returns the number of pages written * * @rs: current RAM state - * @ms: current migration state * @pss: data about the page we want to send * @last_stage: if we are at the completion stage */ static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss, bool last_stage) { - int res = 0; - - /* Check the pages is dirty and if it is send it */ - if (migration_bitmap_clear_dirty(rs, pss->block, pss->page)) { - /* - * If xbzrle is on, stop using the data compression after first - * round of migration even if compression is enabled. In theory, - * xbzrle can do better than compression. - */ - if (migrate_use_compression() && - (rs->ram_bulk_stage || !migrate_use_xbzrle())) { - res = ram_save_compressed_page(rs, pss, last_stage); - } else { - res = ram_save_page(rs, pss, last_stage); - } - - if (res < 0) { - return res; - } - if (pss->block->unsentmap) { - clear_bit(pss->page, pss->block->unsentmap); - } + /* + * If xbzrle is on, stop using the data compression after first + * round of migration even if compression is enabled. In theory, + * xbzrle can do better than compression. + */ + if (migrate_use_compression() && + (rs->ram_bulk_stage || !migrate_use_xbzrle())) { + return ram_save_compressed_page(rs, pss, last_stage); } - return res; + return ram_save_page(rs, pss, last_stage); } /** @@ -1545,12 +1530,22 @@ static int ram_save_host_page(RAMState *rs, PageSearchStatus *pss, qemu_ram_pagesize(pss->block) >> TARGET_PAGE_BITS; do { + /* Check the pages is dirty and if it is send it */ + if (!migration_bitmap_clear_dirty(rs, pss->block, pss->page)) { + pss->page++; + continue; + } + tmppages = ram_save_target_page(rs, pss, last_stage); if (tmppages < 0) { return tmppages; } pages += tmppages; + if (pss->block->unsentmap) { + clear_bit(pss->page, pss->block->unsentmap); + } + pss->page++; } while ((pss->page & (pagesize_bits - 1)) && offset_in_ramblock(pss->block, pss->page << TARGET_PAGE_BITS)); From patchwork Tue Mar 27 09:10:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 892042 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="mglAyVAj"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40B2Qt1pWXz9s0m for ; Wed, 28 Mar 2018 20:20:04 +1100 (AEDT) Received: from localhost ([::1]:38193 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f17FR-0008Fg-48 for incoming@patchwork.ozlabs.org; Wed, 28 Mar 2018 05:20:01 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56740) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f176Z-000132-7p for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f176U-00059f-J0 for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:51 -0400 Received: from mail-pf0-x244.google.com ([2607:f8b0:400e:c00::244]:35815) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1f176U-00059J-D1 for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:46 -0400 Received: by mail-pf0-x244.google.com with SMTP id u86so740662pfd.2 for ; Wed, 28 Mar 2018 02:10:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=a4yJQ70gbcmr1a7GEjaWfWJaZjMnfyFa6mdz4JKGZqA=; b=mglAyVAjsravC+VR+USAPRf3Jabk4fGPO3Opva7Cm2f/xRw/U0BecIh7ET7ajt5Wfd U5Y5Xr/5nkdCKC3PizrMpHIOXomEjgGMomq8K+DJ1hBYdiFaE3m5VKO5Nkl5T8Qlc7m2 q4w9odSfBMBEJ9Z/dDspcaZ0tWQ8tMKvUthx6Vx0Ir6EkuiUAAdgOtstkvmjtOqxW0Pq 2xOHN3OfJ12hR/V1k3HbCetKKlDwS2kGy+YrZX6FIoXyZWqU/YtZn8nz7jSGE6V0KYzp ysRQcaJeL6KLB3OCe8VQtNx1QaghZPakvSw9cRDAeolxkFnUzHpqsgmkNDzle3fi56ew ISGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=a4yJQ70gbcmr1a7GEjaWfWJaZjMnfyFa6mdz4JKGZqA=; b=AtGTnRTorjaaet0oPBELMCQ9LGiqB+kA7KtsTMBPNHMC7Mio1oFwd6sOrE0bw7Wa+6 cWvGi8JDBcTBqDGyYa75ivKwxZyRue+F3ZRKdkxmsHoeRLLgUvACLHNfxruk+2cW2Xqg WrfpT3GeNCMWKCSVQHu7sa7as8cwOlsh0IrmIqbFccJuM4yl5XTwmLRTBr23IpLw+CiD zRyvUCSYkdg/2rhNdK3Nea/miMcgRiuputt6YAC8O/t2Um8VQrMb2+/NZOleT2BXCFrI aPlYFBbEy+PihCtH3cvTggMBq/CoxutwiW5czEltV/PM5SMmZylqJ/9qJss9L5nN9jXX j0lw== X-Gm-Message-State: AElRT7GoCAWPqnbdKXPvVPovsftEQarsHC+Ux1H0YuWNqmujIhBWQa24 gSOPbmXlaZhVJU2G/ERz2Y4= X-Google-Smtp-Source: AIpwx48uGE6OcQ8KMF1K71QtRYYJhNpMjdmsVntrQAQcgdLI6TPfNs2jE2AVnLKAYV3VYfGfbsILAw== X-Received: by 10.101.101.133 with SMTP id u5mr1967153pgv.151.1522228245674; Wed, 28 Mar 2018 02:10:45 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.38]) by smtp.gmail.com with ESMTPSA id q9sm5590648pgs.89.2018.03.28.02.10.42 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 28 Mar 2018 02:10:45 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 27 Mar 2018 17:10:40 +0800 Message-Id: <20180327091043.30220-8-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180327091043.30220-1-xiaoguangrong@tencent.com> References: <20180327091043.30220-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::244 Subject: [Qemu-devel] [PATCH v2 07/10] migration: move calling control_save_page to the common place X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong The function is called by both ram_save_page and ram_save_target_page, so move it to the common caller to cleanup the code Reviewed-by: Peter Xu Signed-off-by: Xiao Guangrong --- migration/ram.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 104d3d3e9f..ce3ef4382d 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1039,10 +1039,6 @@ static int ram_save_page(RAMState *rs, PageSearchStatus *pss, bool last_stage) p = block->host + offset; trace_ram_save_page(block->idstr, (uint64_t)offset, p); - if (control_save_page(rs, block, offset, &pages)) { - return pages; - } - XBZRLE_cache_lock(); pages = save_zero_page(rs, block, offset); if (pages > 0) { @@ -1200,10 +1196,6 @@ static int ram_save_compressed_page(RAMState *rs, PageSearchStatus *pss, p = block->host + offset; - if (control_save_page(rs, block, offset, &pages)) { - return pages; - } - /* When starting the process of a new block, the first page of * the block should be sent out before other pages in the same * block, and all the pages in last block should have been sent @@ -1491,6 +1483,14 @@ err: static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss, bool last_stage) { + RAMBlock *block = pss->block; + ram_addr_t offset = pss->page << TARGET_PAGE_BITS; + int res; + + if (control_save_page(rs, block, offset, &res)) { + return res; + } + /* * If xbzrle is on, stop using the data compression after first * round of migration even if compression is enabled. In theory, From patchwork Tue Mar 27 09:10:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 892031 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="kVsqDVgN"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40B2FV25Pbz9s08 for ; Wed, 28 Mar 2018 20:11:58 +1100 (AEDT) Received: from localhost ([::1]:38155 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f177b-00016z-N6 for incoming@patchwork.ozlabs.org; Wed, 28 Mar 2018 05:11:55 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56746) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f176a-00014O-PO for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f176Y-0005BK-3T for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:52 -0400 Received: from mail-pl0-x231.google.com ([2607:f8b0:400e:c01::231]:45356) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1f176X-0005Ay-Rq for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:50 -0400 Received: by mail-pl0-x231.google.com with SMTP id n15-v6so1193068plp.12 for ; Wed, 28 Mar 2018 02:10:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=XUDzYoUh4OyYELgkFPvZrzIEe7Aa5QqBLX/5X+9TspI=; b=kVsqDVgNp/ZVXjqvo9zLRfQj+/8H3qHxD5dWPVT8ny+u/gaKXJZ1KJ52FzzlpDXU/s An2L/6hf2/j/j5z1k+F93aU+f1vtr12EIZf9ZT1ZULDLjjFKgGVQd2S9ZI4E4qNluYSG yyiVI581LLD608gJFH02xGMBEOS/TGyadASfNsrPeMbyBb13dgHOQEvVqsl/NBbHBUuf 3VhyGfVJl/SIxDTmp8UW7SNXXrzxTHhKdCya44n1MsmTLeZ3sduBRGLBWfEydbV3b4I3 W+eHe4lSuOWDgc5pJ2temUGl3kShM43WS/GLiuH8hR4rU+p6PpyjZtWqpvT9zC/lOURf WEsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=XUDzYoUh4OyYELgkFPvZrzIEe7Aa5QqBLX/5X+9TspI=; b=M+5NGuzoasusqGELIxRKKOnDPDtUk8WFxQ9gnWenW08ZVqScQsYYlJ/X16a6wjECho WazL80dEF27YifisThXYslVtCgRjeIqks+3bbrdGSrlJ+11ccfd9Uc11771+tlMCPyny CuOstKxiRD+LQH7d75DuiIQ1IShM82lQLqwJAmXjXWh/lRVuG3HPwmBy2+lkrRE8Ii+W pzFZIyQinQUdlPwFJmDUGTb6oo6UPXMgO9BXtyHGCFHzfojeYj6/Ae7iuywuks3O6/uy 54m+OXsENLC5bmIq4/SI+munKHm1NN3hzZsC0WRf3hcgGIL7xSsVuhrWhQ4gZu4xHnco Eoyw== X-Gm-Message-State: AElRT7FsqBWAc1ipCs9WlBig2pWrgHAvWHr2qTQZ1yQVZJdwK7id3O30 jn69F0IAhQIETMMUnLWZ+c9hpw== X-Google-Smtp-Source: AIpwx4/P/Cm0KdUo3LVP4hyh4BY4Jy8coxhBbKrJ7INS9Gb5GebDAZXQOTKhR10P28/Pw+oa/QvbDg== X-Received: by 2002:a17:902:b602:: with SMTP id b2-v6mr2979378pls.11.1522228249059; Wed, 28 Mar 2018 02:10:49 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.38]) by smtp.gmail.com with ESMTPSA id q9sm5590648pgs.89.2018.03.28.02.10.45 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 28 Mar 2018 02:10:48 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 27 Mar 2018 17:10:41 +0800 Message-Id: <20180327091043.30220-9-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180327091043.30220-1-xiaoguangrong@tencent.com> References: <20180327091043.30220-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::231 Subject: [Qemu-devel] [PATCH v2 08/10] migration: move calling save_zero_page to the common place X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong save_zero_page() is always our first approach to try, move it to the common place before calling ram_save_compressed_page and ram_save_page Reviewed-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Xiao Guangrong --- migration/ram.c | 105 +++++++++++++++++++++++++++++++------------------------- 1 file changed, 59 insertions(+), 46 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index ce3ef4382d..771763985d 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1040,15 +1040,8 @@ static int ram_save_page(RAMState *rs, PageSearchStatus *pss, bool last_stage) trace_ram_save_page(block->idstr, (uint64_t)offset, p); XBZRLE_cache_lock(); - pages = save_zero_page(rs, block, offset); - if (pages > 0) { - /* Must let xbzrle know, otherwise a previous (now 0'd) cached - * page would be stale - */ - xbzrle_cache_zero_page(rs, current_addr); - ram_release_pages(block->idstr, offset, pages); - } else if (!rs->ram_bulk_stage && - !migration_in_postcopy() && migrate_use_xbzrle()) { + if (!rs->ram_bulk_stage && !migration_in_postcopy() && + migrate_use_xbzrle()) { pages = save_xbzrle_page(rs, &p, current_addr, block, offset, last_stage); if (!last_stage) { @@ -1196,40 +1189,23 @@ static int ram_save_compressed_page(RAMState *rs, PageSearchStatus *pss, p = block->host + offset; - /* When starting the process of a new block, the first page of - * the block should be sent out before other pages in the same - * block, and all the pages in last block should have been sent - * out, keeping this order is important, because the 'cont' flag - * is used to avoid resending the block name. - */ if (block != rs->last_sent_block) { - flush_compressed_data(rs); - pages = save_zero_page(rs, block, offset); - if (pages > 0) { - ram_release_pages(block->idstr, offset, pages); - } else { - /* - * Make sure the first page is sent out before other pages. - * - * we post it as normal page as compression will take much - * CPU resource. - */ - ram_counters.transferred += save_page_header(rs, rs->f, block, - offset | RAM_SAVE_FLAG_PAGE); - qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE, - migrate_release_ram() & - migration_in_postcopy()); - ram_counters.transferred += TARGET_PAGE_SIZE; - ram_counters.normal++; - pages = 1; - } + /* + * Make sure the first page is sent out before other pages. + * + * we post it as normal page as compression will take much + * CPU resource. + */ + ram_counters.transferred += save_page_header(rs, rs->f, block, + offset | RAM_SAVE_FLAG_PAGE); + qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE, + migrate_release_ram() & + migration_in_postcopy()); + ram_counters.transferred += TARGET_PAGE_SIZE; + ram_counters.normal++; + pages = 1; } else { - pages = save_zero_page(rs, block, offset); - if (pages == -1) { - pages = compress_page_with_multi_thread(rs, block, offset); - } else { - ram_release_pages(block->idstr, offset, pages); - } + pages = compress_page_with_multi_thread(rs, block, offset); } return pages; @@ -1471,6 +1447,24 @@ err: return -1; } +static bool save_page_use_compression(RAMState *rs) +{ + if (!migrate_use_compression()) { + return false; + } + + /* + * If xbzrle is on, stop using the data compression after first + * round of migration even if compression is enabled. In theory, + * xbzrle can do better than compression. + */ + if (rs->ram_bulk_stage || !migrate_use_xbzrle()) { + return true; + } + + return false; +} + /** * ram_save_target_page: save one target page * @@ -1492,12 +1486,31 @@ static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss, } /* - * If xbzrle is on, stop using the data compression after first - * round of migration even if compression is enabled. In theory, - * xbzrle can do better than compression. + * When starting the process of a new block, the first page of + * the block should be sent out before other pages in the same + * block, and all the pages in last block should have been sent + * out, keeping this order is important, because the 'cont' flag + * is used to avoid resending the block name. */ - if (migrate_use_compression() && - (rs->ram_bulk_stage || !migrate_use_xbzrle())) { + if (block != rs->last_sent_block && save_page_use_compression(rs)) { + flush_compressed_data(rs); + } + + res = save_zero_page(rs, block, offset); + if (res > 0) { + /* Must let xbzrle know, otherwise a previous (now 0'd) cached + * page would be stale + */ + if (!save_page_use_compression(rs)) { + XBZRLE_cache_lock(); + xbzrle_cache_zero_page(rs, block->offset + offset); + XBZRLE_cache_unlock(); + } + ram_release_pages(block->idstr, offset, res); + return res; + } + + if (save_page_use_compression(rs)) { return ram_save_compressed_page(rs, pss, last_stage); } From patchwork Tue Mar 27 09:10:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 892039 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="GOlo2Uih"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40B2MR5NrZz9s08 for ; Wed, 28 Mar 2018 20:17:06 +1100 (AEDT) Received: from localhost ([::1]:38181 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f17CW-0005k3-Pg for incoming@patchwork.ozlabs.org; Wed, 28 Mar 2018 05:17:00 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56758) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f176c-00015i-4B for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f176b-0005Cf-2i for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:54 -0400 Received: from mail-pl0-x22c.google.com ([2607:f8b0:400e:c01::22c]:39038) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1f176a-0005CN-Th for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:53 -0400 Received: by mail-pl0-x22c.google.com with SMTP id s24-v6so1201805plq.6 for ; Wed, 28 Mar 2018 02:10:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=F33N57sAf+wp/fVCaGOBBnoIR6diNc3lH+anQmNYRYY=; b=GOlo2Uih1Ab51AFvSJ3hITeyI658hkxbNY1B3mTkrISpx8OfUR8gSGsk+vJHMlULkU fJ5M2fKs28VSsWGIee7y/lF40vRHMwfIZOLwPJjc3vNL+EtmPkutt+NJBtE+/kMI6W8R +xpkBmKK0DvtTwHGMUpqoVFlNtmzuiVuqxQbU52M0lXjtE9EyCwKMqLJnEDXf+7IXBw/ WvZqhaTVlAkjGXQuLJ83eqLA8nvLGAHRXGQShlsmM9t4mONVWy6w1vd+4l5YLoc2dIOx xxdmX9nv4wzY2vdrTSy4oYqrQuQiI9d/4YLB+F3UXinMB3quwdvGLesuv4HhXWMd/okk PjzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=F33N57sAf+wp/fVCaGOBBnoIR6diNc3lH+anQmNYRYY=; b=s75ZQSG5CtOSSjPqSiTuuEmQoGOSHihLRPXewydcXn/Y25/h98ZVIf5ooXCljM2af0 l9gE5/pNwF4JtEkG9cqH/1Podz25BH/7COcoPFk965Culs9hRht2dMiBOhqVtdhdnlZW zBWBqb0OyrrMbaPY+mAJnxi5t3CSyknFOa6XDx4WxIojQnmZWhVvcn9KaE9xrjgtAdA0 XcQI55eGpttKUUnCUt04qsVmmr9JgRTmHXEGbRE+NHyGg4LucCB6Aex4arQhCK/FszJC Ltjv6+FRV0jg3GGHw2lxse6iBokeUDmt93Q0FoGBGtpVUw4+g965VBFVf18uEDY7BPgk c3ng== X-Gm-Message-State: AElRT7FBKHq9/pIFHIEHh/+2WA0CKjF7gGBVk3R2KP1Uy3t+nD+hcOFe Y4zBAu1Wy8Cv2jJCJSBXEoM= X-Google-Smtp-Source: AIpwx4/XMzDdRn7dpLLiKqLCG+J4fQE5IR4HbzoGBC3Z/ZZlSLuS6/lJTpqN2Tqp5zSdo10MfRVFxw== X-Received: by 2002:a17:902:6884:: with SMTP id i4-v6mr2998173plk.259.1522228252183; Wed, 28 Mar 2018 02:10:52 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.38]) by smtp.gmail.com with ESMTPSA id q9sm5590648pgs.89.2018.03.28.02.10.49 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 28 Mar 2018 02:10:51 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 27 Mar 2018 17:10:42 +0800 Message-Id: <20180327091043.30220-10-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180327091043.30220-1-xiaoguangrong@tencent.com> References: <20180327091043.30220-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::22c Subject: [Qemu-devel] [PATCH v2 09/10] migration: introduce save_normal_page() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong It directly sends the page to the stream neither checking zero nor using xbzrle or compression Reviewed-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Xiao Guangrong --- migration/ram.c | 50 ++++++++++++++++++++++++++++++-------------------- 1 file changed, 30 insertions(+), 20 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 771763985d..e71a9aee11 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1014,6 +1014,34 @@ static bool control_save_page(RAMState *rs, RAMBlock *block, ram_addr_t offset, return true; } +/* + * directly send the page to the stream + * + * Returns the number of pages written. + * + * @rs: current RAM state + * @block: block that contains the page we want to send + * @offset: offset inside the block for the page + * @buf: the page to be sent + * @async: send to page asyncly + */ +static int save_normal_page(RAMState *rs, RAMBlock *block, ram_addr_t offset, + uint8_t *buf, bool async) +{ + ram_counters.transferred += save_page_header(rs, rs->f, block, + offset | RAM_SAVE_FLAG_PAGE); + if (async) { + qemu_put_buffer_async(rs->f, buf, TARGET_PAGE_SIZE, + migrate_release_ram() & + migration_in_postcopy()); + } else { + qemu_put_buffer(rs->f, buf, TARGET_PAGE_SIZE); + } + ram_counters.transferred += TARGET_PAGE_SIZE; + ram_counters.normal++; + return 1; +} + /** * ram_save_page: send the given page to the stream * @@ -1054,18 +1082,7 @@ static int ram_save_page(RAMState *rs, PageSearchStatus *pss, bool last_stage) /* XBZRLE overflow or normal page */ if (pages == -1) { - ram_counters.transferred += - save_page_header(rs, rs->f, block, offset | RAM_SAVE_FLAG_PAGE); - if (send_async) { - qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE, - migrate_release_ram() & - migration_in_postcopy()); - } else { - qemu_put_buffer(rs->f, p, TARGET_PAGE_SIZE); - } - ram_counters.transferred += TARGET_PAGE_SIZE; - pages = 1; - ram_counters.normal++; + pages = save_normal_page(rs, block, offset, p, send_async); } XBZRLE_cache_unlock(); @@ -1196,14 +1213,7 @@ static int ram_save_compressed_page(RAMState *rs, PageSearchStatus *pss, * we post it as normal page as compression will take much * CPU resource. */ - ram_counters.transferred += save_page_header(rs, rs->f, block, - offset | RAM_SAVE_FLAG_PAGE); - qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE, - migrate_release_ram() & - migration_in_postcopy()); - ram_counters.transferred += TARGET_PAGE_SIZE; - ram_counters.normal++; - pages = 1; + pages = save_normal_page(rs, block, offset, p, true); } else { pages = compress_page_with_multi_thread(rs, block, offset); } From patchwork Tue Mar 27 09:10:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 892043 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="vbFU56fc"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40B2Rn41Srz9s0b for ; Wed, 28 Mar 2018 20:20:53 +1100 (AEDT) Received: from localhost ([::1]:38198 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f17GE-0000Nv-UM for incoming@patchwork.ozlabs.org; Wed, 28 Mar 2018 05:20:50 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56781) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f176f-00016a-Ch for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:11:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f176e-0005Dv-CY for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:57 -0400 Received: from mail-pl0-x244.google.com ([2607:f8b0:400e:c01::244]:42123) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1f176e-0005Dc-5z for qemu-devel@nongnu.org; Wed, 28 Mar 2018 05:10:56 -0400 Received: by mail-pl0-x244.google.com with SMTP id g20-v6so1197323plo.9 for ; Wed, 28 Mar 2018 02:10:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=UxMSDyw24gAUGieRCnuofhQ6Hgyimbt8gF/MFUmaIiU=; b=vbFU56fc5y6zDso1MCfx8x4XcgE5I6UnnNm2fNYN4aUHfqT44AXvpSPZyA9Wu5XB4V BDwtbRKNWRxEoXFCUrIPUbqlC30VzrpzAvBU2D+nfuVXYO5KRmRyo53vEDq9/J+PFVWO ftUmXktzifgc/z3Z+UymCsDnXTtV80zvBCir3MNoON5DSc1KTuaELLXWDw/M5cr4DpM6 8h4UQEuvdcmb5F3oRlTpUN2HA710pLgDfQeYpvP5b8w4po+xCg07Vix2bYbClbixHBF8 +93gRexbT7trIJGShAlh64EtaD30qvwUTbOyMDkKNBXG8tQCaS8gs/68U0WswKJgwSEb Gj6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=UxMSDyw24gAUGieRCnuofhQ6Hgyimbt8gF/MFUmaIiU=; b=sfbcLDLGkrZnkCruzHIbqCxit6S5fTWL/n4KMaGC29t/1EF5n1lU3U+oqy6oRcjzyv IGm7MA3d7x5lk8PrPwypEglY3Gi5VEWTqZzqhVT+ZL1GVug+5Y7guhQoMW+7RJDMlCmY 0scZ3KUrA0kXar6O7jDFVJMExZSPqXxiDrFoOfXaD49UWPiDs87HgMI+B6rHWu1hp9bt PPNcm8r735gQ020cEvU947cN4SEVFFRi0c+QDA2QPjybbp0PszxAKJ2rz8Ega43bTzsL I7VYvJhaRYNPISSgZjzrW6rP2Di8giN3P8e58H0r0HIhE1WzxReZGnl9n/w5aI+XEB5a CkNg== X-Gm-Message-State: AElRT7GtBznQ/Uep1NtEhERTudE2YujT43BMe/kAkfUGUThQ03rdnGNf 0mQo48gqEvMoOUD+xY2mLfU= X-Google-Smtp-Source: AIpwx48zmfdZXHSy+s/GOXWmMt6JbJ5TgJYNvwYPX2ogg94BEQdZTbEiSbMTCjU/8XcHIyxuiVUaQQ== X-Received: by 2002:a17:902:2c83:: with SMTP id n3-v6mr2978376plb.317.1522228255505; Wed, 28 Mar 2018 02:10:55 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.38]) by smtp.gmail.com with ESMTPSA id q9sm5590648pgs.89.2018.03.28.02.10.52 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 28 Mar 2018 02:10:54 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Tue, 27 Mar 2018 17:10:43 +0800 Message-Id: <20180327091043.30220-11-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180327091043.30220-1-xiaoguangrong@tencent.com> References: <20180327091043.30220-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::244 Subject: [Qemu-devel] [PATCH v2 10/10] migration: remove ram_save_compressed_page() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong Now, we can reuse the path in ram_save_page() to post the page out as normal, then the only thing remained in ram_save_compressed_page() is compression that we can move it out to the caller Reviewed-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Xiao Guangrong --- migration/ram.c | 45 ++++++++------------------------------------- 1 file changed, 8 insertions(+), 37 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index e71a9aee11..5187455a93 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1186,41 +1186,6 @@ static int compress_page_with_multi_thread(RAMState *rs, RAMBlock *block, return pages; } -/** - * ram_save_compressed_page: compress the given page and send it to the stream - * - * Returns the number of pages written. - * - * @rs: current RAM state - * @block: block that contains the page we want to send - * @offset: offset inside the block for the page - * @last_stage: if we are at the completion stage - */ -static int ram_save_compressed_page(RAMState *rs, PageSearchStatus *pss, - bool last_stage) -{ - int pages = -1; - uint8_t *p; - RAMBlock *block = pss->block; - ram_addr_t offset = pss->page << TARGET_PAGE_BITS; - - p = block->host + offset; - - if (block != rs->last_sent_block) { - /* - * Make sure the first page is sent out before other pages. - * - * we post it as normal page as compression will take much - * CPU resource. - */ - pages = save_normal_page(rs, block, offset, p, true); - } else { - pages = compress_page_with_multi_thread(rs, block, offset); - } - - return pages; -} - /** * find_dirty_block: find the next dirty page and update any state * associated with the search process. @@ -1520,8 +1485,14 @@ static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss, return res; } - if (save_page_use_compression(rs)) { - return ram_save_compressed_page(rs, pss, last_stage); + /* + * Make sure the first page is sent out before other pages. + * + * we post it as normal page as compression will take much + * CPU resource. + */ + if (block == rs->last_sent_block && save_page_use_compression(rs)) { + res = compress_page_with_multi_thread(rs, block, offset); } return ram_save_page(rs, pss, last_stage);