From patchwork Thu Apr 2 09:14:48 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Congyang X-Patchwork-Id: 457572 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 944FF140083 for ; Thu, 2 Apr 2015 20:12:07 +1100 (AEDT) Received: from localhost ([::1]:56607 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YdbAS-0008NO-1S for incoming@patchwork.ozlabs.org; Thu, 02 Apr 2015 05:12:04 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57366) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YdbA4-0007wc-6k for qemu-devel@nongnu.org; Thu, 02 Apr 2015 05:11:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YdbA0-00075G-V4 for qemu-devel@nongnu.org; Thu, 02 Apr 2015 05:11:40 -0400 Received: from [59.151.112.132] (port=32370 helo=heian.cn.fujitsu.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Ydb9z-00073l-H4 for qemu-devel@nongnu.org; Thu, 02 Apr 2015 05:11:36 -0400 X-IronPort-AV: E=Sophos;i="5.04,848,1406563200"; d="scan'208";a="90089073" Received: from unknown (HELO edo.cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 02 Apr 2015 17:07:42 +0800 Received: from G08CNEXCHPEKD02.g08.fujitsu.local (localhost.localdomain [127.0.0.1]) by edo.cn.fujitsu.com (8.14.3/8.13.1) with ESMTP id t329AMis016159; Thu, 2 Apr 2015 17:10:22 +0800 Received: from [10.167.226.52] (10.167.226.52) by G08CNEXCHPEKD02.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server id 14.3.181.6; Thu, 2 Apr 2015 17:11:27 +0800 Message-ID: <551D0888.3000705@cn.fujitsu.com> Date: Thu, 2 Apr 2015 17:14:48 +0800 From: Wen Congyang User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: , Kevin Wolf , Stefan Hajnoczi , Michael Tsirkin References: <55128084.2040304@huawei.com> <87a8z12yot.fsf@neno.neno> <5513793B.6020909@cn.fujitsu.com> <874mp8xd9k.fsf@neno.neno> In-Reply-To: <874mp8xd9k.fsf@neno.neno> X-Originating-IP: [10.167.226.52] X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 59.151.112.132 Cc: hangaohuai@huawei.com, zhanghailiang , Li Zhijian , "Dr. David Alan Gilbert \(git\)" , qemu-devel@nongnu.org, "Gonglei \(Arei\)" , Amit Shah , peter.huangpeng@huawei.com, david@gibson.dropbear.id.au Subject: Re: [Qemu-devel] [Migration Bug? ] Occasionally, the content of VM's memory is inconsistent between Source and Destination of migration X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org On 03/26/2015 06:29 PM, Juan Quintela wrote: > Wen Congyang wrote: >> On 03/25/2015 05:50 PM, Juan Quintela wrote: >>> zhanghailiang wrote: >>>> Hi all, >>>> >>>> We found that, sometimes, the content of VM's memory is >>>> inconsistent between Source side and Destination side >>>> when we check it just after finishing migration but before VM continue to Run. >>>> >>>> We use a patch like bellow to find this issue, you can find it from affix, >>>> and Steps to reprduce: >>>> >>>> (1) Compile QEMU: >>>> ./configure --target-list=x86_64-softmmu --extra-ldflags="-lssl" && make >>>> >>>> (2) Command and output: >>>> SRC: # x86_64-softmmu/qemu-system-x86_64 -enable-kvm -cpu >>>> qemu64,-kvmclock -netdev tap,id=hn0-device >>>> virtio-net-pci,id=net-pci0,netdev=hn0 -boot c -drive >>>> file=/mnt/sdb/pure_IMG/sles/sles11_sp3.img,if=none,id=drive-virtio-disk0,cache=unsafe >>>> -device >>>> virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0 >>>> -vnc :7 -m 2048 -smp 2 -device piix3-usb-uhci -device usb-tablet >>>> -monitor stdio >>> >>> Could you try to reproduce: >>> - without vhost >>> - without virtio-net >>> - cache=unsafe is going to give you trouble, but trouble should only >>> happen after migration of pages have finished. >> >> If I use ide disk, it doesn't happen. >> Even if I use virtio-net with vhost=on, it still doesn't happen. I guess >> it is because I migrate the guest when it is booting. The virtio net >> device is not used in this case. > > Kevin, Stefan, Michael, any great idea? The following patch can fix this problem(vhost=off): From ebc024702dd3147e0cbdfd173c599103dc87796c Mon Sep 17 00:00:00 2001 From: Wen Congyang Date: Thu, 2 Apr 2015 16:28:17 +0800 Subject: [PATCH] fix qiov size Signed-off-by: Wen Congyang --- hw/block/virtio-blk.c | 15 +++++++++++++++ include/hw/virtio/virtio-blk.h | 1 + 2 files changed, 16 insertions(+) diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index 000c38d..13967bc 100644 --- a/hw/block/virtio-blk.c +++ b/hw/block/virtio-blk.c @@ -33,6 +33,7 @@ VirtIOBlockReq *virtio_blk_alloc_request(VirtIOBlock *s) VirtIOBlockReq *req = g_slice_new(VirtIOBlockReq); req->dev = s; req->qiov.size = 0; + req->size = 0; req->next = NULL; req->mr_next = NULL; return req; @@ -97,12 +98,20 @@ static void virtio_blk_rw_complete(void *opaque, int ret) * external iovec. It was allocated in submit_merged_requests * to be able to merge requests. */ qemu_iovec_destroy(&req->qiov); + + /* Restore qiov->size here */ + req->qiov.size = req->size; } if (ret) { int p = virtio_ldl_p(VIRTIO_DEVICE(req->dev), &req->out.type); bool is_read = !(p & VIRTIO_BLK_T_OUT); if (virtio_blk_handle_rw_error(req, -ret, is_read)) { + /* + * FIXME: + * The memory may be dirtied on read failure, it will + * break live migration. + */ continue; } } @@ -323,6 +332,12 @@ static inline void submit_requests(BlockBackend *blk, MultiReqBuffer *mrb, struct iovec *tmp_iov = qiov->iov; int tmp_niov = qiov->niov; + /* + * Save old qiov->size, which will used in + * virtio_blk_complete_request() + */ + mrb->reqs[start]->size = qiov->size; + /* mrb->reqs[start]->qiov was initialized from external so we can't * modifiy it here. We need to initialize it locally and then add the * external iovecs. */ diff --git a/include/hw/virtio/virtio-blk.h b/include/hw/virtio/virtio-blk.h index b3ffcd9..7d47310 100644 --- a/include/hw/virtio/virtio-blk.h +++ b/include/hw/virtio/virtio-blk.h @@ -67,6 +67,7 @@ typedef struct VirtIOBlockReq { struct virtio_blk_inhdr *in; struct virtio_blk_outhdr out; QEMUIOVector qiov; + size_t size; struct VirtIOBlockReq *next; struct VirtIOBlockReq *mr_next; BlockAcctCookie acct;