From patchwork Thu Dec 26 16:19:52 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Lieven X-Patchwork-Id: 305286 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 8B8B02C009F for ; Fri, 27 Dec 2013 03:19:57 +1100 (EST) Received: from localhost ([::1]:46148 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VwDf7-0004hi-R2 for incoming@patchwork.ozlabs.org; Thu, 26 Dec 2013 11:19:53 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37245) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VwDen-0004hT-6I for qemu-devel@nongnu.org; Thu, 26 Dec 2013 11:19:38 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VwDeh-0006DC-JK for qemu-devel@nongnu.org; Thu, 26 Dec 2013 11:19:33 -0500 Received: from mx.ipv6.kamp.de ([2a02:248:0:51::16]:54665 helo=mx01.kamp.de) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VwDeh-0006D4-8v for qemu-devel@nongnu.org; Thu, 26 Dec 2013 11:19:27 -0500 Received: (qmail 9852 invoked by uid 89); 26 Dec 2013 16:19:25 -0000 Received: from [82.141.1.145] by client-16-kamp (envelope-from , uid 89) with qmail-scanner-2010/03/19-MF (clamdscan: 0.98/18288. hbedv: 8.2.12.166/7.11.122.50. spamassassin: 3.3.1. Clear:RC:1(82.141.1.145):SA:0(-1.2/5.0):. Processed in 1.57473 secs); 26 Dec 2013 16:19:25 -0000 Received: from ns.kamp-intra.net (HELO dns.kamp-intra.net) ([82.141.1.145]) by mx01.kamp.de with SMTP; 26 Dec 2013 16:19:23 -0000 X-GL_Whitelist: yes Received: from lieven-pc.kamp-intra.net (lieven-pc.kamp-intra.net [172.21.12.60]) by dns.kamp-intra.net (Postfix) with ESMTP id BC54C20688; Thu, 26 Dec 2013 17:18:33 +0100 (CET) Received: by lieven-pc.kamp-intra.net (Postfix, from userid 1000) id 0FC815FCC2; Thu, 26 Dec 2013 17:19:53 +0100 (CET) From: Peter Lieven To: qemu-devel@nongnu.org Date: Thu, 26 Dec 2013 17:19:52 +0100 Message-Id: <1388074792-29946-1-git-send-email-pl@kamp.de> X-Mailer: git-send-email 1.7.9.5 X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2a02:248:0:51::16 Cc: kwolf@redhat.com, famz@redhat.com, stefanha@redhat.com, Peter Lieven , ronniesahlberg@gmail.com, pbonzini@redhat.com Subject: [Qemu-devel] [RFC PATCH] qcow2: add a readahead cache for qcow2_decompress_cluster X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org while evaluatiing compressed qcow2 images as a good basis for virtual machine templates I found out that there are a lot of partly redundant (compressed clusters have common physical sectors) and relatively short reads. This doesn't hurt if the image resides on a local filesystem where we can benefit from the local page cache, but it adds a lot of penalty when accessing remote images on NFS or similar exports. This patch effectevily implements a readahead of 2 * cluster_size which is 2 * 64kB per default resulting in 128kB readahead. This is the common setting for Linux for instance. For example this leads to the following times when converting a compressed qcow2 image to a local tmpfs partition. Old: time ./qemu-img convert nfs://10.0.0.1/export/VC-Ubuntu-LTS-12.04.2-64bit.qcow2 /tmp/test.raw real 0m24.681s user 0m8.597s sys 0m4.084s New: time ./qemu-img convert nfs://10.0.0.1/export/VC-Ubuntu-LTS-12.04.2-64bit.qcow2 /tmp/test.raw real 0m16.121s user 0m7.932s sys 0m2.244s Signed-off-by: Peter Lieven --- block/qcow2-cluster.c | 27 +++++++++++++++++++++++++-- block/qcow2.h | 1 + 2 files changed, 26 insertions(+), 2 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 11f9c50..367f089 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -1321,7 +1321,7 @@ static int decompress_buffer(uint8_t *out_buf, int out_buf_size, int qcow2_decompress_cluster(BlockDriverState *bs, uint64_t cluster_offset) { BDRVQcowState *s = bs->opaque; - int ret, csize, nb_csectors, sector_offset; + int ret, csize, nb_csectors, sector_offset, max_read; uint64_t coffset; coffset = cluster_offset & s->cluster_offset_mask; @@ -1329,9 +1329,32 @@ int qcow2_decompress_cluster(BlockDriverState *bs, uint64_t cluster_offset) nb_csectors = ((cluster_offset >> s->csize_shift) & s->csize_mask) + 1; sector_offset = coffset & 511; csize = nb_csectors * 512 - sector_offset; + max_read = MIN((bs->file->total_sectors - (coffset >> 9)), 2 * s->cluster_sectors); BLKDBG_EVENT(bs->file, BLKDBG_READ_COMPRESSED); - ret = bdrv_read(bs->file, coffset >> 9, s->cluster_data, nb_csectors); + if (s->cluster_cache_offset != -1 && coffset > s->cluster_cache_offset && + (coffset >> 9) < (s->cluster_cache_offset >> 9) + s->cluster_data_sectors) { + int cached_sectors = s->cluster_data_sectors - ((coffset >> 9) - + (s->cluster_cache_offset >> 9)); + memmove(s->cluster_data, + s->cluster_data + (s->cluster_data_sectors - cached_sectors) * 512, + cached_sectors * 512); + s->cluster_data_sectors = cached_sectors; + if (nb_csectors > cached_sectors) { + /* some sectors are missing read them and fill up to max_read sectors */ + ret = bdrv_read(bs->file, (coffset >> 9) + cached_sectors, + s->cluster_data + cached_sectors * 512, + max_read); + s->cluster_data_sectors = cached_sectors + max_read; + } else { + /* all relevant sectors are in the cache */ + ret = 0; + } + } else { + ret = bdrv_read(bs->file, coffset >> 9, s->cluster_data, max_read); + s->cluster_data_sectors = max_read; + } if (ret < 0) { + s->cluster_data_sectors = 0; return ret; } if (decompress_buffer(s->cluster_cache, s->cluster_size, diff --git a/block/qcow2.h b/block/qcow2.h index 922e190..5edad26 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -185,6 +185,7 @@ typedef struct BDRVQcowState { uint8_t *cluster_cache; uint8_t *cluster_data; + int cluster_data_sectors; uint64_t cluster_cache_offset; QLIST_HEAD(QCowClusterAlloc, QCowL2Meta) cluster_allocs;