From patchwork Wed Nov 1 15:44:02 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Nefedov X-Patchwork-Id: 833025 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="fGOE/IMa"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3yRt0z393Dz9sQl for ; Thu, 2 Nov 2017 02:48:35 +1100 (AEDT) Received: from localhost ([::1]:56387 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e9vFp-0002No-AL for incoming@patchwork.ozlabs.org; Wed, 01 Nov 2017 11:48:33 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60654) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e9vCG-0008Cu-Mk for qemu-devel@nongnu.org; Wed, 01 Nov 2017 11:44:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e9vCC-0006hX-0M for qemu-devel@nongnu.org; Wed, 01 Nov 2017 11:44:52 -0400 Received: from mail-eopbgr40111.outbound.protection.outlook.com ([40.107.4.111]:9363 helo=EUR03-DB5-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e9vCB-0006gT-IO; Wed, 01 Nov 2017 11:44:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=yXqct5nEsunXWDQ8jR7IXIAhKj/Rp5xFfRRo4P/7faY=; b=fGOE/IMaXg34OgtEqD59aFkACmBdEm2YPFIxwWLLTlSSz5zaWNNJGLcoh+Am/1q1wuwDdAwyycfyPZZXmFW07RYu0XJW0Tb5iZSqGrRKeXMWZhFzp/AZ0ynkEcewzr6ysEH4g7eEQipUFEMG7BkCA68NZF7YfQvropmFU8mcZxs= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=anton.nefedov@virtuozzo.com; Received: from xantnef-ws.sw.ru (195.214.232.6) by AM5PR0801MB1987.eurprd08.prod.outlook.com (2603:10a6:203:4b::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.178.6; Wed, 1 Nov 2017 15:44:44 +0000 From: Anton Nefedov To: qemu-devel@nongnu.org Date: Wed, 1 Nov 2017 18:44:02 +0300 Message-Id: <1509551048-129830-10-git-send-email-anton.nefedov@virtuozzo.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1509551048-129830-1-git-send-email-anton.nefedov@virtuozzo.com> References: <1509551048-129830-1-git-send-email-anton.nefedov@virtuozzo.com> MIME-Version: 1.0 X-Originating-IP: [195.214.232.6] X-ClientProxiedBy: VI1PR0501CA0017.eurprd05.prod.outlook.com (2603:10a6:800:92::27) To AM5PR0801MB1987.eurprd08.prod.outlook.com (2603:10a6:203:4b::14) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: cc6bdc65-8569-47e8-bdea-08d5213f79f2 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(4534020)(4602075)(2017052603199); SRVR:AM5PR0801MB1987; X-Microsoft-Exchange-Diagnostics: 1; AM5PR0801MB1987; 3:aBYJatlpYlDLKuXe2ABcsIVYEcEUfB62oy+ehY79HJcgjA6cSXM68Mx1pOeQ0/IZQPBqKsrAB7hvO2dLEAJMHmOlr9wfQq3DPGjkHiNumv62KFberCgkI2Uxys52QERjJGa+iAyHhXgWPrkochkFYfaaPNbdakQdosVZOQhpiz/VxRMiq5xKOKPEU+gU7ZX6sU30bT2sNmwNVeAWeqSUgydRjwIO1AB/GPTzaLhn/MrZATbMY8wMgL0jbMuLE9F/; 25:3MWrsUDM74U4gd5RDw4Ti13sreURPcHBzUrd5v/oNXtXjAo0ezuHBEbpKB5jZf6qYB33rcxSVobeeFAeBtcMF/6N6X055Uc8EPA9/TOKBHS0waNl00oHPTSaQEvuFQiHexRLQhEanFr7XHVhSMmpt7XXcHcVq9wNe1V9VXUOJb7G+93I25biUXfT2C09IARtUjJYbYUHoAWG7E8W+Vbl8xJFqWx6RrNjNxUpOYvhbmY+7nod76cCfX/FI8jLCS/xrDSFD2vd+b+sY9BdERK0mGgsZ9DcIJFqXFzmNo2RBTjbVkq6/VUrUbcI0q3Nj9mQYD2la1xYfVAd/xn8XMMbFw==; 31:yifgJf78jZI84CSgcVSz6IkxGNB274Vlh6r34/A6FxZxu8odizOllERnSPUblHzxFo7XFwl5yRGbed3wGgNDoaTA6CKnkrv3x7iIp8BDZyd+VryjqFmNmlXwAm9CP3F4ukvjcAa3qikevjiApt6UnzppRg70ED6tgdOaTLcm45NvVo4V2ZEGBvjTIOzItWgPVH3qElbU+gBvmjbc2gczvwqSqVfF+Xr8tG9Uzx38Gk8= X-MS-TrafficTypeDiagnostic: AM5PR0801MB1987: X-Microsoft-Exchange-Diagnostics: 1; AM5PR0801MB1987; 20:EFcvmE8Mfb7fWMF9vKCMoRxs9ZnyHL1C949AgJdqZp7YXRPGAPdTmjANlfJCnHU8/DmOp1tTy4exeGd7CySeusjjtbm5gYtXxgXg8px5rLx6hkd5IlR364J494CwP8VPC84QOYw9c3Bi3MNSnHjSYqj7qYcF0yl9PqP/Pm502iG+Lw/81yjFrcWJn7B13rn4+/hBwMC9QTAkYY0+Jsx8fZZIdMWLPxOEHDeZcX2YneHzSRj+2Z/Hf1fsHb9xULPY3KiQYnDhOn3VLZbR2r4gjehsyIB4rD4HQrYwTs3eHpE1ZgNI0U8vZ3Lyg8yKQ/7a4S0E/ufhYhFwhA8aQFIOHXvhqQw7OpnRn/suQGrEvs8kJVTcT5pK2T1FZsSZcrbr00wwLSJO0yX18MPFNZIN8WGRcDg3MiR0w8XR+2Af0hI=; 4:Z/P3XpjOPsa2b5p2nKyQzgdQ4hMfl4s7DrDoWv/hXoYPYlw7qrGp+P6v/ODb9YyL62n1y+tsI3qCO9M6p/niXjKoovxgyH4uAEFodhEoU8/B7Nasp7luq2L+m0Rlu0ldDDMZjikW9wv2diiFN8f7nwfEoKkpDDi1eiTvrIbzvs+022OpS4Ua0N+3OXhJI9hIdQ5KCPzAoYfpLY09Zr/bKjoWWimXp/c2s/l+OJcFNJhJimeZUn9BiH1M4ZZ/e0+XNg5b7pRo8QHFRmLIjplNVQ== X-Exchange-Antispam-Report-Test: UriScan:; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(10201501046)(93006095)(93001095)(3231020)(100000703101)(100105400095)(3002001)(6041248)(20161123555025)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(20161123564025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:AM5PR0801MB1987; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:AM5PR0801MB1987; X-Forefront-PRVS: 0478C23FE0 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(6009001)(376002)(346002)(189002)(199003)(6666003)(16526018)(53936002)(107886003)(48376002)(86362001)(6916009)(2950100002)(3846002)(6116002)(101416001)(50986999)(76176999)(106356001)(69596002)(5660300001)(25786009)(105586002)(316002)(53416004)(2351001)(16586007)(2361001)(54906003)(33646002)(4326008)(5003940100001)(478600001)(66066001)(36756003)(50226002)(6512007)(47776003)(2906002)(8936002)(6506006)(97736004)(6486002)(68736007)(305945005)(50466002)(81166006)(81156014)(8676002)(189998001)(7736002); DIR:OUT; SFP:1102; SCL:1; SRVR:AM5PR0801MB1987; H:xantnef-ws.sw.ru; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: virtuozzo.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; AM5PR0801MB1987; 23:x2J7YKdyNmLuoSM0dKxQSSBN2XjhoSwrv7V5LcV?= cy3UdmPPC3cWYWtF8GL5Omitu87A4I9gfyWvUTrJ8nw87F543pf6CJ7vmpNF76fTZQQ+XD432J9tNk7dKZioE835cuytPHfzEWZBb31GYHltmjSKGgPGFn9KGP3R3Gk4EPlWJFjrwQocSgKkJ7QwzjpNJwADJX+/JywGUBX+j3eYtRWl7QyrdTzmtyn0oeHWiVS3rU3D5RZ65J+RqNFeFggocJH2JvTeO3vnGn0iEpeyM1m1w16anS/XeQhUlZAZHmcSs58RE4K89kvHlaN93ruAddHITgGRO0UDr02DITHXKiNAAq+RDffybCbT/onPjC7u1KEYL9JWYdxnAXXWQkqnXOWxOJLlFHyfvn1x/opqvzs5towglQ0IFzGdRENoAIRheup+gWG4qm0VubaxO5n+diod/GGEVLylxo4j0VeuxGFSXbKb9F+JNJcddE5zUEwuiq1snY180dpQmkMYQBfVrDyTEwn9PGIeY1ZYrFByWMh8+rdueZFzGQBBFyRNj3WHpP4SlThNHMGhiRTUbecI9MrqsYepYrw1DaLnrdXjgMfr5B6TcfUw5rdXQmkU/HriWkrTgNY1cc6Mr75WFJ6OLIk0Vlmb00gTeBWbl9YA5MZAwz+Edt+8+7KQCI/Vle6qXHOkW4mCjVBs8XeXXoAkIInu1QtO4tkdPDzuHwIy4rWgZHSINiHT4fuy1PaWftYpj9sQd7fkwfBc2e8VoQTTf44Us1GstFv5cD169E6vft4K1E1WB1k/R0MkKi/hdFtPnGAF7aMlrBChWjtm9iZ69yVDj4nSL6O985xaFrGeaWvt+lt4vkoPKGx/RJe24lGisR6FgAEHWHddUunTduYheRApWaJqrucLXhwAuGrsWNxJl76fMGYFzcbJJ4znKDvOoyVXUOvBVYbejwWPaVmzUO53+ZErllDFMhZvRP1tmXml8CoQcjKdBaShNDqF/tTn7nf0BV0fKwtMaMpiraWOMCqsCaWGodHQdWlj+kWefvopNkiWR/91hlEb0UHUg4FBEsCyrYE16ZwA6o5/WIMk4ji9cvCDdaL28Lqq7ILOEfB/nFpKahjsI9eTypgr6yLdz4qT3w78z5ah658aoytRhwxkTAx9XvED4snlOcSOVQlcz0sekZBRpQ2KHxxVyjGk= X-Microsoft-Exchange-Diagnostics: 1; AM5PR0801MB1987; 6:MOSanLAgkSVmHnFae4dPTRXZTWhRpPmZhdDyi3+axwTgC3qKOsUe+5J3TWQ5Ai/isB3Ilb8hvOW9qgtA07UslNYF922p01HUNcEzuWsZMj/PL5m55c1Mme0jharvTNxONVZBUAV0Kwoxq8nAqLQcVjd/mY0Xa4PpH0kO1CKl6xdsnDNg9b+uBEj6c2u8HyvDJIcJc5wcCd1Yt/e9P3hBnehDg3BiRaQ4MEP+tF6u734EviAH8LMZRzu0wZJ492zLHltksMHHiy7r4plQQgLedEisVclvPGKebR3DCsQLN+yt2ZdxLoa1MToC9EhWccadmwKPyNVziD84IPzNLSADsmNvEMQGrfchPQ2RWbI7ENw=; 5:6G2fPdrjbvr5HkySTRx0b90D3bEiaefgIGrBgMs+brEWegOVvoLUYlN62OrQ9kV4l4yCBX+NK3U8gHl4EqPtIgIz9GTr2uWDMfLsGWh+Bro+aNVVfgFochDVAaP/8vVSp/I6KMXNB0oHcGU1p7bRMRVVn950o9QEbAyvrdsAKQo=; 24:vmMPkRUSZQJBHSaxLMWqMhZVTjNQH89WTiiw9DdNVAyQWYm9v03BAL581yYlFmeU0U1B0/7j2eYMXjhaVLuKgM7LmSM1Jpc6zymT0eT0AcE=; 7:3LcfAeXcOU6xtB58ibLnK8jgQlg5GaU979nOJ2uUl3bFv6VTAss0IYgjARs6/ehLwF16ipSHYQXTSEF85vEchAIxu8DUngNtdFlNCpWIi2bv3KS1rRojrp7JS0zo2/mTEV7c90oBmHIuBUuGq7yG9B/55+yCzW7Squ38t4Gn7RHQOasRt5RK8I2ZzzhO7z0lHYAhq8PiGfW+TViAqQ2IVIcONm5xJdDxlzZwzhBMuznyi/wN0M2pcYucnjYZa7fx SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; AM5PR0801MB1987; 20:an8GhG5hlEBfRNP9B1Ndaid74qZHx3LnsRJ22VRrQxAPxkysy82wG5xjz8Ba0XZyLLozlBAByoGTAF5IS7VqEXg1OlwK4dxXIT9PwPB0B0fCrXd+5Gs8An7sIhKwac+/UinzBmFNHkPGx0lIFAfwxMB2W5heii5Aejh6y/MI/CY= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Nov 2017 15:44:44.6182 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cc6bdc65-8569-47e8-bdea-08d5213f79f2 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0801MB1987 X-detected-operating-system: by eggs.gnu.org: Windows 7 or 8 [fuzzy] X-Received-From: 40.107.4.111 Subject: [Qemu-devel] [PATCH v5 09/15] qcow2: preallocation at image expand X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Anton Nefedov , berto@igalia.com, den@virtuozzo.com, qemu-block@nongnu.org, mreitz@redhat.com, "Denis V. Lunev" Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: "Denis V. Lunev" This patch adds image preallocation at expand to provide better locality of QCOW2 image file and optimize this procedure for some distributed storage where this procedure is slow. Preallocation is not issued upon writing metadata clusters. Possible conflicts are resolved by the common block layer code since ALLOCATE requests are serialising. Signed-off-by: Denis V. Lunev Signed-off-by: Anton Nefedov --- block/qcow2.h | 3 +++ block/qcow2.c | 43 +++++++++++++++++++++++++++++++++++++++++++ qemu-options.hx | 4 ++++ 3 files changed, 50 insertions(+) diff --git a/block/qcow2.h b/block/qcow2.h index e312e48..9a2711e 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -102,6 +102,7 @@ #define QCOW2_OPT_L2_CACHE_SIZE "l2-cache-size" #define QCOW2_OPT_REFCOUNT_CACHE_SIZE "refcount-cache-size" #define QCOW2_OPT_CACHE_CLEAN_INTERVAL "cache-clean-interval" +#define QCOW2_OPT_PREALLOC_SIZE "prealloc-size" typedef struct QCowHeader { uint32_t magic; @@ -327,6 +328,8 @@ typedef struct BDRVQcow2State { * override) */ char *image_backing_file; char *image_backing_format; + + uint64_t prealloc_size; } BDRVQcow2State; typedef struct Qcow2COWRegion { diff --git a/block/qcow2.c b/block/qcow2.c index 827058d..3899524 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -674,6 +674,11 @@ static QemuOptsList qcow2_runtime_opts = { }, BLOCK_CRYPTO_OPT_DEF_KEY_SECRET("encrypt.", "ID of secret providing qcow2 AES key or LUKS passphrase"), + { + .name = QCOW2_OPT_PREALLOC_SIZE, + .type = QEMU_OPT_SIZE, + .help = "Preallocation amount at image expand", + }, { /* end of list */ } }, }; @@ -1016,6 +1021,15 @@ static int qcow2_update_options_prepare(BlockDriverState *bs, goto fail; } + s->prealloc_size = + ROUND_UP(qemu_opt_get_size_del(opts, QCOW2_OPT_PREALLOC_SIZE, 0), + s->cluster_size); + if (s->prealloc_size && + !(bs->file->bs->supported_zero_flags & BDRV_REQ_ALLOCATE)) + { + s->prealloc_size = 0; + } + ret = 0; fail: QDECREF(encryptopts); @@ -1924,6 +1938,31 @@ static bool is_zero_cow(BlockDriverState *bs, QCowL2Meta *m) return true; } +/* + * If the specified area is beyond EOF, allocates it + prealloc_size + * bytes ahead. + */ +static void coroutine_fn handle_prealloc(BlockDriverState *bs, + const QCowL2Meta *m) +{ + BDRVQcow2State *s = bs->opaque; + uint64_t start = m->alloc_offset; + uint64_t end = start + m->nb_clusters * s->cluster_size; + int64_t flen = bdrv_getlength(bs->file->bs); + + if (flen < 0) { + return; + } + + if (end > flen) { + /* try to alloc host space in one chunk for better locality */ + bdrv_co_pwrite_zeroes(bs->file, flen, + QEMU_ALIGN_UP(end + s->prealloc_size - flen, + s->cluster_size), + BDRV_REQ_ALLOCATE); + } +} + static void handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta) { BDRVQcow2State *s = bs->opaque; @@ -1932,6 +1971,10 @@ static void handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta) for (m = l2meta; m != NULL; m = m->next) { int ret; + if (s->prealloc_size) { + handle_prealloc(bs, m); + } + if (!m->cow_start.nb_bytes && !m->cow_end.nb_bytes) { continue; } diff --git a/qemu-options.hx b/qemu-options.hx index 3728e9b..a75a9fd 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -771,6 +771,10 @@ occasions where a cluster gets freed (on/off; default: off) Which overlap checks to perform for writes to the image (none/constant/cached/all; default: cached). For details or finer granularity control refer to the QAPI documentation of @code{blockdev-add}. + +@item prealloc-size +The number of bytes that will be preallocated ahead at qcow2 file expansion +(allocating a new cluster beyond the end of file). @end table Example 1: