From patchwork Tue May 14 13:53:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Nefedov X-Patchwork-Id: 1099489 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="GdsnM6YI"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 453K0l6gt2z9sBb for ; Tue, 14 May 2019 23:53:59 +1000 (AEST) Received: from localhost ([127.0.0.1]:48538 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQXsT-0003UR-UF for incoming@patchwork.ozlabs.org; Tue, 14 May 2019 09:53:57 -0400 Received: from eggs.gnu.org ([209.51.188.92]:58020) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQXrk-0003PC-BR for qemu-devel@nongnu.org; Tue, 14 May 2019 09:53:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hQXrj-0000vK-5g for qemu-devel@nongnu.org; Tue, 14 May 2019 09:53:12 -0400 Received: from mail-db5eur03on0724.outbound.protection.outlook.com ([2a01:111:f400:fe0a::724]:21377 helo=EUR03-DB5-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hQXrg-0000oa-1A; Tue, 14 May 2019 09:53:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Eum1UMLP8GQRjog3i3FY0l4Nm/f3c97bMIJbis6dmqs=; b=GdsnM6YIlSxntXFXx3buNL2dBAvg9SqP/e/IpdqO9F1tDWyeAc7FUdsh/XcM0oz6dgtW+/0xJs7U1sTJoITRhclT69TqflssNWZTfNvOJEhJLdazyQYNJ+hoQAF616wvMWLQQeXk0I949D2Bf3wT/2mCB224psKnN6OcSXDxD70= Received: from VI1PR08MB4078.eurprd08.prod.outlook.com (20.178.127.92) by VI1PR08MB4223.eurprd08.prod.outlook.com (20.178.13.96) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1878.22; Tue, 14 May 2019 13:53:03 +0000 Received: from VI1PR08MB4078.eurprd08.prod.outlook.com ([fe80::8ce8:ffb4:fe9:4900]) by VI1PR08MB4078.eurprd08.prod.outlook.com ([fe80::8ce8:ffb4:fe9:4900%3]) with mapi id 15.20.1878.024; Tue, 14 May 2019 13:53:03 +0000 From: Anton Nefedov To: "qemu-block@nongnu.org" Thread-Topic: [PATCH v13 0/1] qcow2: cluster space preallocation Thread-Index: AQHVClxZLAmdZM7mYka0TPufP1BHSg== Date: Tue, 14 May 2019 13:53:03 +0000 Message-ID: <20190514135248.34146-1-anton.nefedov@virtuozzo.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: HE1PR0901CA0064.eurprd09.prod.outlook.com (2603:10a6:3:45::32) To VI1PR08MB4078.eurprd08.prod.outlook.com (2603:10a6:803:e5::28) authentication-results: spf=none (sender IP is ) smtp.mailfrom=anton.nefedov@virtuozzo.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.17.1 x-originating-ip: [185.231.240.5] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2ae49eda-fa80-40f5-ae7e-08d6d8737c2e x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600141)(711020)(4605104)(2017052603328)(7193020); SRVR:VI1PR08MB4223; x-ms-traffictypediagnostic: VI1PR08MB4223: x-ms-exchange-purlcount: 4 x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:6430; x-forefront-prvs: 0037FD6480 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(366004)(346002)(376002)(39840400004)(396003)(136003)(199004)(189003)(6116002)(64756008)(86362001)(66946007)(66476007)(66446008)(66556008)(2616005)(8676002)(476003)(6436002)(3846002)(8936002)(14444005)(1076003)(50226002)(73956011)(2351001)(6486002)(256004)(5640700003)(44832011)(2906002)(2501003)(6306002)(5660300002)(305945005)(966005)(7736002)(486006)(14454004)(478600001)(107886003)(71200400001)(71190400001)(66066001)(186003)(68736007)(102836004)(26005)(25786009)(6512007)(54906003)(6916009)(4326008)(53936002)(386003)(36756003)(81156014)(99286004)(6506007)(81166006)(52116002)(316002); DIR:OUT; SFP:1102; SCL:1; SRVR:VI1PR08MB4223; H:VI1PR08MB4078.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: virtuozzo.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: NXxdjOr+dLYEFnny5dj/0vXwqSM/tG9/fvdutxCcC7JU3k0Hi/cGBHTU7A4CLeHyZ+hyW3z/6ATxSTytpoHUBIZxOlknkCtSF7LlVnzowqsfVwmDVemaJyDjldHkqw0kovHyNnbQldgl/PCd+AlAvJ0TOgXLTIRjmmkrMUVuigYaNYKEMvgOt0J/gCK4GNr1V/FRq7uh972pufRJJUI4uVDI5EN8SgLGs0cDnDBxXF3z28cxhfQWIyM4NyItH308OEEFA491h+rMCxrDC+37XyVAiBS4WLaL+SQlJkLmG7rDZCEYQlF9szyjlvXuWOuuQg6Y8Zr5xvsUb+Wp+gP103nocf/D5lZV/Cd7BkH+VTFG3chkMmiU3t5xC9zQ6movMIapZFL9pUXHX+PD2cvOxDUIpb8l+UJcKSXUyZlNNqg= MIME-Version: 1.0 X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2ae49eda-fa80-40f5-ae7e-08d6d8737c2e X-MS-Exchange-CrossTenant-originalarrivaltime: 14 May 2019 13:53:03.7768 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4223 X-detected-operating-system: by eggs.gnu.org: Windows 7 or 8 [fuzzy] X-Received-From: 2a01:111:f400:fe0a::724 Subject: [Qemu-devel] [PATCH v13 0/1] qcow2: cluster space preallocation X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "kwolf@redhat.com" , Vladimir Sementsov-Ogievskiy , "berto@igalia.com" , Denis Lunev , "qemu-devel@nongnu.org" , "mreitz@redhat.com" , Anton Nefedov Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" hi, this used to be a large 10-patches series, but now thanks to Kevin's BDRV_REQ_NO_FALLBACK series (http://lists.nongnu.org/archive/html/qemu-devel/2019-03/msg06389.html) the core block layer functionality is already in place: the existing flag can be reused. A few accompanying patches were also dropped from the series as those can be processed separately. So, new in v13: - patch 1 (was patch 9) rebased to use s->data_file pointer - comments fixed/added - other patches dropped ;) v12: http://lists.nongnu.org/archive/html/qemu-devel/2019-01/msg02761.html v11: http://lists.nongnu.org/archive/html/qemu-devel/2018-12/msg04342.html v10: http://lists.nongnu.org/archive/html/qemu-devel/2018-12/msg00121.html ---- This pull request is to start to improve a few performance points of qcow2 format: 1. non cluster-aligned write requests (to unallocated clusters) explicitly pad data with zeroes if there is no backing data. Resulting increase in ops number and potential cluster fragmentation (on the host file) is already solved by: ee22a9d qcow2: Merge the writing of the COW regions with the guest data However, in case of zero COW regions, that can be avoided at all but the whole clusters are preallocated and zeroed in a single efficient write_zeroes() operation 2. moreover, efficient write_zeroes() operation can be used to preallocate space megabytes (*configurable number) ahead which gives noticeable improvement on some storage types (e.g. distributed storage) where the space allocation operation might be expensive) (Not included in this patchset since v6). 3. this will also allow to enable simultaneous writes to the same unallocated cluster after the space has been allocated & zeroed but before the first data is written and the cluster is linked to L2. (Not included in this patchset). Efficient write_zeroes usually implies that the blocks are not actually written to but only reserved and marked as zeroed by the storage. Existing bdrv_write_zeroes() falls back to writing zero buffers if write_zeroes is not supported by the driver, so BDRV_REQ_NO_FALLBACK is used. simple perf test: qemu-img create -f qcow2 test.img 4G && \ qemu-img bench -c $((1024*1024)) -f qcow2 -n -s 4k -t none -w test.img test results (seconds): +-----------+-------+------+-------+------+------+ | file | before | after | gain | +-----------+-------+------+-------+------+------+ | ssd | 61.153 | 36.313 | 41% | | hdd | 112.676 | 122.056 | -8% | +-----------+--------------+--------------+------+ Anton Nefedov (1): qcow2: skip writing zero buffers to empty COW areas qapi/block-core.json | 4 +- block/qcow2.h | 6 +++ block/qcow2-cluster.c | 2 +- block/qcow2.c | 93 +++++++++++++++++++++++++++++++++++++- block/trace-events | 1 + tests/qemu-iotests/060 | 7 ++- tests/qemu-iotests/060.out | 5 +- 7 files changed, 112 insertions(+), 6 deletions(-)