From patchwork Tue May 14 12:10:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Nefedov X-Patchwork-Id: 1099454 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="NTlyKGmq"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 453Gnz5Y5sz9sNl for ; Tue, 14 May 2019 22:14:31 +1000 (AEST) Received: from localhost ([127.0.0.1]:47279 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQWKD-0005lI-Lg for incoming@patchwork.ozlabs.org; Tue, 14 May 2019 08:14:29 -0400 Received: from eggs.gnu.org ([209.51.188.92]:60919) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQWGp-0003NU-UX for qemu-devel@nongnu.org; Tue, 14 May 2019 08:11:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hQWGn-00022b-E5 for qemu-devel@nongnu.org; Tue, 14 May 2019 08:10:59 -0400 Received: from mail-ve1eur01on072b.outbound.protection.outlook.com ([2a01:111:f400:fe1f::72b]:39198 helo=EUR01-VE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hQWGg-0001pj-PI; Tue, 14 May 2019 08:10:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K/zUYUKsAM5aVkIisRlci6D62ji3clpG26Ilc5jZaBc=; b=NTlyKGmqLjxYIw28vS3mlBODY1Hryl/dWplgQVtRU9u7tl8N1aFHEtUIm7dUH++ZfuRIv//sdBAtuhI1JjJ7E0De4dH1kkaO/YnJD7S4n1EPbvtHgHCFCNAAMntD8wZZ2qm7UMCovQYuFi/cYE2hw8Bz1dIh99Te0bm7Z5KYVhc= Received: from VI1PR08MB4078.eurprd08.prod.outlook.com (20.178.127.92) by VI1PR08MB4480.eurprd08.prod.outlook.com (20.179.27.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1900.16; Tue, 14 May 2019 12:10:40 +0000 Received: from VI1PR08MB4078.eurprd08.prod.outlook.com ([fe80::8ce8:ffb4:fe9:4900]) by VI1PR08MB4078.eurprd08.prod.outlook.com ([fe80::8ce8:ffb4:fe9:4900%3]) with mapi id 15.20.1878.024; Tue, 14 May 2019 12:10:40 +0000 From: Anton Nefedov To: "qemu-block@nongnu.org" Thread-Topic: [PATCH v7 0/9] discard blockstats Thread-Index: AQHVCk4LHMttZCqeQUeaJK/Q2TMYQA== Date: Tue, 14 May 2019 12:10:40 +0000 Message-ID: <20190514121015.32190-1-anton.nefedov@virtuozzo.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: HE1PR0701CA0057.eurprd07.prod.outlook.com (2603:10a6:3:9e::25) To VI1PR08MB4078.eurprd08.prod.outlook.com (2603:10a6:803:e5::28) authentication-results: spf=none (sender IP is ) smtp.mailfrom=anton.nefedov@virtuozzo.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.17.1 x-originating-ip: [185.231.240.5] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 5806d194-01c3-4e10-4ca4-08d6d8652e3f x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600141)(711020)(4605104)(2017052603328)(7193020); SRVR:VI1PR08MB4480; x-ms-traffictypediagnostic: VI1PR08MB4480: x-ms-exchange-purlcount: 4 x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 0037FD6480 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(346002)(366004)(136003)(396003)(376002)(39850400004)(189003)(199004)(186003)(86362001)(6116002)(486006)(3846002)(66066001)(4326008)(256004)(52116002)(316002)(26005)(44832011)(36756003)(6512007)(966005)(6486002)(25786009)(6506007)(14444005)(478600001)(102836004)(6916009)(386003)(2351001)(54906003)(64756008)(66446008)(66556008)(5640700003)(7736002)(6306002)(14454004)(2616005)(2906002)(73956011)(53936002)(476003)(71190400001)(71200400001)(99286004)(107886003)(66946007)(66476007)(2501003)(305945005)(6436002)(8936002)(1076003)(81166006)(81156014)(8676002)(5660300002)(50226002)(68736007); DIR:OUT; SFP:1102; SCL:1; SRVR:VI1PR08MB4480; H:VI1PR08MB4078.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: virtuozzo.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: G3E/3srAszU1FI5GTjgSDk7DxD3y+mlp/1AEmrh7zaK+lAPvEPvCPQPFEXG/ASD56QnU1u68J8OZd9wyGLClDPu4IJc6hXyPrz2aBT2veIO2rs9pYGkiV+HWWZP27v+IpZlyA32TYPzNtkD2Ju8IB3KAId2WdlV7VqkY/fCGjrjSPqP08xPlJVa528YTmnEFfuHmrGGP7mSRGs5Wtwo+e84ktkxHfBaHW5bPtJRAdasAXJzMZf1wgegdR/FllVswRlw1OdG4djr87QLCnJ1qMvIzj68o3mNOg/nqiejGnf9wM0vbgVNVOoDWSkm89bwb7BI9a4cDuRjO9OsKPwx29pDl1CMD6LdWPOGQWxkpAORtR16aUiHTlS5tRhQJ6Ma6L952WOrEkn9Y/fwi4U/su0CmtWY2uh26fVD+xIH6f8E= MIME-Version: 1.0 X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5806d194-01c3-4e10-4ca4-08d6d8652e3f X-MS-Exchange-CrossTenant-originalarrivaltime: 14 May 2019 12:10:40.2143 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4480 X-detected-operating-system: by eggs.gnu.org: Windows 7 or 8 [fuzzy] X-Received-From: 2a01:111:f400:fe1f::72b Subject: [Qemu-devel] [PATCH v7 0/9] discard blockstats X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "kwolf@redhat.com" , Vladimir Sementsov-Ogievskiy , "berto@igalia.com" , Denis Lunev , "qemu-devel@nongnu.org" , "mreitz@redhat.com" , Anton Nefedov , "pbonzini@redhat.com" , "jsnow@redhat.com" Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" hi, yet another take for this patch series; please kindly consider these for 4.1 Just a few cosmetic comments were received for v6 so this is mostly a rebase+ping. new in v7: - general rebase - since clauses -> 4.1 - patch 8: not completely trivial rebase: raw_account_discard moved to common raw_do_pdiscard() - patch 9: comment wording fixed v6: http://lists.nongnu.org/archive/html/qemu-devel/2018-11/msg06633.html v5: http://lists.nongnu.org/archive/html/qemu-devel/2018-10/msg06828.html v4: http://lists.nongnu.org/archive/html/qemu-devel/2018-08/msg04308.html v3: http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg03688.html ---- qmp query-blockstats provides stats info for write/read/flush ops. Patches 1-7 implement the similar for discard (unmap) command for scsi and ide disks. Discard stat "unmap_ops / unmap_bytes" is supposed to account the ops that have completed without an error. However, discard operation is advisory. Specifically, - common block layer ignores ENOTSUP error code. That might be returned if the block driver does not support discard, or discard has been configured to be ignored. - format drivers such as qcow2 may ignore discard if they were configured to ignore that, or if the corresponding area is already marked unused (unallocated / zero clusters). And what is actually useful is the number of bytes actually discarded down on the host filesystem. To achieve that, driver-specific statistics has been added to blockstats (patch 9). With patch 8, file-posix driver accounts discard operations on its level too. query-blockstat result: (note the difference between blockdevice unmap and file discard stats. qcow2 sends fewer ops down to the file as the clusters are actually unallocated on qcow2 level) { "device": "drive-scsi0-0-0-0", "node-name": "#block159", "stats": { > "invalid_unmap_operations": 0, > "failed_unmap_operations": 0, "wr_highest_offset": 13411688448, "rd_total_time_ns": 2859566315, "rd_bytes": 103182336, "rd_merged": 0, "flush_operations": 19, "invalid_wr_operations": 0, "flush_total_time_ns": 23111608, "failed_rd_operations": 0, "failed_flush_operations": 0, "invalid_flush_operations": 0, "timed_stats": [ ], "wr_merged": 0, "wr_bytes": 1702912, > "unmap_bytes": 11954954240, > "unmap_operations": 865, "idle_time_ns": 2669508623, "account_invalid": true, > "unmap_total_time_ns": 19698002, "wr_operations": 143, "failed_wr_operations": 0, "rd_operations": 4816, "account_failed": true, > "unmap_merged": 0, "wr_total_time_ns": 1262686124, "invalid_rd_operations": 0 }, "parent": { > "driver-specific": { > "discard-nb-failed": 0, > "discard-bytes-ok": 720896, > "driver": "file", > "discard-nb-ok": 8 > }, "node-name": "#block009", "stats": { [..] } } }, { "device": "floppy0", Anton Nefedov (9): qapi: group BlockDeviceStats fields qapi: add unmap to BlockDeviceStats block: add empty account cookie type ide: account UNMAP (TRIM) operations scsi: store unmap offset and nb_sectors in request struct scsi: move unmap error checking to the complete callback scsi: account unmap operations file-posix: account discard operations qapi: query-blockstat: add driver specific file-posix stats qapi/block-core.json | 81 ++++++++++++++++++++++++++++++++------ include/block/accounting.h | 2 + include/block/block.h | 1 + include/block/block_int.h | 1 + block.c | 9 +++++ block/accounting.c | 6 +++ block/file-posix.c | 54 ++++++++++++++++++++++++- block/qapi.c | 11 ++++++ hw/ide/core.c | 12 ++++++ hw/scsi/scsi-disk.c | 32 +++++++++------ tests/qemu-iotests/227.out | 18 +++++++++ 11 files changed, 204 insertions(+), 23 deletions(-)