From patchwork Tue Aug 21 09:46:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Nefedov X-Patchwork-Id: 960176 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="cHc1eVRj"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41vm7H1F4nz9s4Z for ; Tue, 21 Aug 2018 19:47:39 +1000 (AEST) Received: from localhost ([::1]:51965 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fs3GC-0003ro-Of for incoming@patchwork.ozlabs.org; Tue, 21 Aug 2018 05:47:36 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48644) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fs3FR-0003qo-1X for qemu-devel@nongnu.org; Tue, 21 Aug 2018 05:46:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fs3FL-0007rF-Sj for qemu-devel@nongnu.org; Tue, 21 Aug 2018 05:46:48 -0400 Received: from mail-eopbgr40122.outbound.protection.outlook.com ([40.107.4.122]:7520 helo=EUR03-DB5-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fs3FI-0007oN-VP; Tue, 21 Aug 2018 05:46:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HyJOKP5lt/OZcz1boy1NrAftw4omHRJHwmWqJ+rkbKg=; b=cHc1eVRjsQGhbYcQclD/5YfIlv6FjPpX5a7qxypmQEaTSIk1P2eqNF6CVsuqBtizpk9xVxaVE5O6f/sn8wwS5HFdMQrDxGuhH1wCQ/b3ouHxEz1k/MIArmtuE4sEfT48ZpbhKoFMz/wV2AG66IylM5NHm6rW35XwMNam4NuQ9yk= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=anton.nefedov@virtuozzo.com; Received: from xantnef-ws.sw.ru (185.231.240.5) by AM0PR08MB3347.eurprd08.prod.outlook.com (2603:10a6:208:5f::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1059.22; Tue, 21 Aug 2018 09:46:36 +0000 From: Anton Nefedov To: qemu-devel@nongnu.org Date: Tue, 21 Aug 2018 12:46:11 +0300 Message-Id: <1534844779-118784-1-git-send-email-anton.nefedov@virtuozzo.com> X-Mailer: git-send-email 2.7.4 MIME-Version: 1.0 X-Originating-IP: [185.231.240.5] X-ClientProxiedBy: HE1PR08CA0076.eurprd08.prod.outlook.com (2603:10a6:7:2a::47) To AM0PR08MB3347.eurprd08.prod.outlook.com (2603:10a6:208:5f::20) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4cd0e230-3d70-46af-5fd8-08d6074afd47 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989137)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(2017052603328)(7153060)(7193020); SRVR:AM0PR08MB3347; X-Microsoft-Exchange-Diagnostics: 1; AM0PR08MB3347; 3:3zqjVDBmb5SbRSPHhDL6KCt/YUROtcnVG9MehQ/FXnra1snQU/RzT2YhPZm/SX18CkohgixIgMjrkH6nmAJMueuvDogpg6uAlvl4q08nRxw0ANl7ES+M49bfz37qIeGa89PQXYOJggIX8SlBIvtN4NMqPzctKQzVKaDs2aqaZ9zt0b/oTmXWPmXS9JITSbGXPpM3Qchbe4NoHY+OA0g8wrSXCTh8/rzMHplsRCnk8y1CGv8x+2u39pHkG85ILjmv; 25:K5ZHapBiUp6PzhgCljqYujKdeG+eA0So9s6sCW6PFHCadgPCjSENO8gEbEwd6xqLdm5mdxqwDicESOu4PBABTUlJkzSyWmBa4U1YihGfDRpWfdGT2RVj/PMfqhv9hGAr2mi/nyePGoFQ3rqUILUTyLDzf31D4Ttx4VZeYfDv4TLrhnAdi7B69hZoNoOq7LA7eoO4uHgJd0YuXU0NPMLlcb4jQX56/VEEZy24qRSgYvpT2Z8DuNml8hgUpY5wl+5+bj+hNl4XAjU5krdVrHE9JgXoLSYd75sCO97CYUFbS52Xa2Ye140o+w4ovDm3H2K6ivHnMpvQLnnwrhKARrf0Lg==; 31:b/u2PqosiSKQVTs0n00od3G9HT+eWlVYRhH67J+geUJ8d/KjkuVdqPzUWXRuNlopmnGmX9a+Fsr/dfGYdlgCo2C3sG+r/Nyw+ttKJxMW1GKAhn++UFgjAv42KTouwZNkue5m1+2hi54aW+J6Kw/oKs8szafifupxi+b6eWAyc+tOyyGS1s1I0oRRHvRKD2SIRRx/ljnvLcfZgUiXJWvFYVH5kVQQL5E2plH7Pk49kSA= X-MS-TrafficTypeDiagnostic: AM0PR08MB3347: X-Microsoft-Exchange-Diagnostics: 1; AM0PR08MB3347; 20:wlxcqQ23+6Ml9Ery/J1I6Y4IbQL+gP/MVffb4gfOLqDowVJ/9e05tCipshhbGK6jR9YL0kIv2QryK2Rp65+fpABZw9KxN+S86e8ot0bRv7lOAlwcCJM0HPpMlv2dZscv7bFv9UJK8bCOta8yEfcQHlCxmo+0jYEKhqxQ36SBmjj3Quqk1YdzYbrRbANHavpzNV77bs37ekvoW9YZ/36entjTY3nni1hCUYWBIt6CawVwfeas6NBka2z9kfF/p7pu1Ldz3XtBqPzR28V/e7DkY3q7dBzp9M6FJb4d6PwN1lM4e2SLstv/Dqv0Kkus3A71lglbKv+nE8VLCnH1p/pMHwTieu3kMOINWE1ZzrANZWxjrUaN+d1XiH+EXL+UwrwpEo+hXCFKxwXkCIpMtrCr3fiQxQx8yKEJGIUv6d7ESaBnPEwbXVm8N0HCwjri/xpYzRpQ0MhR6JLHqw7h7i3KF3bJvlLPljicTpb6O5GBGo5/EWmSQr8kMKswLK81DeY1; 4:so+kjlIUE4EqAPzvz2QTQudc3KhPf7HJedYxTsLV4TYeLRa+RYYy6B0UuXvAozI4S6puh3A8mh2nbCG7DsUETlyTeIHH95bV5lXXytUy4NrWagbN+6nJVfhPAi96paQIz/aB4L3Mtaxitc1L7kLFBSApwDg58SMqZzSmApSWdASEsGQv+PsFnhVhSe+sg8OHM96XIRL6c878tSpeNjhn1bLVs/hqimbY8K9TOG+a9sgDdXR1h5o2cG7lwmwN8xfIzlhnfwt6QLDya16ZoOhNDA== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(8121501046)(5005006)(93006095)(93001095)(3002001)(10201501046)(3231311)(944501410)(52105095)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123558120)(20161123560045)(20161123564045)(201708071742011)(7699016); SRVR:AM0PR08MB3347; BCL:0; PCL:0; RULEID:; SRVR:AM0PR08MB3347; X-Forefront-PRVS: 0771670921 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(39850400004)(136003)(396003)(376002)(366004)(346002)(199004)(189003)(36756003)(305945005)(53416004)(105586002)(51416003)(48376002)(50226002)(8936002)(6116002)(7736002)(956004)(3846002)(69596002)(50466002)(68736007)(186003)(106356001)(2616005)(16526019)(16586007)(47776003)(486006)(476003)(52116002)(66066001)(86362001)(14444005)(107886003)(2351001)(4326008)(316002)(5660300001)(25786009)(53936002)(478600001)(2361001)(6506007)(386003)(2906002)(6306002)(6916009)(6486002)(6512007)(81156014)(81166006)(97736004)(966005)(6666003)(8676002)(26005)(7416002)(44832011); DIR:OUT; SFP:1102; SCL:1; SRVR:AM0PR08MB3347; H:xantnef-ws.sw.ru; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: virtuozzo.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; AM0PR08MB3347; 23:IelxTQ1dK/fy5toDHyBgXkOOppW8FhX01J2KayWRb?= r+CdtAO4ZHi8bKRGhjKbmRmxmSaD4g3akMHp+bOo921RluKI+CkSYNSNdfKGUb1pC5zTv6kTJ+VBRyZtsVn2ARzQkdKd5ZaEyUSBm8+GeSUPvzw+vNNB1eIoRIIzxu4ochDqmERUjPwgkKCGOWQpm/HytgHuYddECoqaqtwHThEDMlQDKGqv3pFoQ+rJE3veCTeoxbDOuits1REUHHDK8PkxBBoxH5OF/XaNdkQnfy+USfHqt2gFdWLyKIcSQAskZI0UokHjApyQKq3J7avVJHgQTt1HfxdU8XbTmjqqyaFIycOUn05qKrvyRR/eGjwEizfULmWAH7516kc2PqxXbvsDu99/CurWoaZLF1qO1gfrSF4WbDb3gvhldqzS6k6R08wguYUDJ+AxFgsXwb153YmWmc3TIYryzYXBWNY5c8Cf8nU9cAzSoUHil3NhT5OQPzq9A73q3EPMsGTHlOzE7pEQJL+h7akz6jmwXknmaSIb+kf+eMqQUk4FAOGs7QQvl5/3scOroR7Bh9mLuDgybZ+gl8JKGBU5BkGNUIvBMANxXxl0nJ/tK0UWhE8M+5qtdnVch6uetcoCFVqpQfnFACIVE0vC5peBl/4B8l8uwPtIkSCwemxT6TKlFccmt20ZBrj/j3fb8mxsjIwxXk3dPzYyLFoeCuSXpnBmk+Xv93gPjEFJVtd/r4PBlDTsHGOz1lCGXuqmKcOcP0X7AN0IfQKKFE4K8F43pOAwF614qtL/kx3P7t5Cg1jzjK7HlnP/UJB+I/MxIFh4fhN7Qa4SZ6OKLowVlUI4exjjmbMjuFoIcvVEEiQ/hqZyIPTygbZAz5snyOl65N/l1sY1yXpBdPe9+qRUAWZM3SehTIIkUJHU67VanvAgty2mvb15TU45/C8w68DtS6i5fqEBckTpOLDws5b9Wa9mPCZD95JzzQUvRS0PvL50wx4lNYJrrke8CUIkeRRRi3jX5/yOpQHNvC2vSuzf2zw/XuVx6rwEbzofsT+Yzgn0MrMDpc48ipfC6190XAdn0gU7j83kj8yhTc1wSsO6o73EJShZ8j+ZtQuNUfiF2TZ0DgpIuc8Wgp25a2OXk62MU0EHiRHT+8bCJSFf3fouoDy5YqIF9aOwhvOSv5oNUfQSZqIZfySSeM4Cgm+lE9qNrJax/dVyPYq8A5uk2rWFK8NF6mS1qm+MgDQGAj27BhCjN+AEqbtj7Qnja9+iJM4Vrnzy9Mi9Ia8nnAyJU39tcPJC8t/PKjDilBa7ABHv41WwL+BYLsPgKMHVcM= X-Microsoft-Antispam-Message-Info: ZfvscEd/xSxh2VDH2s5+Avz7ml8lhcFWU00UmuhM9DkZ13SBUWWnOdvYBSbJz05tHFhPFIOc0CJyfWsuT8aFu7TJOsVwviQjA2qhm8CyACJyXx6s2xGvTFYEjisOLIj8n3GFTgCeO00ah44tssi970axFj+iaCn5TOeg4sIrcMBEeacwUkg3RquAL64DZKPJHqyPBd++4ixRLKFtR1C7S5yj0oUOGHDwMw5ade+9lNHMiQurF9x8DuXsnqU5sB89vrVByB7d770jRda701aejhqqhvxxQT2x4qjfqNKI+yLK0+P05/baXLw2yTnr/fzD/SPz+JLKlqsUVh6f2rNrS4JY2asXIHqVhXOawtG0r/0= X-Microsoft-Exchange-Diagnostics: 1; AM0PR08MB3347; 6:j6s/vDC9jA14JxBJdjMhsSUIGORzIUmHybE4rTul/vc8HhlueKU1VvUEsKyFqORT1vAC7RJX+UhAyBuRQ9D1H3+E02jVabgvfs/rAcsISygquVU8rLiyQ11fsbSeba6FezLg/rEyN1EbiUO+UrDb42Y4RNM7GMpi24RQKAFgw6ZqeUZbkk279ULpHt578v2JXUyIedpvZG28mgy5P5OitfPctleDVSqJLQ1/pYv0f8HOYxBXsk6O2bRnzfMlKaifUklAZNn9ymvNy3ICx+0rMQ7VtJ06PG6lFyFzuerFLDzeawSmx+rtX1JzPMXYL89g39isvO9J60jiYU08vUhox8ORgILrBmLM8Auh9VhoL0fTdEtMtpbSSw8mliFBUNOZYeQnmS/SCHfd0brl7beEkkjqEelSiwxM3h/ZjQxvvOK7561LB1u9X/VJ0Y1b6ssfBESUUEPG2eDZTtjDayexdg==; 5:wR35Jd69Uq7nYXDuU52GlH1SNuQzS+pLmdya+/Uo+aCOB1S147IWHs/GeI3eCkfv/N0V3AswE3tEL79QA+kB6LAuttlgyWu1Yqr7qrnzoEBlvErhOoHx2QCwHlgX7JnJoFR5ux3v96BzBM8F1YtAiozCOSFkS+P03TaYZK2MjpU=; 7:lGdbDSvhdJ3OnUvRevGvJY2QzuhYA0ItAAECekGnIAdP32WkJt/9gtWDkBzj2BkZjE93x15fSx/RtvZcfQarBTuYsQIQzNxBx3x+Oyw9wu2Z0+w09dNek0thKttVr2ZlBawvgYP2kuAQGdWCOimSrmDr7Jz4QFFUXy3rUY1VC6gTbA4oqMyOimyj7QflhGbd/7cZRWG9/NL6qwGBn2dFUMt9XbPxkBwNHXG82kkImrKZEWbduQJ6EN0HBnCdXkKD SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; AM0PR08MB3347; 20:lZJ011pxvHiFiecuS9wqwW+s44gmic0HhYoEmjzJBTtU6/UCTUxXUQWZ15uYuHb0FFyii69mjVkNv5dG9wJVUlouE3BmqZoCc7SXjGz2NJ5zsthIfmhqPwrYKApVDSqX3JCLxJz41B7LpWLWU3F+DdZZsdZTG2+Q7b5YgOZ0Mv4= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2018 09:46:36.6697 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4cd0e230-3d70-46af-5fd8-08d6074afd47 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB3347 X-detected-operating-system: by eggs.gnu.org: Windows 7 or 8 [fuzzy] X-Received-From: 40.107.4.122 Subject: [Qemu-devel] [PATCH v4 0/8] discard blockstats X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Anton Nefedov , famz@redhat.com, den@virtuozzo.com, qemu-block@nongnu.org, armbru@redhat.com, mreitz@redhat.com, berto@igalia.com, pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" new in v4: - patch 7: discard and write-zeroes code paths had been separated in 34fa110e: file-posix: Fix write_zeroes with unmap on block devices. This patch now only accounts discards that come explicitly through .bdrv_co_pdiscard handler. - qapi 'Since' clauses changed 3.0 -> 3.1 v3: http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg03688.html ---- qmp query-blockstats provides stats info for write/read/flush ops. Patches 1-6 implement the similar for discard (unmap) command for scsi and ide disks. Discard stat "unmap_ops / unmap_bytes" is supposed to account the ops that have completed without an error. However, discard operation is advisory. Specifically, - common block layer ignores ENOTSUP error code. That might be returned if the block driver does not support discard, or discard has been configured to be ignored. - format drivers such as qcow2 may ignore discard if they were configured to ignore that, or if the corresponding area is already marked unused (unallocated / zero clusters). And what is actually useful is the number of bytes actually discarded down on the host filesystem. To achieve that, driver-specific statistics has been added to blockstats (patch 8). With patch 7, file-posix driver accounts discard operations on its level too. query-blockstat result: (note the difference between blockdevice unmap and file discard stats. qcow2 sends fewer ops down to the file as the clusters are actually unallocated on qcow2 level) { "return": [ { "device": "drive-scsi0-0-0-0", "parent": { > "discard-bytes-ok": 262144, > "discard-nb-ok": 4, "stats": { > "unmap_operations": 0, > "unmap_merged": 0, "flush_total_time_ns": 0, "wr_highest_offset": 8111718400, "wr_total_time_ns": 0, "failed_wr_operations": 0, "failed_rd_operations": 0, "wr_merged": 0, "wr_bytes": 0, "timed_stats": [ ], > "failed_unmap_operations": 0, "failed_flush_operations": 0, "account_invalid": false, "rd_total_time_ns": 0, > "invalid_unmap_operations": 0, "flush_operations": 0, "wr_operations": 0, > "unmap_bytes": 0, "rd_merged": 0, "rd_bytes": 0, > "unmap_total_time_ns": 0, "invalid_flush_operations": 0, "account_failed": false, "rd_operations": 0, "invalid_wr_operations": 0, "invalid_rd_operations": 0 }, "node-name": "#block012", > "driver": "file", > "discard-nb-failed": 0 }, "stats": { > "unmap_operations": 860, > "unmap_merged": 0, "flush_total_time_ns": 21506733, "wr_highest_offset": 13411741696, "wr_total_time_ns": 2212749334, "failed_wr_operations": 0, "failed_rd_operations": 0, "wr_merged": 0, "wr_bytes": 3426304, "timed_stats": [ ], > "failed_unmap_operations": 0, "failed_flush_operations": 0, "account_invalid": true, "rd_total_time_ns": 3617478206, > "invalid_unmap_operations": 0, "flush_operations": 24, "wr_operations": 309, > "unmap_bytes": 11949633536, "rd_merged": 0, "rd_bytes": 141967360, > "unmap_total_time_ns": 14871816, [..] Anton Nefedov (8): qapi: group BlockDeviceStats fields qapi: add unmap to BlockDeviceStats ide: account UNMAP (TRIM) operations scsi: store unmap offset and nb_sectors in request struct scsi: move unmap error checking to the complete callback scsi: account unmap operations file-posix: account discard operations qapi: query-blockstat: add driver specific file-posix stats qapi/block-core.json | 82 +++++++++++++++++++++++++++++++++++++++------- include/block/accounting.h | 1 + include/block/block.h | 1 + include/block/block_int.h | 1 + block.c | 9 +++++ block/file-posix.c | 45 +++++++++++++++++++++++-- block/qapi.c | 11 +++++++ hw/ide/core.c | 12 +++++++ hw/scsi/scsi-disk.c | 29 +++++++++------- tests/qemu-iotests/227.out | 18 ++++++++++ 10 files changed, 184 insertions(+), 25 deletions(-)