From patchwork Thu Dec 17 21:02:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Klaus Jensen X-Patchwork-Id: 1417978 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=irrelevant.dk Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=irrelevant.dk header.i=@irrelevant.dk header.a=rsa-sha256 header.s=fm1 header.b=EUpBt9hp; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm1 header.b=NqbaiNnU; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Cxkyz2z1qz9sVY for ; Fri, 18 Dec 2020 08:04:59 +1100 (AEDT) Received: from localhost ([::1]:53212 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kq0SH-0002KP-8l for incoming@patchwork.ozlabs.org; Thu, 17 Dec 2020 16:04:57 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:36176) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kq0Py-000252-CE; Thu, 17 Dec 2020 16:02:34 -0500 Received: from wout5-smtp.messagingengine.com ([64.147.123.21]:45433) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kq0Pv-0002C0-HC; Thu, 17 Dec 2020 16:02:34 -0500 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 8A8D16A3; Thu, 17 Dec 2020 16:02:26 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Thu, 17 Dec 2020 16:02:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.dk; h=from:to:cc:subject:date:message-id:content-type:mime-version :content-transfer-encoding; s=fm1; bh=Ipn7va8QVQG4eoQiemms2FxBhT 5Y2tr/J7EjvC7vqI4=; b=EUpBt9hpb7vpMVNjWi5bnuxnH5CY28TZYV7ZZKti1g BHIwvZ/Q7aaY48dLlN7VV3Wt+q0kr8DN+Oog07Mg2E8F25vwwVMRDb1oedYaqVW2 HIF/PVb31dAPxO5OPN+mfYU5DUR52xIMQwJCyrURRtFY3bmSu2wjTeY4HFPj4QCk rThHWMiJlAoifxuKtVa22mrPkcVrEa2zHS5UpVvgZpq0o6fler4hLHvuTB0xzGUW 8SdpYJh9U5eE2v641pCaw5GGHnU3VKqKpXeWGlYV/sGkIFTTUKcRzKXZy9XtpuR/ AZG2tfeg6i9EjbxqkgWFv3zdF+wwBSDekbcCEbFzaxag== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; bh=Ipn7va 8QVQG4eoQiemms2FxBhT5Y2tr/J7EjvC7vqI4=; b=NqbaiNnU3wqAuVVmOsbTNX WHJPC02lQpqJlYaUnnR2sh+26cXVVQhFUNvoMU3foFZZ0Xg5TIhtKtA2KXcxAbVg to/q0J6jU2A90J/tYIcTCZ2V0BfeumWpGsmtbQVx9usF1rZhbm6886PKYciSOfWQ ghKw/ZngzS81C4GJN+gcQPKWkLFAqkaDpSnIXGWaMtJgx0Vr/1H+v4a5vclKi8ma +EJbyCiZ2hQY9D81C1bO034+pKHNsQ+fP7mGQ3v34NQQf6QsRQiyRXyZhyMowQ8O B83r+AJZczyKxyzdwAhcaHxvFrK2WO2on10kwYauIpfWnDEkmZm3OQFda+ZqDpRg == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrudelgedgudegudcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvffufffkofgtggfgsehtqhertdertdejnecuhfhrohhmpefmlhgruhhs ucflvghnshgvnhcuoehithhssehirhhrvghlvghvrghnthdrughkqeenucggtffrrghtth gvrhhnpefhgeevkeeigfekvedvteejjeekkedugfdvheeijeffgfekffdvveelffetvdeg hfenucfkphepkedtrdduieejrdelkedrudeltdenucevlhhushhtvghrufhiiigvpedtne curfgrrhgrmhepmhgrihhlfhhrohhmpehithhssehirhhrvghlvghvrghnthdrughk X-ME-Proxy: Received: from apples.local (80-167-98-190-cable.dk.customer.tdc.net [80.167.98.190]) by mail.messagingengine.com (Postfix) with ESMTPA id 8601B24005E; Thu, 17 Dec 2020 16:02:23 -0500 (EST) From: Klaus Jensen To: qemu-devel@nongnu.org Subject: [PATCH RFC 0/3] hw/block/nvme: dif-based end-to-end data protection support Date: Thu, 17 Dec 2020 22:02:19 +0100 Message-Id: <20201217210222.779619-1-its@irrelevant.dk> X-Mailer: git-send-email 2.29.2 MIME-Version: 1.0 Received-SPF: pass client-ip=64.147.123.21; envelope-from=its@irrelevant.dk; helo=wout5-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , qemu-block@nongnu.org, Klaus Jensen , Max Reitz , Klaus Jensen , Stefan Hajnoczi , Keith Busch Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Klaus Jensen This series adds support for extended LBAs and end-to-end data protection. Marked RFC, since there are a bunch of issues that could use some discussion. Storing metadata bytes contiguously with the logical block data and creating a physically extended logical block basically breaks the DULBE and deallocation support I just added. Formatting a namespace with protection information requires the app- and reftags of deallocated or unwritten blocks to be 0xffff and 0xffffffff respectively; this could be used to reintroduce DULBE support in that case, albeit at a somewhat higher cost than the block status flag-based approach. There is basically three ways of storing metadata (and maybe a forth, but that is probably quite the endeavour): 1. Storing metadata as extended blocks directly on the blockdev. That is the approach used in this RFC. 2. Use a separate blockdev. Incidentially, this is also the easiest and most straightforward solution to support MPTR-based "separate metadata". This also allows DULBE and block deallocation to be supported using the existing approach. 3. A hybrid of 1 and 2 where the metadata is stored contiguously at the end of the nvme-ns blockdev. Option 1 obviously works well with DIF-based protection information and extended LBAs since it maps one to one. Option 2 works flawlessly with MPTR-based metadata, but extended LBAs can be "emulated" at the cost of a bunch of scatter/gather operations. The 4th option is extending an existing image format (QCOW2) or create something on top of RAW to supports metadata bytes per block. But both approaches require full API support through the block layer. And probably a lot of other stuff that I did not think about. Anyway, we would love some comments on this. Gollu Appalanaidu (2): nvme: add support for extended LBAs hw/block/nvme: end-to-end data protection Klaus Jensen (1): hw/block/nvme: refactor nvme_dma hw/block/nvme-ns.h | 22 +- hw/block/nvme.h | 36 +++ include/block/nvme.h | 24 +- hw/block/nvme-ns.c | 66 ++++- hw/block/nvme.c | 616 ++++++++++++++++++++++++++++++++++++++---- hw/block/trace-events | 10 + 6 files changed, 704 insertions(+), 70 deletions(-)