From patchwork Thu Jul 11 18:04:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Klaus Jensen X-Patchwork-Id: 1959480 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=irrelevant.dk header.i=@irrelevant.dk header.a=rsa-sha256 header.s=fm1 header.b=H4yQgNbL; dkim=pass (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=m2sfdUvs; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WKjNn5hbBz1xqr for ; Fri, 12 Jul 2024 04:07:05 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sRy9z-0006fs-Eg; Thu, 11 Jul 2024 14:04:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sRy9x-0006bW-Hi; Thu, 11 Jul 2024 14:04:49 -0400 Received: from fhigh5-smtp.messagingengine.com ([103.168.172.156]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sRy9v-0003t4-N3; Thu, 11 Jul 2024 14:04:49 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailfhigh.nyi.internal (Postfix) with ESMTP id B69ED1140395; Thu, 11 Jul 2024 14:04:45 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Thu, 11 Jul 2024 14:04:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.dk; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm1; t=1720721085; x= 1720807485; bh=patIyWlEXlTOaWV0nBfaOYUpTnvgQDVzdrKKr7JCxG8=; b=H 4yQgNbL34vG+ySlW0NEJuI4XLp1o6GHhaKKizV4LXwaBKHiPS93/sYIqJEChyIKx D/miV0BMbq7nYe/1du1J1GyJvSS25Srd4ioYtmGIYJZoMxGSvNVnXTaUCFpi81n2 +YnrEA5VCLM6DixRH9K/6ngdMzRP5sbwMGqliG67P5FJZqTty16EpNHv4+srrDEi EObkDoX7QFGDWkaiWveO0Si6TBW8TJ1K0vzUyE2sehgssTNitMZ60H5VHqvDZnZQ kaDicD3loPktcei3F7WVLjOVS/EOgITEellm0ix2ASBcHPuV+eHVWY1rB2/2Gv5b eSDZ43s5BDqP77lEK/4pA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1720721085; x= 1720807485; bh=patIyWlEXlTOaWV0nBfaOYUpTnvgQDVzdrKKr7JCxG8=; b=m 2sfdUvsdq7Nj7cs5QpE+N68qaD3vzktVC9K4DRnxiFONUhS8To1LXXjyIgupXaII VvG/sWVtPqLXsS6vaNsOzn5VMbdhPbRRgcxoc31Z5XrSmt4NqaYhzaYK+MEOML1i KdmK4Rsvzxf+nq3lk7npn5aL7CuG9B+N3e+JwB6J3eyyhHpIfVpMxourRjCylTwd C1RixqqmYMr4DnPx/hc6S+YNVrAGt9cW9suNcuwWwaYPTkw9Lp1m09jOUljQEbI/ r+i19UJdghsoI3d0K2+pHva2xK78TGMRd1d5eV0NeMAuWhJqfwOB9V1q9iZHYDG4 0bybp2f9Bn7OSY8StUJMQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrfeeggdduvddtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhgggfestdekredtredttdenucfhrhhomhepmfhlrghu shculfgvnhhsvghnuceoihhtshesihhrrhgvlhgvvhgrnhhtrdgukheqnecuggftrfgrth htvghrnhepjefgieelgfeiveehkeeuveehheekfeevgeeigfehfefgjeejhefffeegudej udegnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepih htshesihhrrhgvlhgvvhgrnhhtrdgukh X-ME-Proxy: Feedback-ID: idc91472f:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Jul 2024 14:04:42 -0400 (EDT) From: Klaus Jensen To: qemu-devel@nongnu.org, Peter Maydell Cc: qemu-block@nongnu.org, Kevin Wolf , =?utf-8?q?Philippe?= =?utf-8?q?_Mathieu-Daud=C3=A9?= , Keith Busch , Klaus Jensen , Jesper Devantier , Stefan Hajnoczi , Hanna Reitz , Fam Zheng , John Berg , John Berg , Klaus Jensen Subject: [PULL 1/7] hw/nvme: Add support for setting the MQES for the NVMe emulation Date: Thu, 11 Jul 2024 20:04:38 +0200 Message-ID: <20240711180436.8532-10-its@irrelevant.dk> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240711180436.8532-9-its@irrelevant.dk> References: <20240711180436.8532-9-its@irrelevant.dk> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2892; i=k.jensen@samsung.com; h=from:subject; bh=gQ/L+AStstwGmhQT28OfJbT+GPJlgZz8VyHUWxR47Hw=; b=owJ4nAFtAZL+kA0DAAoBTeGvMW1PDekByyZiAGaQHrTOkoI+mD0j/4T6mQ4bel5XgDaZYBOOJ czkV1C63pHTf4kBMwQAAQoAHRYhBFIoM6p14tzmokdmwE3hrzFtTw3pBQJmkB60AAoJEE3hrzFt Tw3pKeAH+wd+WBLH39yfdSZ5pRBCqh4dfhSxabKKvKwu0t07oLdQLtlxB1kII2xKZ05fyoiQgyV lyfpQ7H4NNcY+wzhG0knoKseCiYZ3rPCvGsu3BHH6/ytDap93LxpmCy41cY0q9aOgB+lAcTqjX2 /RGSMJvmfrxK+Nl5YAgH76mbQI1KrGToskawu95woD6hU/rCnGmgLDBvOEkPmyTN6alWVo04c7X AkYwxqbX8d6Sye4SLhmIGqJWgmDp4rqJDgjabSl5aYDhgu95SlteIaPCkQhjoh3jKMQFSMrWc5O TAR7D4OAww0G/HfRELuOJ5gZwNrrugAK8qjuduU2z5mGHNMC3AD5iaC1 X-Developer-Key: i=k.jensen@samsung.com; a=openpgp; fpr=DDCA4D9C9EF931CC3468427263D56FC5E55DA838 Received-SPF: pass client-ip=103.168.172.156; envelope-from=its@irrelevant.dk; helo=fhigh5-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Berg The MQES field in the CAP register describes the Maximum Queue Entries Supported for the IO queues of an NVMe controller. Adding a +1 to the value in this field results in the total queue size. A full queue is when a queue of size N contains N - 1 entries, and the minimum queue size is 2. Thus the lowest MQES value is 1. This patch adds the new mqes property to the NVMe emulation which allows a user to specify the maximum queue size by setting this property. This is useful as it enables testing of NVMe controller where the MQES is relatively small. The smallest NVMe queue size supported in NVMe is 2 submission and completion entries, which means that the smallest legal mqes value is 1. The following example shows how the mqes can be set for a the NVMe emulation: -drive id=nvme0,if=none,file=nvme.img,format=raw -device nvme,drive=nvme0,serial=foo,mqes=1 If the mqes property is not provided then the default mqes will still be 0x7ff (the queue size is 2048 entries). Signed-off-by: John Berg Reviewed-by: Keith Busch Reviewed-by: Klaus Jensen Signed-off-by: Klaus Jensen --- hw/nvme/ctrl.c | 8 +++++++- hw/nvme/nvme.h | 1 + 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index 066389e391b6..fa7ec0e79490 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -7805,6 +7805,11 @@ static bool nvme_check_params(NvmeCtrl *n, Error **errp) return false; } + if (params->mqes < 1) { + error_setg(errp, "mqes property cannot be less than 1"); + return false; + } + if (n->pmr.dev) { if (params->msix_exclusive_bar) { error_setg(errp, "not enough BARs available to enable PMR"); @@ -8289,7 +8294,7 @@ static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *pci_dev) id->ctratt = cpu_to_le32(ctratt); - NVME_CAP_SET_MQES(cap, 0x7ff); + NVME_CAP_SET_MQES(cap, n->params.mqes); NVME_CAP_SET_CQR(cap, 1); NVME_CAP_SET_TO(cap, 0xf); NVME_CAP_SET_CSS(cap, NVME_CAP_CSS_NVM); @@ -8459,6 +8464,7 @@ static Property nvme_props[] = { params.sriov_max_vq_per_vf, 0), DEFINE_PROP_BOOL("msix-exclusive-bar", NvmeCtrl, params.msix_exclusive_bar, false), + DEFINE_PROP_UINT16("mqes", NvmeCtrl, params.mqes, 0x7ff), DEFINE_PROP_END_OF_LIST(), }; diff --git a/hw/nvme/nvme.h b/hw/nvme/nvme.h index bed8191bd5fd..2e7d31c0ae6d 100644 --- a/hw/nvme/nvme.h +++ b/hw/nvme/nvme.h @@ -521,6 +521,7 @@ typedef struct NvmeParams { uint32_t num_queues; /* deprecated since 5.1 */ uint32_t max_ioqpairs; uint16_t msix_qsize; + uint16_t mqes; uint32_t cmb_size_mb; uint8_t aerl; uint32_t aer_max_queued; From patchwork Thu Jul 11 18:04:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Klaus Jensen X-Patchwork-Id: 1959479 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=irrelevant.dk header.i=@irrelevant.dk header.a=rsa-sha256 header.s=fm1 header.b=BMXk5Hld; dkim=pass (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=BVKBCi+6; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WKjNQ3x2zz1xqr for ; Fri, 12 Jul 2024 04:06:46 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sRyA6-000709-34; Thu, 11 Jul 2024 14:04:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sRyA0-0006ll-SJ; Thu, 11 Jul 2024 14:04:52 -0400 Received: from fhigh5-smtp.messagingengine.com ([103.168.172.156]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sRy9y-0003tU-9m; Thu, 11 Jul 2024 14:04:52 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 238CE11400AD; Thu, 11 Jul 2024 14:04:49 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Thu, 11 Jul 2024 14:04:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.dk; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm1; t=1720721089; x= 1720807489; bh=WO+HZ2heOGTaV73IU7dquxv1Kny0hFhzSzZKp3QVADU=; b=B MXk5HldYxHtXn41xTTI4P82U39a+TeKLGtpZiT0AhmaiCizrTN5E7KYBMPtZAcJA /wz9GbYpzQXOx7AIstQqegz+dE9/O6KfhQ0pRx0HUZDsMJ0u/ZJgy2XVDXWwaK07 KuzFU+kt2deQ5w5aW5fQv1711V7mhY6gQWmw8NkzpSkm/KZ8RJTWv6glDMPHVgUx lV55VO3aAAsoGRJq91MwbxH+y5Bf/nEyiWtCicpRM/Yc7R2KefE73ltx30rPCg1M gTjF+gedIioPRPv1PrVxwvstNvm5cVU1Grz36d+1O3InVHniWBpSPljIT/Vn7kKJ ymAQQ7w11hWSRRp77Mj7Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1720721089; x= 1720807489; bh=WO+HZ2heOGTaV73IU7dquxv1Kny0hFhzSzZKp3QVADU=; b=B VKBCi+6TuPp2arMGOTby29qvYG4TQr5OaThIK1Dp0ACbbVYrDEUxMgxMyvHRQG5H n/rCOWHPjnv9seSU0YpHTGr6VXfLFZ4FkCLhsxuK2a/KE4/Cd4aQPPy8EBcT5nZM YervkDZXNg1ZJoIM19nupQnUDGfNyIGgzFlIG4QMldPbycID8YmrCwVdpHr1JVcP e+ktFsGbJAKY8a+cfpVtKhzMl05T+lFKNJxsuWvStHSrsZlTCdbQHjDM8d/4i2Pu 0h6zoG7lyG0rlegka+WcuqjXoaES9fTEuDRZFss0UcPB0BAvtYGOHM64BgC2bFAg fWN/ALHWLtkop4RA/smuA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrfeeggdduvdduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhgggfestdekredtredttdenucfhrhhomhepmfhlrghu shculfgvnhhsvghnuceoihhtshesihhrrhgvlhgvvhgrnhhtrdgukheqnecuggftrfgrth htvghrnhepjefgieelgfeiveehkeeuveehheekfeevgeeigfehfefgjeejhefffeegudej udegnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepih htshesihhrrhgvlhgvvhgrnhhtrdgukh X-ME-Proxy: Feedback-ID: idc91472f:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Jul 2024 14:04:45 -0400 (EDT) From: Klaus Jensen To: qemu-devel@nongnu.org, Peter Maydell Cc: qemu-block@nongnu.org, Kevin Wolf , =?utf-8?q?Philippe?= =?utf-8?q?_Mathieu-Daud=C3=A9?= , Keith Busch , Klaus Jensen , Jesper Devantier , Stefan Hajnoczi , Hanna Reitz , Fam Zheng , Vincent Fu , qemu-stable@nongnu.org, Vincent Fu , Klaus Jensen Subject: [PULL 2/7] hw/nvme: fix number of PIDs for FDP RUH update Date: Thu, 11 Jul 2024 20:04:39 +0200 Message-ID: <20240711180436.8532-11-its@irrelevant.dk> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240711180436.8532-9-its@irrelevant.dk> References: <20240711180436.8532-9-its@irrelevant.dk> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=985; i=k.jensen@samsung.com; h=from:subject; bh=NRnqEByl/824gnk6FtAb/MAqKkQ6e5hG9fzMuAHEpeg=; b=owJ4nAFtAZL+kA0DAAoBTeGvMW1PDekByyZiAGaQHrRfMflBr6HDt+iTBaCrzZJ0Rz141UYsN HZhx4guplCI6YkBMwQAAQoAHRYhBFIoM6p14tzmokdmwE3hrzFtTw3pBQJmkB60AAoJEE3hrzFt Tw3p/owIAJ5TjvHdhiS1eZlRodUkOd9Rh8l0KiTBUKxVAModT5n4RUvYMo33nBqqFmh4XnmosAO RWQrRPwCtW7LYrlKHwp+21wxJDpqA5PANonUXSxTWH9ffkhnDTJTB9e87tX9GBqudfjdUHiG+/M dEBTm90aS2UyoXt0jo1MhYLQ3QRsIM7y5lE+gktV+bmmSvKefl2dYIj8olROPEagkYevdiCAgvL qsxVlO9c4piNzRt1ncYx5hSwWFo9MW+rSXYGa0+evzRoZKEjLiyP3BgpDY0YBohXFrbSJcz+Zzo P/7ARs1mte6Xz1pinS4uyhjZB3VMA4O+6fmWLS3f4T4FAUZqASdQSaYd X-Developer-Key: i=k.jensen@samsung.com; a=openpgp; fpr=DDCA4D9C9EF931CC3468427263D56FC5E55DA838 Received-SPF: pass client-ip=103.168.172.156; envelope-from=its@irrelevant.dk; helo=fhigh5-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Vincent Fu The number of PIDs is in the upper 16 bits of cdw10. So we need to right-shift by 16 bits instead of only a single bit. Fixes: 73064edfb864 ("hw/nvme: flexible data placement emulation") Cc: qemu-stable@nongnu.org Signed-off-by: Vincent Fu Reviewed-by: Klaus Jensen Signed-off-by: Klaus Jensen --- hw/nvme/ctrl.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index fa7ec0e79490..231e1127cec8 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -4352,7 +4352,7 @@ static uint16_t nvme_io_mgmt_send_ruh_update(NvmeCtrl *n, NvmeRequest *req) NvmeNamespace *ns = req->ns; uint32_t cdw10 = le32_to_cpu(cmd->cdw10); uint16_t ret = NVME_SUCCESS; - uint32_t npid = (cdw10 >> 1) + 1; + uint32_t npid = (cdw10 >> 16) + 1; unsigned int i = 0; g_autofree uint16_t *pids = NULL; uint32_t maxnpid; From patchwork Thu Jul 11 18:04:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Klaus Jensen X-Patchwork-Id: 1959475 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=irrelevant.dk header.i=@irrelevant.dk header.a=rsa-sha256 header.s=fm1 header.b=FBI0+5UG; dkim=pass (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=ZPXtczCd; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WKjMt5Vllz1xqr for ; Fri, 12 Jul 2024 04:06:18 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sRyAD-0007Iw-NF; Thu, 11 Jul 2024 14:05:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sRyA4-0006yj-7B; Thu, 11 Jul 2024 14:04:56 -0400 Received: from fhigh5-smtp.messagingengine.com ([103.168.172.156]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sRyA1-0003wj-32; Thu, 11 Jul 2024 14:04:54 -0400 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 5222F11406A8; Thu, 11 Jul 2024 14:04:52 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Thu, 11 Jul 2024 14:04:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.dk; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm1; t=1720721092; x= 1720807492; bh=4HASoSfZRxPdg/HZsQNUQDHZWGxP6Kj4aXho5mhlYEo=; b=F BI0+5UGkx+eYnP/vmUZIlEd/YZiJBF2gtw4IbL1bO5SG6TMGaQQox0Sd2NJ4wkzY 1yxKqB8X/FoIdTpXcq+jWs1Xr50zAJ4StIyF0DJy4ZS5XA3Y+cOTCgrB5/F7IUYG rJteg8nCBO0CrLC+z6KOL9gLax1yUZ+ss1bS8WP//rvlOOXlm2BmmRPv9XjMgy34 O/uwRGvizcvk6m9Qt8TM9QsoD+NH/7JXrBHSOzgUvZT1IuYKVloZ9f48arNtmGMn 9sXNhO0Wrn0oHgxBESD6mI3QCf4Mg984j6dhvXmUoXuc99NXEUo7CGwTtRpWvJQT NSQexcP5KCKoYh59giy1A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1720721092; x= 1720807492; bh=4HASoSfZRxPdg/HZsQNUQDHZWGxP6Kj4aXho5mhlYEo=; b=Z PXtczCd7qVmNQTiiApBcXjEh+BJzSt9P8n90LoCphJ9NTsl6yfXMtu5D97jImp/s M7a/cZYtRWGAHYspcWO0qAPjdAcY0kwEhtCwHJ+HFLlV+VwTYu1EAZI8TGhjTKvD 26eUCuFgadBdJF0YBvn5X5WfiboUWmNkbPCuljhiqxV8zYjFfeQhl3ncppeKZWq2 t1HG3dtNMoWQkCtKcHtVNiSmdqBPWuW14Ixksh8mybEZOwYbibGc5+a8Jh96VwM0 5zNpTOa/xa4QDHZMHbXQOYuSv6FWlO4NkvlUhxesSueXevwJIyOFu9gIi3/LN9ZZ otGfPQ0ghoezdl4ZRq12A== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrfeeggdduvdduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhgggfestdekredtredttdenucfhrhhomhepmfhlrghu shculfgvnhhsvghnuceoihhtshesihhrrhgvlhgvvhgrnhhtrdgukheqnecuggftrfgrth htvghrnhepjefgieelgfeiveehkeeuveehheekfeevgeeigfehfefgjeejhefffeegudej udegnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepih htshesihhrrhgvlhgvvhgrnhhtrdgukh X-ME-Proxy: Feedback-ID: idc91472f:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Jul 2024 14:04:49 -0400 (EDT) From: Klaus Jensen To: qemu-devel@nongnu.org, Peter Maydell Cc: qemu-block@nongnu.org, Kevin Wolf , =?utf-8?q?Philippe?= =?utf-8?q?_Mathieu-Daud=C3=A9?= , Keith Busch , Klaus Jensen , Jesper Devantier , Stefan Hajnoczi , Hanna Reitz , Fam Zheng , Minwoo Im , Klaus Jensen Subject: [PULL 3/7] hw/nvme: fix BAR size mismatch of SR-IOV VF Date: Thu, 11 Jul 2024 20:04:40 +0200 Message-ID: <20240711180436.8532-12-its@irrelevant.dk> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240711180436.8532-9-its@irrelevant.dk> References: <20240711180436.8532-9-its@irrelevant.dk> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3605; i=k.jensen@samsung.com; h=from:subject; bh=ywvt19DaycrPLQkuZuEEUyxvLK5gVPKhT1LHVJGcO7E=; b=owJ4nAFtAZL+kA0DAAoBTeGvMW1PDekByyZiAGaQHrVsA41B5SgfmIxk0DTZY68fp1c4Y3W9B sGFKD/4N86spIkBMwQAAQoAHRYhBFIoM6p14tzmokdmwE3hrzFtTw3pBQJmkB61AAoJEE3hrzFt Tw3pOyEH/1zYW5Xl9xW2BWdhcZruugVIjhwj848zZ1vGdF0wItcCE1FIu8wM2WarG0dhGmly0Pp loo1nYrQPr3kKYMl8SKLsIEup2V+pntajoV11oeArKUaNA2wv8X4YUdouvjTgXmiXfJronHMi6W wH8goCoPdGfX8S61y4AXhJMUJIUOh3RSIOxPMeYDvgyTvzfVvmusbi7sh2fX1LO1MbrMddPbvaJ vDg3/R7yQaLVD3c9iYTBJ38NWJX2OLQjdmJhie5aEVgALtI8y7/aiNJMQ3g4VYumUI77/3qRnMw ICRHq+0LwljIC8yvJQXQj5vIq1iceFeYuWd1UZolCETkjioP+JsI1qSl X-Developer-Key: i=k.jensen@samsung.com; a=openpgp; fpr=DDCA4D9C9EF931CC3468427263D56FC5E55DA838 Received-SPF: pass client-ip=103.168.172.156; envelope-from=its@irrelevant.dk; helo=fhigh5-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Minwoo Im PF initializes SR-IOV VF BAR0 region in nvme_init_sriov() with bar_size calcaulted by Primary Controller Capability such as VQFRSM and VIFRSM rather than `max_ioqpairs` and `msix_qsize` which is for PF only. In this case, the bar size reported in nvme_init_sriov() by PF and nvme_init_pci() by VF might differ especially with large number of sriov_max_vfs (e.g., 127 which is curret maximum number of VFs). And this reports invalid BAR0 address of VFs to the host operating system so that MMIO access will not be caught properly and, of course, NVMe driver initialization is failed. For example, if we give the following options, BAR size will be initialized by PF with 4K, but VF will try to allocate 8K BAR0 size in nvme_init_pci(). #!/bin/bash nr_vf=$((127)) nr_vq=$(($nr_vf * 2 + 2)) nr_vi=$(($nr_vq / 2 + 1)) nr_ioq=$(($nr_vq + 2)) ... -device nvme,serial=foo,id=nvme0,bus=rp2,subsys=subsys0,mdts=9,msix_qsize=$nr_ioq,max_ioqpairs=$nr_ioq,sriov_max_vfs=$nr_vf,sriov_vq_flexible=$nr_vq,sriov_vi_flexible=$nr_vi \ To fix this issue, this patch modifies the calculation of BAR size in the PF and VF initialization by using different elements: PF: `max_ioqpairs + 1` with `msix_qsize` VF: VQFRSM with VIFRSM Signed-off-by: Minwoo Im Reviewed-by: Klaus Jensen Signed-off-by: Klaus Jensen --- hw/nvme/ctrl.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index 231e1127cec8..f3ae54896f6d 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -8104,6 +8104,7 @@ static bool nvme_init_pci(NvmeCtrl *n, PCIDevice *pci_dev, Error **errp) uint8_t *pci_conf = pci_dev->config; uint64_t bar_size; unsigned msix_table_offset = 0, msix_pba_offset = 0; + unsigned nr_vectors; int ret; pci_conf[PCI_INTERRUPT_PIN] = 1; @@ -8136,9 +8137,19 @@ static bool nvme_init_pci(NvmeCtrl *n, PCIDevice *pci_dev, Error **errp) assert(n->params.msix_qsize >= 1); /* add one to max_ioqpairs to account for the admin queue pair */ - bar_size = nvme_mbar_size(n->params.max_ioqpairs + 1, - n->params.msix_qsize, &msix_table_offset, - &msix_pba_offset); + if (!pci_is_vf(pci_dev)) { + nr_vectors = n->params.msix_qsize; + bar_size = nvme_mbar_size(n->params.max_ioqpairs + 1, + nr_vectors, &msix_table_offset, + &msix_pba_offset); + } else { + NvmeCtrl *pn = NVME(pcie_sriov_get_pf(pci_dev)); + NvmePriCtrlCap *cap = &pn->pri_ctrl_cap; + + nr_vectors = le16_to_cpu(cap->vifrsm); + bar_size = nvme_mbar_size(le16_to_cpu(cap->vqfrsm), nr_vectors, + &msix_table_offset, &msix_pba_offset); + } memory_region_init(&n->bar0, OBJECT(n), "nvme-bar0", bar_size); memory_region_init_io(&n->iomem, OBJECT(n), &nvme_mmio_ops, n, "nvme", @@ -8152,7 +8163,7 @@ static bool nvme_init_pci(NvmeCtrl *n, PCIDevice *pci_dev, Error **errp) PCI_BASE_ADDRESS_MEM_TYPE_64, &n->bar0); } - ret = msix_init(pci_dev, n->params.msix_qsize, + ret = msix_init(pci_dev, nr_vectors, &n->bar0, 0, msix_table_offset, &n->bar0, 0, msix_pba_offset, 0, errp); } From patchwork Thu Jul 11 18:04:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Klaus Jensen X-Patchwork-Id: 1959473 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=irrelevant.dk header.i=@irrelevant.dk header.a=rsa-sha256 header.s=fm1 header.b=I3hcItcK; dkim=pass (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=tddzMYDh; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WKjMT57g2z1xqr for ; Fri, 12 Jul 2024 04:05:57 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sRyAF-0007Mt-6n; Thu, 11 Jul 2024 14:05:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sRyA6-00075l-GC; Thu, 11 Jul 2024 14:05:00 -0400 Received: from fhigh5-smtp.messagingengine.com ([103.168.172.156]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sRyA4-0003x1-P8; Thu, 11 Jul 2024 14:04:58 -0400 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 8091D1140395; Thu, 11 Jul 2024 14:04:55 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Thu, 11 Jul 2024 14:04:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.dk; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm1; t=1720721095; x= 1720807495; bh=fU0R5yFm01ORA9K//V//NlC8z2SUntdN8U1tV783OlM=; b=I 3hcItcKwDfVkO8U2XZYnzKK2s/kesvma/BsXzX5IJ2ulfukxaeffvvaneKY8t9KC KIUW6aO0zGklWvmgQT2HtNMMxJLycVct7oiVplFH6sVghRFkeH4s0yoIkFhbw6Lc b1F22tfzHLvng61koP8Z7Wru1PBLV6zmNPNxkbh7RteopbCAvWWbb7iO7Ff3Ut1d hFLSn1qy3NE8nanlZ7BZO0tVDJwFxI95KpehYNA1+2DxTy1whQmaS3hAvoYJ+rK9 ZnEbdeAnaE9K+LXFKylfzHC5/7VWjMmE3E0FUcv3LDI0hJXSo/OuNZ1gOXpOjV/y toC1yAFCrbYkoGwOR74YA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1720721095; x= 1720807495; bh=fU0R5yFm01ORA9K//V//NlC8z2SUntdN8U1tV783OlM=; b=t ddzMYDh9i/oR0o74aSJDdeld9cg4Yztw0VHESVvoGpuU/4bq5lS5blpk2WP+WsPz p7eACwl1jLj2+4JLJtPnTieKZvgSzVOfExIktg950j64t2R6J8XE/hs2YVijLikh vHpOwwYto2MVBgL/il6CCJnpz9lD1TzB+QTIJS445mk2pof+IWNvuM7mKUDBeLjI nNSh3rKofjJkxjjaY4nVxYzB8yoPesDLhi2SKdWaCBc39uJDTkaoiaUz8Lk0rOvW f+5btSzthMvIHjlKADyrv3Sv7lXwACjcQq5NhP3VdFSYSaRYhl9sJVp6oVsH6Bzr r7VPtlU7WzYfDjak6xUJQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrfeeggdduvdduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhgggfestdekredtredttdenucfhrhhomhepmfhlrghu shculfgvnhhsvghnuceoihhtshesihhrrhgvlhgvvhgrnhhtrdgukheqnecuggftrfgrth htvghrnhepjefgieelgfeiveehkeeuveehheekfeevgeeigfehfefgjeejhefffeegudej udegnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepih htshesihhrrhgvlhgvvhgrnhhtrdgukh X-ME-Proxy: Feedback-ID: idc91472f:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Jul 2024 14:04:52 -0400 (EDT) From: Klaus Jensen To: qemu-devel@nongnu.org, Peter Maydell Cc: qemu-block@nongnu.org, Kevin Wolf , =?utf-8?q?Philippe?= =?utf-8?q?_Mathieu-Daud=C3=A9?= , Keith Busch , Klaus Jensen , Jesper Devantier , Stefan Hajnoczi , Hanna Reitz , Fam Zheng , Minwoo Im , Klaus Jensen Subject: [PULL 4/7] hw/nvme: add Identify Endurance Group List Date: Thu, 11 Jul 2024 20:04:41 +0200 Message-ID: <20240711180436.8532-13-its@irrelevant.dk> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240711180436.8532-9-its@irrelevant.dk> References: <20240711180436.8532-9-its@irrelevant.dk> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2675; i=k.jensen@samsung.com; h=from:subject; bh=bLLqYH+xpFPc6bj1dQ4f6vAV7JLMQj/+ItiUpoYuPOA=; b=owJ4nAFtAZL+kA0DAAoBTeGvMW1PDekByyZiAGaQHrU8bZRr6zIgFxItj9xrzxUI9CElqs2CU CUAnZF7nVMpwYkBMwQAAQoAHRYhBFIoM6p14tzmokdmwE3hrzFtTw3pBQJmkB61AAoJEE3hrzFt Tw3pXEwH/0wPzyvsHQ1QZHI7kShq8cgvnoXGtheuhaxQk370jOv5Ulvna4iaJpYk9W8MovYp3S+ lF/XGz1goxNsqkO2s0nwdtzk0kBvP8G1+8ZT24Bc8fo4ELx+DEFRWfVhh0OTtNEiFFu+UJRdYVF Us0xwhZCN8kWbMLoDDBiWXi2Lqm+sE8FSWapz7nEsD6w/BK/4Nca6z0iLEN1cIUKjkXcqt4jQ9W uiIODdwdoB8EQ+vkWK24aiZPYjLcDoLeOiDRwMFBmycW9RsEizvew1FKL0OefPMqMZ08BE83G5M QblLBznpIeCjkjfG6KT3Q4jcnyr0qjqZebzZo7BDW3QOIkeafjUMdaLR X-Developer-Key: i=k.jensen@samsung.com; a=openpgp; fpr=DDCA4D9C9EF931CC3468427263D56FC5E55DA838 Received-SPF: pass client-ip=103.168.172.156; envelope-from=its@irrelevant.dk; helo=fhigh5-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Minwoo Im Commit 73064edfb864 ("hw/nvme: flexible data placement emulation") intorudced NVMe FDP feature to nvme-subsys and nvme-ctrl with a single endurance group #1 supported. This means that controller should return proper identify data to host with Identify Endurance Group List (CNS 19h). But, yes, only just for the endurance group #1. This patch allows host applications to ask for which endurance group is available and utilize FDP through that endurance group. Reviewed-by: Klaus Jensen Signed-off-by: Minwoo Im Signed-off-by: Klaus Jensen --- hw/nvme/ctrl.c | 22 ++++++++++++++++++++++ include/block/nvme.h | 1 + 2 files changed, 23 insertions(+) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index f3ae54896f6d..50f8cc90b038 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -5629,6 +5629,26 @@ static uint16_t nvme_identify_nslist_csi(NvmeCtrl *n, NvmeRequest *req, return nvme_c2h(n, list, data_len, req); } +static uint16_t nvme_endurance_group_list(NvmeCtrl *n, NvmeRequest *req) +{ + uint16_t list[NVME_CONTROLLER_LIST_SIZE] = {}; + uint16_t *nr_ids = &list[0]; + uint16_t *ids = &list[1]; + uint16_t endgid = le32_to_cpu(req->cmd.cdw11) & 0xffff; + + /* + * The current nvme-subsys only supports Endurance Group #1. + */ + if (!endgid) { + *nr_ids = 1; + ids[0] = 1; + } else { + *nr_ids = 0; + } + + return nvme_c2h(n, list, sizeof(list), req); +} + static uint16_t nvme_identify_ns_descr_list(NvmeCtrl *n, NvmeRequest *req) { NvmeNamespace *ns; @@ -5744,6 +5764,8 @@ static uint16_t nvme_identify(NvmeCtrl *n, NvmeRequest *req) return nvme_identify_nslist(n, req, false); case NVME_ID_CNS_CS_NS_ACTIVE_LIST: return nvme_identify_nslist_csi(n, req, true); + case NVME_ID_CNS_ENDURANCE_GROUP_LIST: + return nvme_endurance_group_list(n, req); case NVME_ID_CNS_CS_NS_PRESENT_LIST: return nvme_identify_nslist_csi(n, req, false); case NVME_ID_CNS_NS_DESCR_LIST: diff --git a/include/block/nvme.h b/include/block/nvme.h index bb231d0b9ad0..7c77d38174a7 100644 --- a/include/block/nvme.h +++ b/include/block/nvme.h @@ -1074,6 +1074,7 @@ enum NvmeIdCns { NVME_ID_CNS_CTRL_LIST = 0x13, NVME_ID_CNS_PRIMARY_CTRL_CAP = 0x14, NVME_ID_CNS_SECONDARY_CTRL_LIST = 0x15, + NVME_ID_CNS_ENDURANCE_GROUP_LIST = 0x19, NVME_ID_CNS_CS_NS_PRESENT_LIST = 0x1a, NVME_ID_CNS_CS_NS_PRESENT = 0x1b, NVME_ID_CNS_IO_COMMAND_SET = 0x1c, From patchwork Thu Jul 11 18:04:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Klaus Jensen X-Patchwork-Id: 1959478 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=irrelevant.dk header.i=@irrelevant.dk header.a=rsa-sha256 header.s=fm1 header.b=oVmeWd8t; dkim=pass (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=Kq4meD7/; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WKjNN4tmmz1xqr for ; Fri, 12 Jul 2024 04:06:44 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sRyAD-0007HJ-7s; Thu, 11 Jul 2024 14:05:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sRyA9-0007B6-Qy; Thu, 11 Jul 2024 14:05:02 -0400 Received: from fout1-smtp.messagingengine.com ([103.168.172.144]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sRyA8-0003xf-0l; Thu, 11 Jul 2024 14:05:01 -0400 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailfout.nyi.internal (Postfix) with ESMTP id ADFDB138815B; Thu, 11 Jul 2024 14:04:58 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Thu, 11 Jul 2024 14:04:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.dk; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm1; t=1720721098; x= 1720807498; bh=BKUuf6//UbpYmLZqAbqnAhGjR5i4tJ9ytdfz09jbFdE=; b=o VmeWd8tpX148ruqfSDSSadtVnT22m91uky+j0cUc/3M1clIGQOh9jijVOEGo6n3S kEfoiX4w4aPBW6rastgvQ47ZCy8ecS8YRsOJGFPLoCOarTX5xwoMvBnVVDlU2lz3 dHUziAEZHHT2Zn7ztP3YMxP+L7egh83IIh8RRYRvxHmqvV5MxLnX67y2zEj6wrbO abuWbAp+QL56dbibSMLfmED+rx+pNSOrchqF7XNYa/4KGJLGLRDxthhEDnWGI/vF sHnDblJCzB0vhw/hd2jdtJ886NkTYuihNdWxgJjWZ6I/8bWZgAvRGmEqzYlEqQAl 5wMnv7WFS7xO8eK7WAYKA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1720721098; x= 1720807498; bh=BKUuf6//UbpYmLZqAbqnAhGjR5i4tJ9ytdfz09jbFdE=; b=K q4meD7/NYmnwmxbXYqLWSJHeyuPAzdYA9t/ee4U7ZZjUz+fMIbXOORkwyx6p7wOB IWF8M1HBtc7Wfs/Utw8t82YtbdI5sUfoFGEKOBWLba71g7/F5V/B8rET5SLLg9ZI 6JVleF60NbRr9mu4on/zzAgXQAQEa+WpXjkTt0D905n/pjMWfsfQQ/aIix2w6aSH hI066ZbS9HsxzaCI5foPJTciELbHAVJuUYVMhfFQM6j42+jZjrRByU09s5uiE2eP izHwrwgDUfyEhInH7V1CH1g5AtxlBWlAa5gGA9evEFYDjnVvkOV52SfpKP+DQYQk BDNL5dJZUfWjW0EyjSVmg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrfeeggdduvdduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhgggfestdekredtredttdenucfhrhhomhepmfhlrghu shculfgvnhhsvghnuceoihhtshesihhrrhgvlhgvvhgrnhhtrdgukheqnecuggftrfgrth htvghrnhepjefgieelgfeiveehkeeuveehheekfeevgeeigfehfefgjeejhefffeegudej udegnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepih htshesihhrrhgvlhgvvhgrnhhtrdgukh X-ME-Proxy: Feedback-ID: idc91472f:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Jul 2024 14:04:55 -0400 (EDT) From: Klaus Jensen To: qemu-devel@nongnu.org, Peter Maydell Cc: qemu-block@nongnu.org, Kevin Wolf , =?utf-8?q?Philippe?= =?utf-8?q?_Mathieu-Daud=C3=A9?= , Keith Busch , Klaus Jensen , Jesper Devantier , Stefan Hajnoczi , Hanna Reitz , Fam Zheng , Minwoo Im , Klaus Jensen Subject: [PULL 5/7] hw/nvme: separate identify data for sec. ctrl list Date: Thu, 11 Jul 2024 20:04:42 +0200 Message-ID: <20240711180436.8532-14-its@irrelevant.dk> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240711180436.8532-9-its@irrelevant.dk> References: <20240711180436.8532-9-its@irrelevant.dk> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=6763; i=k.jensen@samsung.com; h=from:subject; bh=rH0hF3TT/+xqP5ifmpt5IZWcD7feHCeFEaDZHk42zFo=; b=owJ4nAFtAZL+kA0DAAoBTeGvMW1PDekByyZiAGaQHrVLqBVL4LozslJlQqqPXUZApasNAsKto zgw1C8UhS8Fd4kBMwQAAQoAHRYhBFIoM6p14tzmokdmwE3hrzFtTw3pBQJmkB61AAoJEE3hrzFt Tw3pHSsH/0mNL2CS4CXNMrnTASwnIv50k6qzqbERlBvshx8Odpu9y9JfVis1zGS0J2SEFyIyAt/ XrBQkb9HjvWB6rbPUXkO2FxS0yByw7ZxPlx1aZhTWQ3N9hFbTt9B+p+egc6DQehnOFzt4y67VDS 2RO0cZtTiCtqgx8swIlSZ2PrPQ+CrosMkLr0lTHJw2dGBrO9LUP+OVkikHcEsQapDdIJY/4B/9c YOsh3IguX+Sa6bgH2YDgVb7epK97xre5fPAzt1bUJ17pjFOtRtGk/ImiF4RpqGtvTzRkUGwoHDX ZzXTQoypyCIQGaCplV4RCYbSVC5IU5cWi62jtxhfOaFUDclIvMRBtaLB X-Developer-Key: i=k.jensen@samsung.com; a=openpgp; fpr=DDCA4D9C9EF931CC3468427263D56FC5E55DA838 Received-SPF: pass client-ip=103.168.172.144; envelope-from=its@irrelevant.dk; helo=fout1-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Minwoo Im Secondary controller list for virtualization has been managed by Identify Secondary Controller List data structure with NvmeSecCtrlList where up to 127 secondary controller entries can be managed. The problem hasn't arisen so far because NVME_MAX_VFS has been 127. This patch separated identify data itself from the actual secondary controller list managed by controller to support more than 127 secondary controllers with the following patch. This patch reused NvmeSecCtrlEntry structure to manage all the possible secondary controllers, and copy entries to identify data structure when the command comes in. Reviewed-by: Klaus Jensen Signed-off-by: Minwoo Im Signed-off-by: Klaus Jensen --- hw/nvme/ctrl.c | 21 ++++++++++----------- hw/nvme/nvme.h | 14 ++++++++------ hw/nvme/subsys.c | 8 ++++---- 3 files changed, 22 insertions(+), 21 deletions(-) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index 50f8cc90b038..8a838e5b658b 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -219,7 +219,6 @@ #define NVME_TEMPERATURE_CRITICAL 0x175 #define NVME_NUM_FW_SLOTS 1 #define NVME_DEFAULT_MAX_ZA_SIZE (128 * KiB) -#define NVME_MAX_VFS 127 #define NVME_VF_RES_GRANULARITY 1 #define NVME_VF_OFFSET 0x1 #define NVME_VF_STRIDE 1 @@ -5480,14 +5479,14 @@ static uint16_t nvme_identify_sec_ctrl_list(NvmeCtrl *n, NvmeRequest *req) NvmeIdentify *c = (NvmeIdentify *)&req->cmd; uint16_t pri_ctrl_id = le16_to_cpu(n->pri_ctrl_cap.cntlid); uint16_t min_id = le16_to_cpu(c->ctrlid); - uint8_t num_sec_ctrl = n->sec_ctrl_list.numcntl; + uint8_t num_sec_ctrl = n->nr_sec_ctrls; NvmeSecCtrlList list = {0}; uint8_t i; for (i = 0; i < num_sec_ctrl; i++) { - if (n->sec_ctrl_list.sec[i].scid >= min_id) { - list.numcntl = num_sec_ctrl - i; - memcpy(&list.sec, n->sec_ctrl_list.sec + i, + if (n->sec_ctrl_list[i].scid >= min_id) { + list.numcntl = MIN(num_sec_ctrl - i, 127); + memcpy(&list.sec, n->sec_ctrl_list + i, list.numcntl * sizeof(NvmeSecCtrlEntry)); break; } @@ -7144,8 +7143,8 @@ static void nvme_ctrl_reset(NvmeCtrl *n, NvmeResetType rst) if (n->params.sriov_max_vfs) { if (!pci_is_vf(pci_dev)) { - for (i = 0; i < n->sec_ctrl_list.numcntl; i++) { - sctrl = &n->sec_ctrl_list.sec[i]; + for (i = 0; i < n->nr_sec_ctrls; i++) { + sctrl = &n->sec_ctrl_list[i]; nvme_virt_set_state(n, le16_to_cpu(sctrl->scid), false); } } @@ -7939,7 +7938,7 @@ static bool nvme_check_params(NvmeCtrl *n, Error **errp) static void nvme_init_state(NvmeCtrl *n) { NvmePriCtrlCap *cap = &n->pri_ctrl_cap; - NvmeSecCtrlList *list = &n->sec_ctrl_list; + NvmeSecCtrlEntry *list = n->sec_ctrl_list; NvmeSecCtrlEntry *sctrl; PCIDevice *pci = PCI_DEVICE(n); uint8_t max_vfs; @@ -7964,9 +7963,9 @@ static void nvme_init_state(NvmeCtrl *n) n->aer_reqs = g_new0(NvmeRequest *, n->params.aerl + 1); QTAILQ_INIT(&n->aer_queue); - list->numcntl = max_vfs; + n->nr_sec_ctrls = max_vfs; for (i = 0; i < max_vfs; i++) { - sctrl = &list->sec[i]; + sctrl = &list[i]; sctrl->pcid = cpu_to_le16(n->cntlid); sctrl->vfn = cpu_to_le16(i + 1); } @@ -8559,7 +8558,7 @@ static void nvme_sriov_post_write_config(PCIDevice *dev, uint16_t old_num_vfs) int i; for (i = pcie_sriov_num_vfs(dev); i < old_num_vfs; i++) { - sctrl = &n->sec_ctrl_list.sec[i]; + sctrl = &n->sec_ctrl_list[i]; nvme_virt_set_state(n, le16_to_cpu(sctrl->scid), false); } } diff --git a/hw/nvme/nvme.h b/hw/nvme/nvme.h index 2e7d31c0ae6d..9da5343ffe90 100644 --- a/hw/nvme/nvme.h +++ b/hw/nvme/nvme.h @@ -26,6 +26,7 @@ #define NVME_MAX_CONTROLLERS 256 #define NVME_MAX_NAMESPACES 256 +#define NVME_MAX_VFS 127 #define NVME_EUI64_DEFAULT ((uint64_t)0x5254000000000000) #define NVME_FDP_MAX_EVENTS 63 #define NVME_FDP_MAXPIDS 128 @@ -613,7 +614,8 @@ typedef struct NvmeCtrl { } features; NvmePriCtrlCap pri_ctrl_cap; - NvmeSecCtrlList sec_ctrl_list; + uint32_t nr_sec_ctrls; + NvmeSecCtrlEntry sec_ctrl_list[NVME_MAX_VFS]; struct { uint16_t vqrfap; uint16_t virfap; @@ -663,7 +665,7 @@ static inline NvmeSecCtrlEntry *nvme_sctrl(NvmeCtrl *n) NvmeCtrl *pf = NVME(pcie_sriov_get_pf(pci_dev)); if (pci_is_vf(pci_dev)) { - return &pf->sec_ctrl_list.sec[pcie_sriov_vf_number(pci_dev)]; + return &pf->sec_ctrl_list[pcie_sriov_vf_number(pci_dev)]; } return NULL; @@ -672,12 +674,12 @@ static inline NvmeSecCtrlEntry *nvme_sctrl(NvmeCtrl *n) static inline NvmeSecCtrlEntry *nvme_sctrl_for_cntlid(NvmeCtrl *n, uint16_t cntlid) { - NvmeSecCtrlList *list = &n->sec_ctrl_list; + NvmeSecCtrlEntry *list = n->sec_ctrl_list; uint8_t i; - for (i = 0; i < list->numcntl; i++) { - if (le16_to_cpu(list->sec[i].scid) == cntlid) { - return &list->sec[i]; + for (i = 0; i < n->nr_sec_ctrls; i++) { + if (le16_to_cpu(list[i].scid) == cntlid) { + return &list[i]; } } diff --git a/hw/nvme/subsys.c b/hw/nvme/subsys.c index d30bb8bfd5b4..561ed04a5317 100644 --- a/hw/nvme/subsys.c +++ b/hw/nvme/subsys.c @@ -17,13 +17,13 @@ static int nvme_subsys_reserve_cntlids(NvmeCtrl *n, int start, int num) { NvmeSubsystem *subsys = n->subsys; - NvmeSecCtrlList *list = &n->sec_ctrl_list; + NvmeSecCtrlEntry *list = n->sec_ctrl_list; NvmeSecCtrlEntry *sctrl; int i, cnt = 0; for (i = start; i < ARRAY_SIZE(subsys->ctrls) && cnt < num; i++) { if (!subsys->ctrls[i]) { - sctrl = &list->sec[cnt]; + sctrl = &list[cnt]; sctrl->scid = cpu_to_le16(i); subsys->ctrls[i] = SUBSYS_SLOT_RSVD; cnt++; @@ -36,12 +36,12 @@ static int nvme_subsys_reserve_cntlids(NvmeCtrl *n, int start, int num) static void nvme_subsys_unreserve_cntlids(NvmeCtrl *n) { NvmeSubsystem *subsys = n->subsys; - NvmeSecCtrlList *list = &n->sec_ctrl_list; + NvmeSecCtrlEntry *list = n->sec_ctrl_list; NvmeSecCtrlEntry *sctrl; int i, cntlid; for (i = 0; i < n->params.sriov_max_vfs; i++) { - sctrl = &list->sec[i]; + sctrl = &list[i]; cntlid = le16_to_cpu(sctrl->scid); if (cntlid) { From patchwork Thu Jul 11 18:04:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Klaus Jensen X-Patchwork-Id: 1959474 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=irrelevant.dk header.i=@irrelevant.dk header.a=rsa-sha256 header.s=fm1 header.b=PuurNt7Z; dkim=pass (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=G9o/CJnr; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WKjMs3BVMz1xqr for ; Fri, 12 Jul 2024 04:06:17 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sRyAR-0007uh-RY; Thu, 11 Jul 2024 14:05:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sRyAE-0007Kk-2t; Thu, 11 Jul 2024 14:05:06 -0400 Received: from fout1-smtp.messagingengine.com ([103.168.172.144]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sRyAC-0003yC-Aq; Thu, 11 Jul 2024 14:05:05 -0400 Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailfout.nyi.internal (Postfix) with ESMTP id CFB76138815B; Thu, 11 Jul 2024 14:05:01 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute7.internal (MEProxy); Thu, 11 Jul 2024 14:05:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.dk; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm1; t=1720721101; x= 1720807501; bh=Fq1LlHL5Af2WmagmF5wE56kSRxwIsdDG12xAo8ifsT0=; b=P uurNt7Zos6RC5jsWSPBpjUqveuGaQmz59pgq9eDw9ziZuYW44uxYSvFIi0kzQSqO md6FTRCsPYRoWaYK2KT65d1RTxyi5LENbBLeDIgMKX61PlrBfMYWHJIvp+agiV6W TLEZbtGJ24xhvRabSUDvPT+kOGUF6AoKDzR8Xod4/Meiiy7UOP9HgSduqFM0LeVz ifHgU3BBw8wgxpf5OHjnd9TjkZlapuPi5fBbK1ib/7LniGwzqOABqpaYpYT6UhkO AZERJVCU489PJaw7OvuI0d/guInFnrV9IuiWgZJELZB3ztNgQNlQemMhtIVHIYRO BkgmnoML/5BU3VkWLLhhQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1720721101; x= 1720807501; bh=Fq1LlHL5Af2WmagmF5wE56kSRxwIsdDG12xAo8ifsT0=; b=G 9o/CJnrYz3SKZIvwCEwtblxQqR5Cgn9hvith2+ahS+9PeO84t47MxuPCnJAQVjLC fwjTnR3g+QXR2xyuNJHDELRngvdP/+hD2/5wT0GofHOLr84WNXIeJkP3diKxB4QN AavRBezVZiNlnJbDL3axUeFMPoipX3lrDp7UUfc62895g8L6Ibk0ww476p8veL/p gUeUgLD/P/My94G4vy3IHkNTtaoGfET4xiVL1KeknYGT4axpAzkZfX3Z3HddELDi MDh2nnHE+9pAB0377pAb7hThhOjCLUzl2Z5Yb0QxBDVxFXQRx/hcquQ4tfRDv5As Vsg1axC3vpr7Ru264cFLA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrfeeggdduvdduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhgggfestdekredtredttdenucfhrhhomhepmfhlrghu shculfgvnhhsvghnuceoihhtshesihhrrhgvlhgvvhgrnhhtrdgukheqnecuggftrfgrth htvghrnhepjefgieelgfeiveehkeeuveehheekfeevgeeigfehfefgjeejhefffeegudej udegnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepih htshesihhrrhgvlhgvvhgrnhhtrdgukh X-ME-Proxy: Feedback-ID: idc91472f:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Jul 2024 14:04:58 -0400 (EDT) From: Klaus Jensen To: qemu-devel@nongnu.org, Peter Maydell Cc: qemu-block@nongnu.org, Kevin Wolf , =?utf-8?q?Philippe?= =?utf-8?q?_Mathieu-Daud=C3=A9?= , Keith Busch , Klaus Jensen , Jesper Devantier , Stefan Hajnoczi , Hanna Reitz , Fam Zheng , Minwoo Im , Klaus Jensen Subject: [PULL 6/7] hw/nvme: Allocate sec-ctrl-list as a dynamic array Date: Thu, 11 Jul 2024 20:04:43 +0200 Message-ID: <20240711180436.8532-15-its@irrelevant.dk> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240711180436.8532-9-its@irrelevant.dk> References: <20240711180436.8532-9-its@irrelevant.dk> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3151; i=k.jensen@samsung.com; h=from:subject; bh=k8m80l/QsMvWexioTVhcTPvnL8hEaDZKufoo+Ej37ds=; b=owJ4nJvAy8zAxej7cL1hrj/vS8bTakkMaRPkts689SH5mPuNEPuCKWtl3qvvbd9Rd4ltyo38b LcALpu972w7GY1ZGBi5GGTFFFmCNIxXlT6682yRe9oBmEGsTCBTGLg4BWAiF+M5GLaHxHqIVR7X UZYXXVd2SErqf+WvhqNswikZIim6M1MPaNp56rCf/cFuHdQVkD/9UNsGfYWVn1S+v/ppoXX+JO9 WtpCdFf8msZ187H80SfOXZPSRiS+YDGsjvZqMrXM/mDgoxYjxnjvmKyz10Wdvut2J0g1FUp4XSh 9wLFOTS2vfJtFttro5n7erf8qCr22LHjrPmb9ackX/r5rSNcm1IZzR+5Qe8q43erH6ctddA9lrD 0y+R75szHOOmFBdF7zQlEPp7eGTz3baWrncCQnY9uTLwpqWtJ76zvgE031WtQImpnMFEjY7xHre qr20UnLtr5lvK6q063kfbHP++pn3eOX7OToSLUyc3pLsOhEAbzKhiw== X-Developer-Key: i=k.jensen@samsung.com; a=openpgp; fpr=DDCA4D9C9EF931CC3468427263D56FC5E55DA838 Received-SPF: pass client-ip=103.168.172.144; envelope-from=its@irrelevant.dk; helo=fout1-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Minwoo Im To prevent further bumping up the number of maximum VF te support, this patch allocates a dynamic array (NvmeCtrl *)->sec_ctrl_list based on number of VF supported by sriov_max_vfs property. Reviewed-by: Klaus Jensen Signed-off-by: Minwoo Im Signed-off-by: Klaus Jensen --- hw/nvme/ctrl.c | 8 +------- hw/nvme/nvme.h | 5 ++--- hw/nvme/subsys.c | 2 ++ 3 files changed, 5 insertions(+), 10 deletions(-) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index 8a838e5b658b..1e50b57707ba 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -7868,12 +7868,6 @@ static bool nvme_check_params(NvmeCtrl *n, Error **errp) return false; } - if (params->sriov_max_vfs > NVME_MAX_VFS) { - error_setg(errp, "sriov_max_vfs must be between 0 and %d", - NVME_MAX_VFS); - return false; - } - if (params->cmb_size_mb) { error_setg(errp, "CMB is not supported with SR-IOV"); return false; @@ -8485,7 +8479,7 @@ static Property nvme_props[] = { DEFINE_PROP_UINT8("zoned.zasl", NvmeCtrl, params.zasl, 0), DEFINE_PROP_BOOL("zoned.auto_transition", NvmeCtrl, params.auto_transition_zones, true), - DEFINE_PROP_UINT8("sriov_max_vfs", NvmeCtrl, params.sriov_max_vfs, 0), + DEFINE_PROP_UINT16("sriov_max_vfs", NvmeCtrl, params.sriov_max_vfs, 0), DEFINE_PROP_UINT16("sriov_vq_flexible", NvmeCtrl, params.sriov_vq_flexible, 0), DEFINE_PROP_UINT16("sriov_vi_flexible", NvmeCtrl, diff --git a/hw/nvme/nvme.h b/hw/nvme/nvme.h index 9da5343ffe90..180df26ccea0 100644 --- a/hw/nvme/nvme.h +++ b/hw/nvme/nvme.h @@ -26,7 +26,6 @@ #define NVME_MAX_CONTROLLERS 256 #define NVME_MAX_NAMESPACES 256 -#define NVME_MAX_VFS 127 #define NVME_EUI64_DEFAULT ((uint64_t)0x5254000000000000) #define NVME_FDP_MAX_EVENTS 63 #define NVME_FDP_MAXPIDS 128 @@ -533,7 +532,7 @@ typedef struct NvmeParams { bool auto_transition_zones; bool legacy_cmb; bool ioeventfd; - uint8_t sriov_max_vfs; + uint16_t sriov_max_vfs; uint16_t sriov_vq_flexible; uint16_t sriov_vi_flexible; uint8_t sriov_max_vq_per_vf; @@ -615,7 +614,7 @@ typedef struct NvmeCtrl { NvmePriCtrlCap pri_ctrl_cap; uint32_t nr_sec_ctrls; - NvmeSecCtrlEntry sec_ctrl_list[NVME_MAX_VFS]; + NvmeSecCtrlEntry *sec_ctrl_list; struct { uint16_t vqrfap; uint16_t virfap; diff --git a/hw/nvme/subsys.c b/hw/nvme/subsys.c index 561ed04a5317..77deaf2c2c97 100644 --- a/hw/nvme/subsys.c +++ b/hw/nvme/subsys.c @@ -61,6 +61,8 @@ int nvme_subsys_register_ctrl(NvmeCtrl *n, Error **errp) if (pci_is_vf(&n->parent_obj)) { cntlid = le16_to_cpu(sctrl->scid); } else { + n->sec_ctrl_list = g_new0(NvmeSecCtrlEntry, num_vfs); + for (cntlid = 0; cntlid < ARRAY_SIZE(subsys->ctrls); cntlid++) { if (!subsys->ctrls[cntlid]) { break; From patchwork Thu Jul 11 18:04:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Klaus Jensen X-Patchwork-Id: 1959476 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=irrelevant.dk header.i=@irrelevant.dk header.a=rsa-sha256 header.s=fm1 header.b=C/I3b3XQ; dkim=pass (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=aUS9kXSb; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WKjNF0Jzrz1xqr for ; Fri, 12 Jul 2024 04:06:37 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sRyAR-0007tX-A3; Thu, 11 Jul 2024 14:05:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sRyAF-0007Qv-Oj; Thu, 11 Jul 2024 14:05:08 -0400 Received: from fout1-smtp.messagingengine.com ([103.168.172.144]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sRyAD-00049P-Ur; Thu, 11 Jul 2024 14:05:07 -0400 Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailfout.nyi.internal (Postfix) with ESMTP id 062D313882AA; Thu, 11 Jul 2024 14:05:05 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute7.internal (MEProxy); Thu, 11 Jul 2024 14:05:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.dk; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm1; t=1720721105; x= 1720807505; bh=16XE17NJxZw2A8g+Vm9cyATPlkBsPC8Nj8Sy6oWvyTM=; b=C /I3b3XQOzsPrh+pdssubRMr5R+b37lbbXYSfc8xff4qsSNhFCFUDczxZjv1IYASG EulvEoUd1g2IXTRC/adhvUpu5Er/+3iD9tzMs0ZkzV8wGnu+lfOjNZhv3dzgfR8W 6Y3dXU4XAmoO+ynLVrqtyrmRW53p/rMFN1M7CmyFDilt8lyzySbPzl+uOJC+5/UV lzlnuYP/G5XofI8Sfpms4TuIBzEJ7tJobsu/KwWYlHbrJnI6HTXdK/JUOhugfdQl IpG9GZBJN2jRN/cgIeVATD3kcAjwfe9oVzvRlbQJjZt3tnNIqQ0Dzj2DW2p2Yz9d oA/tM+GAm6qtspax9B2tA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1720721105; x= 1720807505; bh=16XE17NJxZw2A8g+Vm9cyATPlkBsPC8Nj8Sy6oWvyTM=; b=a US9kXSbpjRO8pTBW0xtdE9yBUvjr05A7k5I5DnkEIfgrhRKUtUIchMb9HsMpt4u+ TjFCOyVzHrEahiClfQ2k0jetlBcVV/qZfkeFveZYGTqSbX82bIAECSmREHv6Fp/w zfYNkxnKQ1b6wohKy6HUXFe+V3Fypj4O4l5Dog3rFf49XtyiWSQkIwE0k8K9q4SD ABuSYN+l5xccCiINn2U8E7/KHO/TpO+tPGrpT9CJASfNIqFO6lP5WMeFriXQiH2z /0D936HOpxB+i60JMmX3xQx6NHjRkAjYD6kb2zJqta2KkVDlq+BDZHO7mfgGBvD0 SPlmyKchGATCZoPB2SOGQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrfeeggdduvdduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhgggfestdekredtredttdenucfhrhhomhepmfhlrghu shculfgvnhhsvghnuceoihhtshesihhrrhgvlhgvvhgrnhhtrdgukheqnecuggftrfgrth htvghrnhepjefgieelgfeiveehkeeuveehheekfeevgeeigfehfefgjeejhefffeegudej udegnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepih htshesihhrrhgvlhgvvhgrnhhtrdgukh X-ME-Proxy: Feedback-ID: idc91472f:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Jul 2024 14:05:01 -0400 (EDT) From: Klaus Jensen To: qemu-devel@nongnu.org, Peter Maydell Cc: qemu-block@nongnu.org, Kevin Wolf , =?utf-8?q?Philippe?= =?utf-8?q?_Mathieu-Daud=C3=A9?= , Keith Busch , Klaus Jensen , Jesper Devantier , Stefan Hajnoczi , Hanna Reitz , Fam Zheng , Minwoo Im , Klaus Jensen Subject: [PULL 7/7] hw/nvme: Expand VI/VQ resource to uint32 Date: Thu, 11 Jul 2024 20:04:44 +0200 Message-ID: <20240711180436.8532-16-its@irrelevant.dk> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240711180436.8532-9-its@irrelevant.dk> References: <20240711180436.8532-9-its@irrelevant.dk> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2047; i=k.jensen@samsung.com; h=from:subject; bh=z5OhNC9T5HEBL6r91CS4qYLYhcYx5tXrG/ktqbgiBT4=; b=owJ4nAFtAZL+kA0DAAoBTeGvMW1PDekByyZiAGaQHrV7oG7Ny2PuLmk2KP8hxwaafPovrJyX1 qacs58WP4XIj4kBMwQAAQoAHRYhBFIoM6p14tzmokdmwE3hrzFtTw3pBQJmkB61AAoJEE3hrzFt Tw3pgJQH/1DBJAF9KZjsIC6LI04Pfxzoi8SQvuFKSJTXxVU12YuskXaqs/Ak185h8QDwILFqYTx CgZUyXT90ei+Qly8utp46VrOuNZjZlmF8gGRZCp4ps2xiqmUKBvg2TVMXGO079qdq7DkvkjN53O dJLbSYl922QiIINd5bGMhmDwTtdK5+iw4vr7W+KMZpGTm3O2KSC3i1EZQYGTiuO+RJ2oKIvToNT C8Zxu/pwd/9mn88jUunCPNxy3K0C0AEskPX0hcFmVO1zas7X/ZqdZ8MQaRSy2XqyMnzy62MXHmS XyQTNuUg2RcOAA1Baa0vf98Bo9GcWUJ3bxV7e9BkHISRA8BWvorR8aJv X-Developer-Key: i=k.jensen@samsung.com; a=openpgp; fpr=DDCA4D9C9EF931CC3468427263D56FC5E55DA838 Received-SPF: pass client-ip=103.168.172.144; envelope-from=its@irrelevant.dk; helo=fout1-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Minwoo Im VI and VQ resources cover queue resources in each VFs in SR-IOV. Current maximum I/O queue pair size is 0xffff, we can expand them to cover the full number of I/O queue pairs. This patch also fixed Identify Secondary Controller List overflow due to expand of number of secondary controllers. Reviewed-by: Klaus Jensen Signed-off-by: Minwoo Im Signed-off-by: Klaus Jensen --- hw/nvme/ctrl.c | 8 ++++---- hw/nvme/nvme.h | 4 ++-- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index 1e50b57707ba..5b1b0cabcfc3 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -8484,10 +8484,10 @@ static Property nvme_props[] = { params.sriov_vq_flexible, 0), DEFINE_PROP_UINT16("sriov_vi_flexible", NvmeCtrl, params.sriov_vi_flexible, 0), - DEFINE_PROP_UINT8("sriov_max_vi_per_vf", NvmeCtrl, - params.sriov_max_vi_per_vf, 0), - DEFINE_PROP_UINT8("sriov_max_vq_per_vf", NvmeCtrl, - params.sriov_max_vq_per_vf, 0), + DEFINE_PROP_UINT32("sriov_max_vi_per_vf", NvmeCtrl, + params.sriov_max_vi_per_vf, 0), + DEFINE_PROP_UINT32("sriov_max_vq_per_vf", NvmeCtrl, + params.sriov_max_vq_per_vf, 0), DEFINE_PROP_BOOL("msix-exclusive-bar", NvmeCtrl, params.msix_exclusive_bar, false), DEFINE_PROP_UINT16("mqes", NvmeCtrl, params.mqes, 0x7ff), diff --git a/hw/nvme/nvme.h b/hw/nvme/nvme.h index 180df26ccea0..781985754d0d 100644 --- a/hw/nvme/nvme.h +++ b/hw/nvme/nvme.h @@ -535,8 +535,8 @@ typedef struct NvmeParams { uint16_t sriov_max_vfs; uint16_t sriov_vq_flexible; uint16_t sriov_vi_flexible; - uint8_t sriov_max_vq_per_vf; - uint8_t sriov_max_vi_per_vf; + uint32_t sriov_max_vq_per_vf; + uint32_t sriov_max_vi_per_vf; bool msix_exclusive_bar; } NvmeParams;