From patchwork Sat Nov 6 12:10:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kai-Heng Feng X-Patchwork-Id: 1551662 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=NyAdd6ix; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Hmbnk4lc3z9t0T for ; Sat, 6 Nov 2021 23:11:21 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1mjKXO-0005c6-5D; Sat, 06 Nov 2021 12:11:10 +0000 Received: from smtp-relay-canonical-0.internal ([10.131.114.83] helo=smtp-relay-canonical-0.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1mjKXJ-0005bJ-Nq for kernel-team@lists.ubuntu.com; Sat, 06 Nov 2021 12:11:05 +0000 Received: from HP-EliteBook-840-G7.. (1-171-213-210.dynamic-ip.hinet.net [1.171.213.210]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-0.canonical.com (Postfix) with ESMTPSA id B78F93F1B4 for ; Sat, 6 Nov 2021 12:11:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1636200665; bh=rgrDnETUPntGcTbIj1P/H9a7YeeQJv1ngaFyV/svoZI=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NyAdd6ixcNN9qTxSuHFtIJ6Xktfa1CTdoVQberHgXVn8RiTVLVKSpxUQyVX9gYZ+q s8771HgwtMxlyD1NOffbkhftkXp/aNcaGS91gaWcEb/vI3xBAxP0rfYbYQlNaCFTNv Aj4FQZu6SJNB1SDfZEp1HphexxaVVj1ZrW/ZQmrnLvSePXe6dmRhiTQkksDLB4r6+i M5fj80Zl396zMWLpXPuSUN0kD8MSl9oj/r1Z0naLwuSFoYxnkDyJ4NsU+mq+dB0L97 3ISVtNcAxngP4wEPv2++So/6XsFcYVN9lQuHWLoXWKdNhEJbEybhmqe7jHmpd5wqLN B69iYrukSgAig== From: Kai-Heng Feng To: kernel-team@lists.ubuntu.com Subject: [H/I/OEM-5.10/OEM-5.13/OEM-5.14] [PATCH 1/4] nvme-pci: use attribute group for cmb sysfs Date: Sat, 6 Nov 2021 20:10:55 +0800 Message-Id: <20211106121058.1843173-2-kai.heng.feng@canonical.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211106121058.1843173-1-kai.heng.feng@canonical.com> References: <20211106121058.1843173-1-kai.heng.feng@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Keith Busch BugLink: https://bugs.launchpad.net/bugs/1950042 Appending sysfs files to the controller kobject is a bit clunky and becomes a maintenance problem as more attributes are added. The attribute group infrastructure handles this better, so use that. Signed-off-by: Keith Busch Signed-off-by: Christoph Hellwig (cherry picked from commit 0521905e859fd1a07949cb18efb20cdd4aab3b20) Signed-off-by: Kai-Heng Feng --- drivers/nvme/host/pci.c | 72 ++++++++++++++++++++++++++--------------- 1 file changed, 46 insertions(+), 26 deletions(-) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 1f41619018c16..724be85141b75 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -153,6 +153,8 @@ struct nvme_dev { unsigned int nr_allocated_queues; unsigned int nr_write_queues; unsigned int nr_poll_queues; + + bool attrs_added; }; static int io_queue_depth_set(const char *val, const struct kernel_param *kp) @@ -1808,17 +1810,6 @@ static int nvme_create_io_queues(struct nvme_dev *dev) return ret >= 0 ? 0 : ret; } -static ssize_t nvme_cmb_show(struct device *dev, - struct device_attribute *attr, - char *buf) -{ - struct nvme_dev *ndev = to_nvme_dev(dev_get_drvdata(dev)); - - return scnprintf(buf, PAGE_SIZE, "cmbloc : x%08x\ncmbsz : x%08x\n", - ndev->cmbloc, ndev->cmbsz); -} -static DEVICE_ATTR(cmb, S_IRUGO, nvme_cmb_show, NULL); - static u64 nvme_cmb_size_unit(struct nvme_dev *dev) { u8 szu = (dev->cmbsz >> NVME_CMBSZ_SZU_SHIFT) & NVME_CMBSZ_SZU_MASK; @@ -1887,20 +1878,6 @@ static void nvme_map_cmb(struct nvme_dev *dev) if ((dev->cmbsz & (NVME_CMBSZ_WDS | NVME_CMBSZ_RDS)) == (NVME_CMBSZ_WDS | NVME_CMBSZ_RDS)) pci_p2pmem_publish(pdev, true); - - if (sysfs_add_file_to_group(&dev->ctrl.device->kobj, - &dev_attr_cmb.attr, NULL)) - dev_warn(dev->ctrl.device, - "failed to add sysfs attribute for CMB\n"); -} - -static inline void nvme_release_cmb(struct nvme_dev *dev) -{ - if (dev->cmb_size) { - sysfs_remove_file_from_group(&dev->ctrl.device->kobj, - &dev_attr_cmb.attr, NULL); - dev->cmb_size = 0; - } } static int nvme_set_host_mem(struct nvme_dev *dev, u32 bits) @@ -2080,6 +2057,38 @@ static int nvme_setup_host_mem(struct nvme_dev *dev) return ret; } +static ssize_t cmb_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct nvme_dev *ndev = to_nvme_dev(dev_get_drvdata(dev)); + + return sysfs_emit(buf, "cmbloc : x%08x\ncmbsz : x%08x\n", + ndev->cmbloc, ndev->cmbsz); +} +static DEVICE_ATTR_RO(cmb); + +static umode_t nvme_pci_attrs_are_visible(struct kobject *kobj, + struct attribute *a, int n) +{ + struct nvme_ctrl *ctrl = + dev_get_drvdata(container_of(kobj, struct device, kobj)); + struct nvme_dev *dev = to_nvme_dev(ctrl); + + if (a == &dev_attr_cmb.attr && !dev->cmbsz) + return 0; + return a->mode; +} + +static struct attribute *nvme_pci_attrs[] = { + &dev_attr_cmb.attr, + NULL, +}; + +static const struct attribute_group nvme_pci_attr_group = { + .attrs = nvme_pci_attrs, + .is_visible = nvme_pci_attrs_are_visible, +}; + /* * nirqs is the number of interrupts available for write and read * queues. The core already reserved an interrupt for the admin queue. @@ -2751,6 +2760,10 @@ static void nvme_reset_work(struct work_struct *work) goto out; } + if (!dev->attrs_added && !sysfs_create_group(&dev->ctrl.device->kobj, + &nvme_pci_attr_group)) + dev->attrs_added = true; + nvme_start_ctrl(&dev->ctrl); return; @@ -2999,6 +3012,13 @@ static void nvme_shutdown(struct pci_dev *pdev) nvme_disable_prepare_reset(dev, true); } +static void nvme_remove_attrs(struct nvme_dev *dev) +{ + if (dev->attrs_added) + sysfs_remove_group(&dev->ctrl.device->kobj, + &nvme_pci_attr_group); +} + /* * The driver's remove may be called on a device in a partially initialized * state. This function must not have any dependencies on the device state in @@ -3020,7 +3040,7 @@ static void nvme_remove(struct pci_dev *pdev) nvme_stop_ctrl(&dev->ctrl); nvme_remove_namespaces(&dev->ctrl); nvme_dev_disable(dev, true); - nvme_release_cmb(dev); + nvme_remove_attrs(dev); nvme_free_host_mem(dev); nvme_dev_remove_admin(dev); nvme_free_queues(dev, 0); From patchwork Sat Nov 6 12:10:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kai-Heng Feng X-Patchwork-Id: 1551659 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=fhqgBcww; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Hmbnk4RC8z9ssD for ; Sat, 6 Nov 2021 23:11:21 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1mjKXQ-0005co-9x; Sat, 06 Nov 2021 12:11:12 +0000 Received: from smtp-relay-canonical-0.internal ([10.131.114.83] helo=smtp-relay-canonical-0.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1mjKXL-0005bR-Aj for kernel-team@lists.ubuntu.com; Sat, 06 Nov 2021 12:11:07 +0000 Received: from HP-EliteBook-840-G7.. (1-171-213-210.dynamic-ip.hinet.net [1.171.213.210]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-0.canonical.com (Postfix) with ESMTPSA id 3C8BD3F1B4 for ; Sat, 6 Nov 2021 12:11:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1636200667; bh=bMtGZeo4LBT81hXKNue9XLMkzaKrPDo9sjsUHOXCCWI=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=fhqgBcwwrNBY/IvKRvS8uJKsvvL2JZ5xIdo/Zhf5lZccF8XlM9se5MxTC6ARSgU4x 99Y144Y+eNMta3rzPgAqnbD5ghb4+tTe4kCdQ1vhfp3oni7mDmsFE9FSbvba0L2Rnn 5CihKyLyzgO7y5mu0b3QhgY5LwQC+jBz2nTn4WpNoGh2qbgIa3/2xyXweC9RRB+IbX JQV/AtU724xJ3N1vdB3rd9t+QeNqoyn/a4XQgzc9ND9ETzo61KTD27Qc1PjVqdb3ju EdKCEEJD1DKsc/UzUQr9ikggSQpTgPJOBG5ZCy30ZKBgj9eFTjCKaCx9n8XlLTGRoD y4D6bHNOI6GCQ== From: Kai-Heng Feng To: kernel-team@lists.ubuntu.com Subject: [H/I/OEM-5.10/OEM-5.13/OEM-5.14] [PATCH 2/4] nvme-pci: cmb sysfs: one file, one value Date: Sat, 6 Nov 2021 20:10:56 +0800 Message-Id: <20211106121058.1843173-3-kai.heng.feng@canonical.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211106121058.1843173-1-kai.heng.feng@canonical.com> References: <20211106121058.1843173-1-kai.heng.feng@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Keith Busch BugLink: https://bugs.launchpad.net/bugs/1950042 An attribute should only be exporting one value as recommended in Documentation/filesystems/sysfs.rst. Implement CMB attributes this way. The old attribute will remain for backward compatibility. Signed-off-by: Keith Busch Signed-off-by: Christoph Hellwig (cherry picked from commit 1751e97aa940656b5de0e620f02cf193a275e014) Signed-off-by: Kai-Heng Feng --- drivers/nvme/host/pci.c | 28 ++++++++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 724be85141b75..3d734e60f2146 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -2067,6 +2067,24 @@ static ssize_t cmb_show(struct device *dev, struct device_attribute *attr, } static DEVICE_ATTR_RO(cmb); +static ssize_t cmbloc_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct nvme_dev *ndev = to_nvme_dev(dev_get_drvdata(dev)); + + return sysfs_emit(buf, "%u\n", ndev->cmbloc); +} +static DEVICE_ATTR_RO(cmbloc); + +static ssize_t cmbsz_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct nvme_dev *ndev = to_nvme_dev(dev_get_drvdata(dev)); + + return sysfs_emit(buf, "%u\n", ndev->cmbsz); +} +static DEVICE_ATTR_RO(cmbsz); + static umode_t nvme_pci_attrs_are_visible(struct kobject *kobj, struct attribute *a, int n) { @@ -2074,13 +2092,19 @@ static umode_t nvme_pci_attrs_are_visible(struct kobject *kobj, dev_get_drvdata(container_of(kobj, struct device, kobj)); struct nvme_dev *dev = to_nvme_dev(ctrl); - if (a == &dev_attr_cmb.attr && !dev->cmbsz) - return 0; + if (a == &dev_attr_cmb.attr || + a == &dev_attr_cmbloc.attr || + a == &dev_attr_cmbsz.attr) { + if (!dev->cmbsz) + return 0; + } return a->mode; } static struct attribute *nvme_pci_attrs[] = { &dev_attr_cmb.attr, + &dev_attr_cmbloc.attr, + &dev_attr_cmbsz.attr, NULL, }; From patchwork Sat Nov 6 12:10:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kai-Heng Feng X-Patchwork-Id: 1551661 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=AJabNxfF; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Hmbnk4dF6z9t0G for ; Sat, 6 Nov 2021 23:11:21 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1mjKXR-0005df-Fe; Sat, 06 Nov 2021 12:11:13 +0000 Received: from smtp-relay-canonical-0.internal ([10.131.114.83] helo=smtp-relay-canonical-0.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1mjKXN-0005be-1V for kernel-team@lists.ubuntu.com; Sat, 06 Nov 2021 12:11:09 +0000 Received: from HP-EliteBook-840-G7.. (1-171-213-210.dynamic-ip.hinet.net [1.171.213.210]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-0.canonical.com (Postfix) with ESMTPSA id BF9AB3F1B4 for ; Sat, 6 Nov 2021 12:11:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1636200668; bh=MZ0/EqPdZExXKdCUs+Myu1RKpK0wzmwWxTHVZt6+5fM=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AJabNxfFSdbgROzmBhpAq2PiTILx3ENLyo6snJIKSiH+/8g9/Wr9TdxpvJb93vdDk DIKEp0PPgpTpzqveb/zhG696h4QX5TuOrwk6e67QRZ7Bh3NOU/itEAkfUY4cbO9zRG TPWsmkN92QSx4Blhvv0UXCAboQBV8yhfa0LL/zsGFMXGQhyELnE8nk1hgu8eN0hWLC Sk8WhJ7QyNyPC9d9m9E71RLkaYVdrouw3sUe07Wb94OURAlzQb6KaB6kLC51JD7tBP o0gbioR5LB7dr+q6GxxMBaH6bp8DIJi7Iq1e/B088vzLxXgBc/uhnjPGyHdJY7V5gR Hx+9xBYwi0R3g== From: Kai-Heng Feng To: kernel-team@lists.ubuntu.com Subject: [H/I/OEM-5.10/OEM-5.13/OEM-5.14] [PATCH 3/4] nvme-pci: disable hmb on idle suspend Date: Sat, 6 Nov 2021 20:10:57 +0800 Message-Id: <20211106121058.1843173-4-kai.heng.feng@canonical.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211106121058.1843173-1-kai.heng.feng@canonical.com> References: <20211106121058.1843173-1-kai.heng.feng@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Keith Busch BugLink: https://bugs.launchpad.net/bugs/1950042 An idle suspend may or may not disable host memory access from devices placed in low power mode. Either way, it should always be safe to disable the host memory buffer prior to entering the low power mode, and this should also always be faster than a full device shutdown. Signed-off-by: Keith Busch Reviewed-by: Sagi Grimberg Signed-off-by: Christoph Hellwig (cherry picked from commit e5ad96f388b765fe6b52f64f37e910c0ba4f3de7) Signed-off-by: Kai-Heng Feng --- drivers/nvme/host/pci.c | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7 deletions(-) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 3d734e60f2146..ee7ef813fb18a 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -3091,8 +3091,13 @@ static int nvme_resume(struct device *dev) if (ndev->last_ps == U32_MAX || nvme_set_power_state(ctrl, ndev->last_ps) != 0) - return nvme_try_sched_reset(&ndev->ctrl); + goto reset; + if (ctrl->hmpre && nvme_setup_host_mem(ndev)) + goto reset; + return 0; +reset: + return nvme_try_sched_reset(ctrl); } static int nvme_suspend(struct device *dev) @@ -3116,15 +3121,9 @@ static int nvme_suspend(struct device *dev) * the PCI bus layer to put it into D3 in order to take the PCIe link * down, so as to allow the platform to achieve its minimum low-power * state (which may not be possible if the link is up). - * - * If a host memory buffer is enabled, shut down the device as the NVMe - * specification allows the device to access the host memory buffer in - * host DRAM from all power states, but hosts will fail access to DRAM - * during S3. */ if (pm_suspend_via_firmware() || !ctrl->npss || !pcie_aspm_enabled(pdev) || - ndev->nr_host_mem_descs || (ndev->ctrl.quirks & NVME_QUIRK_SIMPLE_SUSPEND)) return nvme_disable_prepare_reset(ndev, true); @@ -3135,6 +3134,17 @@ static int nvme_suspend(struct device *dev) if (ctrl->state != NVME_CTRL_LIVE) goto unfreeze; + /* + * Host memory access may not be successful in a system suspend state, + * but the specification allows the controller to access memory in a + * non-operational power state. + */ + if (ndev->hmb) { + ret = nvme_set_host_mem(ndev, 0); + if (ret < 0) + goto unfreeze; + } + ret = nvme_get_power_state(ctrl, &ndev->last_ps); if (ret < 0) goto unfreeze; From patchwork Sat Nov 6 12:10:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kai-Heng Feng X-Patchwork-Id: 1551660 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=HU8F51nV; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Hmbnk5ndyz9t0k for ; Sat, 6 Nov 2021 23:11:21 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1mjKXR-0005e3-Kn; Sat, 06 Nov 2021 12:11:13 +0000 Received: from smtp-relay-canonical-0.internal ([10.131.114.83] helo=smtp-relay-canonical-0.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1mjKXO-0005cG-CQ for kernel-team@lists.ubuntu.com; Sat, 06 Nov 2021 12:11:10 +0000 Received: from HP-EliteBook-840-G7.. (1-171-213-210.dynamic-ip.hinet.net [1.171.213.210]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-0.canonical.com (Postfix) with ESMTPSA id 4E58F3F1B4 for ; Sat, 6 Nov 2021 12:11:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1636200670; bh=ZO2d3x3s/d0nedXo6oaoGSss61t+k/acVjhgS8xuzeI=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=HU8F51nV1Vxaluh+9B8E1ynDr/9prDqtXkad8tCkUee8a8//S3mhVpa5sVehR5FP4 aLJk409dkIIpF1X0As6T5Shkd3Qylzzk/R38zcsCb3bTNzEIW19uF8wgvSRfpVh6Zw MtWeI+IpvRMeiTfrXWay8bAoDbgXuQ3swBKmg59MR3asNRnDgSlHWyjZOYG3OwYsLi pVh4bdubzQwXfoolnG2Hj5fRPuvuXYr3B0JE1uooIbXy10HcCSDqwLVu4vNNt3IxTD pidBytS+C0zT4GBPGuAwSxAiUjX/pKZMHsjVPSU2MYYxSAPsxiFBPGtnamrFSQ93yK mtw/XfzR0LZkQ== From: Kai-Heng Feng To: kernel-team@lists.ubuntu.com Subject: [H/I/OEM-5.10/OEM-5.13/OEM-5.14] [PATCH 4/4] nvme: allow user toggling hmb usage Date: Sat, 6 Nov 2021 20:10:58 +0800 Message-Id: <20211106121058.1843173-5-kai.heng.feng@canonical.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211106121058.1843173-1-kai.heng.feng@canonical.com> References: <20211106121058.1843173-1-kai.heng.feng@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Keith Busch BugLink: https://bugs.launchpad.net/bugs/1950042 The NVMe host memory buffer may consume a non-negligable amount of memory. Controllers are required to function without the host memory buffer enabled, but with possibly degraded performance. Export a sysfs property to toggle this feature on a per-device granularity so users may choose to reclaim memory at the expense of storage performance. Signed-off-by: Keith Busch Reviewed-by: Sagi Grimberg Signed-off-by: Christoph Hellwig (cherry picked from commit a5df5e79c43c84d9fb88f56b707c5ff52b27ccca) Signed-off-by: Kai-Heng Feng --- drivers/nvme/host/pci.c | 45 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 44 insertions(+), 1 deletion(-) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index ee7ef813fb18a..f29d892eae5f6 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -135,6 +135,7 @@ struct nvme_dev { u32 cmbloc; struct nvme_ctrl ctrl; u32 last_ps; + bool hmb; mempool_t *iod_mempool; @@ -1900,7 +1901,9 @@ static int nvme_set_host_mem(struct nvme_dev *dev, u32 bits) dev_warn(dev->ctrl.device, "failed to set host mem (err %d, flags %#x).\n", ret, bits); - } + } else + dev->hmb = bits & NVME_HOST_MEM_ENABLE; + return ret; } @@ -2085,6 +2088,42 @@ static ssize_t cmbsz_show(struct device *dev, struct device_attribute *attr, } static DEVICE_ATTR_RO(cmbsz); +static ssize_t hmb_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct nvme_dev *ndev = to_nvme_dev(dev_get_drvdata(dev)); + + return sysfs_emit(buf, "%d\n", ndev->hmb); +} + +static ssize_t hmb_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct nvme_dev *ndev = to_nvme_dev(dev_get_drvdata(dev)); + bool new; + int ret; + + if (strtobool(buf, &new) < 0) + return -EINVAL; + + if (new == ndev->hmb) + return count; + + if (new) { + ret = nvme_setup_host_mem(ndev); + } else { + ret = nvme_set_host_mem(ndev, 0); + if (!ret) + nvme_free_host_mem(ndev); + } + + if (ret < 0) + return ret; + + return count; +} +static DEVICE_ATTR_RW(hmb); + static umode_t nvme_pci_attrs_are_visible(struct kobject *kobj, struct attribute *a, int n) { @@ -2098,6 +2137,9 @@ static umode_t nvme_pci_attrs_are_visible(struct kobject *kobj, if (!dev->cmbsz) return 0; } + if (a == &dev_attr_hmb.attr && !ctrl->hmpre) + return 0; + return a->mode; } @@ -2105,6 +2147,7 @@ static struct attribute *nvme_pci_attrs[] = { &dev_attr_cmb.attr, &dev_attr_cmbloc.attr, &dev_attr_cmbsz.attr, + &dev_attr_hmb.attr, NULL, };