From patchwork Tue Sep 6 22:23:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 1675050 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=KoWQN+Dt; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4MMg0L1JMTz1ynw for ; Wed, 7 Sep 2022 08:24:02 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229844AbiIFWYA (ORCPT ); Tue, 6 Sep 2022 18:24:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48130 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229810AbiIFWX7 (ORCPT ); Tue, 6 Sep 2022 18:23:59 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 05851A9C27; Tue, 6 Sep 2022 15:23:59 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A2C7461595; Tue, 6 Sep 2022 22:23:58 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C7980C433C1; Tue, 6 Sep 2022 22:23:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1662503038; bh=RT3RQzwcZmsdKMDfRVB08SNXYiMqaAszRBdVwIagBPc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KoWQN+DtFhGgaXGAEecu3YKNsWQGG8N0v6iE7sATpBchhXm81rsAlv98qJNIh9Wpd cmA+qKlBeGudRRi5UO7LtHjB7er9Ovu6uwvHXrMRCLp6SYZlUcyRoGjQ8skCEsjALw tQbrfA/dG5mD9GFkmLvzmats17r8MvOPMTIcrKAVUCHNoOTRGBkbnfHqwx2e4rlPVI caS99Ak3oYEFWQbHW1M7RUOaoi+zUItiul2iXoMdgpAOyUqIY2qF3IWBBzWbTbB32P sZZeciu0P1DXzhqK9+4CPNqX/ByvEtKyqCEAE2Fe7EKYaZ7S9hn/5IqreTJ/IPyJB9 10Uu0OAAIPx0g== From: Bjorn Helgaas To: Kai-Heng Feng , Rajvi Jingar , "Rafael J . Wysocki" Cc: Koba Ko , Mika Westerberg , "David E . Box" , Sathyanarayanan Kuppuswamy , linux-pci@vger.kernel.org, linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Bjorn Helgaas Subject: [PATCH v3 01/10] PCI/PTM: Preserve PTM Root Select Date: Tue, 6 Sep 2022 17:23:42 -0500 Message-Id: <20220906222351.64760-2-helgaas@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906222351.64760-1-helgaas@kernel.org> References: <20220906222351.64760-1-helgaas@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Bjorn Helgaas When disabling PTM, there's no need to clear the Root Select bit. We disable PTM during suspend, and we want to re-enable it during resume. Clearing Root Select here makes re-enabling more complicated. Per PCIe r6.0, sec 7.9.15.3, "When set, if the PTM Enable bit is also Set, this Time Source is the PTM Root," so if PTM Enable is cleared, the value of Root Select should be irrelevant. Preserve Root Select to simplify re-enabling PTM. Signed-off-by: Bjorn Helgaas Cc: David E. Box --- drivers/pci/pcie/ptm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/pci/pcie/ptm.c b/drivers/pci/pcie/ptm.c index 368a254e3124..b6a417247ce3 100644 --- a/drivers/pci/pcie/ptm.c +++ b/drivers/pci/pcie/ptm.c @@ -42,7 +42,7 @@ void pci_disable_ptm(struct pci_dev *dev) return; pci_read_config_word(dev, ptm + PCI_PTM_CTRL, &ctrl); - ctrl &= ~(PCI_PTM_CTRL_ENABLE | PCI_PTM_CTRL_ROOT); + ctrl &= ~PCI_PTM_CTRL_ENABLE; pci_write_config_word(dev, ptm + PCI_PTM_CTRL, ctrl); } From patchwork Tue Sep 6 22:23:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 1675051 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=ur8wkfXm; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4MMg0W2px3z1yhP for ; Wed, 7 Sep 2022 08:24:11 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229976AbiIFWYJ (ORCPT ); Tue, 6 Sep 2022 18:24:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48274 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229932AbiIFWYE (ORCPT ); Tue, 6 Sep 2022 18:24:04 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D8E3EAA346; Tue, 6 Sep 2022 15:24:02 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 1DB23B81A6B; Tue, 6 Sep 2022 22:24:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9FF03C433C1; Tue, 6 Sep 2022 22:23:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1662503039; bh=Dl3nM+fYYrCbUQ97Bp2WKBRN00kujXBqOz/puvMrmQo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ur8wkfXmer+PDTCjml9BVH31hJucY0Q+URaOxua80LJow1vpQ6GvrSJ184lLeHWdJ pLfOam5pC9ZbBTfRRWofQMEdEmPD7lysx3CWGNJtTLXxFUPOBUiwBCh0RRo2P2nHSZ 6ljLmbQhvqpIDrtYm6yqdeQ/qBeIL5Np8/bdh7Jvc7mEP6F8FHipFD7BEc7A67HqNR FWQVYlrbiy31e4l1lGy1JKeHJd5ZtJVk4gGHNoyumDvUKlQ3t+rH7iA0+3VfoWhsCx 9C6dDPMUga2lxP0bC1ipG/tDdZrh+j2KRO116lpeNl/1vub+pO3VOx5hc6iR3QFDPh +MEJB7bEIenXg== From: Bjorn Helgaas To: Kai-Heng Feng , Rajvi Jingar , "Rafael J . Wysocki" Cc: Koba Ko , Mika Westerberg , "David E . Box" , Sathyanarayanan Kuppuswamy , linux-pci@vger.kernel.org, linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Bjorn Helgaas Subject: [PATCH v3 02/10] PCI/PTM: Cache PTM Capability offset Date: Tue, 6 Sep 2022 17:23:43 -0500 Message-Id: <20220906222351.64760-3-helgaas@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906222351.64760-1-helgaas@kernel.org> References: <20220906222351.64760-1-helgaas@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Bjorn Helgaas Cache the PTM Capability offset instead of searching for it every time we enable/disable PTM or save/restore PTM state. Signed-off-by: Bjorn Helgaas --- drivers/pci/pcie/ptm.c | 41 +++++++++++++++++------------------------ include/linux/pci.h | 1 + 2 files changed, 18 insertions(+), 24 deletions(-) diff --git a/drivers/pci/pcie/ptm.c b/drivers/pci/pcie/ptm.c index b6a417247ce3..6ac7ff48be57 100644 --- a/drivers/pci/pcie/ptm.c +++ b/drivers/pci/pcie/ptm.c @@ -31,13 +31,9 @@ static void pci_ptm_info(struct pci_dev *dev) void pci_disable_ptm(struct pci_dev *dev) { - int ptm; + int ptm = dev->ptm_cap; u16 ctrl; - if (!pci_is_pcie(dev)) - return; - - ptm = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_PTM); if (!ptm) return; @@ -48,14 +44,10 @@ void pci_disable_ptm(struct pci_dev *dev) void pci_save_ptm_state(struct pci_dev *dev) { - int ptm; + int ptm = dev->ptm_cap; struct pci_cap_saved_state *save_state; u16 *cap; - if (!pci_is_pcie(dev)) - return; - - ptm = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_PTM); if (!ptm) return; @@ -69,16 +61,15 @@ void pci_save_ptm_state(struct pci_dev *dev) void pci_restore_ptm_state(struct pci_dev *dev) { + int ptm = dev->ptm_cap; struct pci_cap_saved_state *save_state; - int ptm; u16 *cap; - if (!pci_is_pcie(dev)) + if (!ptm) return; save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_PTM); - ptm = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_PTM); - if (!save_state || !ptm) + if (!save_state) return; cap = (u16 *)&save_state->cap.data[0]; @@ -87,7 +78,7 @@ void pci_restore_ptm_state(struct pci_dev *dev) void pci_ptm_init(struct pci_dev *dev) { - int pos; + int ptm; u32 cap, ctrl; u8 local_clock; struct pci_dev *ups; @@ -117,13 +108,14 @@ void pci_ptm_init(struct pci_dev *dev) return; } - pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_PTM); - if (!pos) + ptm = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_PTM); + if (!ptm) return; + dev->ptm_cap = ptm; pci_add_ext_cap_save_buffer(dev, PCI_EXT_CAP_ID_PTM, sizeof(u16)); - pci_read_config_dword(dev, pos + PCI_PTM_CAP, &cap); + pci_read_config_dword(dev, ptm + PCI_PTM_CAP, &cap); local_clock = (cap & PCI_PTM_GRANULARITY_MASK) >> 8; /* @@ -148,7 +140,7 @@ void pci_ptm_init(struct pci_dev *dev) } ctrl |= dev->ptm_granularity << 8; - pci_write_config_dword(dev, pos + PCI_PTM_CTRL, ctrl); + pci_write_config_dword(dev, ptm + PCI_PTM_CTRL, ctrl); dev->ptm_enabled = 1; pci_ptm_info(dev); @@ -156,18 +148,19 @@ void pci_ptm_init(struct pci_dev *dev) int pci_enable_ptm(struct pci_dev *dev, u8 *granularity) { - int pos; + int ptm; u32 cap, ctrl; struct pci_dev *ups; if (!pci_is_pcie(dev)) return -EINVAL; - pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_PTM); - if (!pos) + ptm = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_PTM); + if (!ptm) return -EINVAL; - pci_read_config_dword(dev, pos + PCI_PTM_CAP, &cap); + dev->ptm_cap = ptm; + pci_read_config_dword(dev, ptm + PCI_PTM_CAP, &cap); if (!(cap & PCI_PTM_CAP_REQ)) return -EINVAL; @@ -192,7 +185,7 @@ int pci_enable_ptm(struct pci_dev *dev, u8 *granularity) ctrl = PCI_PTM_CTRL_ENABLE; ctrl |= dev->ptm_granularity << 8; - pci_write_config_dword(dev, pos + PCI_PTM_CTRL, ctrl); + pci_write_config_dword(dev, ptm + PCI_PTM_CTRL, ctrl); dev->ptm_enabled = 1; pci_ptm_info(dev); diff --git a/include/linux/pci.h b/include/linux/pci.h index 060af91bafcd..54be939023a3 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -475,6 +475,7 @@ struct pci_dev { unsigned int broken_cmd_compl:1; /* No compl for some cmds */ #endif #ifdef CONFIG_PCIE_PTM + u16 ptm_cap; /* PTM Capability */ unsigned int ptm_root:1; unsigned int ptm_enabled:1; u8 ptm_granularity; From patchwork Tue Sep 6 22:23:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 1675052 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=UJNy3ohI; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4MMg0X19Lyz1yhP for ; Wed, 7 Sep 2022 08:24:12 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229910AbiIFWYK (ORCPT ); Tue, 6 Sep 2022 18:24:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48452 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229886AbiIFWYI (ORCPT ); Tue, 6 Sep 2022 18:24:08 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6320CAA37A; Tue, 6 Sep 2022 15:24:04 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id DA27DB81AAC; Tue, 6 Sep 2022 22:24:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5E235C43470; Tue, 6 Sep 2022 22:24:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1662503041; bh=9D8mgVY5gHcE6/P2q6z5Fluh/zLBUAmVl7NPUlk2Lew=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UJNy3ohIHRZmJ7ExidmVPQszAoQxTHEG0ha6tBNlcF6B61WQ1ytjO30zoGmWeQg8v sliC9dTUVQCvcMju7iHIapWq3TD0CunwOeCCZtCdwmfI5SWKcblCxwl0iKGCRtRA2o rtqLmbxjE4V4z2XmDfBw2kYTAUiaQX7JtOSRkR1zpEVDRo+rz90N+e97fgJBCymMs2 /m7Df03YMEV7HSqkoE8Ilgwq+6v163+Rmkfws3Q54X81M6JBzdgQb0aISzkof5/xRC 0yYya7s7J5uOwtpDiFlAqRJZA+Mmhi1z0nTXUuIFIm44jlIM3HumNDB8LVDWNxRqRw zSiDW4+4ytdwA== From: Bjorn Helgaas To: Kai-Heng Feng , Rajvi Jingar , "Rafael J . Wysocki" Cc: Koba Ko , Mika Westerberg , "David E . Box" , Sathyanarayanan Kuppuswamy , linux-pci@vger.kernel.org, linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Bjorn Helgaas Subject: [PATCH v3 03/10] PCI/PTM: Add pci_upstream_ptm() helper Date: Tue, 6 Sep 2022 17:23:44 -0500 Message-Id: <20220906222351.64760-4-helgaas@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906222351.64760-1-helgaas@kernel.org> References: <20220906222351.64760-1-helgaas@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Bjorn Helgaas PTM requires an unbroken path of PTM-supporting devices between the PTM Root and the ultimate PTM Requester, but if a Switch supports PTM, only the Upstream Port can have a PTM Capability; the Downstream Ports do not. Previously we copied the PTM configuration from the Switch Upstream Port to the Downstream Ports so dev->ptm_enabled for any device implied that all the upstream devices support PTM. Instead of making it look like Downstream Ports have their own PTM config, add pci_upstream_ptm(), which returns the upstream device that has a PTM Capability (either a Root Port or a Switch Upstream Port). Signed-off-by: Bjorn Helgaas --- drivers/pci/pcie/ptm.c | 39 +++++++++++++++++++++++++-------------- 1 file changed, 25 insertions(+), 14 deletions(-) diff --git a/drivers/pci/pcie/ptm.c b/drivers/pci/pcie/ptm.c index 6ac7ff48be57..8729c3e452ee 100644 --- a/drivers/pci/pcie/ptm.c +++ b/drivers/pci/pcie/ptm.c @@ -76,6 +76,29 @@ void pci_restore_ptm_state(struct pci_dev *dev) pci_write_config_word(dev, ptm + PCI_PTM_CTRL, *cap); } +/* + * If the next upstream device supports PTM, return it; otherwise return + * NULL. PTM Messages are local, so both link partners must support it. + */ +static struct pci_dev *pci_upstream_ptm(struct pci_dev *dev) +{ + struct pci_dev *ups = pci_upstream_bridge(dev); + + /* + * Switch Downstream Ports are not permitted to have a PTM + * capability; their PTM behavior is controlled by the Upstream + * Port (PCIe r5.0, sec 7.9.16), so if the upstream bridge is a + * Switch Downstream Port, look up one more level. + */ + if (ups && pci_pcie_type(ups) == PCI_EXP_TYPE_DOWNSTREAM) + ups = pci_upstream_bridge(ups); + + if (ups && ups->ptm_cap) + return ups; + + return NULL; +} + void pci_ptm_init(struct pci_dev *dev) { int ptm; @@ -95,19 +118,6 @@ void pci_ptm_init(struct pci_dev *dev) pci_pcie_type(dev) == PCI_EXP_TYPE_RC_END)) return; - /* - * Switch Downstream Ports are not permitted to have a PTM - * capability; their PTM behavior is controlled by the Upstream - * Port (PCIe r5.0, sec 7.9.16). - */ - ups = pci_upstream_bridge(dev); - if (pci_pcie_type(dev) == PCI_EXP_TYPE_DOWNSTREAM && - ups && ups->ptm_enabled) { - dev->ptm_granularity = ups->ptm_granularity; - dev->ptm_enabled = 1; - return; - } - ptm = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_PTM); if (!ptm) return; @@ -124,6 +134,7 @@ void pci_ptm_init(struct pci_dev *dev) * the spec recommendation (PCIe r3.1, sec 7.32.3), select the * furthest upstream Time Source as the PTM Root. */ + ups = pci_upstream_ptm(dev); if (ups && ups->ptm_enabled) { ctrl = PCI_PTM_CTRL_ENABLE; if (ups->ptm_granularity == 0) @@ -173,7 +184,7 @@ int pci_enable_ptm(struct pci_dev *dev, u8 *granularity) * associate the endpoint with a time source. */ if (pci_pcie_type(dev) == PCI_EXP_TYPE_ENDPOINT) { - ups = pci_upstream_bridge(dev); + ups = pci_upstream_ptm(dev); if (!ups || !ups->ptm_enabled) return -EINVAL; From patchwork Tue Sep 6 22:23:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 1675053 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=uFOXnfAX; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4MMg0d28Yhz1yhP for ; Wed, 7 Sep 2022 08:24:17 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230022AbiIFWYN (ORCPT ); Tue, 6 Sep 2022 18:24:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48526 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229999AbiIFWYK (ORCPT ); Tue, 6 Sep 2022 18:24:10 -0400 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6AA4CAB18A; Tue, 6 Sep 2022 15:24:07 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sin.source.kernel.org (Postfix) with ESMTPS id 2F9A6CE193F; Tue, 6 Sep 2022 22:24:05 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3F530C43470; Tue, 6 Sep 2022 22:24:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1662503043; bh=rPl1TPqw2/PuhdUCzI4nj4Vdv7gDiO1VXLiBI/IcQQM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uFOXnfAX63txEqjxwpkQFwdA/yiuCn4en+X6TwNGJn9je5pFG6jNxyg4tr5tmM32A +nWNCab2Z7f6V1kWbqyD04p5mRM9ObysscvvPHyZrODgqrGcQ/NBk7qtBXZ1L1mZRa +2WrY2hrPzeOfL6fEh/z/LvlbNEqWSueBkhOlxAi8OMUByCtn55J+dG3dDmpY1TIF2 D58DyyFbN2DcNordDrFgkcwJRf2tGg5RKYqQz4yrq3TaNTi6Rkf7BDIwIFZoVDwCfj WamHSihYAJRzWCY1LjBkFYTZukgzoXZJ674gsIvQvfLp0/IusU7Jjk6xux+/Tf9fnu CX1ps7h7SPfAA== From: Bjorn Helgaas To: Kai-Heng Feng , Rajvi Jingar , "Rafael J . Wysocki" Cc: Koba Ko , Mika Westerberg , "David E . Box" , Sathyanarayanan Kuppuswamy , linux-pci@vger.kernel.org, linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Bjorn Helgaas Subject: [PATCH v3 04/10] PCI/PTM: Separate configuration and enable Date: Tue, 6 Sep 2022 17:23:45 -0500 Message-Id: <20220906222351.64760-5-helgaas@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906222351.64760-1-helgaas@kernel.org> References: <20220906222351.64760-1-helgaas@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Bjorn Helgaas PTM configuration and enabling were previously mixed together: pci_ptm_init() collected granularity info and enabled PTM for Root Ports and Switch Upstream Ports; pci_enable_ptm() did the same for Endpoints. Move everything related to the PTM Capability register to pci_ptm_init() for all devices, and everything related to the PTM Control register to pci_enable_ptm(). Signed-off-by: Bjorn Helgaas --- drivers/pci/pcie/ptm.c | 81 ++++++++++++++---------------------------- 1 file changed, 27 insertions(+), 54 deletions(-) diff --git a/drivers/pci/pcie/ptm.c b/drivers/pci/pcie/ptm.c index 8729c3e452ee..ac51cd84793f 100644 --- a/drivers/pci/pcie/ptm.c +++ b/drivers/pci/pcie/ptm.c @@ -102,22 +102,12 @@ static struct pci_dev *pci_upstream_ptm(struct pci_dev *dev) void pci_ptm_init(struct pci_dev *dev) { int ptm; - u32 cap, ctrl; - u8 local_clock; + u32 cap; struct pci_dev *ups; if (!pci_is_pcie(dev)) return; - /* - * Enable PTM only on interior devices (root ports, switch ports, - * etc.) on the assumption that it causes no link traffic until an - * endpoint enables it. - */ - if ((pci_pcie_type(dev) == PCI_EXP_TYPE_ENDPOINT || - pci_pcie_type(dev) == PCI_EXP_TYPE_RC_END)) - return; - ptm = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_PTM); if (!ptm) return; @@ -126,76 +116,59 @@ void pci_ptm_init(struct pci_dev *dev) pci_add_ext_cap_save_buffer(dev, PCI_EXT_CAP_ID_PTM, sizeof(u16)); pci_read_config_dword(dev, ptm + PCI_PTM_CAP, &cap); - local_clock = (cap & PCI_PTM_GRANULARITY_MASK) >> 8; + dev->ptm_granularity = (cap & PCI_PTM_GRANULARITY_MASK) >> 8; /* - * There's no point in enabling PTM unless it's enabled in the - * upstream device or this device can be a PTM Root itself. Per - * the spec recommendation (PCIe r3.1, sec 7.32.3), select the + * Per the spec recommendation (PCIe r6.0, sec 7.9.15.3), select the * furthest upstream Time Source as the PTM Root. */ ups = pci_upstream_ptm(dev); - if (ups && ups->ptm_enabled) { - ctrl = PCI_PTM_CTRL_ENABLE; + if (ups) { if (ups->ptm_granularity == 0) dev->ptm_granularity = 0; - else if (ups->ptm_granularity > local_clock) + else if (ups->ptm_granularity > dev->ptm_granularity) dev->ptm_granularity = ups->ptm_granularity; - } else { - if (cap & PCI_PTM_CAP_ROOT) { - ctrl = PCI_PTM_CTRL_ENABLE | PCI_PTM_CTRL_ROOT; - dev->ptm_root = 1; - dev->ptm_granularity = local_clock; - } else - return; + } else if (cap & PCI_PTM_CAP_ROOT) { + dev->ptm_root = 1; + } else if (pci_pcie_type(dev) == PCI_EXP_TYPE_RC_END) { + + /* + * Per sec 7.9.15.3, this should be the Local Clock + * Granularity of the associated Time Source. But it + * doesn't say how to find that Time Source. + */ + dev->ptm_granularity = 0; } - ctrl |= dev->ptm_granularity << 8; - pci_write_config_dword(dev, ptm + PCI_PTM_CTRL, ctrl); - dev->ptm_enabled = 1; - - pci_ptm_info(dev); + if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT || + pci_pcie_type(dev) == PCI_EXP_TYPE_UPSTREAM) + pci_enable_ptm(dev, NULL); } int pci_enable_ptm(struct pci_dev *dev, u8 *granularity) { - int ptm; - u32 cap, ctrl; + int ptm = dev->ptm_cap; struct pci_dev *ups; + u32 ctrl; - if (!pci_is_pcie(dev)) - return -EINVAL; - - ptm = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_PTM); if (!ptm) return -EINVAL; - dev->ptm_cap = ptm; - pci_read_config_dword(dev, ptm + PCI_PTM_CAP, &cap); - if (!(cap & PCI_PTM_CAP_REQ)) - return -EINVAL; - /* - * For a PCIe Endpoint, PTM is only useful if the endpoint can - * issue PTM requests to upstream devices that have PTM enabled. - * - * For Root Complex Integrated Endpoints, there is no upstream - * device, so there must be some implementation-specific way to - * associate the endpoint with a time source. + * If this device is not a PTM Root, the upstream link partner must + * have PTM enabled before we can enable PTM. */ - if (pci_pcie_type(dev) == PCI_EXP_TYPE_ENDPOINT) { + if (!dev->ptm_root) { ups = pci_upstream_ptm(dev); if (!ups || !ups->ptm_enabled) return -EINVAL; - - dev->ptm_granularity = ups->ptm_granularity; - } else if (pci_pcie_type(dev) == PCI_EXP_TYPE_RC_END) { - dev->ptm_granularity = 0; - } else - return -EINVAL; + } ctrl = PCI_PTM_CTRL_ENABLE; ctrl |= dev->ptm_granularity << 8; + if (dev->ptm_root) + ctrl |= PCI_PTM_CTRL_ROOT; + pci_write_config_dword(dev, ptm + PCI_PTM_CTRL, ctrl); dev->ptm_enabled = 1; From patchwork Tue Sep 6 22:23:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 1675054 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=vPfusRYK; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4MMg0g6hJGz1yhP for ; Wed, 7 Sep 2022 08:24:19 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230071AbiIFWYS (ORCPT ); Tue, 6 Sep 2022 18:24:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48478 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229916AbiIFWYL (ORCPT ); Tue, 6 Sep 2022 18:24:11 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6BB9BAA3E4; Tue, 6 Sep 2022 15:24:08 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 63BF2B81AAA; Tue, 6 Sep 2022 22:24:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EB9A0C433D6; Tue, 6 Sep 2022 22:24:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1662503045; bh=cD6LaKtHHzCoaSefmit3nfjumtZ0PHa2t3oTDXVD7L0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vPfusRYKGLk3TY22O1v2vmVECesOBY1pyCymDRrMmhZTxwwMfxH0QL9+8jceQ4FsD 3lVV+/x4o0Awsf4PzqYz3/b7D2BUIOYFhLpK6R1lc0QS3i03lTgPPz4+OMTwVsCHqU CFsj66y4Oevjj+CKseAunSovND2QHNJJxr4izQZbVX6AeIN7/+D6xaJt80fLmmcR1f 3QD4Gflny2ftp4Rh3v1BTfXurPs1yUA5DNO5+5NpQxwg9n1rAZbKw54nhk2sx9WJBz tmmpzwLx6snG5lkhUwWTzSArO9FqPB691kJxHlsv5jHrVRYxjmlZzhExNuJ2h7zuc4 4SAE+HtCYM/sw== From: Bjorn Helgaas To: Kai-Heng Feng , Rajvi Jingar , "Rafael J . Wysocki" Cc: Koba Ko , Mika Westerberg , "David E . Box" , Sathyanarayanan Kuppuswamy , linux-pci@vger.kernel.org, linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Bjorn Helgaas Subject: [PATCH v3 05/10] PCI/PTM: Add pci_disable_ptm() wrapper Date: Tue, 6 Sep 2022 17:23:46 -0500 Message-Id: <20220906222351.64760-6-helgaas@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906222351.64760-1-helgaas@kernel.org> References: <20220906222351.64760-1-helgaas@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Bjorn Helgaas Implement pci_disable_ptm() as a wrapper around an internal __pci_disable_ptm() that we can use during suspend to disable PTM without clearing dev->ptm_enabled. A future commit will re-enable PTM during resume when dev->ptm_enabled is set. Export pci_disable_ptm(), which *does* clear dev->ptm_enabled, for use by drivers that want to disable PTM. Signed-off-by: Bjorn Helgaas --- drivers/pci/pci.h | 2 -- drivers/pci/pcie/ptm.c | 9 ++++++++- include/linux/pci.h | 2 ++ 3 files changed, 10 insertions(+), 3 deletions(-) diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 785f31086313..91a465460d0f 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -507,11 +507,9 @@ static inline int pci_iov_bus_range(struct pci_bus *bus) #ifdef CONFIG_PCIE_PTM void pci_save_ptm_state(struct pci_dev *dev); void pci_restore_ptm_state(struct pci_dev *dev); -void pci_disable_ptm(struct pci_dev *dev); #else static inline void pci_save_ptm_state(struct pci_dev *dev) { } static inline void pci_restore_ptm_state(struct pci_dev *dev) { } -static inline void pci_disable_ptm(struct pci_dev *dev) { } #endif unsigned long pci_cardbus_resource_alignment(struct resource *); diff --git a/drivers/pci/pcie/ptm.c b/drivers/pci/pcie/ptm.c index ac51cd84793f..762299984469 100644 --- a/drivers/pci/pcie/ptm.c +++ b/drivers/pci/pcie/ptm.c @@ -29,7 +29,7 @@ static void pci_ptm_info(struct pci_dev *dev) dev->ptm_root ? " (root)" : "", clock_desc); } -void pci_disable_ptm(struct pci_dev *dev) +static void __pci_disable_ptm(struct pci_dev *dev) { int ptm = dev->ptm_cap; u16 ctrl; @@ -42,6 +42,13 @@ void pci_disable_ptm(struct pci_dev *dev) pci_write_config_word(dev, ptm + PCI_PTM_CTRL, ctrl); } +void pci_disable_ptm(struct pci_dev *dev) +{ + __pci_disable_ptm(dev); + dev->ptm_enabled = 0; +} +EXPORT_SYMBOL(pci_disable_ptm); + void pci_save_ptm_state(struct pci_dev *dev) { int ptm = dev->ptm_cap; diff --git a/include/linux/pci.h b/include/linux/pci.h index 54be939023a3..cb5f796e3319 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1678,10 +1678,12 @@ bool pci_ats_disabled(void); #ifdef CONFIG_PCIE_PTM int pci_enable_ptm(struct pci_dev *dev, u8 *granularity); +void pci_disable_ptm(struct pci_dev *dev); bool pcie_ptm_enabled(struct pci_dev *dev); #else static inline int pci_enable_ptm(struct pci_dev *dev, u8 *granularity) { return -EINVAL; } +static inline void pci_disable_ptm(struct pci_dev *dev) { } static inline bool pcie_ptm_enabled(struct pci_dev *dev) { return false; } #endif From patchwork Tue Sep 6 22:23:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 1675055 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=WB/Q/NGN; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4MMg0y3xH8z1yhP for ; Wed, 7 Sep 2022 08:24:34 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230092AbiIFWYc (ORCPT ); Tue, 6 Sep 2022 18:24:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48510 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229980AbiIFWYM (ORCPT ); Tue, 6 Sep 2022 18:24:12 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 192E5AA37A; Tue, 6 Sep 2022 15:24:09 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 2C3FBB81AAC; Tue, 6 Sep 2022 22:24:08 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A52FEC433C1; Tue, 6 Sep 2022 22:24:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1662503046; bh=JiLQVQZBxN5nRTMlR7cEglY0khs6D7MoN1CEw3gMigY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WB/Q/NGNxNddoHQZVZnZ49FOmWp0IHWFoYWk8jnMiTy3qwlpYySJ/m34isSAP38/v p4cDMYCxZWrY498gXfajG5pAummooJHA7LICDLDdyRvhzQNYuZbD2T2MOsu5gl2QBZ NP524JN2QOFsL7jcfNnHMQvVhICL4c+HqQ/d3hfIVQJcsNern9p99hRMRKJZyrsS7L iiRsBKbr+MCaP3z2UHgarMwy9zQ/6n1voHtQjhYTDj9COoKpQRz42BlemYwscF9VuH CQns395hbUc3wCGHBxyEVOLDOIYcQuogbVtzNPF6qhtYHC2AHQwAChQPLjIRb240oI 1WxXcZogNOwmw== From: Bjorn Helgaas To: Kai-Heng Feng , Rajvi Jingar , "Rafael J . Wysocki" Cc: Koba Ko , Mika Westerberg , "David E . Box" , Sathyanarayanan Kuppuswamy , linux-pci@vger.kernel.org, linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Bjorn Helgaas Subject: [PATCH v3 06/10] PCI/PTM: Add pci_enable_ptm() wrapper Date: Tue, 6 Sep 2022 17:23:47 -0500 Message-Id: <20220906222351.64760-7-helgaas@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906222351.64760-1-helgaas@kernel.org> References: <20220906222351.64760-1-helgaas@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Bjorn Helgaas Implement pci_enable_ptm() as a wrapper around an internal __pci_enable_ptm() that we can use during resume to enable PTM without emitting log messages. Signed-off-by: Bjorn Helgaas --- drivers/pci/pcie/ptm.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/drivers/pci/pcie/ptm.c b/drivers/pci/pcie/ptm.c index 762299984469..4a9f045126ca 100644 --- a/drivers/pci/pcie/ptm.c +++ b/drivers/pci/pcie/ptm.c @@ -152,7 +152,7 @@ void pci_ptm_init(struct pci_dev *dev) pci_enable_ptm(dev, NULL); } -int pci_enable_ptm(struct pci_dev *dev, u8 *granularity) +static int __pci_enable_ptm(struct pci_dev *dev) { int ptm = dev->ptm_cap; struct pci_dev *ups; @@ -177,6 +177,17 @@ int pci_enable_ptm(struct pci_dev *dev, u8 *granularity) ctrl |= PCI_PTM_CTRL_ROOT; pci_write_config_dword(dev, ptm + PCI_PTM_CTRL, ctrl); + return 0; +} + +int pci_enable_ptm(struct pci_dev *dev, u8 *granularity) +{ + int rc; + + rc = __pci_enable_ptm(dev); + if (rc) + return rc; + dev->ptm_enabled = 1; pci_ptm_info(dev); From patchwork Tue Sep 6 22:23:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 1675056 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=DeD/nKSA; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4MMg0y72G6z1yhP for ; Wed, 7 Sep 2022 08:24:34 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230103AbiIFWYd (ORCPT ); Tue, 6 Sep 2022 18:24:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48466 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229995AbiIFWYN (ORCPT ); Tue, 6 Sep 2022 18:24:13 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4EB26ABF0E; Tue, 6 Sep 2022 15:24:10 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 30606616F5; Tue, 6 Sep 2022 22:24:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6CA26C433C1; Tue, 6 Sep 2022 22:24:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1662503048; bh=GZ1t3eEPe9WAKIeqnRdMruDBNlk/eilfZ8AziF6/6PI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DeD/nKSAZsxT6ztAhrLodFx5OfRGCevqJTbqV9M0/SXRzVpdVIJkCYT9MOMs6v5rQ 9fgsbljrwM7837zZsdZsuYe0kfiNoZDsq1QyXojGodg+FmI7V7w1YEY5ntG4XF2HQz yikQXO7zGnnqozfDYCVe0YZm5Dd0pThK1+kPO40FztEjlesNUPvM+W8wD5+zabncp7 DJuKd9X8qxaDiiemhtP6P4mdxplnjkZhjjcz0M+MlMt5SLMxw0+HuGY9go68I61/02 kx6z22rlodkosTtmoV/p0pAvpmyfhURIgG4ZYfHbOympDprTJJ6lV4B/2Lps/aMcIG lHyeBSD3YLeVQ== From: Bjorn Helgaas To: Kai-Heng Feng , Rajvi Jingar , "Rafael J . Wysocki" Cc: Koba Ko , Mika Westerberg , "David E . Box" , Sathyanarayanan Kuppuswamy , linux-pci@vger.kernel.org, linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Bjorn Helgaas Subject: [PATCH v3 07/10] PCI/PTM: Add suspend/resume Date: Tue, 6 Sep 2022 17:23:48 -0500 Message-Id: <20220906222351.64760-8-helgaas@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906222351.64760-1-helgaas@kernel.org> References: <20220906222351.64760-1-helgaas@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Bjorn Helgaas --- drivers/pci/pci.c | 4 ++-- drivers/pci/pci.h | 4 ++++ drivers/pci/pcie/ptm.c | 15 +++++++++++++++ 3 files changed, 21 insertions(+), 2 deletions(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 95bc329e74c0..83818f81577d 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -2714,7 +2714,7 @@ int pci_prepare_to_sleep(struct pci_dev *dev) * lower-power idle state as a whole. */ if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) - pci_disable_ptm(dev); + pci_suspend_ptm(dev); pci_enable_wake(dev, target_state, wakeup); @@ -2772,7 +2772,7 @@ int pci_finish_runtime_suspend(struct pci_dev *dev) * lower-power idle state as a whole. */ if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) - pci_disable_ptm(dev); + pci_suspend_ptm(dev); __pci_enable_wake(dev, target_state, pci_dev_run_wake(dev)); diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 91a465460d0f..ce4a277e3f41 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -507,9 +507,13 @@ static inline int pci_iov_bus_range(struct pci_bus *bus) #ifdef CONFIG_PCIE_PTM void pci_save_ptm_state(struct pci_dev *dev); void pci_restore_ptm_state(struct pci_dev *dev); +void pci_suspend_ptm(struct pci_dev *dev); +void pci_resume_ptm(struct pci_dev *dev); #else static inline void pci_save_ptm_state(struct pci_dev *dev) { } static inline void pci_restore_ptm_state(struct pci_dev *dev) { } +static inline void pci_suspend_ptm(struct pci_dev *dev) { } +static inline void pci_resume_ptm(struct pci_dev *dev) { } #endif unsigned long pci_cardbus_resource_alignment(struct resource *); diff --git a/drivers/pci/pcie/ptm.c b/drivers/pci/pcie/ptm.c index 4a9f045126ca..8ac844212436 100644 --- a/drivers/pci/pcie/ptm.c +++ b/drivers/pci/pcie/ptm.c @@ -198,6 +198,21 @@ int pci_enable_ptm(struct pci_dev *dev, u8 *granularity) } EXPORT_SYMBOL(pci_enable_ptm); +/* + * Disable PTM, but leave dev->ptm_enabled so we silently re-enable it on + * resume. + */ +void pci_suspend_ptm(struct pci_dev *dev) +{ + __pci_disable_ptm(dev); +} + +void pci_resume_ptm(struct pci_dev *dev) +{ + if (dev->ptm_enabled) + __pci_enable_ptm(dev); +} + bool pcie_ptm_enabled(struct pci_dev *dev) { if (!dev) From patchwork Tue Sep 6 22:23:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 1675057 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=bmcv0F9F; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4MMg153zmwz1yhP for ; Wed, 7 Sep 2022 08:24:41 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230129AbiIFWYi (ORCPT ); Tue, 6 Sep 2022 18:24:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48790 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230061AbiIFWYR (ORCPT ); Tue, 6 Sep 2022 18:24:17 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BA2E6AB417; Tue, 6 Sep 2022 15:24:11 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0D14361714; Tue, 6 Sep 2022 22:24:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 35126C433D6; Tue, 6 Sep 2022 22:24:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1662503050; bh=AvPvkozYP8lvZ3lyaRCBEZlCgNXntwO4lZojFieDtfc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bmcv0F9Ft7fOIr0Upkvq3lgYPE3ClrJBD26uNzovQwCifO9S6MuQ+jYyQlVlZ/H1S u47asImqLq1MnIYTGIQMAZw9LYscFev6hTLT6PIJ/EcMbzJOBkmnLhPsjtOwcEHp++ xirr5l1F3rHUdwcquIJ9wiYwH6PUEdwNwa3oCX0MPLXdHp7CQPawsSu8JTeX0VsyDw TI4KXJ0sPwoB67qK9bApSQ7GNg9qffSMn6C0mXUMDsdjAS0+FvvZZdk6T15BnSFpY2 ftKc1f08d0KMKMBSCAMOCpf9Y1ACDHhkSiNcWfvZ5Ptyswk6fNq3xNSS0CyyfXyPRD VFG0HdegCOj+A== From: Bjorn Helgaas To: Kai-Heng Feng , Rajvi Jingar , "Rafael J . Wysocki" Cc: Koba Ko , Mika Westerberg , "David E . Box" , Sathyanarayanan Kuppuswamy , linux-pci@vger.kernel.org, linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Bjorn Helgaas Subject: [PATCH v3 08/10] PCI/PTM: Move pci_ptm_info() body into its only caller Date: Tue, 6 Sep 2022 17:23:49 -0500 Message-Id: <20220906222351.64760-9-helgaas@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906222351.64760-1-helgaas@kernel.org> References: <20220906222351.64760-1-helgaas@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Bjorn Helgaas pci_ptm_info() is simple and is only called by pci_enable_ptm(). Move the entire body there. Signed-off-by: Bjorn Helgaas --- drivers/pci/pcie/ptm.c | 39 +++++++++++++++++---------------------- 1 file changed, 17 insertions(+), 22 deletions(-) diff --git a/drivers/pci/pcie/ptm.c b/drivers/pci/pcie/ptm.c index 8ac844212436..d65f5af9700d 100644 --- a/drivers/pci/pcie/ptm.c +++ b/drivers/pci/pcie/ptm.c @@ -9,26 +9,6 @@ #include #include "../pci.h" -static void pci_ptm_info(struct pci_dev *dev) -{ - char clock_desc[8]; - - switch (dev->ptm_granularity) { - case 0: - snprintf(clock_desc, sizeof(clock_desc), "unknown"); - break; - case 255: - snprintf(clock_desc, sizeof(clock_desc), ">254ns"); - break; - default: - snprintf(clock_desc, sizeof(clock_desc), "%uns", - dev->ptm_granularity); - break; - } - pci_info(dev, "PTM enabled%s, %s granularity\n", - dev->ptm_root ? " (root)" : "", clock_desc); -} - static void __pci_disable_ptm(struct pci_dev *dev) { int ptm = dev->ptm_cap; @@ -183,6 +163,7 @@ static int __pci_enable_ptm(struct pci_dev *dev) int pci_enable_ptm(struct pci_dev *dev, u8 *granularity) { int rc; + char clock_desc[8]; rc = __pci_enable_ptm(dev); if (rc) @@ -190,10 +171,24 @@ int pci_enable_ptm(struct pci_dev *dev, u8 *granularity) dev->ptm_enabled = 1; - pci_ptm_info(dev); - if (granularity) *granularity = dev->ptm_granularity; + + switch (dev->ptm_granularity) { + case 0: + snprintf(clock_desc, sizeof(clock_desc), "unknown"); + break; + case 255: + snprintf(clock_desc, sizeof(clock_desc), ">254ns"); + break; + default: + snprintf(clock_desc, sizeof(clock_desc), "%uns", + dev->ptm_granularity); + break; + } + pci_info(dev, "PTM enabled%s, %s granularity\n", + dev->ptm_root ? " (root)" : "", clock_desc); + return 0; } EXPORT_SYMBOL(pci_enable_ptm); From patchwork Tue Sep 6 22:23:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 1675059 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=WVzSCHFQ; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4MMg1J2l4yz1yhP for ; Wed, 7 Sep 2022 08:24:52 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230089AbiIFWYt (ORCPT ); Tue, 6 Sep 2022 18:24:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49704 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230083AbiIFWYa (ORCPT ); Tue, 6 Sep 2022 18:24:30 -0400 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 41EA4AEDA9; Tue, 6 Sep 2022 15:24:15 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sin.source.kernel.org (Postfix) with ESMTPS id E87D8CE18A3; Tue, 6 Sep 2022 22:24:13 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 10D77C43149; Tue, 6 Sep 2022 22:24:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1662503052; bh=RqustZvweOUGMZRzXAxz1L58dl8uzoXmJZ6vgqdeZw0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WVzSCHFQefHgDDstD/MthGIhvD0pGHpJ+v3D2GtaJxL/NlaD0nIbblnfoctu7hbr6 QsFxSOS724ILyn1x6fALh2JzRt2SXKig/m8zF1D1Sx+ON/sefyroJkWXH6pWBGUzmO Wuw9Ag22NPPAIZ1VXf5J0puU3BdLtM47iOrN6g/xOIaNgz96UegXZ49S+jmTKTxCKS ausPxIZMnThbUldg0CLzLki/pwFQQQlKN8d6NFkypHEfDc2KbKqYrkIwLgRdyHU2xh k88FISCnglQ7o6l59yluzs6ayy+NDGAMnmW7UobDyEZUHglQH3IVsRCrS4HHe1TA7+ WeqLHF3GL9mFQ== From: Bjorn Helgaas To: Kai-Heng Feng , Rajvi Jingar , "Rafael J . Wysocki" Cc: Koba Ko , Mika Westerberg , "David E . Box" , Sathyanarayanan Kuppuswamy , linux-pci@vger.kernel.org, linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Bjorn Helgaas Subject: [PATCH v3 09/10] PCI/PTM: Reorder functions in logical order Date: Tue, 6 Sep 2022 17:23:50 -0500 Message-Id: <20220906222351.64760-10-helgaas@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906222351.64760-1-helgaas@kernel.org> References: <20220906222351.64760-1-helgaas@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Bjorn Helgaas pci_enable_ptm() and pci_disable_ptm() were separated. pci_save_ptm_state() and pci_restore_ptm_state() dangled at the top. Move them to logical places. Signed-off-by: Bjorn Helgaas --- drivers/pci/pcie/ptm.c | 108 ++++++++++++++++++++--------------------- 1 file changed, 54 insertions(+), 54 deletions(-) diff --git a/drivers/pci/pcie/ptm.c b/drivers/pci/pcie/ptm.c index d65f5af9700d..6c09e00a7b51 100644 --- a/drivers/pci/pcie/ptm.c +++ b/drivers/pci/pcie/ptm.c @@ -9,60 +9,6 @@ #include #include "../pci.h" -static void __pci_disable_ptm(struct pci_dev *dev) -{ - int ptm = dev->ptm_cap; - u16 ctrl; - - if (!ptm) - return; - - pci_read_config_word(dev, ptm + PCI_PTM_CTRL, &ctrl); - ctrl &= ~PCI_PTM_CTRL_ENABLE; - pci_write_config_word(dev, ptm + PCI_PTM_CTRL, ctrl); -} - -void pci_disable_ptm(struct pci_dev *dev) -{ - __pci_disable_ptm(dev); - dev->ptm_enabled = 0; -} -EXPORT_SYMBOL(pci_disable_ptm); - -void pci_save_ptm_state(struct pci_dev *dev) -{ - int ptm = dev->ptm_cap; - struct pci_cap_saved_state *save_state; - u16 *cap; - - if (!ptm) - return; - - save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_PTM); - if (!save_state) - return; - - cap = (u16 *)&save_state->cap.data[0]; - pci_read_config_word(dev, ptm + PCI_PTM_CTRL, cap); -} - -void pci_restore_ptm_state(struct pci_dev *dev) -{ - int ptm = dev->ptm_cap; - struct pci_cap_saved_state *save_state; - u16 *cap; - - if (!ptm) - return; - - save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_PTM); - if (!save_state) - return; - - cap = (u16 *)&save_state->cap.data[0]; - pci_write_config_word(dev, ptm + PCI_PTM_CTRL, *cap); -} - /* * If the next upstream device supports PTM, return it; otherwise return * NULL. PTM Messages are local, so both link partners must support it. @@ -132,6 +78,40 @@ void pci_ptm_init(struct pci_dev *dev) pci_enable_ptm(dev, NULL); } +void pci_save_ptm_state(struct pci_dev *dev) +{ + int ptm = dev->ptm_cap; + struct pci_cap_saved_state *save_state; + u16 *cap; + + if (!ptm) + return; + + save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_PTM); + if (!save_state) + return; + + cap = (u16 *)&save_state->cap.data[0]; + pci_read_config_word(dev, ptm + PCI_PTM_CTRL, cap); +} + +void pci_restore_ptm_state(struct pci_dev *dev) +{ + int ptm = dev->ptm_cap; + struct pci_cap_saved_state *save_state; + u16 *cap; + + if (!ptm) + return; + + save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_PTM); + if (!save_state) + return; + + cap = (u16 *)&save_state->cap.data[0]; + pci_write_config_word(dev, ptm + PCI_PTM_CTRL, *cap); +} + static int __pci_enable_ptm(struct pci_dev *dev) { int ptm = dev->ptm_cap; @@ -193,6 +173,26 @@ int pci_enable_ptm(struct pci_dev *dev, u8 *granularity) } EXPORT_SYMBOL(pci_enable_ptm); +static void __pci_disable_ptm(struct pci_dev *dev) +{ + int ptm = dev->ptm_cap; + u16 ctrl; + + if (!ptm) + return; + + pci_read_config_word(dev, ptm + PCI_PTM_CTRL, &ctrl); + ctrl &= ~PCI_PTM_CTRL_ENABLE; + pci_write_config_word(dev, ptm + PCI_PTM_CTRL, ctrl); +} + +void pci_disable_ptm(struct pci_dev *dev) +{ + __pci_disable_ptm(dev); + dev->ptm_enabled = 0; +} +EXPORT_SYMBOL(pci_disable_ptm); + /* * Disable PTM, but leave dev->ptm_enabled so we silently re-enable it on * resume. From patchwork Tue Sep 6 22:23:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 1675058 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=jTF2mls+; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4MMg1G4y2pz1yhP for ; Wed, 7 Sep 2022 08:24:50 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230171AbiIFWYr (ORCPT ); Tue, 6 Sep 2022 18:24:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49482 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230080AbiIFWY0 (ORCPT ); Tue, 6 Sep 2022 18:24:26 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9EE80AE86F; Tue, 6 Sep 2022 15:24:15 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A378861713; Tue, 6 Sep 2022 22:24:14 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DAE96C433C1; Tue, 6 Sep 2022 22:24:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1662503054; bh=8zBtTybYX8wofqg/4M5b5Eesn1tmGcj3alXYa5B6SS8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jTF2mls+dk/XL02WdDwayJCVNRSAdDhBURu3wckxOru9a5/wQKNLb4P6TPbmkSIKh bzQb8xJ/qYcu56kRUVpLaWHZLUwzo8iFfG6qZ6etUMrP0R7idmfxLO/yxpNEAlPebj XgEpTnQgLZDegIE+t+qwm5vozzOLCSlS7KORLibCFL8GZ+0czg7iQEr+fqKN5GtM7N wgCtCDsDJE5DdbD6/kaEQGCUhhXaJLnXULnB2hZ8xqYMHZ/OD9rZagdEY7ZfdeqPJ0 INRsniFzvKiQE7cXiouUc4v7XEW7341ma0ApVVjqFnw0ZAt5kDLcTajzAXGTxd7GMA YRVqMpdEIuKyw== From: Bjorn Helgaas To: Kai-Heng Feng , Rajvi Jingar , "Rafael J . Wysocki" Cc: Koba Ko , Mika Westerberg , "David E . Box" , Sathyanarayanan Kuppuswamy , linux-pci@vger.kernel.org, linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Bjorn Helgaas Subject: [PATCH v3 10/10] PCI/PM: Always disable PTM for all devices during suspend Date: Tue, 6 Sep 2022 17:23:51 -0500 Message-Id: <20220906222351.64760-11-helgaas@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906222351.64760-1-helgaas@kernel.org> References: <20220906222351.64760-1-helgaas@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Bjorn Helgaas We want to disable PTM on Root Ports because that allows some chips, e.g., Intel mobile chips since Coffee Lake, to enter a lower-power PM state. That means we also have to disable PTM on downstream devices. PCIe r6.0, sec 2.2.8, recommends that functions support generation of messages in non-D0 states, so we have to assume Switch Upstream Ports or Endpoints may send PTM Requests while in D1, D2, and D3hot. A PTM message received by a Downstream Port (including a Root Port) with PTM disabled must be treated as an Unsupported Request (sec 6.21.3). PTM was previously disabled only for Root Ports, and it was disabled in pci_prepare_to_sleep(), which is not called at all if a driver supports legacy PM or does its own state saving. Instead, disable PTM early in pci_pm_suspend() and pci_pm_runtime_suspend() so we do it in all cases. Previously PTM was disabled *after* saving device state, so the state restore on resume automatically re-enabled it. Since we now disable PTM *before* saving state, we must explicitly re-enable it in pci_pm_resume() and pci_pm_runtime_resume(). Here's a sample of errors that occur when PTM is disabled only on the Root Port. With this topology: 0000:00:1d.0 Root Port to [bus 08-71] 0000:08:00.0 Switch Upstream Port to [bus 09-71] Kai-Heng reported errors like this: pcieport 0000:00:1d.0: [20] UnsupReq (First) pcieport 0000:00:1d.0: AER: TLP Header: 34000000 08000052 00000000 00000000 Decoding TLP header 0x34...... (0011 0100b) and 0x08000052: Fmt 001b 4 DW header, no data Type 1 0100b Msg (Local - Terminate at Receiver) Requester ID 0x0800 Bus 08 Devfn 00.0 Message Code 0x52 0101 0010b PTM Request The 00:1d.0 Root Port logged an Unsupported Request error when it received a PTM Request with Requester ID 08:00.0. Fixes: a697f072f5da ("PCI: Disable PTM during suspend to save power") Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=215453 Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=216210 Reported-by: Kai-Heng Feng Signed-off-by: Bjorn Helgaas --- drivers/pci/pci-driver.c | 11 +++++++++++ drivers/pci/pci.c | 28 ++-------------------------- 2 files changed, 13 insertions(+), 26 deletions(-) diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index 2815922ac525..107d77f3c846 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -774,6 +774,12 @@ static int pci_pm_suspend(struct device *dev) pci_dev->skip_bus_pm = false; + /* + * Disabling PTM allows some systems, e.g., Intel mobile chips + * since Coffee Lake, to enter a lower-power PM state. + */ + pci_suspend_ptm(pci_dev); + if (pci_has_legacy_pm_support(pci_dev)) return pci_legacy_suspend(dev, PMSG_SUSPEND); @@ -982,6 +988,8 @@ static int pci_pm_resume(struct device *dev) if (pci_dev->state_saved) pci_restore_standard_config(pci_dev); + pci_resume_ptm(pci_dev); + if (pci_has_legacy_pm_support(pci_dev)) return pci_legacy_resume(dev); @@ -1269,6 +1277,8 @@ static int pci_pm_runtime_suspend(struct device *dev) pci_power_t prev = pci_dev->current_state; int error; + pci_suspend_ptm(pci_dev); + /* * If pci_dev->driver is not set (unbound), we leave the device in D0, * but it may go to D3cold when the bridge above it runtime suspends. @@ -1330,6 +1340,7 @@ static int pci_pm_runtime_resume(struct device *dev) * D3cold when the bridge above it runtime suspended. */ pci_pm_default_resume_early(pci_dev); + pci_resume_ptm(pci_dev); if (!pci_dev->driver) return 0; diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 83818f81577d..107afa0a5b03 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -2706,24 +2706,12 @@ int pci_prepare_to_sleep(struct pci_dev *dev) if (target_state == PCI_POWER_ERROR) return -EIO; - /* - * There are systems (for example, Intel mobile chips since Coffee - * Lake) where the power drawn while suspended can be significantly - * reduced by disabling PTM on PCIe root ports as this allows the - * port to enter a lower-power PM state and the SoC to reach a - * lower-power idle state as a whole. - */ - if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) - pci_suspend_ptm(dev); - pci_enable_wake(dev, target_state, wakeup); error = pci_set_power_state(dev, target_state); - if (error) { + if (error) pci_enable_wake(dev, target_state, false); - pci_restore_ptm_state(dev); - } return error; } @@ -2764,24 +2752,12 @@ int pci_finish_runtime_suspend(struct pci_dev *dev) if (target_state == PCI_POWER_ERROR) return -EIO; - /* - * There are systems (for example, Intel mobile chips since Coffee - * Lake) where the power drawn while suspended can be significantly - * reduced by disabling PTM on PCIe root ports as this allows the - * port to enter a lower-power PM state and the SoC to reach a - * lower-power idle state as a whole. - */ - if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) - pci_suspend_ptm(dev); - __pci_enable_wake(dev, target_state, pci_dev_run_wake(dev)); error = pci_set_power_state(dev, target_state); - if (error) { + if (error) pci_enable_wake(dev, target_state, false); - pci_restore_ptm_state(dev); - } return error; }