From patchwork Thu Aug 24 04:46:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kai-Heng Feng X-Patchwork-Id: 1825118 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=TNfcEKTW; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RWVvY1wcPz1yg8 for ; Thu, 24 Aug 2023 14:48:07 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=TNfcEKTW; dkim-atps=neutral Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4RWVvR4qblz3cBY for ; Thu, 24 Aug 2023 14:48:03 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=TNfcEKTW; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=canonical.com (client-ip=185.125.188.120; helo=smtp-relay-canonical-0.canonical.com; envelope-from=kai.heng.feng@canonical.com; receiver=lists.ozlabs.org) Received: from smtp-relay-canonical-0.canonical.com (smtp-relay-canonical-0.canonical.com [185.125.188.120]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4RWVv51r5vz2yst for ; Thu, 24 Aug 2023 14:47:43 +1000 (AEST) Received: from localhost.localdomain (unknown [10.101.196.174]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-0.canonical.com (Postfix) with ESMTPSA id 7C121400F1; Thu, 24 Aug 2023 04:47:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1692852459; bh=lT+xOUvkqgknE0fDL1EWLyC+g3xCP3X2QeaziUTo0D0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=TNfcEKTW4L3BY06akVpl+Mj56NQYEVdA96PNmn8X2Bwm0aCHupWR+kxhmVqZ1bLNL 3389hkPLWmf3bqR+vFvDTfSmy9lnJHXmbpDqOFOxGYzE9UQSOAjqt34VQB2aWX5HvO 74b2J7i0K4GB+hYL8bnqtlgJJwrqZd67WP/kZeMqHaFJeyNDiv31ircWXa4SGj9/Bl M6S4YLXPBt4Ys+YpvD4a0qXqRE4A3XjpJvKSOwaRgj4lbTtTbJK9AGTzncVaPkemcq wFCNWaZnbm6xTX0oDg2dyRsecdmnJw8TIj50QrqtCMg06/IzwejuDZMKxpeAIiMb5a tCxYjdBFn8loA== From: Kai-Heng Feng To: bhelgaas@google.com Subject: [PATCH 2/3] PCI/AER: Disable AER service on suspend Date: Thu, 24 Aug 2023 12:46:44 +0800 Message-Id: <20230824044645.423378-2-kai.heng.feng@canonical.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230824044645.423378-1-kai.heng.feng@canonical.com> References: <20230824044645.423378-1-kai.heng.feng@canonical.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: sathyanarayanan.kuppuswamy@linux.intel.com, linuxppc-dev@lists.ozlabs.org, Mahesh J Salgaonkar , linux-kernel@vger.kernel.org, koba.ko@canonical.com, Kai-Heng Feng , Oliver O'Halloran , linux-pci@vger.kernel.org, mika.westerberg@linux.intel.com Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" When the power rail gets cut off, the hardware can create some electric noise on the link that triggers AER. If IRQ is shared between AER with PME, such AER noise will cause a spurious wakeup on system suspend. When the power rail gets back, the firmware of the device resets itself and can create unexpected behavior like sending PTM messages. For this case, the driver will always be too late to toggle off features should be disabled. As Per PCIe Base Spec 5.0, section 5.2, titled "Link State Power Management", TLP and DLLP transmission are disabled for a Link in L2/L3 Ready (D3hot), L2 (D3cold with aux power) and L3 (D3cold) states. So if the power will be turned off during suspend process, disable AER service and re-enable it during the resume process. This should not affect the basic functionality. Link: https://bugzilla.kernel.org/show_bug.cgi?id=209149 Link: https://bugzilla.kernel.org/show_bug.cgi?id=216295 v7: - Wording - Disable AER completely (again) if power will be turned off v6: v5: - Wording. v4: v3: - No change. v2: - Only disable AER IRQ. - No more check on PME IRQ#. - Use helper. Signed-off-by: Kai-Heng Feng --- drivers/pci/pcie/aer.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index f6c24ded134c..b5161bfedd8b 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -28,6 +28,7 @@ #include #include #include +#include #include #include @@ -1340,6 +1341,28 @@ static int aer_probe(struct pcie_device *dev) return 0; } +static int aer_suspend(struct pcie_device *dev) +{ + struct aer_rpc *rpc = get_service_data(dev); + struct pci_dev *pdev = rpc->rpd; + + if (pci_ancestor_pr3_present(pdev) || pm_suspend_via_firmware()) + aer_disable_rootport(rpc); + + return 0; +} + +static int aer_resume(struct pcie_device *dev) +{ + struct aer_rpc *rpc = get_service_data(dev); + struct pci_dev *pdev = rpc->rpd; + + if (pci_ancestor_pr3_present(pdev) || pm_resume_via_firmware()) + aer_enable_rootport(rpc); + + return 0; +} + /** * aer_root_reset - reset Root Port hierarchy, RCEC, or RCiEP * @dev: pointer to Root Port, RCEC, or RCiEP @@ -1411,6 +1434,8 @@ static struct pcie_port_service_driver aerdriver = { .service = PCIE_PORT_SERVICE_AER, .probe = aer_probe, + .suspend = aer_suspend, + .resume = aer_resume, .remove = aer_remove, }; From patchwork Thu Aug 24 04:46:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kai-Heng Feng X-Patchwork-Id: 1825122 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=acNwiT+K; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RWVwN5Qwlz1yg8 for ; Thu, 24 Aug 2023 14:48:52 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=acNwiT+K; dkim-atps=neutral Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4RWVwN4MRbz3cGJ for ; Thu, 24 Aug 2023 14:48:52 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=acNwiT+K; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=canonical.com (client-ip=185.125.188.120; helo=smtp-relay-canonical-0.canonical.com; envelope-from=kai.heng.feng@canonical.com; receiver=lists.ozlabs.org) Received: from smtp-relay-canonical-0.canonical.com (smtp-relay-canonical-0.canonical.com [185.125.188.120]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4RWVv74H6Dz2yst for ; Thu, 24 Aug 2023 14:47:47 +1000 (AEST) Received: from localhost.localdomain (unknown [10.101.196.174]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-0.canonical.com (Postfix) with ESMTPSA id 528C6401C0; Thu, 24 Aug 2023 04:47:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1692852464; bh=gzh8CFjOP3xGUTmBskUbhLru7uSiTvUwnZrR616fQVw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=acNwiT+K3CAJqMILNvkz//RH7LxUzOHRB996jP69N1l3uflJPrHjRjQOjO0XK3D6q eQKbcLJBxZsYJDlfN3NBUIlru/zxfRIvwMKzSVDRrnGYhaqJLgWIE2VomepbQcYW3O h00GK0E2KDuc8TVpJMqQlslavRm2djPPBmdQgpPdwL7rDLrmOoDlX7dWzbCqsYwm40 wBMWWYBlurTE7kLt+dZH1oZgO2eTgJH/vXXu2Afm6bzBDpjcW3snoWBWRKw+IJtEMG q/+or3+ZdhMi3L2WfyHs3qRmtPtL/Hx0P11qICmXSDHFXeHC+30GU44+j15jkSxI66 LX06IKU95LZYg== From: Kai-Heng Feng To: bhelgaas@google.com Subject: [PATCH 3/3] PCI/DPC: Disable DPC service on suspend Date: Thu, 24 Aug 2023 12:46:45 +0800 Message-Id: <20230824044645.423378-3-kai.heng.feng@canonical.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230824044645.423378-1-kai.heng.feng@canonical.com> References: <20230824044645.423378-1-kai.heng.feng@canonical.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: sathyanarayanan.kuppuswamy@linux.intel.com, linuxppc-dev@lists.ozlabs.org, Mahesh J Salgaonkar , linux-kernel@vger.kernel.org, koba.ko@canonical.com, Kai-Heng Feng , Oliver O'Halloran , linux-pci@vger.kernel.org, mika.westerberg@linux.intel.com Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" When the power rail gets cut off, the hardware can create some electric noise on the link that triggers AER. If IRQ is shared between AER with PME, such AER noise will cause a spurious wakeup on system suspend. When the power rail gets back, the firmware of the device resets itself and can create unexpected behavior like sending PTM messages. For this case, the driver will always be too late to toggle off features should be disabled. As Per PCIe Base Spec 5.0, section 5.2, titled "Link State Power Management", TLP and DLLP transmission are disabled for a Link in L2/L3 Ready (D3hot), L2 (D3cold with aux power) and L3 (D3cold) states. So if the power will be turned off during suspend process, disable DPC service and re-enable it during the resume process. This should not affect the basic functionality. Since DPC depends on AER to function, also disable DPC. Link: https://bugzilla.kernel.org/show_bug.cgi?id=209149 Link: https://bugzilla.kernel.org/show_bug.cgi?id=216295 v7: - Wording. - Disable DPC completely (again) if power will be turned off v6: v5: - Wording. v4: v3: - No change. v2: - Only disable DPC IRQ. - No more check on PME IRQ#. Signed-off-by: Kai-Heng Feng --- drivers/pci/pcie/dpc.c | 56 ++++++++++++++++++++++++++++++++++-------- 1 file changed, 46 insertions(+), 10 deletions(-) diff --git a/drivers/pci/pcie/dpc.c b/drivers/pci/pcie/dpc.c index 3ceed8e3de41..73426addb2f1 100644 --- a/drivers/pci/pcie/dpc.c +++ b/drivers/pci/pcie/dpc.c @@ -13,6 +13,7 @@ #include #include #include +#include #include "portdrv.h" #include "../pci.h" @@ -347,13 +348,34 @@ void pci_dpc_init(struct pci_dev *pdev) } } +static void dpc_enable(struct pcie_device *dev) +{ + struct pci_dev *pdev = dev->port; + u16 ctl; + + pci_read_config_word(pdev, pdev->dpc_cap + PCI_EXP_DPC_CTL, &ctl); + + ctl = (ctl & 0xfff4) | PCI_EXP_DPC_CTL_EN_FATAL | PCI_EXP_DPC_CTL_INT_EN; + pci_write_config_word(pdev, pdev->dpc_cap + PCI_EXP_DPC_CTL, ctl); +} + +static void dpc_disable(struct pcie_device *dev) +{ + struct pci_dev *pdev = dev->port; + u16 ctl; + + pci_read_config_word(pdev, pdev->dpc_cap + PCI_EXP_DPC_CTL, &ctl); + ctl &= ~(PCI_EXP_DPC_CTL_EN_FATAL | PCI_EXP_DPC_CTL_INT_EN); + pci_write_config_word(pdev, pdev->dpc_cap + PCI_EXP_DPC_CTL, ctl); +} + #define FLAG(x, y) (((x) & (y)) ? '+' : '-') static int dpc_probe(struct pcie_device *dev) { struct pci_dev *pdev = dev->port; struct device *device = &dev->device; int status; - u16 ctl, cap; + u16 cap; if (!pcie_aer_is_native(pdev) && !pcie_ports_dpc_native) return -ENOTSUPP; @@ -368,10 +390,7 @@ static int dpc_probe(struct pcie_device *dev) } pci_read_config_word(pdev, pdev->dpc_cap + PCI_EXP_DPC_CAP, &cap); - pci_read_config_word(pdev, pdev->dpc_cap + PCI_EXP_DPC_CTL, &ctl); - - ctl = (ctl & 0xfff4) | PCI_EXP_DPC_CTL_EN_FATAL | PCI_EXP_DPC_CTL_INT_EN; - pci_write_config_word(pdev, pdev->dpc_cap + PCI_EXP_DPC_CTL, ctl); + dpc_enable(dev); pci_info(pdev, "enabled with IRQ %d\n", dev->irq); pci_info(pdev, "error containment capabilities: Int Msg #%d, RPExt%c PoisonedTLP%c SwTrigger%c RP PIO Log %d, DL_ActiveErr%c\n", @@ -384,14 +403,29 @@ static int dpc_probe(struct pcie_device *dev) return status; } -static void dpc_remove(struct pcie_device *dev) +static int dpc_suspend(struct pcie_device *dev) { struct pci_dev *pdev = dev->port; - u16 ctl; - pci_read_config_word(pdev, pdev->dpc_cap + PCI_EXP_DPC_CTL, &ctl); - ctl &= ~(PCI_EXP_DPC_CTL_EN_FATAL | PCI_EXP_DPC_CTL_INT_EN); - pci_write_config_word(pdev, pdev->dpc_cap + PCI_EXP_DPC_CTL, ctl); + if (pci_ancestor_pr3_present(pdev) || pm_suspend_via_firmware()) + dpc_disable(dev); + + return 0; +} + +static int dpc_resume(struct pcie_device *dev) +{ + struct pci_dev *pdev = dev->port; + + if (pci_ancestor_pr3_present(pdev) || pm_resume_via_firmware()) + dpc_enable(dev); + + return 0; +} + +static void dpc_remove(struct pcie_device *dev) +{ + dpc_disable(dev); } static struct pcie_port_service_driver dpcdriver = { @@ -399,6 +433,8 @@ static struct pcie_port_service_driver dpcdriver = { .port_type = PCIE_ANY_PORT, .service = PCIE_PORT_SERVICE_DPC, .probe = dpc_probe, + .suspend = dpc_suspend, + .resume = dpc_resume, .remove = dpc_remove, };