From patchwork Mon Oct 21 09:28:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: En-Wei Wu X-Patchwork-Id: 1999817 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=185.125.189.65; helo=lists.ubuntu.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=patchwork.ozlabs.org) Received: from lists.ubuntu.com (lists.ubuntu.com [185.125.189.65]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XX93d4bskz1xwK for ; Mon, 21 Oct 2024 20:28:45 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=lists.ubuntu.com) by lists.ubuntu.com with esmtp (Exim 4.86_2) (envelope-from ) id 1t2oiN-0000wy-5w; Mon, 21 Oct 2024 09:28:39 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by lists.ubuntu.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1t2oiJ-0000ve-OD for kernel-team@lists.ubuntu.com; Mon, 21 Oct 2024 09:28:35 +0000 Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 306BC3F1E5 for ; Mon, 21 Oct 2024 09:28:35 +0000 (UTC) Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-4316138aff6so24543355e9.1 for ; Mon, 21 Oct 2024 02:28:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729502914; x=1730107714; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Nespvzk6miXFbzTWd6++QzFxnRBgMjPCMiJL3+f77/c=; b=VO3fUzuTunh8msCJDnvqKWQcOggEhwWsHd9q1TR7V1jQPVx1laCSCwrQ7p0muYu2E6 alBAFG4Jo1MTDjrmUNhU0rtJubRUmKTR4Pr3LZXmWhRzx55CJbsM9rAeQzULcnMvHcAX nm3t5z90vw9/hI4aihxwrQxc/WRLbwdhh+gRGcvZ0shkb+0ylsxBHn7fL6qyyIW4seF8 hNbscn9+VC8/vuojE3PAr2HvlQF6hYz8zdtVhfppbjd2j/h7ZiacUKKBGBc1NaxEUDrz jfwRLF8LLF602Qn9gaC2yWlT56jlJ+1O6tQvrheQVR89BdxD7bra0OM50n1GCKjvniBj E6dg== X-Gm-Message-State: AOJu0YxZ3cpTKPuK8P4i2JGGc0TauPVpuVVykX3su8gZImS/hvl2U2E1 w8YhM0g5FMfJXfYjdr42CKt08eVAjGC7Z5bY9JEmTDR9tdySO1pNCIapqSluYFBGy8RAYTOrPVx 4YhbWs8bqRN2IaTHBXr64ptFX3XMQ1g7IJrpKvVEU8BvveKrHDMYxKBgaV4+7TV/XEB7B+J62dC 6NFZmh3thFb/CI X-Received: by 2002:a05:600c:3b25:b0:42f:8515:e4a8 with SMTP id 5b1f17b1804b1-431616331e8mr87166775e9.6.1729502914599; Mon, 21 Oct 2024 02:28:34 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHyPBpauxgG6PnXhcAy3ON/JQqN0R+ut8Mf6dstj+9AfZ8PiwtC9fvQm+1y9hSd5YkuKdTjzQ== X-Received: by 2002:a05:600c:3b25:b0:42f:8515:e4a8 with SMTP id 5b1f17b1804b1-431616331e8mr87166575e9.6.1729502914181; Mon, 21 Oct 2024 02:28:34 -0700 (PDT) Received: from rickywu0421-ThinkPad-X1-Carbon-Gen-11.. (dhcp-138-246-3-198.dynamic.eduroam.mwn.de. [138.246.3.198]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-43160682586sm97516225e9.1.2024.10.21.02.28.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Oct 2024 02:28:33 -0700 (PDT) From: En-Wei Wu To: kernel-team@lists.ubuntu.com Subject: [SRU][N][PATCH 1/3] UBUNTU: SAUCE: PCI: vmd: Set PCI devices to D0 before enable PCI PM's L1 substates Date: Mon, 21 Oct 2024 11:28:30 +0200 Message-ID: <20241021092832.475663-2-en-wei.wu@canonical.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241021092832.475663-1-en-wei.wu@canonical.com> References: <20241021092832.475663-1-en-wei.wu@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Jian-Hong Pan BugLink: https://bugs.launchpad.net/bugs/2085092 The remapped PCIe Root Port and the child device have PCI PM L1 substates capability, but they are disabled originally. Here is a failed example on ASUS B1400CEAE: Capabilities: [900 v1] L1 PM Substates L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- L1_PM_Substates+ PortCommonModeRestoreTime=32us PortTPowerOnTime=10us L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- T_CommonMode=0us LTR1.2_Threshold=101376ns L1SubCtl2: T_PwrOn=50us Power on all of the VMD remapped PCI devices to D0 before enable PCI-PM L1 PM Substates by following "PCIe r6.0, sec 5.5.4". Link: https://bugzilla.kernel.org/show_bug.cgi?id=218394 Signed-off-by: Jian-Hong Pan Reviewed-by: Kuppuswamy Sathyanarayanan (backported from https://lore.kernel.org/linux-pci/20241001083438.10070-2-jhp@endlessos.org) [En-Wei - need to fix this issue ASAP and the patch hasn't been merged yet] Signed-off-by: En-Wei Wu --- drivers/pci/controller/vmd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c index e5a0c6c57524..732bc3b7e69f 100644 --- a/drivers/pci/controller/vmd.c +++ b/drivers/pci/controller/vmd.c @@ -750,7 +750,7 @@ static int vmd_pm_enable_quirk(struct pci_dev *pdev, void *userdata) if (!(features & VMD_FEAT_BIOS_PM_QUIRK)) return 0; - + pdev->aspm_os_control = 1; pci_enable_link_state_locked(pdev, PCIE_LINK_STATE_ALL); From patchwork Mon Oct 21 09:28:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: En-Wei Wu X-Patchwork-Id: 1999816 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=185.125.189.65; helo=lists.ubuntu.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=patchwork.ozlabs.org) Received: from lists.ubuntu.com (lists.ubuntu.com [185.125.189.65]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XX93c43dCz1xwb for ; Mon, 21 Oct 2024 20:28:44 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=lists.ubuntu.com) by lists.ubuntu.com with esmtp (Exim 4.86_2) (envelope-from ) id 1t2oiM-0000wE-0u; Mon, 21 Oct 2024 09:28:38 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by lists.ubuntu.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1t2oiJ-0000vq-Vg for kernel-team@lists.ubuntu.com; Mon, 21 Oct 2024 09:28:35 +0000 Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id C046B3F1E5 for ; Mon, 21 Oct 2024 09:28:35 +0000 (UTC) Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-431604a3b47so24683485e9.3 for ; Mon, 21 Oct 2024 02:28:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729502915; x=1730107715; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LnHBxZK0D8I+0TglovCKg8DLYqgjNCHiU8DQzfBW+uc=; b=EJRkCwROfeiru92Z6elMzmaZ2JoMaazfTiiik4YWd3LyAq9PWBbSKilQWXtDfLx5uu DgMc9dTJXt9fXc9QTD3SXqV+2FZgtltQxnE6VReE8czA5rydepKt1Vu+cofz/vUHLBZu gwZmeDJTyrNrrw0Kj9FPxFSl2hGLv27r4GSd42+Vvp/4J2lFwbguZUmnf9Tvy/8ysQrw rJNZB4kR2v8sdjJlPTN2/gIOx9UU2ThViDDXQhln+5+G470KcDIuJ2UGTHNGnf+kBbt+ 0QfgUAQPqf4qPAHYLVVEB398zIB0NZ3Fwkc13fqrLMW4egg/0igSt/tJnHbiXNx7VTHg O04A== X-Gm-Message-State: AOJu0YxEwMh6z7EWXSSB1WnlhYepOm9Wi9vzvL32gUyyE52pEYh6q1Xo mGhAYBiiVZUkhfjjE/WZlSvHeF9vm8Blov78iTyKVdjQ5/d7nGz91jkaB9VluWdjj8hARS5BAJH KGS0hmJ2BoVEGDWmDR+ENLc9+lRi9o9t/h41V62LGt5XP7wrm1br8dq4kSvgBmsAWqRU7dCKDKY 9LS9YbO6ndOxhD X-Received: by 2002:a05:600c:3b8c:b0:42c:a580:71cf with SMTP id 5b1f17b1804b1-43161693173mr87289395e9.30.1729502914913; Mon, 21 Oct 2024 02:28:34 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH0EHwyALcGSAjWJrC711dAXh+UyrbRnDOx9jsRulzUjY8m3LlSgW6tPZGxf3y1g5msBfHOug== X-Received: by 2002:a05:600c:3b8c:b0:42c:a580:71cf with SMTP id 5b1f17b1804b1-43161693173mr87289195e9.30.1729502914561; Mon, 21 Oct 2024 02:28:34 -0700 (PDT) Received: from rickywu0421-ThinkPad-X1-Carbon-Gen-11.. (dhcp-138-246-3-198.dynamic.eduroam.mwn.de. [138.246.3.198]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-43160682586sm97516225e9.1.2024.10.21.02.28.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Oct 2024 02:28:34 -0700 (PDT) From: En-Wei Wu To: kernel-team@lists.ubuntu.com Subject: [SRU][N][PATCH 2/3] UBUNTU: SAUCE: PCI/ASPM: Add notes about enabling PCI-PM L1SS to pci_enable_link_state(_locked) Date: Mon, 21 Oct 2024 11:28:31 +0200 Message-ID: <20241021092832.475663-3-en-wei.wu@canonical.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241021092832.475663-1-en-wei.wu@canonical.com> References: <20241021092832.475663-1-en-wei.wu@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Jian-Hong Pan BugLink: https://bugs.launchpad.net/bugs/2085092 According to "PCIe r6.0, sec 5.5.4", add note about D0 requirement in pci_enable_link_state() kernel-doc. Signed-off-by: Jian-Hong Pan Acked-by: Bjorn Helgaas Reviewed-by: Kuppuswamy Sathyanarayanan (cherry picked from https://lore.kernel.org/linux-pci/20241001083438.10070-2-jhp@endlessos.org) [En-Wei - need to fix this issue ASAP and the patch hasn't been merged yet] Signed-off-by: En-Wei Wu --- drivers/pci/pcie/aspm.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c index 138ff0efa6dd..182226ce5c4e 100644 --- a/drivers/pci/pcie/aspm.c +++ b/drivers/pci/pcie/aspm.c @@ -1450,6 +1450,9 @@ static int __pci_enable_link_state(struct pci_dev *pdev, int state, bool locked) * touch the LNKCTL register. Also note that this does not enable states * disabled by pci_disable_link_state(). Return 0 or a negative errno. * + * Note: Ensure devices are in D0 before enabling PCI-PM L1 PM Substates, per + * PCIe r6.0, sec 5.5.4. + * * @pdev: PCI device * @state: Mask of ASPM link states to enable */ @@ -1466,6 +1469,9 @@ EXPORT_SYMBOL(pci_enable_link_state); * can't touch the LNKCTL register. Also note that this does not enable states * disabled by pci_disable_link_state(). Return 0 or a negative errno. * + * Note: Ensure devices are in D0 before enabling PCI-PM L1 PM Substates, per + * PCIe r6.0, sec 5.5.4. + * * @pdev: PCI device * @state: Mask of ASPM link states to enable * From patchwork Mon Oct 21 09:28:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: En-Wei Wu X-Patchwork-Id: 1999818 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=185.125.189.65; helo=lists.ubuntu.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=patchwork.ozlabs.org) Received: from lists.ubuntu.com (lists.ubuntu.com [185.125.189.65]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XX93d6hFJz1xwb for ; Mon, 21 Oct 2024 20:28:45 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=lists.ubuntu.com) by lists.ubuntu.com with esmtp (Exim 4.86_2) (envelope-from ) id 1t2oiN-0000x7-Bd; Mon, 21 Oct 2024 09:28:39 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by lists.ubuntu.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1t2oiK-0000vy-Hy for kernel-team@lists.ubuntu.com; Mon, 21 Oct 2024 09:28:36 +0000 Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 5F3F43F5B4 for ; Mon, 21 Oct 2024 09:28:36 +0000 (UTC) Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-4315d98a873so27682675e9.1 for ; Mon, 21 Oct 2024 02:28:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729502915; x=1730107715; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nUq7RvE2k7PMFYCeKCBjxWjtnpyOfuVm6ZeODa4sGho=; b=RKe8UAbqlncHMgX3A7w4ABpsIBuIYBXzKRVHtRQHVTlKe3toI/kwoeK8ORiCjxlhvk wwYzkWXSIdp2sG2ahwtWDFxVjlay0zLOE2QsZuq/m+2WRSryARq0mcpMQ5pfA72bpGs3 lPOrmZMt1HUho2Y8oslGsvbMsyclFFTWx8vrOBkx0MnPiIma3IjZdNFOtEpJuUA6pH7e kN7XRAQUJ7F4nOlgtQS7ofMsbYxrCbd97KCR6Hz0GLofRLi6EEGaRrXV4OZAwDovv6oM xz0a//uKndzMpLtz2U5UgIbUTgwYncUFMpT40Ks7SP646FHXjy+7AgGZNi2z2ZqhhzIz 6/tA== X-Gm-Message-State: AOJu0Ywm5uXhAGLnHGZ1g6lHshakU9ZqcN2kr6UZcRgsNbUz8bt9RfvD Z1FSV7kAE7snFkMHSNJjelyZPFd+gdwaac3mMP+gTltMR//23kL2SL16hQ3MYD/AIKtLfRYnQBR A1nYXzwXUhvR3ww+ls1n/kCM3MtEyUm8PwWbvIACwOCvKAcA03mQjVs7ccb61T1Py5W+i77zXtj JKyXB1+3/McBCe X-Received: by 2002:a05:600c:1c9b:b0:431:5632:4497 with SMTP id 5b1f17b1804b1-4316168f1fbmr88591075e9.26.1729502915401; Mon, 21 Oct 2024 02:28:35 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFeruiwDEBiLK/OInF8vDq5bJeF5xL/gOTzddOEnQsvcdimYS/ct7ctgBknC43+clUpiUyodw== X-Received: by 2002:a05:600c:1c9b:b0:431:5632:4497 with SMTP id 5b1f17b1804b1-4316168f1fbmr88590895e9.26.1729502914969; Mon, 21 Oct 2024 02:28:34 -0700 (PDT) Received: from rickywu0421-ThinkPad-X1-Carbon-Gen-11.. (dhcp-138-246-3-198.dynamic.eduroam.mwn.de. [138.246.3.198]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-43160682586sm97516225e9.1.2024.10.21.02.28.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Oct 2024 02:28:34 -0700 (PDT) From: En-Wei Wu To: kernel-team@lists.ubuntu.com Subject: [SRU][N][PATCH 3/3] UBUNTU: SAUCE: PCI/ASPM: Make pci_save_aspm_l1ss_state save both child and parent's L1SS configuration Date: Mon, 21 Oct 2024 11:28:32 +0200 Message-ID: <20241021092832.475663-4-en-wei.wu@canonical.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241021092832.475663-1-en-wei.wu@canonical.com> References: <20241021092832.475663-1-en-wei.wu@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Jian-Hong Pan BugLink: https://bugs.launchpad.net/bugs/2085092 PCI devices' parameters on the VMD bus have been programmed properly originally. But, cleared after pci_reset_bus() and have not been restored correctly. This leads the link's L1.2 between PCIe Root Port and child device gets wrong configs. Here is a failed example on ASUS B1400CEAE with enabled VMD. Both PCIe bridge and NVMe device should have the same LTR1.2_Threshold value. However, they are configured as different values in this case: 10000:e0:06.0 PCI bridge [0604]: Intel Corporation 11th Gen Core Processor PCIe Controller [8086:9a09] (rev 01) (prog-if 00 [Normal decode]) ... Capabilities: [200 v1] L1 PM Substates L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+ PortCommonModeRestoreTime=45us PortTPowerOnTime=50us L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- T_CommonMode=0us LTR1.2_Threshold=0ns L1SubCtl2: T_PwrOn=0us 10000:e1:00.0 Non-Volatile memory controller [0108]: Sandisk Corp WD Blue SN550 NVMe SSD [15b7:5009] (rev 01) (prog-if 02 [NVM Express]) ... Capabilities: [900 v1] L1 PM Substates L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- L1_PM_Substates+ PortCommonModeRestoreTime=32us PortTPowerOnTime=10us L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- T_CommonMode=0us LTR1.2_Threshold=101376ns L1SubCtl2: T_PwrOn=50us Here is VMD mapped PCI device tree: -+-[0000:00]-+-00.0 Intel Corporation Device 9a04 | ... \-[10000:e0]-+-06.0-[e1]----00.0 Sandisk Corp WD Blue SN550 NVMe SSD \-17.0 Intel Corporation Tiger Lake-LP SATA Controller When pci_reset_bus() resets the bus [e1] of the NVMe, it only saves and restores NVMe's state before and after reset. Then, when it restores the NVMe's state, ASPM code restores L1SS for both the parent bridge and the NVMe in pci_restore_aspm_l1ss_state(). The NVMe's L1SS is restored correctly. But, the parent bridge's L1SS is restored with a wrong value 0x0 because the parent bridge's L1SS wasn't saved by pci_save_aspm_l1ss_state() before reset. So, if the PCI device has a parent, make pci_save_aspm_l1ss_state() save the parent's L1SS configuration, too. This is symmetric on pci_restore_aspm_l1ss_state(). Link: https://lore.kernel.org/linux-pci/CAPpJ_eexU0gCHMbXw_z924WxXw0+B6SdS4eG9oGpEX1wmnMLkQ@mail.gmail.com/ Closes: https://bugzilla.kernel.org/show_bug.cgi?id=218394 Fixes: 17423360a27a ("PCI/ASPM: Save L1 PM Substates Capability for suspend/resume") Suggested-by: Ilpo Järvinen Signed-off-by: Jian-Hong Pan Reviewed-by: Ilpo Järvinen Reviewed-by: David E. Box (cherry picked from https://lore.kernel.org/linux-pci/20241001083438.10070-2-jhp@endlessos.org) [En-Wei - need to fix this issue ASAP and the patch hasn't been merged yet] Signed-off-by: En-Wei Wu --- drivers/pci/pcie/aspm.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c index 182226ce5c4e..66e9b7091a30 100644 --- a/drivers/pci/pcie/aspm.c +++ b/drivers/pci/pcie/aspm.c @@ -77,7 +77,7 @@ void pci_configure_aspm_l1ss(struct pci_dev *pdev) ERR_PTR(rc)); } -void pci_save_aspm_l1ss_state(struct pci_dev *pdev) +static void __pci_save_aspm_l1ss_state(struct pci_dev *pdev) { struct pci_cap_saved_state *save_state; u16 l1ss = pdev->l1ss; @@ -99,6 +99,24 @@ void pci_save_aspm_l1ss_state(struct pci_dev *pdev) pci_read_config_dword(pdev, l1ss + PCI_L1SS_CTL1, cap++); } +void pci_save_aspm_l1ss_state(struct pci_dev *pdev) +{ + struct pci_dev *parent; + + __pci_save_aspm_l1ss_state(pdev); + + /* + * To be symmetric on pci_restore_aspm_l1ss_state(), save parent's L1 + * substate configuration, if the parent has not saved state. + */ + if (!pdev->bus || !pdev->bus->self) + return; + + parent = pdev->bus->self; + if (!parent->state_saved) + __pci_save_aspm_l1ss_state(parent); +} + void pci_restore_aspm_l1ss_state(struct pci_dev *pdev) { struct pci_cap_saved_state *pl_save_state, *cl_save_state;