diff mbox series

[v3,3/3] PCI/ASPM: Fix L1SS parameters & only enable supported features when enable link state

Message ID 20240207111854.576402-2-jhp@endlessos.org
State New
Headers show
Series [v3,1/3] PCI: vmd: Enable PCI PM's L1 substates of remapped PCIe Root Port and NVMe | expand

Commit Message

Jian-Hong Pan Feb. 7, 2024, 11:18 a.m. UTC 
The original __pci_enable_link_state() configs the links directly without:
* Check the L1 substates features which are supported, or not
* Calculate & program related parameters for L1.2, such as T_POWER_ON,
  Common_Mode_Restore_Time, and LTR_L1.2_THRESHOLD

This leads some supported L1 PM substates of the link between VMD remapped
PCIe Root Port and NVMe get wrong configs when a caller tries to enabled
them.

Here is a failed example on ASUS B1400CEAE with enabled VMD:

Capabilities: [900 v1] L1 PM Substates
        L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- L1_PM_Substates+
                  PortCommonModeRestoreTime=32us PortTPowerOnTime=10us
        L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1-
                   T_CommonMode=0us LTR1.2_Threshold=0ns
        L1SubCtl2: T_PwrOn=10us

This patch initializes the link's L1 PM substates to get the supported
features and programs relating paramters, if some of them are going to be
enabled in __pci_enable_link_state(). Then, enables the L1 PM substates if
the caller intends to enable them and they are supported.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=218394
Signed-off-by: Jian-Hong Pan <jhp@endlessos.org>
---
v2:
- Prepare the PCIe LTR parameters before enable L1 Substates

v3:
- Only enable supported features for the L1 Substates part

 drivers/pci/pcie/aspm.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

Comments

Ilpo Järvinen Feb. 7, 2024, 12:54 p.m. UTC | #1 
On Wed, 7 Feb 2024, Jian-Hong Pan wrote:

> The original __pci_enable_link_state() configs the links directly without:
> * Check the L1 substates features which are supported, or not
> * Calculate & program related parameters for L1.2, such as T_POWER_ON,
>   Common_Mode_Restore_Time, and LTR_L1.2_THRESHOLD
> 
> This leads some supported L1 PM substates of the link between VMD remapped
> PCIe Root Port and NVMe get wrong configs when a caller tries to enabled
> them.
> 
> Here is a failed example on ASUS B1400CEAE with enabled VMD:
> 
> Capabilities: [900 v1] L1 PM Substates
>         L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- L1_PM_Substates+
>                   PortCommonModeRestoreTime=32us PortTPowerOnTime=10us
>         L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1-
>                    T_CommonMode=0us LTR1.2_Threshold=0ns
>         L1SubCtl2: T_PwrOn=10us
> 
> This patch initializes the link's L1 PM substates to get the supported
> features and programs relating paramters, if some of them are going to be
> enabled in __pci_enable_link_state(). Then, enables the L1 PM substates if
> the caller intends to enable them and they are supported.
> 
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=218394
> Signed-off-by: Jian-Hong Pan <jhp@endlessos.org>
> ---
> v2:
> - Prepare the PCIe LTR parameters before enable L1 Substates
> 
> v3:
> - Only enable supported features for the L1 Substates part
> 
>  drivers/pci/pcie/aspm.c | 12 +++++++-----
>  1 file changed, 7 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
> index a39d2ee744cb..c866971cae70 100644
> --- a/drivers/pci/pcie/aspm.c
> +++ b/drivers/pci/pcie/aspm.c
> @@ -1389,14 +1389,16 @@ static int __pci_enable_link_state(struct pci_dev *pdev, int state, bool locked)
>  		link->aspm_default |= ASPM_STATE_L0S;
>  	if (state & PCIE_LINK_STATE_L1)
>  		link->aspm_default |= ASPM_STATE_L1;
> -	/* L1 PM substates require L1 */
> -	if (state & PCIE_LINK_STATE_L1_1)
> +	if (state & ASPM_STATE_L1_2_MASK)
> +		aspm_l1ss_init(link);

Your commit message didn't explain why you need to add this call here, why 
isn't the existing call to aspm_l1ss_init() enough to initialize what is 
needed?

> +	/* L1 PM substates require L1 and should be in supported list */
> +	if (state & link->aspm_support & PCIE_LINK_STATE_L1_1)
>  		link->aspm_default |= ASPM_STATE_L1_1 | ASPM_STATE_L1;
> -	if (state & PCIE_LINK_STATE_L1_2)
> +	if (state & link->aspm_support & PCIE_LINK_STATE_L1_2)
>  		link->aspm_default |= ASPM_STATE_L1_2 | ASPM_STATE_L1;
> -	if (state & PCIE_LINK_STATE_L1_1_PCIPM)
> +	if (state & link->aspm_support & PCIE_LINK_STATE_L1_1_PCIPM)
>  		link->aspm_default |= ASPM_STATE_L1_1_PCIPM | ASPM_STATE_L1;
> -	if (state & PCIE_LINK_STATE_L1_2_PCIPM)
> +	if (state & link->aspm_support & PCIE_LINK_STATE_L1_2_PCIPM)
>  		link->aspm_default |= ASPM_STATE_L1_2_PCIPM | ASPM_STATE_L1;

It would be simpler and cleaner to just AND once with link->aspm_support 
to clear unsupported bits from state rather than doing it on every line 
like that.
David E. Box Feb. 7, 2024, 4:02 p.m. UTC | #2 
On Wed, 2024-02-07 at 19:18 +0800, Jian-Hong Pan wrote:
> The original __pci_enable_link_state() configs the links directly without:
> * Check the L1 substates features which are supported, or not
> * Calculate & program related parameters for L1.2, such as T_POWER_ON,
>   Common_Mode_Restore_Time, and LTR_L1.2_THRESHOLD
> 
> This leads some supported L1 PM substates of the link between VMD remapped
> PCIe Root Port and NVMe get wrong configs when a caller tries to enabled
> them.
> 
> Here is a failed example on ASUS B1400CEAE with enabled VMD:
> 
> Capabilities: [900 v1] L1 PM Substates
>         L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1-
> L1_PM_Substates+
>                   PortCommonModeRestoreTime=32us PortTPowerOnTime=10us
>         L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1-
>                    T_CommonMode=0us LTR1.2_Threshold=0ns
>         L1SubCtl2: T_PwrOn=10us
> 
> This patch initializes the link's L1 PM substates to get the supported
> features and programs relating paramters, if some of them are going to be
> enabled in __pci_enable_link_state(). Then, enables the L1 PM substates if
> the caller intends to enable them and they are supported.
> 
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=218394
> Signed-off-by: Jian-Hong Pan <jhp@endlessos.org>
> ---
> v2:
> - Prepare the PCIe LTR parameters before enable L1 Substates
> 
> v3:
> - Only enable supported features for the L1 Substates part
> 
>  drivers/pci/pcie/aspm.c | 12 +++++++-----
>  1 file changed, 7 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
> index a39d2ee744cb..c866971cae70 100644
> --- a/drivers/pci/pcie/aspm.c
> +++ b/drivers/pci/pcie/aspm.c
> @@ -1389,14 +1389,16 @@ static int __pci_enable_link_state(struct pci_dev
> *pdev, int state, bool locked)
>                 link->aspm_default |= ASPM_STATE_L0S;
>         if (state & PCIE_LINK_STATE_L1)
>                 link->aspm_default |= ASPM_STATE_L1;
> -       /* L1 PM substates require L1 */
> -       if (state & PCIE_LINK_STATE_L1_1)
> +       if (state & ASPM_STATE_L1_2_MASK)
> +               aspm_l1ss_init(link);

This mixes ASPM_STATE flags with PCIE_LINK_STATE register mapping. This may work
but I don't know if it's intended to. Rather do,

    if (link->default & ASPM_STATE_L1_2_MASK)

after collecting all of the states to be enabled.

I understand that you are calling aspm_l1ss_init() to do the L1.2 calculations
but it does more than this that you don't need. Maybe it would be more
appropriate to call aspm_calc_l12_info() directly through an additional function
that finds the parent and determines both ends of the link support L1SS.

> +       /* L1 PM substates require L1 and should be in supported list */
> +       if (state & link->aspm_support & PCIE_LINK_STATE_L1_1)
>                 link->aspm_default |= ASPM_STATE_L1_1 | ASPM_STATE_L1;
> -       if (state & PCIE_LINK_STATE_L1_2)
> +       if (state & link->aspm_support & PCIE_LINK_STATE_L1_2)
>                 link->aspm_default |= ASPM_STATE_L1_2 | ASPM_STATE_L1;
> -       if (state & PCIE_LINK_STATE_L1_1_PCIPM)
> +       if (state & link->aspm_support & PCIE_LINK_STATE_L1_1_PCIPM)
>                 link->aspm_default |= ASPM_STATE_L1_1_PCIPM | ASPM_STATE_L1;
> -       if (state & PCIE_LINK_STATE_L1_2_PCIPM)
> +       if (state & link->aspm_support & PCIE_LINK_STATE_L1_2_PCIPM)
>                 link->aspm_default |= ASPM_STATE_L1_2_PCIPM | ASPM_STATE_L1;
>         pcie_config_aspm_link(link, policy_to_aspm_state(link));
>  

I don't think these changes are necessary. pcie_config_aspm_link() already
checks link->aspm_capable which was initialized from link->aspm_support.

David
Kuppuswamy Sathyanarayanan Feb. 7, 2024, 5:19 p.m. UTC | #3 
On 2/7/24 3:18 AM, Jian-Hong Pan wrote:
> The original __pci_enable_link_state() configs the links directly without:

Since you are referring to the current version of this function, just start
with "Currently, "

> * Check the L1 substates features which are supported, or not
> * Calculate & program related parameters for L1.2, such as T_POWER_ON,
>   Common_Mode_Restore_Time, and LTR_L1.2_THRESHOLD
>
> This leads some supported L1 PM substates of the link between VMD remapped
> PCIe Root Port and NVMe get wrong configs when a caller tries to enabled
> them.
>
> Here is a failed example on ASUS B1400CEAE with enabled VMD:
>
> Capabilities: [900 v1] L1 PM Substates
>         L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- L1_PM_Substates+
>                   PortCommonModeRestoreTime=32us PortTPowerOnTime=10us
>         L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1-
>                    T_CommonMode=0us LTR1.2_Threshold=0ns
>         L1SubCtl2: T_PwrOn=10us
>
> This patch initializes the link's L1 PM substates to get the supported
> features and programs relating paramters, if some of them are going to be
> enabled in __pci_enable_link_state(). Then, enables the L1 PM substates if
> the caller intends to enable them and they are supported.
>
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=218394
> Signed-off-by: Jian-Hong Pan <jhp@endlessos.org>
> ---
> v2:
> - Prepare the PCIe LTR parameters before enable L1 Substates
>
> v3:
> - Only enable supported features for the L1 Substates part
>
>  drivers/pci/pcie/aspm.c | 12 +++++++-----
>  1 file changed, 7 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
> index a39d2ee744cb..c866971cae70 100644
> --- a/drivers/pci/pcie/aspm.c
> +++ b/drivers/pci/pcie/aspm.c
> @@ -1389,14 +1389,16 @@ static int __pci_enable_link_state(struct pci_dev *pdev, int state, bool locked)
>  		link->aspm_default |= ASPM_STATE_L0S;
>  	if (state & PCIE_LINK_STATE_L1)
>  		link->aspm_default |= ASPM_STATE_L1;
> -	/* L1 PM substates require L1 */
> -	if (state & PCIE_LINK_STATE_L1_1)
> +	if (state & ASPM_STATE_L1_2_MASK)
> +		aspm_l1ss_init(link);

This is a new change. Explain why you are doing it?

> +	/* L1 PM substates require L1 and should be in supported list */
> +	if (state & link->aspm_support & PCIE_LINK_STATE_L1_1)
>  		link->aspm_default |= ASPM_STATE_L1_1 | ASPM_STATE_L1;
> -	if (state & PCIE_LINK_STATE_L1_2)
> +	if (state & link->aspm_support & PCIE_LINK_STATE_L1_2)
>  		link->aspm_default |= ASPM_STATE_L1_2 | ASPM_STATE_L1;
> -	if (state & PCIE_LINK_STATE_L1_1_PCIPM)
> +	if (state & link->aspm_support & PCIE_LINK_STATE_L1_1_PCIPM)
>  		link->aspm_default |= ASPM_STATE_L1_1_PCIPM | ASPM_STATE_L1;
> -	if (state & PCIE_LINK_STATE_L1_2_PCIPM)
> +	if (state & link->aspm_support & PCIE_LINK_STATE_L1_2_PCIPM)
>  		link->aspm_default |= ASPM_STATE_L1_2_PCIPM | ASPM_STATE_L1;
>  	pcie_config_aspm_link(link, policy_to_aspm_state(link));
>
Jian-Hong Pan Feb. 16, 2024, 3:13 a.m. UTC | #4 
David E. Box <david.e.box@linux.intel.com> 於 2024年2月8日 週四 上午12:02寫道：
>
> On Wed, 2024-02-07 at 19:18 +0800, Jian-Hong Pan wrote:
> > The original __pci_enable_link_state() configs the links directly without:
> > * Check the L1 substates features which are supported, or not
> > * Calculate & program related parameters for L1.2, such as T_POWER_ON,
> >   Common_Mode_Restore_Time, and LTR_L1.2_THRESHOLD
> >
> > This leads some supported L1 PM substates of the link between VMD remapped
> > PCIe Root Port and NVMe get wrong configs when a caller tries to enabled
> > them.
> >
> > Here is a failed example on ASUS B1400CEAE with enabled VMD:
> >
> > Capabilities: [900 v1] L1 PM Substates
> >         L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1-
> > L1_PM_Substates+
> >                   PortCommonModeRestoreTime=32us PortTPowerOnTime=10us
> >         L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1-
> >                    T_CommonMode=0us LTR1.2_Threshold=0ns
> >         L1SubCtl2: T_PwrOn=10us
> >
> > This patch initializes the link's L1 PM substates to get the supported
> > features and programs relating paramters, if some of them are going to be
> > enabled in __pci_enable_link_state(). Then, enables the L1 PM substates if
> > the caller intends to enable them and they are supported.
> >
> > Link: https://bugzilla.kernel.org/show_bug.cgi?id=218394
> > Signed-off-by: Jian-Hong Pan <jhp@endlessos.org>
> > ---
> > v2:
> > - Prepare the PCIe LTR parameters before enable L1 Substates
> >
> > v3:
> > - Only enable supported features for the L1 Substates part
> >
> >  drivers/pci/pcie/aspm.c | 12 +++++++-----
> >  1 file changed, 7 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
> > index a39d2ee744cb..c866971cae70 100644
> > --- a/drivers/pci/pcie/aspm.c
> > +++ b/drivers/pci/pcie/aspm.c
> > @@ -1389,14 +1389,16 @@ static int __pci_enable_link_state(struct pci_dev
> > *pdev, int state, bool locked)
> >                 link->aspm_default |= ASPM_STATE_L0S;
> >         if (state & PCIE_LINK_STATE_L1)
> >                 link->aspm_default |= ASPM_STATE_L1;
> > -       /* L1 PM substates require L1 */
> > -       if (state & PCIE_LINK_STATE_L1_1)
> > +       if (state & ASPM_STATE_L1_2_MASK)
> > +               aspm_l1ss_init(link);
>
> This mixes ASPM_STATE flags with PCIE_LINK_STATE register mapping. This may work
> but I don't know if it's intended to. Rather do,
>
>     if (link->default & ASPM_STATE_L1_2_MASK)
>
> after collecting all of the states to be enabled.
>
> I understand that you are calling aspm_l1ss_init() to do the L1.2 calculations
> but it does more than this that you don't need. Maybe it would be more
> appropriate to call aspm_calc_l12_info() directly through an additional function
> that finds the parent and determines both ends of the link support L1SS.

After think & check twice, focusing on fixing the L1.2 parameters makes sense.

I am preparing a new patch.  Thanks!

Jian-Hong Pan

> > +       /* L1 PM substates require L1 and should be in supported list */
> > +       if (state & link->aspm_support & PCIE_LINK_STATE_L1_1)
> >                 link->aspm_default |= ASPM_STATE_L1_1 | ASPM_STATE_L1;
> > -       if (state & PCIE_LINK_STATE_L1_2)
> > +       if (state & link->aspm_support & PCIE_LINK_STATE_L1_2)
> >                 link->aspm_default |= ASPM_STATE_L1_2 | ASPM_STATE_L1;
> > -       if (state & PCIE_LINK_STATE_L1_1_PCIPM)
> > +       if (state & link->aspm_support & PCIE_LINK_STATE_L1_1_PCIPM)
> >                 link->aspm_default |= ASPM_STATE_L1_1_PCIPM | ASPM_STATE_L1;
> > -       if (state & PCIE_LINK_STATE_L1_2_PCIPM)
> > +       if (state & link->aspm_support & PCIE_LINK_STATE_L1_2_PCIPM)
> >                 link->aspm_default |= ASPM_STATE_L1_2_PCIPM | ASPM_STATE_L1;
> >         pcie_config_aspm_link(link, policy_to_aspm_state(link));
> >
>
> I don't think these changes are necessary. pcie_config_aspm_link() already
> checks link->aspm_capable which was initialized from link->aspm_support.
>
> David
diff mbox series

Patch

diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
index a39d2ee744cb..c866971cae70 100644
--- a/drivers/pci/pcie/aspm.c
+++ b/drivers/pci/pcie/aspm.c
@@ -1389,14 +1389,16 @@  static int __pci_enable_link_state(struct pci_dev *pdev, int state, bool locked)
 		link->aspm_default |= ASPM_STATE_L0S;
 	if (state & PCIE_LINK_STATE_L1)
 		link->aspm_default |= ASPM_STATE_L1;
-	/* L1 PM substates require L1 */
-	if (state & PCIE_LINK_STATE_L1_1)
+	if (state & ASPM_STATE_L1_2_MASK)
+		aspm_l1ss_init(link);
+	/* L1 PM substates require L1 and should be in supported list */
+	if (state & link->aspm_support & PCIE_LINK_STATE_L1_1)
 		link->aspm_default |= ASPM_STATE_L1_1 | ASPM_STATE_L1;
-	if (state & PCIE_LINK_STATE_L1_2)
+	if (state & link->aspm_support & PCIE_LINK_STATE_L1_2)
 		link->aspm_default |= ASPM_STATE_L1_2 | ASPM_STATE_L1;
-	if (state & PCIE_LINK_STATE_L1_1_PCIPM)
+	if (state & link->aspm_support & PCIE_LINK_STATE_L1_1_PCIPM)
 		link->aspm_default |= ASPM_STATE_L1_1_PCIPM | ASPM_STATE_L1;
-	if (state & PCIE_LINK_STATE_L1_2_PCIPM)
+	if (state & link->aspm_support & PCIE_LINK_STATE_L1_2_PCIPM)
 		link->aspm_default |= ASPM_STATE_L1_2_PCIPM | ASPM_STATE_L1;
 	pcie_config_aspm_link(link, policy_to_aspm_state(link));