mbox series

[v9,00/10] phy: Add support for Lynx 10G SerDes

Message ID 20221230000139.2846763-1-sean.anderson@seco.com
Headers show
Series phy: Add support for Lynx 10G SerDes | expand

Message

Sean Anderson Dec. 30, 2022, 12:01 a.m. UTC
This adds support for the Lynx 10G SerDes found on the QorIQ T-series
and Layerscape series. Due to limited time and hardware, only support
for the LS1046ARDB and LS1088ARDB is added in this initial series.

This series is based on phy/next, but it requires phylink support. This
is already present for the LS1088A, and it was recently added for the
LS1046A in net-next/master.

Major reconfiguration of baud rate (e.g. 1G->10G) does not work. From my
testing, SerDes register settings appear identical. The issue appears to
be between the PCS and the MAC. The link itself comes up at both ends,
and a mac loopback succeeds. However, a PCS loopback results in dropped
packets. Perhaps there is some undocumented register in the PCS?

I suspect this driver is around 95% complete, but I don't have the
documentation to make it work completely. At the very least it is useful
for two cases:

- Although this is untested, it should support 2.5G SGMII as well as
  1000BASE-KX. The latter needs MAC and PCS support, but the former
  should work out of the box.
- It allows for clock configurations not supported by the RCW. This is
  very useful if you want to use e.g. SRDS_PRTCL_S1=0x3333 and =0x1133
  on the same board. This is because the former setting will use PLL1
  as the 1G reference, but the latter will use PLL1 as the 10G
  reference. Because we can reconfigure the PLLs, it is possible to
  always use PLL1 as the 1G reference.

The final patch in this series depends on [1].

[1] https://lore.kernel.org/netdev/20221227230918.2440351-1-sean.anderson@seco.com/

Changes in v9:
- Add fsl,unused-lanes-reserved to allow for a gradual transition
  between firmware and Linux control of the SerDes
- Change phy-type back to fsl,type, as I was getting the error
    '#phy-cells' is a dependency of 'phy-type'
- Convert some u32s to unsigned long to match arguments
- Switch from round_rate to determine_rate
- Drop explicit reference to reference clock
- Use .parent_names when requesting parents
- Use devm_clk_hw_get_clk to pass clocks back to serdes
- Fix indentation
- Split off clock "driver" into its own patch to allow for better
  review.
- Add ability to defer lane initialization to phy_init. This allows
  for easier transitioning between firmware-managed serdes and Linux-
  managed serdes, as the consumer (such as dpaa2, which knows what the
  firmware is doing) has the last say on who gets control.
- Fix name of phy mode node
- Add fsl,unused-lanes-reserved to allow a gradual transition, depending
  on the mac link type.
- Remove unused clocks
- Fix some phy mode node names

Changes in v8:
- Remove unused variable from lynx_ls_mode_init
- Rename serdes phy handles to use _A, _B, etc. instead of _0, _1, etc.
  This should help remind readers that the numbering corresponds to the
  physical layout of the registers, and not the lane (pin) number.
- Prevent PCSs from probing as phys
- Rename serdes phy handles like the LS1046A
- Add SFP slot binding
- Fix incorrect lane ordering (it's backwards on the LS1088A just like it is in
  the LS1046A).
- Fix duplicated lane 2 (it should have been lane 3).
- Fix incorrectly-documented value for XFI1.
- Remove interrupt for aquantia phy. It never fired for whatever reason,
  preventing the link from coming up.
- Add GPIOs for QIXIS FPGA.
- Enable MAC1 PCS
- Remove si5341 binding

Changes in v7:
- Use double quotes everywhere in yaml
- Break out call order into generic documentation
- Refuse to switch "major" protocols
- Update Kconfig to reflect restrictions
- Remove set/clear of "pcs reset" bit, since it doesn't seem to fix
  anything.

Changes in v6:
- Bump PHY_TYPE_2500BASEX to 13, since PHY_TYPE_USXGMII was added in the
  meantime
- fsl,type -> phy-type
- frequence -> frequency
- Update MAINTAINERS to include new files
- Include bitfield.h and slab.h to allow compilation on non-arm64
  arches.
- Depend on COMMON_CLK and either layerscape/ppc
- XGI.9 -> XFI.9

Changes in v5:
- Update commit description
- Dual id header
- Remove references to PHY_INTERFACE_MODE_1000BASEKX to allow this
  series to be applied directly to linux/master.
- Add fsl,lynx-10g.h to MAINTAINERS

Changes in v4:
- Add 2500BASE-X and 10GBASE-R phy types
- Use subnodes to describe lane configuration, instead of describing
  PCCRs. This is the same style used by phy-cadence-sierra et al.
- Add ids for Lynx 10g PLLs
- Rework all debug statements to remove use of __func__. Additional
  information has been provided as necessary.
- Consider alternative parent rates in round_rate and not in set_rate.
  Trying to modify out parent's rate in set_rate will deadlock.
- Explicitly perform a stop/reset sequence in set_rate. This way we
  always ensure that the PLL is properly stopped.
- Set the power-down bit when disabling the PLL. We can do this now that
  enable/disable aren't abused during the set rate sequence.
- Fix typos in QSGMII_OFFSET and XFI_OFFSET
- Rename LNmTECR0_TEQ_TYPE_PRE to LNmTECR0_TEQ_TYPE_POST to better
  reflect its function (adding post-cursor equalization).
- Use of_clk_hw_onecell_get instead of a custom function.
- Return struct clks from lynx_clks_init instead of embedding lynx_clk
  in lynx_priv.
- Rework PCCR helper functions; T-series SoCs differ from Layerscape SoCs
  primarily in the layout and offset of the PCCRs. This will help bring a
  cleaner abstraction layer. The caps have been removed, since this handles the
  only current usage.
- Convert to use new binding format. As a result of this, we no longer need to
  have protocols for PCIe or SATA. Additionally, modes now live in lynx_group
  instead of lynx_priv.
- Remove teq from lynx_proto_params, since it can be determined from
  preq_ratio/postq_ratio.
- Fix an early return from lynx_set_mode not releasing serdes->lock.
- Rename lynx_priv.conf to .cfg, since I kept mistyping it.

Changes in v3:
- Manually expand yaml references
- Add mode configuration to device tree
- Rename remaining references to QorIQ SerDes to Lynx 10G
- Fix PLL enable sequence by waiting for our reset request to be cleared
  before continuing. Do the same for the lock, even though it isn't as
  critical. Because we will delay for 1.5ms on average, use prepare
  instead of enable so we can sleep.
- Document the status of each protocol
- Fix offset of several bitfields in RECR0
- Take into account PLLRST_B, SDRST_B, and SDEN when considering whether
  a PLL is "enabled."
- Only power off unused lanes.
- Split mode lane mask into first/last lane (like group)
- Read modes from device tree
- Use caps to determine whether KX/KR are supported
- Move modes to lynx_priv
- Ensure that the protocol controller is not already in-use when we try
  to configure a new mode. This should only occur if the device tree is
  misconfigured (e.g. when QSGMII is selected on two lanes but there is
  only one QSGMII controller).
- Split PLL drivers off into their own file
- Add clock for "ext_dly" instead of writing the bit directly (and
  racing with any clock code).
- Use kasprintf instead of open-coding the snprintf dance
- Support 1000BASE-KX in lynx_lookup_proto. This still requires PCS
  support, so nothing is truly "enabled" yet.
- Describe modes in device tree
- ls1088a: Add serdes bindings

Changes in v2:
- Rename to fsl,lynx-10g.yaml
- Refer to the device in the documentation, rather than the binding
- Move compatible first
- Document phy cells in the description
- Allow a value of 1 for phy-cells. This allows for compatibility with
  the similar (but according to Ioana Ciornei different enough) lynx-28g
  binding.
- Remove minItems
- Use list for clock-names
- Fix example binding having too many cells in regs
- Add #clock-cells. This will allow using assigned-clocks* to configure
  the PLLs.
- Document the structure of the compatible strings
- Rename driver to Lynx 10G (etc.)
- Fix not clearing group->pll after disabling it
- Support 1 and 2 phy-cells
- Power off lanes during probe
- Clear SGMIIaCR1_PCS_EN during probe
- Rename LYNX_PROTO_UNKNOWN to LYNX_PROTO_NONE
- Handle 1000BASE-KX in lynx_proto_mode_prep
- Use one phy cell for SerDes1, since no lanes can be grouped
- Disable SerDes by default to prevent breaking boards inadvertently.

Sean Anderson (10):
  dt-bindings: phy: Add 2500BASE-X and 10GBASE-R
  dt-bindings: phy: Add Lynx 10G phy binding
  dt-bindings: clock: Add ids for Lynx 10g PLLs
  clk: Add Lynx 10G SerDes PLL driver
  phy: fsl: Add Lynx 10G SerDes driver
  arm64: dts: ls1046a: Add serdes bindings
  arm64: dts: ls1046ardb: Add serdes bindings
  arm64: dts: ls1088a: Add serdes bindings
  arm64: dts: ls1088a: Prevent PCSs from probing as phys
  arm64: dts: ls1088ardb: Add serdes bindings

 .../devicetree/bindings/phy/fsl,lynx-10g.yaml |  248 ++++
 Documentation/driver-api/phy/index.rst        |    1 +
 Documentation/driver-api/phy/lynx_10g.rst     |   58 +
 MAINTAINERS                                   |    9 +
 .../boot/dts/freescale/fsl-ls1046a-rdb.dts    |  112 ++
 .../arm64/boot/dts/freescale/fsl-ls1046a.dtsi |   18 +
 .../boot/dts/freescale/fsl-ls1088a-rdb.dts    |  162 ++-
 .../arm64/boot/dts/freescale/fsl-ls1088a.dtsi |   48 +-
 drivers/clk/Makefile                          |    1 +
 drivers/clk/clk-fsl-lynx-10g.c                |  509 +++++++
 drivers/phy/freescale/Kconfig                 |   23 +
 drivers/phy/freescale/Makefile                |    1 +
 drivers/phy/freescale/phy-fsl-lynx-10g.c      | 1224 +++++++++++++++++
 include/dt-bindings/clock/fsl,lynx-10g.h      |   14 +
 include/dt-bindings/phy/phy.h                 |    2 +
 include/linux/phy/lynx-10g.h                  |   16 +
 16 files changed, 2434 insertions(+), 12 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/phy/fsl,lynx-10g.yaml
 create mode 100644 Documentation/driver-api/phy/lynx_10g.rst
 create mode 100644 drivers/clk/clk-fsl-lynx-10g.c
 create mode 100644 drivers/phy/freescale/phy-fsl-lynx-10g.c
 create mode 100644 include/dt-bindings/clock/fsl,lynx-10g.h
 create mode 100644 include/linux/phy/lynx-10g.h

Comments

Sean Anderson Jan. 17, 2023, 4:46 p.m. UTC | #1
On 12/29/22 19:01, Sean Anderson wrote:
> This adds support for the Lynx 10G SerDes found on the QorIQ T-series
> and Layerscape series. Due to limited time and hardware, only support
> for the LS1046ARDB and LS1088ARDB is added in this initial series.
> 
> This series is based on phy/next, but it requires phylink support. This
> is already present for the LS1088A, and it was recently added for the
> LS1046A in net-next/master.
> 
> Major reconfiguration of baud rate (e.g. 1G->10G) does not work. From my
> testing, SerDes register settings appear identical. The issue appears to
> be between the PCS and the MAC. The link itself comes up at both ends,
> and a mac loopback succeeds. However, a PCS loopback results in dropped
> packets. Perhaps there is some undocumented register in the PCS?
> 
> I suspect this driver is around 95% complete, but I don't have the
> documentation to make it work completely. At the very least it is useful
> for two cases:
> 
> - Although this is untested, it should support 2.5G SGMII as well as
>   1000BASE-KX. The latter needs MAC and PCS support, but the former
>   should work out of the box.
> - It allows for clock configurations not supported by the RCW. This is
>   very useful if you want to use e.g. SRDS_PRTCL_S1=0x3333 and =0x1133
>   on the same board. This is because the former setting will use PLL1
>   as the 1G reference, but the latter will use PLL1 as the 10G
>   reference. Because we can reconfigure the PLLs, it is possible to
>   always use PLL1 as the 1G reference.
> 
> The final patch in this series depends on [1].
> 
> [1] https://lore.kernel.org/netdev/20221227230918.2440351-1-sean.anderson@seco.com/
> 
> Changes in v9:
> - Add fsl,unused-lanes-reserved to allow for a gradual transition
>   between firmware and Linux control of the SerDes
> - Change phy-type back to fsl,type, as I was getting the error
>     '#phy-cells' is a dependency of 'phy-type'
> - Convert some u32s to unsigned long to match arguments
> - Switch from round_rate to determine_rate
> - Drop explicit reference to reference clock
> - Use .parent_names when requesting parents
> - Use devm_clk_hw_get_clk to pass clocks back to serdes
> - Fix indentation
> - Split off clock "driver" into its own patch to allow for better
>   review.
> - Add ability to defer lane initialization to phy_init. This allows
>   for easier transitioning between firmware-managed serdes and Linux-
>   managed serdes, as the consumer (such as dpaa2, which knows what the
>   firmware is doing) has the last say on who gets control.
> - Fix name of phy mode node
> - Add fsl,unused-lanes-reserved to allow a gradual transition, depending
>   on the mac link type.
> - Remove unused clocks
> - Fix some phy mode node names
> 
> Changes in v8:
> - Remove unused variable from lynx_ls_mode_init
> - Rename serdes phy handles to use _A, _B, etc. instead of _0, _1, etc.
>   This should help remind readers that the numbering corresponds to the
>   physical layout of the registers, and not the lane (pin) number.
> - Prevent PCSs from probing as phys
> - Rename serdes phy handles like the LS1046A
> - Add SFP slot binding
> - Fix incorrect lane ordering (it's backwards on the LS1088A just like it is in
>   the LS1046A).
> - Fix duplicated lane 2 (it should have been lane 3).
> - Fix incorrectly-documented value for XFI1.
> - Remove interrupt for aquantia phy. It never fired for whatever reason,
>   preventing the link from coming up.
> - Add GPIOs for QIXIS FPGA.
> - Enable MAC1 PCS
> - Remove si5341 binding
> 
> Changes in v7:
> - Use double quotes everywhere in yaml
> - Break out call order into generic documentation
> - Refuse to switch "major" protocols
> - Update Kconfig to reflect restrictions
> - Remove set/clear of "pcs reset" bit, since it doesn't seem to fix
>   anything.
> 
> Changes in v6:
> - Bump PHY_TYPE_2500BASEX to 13, since PHY_TYPE_USXGMII was added in the
>   meantime
> - fsl,type -> phy-type
> - frequence -> frequency
> - Update MAINTAINERS to include new files
> - Include bitfield.h and slab.h to allow compilation on non-arm64
>   arches.
> - Depend on COMMON_CLK and either layerscape/ppc
> - XGI.9 -> XFI.9
> 
> Changes in v5:
> - Update commit description
> - Dual id header
> - Remove references to PHY_INTERFACE_MODE_1000BASEKX to allow this
>   series to be applied directly to linux/master.
> - Add fsl,lynx-10g.h to MAINTAINERS
> 
> Changes in v4:
> - Add 2500BASE-X and 10GBASE-R phy types
> - Use subnodes to describe lane configuration, instead of describing
>   PCCRs. This is the same style used by phy-cadence-sierra et al.
> - Add ids for Lynx 10g PLLs
> - Rework all debug statements to remove use of __func__. Additional
>   information has been provided as necessary.
> - Consider alternative parent rates in round_rate and not in set_rate.
>   Trying to modify out parent's rate in set_rate will deadlock.
> - Explicitly perform a stop/reset sequence in set_rate. This way we
>   always ensure that the PLL is properly stopped.
> - Set the power-down bit when disabling the PLL. We can do this now that
>   enable/disable aren't abused during the set rate sequence.
> - Fix typos in QSGMII_OFFSET and XFI_OFFSET
> - Rename LNmTECR0_TEQ_TYPE_PRE to LNmTECR0_TEQ_TYPE_POST to better
>   reflect its function (adding post-cursor equalization).
> - Use of_clk_hw_onecell_get instead of a custom function.
> - Return struct clks from lynx_clks_init instead of embedding lynx_clk
>   in lynx_priv.
> - Rework PCCR helper functions; T-series SoCs differ from Layerscape SoCs
>   primarily in the layout and offset of the PCCRs. This will help bring a
>   cleaner abstraction layer. The caps have been removed, since this handles the
>   only current usage.
> - Convert to use new binding format. As a result of this, we no longer need to
>   have protocols for PCIe or SATA. Additionally, modes now live in lynx_group
>   instead of lynx_priv.
> - Remove teq from lynx_proto_params, since it can be determined from
>   preq_ratio/postq_ratio.
> - Fix an early return from lynx_set_mode not releasing serdes->lock.
> - Rename lynx_priv.conf to .cfg, since I kept mistyping it.
> 
> Changes in v3:
> - Manually expand yaml references
> - Add mode configuration to device tree
> - Rename remaining references to QorIQ SerDes to Lynx 10G
> - Fix PLL enable sequence by waiting for our reset request to be cleared
>   before continuing. Do the same for the lock, even though it isn't as
>   critical. Because we will delay for 1.5ms on average, use prepare
>   instead of enable so we can sleep.
> - Document the status of each protocol
> - Fix offset of several bitfields in RECR0
> - Take into account PLLRST_B, SDRST_B, and SDEN when considering whether
>   a PLL is "enabled."
> - Only power off unused lanes.
> - Split mode lane mask into first/last lane (like group)
> - Read modes from device tree
> - Use caps to determine whether KX/KR are supported
> - Move modes to lynx_priv
> - Ensure that the protocol controller is not already in-use when we try
>   to configure a new mode. This should only occur if the device tree is
>   misconfigured (e.g. when QSGMII is selected on two lanes but there is
>   only one QSGMII controller).
> - Split PLL drivers off into their own file
> - Add clock for "ext_dly" instead of writing the bit directly (and
>   racing with any clock code).
> - Use kasprintf instead of open-coding the snprintf dance
> - Support 1000BASE-KX in lynx_lookup_proto. This still requires PCS
>   support, so nothing is truly "enabled" yet.
> - Describe modes in device tree
> - ls1088a: Add serdes bindings
> 
> Changes in v2:
> - Rename to fsl,lynx-10g.yaml
> - Refer to the device in the documentation, rather than the binding
> - Move compatible first
> - Document phy cells in the description
> - Allow a value of 1 for phy-cells. This allows for compatibility with
>   the similar (but according to Ioana Ciornei different enough) lynx-28g
>   binding.
> - Remove minItems
> - Use list for clock-names
> - Fix example binding having too many cells in regs
> - Add #clock-cells. This will allow using assigned-clocks* to configure
>   the PLLs.
> - Document the structure of the compatible strings
> - Rename driver to Lynx 10G (etc.)
> - Fix not clearing group->pll after disabling it
> - Support 1 and 2 phy-cells
> - Power off lanes during probe
> - Clear SGMIIaCR1_PCS_EN during probe
> - Rename LYNX_PROTO_UNKNOWN to LYNX_PROTO_NONE
> - Handle 1000BASE-KX in lynx_proto_mode_prep
> - Use one phy cell for SerDes1, since no lanes can be grouped
> - Disable SerDes by default to prevent breaking boards inadvertently.
> 
> Sean Anderson (10):
>   dt-bindings: phy: Add 2500BASE-X and 10GBASE-R
>   dt-bindings: phy: Add Lynx 10G phy binding
>   dt-bindings: clock: Add ids for Lynx 10g PLLs
>   clk: Add Lynx 10G SerDes PLL driver
>   phy: fsl: Add Lynx 10G SerDes driver
>   arm64: dts: ls1046a: Add serdes bindings
>   arm64: dts: ls1046ardb: Add serdes bindings
>   arm64: dts: ls1088a: Add serdes bindings
>   arm64: dts: ls1088a: Prevent PCSs from probing as phys
>   arm64: dts: ls1088ardb: Add serdes bindings
> 
>  .../devicetree/bindings/phy/fsl,lynx-10g.yaml |  248 ++++
>  Documentation/driver-api/phy/index.rst        |    1 +
>  Documentation/driver-api/phy/lynx_10g.rst     |   58 +
>  MAINTAINERS                                   |    9 +
>  .../boot/dts/freescale/fsl-ls1046a-rdb.dts    |  112 ++
>  .../arm64/boot/dts/freescale/fsl-ls1046a.dtsi |   18 +
>  .../boot/dts/freescale/fsl-ls1088a-rdb.dts    |  162 ++-
>  .../arm64/boot/dts/freescale/fsl-ls1088a.dtsi |   48 +-
>  drivers/clk/Makefile                          |    1 +
>  drivers/clk/clk-fsl-lynx-10g.c                |  509 +++++++
>  drivers/phy/freescale/Kconfig                 |   23 +
>  drivers/phy/freescale/Makefile                |    1 +
>  drivers/phy/freescale/phy-fsl-lynx-10g.c      | 1224 +++++++++++++++++
>  include/dt-bindings/clock/fsl,lynx-10g.h      |   14 +
>  include/dt-bindings/phy/phy.h                 |    2 +
>  include/linux/phy/lynx-10g.h                  |   16 +
>  16 files changed, 2434 insertions(+), 12 deletions(-)
>  create mode 100644 Documentation/devicetree/bindings/phy/fsl,lynx-10g.yaml
>  create mode 100644 Documentation/driver-api/phy/lynx_10g.rst
>  create mode 100644 drivers/clk/clk-fsl-lynx-10g.c
>  create mode 100644 drivers/phy/freescale/phy-fsl-lynx-10g.c
>  create mode 100644 include/dt-bindings/clock/fsl,lynx-10g.h
>  create mode 100644 include/linux/phy/lynx-10g.h
> 

I noticed that this series is marked "changes requested" on patchwork.
However, I have received only automated feedback. I have done my best
effort to address feedback I have received on prior revisions. I would
appreciate getting another round of review before resending this series.

--Sean
Vinod Koul Jan. 18, 2023, 4:54 p.m. UTC | #2
On 17-01-23, 11:46, Sean Anderson wrote:
> 
> I noticed that this series is marked "changes requested" on patchwork.
> However, I have received only automated feedback. I have done my best
> effort to address feedback I have received on prior revisions. I would
> appreciate getting another round of review before resending this series.

Looking at the series, looks like kernel-bot sent some warnings on the
series so I was expecting an updated series for review
Sean Anderson Jan. 19, 2023, 4:22 p.m. UTC | #3
On 1/18/23 11:54, Vinod Koul wrote:
> On 17-01-23, 11:46, Sean Anderson wrote:
>> 
>> I noticed that this series is marked "changes requested" on patchwork.
>> However, I have received only automated feedback. I have done my best
>> effort to address feedback I have received on prior revisions. I would
>> appreciate getting another round of review before resending this series.
> 
> Looking at the series, looks like kernel-bot sent some warnings on the
> series so I was expecting an updated series for review
> 

Generally, multiple reviewers will comment on a patch, even if another
reviewer finds something which needs to be changed. This is a one-line
fix, so I would appreciate getting more substantial feedback before
respinning. Every time I send a new series I have to rebase and test on
hardware. It's work that I would rather do when there is something to be
gained.

--Sean
Vinod Koul Jan. 20, 2023, 8:06 a.m. UTC | #4
On 19-01-23, 11:22, Sean Anderson wrote:
> On 1/18/23 11:54, Vinod Koul wrote:
> > On 17-01-23, 11:46, Sean Anderson wrote:
> >> 
> >> I noticed that this series is marked "changes requested" on patchwork.
> >> However, I have received only automated feedback. I have done my best
> >> effort to address feedback I have received on prior revisions. I would
> >> appreciate getting another round of review before resending this series.
> > 
> > Looking at the series, looks like kernel-bot sent some warnings on the
> > series so I was expecting an updated series for review
> > 
> 
> Generally, multiple reviewers will comment on a patch, even if another
> reviewer finds something which needs to be changed. This is a one-line
> fix, so I would appreciate getting more substantial feedback before
> respinning. Every time I send a new series I have to rebase and test on
> hardware. It's work that I would rather do when there is something to be
> gained.

I review to apply, if I can apply, I would typically skip this
Sean Anderson Jan. 20, 2023, 4:43 p.m. UTC | #5
On 1/20/23 03:06, Vinod Koul wrote:
> On 19-01-23, 11:22, Sean Anderson wrote:
>> On 1/18/23 11:54, Vinod Koul wrote:
>> > On 17-01-23, 11:46, Sean Anderson wrote:
>> >> 
>> >> I noticed that this series is marked "changes requested" on patchwork.
>> >> However, I have received only automated feedback. I have done my best
>> >> effort to address feedback I have received on prior revisions. I would
>> >> appreciate getting another round of review before resending this series.
>> > 
>> > Looking at the series, looks like kernel-bot sent some warnings on the
>> > series so I was expecting an updated series for review
>> > 
>> 
>> Generally, multiple reviewers will comment on a patch, even if another
>> reviewer finds something which needs to be changed. This is a one-line
>> fix, so I would appreciate getting more substantial feedback before
>> respinning. Every time I send a new series I have to rebase and test on
>> hardware. It's work that I would rather do when there is something to be
>> gained.
> 
> I review to apply, if I can apply, I would typically skip this
> 

It is much more efficient to conduct reviews in parallel. So e.g. the
bindings can be reviewed at the same time as the driver, at the same
time as the device tree changes. This way, I can get a series applied
after max(N, M, ...) revisions, where I would otherwise need N revisions
to get the bindings ready, M revisions to get the driver ready, etc.
But what's happening is that I have to make N + M + ... revisions! I am
very frustrated by your refusal to review anything until there are no
other comments, since it unnecessarily extends the process of getting a
series applied. I have been trying to get this series applied since
June, with nine revisions, and you have reviewed it *twice*! I think the
driver is in a good state and is ready to be applied (aside from the one
known issue), but I have no idea if you agree with that assessment.

--Sean
Shawn Guo Jan. 25, 2023, 11:43 p.m. UTC | #6
On Thu, Dec 29, 2022 at 07:01:36PM -0500, Sean Anderson wrote:
> This adds appropriate bindings for the macs which use the SerDes. The
> 156.25MHz fixed clock is a crystal. The 100MHz clocks (there are
> actually 3) come from a Renesas 6V49205B at address 69 on i2c0. There is
> no driver for this device (and as far as I know all you can do with the
> 100MHz clocks is gate them), so I have chosen to model it as a single
> fixed clock.
> 
> Note: the SerDes1 lane numbering for the LS1046A is *reversed*.
> This means that Lane A (what the driver thinks is lane 0) uses pins
> SD1_TX3_P/N.
> 
> Because this will break ethernet if the serdes is not enabled, enable
> the serdes driver by default on Layerscape.
> 
> Signed-off-by: Sean Anderson <sean.anderson@seco.com>
> ---
> This depends on [1].
> 
> [1] https://lore.kernel.org/netdev/20220804194705.459670-4-sean.anderson@seco.com/
> 
> Changes in v9:
> - Fix name of phy mode node
> - phy-type -> fsl,phy
> 
> Changes in v8:
> - Rename serdes phy handles to use _A, _B, etc. instead of _0, _1, etc.
>   This should help remind readers that the numbering corresponds to the
>   physical layout of the registers, and not the lane (pin) number.
> 
> Changes in v6:
> - XGI.9 -> XFI.9
> 
> Changes in v4:
> - Convert to new bindings
> 
>  .../boot/dts/freescale/fsl-ls1046a-rdb.dts    | 112 ++++++++++++++++++
>  drivers/phy/freescale/Kconfig                 |   1 +

The phy driver Kconfig change shouldn't be part of this patch.

>  2 files changed, 113 insertions(+)
> 
> diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1046a-rdb.dts b/arch/arm64/boot/dts/freescale/fsl-ls1046a-rdb.dts
> index 7025aad8ae89..534f19855b47 100644
> --- a/arch/arm64/boot/dts/freescale/fsl-ls1046a-rdb.dts
> +++ b/arch/arm64/boot/dts/freescale/fsl-ls1046a-rdb.dts
> @@ -10,6 +10,8 @@
>  
>  /dts-v1/;
>  
> +#include <dt-bindings/phy/phy.h>
> +
>  #include "fsl-ls1046a.dtsi"
>  
>  / {
> @@ -26,8 +28,110 @@ aliases {
>  	chosen {
>  		stdout-path = "serial0:115200n8";
>  	};
> +
> +	clocks {

Drop this container node.

Shawn

> +		clk_100mhz: clock-100mhz {
> +			compatible = "fixed-clock";
> +			#clock-cells = <0>;
> +			clock-frequency = <100000000>;
> +		};
> +
> +		clk_156mhz: clock-156mhz {
> +			compatible = "fixed-clock";
> +			#clock-cells = <0>;
> +			clock-frequency = <156250000>;
> +		};
> +	};
>  };
>  
> +&serdes1 {
> +	clocks = <&clk_100mhz>, <&clk_156mhz>;
> +	clock-names = "ref0", "ref1";
> +	status = "okay";
> +
> +	/*
> +	 * XXX: Lane A uses pins SD1_RX3_P/N! That is, the lane numbers and pin
> +	 * numbers are _reversed_. In addition, the PCCR documentation is
> +	 * _inconsistent_ in its usage of these terms!
> +	 *
> +	 * PCCR "Lane 0" refers to...
> +	 * ==== =====================
> +	 *    0 Lane A
> +	 *    2 Lane A
> +	 *    8 Lane A
> +	 *    9 Lane A
> +	 *    B Lane D!
> +	 */
> +	serdes1_A: phy@0 {
> +		#phy-cells = <0>;
> +		reg = <0>;
> +
> +		/* SGMII.6 */
> +		sgmii-0 {
> +			fsl,pccr = <0x8>;
> +			fsl,index = <0>;
> +			fsl,cfg = <0x1>;
> +			fsl,type = <PHY_TYPE_SGMII>;
> +		};
> +	};
> +
> +	serdes1_B: phy@1 {
> +		#phy-cells = <0>;
> +		reg = <1>;
> +
> +		/* SGMII.5 */
> +		sgmii-1 {
> +			fsl,pccr = <0x8>;
> +			fsl,index = <1>;
> +			fsl,cfg = <0x1>;
> +			fsl,type = <PHY_TYPE_2500BASEX>;
> +		};
> +	};
> +
> +	serdes1_C: phy@2 {
> +		#phy-cells = <0>;
> +		reg = <2>;
> +
> +		/* SGMII.10 */
> +		sgmii-2 {
> +			fsl,pccr = <0x8>;
> +			fsl,index = <2>;
> +			fsl,cfg = <0x1>;
> +			fsl,type = <PHY_TYPE_2500BASEX>;
> +		};
> +
> +		/* XFI.10 */
> +		xfi-0 {
> +			fsl,pccr = <0xb>;
> +			fsl,index = <0>;
> +			fsl,cfg = <0x2>;
> +			fsl,type = <PHY_TYPE_10GBASER>;
> +		};
> +	};
> +
> +	serdes1_D: phy@3 {
> +		#phy-cells = <0>;
> +		reg = <3>;
> +
> +		/* SGMII.9 */
> +		sgmii-3 {
> +			fsl,pccr = <0x8>;
> +			fsl,index = <3>;
> +			fsl,cfg = <0x1>;
> +			fsl,type = <PHY_TYPE_2500BASEX>;
> +		};
> +
> +		/* XFI.9 */
> +		xfi-1 {
> +			fsl,pccr = <0xb>;
> +			fsl,index = <1>;
> +			fsl,cfg = <0x1>;
> +			fsl,type = <PHY_TYPE_10GBASER>;
> +		};
> +	};
> +};
> +
> +
>  &duart0 {
>  	status = "okay";
>  };
> @@ -140,21 +244,29 @@ ethernet@e6000 {
>  	ethernet@e8000 {
>  		phy-handle = <&sgmii_phy1>;
>  		phy-connection-type = "sgmii";
> +		phys = <&serdes1_B>;
> +		phy-names = "serdes";
>  	};
>  
>  	ethernet@ea000 {
>  		phy-handle = <&sgmii_phy2>;
>  		phy-connection-type = "sgmii";
> +		phys = <&serdes1_A>;
> +		phy-names = "serdes";
>  	};
>  
>  	ethernet@f0000 { /* 10GEC1 */
>  		phy-handle = <&aqr106_phy>;
>  		phy-connection-type = "xgmii";
> +		phys = <&serdes1_D>;
> +		phy-names = "serdes";
>  	};
>  
>  	ethernet@f2000 { /* 10GEC2 */
>  		fixed-link = <0 1 1000 0 0>;
>  		phy-connection-type = "xgmii";
> +		phys = <&serdes1_C>;
> +		phy-names = "serdes";
>  	};
>  
>  	mdio@fc000 {
> diff --git a/drivers/phy/freescale/Kconfig b/drivers/phy/freescale/Kconfig
> index 6bebe00f5889..b396162dc859 100644
> --- a/drivers/phy/freescale/Kconfig
> +++ b/drivers/phy/freescale/Kconfig
> @@ -54,6 +54,7 @@ config PHY_FSL_LYNX_10G
>  	depends on ARCH_LAYERSCAPE || PPC || COMPILE_TEST
>  	select GENERIC_PHY
>  	select REGMAP_MMIO
> +	default y if ARCH_LAYERSCAPE
>  	help
>  	  This adds support for the Lynx "SerDes" devices found on various QorIQ
>  	  SoCs. There may be up to four SerDes devices on each SoC, and each
> -- 
> 2.35.1.1320.gc452695387.dirty
>
Shawn Guo Jan. 25, 2023, 11:46 p.m. UTC | #7
On Thu, Dec 29, 2022 at 07:01:35PM -0500, Sean Anderson wrote:
> This adds bindings for the SerDes devices. They are disabled by default

s/bindings/descriptions?

The term "bindings" generally means the schema/doc in
Documentation/devicetree/bindings/.

Shawn

> to prevent any breakage on existing boards.
> 
> Signed-off-by: Sean Anderson <sean.anderson@seco.com>
> ---
> 
> (no changes since v4)
> 
> Changes in v4:
> - Convert to new bindings
> 
> Changes in v3:
> - Describe modes in device tree
> 
> Changes in v2:
> - Use one phy cell for SerDes1, since no lanes can be grouped
> - Disable SerDes by default to prevent breaking boards inadvertently.
> 
>  arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi | 18 ++++++++++++++++++
>  1 file changed, 18 insertions(+)
> 
> diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
> index a01e3cfec77f..12adccd5caae 100644
> --- a/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
> +++ b/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
> @@ -424,6 +424,24 @@ sfp: efuse@1e80000 {
>  			clock-names = "sfp";
>  		};
>  
> +		serdes1: serdes@1ea0000 {
> +			#address-cells = <1>;
> +			#size-cells = <0>;
> +			#clock-cells = <1>;
> +			compatible = "fsl,ls1046a-serdes", "fsl,lynx-10g";
> +			reg = <0x0 0x1ea0000 0x0 0x2000>;
> +			status = "disabled";
> +		};
> +
> +		serdes2: serdes@1eb0000 {
> +			#address-cells = <1>;
> +			#size-cells = <0>;
> +			#clock-cells = <1>;
> +			compatible = "fsl,ls1046a-serdes", "fsl,lynx-10g";
> +			reg = <0x0 0x1eb0000 0x0 0x2000>;
> +			status = "disabled";
> +		};
> +
>  		dcfg: dcfg@1ee0000 {
>  			compatible = "fsl,ls1046a-dcfg", "syscon";
>  			reg = <0x0 0x1ee0000 0x0 0x1000>;
> -- 
> 2.35.1.1320.gc452695387.dirty
>
Shawn Guo Jan. 25, 2023, 11:48 p.m. UTC | #8
On Thu, Dec 29, 2022 at 07:01:37PM -0500, Sean Anderson wrote:
> This adds bindings for the SerDes devices. They are disabled by default
> to prevent any breakage on existing boards.
> 
> Signed-off-by: Sean Anderson <sean.anderson@seco.com>
> ---
> 
> (no changes since v4)
> 
> Changes in v4:
> - Convert to new bindings
> 
> Changes in v3:
> - New
> 
>  arch/arm64/boot/dts/freescale/fsl-ls1088a.dtsi | 18 ++++++++++++++++++
>  1 file changed, 18 insertions(+)
> 
> diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1088a.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls1088a.dtsi
> index 260d045dbd9a..ecf9d830e36f 100644
> --- a/arch/arm64/boot/dts/freescale/fsl-ls1088a.dtsi
> +++ b/arch/arm64/boot/dts/freescale/fsl-ls1088a.dtsi
> @@ -238,6 +238,24 @@ reset: syscon@1e60000 {
>  			reg = <0x0 0x1e60000 0x0 0x10000>;
>  		};
>  
> +		serdes1: serdes@1ea0000 {
> +			#address-cells = <1>;
> +			#size-cells = <0>;
> +			#clock-cells = <1>;
> +			compatible = "fsl,ls1088a-serdes", "fsl,lynx-10g";
> +			reg = <0x0 0x1ea0000 0x0 0x2000>;

Can we start the properties with compatible (and reg) like most of other
device nodes?

Shawn

> +			status = "disabled";
> +		};
> +
> +		serdes2: serdes@1eb0000 {
> +			#address-cells = <1>;
> +			#size-cells = <0>;
> +			#clock-cells = <1>;
> +			compatible = "fsl,ls1088a-serdes", "fsl,lynx-10g";
> +			reg = <0x0 0x1eb0000 0x0 0x2000>;
> +			status = "disabled";
> +		};
> +
>  		isc: syscon@1f70000 {
>  			compatible = "fsl,ls1088a-isc", "syscon";
>  			reg = <0x0 0x1f70000 0x0 0x10000>;
> -- 
> 2.35.1.1320.gc452695387.dirty
>
Sean Anderson Jan. 26, 2023, 4:43 p.m. UTC | #9
On 1/25/23 18:46, Shawn Guo wrote:
> On Thu, Dec 29, 2022 at 07:01:35PM -0500, Sean Anderson wrote:
>> This adds bindings for the SerDes devices. They are disabled by default
> 
> s/bindings/descriptions?
> 
> The term "bindings" generally means the schema/doc in
> Documentation/devicetree/bindings/.

How about "nodes"?

--Sean

> Shawn
> 
>> to prevent any breakage on existing boards.
>> 
>> Signed-off-by: Sean Anderson <sean.anderson@seco.com>
>> ---
>> 
>> (no changes since v4)
>> 
>> Changes in v4:
>> - Convert to new bindings
>> 
>> Changes in v3:
>> - Describe modes in device tree
>> 
>> Changes in v2:
>> - Use one phy cell for SerDes1, since no lanes can be grouped
>> - Disable SerDes by default to prevent breaking boards inadvertently.
>> 
>>  arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi | 18 ++++++++++++++++++
>>  1 file changed, 18 insertions(+)
>> 
>> diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
>> index a01e3cfec77f..12adccd5caae 100644
>> --- a/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
>> +++ b/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
>> @@ -424,6 +424,24 @@ sfp: efuse@1e80000 {
>>  			clock-names = "sfp";
>>  		};
>>  
>> +		serdes1: serdes@1ea0000 {
>> +			#address-cells = <1>;
>> +			#size-cells = <0>;
>> +			#clock-cells = <1>;
>> +			compatible = "fsl,ls1046a-serdes", "fsl,lynx-10g";
>> +			reg = <0x0 0x1ea0000 0x0 0x2000>;
>> +			status = "disabled";
>> +		};
>> +
>> +		serdes2: serdes@1eb0000 {
>> +			#address-cells = <1>;
>> +			#size-cells = <0>;
>> +			#clock-cells = <1>;
>> +			compatible = "fsl,ls1046a-serdes", "fsl,lynx-10g";
>> +			reg = <0x0 0x1eb0000 0x0 0x2000>;
>> +			status = "disabled";
>> +		};
>> +
>>  		dcfg: dcfg@1ee0000 {
>>  			compatible = "fsl,ls1046a-dcfg", "syscon";
>>  			reg = <0x0 0x1ee0000 0x0 0x1000>;
>> -- 
>> 2.35.1.1320.gc452695387.dirty
>>
Sean Anderson Jan. 26, 2023, 4:48 p.m. UTC | #10
On 1/25/23 18:43, Shawn Guo wrote:
> On Thu, Dec 29, 2022 at 07:01:36PM -0500, Sean Anderson wrote:
>> This adds appropriate bindings for the macs which use the SerDes. The
>> 156.25MHz fixed clock is a crystal. The 100MHz clocks (there are
>> actually 3) come from a Renesas 6V49205B at address 69 on i2c0. There is
>> no driver for this device (and as far as I know all you can do with the
>> 100MHz clocks is gate them), so I have chosen to model it as a single
>> fixed clock.
>> 
>> Note: the SerDes1 lane numbering for the LS1046A is *reversed*.
>> This means that Lane A (what the driver thinks is lane 0) uses pins
>> SD1_TX3_P/N.
>> 
>> Because this will break ethernet if the serdes is not enabled, enable
>> the serdes driver by default on Layerscape.
>> 
>> Signed-off-by: Sean Anderson <sean.anderson@seco.com>
>> ---
>> This depends on [1].
>> 
>> [1] https://lore.kernel.org/netdev/20220804194705.459670-4-sean.anderson@seco.com/
>> 
>> Changes in v9:
>> - Fix name of phy mode node
>> - phy-type -> fsl,phy
>> 
>> Changes in v8:
>> - Rename serdes phy handles to use _A, _B, etc. instead of _0, _1, etc.
>>   This should help remind readers that the numbering corresponds to the
>>   physical layout of the registers, and not the lane (pin) number.
>> 
>> Changes in v6:
>> - XGI.9 -> XFI.9
>> 
>> Changes in v4:
>> - Convert to new bindings
>> 
>>  .../boot/dts/freescale/fsl-ls1046a-rdb.dts    | 112 ++++++++++++++++++
>>  drivers/phy/freescale/Kconfig                 |   1 +
> 
> The phy driver Kconfig change shouldn't be part of this patch.

I put it here for bisectability, since this is the point where we need
to enable it. But I can do this in a separate patch if you want.

>>  2 files changed, 113 insertions(+)
>> 
>> diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1046a-rdb.dts b/arch/arm64/boot/dts/freescale/fsl-ls1046a-rdb.dts
>> index 7025aad8ae89..534f19855b47 100644
>> --- a/arch/arm64/boot/dts/freescale/fsl-ls1046a-rdb.dts
>> +++ b/arch/arm64/boot/dts/freescale/fsl-ls1046a-rdb.dts
>> @@ -10,6 +10,8 @@
>>  
>>  /dts-v1/;
>>  
>> +#include <dt-bindings/phy/phy.h>
>> +
>>  #include "fsl-ls1046a.dtsi"
>>  
>>  / {
>> @@ -26,8 +28,110 @@ aliases {
>>  	chosen {
>>  		stdout-path = "serial0:115200n8";
>>  	};
>> +
>> +	clocks {
> 
> Drop this container node.

OK

--Sean
 
>> +		clk_100mhz: clock-100mhz {
>> +			compatible = "fixed-clock";
>> +			#clock-cells = <0>;
>> +			clock-frequency = <100000000>;
>> +		};
>> +
>> +		clk_156mhz: clock-156mhz {
>> +			compatible = "fixed-clock";
>> +			#clock-cells = <0>;
>> +			clock-frequency = <156250000>;
>> +		};
>> +	};
>>  };
>>  
>> +&serdes1 {
>> +	clocks = <&clk_100mhz>, <&clk_156mhz>;
>> +	clock-names = "ref0", "ref1";
>> +	status = "okay";
>> +
>> +	/*
>> +	 * XXX: Lane A uses pins SD1_RX3_P/N! That is, the lane numbers and pin
>> +	 * numbers are _reversed_. In addition, the PCCR documentation is
>> +	 * _inconsistent_ in its usage of these terms!
>> +	 *
>> +	 * PCCR "Lane 0" refers to...
>> +	 * ==== =====================
>> +	 *    0 Lane A
>> +	 *    2 Lane A
>> +	 *    8 Lane A
>> +	 *    9 Lane A
>> +	 *    B Lane D!
>> +	 */
>> +	serdes1_A: phy@0 {
>> +		#phy-cells = <0>;
>> +		reg = <0>;
>> +
>> +		/* SGMII.6 */
>> +		sgmii-0 {
>> +			fsl,pccr = <0x8>;
>> +			fsl,index = <0>;
>> +			fsl,cfg = <0x1>;
>> +			fsl,type = <PHY_TYPE_SGMII>;
>> +		};
>> +	};
>> +
>> +	serdes1_B: phy@1 {
>> +		#phy-cells = <0>;
>> +		reg = <1>;
>> +
>> +		/* SGMII.5 */
>> +		sgmii-1 {
>> +			fsl,pccr = <0x8>;
>> +			fsl,index = <1>;
>> +			fsl,cfg = <0x1>;
>> +			fsl,type = <PHY_TYPE_2500BASEX>;
>> +		};
>> +	};
>> +
>> +	serdes1_C: phy@2 {
>> +		#phy-cells = <0>;
>> +		reg = <2>;
>> +
>> +		/* SGMII.10 */
>> +		sgmii-2 {
>> +			fsl,pccr = <0x8>;
>> +			fsl,index = <2>;
>> +			fsl,cfg = <0x1>;
>> +			fsl,type = <PHY_TYPE_2500BASEX>;
>> +		};
>> +
>> +		/* XFI.10 */
>> +		xfi-0 {
>> +			fsl,pccr = <0xb>;
>> +			fsl,index = <0>;
>> +			fsl,cfg = <0x2>;
>> +			fsl,type = <PHY_TYPE_10GBASER>;
>> +		};
>> +	};
>> +
>> +	serdes1_D: phy@3 {
>> +		#phy-cells = <0>;
>> +		reg = <3>;
>> +
>> +		/* SGMII.9 */
>> +		sgmii-3 {
>> +			fsl,pccr = <0x8>;
>> +			fsl,index = <3>;
>> +			fsl,cfg = <0x1>;
>> +			fsl,type = <PHY_TYPE_2500BASEX>;
>> +		};
>> +
>> +		/* XFI.9 */
>> +		xfi-1 {
>> +			fsl,pccr = <0xb>;
>> +			fsl,index = <1>;
>> +			fsl,cfg = <0x1>;
>> +			fsl,type = <PHY_TYPE_10GBASER>;
>> +		};
>> +	};
>> +};
>> +
>> +
>>  &duart0 {
>>  	status = "okay";
>>  };
>> @@ -140,21 +244,29 @@ ethernet@e6000 {
>>  	ethernet@e8000 {
>>  		phy-handle = <&sgmii_phy1>;
>>  		phy-connection-type = "sgmii";
>> +		phys = <&serdes1_B>;
>> +		phy-names = "serdes";
>>  	};
>>  
>>  	ethernet@ea000 {
>>  		phy-handle = <&sgmii_phy2>;
>>  		phy-connection-type = "sgmii";
>> +		phys = <&serdes1_A>;
>> +		phy-names = "serdes";
>>  	};
>>  
>>  	ethernet@f0000 { /* 10GEC1 */
>>  		phy-handle = <&aqr106_phy>;
>>  		phy-connection-type = "xgmii";
>> +		phys = <&serdes1_D>;
>> +		phy-names = "serdes";
>>  	};
>>  
>>  	ethernet@f2000 { /* 10GEC2 */
>>  		fixed-link = <0 1 1000 0 0>;
>>  		phy-connection-type = "xgmii";
>> +		phys = <&serdes1_C>;
>> +		phy-names = "serdes";
>>  	};
>>  
>>  	mdio@fc000 {
>> diff --git a/drivers/phy/freescale/Kconfig b/drivers/phy/freescale/Kconfig
>> index 6bebe00f5889..b396162dc859 100644
>> --- a/drivers/phy/freescale/Kconfig
>> +++ b/drivers/phy/freescale/Kconfig
>> @@ -54,6 +54,7 @@ config PHY_FSL_LYNX_10G
>>  	depends on ARCH_LAYERSCAPE || PPC || COMPILE_TEST
>>  	select GENERIC_PHY
>>  	select REGMAP_MMIO
>> +	default y if ARCH_LAYERSCAPE
>>  	help
>>  	  This adds support for the Lynx "SerDes" devices found on various QorIQ
>>  	  SoCs. There may be up to four SerDes devices on each SoC, and each
>> -- 
>> 2.35.1.1320.gc452695387.dirty
>>
Sean Anderson Jan. 26, 2023, 5:54 p.m. UTC | #11
On 1/25/23 18:48, Shawn Guo wrote:
> On Thu, Dec 29, 2022 at 07:01:37PM -0500, Sean Anderson wrote:
>> This adds bindings for the SerDes devices. They are disabled by default
>> to prevent any breakage on existing boards.
>> 
>> Signed-off-by: Sean Anderson <sean.anderson@seco.com>
>> ---
>> 
>> (no changes since v4)
>> 
>> Changes in v4:
>> - Convert to new bindings
>> 
>> Changes in v3:
>> - New
>> 
>>  arch/arm64/boot/dts/freescale/fsl-ls1088a.dtsi | 18 ++++++++++++++++++
>>  1 file changed, 18 insertions(+)
>> 
>> diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1088a.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls1088a.dtsi
>> index 260d045dbd9a..ecf9d830e36f 100644
>> --- a/arch/arm64/boot/dts/freescale/fsl-ls1088a.dtsi
>> +++ b/arch/arm64/boot/dts/freescale/fsl-ls1088a.dtsi
>> @@ -238,6 +238,24 @@ reset: syscon@1e60000 {
>>  			reg = <0x0 0x1e60000 0x0 0x10000>;
>>  		};
>>  
>> +		serdes1: serdes@1ea0000 {
>> +			#address-cells = <1>;
>> +			#size-cells = <0>;
>> +			#clock-cells = <1>;
>> +			compatible = "fsl,ls1088a-serdes", "fsl,lynx-10g";
>> +			reg = <0x0 0x1ea0000 0x0 0x2000>;
> 
> Can we start the properties with compatible (and reg) like most of other
> device nodes?

Sure.

--Sean

>> +			status = "disabled";
>> +		};
>> +
>> +		serdes2: serdes@1eb0000 {
>> +			#address-cells = <1>;
>> +			#size-cells = <0>;
>> +			#clock-cells = <1>;
>> +			compatible = "fsl,ls1088a-serdes", "fsl,lynx-10g";
>> +			reg = <0x0 0x1eb0000 0x0 0x2000>;
>> +			status = "disabled";
>> +		};
>> +
>>  		isc: syscon@1f70000 {
>>  			compatible = "fsl,ls1088a-isc", "syscon";
>>  			reg = <0x0 0x1f70000 0x0 0x10000>;
>> -- 
>> 2.35.1.1320.gc452695387.dirty
>>
Shawn Guo Jan. 27, 2023, 7:52 a.m. UTC | #12
On Thu, Jan 26, 2023 at 11:48:53AM -0500, Sean Anderson wrote:
> On 1/25/23 18:43, Shawn Guo wrote:
> > On Thu, Dec 29, 2022 at 07:01:36PM -0500, Sean Anderson wrote:
> >> This adds appropriate bindings for the macs which use the SerDes. The
> >> 156.25MHz fixed clock is a crystal. The 100MHz clocks (there are
> >> actually 3) come from a Renesas 6V49205B at address 69 on i2c0. There is
> >> no driver for this device (and as far as I know all you can do with the
> >> 100MHz clocks is gate them), so I have chosen to model it as a single
> >> fixed clock.
> >> 
> >> Note: the SerDes1 lane numbering for the LS1046A is *reversed*.
> >> This means that Lane A (what the driver thinks is lane 0) uses pins
> >> SD1_TX3_P/N.
> >> 
> >> Because this will break ethernet if the serdes is not enabled, enable
> >> the serdes driver by default on Layerscape.
> >> 
> >> Signed-off-by: Sean Anderson <sean.anderson@seco.com>
> >> ---
> >> This depends on [1].
> >> 
> >> [1] https://lore.kernel.org/netdev/20220804194705.459670-4-sean.anderson@seco.com/
> >> 
> >> Changes in v9:
> >> - Fix name of phy mode node
> >> - phy-type -> fsl,phy
> >> 
> >> Changes in v8:
> >> - Rename serdes phy handles to use _A, _B, etc. instead of _0, _1, etc.
> >>   This should help remind readers that the numbering corresponds to the
> >>   physical layout of the registers, and not the lane (pin) number.
> >> 
> >> Changes in v6:
> >> - XGI.9 -> XFI.9
> >> 
> >> Changes in v4:
> >> - Convert to new bindings
> >> 
> >>  .../boot/dts/freescale/fsl-ls1046a-rdb.dts    | 112 ++++++++++++++++++
> >>  drivers/phy/freescale/Kconfig                 |   1 +
> > 
> > The phy driver Kconfig change shouldn't be part of this patch.
> 
> I put it here for bisectability, since this is the point where we need
> to enable it. But I can do this in a separate patch if you want.

From DT ABI perspective, it's already broken anyway if you need to change
kernel and DT atomically.

Shawn
Shawn Guo Jan. 27, 2023, 7:53 a.m. UTC | #13
On Thu, Jan 26, 2023 at 11:43:26AM -0500, Sean Anderson wrote:
> On 1/25/23 18:46, Shawn Guo wrote:
> > On Thu, Dec 29, 2022 at 07:01:35PM -0500, Sean Anderson wrote:
> >> This adds bindings for the SerDes devices. They are disabled by default
> > 
> > s/bindings/descriptions?
> > 
> > The term "bindings" generally means the schema/doc in
> > Documentation/devicetree/bindings/.
> 
> How about "nodes"?

Yeah, or device nodes.

Shawn
Sean Anderson Jan. 27, 2023, 4:11 p.m. UTC | #14
On 1/27/23 02:52, Shawn Guo wrote:
> On Thu, Jan 26, 2023 at 11:48:53AM -0500, Sean Anderson wrote:
>> On 1/25/23 18:43, Shawn Guo wrote:
>> > On Thu, Dec 29, 2022 at 07:01:36PM -0500, Sean Anderson wrote:
>> >> This adds appropriate bindings for the macs which use the SerDes. The
>> >> 156.25MHz fixed clock is a crystal. The 100MHz clocks (there are
>> >> actually 3) come from a Renesas 6V49205B at address 69 on i2c0. There is
>> >> no driver for this device (and as far as I know all you can do with the
>> >> 100MHz clocks is gate them), so I have chosen to model it as a single
>> >> fixed clock.
>> >> 
>> >> Note: the SerDes1 lane numbering for the LS1046A is *reversed*.
>> >> This means that Lane A (what the driver thinks is lane 0) uses pins
>> >> SD1_TX3_P/N.
>> >> 
>> >> Because this will break ethernet if the serdes is not enabled, enable
>> >> the serdes driver by default on Layerscape.
>> >> 
>> >> Signed-off-by: Sean Anderson <sean.anderson@seco.com>
>> >> ---
>> >> This depends on [1].
>> >> 
>> >> [1] https://lore.kernel.org/netdev/20220804194705.459670-4-sean.anderson@seco.com/
>> >> 
>> >> Changes in v9:
>> >> - Fix name of phy mode node
>> >> - phy-type -> fsl,phy
>> >> 
>> >> Changes in v8:
>> >> - Rename serdes phy handles to use _A, _B, etc. instead of _0, _1, etc.
>> >>   This should help remind readers that the numbering corresponds to the
>> >>   physical layout of the registers, and not the lane (pin) number.
>> >> 
>> >> Changes in v6:
>> >> - XGI.9 -> XFI.9
>> >> 
>> >> Changes in v4:
>> >> - Convert to new bindings
>> >> 
>> >>  .../boot/dts/freescale/fsl-ls1046a-rdb.dts    | 112 ++++++++++++++++++
>> >>  drivers/phy/freescale/Kconfig                 |   1 +
>> > 
>> > The phy driver Kconfig change shouldn't be part of this patch.
>> 
>> I put it here for bisectability, since this is the point where we need
>> to enable it. But I can do this in a separate patch if you want.
> 
> From DT ABI perspective, it's already broken anyway if you need to change
> kernel and DT atomically.

AIUI new kernels must work with old device trees, but new device trees need not
work with old kernels. So a change like this is fine, since the kernel won't
touch the serdes if it isn't supplied.

--Sean
Krzysztof Kozlowski Jan. 27, 2023, 4:15 p.m. UTC | #15
On 27/01/2023 17:11, Sean Anderson wrote:
>>>>>
>>>>>  .../boot/dts/freescale/fsl-ls1046a-rdb.dts    | 112 ++++++++++++++++++
>>>>>  drivers/phy/freescale/Kconfig                 |   1 +
>>>>
>>>> The phy driver Kconfig change shouldn't be part of this patch.
>>>
>>> I put it here for bisectability, since this is the point where we need
>>> to enable it. But I can do this in a separate patch if you want.
>>
>> From DT ABI perspective, it's already broken anyway if you need to change
>> kernel and DT atomically.
> 
> AIUI new kernels must work with old device trees, but new device trees need not
> work with old kernels. So a change like this is fine, since the kernel won't
> touch the serdes if it isn't supplied.

You used the argument "bisectability". If the patchset is not
bisectable, the ABI is broken.

Best regards,
Krzysztof
Sean Anderson Jan. 27, 2023, 4:22 p.m. UTC | #16
On 1/27/23 11:15, Krzysztof Kozlowski wrote:
> On 27/01/2023 17:11, Sean Anderson wrote:
>>>>>>
>>>>>>  .../boot/dts/freescale/fsl-ls1046a-rdb.dts    | 112 ++++++++++++++++++
>>>>>>  drivers/phy/freescale/Kconfig                 |   1 +
>>>>>
>>>>> The phy driver Kconfig change shouldn't be part of this patch.
>>>>
>>>> I put it here for bisectability, since this is the point where we need
>>>> to enable it. But I can do this in a separate patch if you want.
>>>
>>> From DT ABI perspective, it's already broken anyway if you need to change
>>> kernel and DT atomically.
>> 
>> AIUI new kernels must work with old device trees, but new device trees need not
>> work with old kernels. So a change like this is fine, since the kernel won't
>> touch the serdes if it isn't supplied.
> 
> You used the argument "bisectability". If the patchset is not
> bisectable, the ABI is broken.

Well, because Shawn wants it in a separate patch I am just going to enable
the driver by default on Layerscape before adding the device nodes. That way we have

1. Base state, driver not enabled and node is disabled
2. Driver enabled but not used because the node is disabled
3. Driver enabled and bound to node

So there is never a case where the node is bound but the driver isn't enabled
(which would cause the ethernet drivers to fail to probe).

--Sean
Krzysztof Kozlowski Jan. 27, 2023, 4:41 p.m. UTC | #17
On 27/01/2023 17:22, Sean Anderson wrote:
> On 1/27/23 11:15, Krzysztof Kozlowski wrote:
>> On 27/01/2023 17:11, Sean Anderson wrote:
>>>>>>>
>>>>>>>  .../boot/dts/freescale/fsl-ls1046a-rdb.dts    | 112 ++++++++++++++++++
>>>>>>>  drivers/phy/freescale/Kconfig                 |   1 +
>>>>>>
>>>>>> The phy driver Kconfig change shouldn't be part of this patch.
>>>>>
>>>>> I put it here for bisectability, since this is the point where we need
>>>>> to enable it. But I can do this in a separate patch if you want.
>>>>
>>>> From DT ABI perspective, it's already broken anyway if you need to change
>>>> kernel and DT atomically.
>>>
>>> AIUI new kernels must work with old device trees, but new device trees need not
>>> work with old kernels. So a change like this is fine, since the kernel won't
>>> touch the serdes if it isn't supplied.
>>
>> You used the argument "bisectability". If the patchset is not
>> bisectable, the ABI is broken.
> 
> Well, because Shawn wants it in a separate patch I am just going to enable
> the driver by default on Layerscape before adding the device nodes. That way we have
> 
> 1. Base state, driver not enabled and node is disabled
> 2. Driver enabled but not used because the node is disabled
> 3. Driver enabled and bound to node
> 
> So there is never a case where the node is bound but the driver isn't enabled
> (which would cause the ethernet drivers to fail to probe).

Then there is no bisectability issues and the Kconfig patch should have
been squashed here... Mentioning bisectability and that squash just
confuses.

Best regards,
Krzysztof
Sean Anderson Jan. 27, 2023, 4:42 p.m. UTC | #18
On 1/27/23 11:41, Krzysztof Kozlowski wrote:
> On 27/01/2023 17:22, Sean Anderson wrote:
>> On 1/27/23 11:15, Krzysztof Kozlowski wrote:
>>> On 27/01/2023 17:11, Sean Anderson wrote:
>>>>>>>>
>>>>>>>>  .../boot/dts/freescale/fsl-ls1046a-rdb.dts    | 112 ++++++++++++++++++
>>>>>>>>  drivers/phy/freescale/Kconfig                 |   1 +
>>>>>>>
>>>>>>> The phy driver Kconfig change shouldn't be part of this patch.
>>>>>>
>>>>>> I put it here for bisectability, since this is the point where we need
>>>>>> to enable it. But I can do this in a separate patch if you want.
>>>>>
>>>>> From DT ABI perspective, it's already broken anyway if you need to change
>>>>> kernel and DT atomically.
>>>>
>>>> AIUI new kernels must work with old device trees, but new device trees need not
>>>> work with old kernels. So a change like this is fine, since the kernel won't
>>>> touch the serdes if it isn't supplied.
>>>
>>> You used the argument "bisectability". If the patchset is not
>>> bisectable, the ABI is broken.
>> 
>> Well, because Shawn wants it in a separate patch I am just going to enable
>> the driver by default on Layerscape before adding the device nodes. That way we have
>> 
>> 1. Base state, driver not enabled and node is disabled
>> 2. Driver enabled but not used because the node is disabled
>> 3. Driver enabled and bound to node
>> 
>> So there is never a case where the node is bound but the driver isn't enabled
>> (which would cause the ethernet drivers to fail to probe).
> 
> Then there is no bisectability issues and the Kconfig patch should have
> been squashed here... Mentioning bisectability and that squash just
> confuses.

The Kconfig is currently squashed, but I am going to split it off as requested.

--Sean
Krzysztof Kozlowski Jan. 27, 2023, 4:44 p.m. UTC | #19
On 27/01/2023 17:42, Sean Anderson wrote:
> On 1/27/23 11:41, Krzysztof Kozlowski wrote:
>> On 27/01/2023 17:22, Sean Anderson wrote:
>>> On 1/27/23 11:15, Krzysztof Kozlowski wrote:
>>>> On 27/01/2023 17:11, Sean Anderson wrote:
>>>>>>>>>
>>>>>>>>>  .../boot/dts/freescale/fsl-ls1046a-rdb.dts    | 112 ++++++++++++++++++
>>>>>>>>>  drivers/phy/freescale/Kconfig                 |   1 +
>>>>>>>>
>>>>>>>> The phy driver Kconfig change shouldn't be part of this patch.
>>>>>>>
>>>>>>> I put it here for bisectability, since this is the point where we need
>>>>>>> to enable it. But I can do this in a separate patch if you want.
>>>>>>
>>>>>> From DT ABI perspective, it's already broken anyway if you need to change
>>>>>> kernel and DT atomically.
>>>>>
>>>>> AIUI new kernels must work with old device trees, but new device trees need not
>>>>> work with old kernels. So a change like this is fine, since the kernel won't
>>>>> touch the serdes if it isn't supplied.
>>>>
>>>> You used the argument "bisectability". If the patchset is not
>>>> bisectable, the ABI is broken.
>>>
>>> Well, because Shawn wants it in a separate patch I am just going to enable
>>> the driver by default on Layerscape before adding the device nodes. That way we have
>>>
>>> 1. Base state, driver not enabled and node is disabled
>>> 2. Driver enabled but not used because the node is disabled
>>> 3. Driver enabled and bound to node
>>>
>>> So there is never a case where the node is bound but the driver isn't enabled
>>> (which would cause the ethernet drivers to fail to probe).
>>
>> Then there is no bisectability issues and the Kconfig patch should have

"should have not been squashed", of course...

>> been squashed here... Mentioning bisectability and that squash just
>> confuses.
> 
> The Kconfig is currently squashed, but I am going to split it off as requested.

Yes, thanks.


Best regards,
Krzysztof
Stephen Boyd Jan. 27, 2023, 8:59 p.m. UTC | #20
Quoting Sean Anderson (2022-12-29 16:01:33)
> This adds support for the PLLs found in Lynx 10G "SerDes" devices found on
> various NXP QorIQ SoCs. There are two PLLs in each SerDes. This driver has
> been split from the main PHY driver to allow for better review, even though
> these PLLs are not present anywhere else besides the SerDes. An auxiliary
> device is not used as it offers no benefits over a function call (and there
> is no need to have a separate device).
> 
> The PLLs are modeled as clocks proper to let us take advantage of the
> existing clock infrastructure.

What advantage do we gain?

> I have not given the same treatment to the
> per-lane clocks because they need to be programmed in-concert with the rest
> of the lane settings. One tricky thing is that the VCO (PLL) rate exceeds
> 2^32 (maxing out at around 5GHz). This will be a problem on 32-bit
> platforms, since clock rates are stored as unsigned longs. To work around
> this, the pll clock rate is generally treated in units of kHz.

This looks like a disadvantage. Are we reporting the frequency in kHz to
the clk framework?

> 
> The PLLs are configured rather interestingly. Instead of the usual direct
> programming of the appropriate divisors, the input and output clock rates
> are selected directly. Generally, the only restriction is that the input
> and output must be integer multiples of each other. This suggests some kind
> of internal look-up table. The datasheets generally list out the supported
> combinations explicitly, and not all input/output combinations are
> documented. I'm not sure if this is due to lack of support, or due to an
> oversight. If this becomes an issue, then some combinations can be
> blacklisted (or whitelisted). This may also be necessary for other SoCs
> which have more stringent clock requirements.

I'm wondering if a clk provider should be created at all here. Who is
the consumer of the clk? The phy driver itself? Does the clk provided
need to interact with other clks in the system? Or is the clk tree
wholly self-contained?

Can the phy consumer configure the output frequency directly via
phy_configure() or when the phy is enabled? I'm thinking the phy driver
can call clk_set_rate() on the parent 'rfclk' before or after setting
the bits to control the output rate, and use clk_round_rate() to figure
out what input frequencies are supported for the output frequency
desired. This would avoid kHz overflowing 32-bits, and the big clk lock
getting blocked on some other clk in the system changing rates.

BTW, what sort of phy is this? Some networking device?

> 
> diff --git a/drivers/clk/clk-fsl-lynx-10g.c b/drivers/clk/clk-fsl-lynx-10g.c
> new file mode 100644
> index 000000000000..61f68b5ae675
> --- /dev/null
> +++ b/drivers/clk/clk-fsl-lynx-10g.c
> @@ -0,0 +1,509 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (C) 2022 Sean Anderson <sean.anderson@seco.com>
> + *
> + * This file contains the implementation for the PLLs found on Lynx 10G phys.
> + *
> + * XXX: The VCO rate of the PLLs can exceed ~4GHz, which is the maximum rate
> + * expressable in an unsigned long. To work around this, rates are specified in
> + * kHz. This is as if there was a division by 1000 in the PLL.
> + */
> +
> +#include <linux/clk.h>

Is this include used? If not, please remove.

> +#include <linux/clk-provider.h>
> +#include <linux/device.h>
> +#include <linux/bitfield.h>
> +#include <linux/math64.h>
> +#include <linux/phy/lynx-10g.h>
> +#include <linux/regmap.h>
> +#include <linux/slab.h>
> +#include <linux/units.h>
> +#include <dt-bindings/clock/fsl,lynx-10g.h>
> +
> +#define PLL_STRIDE     0x20
> +#define PLLa(a, off)   ((a) * PLL_STRIDE + (off))
> +#define PLLaRSTCTL(a)  PLLa(a, 0x00)
> +#define PLLaCR0(a)     PLLa(a, 0x04)
> +
> +#define PLLaRSTCTL_RSTREQ      BIT(31)
> +#define PLLaRSTCTL_RST_DONE    BIT(30)
> +#define PLLaRSTCTL_RST_ERR     BIT(29)
[...]
> +
> +static int lynx_clk_init(struct clk_hw_onecell_data *hw_data,
> +                        struct device *dev, struct regmap *regmap,
> +                        unsigned int index)
> +{
> +       const struct clk_hw *ex_dly_parents;
> +       struct clk_parent_data pll_parents[1] = { };
> +       struct clk_init_data pll_init = {
> +               .ops = &lynx_pll_clk_ops,
> +               .parent_data = pll_parents,
> +               .num_parents = 1,
> +               .flags = CLK_GET_RATE_NOCACHE | CLK_SET_RATE_PARENT |

Why is the nocache flag used?

> +                        CLK_OPS_PARENT_ENABLE,
> +       };
> +       struct clk_init_data ex_dly_init = {
> +               .ops = &lynx_ex_dly_clk_ops,
> +               .parent_hws = &ex_dly_parents,
> +               .num_parents = 1,
> +       };
> +       struct lynx_clk *clk;
> +       int ret;
Sean Anderson Jan. 27, 2023, 9:51 p.m. UTC | #21
On 1/27/23 15:59, Stephen Boyd wrote:
> Quoting Sean Anderson (2022-12-29 16:01:33)
>> This adds support for the PLLs found in Lynx 10G "SerDes" devices found on
>> various NXP QorIQ SoCs. There are two PLLs in each SerDes. This driver has
>> been split from the main PHY driver to allow for better review, even though
>> these PLLs are not present anywhere else besides the SerDes. An auxiliary
>> device is not used as it offers no benefits over a function call (and there
>> is no need to have a separate device).
>> 
>> The PLLs are modeled as clocks proper to let us take advantage of the
>> existing clock infrastructure.
> 
> What advantage do we gain?

The handling of rate changes, locking, etc is all done by the clock
subsystem. We can use the existing debugfs stuff as well. And everything
is organized by existing API boundaries.

>> I have not given the same treatment to the
>> per-lane clocks because they need to be programmed in-concert with the rest
>> of the lane settings. One tricky thing is that the VCO (PLL) rate exceeds
>> 2^32 (maxing out at around 5GHz). This will be a problem on 32-bit
>> platforms, since clock rates are stored as unsigned longs. To work around
>> this, the pll clock rate is generally treated in units of kHz.
> 
> This looks like a disadvantage. Are we reporting the frequency in kHz to
> the clk framework?

It is, and yes.

>> 
>> The PLLs are configured rather interestingly. Instead of the usual direct
>> programming of the appropriate divisors, the input and output clock rates
>> are selected directly. Generally, the only restriction is that the input
>> and output must be integer multiples of each other. This suggests some kind
>> of internal look-up table. The datasheets generally list out the supported
>> combinations explicitly, and not all input/output combinations are
>> documented. I'm not sure if this is due to lack of support, or due to an
>> oversight. If this becomes an issue, then some combinations can be
>> blacklisted (or whitelisted). This may also be necessary for other SoCs
>> which have more stringent clock requirements.
> 
> I'm wondering if a clk provider should be created at all here. Who is
> the consumer of the clk? The phy driver itself? Does the clk provided
> need to interact with other clks in the system? Or is the clk tree
> wholly self-contained?

It's wholly self-contained, aside from an undocumented mode where you
can output a fixed frequency (for testing). The provider exists so you
can use assigned-clocks to ensure a particular PLL is used for a
particular frequency. This prevents a problem where e.g. you have
reference clocks for 100 Hz and 156.25 Hz, one lane requests 5 GHz and
the PLL with the 156.25 Hz clock gets used. Then, later another lane
requests 5.15625 GHz and the remaining 100 Hz PLL can't provide it. To
prevent this, you can assign the PLLs their rates up front and the lanes
will use the existing rates. Most of the time this should be done by the
RCW, but there are cases where the RCW can't set up the clocks properly.

> Can the phy consumer configure the output frequency directly via
> phy_configure() or when the phy is enabled? I'm thinking the phy driver
> can call clk_set_rate() on the parent 'rfclk' before or after setting
> the bits to control the output rate,

This is what currently happens. See lynx_set_mode in the next patch.

> and use clk_round_rate() to figure
> out what input frequencies are supported for the output frequency
> desired. This would avoid kHz overflowing 32-bits, and the big clk lock
> getting blocked on some other clk in the system changing rates.

Can you describe this in a bit more detail? Are you suggesting to skip
the clock framework?

> BTW, what sort of phy is this? Some networking device?

From the Kconfig in the next commit:

This adds support for the Lynx "SerDes" devices found on various QorIQ
SoCs. There may be up to four SerDes devices on each SoC, and each
device supports up to eight lanes. The SerDes is configured by
default by the RCW, but this module is necessary in order to support
some modes (such as 2.5G SGMII or 1000BASE-KX), or clock setups (as
only as subset of clock configurations are supported by the RCW).
The hardware supports a variety of protocols, including Ethernet,
SATA, PCIe, and more exotic links such as Interlaken and Aurora. This
driver only supports Ethernet, but it will try not to touch lanes
configured for other protocols.

>> 
>> diff --git a/drivers/clk/clk-fsl-lynx-10g.c b/drivers/clk/clk-fsl-lynx-10g.c
>> new file mode 100644
>> index 000000000000..61f68b5ae675
>> --- /dev/null
>> +++ b/drivers/clk/clk-fsl-lynx-10g.c
>> @@ -0,0 +1,509 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * Copyright (C) 2022 Sean Anderson <sean.anderson@seco.com>
>> + *
>> + * This file contains the implementation for the PLLs found on Lynx 10G phys.
>> + *
>> + * XXX: The VCO rate of the PLLs can exceed ~4GHz, which is the maximum rate
>> + * expressable in an unsigned long. To work around this, rates are specified in
>> + * kHz. This is as if there was a division by 1000 in the PLL.
>> + */
>> +
>> +#include <linux/clk.h>
> 
> Is this include used? If not, please remove.

OK

>> +#include <linux/clk-provider.h>
>> +#include <linux/device.h>
>> +#include <linux/bitfield.h>
>> +#include <linux/math64.h>
>> +#include <linux/phy/lynx-10g.h>
>> +#include <linux/regmap.h>
>> +#include <linux/slab.h>
>> +#include <linux/units.h>
>> +#include <dt-bindings/clock/fsl,lynx-10g.h>
>> +
>> +#define PLL_STRIDE     0x20
>> +#define PLLa(a, off)   ((a) * PLL_STRIDE + (off))
>> +#define PLLaRSTCTL(a)  PLLa(a, 0x00)
>> +#define PLLaCR0(a)     PLLa(a, 0x04)
>> +
>> +#define PLLaRSTCTL_RSTREQ      BIT(31)
>> +#define PLLaRSTCTL_RST_DONE    BIT(30)
>> +#define PLLaRSTCTL_RST_ERR     BIT(29)
> [...]
>> +
>> +static int lynx_clk_init(struct clk_hw_onecell_data *hw_data,
>> +                        struct device *dev, struct regmap *regmap,
>> +                        unsigned int index)
>> +{
>> +       const struct clk_hw *ex_dly_parents;
>> +       struct clk_parent_data pll_parents[1] = { };
>> +       struct clk_init_data pll_init = {
>> +               .ops = &lynx_pll_clk_ops,
>> +               .parent_data = pll_parents,
>> +               .num_parents = 1,
>> +               .flags = CLK_GET_RATE_NOCACHE | CLK_SET_RATE_PARENT |
> 
> Why is the nocache flag used?

PCIe rate selection can automatically modify the frequency of the
PLL. From the reference manual:

| For PCIe 8 Gbaud, the auto-negotiation also involves a combination of
| switching PLL selects, and/or reconfiguring a PLL. There are two
| scenarios for PLL reconfiguration:
|
| * PCIe starts on any PLL which runs at 5GHz, and the other PLL is off.
| PHY wrapper will switch to the other PLL as part of Gen 3 speed
| switch. In this case, both PLLs must be supplied with valid reference
| clocks.
| * PCIe starts on any PLL which runs at 5GHz, and the other PLL is
| being used for other protocols. PHY wrapper will reconfigure the
| current 5GHz PLL as part of Gen 3 speed switch. PCIe stays on the same
| PLL in this case. This also applies to the scenario where one PCIe
| port runs on all lanes.

As far as I know there's no way to configure or disable this. PCIe isn't
implemented yet, but it would likely need some way to mark PLLs for
exclusive use (not just exclusive rate). I figured it would be better
not to cache the rate so that things like the above would get detected
properly.

--Sean

>> +                        CLK_OPS_PARENT_ENABLE,
>> +       };
>> +       struct clk_init_data ex_dly_init = {
>> +               .ops = &lynx_ex_dly_clk_ops,
>> +               .parent_hws = &ex_dly_parents,
>> +               .num_parents = 1,
>> +       };
>> +       struct lynx_clk *clk;
>> +       int ret;
Sean Anderson March 3, 2023, 5:25 p.m. UTC | #22
Hi Stephen,

On 1/27/23 16:51, Sean Anderson wrote:
> On 1/27/23 15:59, Stephen Boyd wrote:
>> Quoting Sean Anderson (2022-12-29 16:01:33)
>>> This adds support for the PLLs found in Lynx 10G "SerDes" devices found on
>>> various NXP QorIQ SoCs. There are two PLLs in each SerDes. This driver has
>>> been split from the main PHY driver to allow for better review, even though
>>> these PLLs are not present anywhere else besides the SerDes. An auxiliary
>>> device is not used as it offers no benefits over a function call (and there
>>> is no need to have a separate device).
>>> 
>>> The PLLs are modeled as clocks proper to let us take advantage of the
>>> existing clock infrastructure.
>> 
>> What advantage do we gain?
> 
> The handling of rate changes, locking, etc is all done by the clock
> subsystem. We can use the existing debugfs stuff as well. And everything
> is organized by existing API boundaries.
> 
>>> I have not given the same treatment to the
>>> per-lane clocks because they need to be programmed in-concert with the rest
>>> of the lane settings. One tricky thing is that the VCO (PLL) rate exceeds
>>> 2^32 (maxing out at around 5GHz). This will be a problem on 32-bit
>>> platforms, since clock rates are stored as unsigned longs. To work around
>>> this, the pll clock rate is generally treated in units of kHz.
>> 
>> This looks like a disadvantage. Are we reporting the frequency in kHz to
>> the clk framework?
> 
> It is, and yes.
> 
>>> 
>>> The PLLs are configured rather interestingly. Instead of the usual direct
>>> programming of the appropriate divisors, the input and output clock rates
>>> are selected directly. Generally, the only restriction is that the input
>>> and output must be integer multiples of each other. This suggests some kind
>>> of internal look-up table. The datasheets generally list out the supported
>>> combinations explicitly, and not all input/output combinations are
>>> documented. I'm not sure if this is due to lack of support, or due to an
>>> oversight. If this becomes an issue, then some combinations can be
>>> blacklisted (or whitelisted). This may also be necessary for other SoCs
>>> which have more stringent clock requirements.
>> 
>> I'm wondering if a clk provider should be created at all here. Who is
>> the consumer of the clk? The phy driver itself? Does the clk provided
>> need to interact with other clks in the system? Or is the clk tree
>> wholly self-contained?
> 
> It's wholly self-contained, aside from an undocumented mode where you
> can output a fixed frequency (for testing). The provider exists so you
> can use assigned-clocks to ensure a particular PLL is used for a
> particular frequency. This prevents a problem where e.g. you have
> reference clocks for 100 Hz and 156.25 Hz, one lane requests 5 GHz and
> the PLL with the 156.25 Hz clock gets used. Then, later another lane
> requests 5.15625 GHz and the remaining 100 Hz PLL can't provide it. To
> prevent this, you can assign the PLLs their rates up front and the lanes
> will use the existing rates. Most of the time this should be done by the
> RCW, but there are cases where the RCW can't set up the clocks properly.
> 
>> Can the phy consumer configure the output frequency directly via
>> phy_configure() or when the phy is enabled? I'm thinking the phy driver
>> can call clk_set_rate() on the parent 'rfclk' before or after setting
>> the bits to control the output rate,
> 
> This is what currently happens. See lynx_set_mode in the next patch.
> 
>> and use clk_round_rate() to figure
>> out what input frequencies are supported for the output frequency
>> desired. This would avoid kHz overflowing 32-bits, and the big clk lock
>> getting blocked on some other clk in the system changing rates.
> 
> Can you describe this in a bit more detail? Are you suggesting to skip
> the clock framework?
> 
>> BTW, what sort of phy is this? Some networking device?
> 
> From the Kconfig in the next commit:
> 
> This adds support for the Lynx "SerDes" devices found on various QorIQ
> SoCs. There may be up to four SerDes devices on each SoC, and each
> device supports up to eight lanes. The SerDes is configured by
> default by the RCW, but this module is necessary in order to support
> some modes (such as 2.5G SGMII or 1000BASE-KX), or clock setups (as
> only as subset of clock configurations are supported by the RCW).
> The hardware supports a variety of protocols, including Ethernet,
> SATA, PCIe, and more exotic links such as Interlaken and Aurora. This
> driver only supports Ethernet, but it will try not to touch lanes
> configured for other protocols.
> 
>>> 
>>> diff --git a/drivers/clk/clk-fsl-lynx-10g.c b/drivers/clk/clk-fsl-lynx-10g.c
>>> new file mode 100644
>>> index 000000000000..61f68b5ae675
>>> --- /dev/null
>>> +++ b/drivers/clk/clk-fsl-lynx-10g.c
>>> @@ -0,0 +1,509 @@
>>> +// SPDX-License-Identifier: GPL-2.0
>>> +/*
>>> + * Copyright (C) 2022 Sean Anderson <sean.anderson@seco.com>
>>> + *
>>> + * This file contains the implementation for the PLLs found on Lynx 10G phys.
>>> + *
>>> + * XXX: The VCO rate of the PLLs can exceed ~4GHz, which is the maximum rate
>>> + * expressable in an unsigned long. To work around this, rates are specified in
>>> + * kHz. This is as if there was a division by 1000 in the PLL.
>>> + */
>>> +
>>> +#include <linux/clk.h>
>> 
>> Is this include used? If not, please remove.
> 
> OK
> 
>>> +#include <linux/clk-provider.h>
>>> +#include <linux/device.h>
>>> +#include <linux/bitfield.h>
>>> +#include <linux/math64.h>
>>> +#include <linux/phy/lynx-10g.h>
>>> +#include <linux/regmap.h>
>>> +#include <linux/slab.h>
>>> +#include <linux/units.h>
>>> +#include <dt-bindings/clock/fsl,lynx-10g.h>
>>> +
>>> +#define PLL_STRIDE     0x20
>>> +#define PLLa(a, off)   ((a) * PLL_STRIDE + (off))
>>> +#define PLLaRSTCTL(a)  PLLa(a, 0x00)
>>> +#define PLLaCR0(a)     PLLa(a, 0x04)
>>> +
>>> +#define PLLaRSTCTL_RSTREQ      BIT(31)
>>> +#define PLLaRSTCTL_RST_DONE    BIT(30)
>>> +#define PLLaRSTCTL_RST_ERR     BIT(29)
>> [...]
>>> +
>>> +static int lynx_clk_init(struct clk_hw_onecell_data *hw_data,
>>> +                        struct device *dev, struct regmap *regmap,
>>> +                        unsigned int index)
>>> +{
>>> +       const struct clk_hw *ex_dly_parents;
>>> +       struct clk_parent_data pll_parents[1] = { };
>>> +       struct clk_init_data pll_init = {
>>> +               .ops = &lynx_pll_clk_ops,
>>> +               .parent_data = pll_parents,
>>> +               .num_parents = 1,
>>> +               .flags = CLK_GET_RATE_NOCACHE | CLK_SET_RATE_PARENT |
>> 
>> Why is the nocache flag used?
> 
> PCIe rate selection can automatically modify the frequency of the
> PLL. From the reference manual:
> 
> | For PCIe 8 Gbaud, the auto-negotiation also involves a combination of
> | switching PLL selects, and/or reconfiguring a PLL. There are two
> | scenarios for PLL reconfiguration:
> |
> | * PCIe starts on any PLL which runs at 5GHz, and the other PLL is off.
> | PHY wrapper will switch to the other PLL as part of Gen 3 speed
> | switch. In this case, both PLLs must be supplied with valid reference
> | clocks.
> | * PCIe starts on any PLL which runs at 5GHz, and the other PLL is
> | being used for other protocols. PHY wrapper will reconfigure the
> | current 5GHz PLL as part of Gen 3 speed switch. PCIe stays on the same
> | PLL in this case. This also applies to the scenario where one PCIe
> | port runs on all lanes.
> 
> As far as I know there's no way to configure or disable this. PCIe isn't
> implemented yet, but it would likely need some way to mark PLLs for
> exclusive use (not just exclusive rate). I figured it would be better
> not to cache the rate so that things like the above would get detected
> properly.
> 
> --Sean
> 
>>> +                        CLK_OPS_PARENT_ENABLE,
>>> +       };
>>> +       struct clk_init_data ex_dly_init = {
>>> +               .ops = &lynx_ex_dly_clk_ops,
>>> +               .parent_hws = &ex_dly_parents,
>>> +               .num_parents = 1,
>>> +       };
>>> +       struct lynx_clk *clk;
>>> +       int ret;

I am going to try and send out v10 soon. Do you have any comments on my
reply here? At the moment the only change I am planning to make is to
remove the clk.h include.

--Sean