| Message ID | 20230321201313.2507539-1-sean.anderson@seco.com |
|---|---|
| Headers | show |
| Series | phy: Add support for Lynx 10G SerDes | expand |
On Tue, Mar 21, 2023 at 04:13:12PM -0400, Sean Anderson wrote: > This adds serdes support to the LS1088ARDB. I have tested the QSGMII > ports as well as the two 10G ports. The SFP slot is now fully supported, > instead of being modeled as a fixed-link. > > Linux hangs around when the serdes is initialized if the si5341 is > enabled with the in-tree driver, so I have modeled it as a two fixed > clocks instead. There are a few registers in the QIXIS FPGA which > control the SFP GPIOs; I have modeled them as discrete GPIO controllers > for now. I never saw the AQR105 interrupt fire; not sure what was going > on, but I have removed it to force polling. So you didn't see the interrupt fire even without these patches? I just tested this on a LS1088ARDB and it works. root@localhost:~# cat /proc/interrupts | grep extirq 99: 5 ls-extirq 2 Level 0x0000000008b97000:00 root@localhost:~# ip link set dev endpmac2 up root@localhost:~# cat /proc/interrupts | grep extirq 99: 6 ls-extirq 2 Level 0x0000000008b97000:00 root@localhost:~# ip link set dev endpmac2 down root@localhost:~# cat /proc/interrupts | grep extirq 99: 7 ls-extirq 2 Level 0x0000000008b97000:00 Please don't just remove things. Ioana
On 3/24/23 09:17, Ioana Ciornei wrote: > On Tue, Mar 21, 2023 at 04:13:12PM -0400, Sean Anderson wrote: >> This adds serdes support to the LS1088ARDB. I have tested the QSGMII >> ports as well as the two 10G ports. The SFP slot is now fully supported, >> instead of being modeled as a fixed-link. >> >> Linux hangs around when the serdes is initialized if the si5341 is >> enabled with the in-tree driver, so I have modeled it as a two fixed >> clocks instead. There are a few registers in the QIXIS FPGA which >> control the SFP GPIOs; I have modeled them as discrete GPIO controllers >> for now. I never saw the AQR105 interrupt fire; not sure what was going >> on, but I have removed it to force polling. > > So you didn't see the interrupt fire even without these patches? Not sure. I went to check this, and discovered I could no longer get the link to come up in Linux, even on v6.0 (before the rate adaptation tuff). I see the LEDs blinking in U-Boot, so presumably it's some configuration problem. I'm going to look into this further when I have more time. > I just tested this on a LS1088ARDB and it works. > > root@localhost:~# cat /proc/interrupts | grep extirq > 99: 5 ls-extirq 2 Level 0x0000000008b97000:00 > root@localhost:~# ip link set dev endpmac2 up > root@localhost:~# cat /proc/interrupts | grep extirq > 99: 6 ls-extirq 2 Level 0x0000000008b97000:00 > root@localhost:~# ip link set dev endpmac2 down > root@localhost:~# cat /proc/interrupts | grep extirq > 99: 7 ls-extirq 2 Level 0x0000000008b97000:00 > > Please don't just remove things. Well, polling isn't the worst thing for a single interface... I do remember having a problem with the interrupt. If this series works with interrupts enabled, I can leave it in. Did you have a chance to look at the core (patches 7 and 8) of this series? Does it make sense to you? Am I missing something which would allow switching from 1G->10G? --Sean
On 3/27/23 14:15, Sean Anderson wrote: > On 3/24/23 09:17, Ioana Ciornei wrote: >> On Tue, Mar 21, 2023 at 04:13:12PM -0400, Sean Anderson wrote: >>> This adds serdes support to the LS1088ARDB. I have tested the QSGMII >>> ports as well as the two 10G ports. The SFP slot is now fully supported, >>> instead of being modeled as a fixed-link. >>> >>> Linux hangs around when the serdes is initialized if the si5341 is >>> enabled with the in-tree driver, so I have modeled it as a two fixed >>> clocks instead. There are a few registers in the QIXIS FPGA which >>> control the SFP GPIOs; I have modeled them as discrete GPIO controllers >>> for now. I never saw the AQR105 interrupt fire; not sure what was going >>> on, but I have removed it to force polling. >> >> So you didn't see the interrupt fire even without these patches? > > Not sure. I went to check this, and discovered I could no longer get the > link to come up in Linux, even on v6.0 (before the rate adaptation > tuff). I see the LEDs blinking in U-Boot, so presumably it's some > configuration problem. I'm going to look into this further when I have > more time. I figured it out. I forgot the dpmac2 was 10G only (no rate adaptation for the AQR105). And the interrupt does not fire on net/main: # cat /proc/interrupts | grep extirq 22: 0 0 0 0 0 0 0 0 ls-extirq 2 Level 0x0000000008b97000:00 Inspecting the phy manually shows the link coming up. By removing the interrupt, the link comes up as usual. I wanted to look into this further, but the IRQ goes through the QIXIS and the firmware source isn't available so I wasn't able to do so. If you'd like, I can try probing the signal (to see where the problem is), but I won't have time for a bit. >> I just tested this on a LS1088ARDB and it works. >> >> root@localhost:~# cat /proc/interrupts | grep extirq >> 99: 5 ls-extirq 2 Level 0x0000000008b97000:00 >> root@localhost:~# ip link set dev endpmac2 up >> root@localhost:~# cat /proc/interrupts | grep extirq >> 99: 6 ls-extirq 2 Level 0x0000000008b97000:00 >> root@localhost:~# ip link set dev endpmac2 down >> root@localhost:~# cat /proc/interrupts | grep extirq >> 99: 7 ls-extirq 2 Level 0x0000000008b97000:00 >> >> Please don't just remove things. > > Well, polling isn't the worst thing for a single interface... I do > remember having a problem with the interrupt. If this series works > with interrupts enabled, I can leave it in. Anyway, given that interrupts seem to be broken for some boards? some configurations? I would like to keep polling. > Did you have a chance to look at the core (patches 7 and 8) of this > series? Does it make sense to you? Am I missing something which would > allow switching from 1G->10G? --Sean
On Mon, Mar 27, 2023 at 02:15:47PM -0400, Sean Anderson wrote: > On 3/24/23 09:17, Ioana Ciornei wrote: > > On Tue, Mar 21, 2023 at 04:13:12PM -0400, Sean Anderson wrote: > >> This adds serdes support to the LS1088ARDB. I have tested the QSGMII > >> ports as well as the two 10G ports. The SFP slot is now fully supported, > >> instead of being modeled as a fixed-link. > >> > >> Linux hangs around when the serdes is initialized if the si5341 is > >> enabled with the in-tree driver, so I have modeled it as a two fixed > >> clocks instead. There are a few registers in the QIXIS FPGA which > >> control the SFP GPIOs; I have modeled them as discrete GPIO controllers > >> for now. I never saw the AQR105 interrupt fire; not sure what was going > >> on, but I have removed it to force polling. > > > > So you didn't see the interrupt fire even without these patches? > > Not sure. I went to check this, and discovered I could no longer get the > link to come up in Linux, even on v6.0 (before the rate adaptation > tuff). I see the LEDs blinking in U-Boot, so presumably it's some > configuration problem. I'm going to look into this further when I have > more time. > > > I just tested this on a LS1088ARDB and it works. > > > > root@localhost:~# cat /proc/interrupts | grep extirq > > 99: 5 ls-extirq 2 Level 0x0000000008b97000:00 > > root@localhost:~# ip link set dev endpmac2 up > > root@localhost:~# cat /proc/interrupts | grep extirq > > 99: 6 ls-extirq 2 Level 0x0000000008b97000:00 > > root@localhost:~# ip link set dev endpmac2 down > > root@localhost:~# cat /proc/interrupts | grep extirq > > 99: 7 ls-extirq 2 Level 0x0000000008b97000:00 > > > > Please don't just remove things. > > Well, polling isn't the worst thing for a single interface... I do > remember having a problem with the interrupt. If this series works > with interrupts enabled, I can leave it in. > > Did you have a chance to look at the core (patches 7 and 8) of this > series? Does it make sense to you? Am I missing something which would > allow switching from 1G->10G? > For a bit of context, I also attempted dynamic switching from 1G to 10G on my own even before this patch set but I did not get a link up on the PCS (CDR lock was there through). Pretty much the same state as you. What I propose is to take this whole endeavor step by step. I am also interrested in getting this feature to actually work but I just didn't have the time to investigate in depth was is missing. And without the dynamic switching I cannot say that I find the addition of the SerDes PHY driver useful. I have the Lynx 10G on my TODO list but I still have some other tasks on the Lynx 28G for the next 2-3 weeks. Once I get those done, I will look closer at the patches. In the meantime, some small thigs from this patch set can be submitted separately. For example, describing the SFP cage on the LS1088ARDB. I still have some small questions on the DTS implementation for the gpio controllers but I would be able to submit this myself if you do not find the time (with your authorship of course). Ioana
On 3/28/23 05:25, Ioana Ciornei wrote: > On Mon, Mar 27, 2023 at 02:15:47PM -0400, Sean Anderson wrote: >> On 3/24/23 09:17, Ioana Ciornei wrote: >> > On Tue, Mar 21, 2023 at 04:13:12PM -0400, Sean Anderson wrote: >> >> This adds serdes support to the LS1088ARDB. I have tested the QSGMII >> >> ports as well as the two 10G ports. The SFP slot is now fully supported, >> >> instead of being modeled as a fixed-link. >> >> >> >> Linux hangs around when the serdes is initialized if the si5341 is >> >> enabled with the in-tree driver, so I have modeled it as a two fixed >> >> clocks instead. There are a few registers in the QIXIS FPGA which >> >> control the SFP GPIOs; I have modeled them as discrete GPIO controllers >> >> for now. I never saw the AQR105 interrupt fire; not sure what was going >> >> on, but I have removed it to force polling. >> > >> > So you didn't see the interrupt fire even without these patches? >> >> Not sure. I went to check this, and discovered I could no longer get the >> link to come up in Linux, even on v6.0 (before the rate adaptation >> tuff). I see the LEDs blinking in U-Boot, so presumably it's some >> configuration problem. I'm going to look into this further when I have >> more time. >> >> > I just tested this on a LS1088ARDB and it works. >> > >> > root@localhost:~# cat /proc/interrupts | grep extirq >> > 99: 5 ls-extirq 2 Level 0x0000000008b97000:00 >> > root@localhost:~# ip link set dev endpmac2 up >> > root@localhost:~# cat /proc/interrupts | grep extirq >> > 99: 6 ls-extirq 2 Level 0x0000000008b97000:00 >> > root@localhost:~# ip link set dev endpmac2 down >> > root@localhost:~# cat /proc/interrupts | grep extirq >> > 99: 7 ls-extirq 2 Level 0x0000000008b97000:00 >> > >> > Please don't just remove things. >> >> Well, polling isn't the worst thing for a single interface... I do >> remember having a problem with the interrupt. If this series works >> with interrupts enabled, I can leave it in. >> >> Did you have a chance to look at the core (patches 7 and 8) of this >> series? Does it make sense to you? Am I missing something which would >> allow switching from 1G->10G? >> > > For a bit of context, I also attempted dynamic switching from 1G to 10G > on my own even before this patch set but I did not get a link up on the > PCS (CDR lock was there through). Pretty much the same state as you. > > What I propose is to take this whole endeavor step by step. > I am also interrested in getting this feature to actually work but I > just didn't have the time to investigate in depth was is missing. > And without the dynamic switching I cannot say that I find the addition > of the SerDes PHY driver useful. Well, it's still useful for supporting 1G and 10G. I touched on this in the cover letter, but there are conflicting PLL defaults on the LS1046A. If you use SRDS_PRCTL 1133, the PLL mapping is 2211. But for 3333 it's 2222. This means PLL2 is used for both 1G and 10G, but no reference frequency can work for both. This will cause the PBL to enter a reset loop, since it wants the PLLs to lock before booting, and this happens before any user configuration. To get around this, we disconnected RESET_REQ_B on our board, and we use this driver to configure the PLLs correctly for whatever SRDS_PRCTL we boot up with. This way we can have two RCWs for 1G and 10G configuration. > I have the Lynx 10G on my TODO list but I still have some other tasks > on the Lynx 28G for the next 2-3 weeks. Once I get those done, I will > look closer at the patches. OK, thanks. > In the meantime, some small thigs from this patch set can be submitted > separately. For example, describing the SFP cage on the LS1088ARDB. I'll have a look at this. I suppose I will also split off the IRQ thing. > I still have some small questions on the DTS implementation for the gpio > controllers but I would be able to submit this myself if you do not find > the time (with your authorship of course). I am going to do another revision to address the GPIO binding problem, so please ask away. --Sean
This adds support for the Lynx 10G SerDes found on the QorIQ T-series and Layerscape series. Due to limited time and hardware, only support for the LS1046ARDB and LS1088ARDB is added in this initial series. This series is ready for review by the phy maintainers. I have addressed all known feedback and there are no outstanding issues. Major reconfiguration of baud rate (e.g. 1G->10G) does not work. From my testing, SerDes register settings appear identical. The issue appears to be between the PCS and the MAC. The link itself comes up at both ends, and a mac loopback succeeds. However, a PCS loopback results in dropped packets. Perhaps there is some undocumented register in the PCS? I suspect this driver is around 95% complete, but I don't have the documentation to make it work completely. At the very least it is useful for two cases: - Although this is untested, it should support 2.5G SGMII as well as 1000BASE-KX. The latter needs MAC and PCS support, but the former should work out of the box. - It allows for clock configurations not supported by the RCW. This is very useful if you want to use e.g. SRDS_PRTCL_S1=0x3333 and =0x1133 on the same board. This is because the former setting will use PLL1 as the 1G reference, but the latter will use PLL1 as the 10G reference. Because we can reconfigure the PLLs, it is possible to always use PLL1 as the 1G reference. Changes in v12: - Put compatible first - Keep gpio-controller to one line - Add little-endian property - Alphabetize compatibles - Remove some comments - Remove some examples with insufficient novelty Changes in v11: - Keep empty (or almost-empty) properties on a single line - Don't use | unnecessarily - Use gpio as the node name for examples - Rename brcm,bcm6345-gpio.yaml to brcm,bcm63xx-gpio.yaml Changes in v10: - Convert gpio-mmio to yaml - Add compatible for QIXIS - Remove unnecessary inclusion of clk.h - Don't gate clocks in compatibility mode - Fix debugging print with incorrect error variable - Move serdes bindings to SoC dtsi - Add support for all (ethernet) serdes modes - Refer to "nodes" instead of "bindings" - Move compatible/reg first Changes in v9: - Add fsl,unused-lanes-reserved to allow for a gradual transition between firmware and Linux control of the SerDes - Change phy-type back to fsl,type, as I was getting the error '#phy-cells' is a dependency of 'phy-type' - Convert some u32s to unsigned long to match arguments - Switch from round_rate to determine_rate - Drop explicit reference to reference clock - Use .parent_names when requesting parents - Use devm_clk_hw_get_clk to pass clocks back to serdes - Fix indentation - Split off clock "driver" into its own patch to allow for better review. - Add ability to defer lane initialization to phy_init. This allows for easier transitioning between firmware-managed serdes and Linux- managed serdes, as the consumer (such as dpaa2, which knows what the firmware is doing) has the last say on who gets control. - Fix name of phy mode node - Add fsl,unused-lanes-reserved to allow a gradual transition, depending on the mac link type. - Remove unused clocks - Fix some phy mode node names Changes in v8: - Remove unused variable from lynx_ls_mode_init - Rename serdes phy handles to use _A, _B, etc. instead of _0, _1, etc. This should help remind readers that the numbering corresponds to the physical layout of the registers, and not the lane (pin) number. - Prevent PCSs from probing as phys - Rename serdes phy handles like the LS1046A - Add SFP slot binding - Fix incorrect lane ordering (it's backwards on the LS1088A just like it is in the LS1046A). - Fix duplicated lane 2 (it should have been lane 3). - Fix incorrectly-documented value for XFI1. - Remove interrupt for aquantia phy. It never fired for whatever reason, preventing the link from coming up. - Add GPIOs for QIXIS FPGA. - Enable MAC1 PCS - Remove si5341 binding Changes in v7: - Use double quotes everywhere in yaml - Break out call order into generic documentation - Refuse to switch "major" protocols - Update Kconfig to reflect restrictions - Remove set/clear of "pcs reset" bit, since it doesn't seem to fix anything. Changes in v6: - Bump PHY_TYPE_2500BASEX to 13, since PHY_TYPE_USXGMII was added in the meantime - fsl,type -> phy-type - frequence -> frequency - Update MAINTAINERS to include new files - Include bitfield.h and slab.h to allow compilation on non-arm64 arches. - Depend on COMMON_CLK and either layerscape/ppc - XGI.9 -> XFI.9 Changes in v5: - Update commit description - Dual id header - Remove references to PHY_INTERFACE_MODE_1000BASEKX to allow this series to be applied directly to linux/master. - Add fsl,lynx-10g.h to MAINTAINERS Changes in v4: - Add 2500BASE-X and 10GBASE-R phy types - Use subnodes to describe lane configuration, instead of describing PCCRs. This is the same style used by phy-cadence-sierra et al. - Add ids for Lynx 10g PLLs - Rework all debug statements to remove use of __func__. Additional information has been provided as necessary. - Consider alternative parent rates in round_rate and not in set_rate. Trying to modify out parent's rate in set_rate will deadlock. - Explicitly perform a stop/reset sequence in set_rate. This way we always ensure that the PLL is properly stopped. - Set the power-down bit when disabling the PLL. We can do this now that enable/disable aren't abused during the set rate sequence. - Fix typos in QSGMII_OFFSET and XFI_OFFSET - Rename LNmTECR0_TEQ_TYPE_PRE to LNmTECR0_TEQ_TYPE_POST to better reflect its function (adding post-cursor equalization). - Use of_clk_hw_onecell_get instead of a custom function. - Return struct clks from lynx_clks_init instead of embedding lynx_clk in lynx_priv. - Rework PCCR helper functions; T-series SoCs differ from Layerscape SoCs primarily in the layout and offset of the PCCRs. This will help bring a cleaner abstraction layer. The caps have been removed, since this handles the only current usage. - Convert to use new binding format. As a result of this, we no longer need to have protocols for PCIe or SATA. Additionally, modes now live in lynx_group instead of lynx_priv. - Remove teq from lynx_proto_params, since it can be determined from preq_ratio/postq_ratio. - Fix an early return from lynx_set_mode not releasing serdes->lock. - Rename lynx_priv.conf to .cfg, since I kept mistyping it. Changes in v3: - Manually expand yaml references - Add mode configuration to device tree - Rename remaining references to QorIQ SerDes to Lynx 10G - Fix PLL enable sequence by waiting for our reset request to be cleared before continuing. Do the same for the lock, even though it isn't as critical. Because we will delay for 1.5ms on average, use prepare instead of enable so we can sleep. - Document the status of each protocol - Fix offset of several bitfields in RECR0 - Take into account PLLRST_B, SDRST_B, and SDEN when considering whether a PLL is "enabled." - Only power off unused lanes. - Split mode lane mask into first/last lane (like group) - Read modes from device tree - Use caps to determine whether KX/KR are supported - Move modes to lynx_priv - Ensure that the protocol controller is not already in-use when we try to configure a new mode. This should only occur if the device tree is misconfigured (e.g. when QSGMII is selected on two lanes but there is only one QSGMII controller). - Split PLL drivers off into their own file - Add clock for "ext_dly" instead of writing the bit directly (and racing with any clock code). - Use kasprintf instead of open-coding the snprintf dance - Support 1000BASE-KX in lynx_lookup_proto. This still requires PCS support, so nothing is truly "enabled" yet. - Describe modes in device tree - ls1088a: Add serdes bindings Changes in v2: - Rename to fsl,lynx-10g.yaml - Refer to the device in the documentation, rather than the binding - Move compatible first - Document phy cells in the description - Allow a value of 1 for phy-cells. This allows for compatibility with the similar (but according to Ioana Ciornei different enough) lynx-28g binding. - Remove minItems - Use list for clock-names - Fix example binding having too many cells in regs - Add #clock-cells. This will allow using assigned-clocks* to configure the PLLs. - Document the structure of the compatible strings - Rename driver to Lynx 10G (etc.) - Fix not clearing group->pll after disabling it - Support 1 and 2 phy-cells - Power off lanes during probe - Clear SGMIIaCR1_PCS_EN during probe - Rename LYNX_PROTO_UNKNOWN to LYNX_PROTO_NONE - Handle 1000BASE-KX in lynx_proto_mode_prep - Use one phy cell for SerDes1, since no lanes can be grouped - Disable SerDes by default to prevent breaking boards inadvertently. Sean Anderson (13): dt-bindings: phy: Add 2500BASE-X and 10GBASE-R dt-bindings: phy: Add Lynx 10G phy binding dt-bindings: Convert gpio-mmio to yaml dt-bindings: gpio-mmio: Add compatible for QIXIS dt-bindings: clock: Add ids for Lynx 10g PLLs clk: Add Lynx 10G SerDes PLL driver phy: fsl: Add Lynx 10G SerDes driver phy: lynx10g: Enable by default on Layerscape arm64: dts: ls1046a: Add serdes nodes arm64: dts: ls1046ardb: Add serdes descriptions arm64: dts: ls1088a: Add serdes nodes arm64: dts: ls1088a: Prevent PCSs from probing as phys arm64: dts: ls1088ardb: Add serdes descriptions ...m6345-gpio.yaml => brcm,bcm63xx-gpio.yaml} | 16 +- .../devicetree/bindings/gpio/gpio-mmio.yaml | 123 ++ .../bindings/gpio/ni,169445-nand-gpio.txt | 38 - .../devicetree/bindings/gpio/wd,mbl-gpio.txt | 38 - .../devicetree/bindings/phy/fsl,lynx-10g.yaml | 248 ++++ Documentation/driver-api/phy/index.rst | 1 + Documentation/driver-api/phy/lynx_10g.rst | 58 + MAINTAINERS | 9 + .../boot/dts/freescale/fsl-ls1046a-rdb.dts | 26 + .../arm64/boot/dts/freescale/fsl-ls1046a.dtsi | 111 ++ .../boot/dts/freescale/fsl-ls1088a-rdb.dts | 82 +- .../arm64/boot/dts/freescale/fsl-ls1088a.dtsi | 156 ++- drivers/clk/Makefile | 1 + drivers/clk/clk-fsl-lynx-10g.c | 510 +++++++ drivers/phy/freescale/Kconfig | 23 + drivers/phy/freescale/Makefile | 1 + drivers/phy/freescale/phy-fsl-lynx-10g.c | 1224 +++++++++++++++++ include/dt-bindings/clock/fsl,lynx-10g.h | 14 + include/dt-bindings/phy/phy.h | 2 + include/linux/phy/lynx-10g.h | 16 + 20 files changed, 2594 insertions(+), 103 deletions(-) rename Documentation/devicetree/bindings/gpio/{brcm,bcm6345-gpio.yaml => brcm,bcm63xx-gpio.yaml} (78%) create mode 100644 Documentation/devicetree/bindings/gpio/gpio-mmio.yaml delete mode 100644 Documentation/devicetree/bindings/gpio/ni,169445-nand-gpio.txt delete mode 100644 Documentation/devicetree/bindings/gpio/wd,mbl-gpio.txt create mode 100644 Documentation/devicetree/bindings/phy/fsl,lynx-10g.yaml create mode 100644 Documentation/driver-api/phy/lynx_10g.rst create mode 100644 drivers/clk/clk-fsl-lynx-10g.c create mode 100644 drivers/phy/freescale/phy-fsl-lynx-10g.c create mode 100644 include/dt-bindings/clock/fsl,lynx-10g.h create mode 100644 include/linux/phy/lynx-10g.h