[u-boot-marvell] ddr: marvell: a38x: Fix Synchronous vs Asynchronous mode determination

From: Marek Behún <marek.behun@nic.cz>

Before commit 4c289425752f ("mv_ddr: a38x: add support for ddr async
mode"), Asynchornous Mode was only used when the CPU Subsystem Clock
Options[4:0] field in the SAR1 register was set to value 0x13: CPU at
2 GHz and DDR at 933 MHz.

Then commit 4c289425752f ("mv_ddr: a38x: add support for ddr async
mode") added support for Asynchornous Modes with frequencies other than
933 MHz (but at least 467 MHz), but the code it added to check for
whether Asynchornous Mode should be used is wrong: it checks whether the
frequency setting in board DDR topology map is set to value other than
MV_DDR_FREQ_SAR.

Thus boards which define a specific value, greater than 400 MHz, for DDR
frequency in their board topology (e.g. Turris Omnia defines
MV_DDR_FREQ_800), are incorrectly put into Asynchornous Mode after that
commit.

The A38x Functional Specification, section 10.12 DRAM Clocking, says:
  In Synchornous mode, the DRAM and CPU clocks are edge aligned and run
  in 1:2 or 1:3 CPU to DRAM frequency ratios.

Change the check for whether Asynchornous Mode should be used according
to this explanation in Functional Specification.

Signed-off-by: Marek Behún <marek.behun@nic.cz>
---
A PR was also created for mv-ddr-marvell:
  https://github.com/MarvellEmbeddedProcessors/mv-ddr-marvell/pull/35

Please test this. It is possible this commit will fix DDR training
issues, since commit 4c289425752f in mv-ddr-marvell started using
Asynchronous Mode where Synchronous Mode was used previously.
---
 drivers/ddr/marvell/a38x/mv_ddr_plat.c | 19 ++++++++-----------
 1 file changed, 8 insertions(+), 11 deletions(-)

On Tue,  4 Jan 2022 15:57:49 +0100
Marek Behún <kabel@kernel.org> wrote:

> From: Marek Behún <marek.behun@nic.cz>
> 
> Before commit 4c289425752f ("mv_ddr: a38x: add support for ddr async
> mode"), Asynchornous Mode was only used when the CPU Subsystem Clock
> Options[4:0] field in the SAR1 register was set to value 0x13: CPU at
> 2 GHz and DDR at 933 MHz.
> 
> Then commit 4c289425752f ("mv_ddr: a38x: add support for ddr async
> mode") added support for Asynchornous Modes with frequencies other than
> 933 MHz (but at least 467 MHz), but the code it added to check for
> whether Asynchornous Mode should be used is wrong: it checks whether the
> frequency setting in board DDR topology map is set to value other than
> MV_DDR_FREQ_SAR.
> 
> Thus boards which define a specific value, greater than 400 MHz, for DDR
> frequency in their board topology (e.g. Turris Omnia defines
> MV_DDR_FREQ_800), are incorrectly put into Asynchornous Mode after that
> commit.
> 
> The A38x Functional Specification, section 10.12 DRAM Clocking, says:
>   In Synchornous mode, the DRAM and CPU clocks are edge aligned and run
>   in 1:2 or 1:3 CPU to DRAM frequency ratios.
> 
> Change the check for whether Asynchornous Mode should be used according
> to this explanation in Functional Specification.
> 
> Signed-off-by: Marek Behún <marek.behun@nic.cz>

I forgot to mention that we discovered this on Turris Omnia by comparing
DDR speed with the
  time mtest 10000000 10100000 0 1
command.

In Asynchornous Mode this takes ~27 seconds, in Synchronous mode ~22
seconds.

Marek

On Wed, Jan 5, 2022 at 3:57 AM Marek Behún <kabel@kernel.org> wrote:
>
> From: Marek Behún <marek.behun@nic.cz>
>
> Before commit 4c289425752f ("mv_ddr: a38x: add support for ddr async
> mode"), Asynchornous Mode was only used when the CPU Subsystem Clock
> Options[4:0] field in the SAR1 register was set to value 0x13: CPU at
> 2 GHz and DDR at 933 MHz.
>
> Then commit 4c289425752f ("mv_ddr: a38x: add support for ddr async
> mode") added support for Asynchornous Modes with frequencies other than
> 933 MHz (but at least 467 MHz), but the code it added to check for
> whether Asynchornous Mode should be used is wrong: it checks whether the
> frequency setting in board DDR topology map is set to value other than
> MV_DDR_FREQ_SAR.
>
> Thus boards which define a specific value, greater than 400 MHz, for DDR
> frequency in their board topology (e.g. Turris Omnia defines
> MV_DDR_FREQ_800), are incorrectly put into Asynchornous Mode after that
> commit.
>
> The A38x Functional Specification, section 10.12 DRAM Clocking, says:
>   In Synchornous mode, the DRAM and CPU clocks are edge aligned and run
>   in 1:2 or 1:3 CPU to DRAM frequency ratios.
>
> Change the check for whether Asynchornous Mode should be used according
> to this explanation in Functional Specification.
>
> Signed-off-by: Marek Behún <marek.behun@nic.cz>

Tested-by: Chris Packham <judge.packham@gmail.com>

> ---
> A PR was also created for mv-ddr-marvell:
>   https://github.com/MarvellEmbeddedProcessors/mv-ddr-marvell/pull/35
>
> Please test this. It is possible this commit will fix DDR training
> issues, since commit 4c289425752f in mv-ddr-marvell started using
> Asynchronous Mode where Synchronous Mode was used previously.
> ---
>  drivers/ddr/marvell/a38x/mv_ddr_plat.c | 19 ++++++++-----------
>  1 file changed, 8 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/ddr/marvell/a38x/mv_ddr_plat.c b/drivers/ddr/marvell/a38x/mv_ddr_plat.c
> index faafc86ea2..7c7bce73a3 100644
> --- a/drivers/ddr/marvell/a38x/mv_ddr_plat.c
> +++ b/drivers/ddr/marvell/a38x/mv_ddr_plat.c
> @@ -167,8 +167,6 @@ static u16 a38x_vco_freq_per_sar_ref_clk_40_mhz[] = {
>  };
>
>
> -static u32 async_mode_at_tf;
> -
>  static u32 dq_bit_map_2_phy_pin[] = {
>         1, 0, 2, 6, 9, 8, 3, 7, /* 0 */
>         8, 9, 1, 7, 2, 6, 3, 0, /* 1 */
> @@ -734,7 +732,8 @@ static int ddr3_tip_a38x_set_divider(u8 dev_num, u32 if_id,
>         u32 divider = 0;
>         u32 sar_val, ref_clk_satr;
>         u32 async_val;
> -       u32 freq = mv_ddr_freq_get(frequency);
> +       u32 cpu_freq;
> +       u32 ddr_freq = mv_ddr_freq_get(frequency);
>
>         if (if_id != 0) {
>                 DEBUG_TRAINING_ACCESS(DEBUG_LEVEL_ERROR,
> @@ -751,11 +750,14 @@ static int ddr3_tip_a38x_set_divider(u8 dev_num, u32 if_id,
>         ref_clk_satr = reg_read(DEVICE_SAMPLE_AT_RESET2_REG);
>         if (((ref_clk_satr >> DEVICE_SAMPLE_AT_RESET2_REG_REFCLK_OFFSET) & 0x1) ==
>             DEVICE_SAMPLE_AT_RESET2_REG_REFCLK_25MHZ)
> -               divider = a38x_vco_freq_per_sar_ref_clk_25_mhz[sar_val] / freq;
> +               cpu_freq = a38x_vco_freq_per_sar_ref_clk_25_mhz[sar_val];
>         else
> -               divider = a38x_vco_freq_per_sar_ref_clk_40_mhz[sar_val] / freq;
> +               cpu_freq = a38x_vco_freq_per_sar_ref_clk_40_mhz[sar_val];
> +
> +       divider = cpu_freq / ddr_freq;
>
> -       if ((async_mode_at_tf == 1) && (freq > 400)) {
> +       if (((cpu_freq % ddr_freq != 0) || (divider != 2 && divider != 3)) &&
> +           (ddr_freq > 400)) {
>                 /* Set async mode */
>                 dunit_write(0x20220, 0x1000, 0x1000);
>                 dunit_write(0xe42f4, 0x200, 0x200);
> @@ -869,8 +871,6 @@ int ddr3_tip_ext_write(u32 dev_num, u32 if_id, u32 reg_addr,
>
>  int mv_ddr_early_init(void)
>  {
> -       struct mv_ddr_topology_map *tm = mv_ddr_topology_map_get();
> -
>         /* FIXME: change this configuration per ddr type
>          * configure a380 and a390 to work with receiver odt timing
>          * the odt_config is defined:
> @@ -882,9 +882,6 @@ int mv_ddr_early_init(void)
>
>         mv_ddr_sw_db_init(0, 0);
>
> -       if (tm->interface_params[0].memory_freq != MV_DDR_FREQ_SAR)
> -               async_mode_at_tf = 1;
> -
>         return MV_OK;
>  }
>
> --
> 2.34.1
>

On 1/4/22 15:57, Marek Behún wrote:
> From: Marek Behún <marek.behun@nic.cz>
> 
> Before commit 4c289425752f ("mv_ddr: a38x: add support for ddr async
> mode"), Asynchornous Mode was only used when the CPU Subsystem Clock
> Options[4:0] field in the SAR1 register was set to value 0x13: CPU at
> 2 GHz and DDR at 933 MHz.
> 
> Then commit 4c289425752f ("mv_ddr: a38x: add support for ddr async
> mode") added support for Asynchornous Modes with frequencies other than
> 933 MHz (but at least 467 MHz), but the code it added to check for
> whether Asynchornous Mode should be used is wrong: it checks whether the
> frequency setting in board DDR topology map is set to value other than
> MV_DDR_FREQ_SAR.
> 
> Thus boards which define a specific value, greater than 400 MHz, for DDR
> frequency in their board topology (e.g. Turris Omnia defines
> MV_DDR_FREQ_800), are incorrectly put into Asynchornous Mode after that
> commit.
> 
> The A38x Functional Specification, section 10.12 DRAM Clocking, says:
>    In Synchornous mode, the DRAM and CPU clocks are edge aligned and run
>    in 1:2 or 1:3 CPU to DRAM frequency ratios.
> 
> Change the check for whether Asynchornous Mode should be used according
> to this explanation in Functional Specification.
> 
> Signed-off-by: Marek Behún <marek.behun@nic.cz>

Reviewed-by: Stefan Roese <sr@denx.de>

Thanks,
Stefan

> ---
> A PR was also created for mv-ddr-marvell:
>    https://github.com/MarvellEmbeddedProcessors/mv-ddr-marvell/pull/35
> 
> Please test this. It is possible this commit will fix DDR training
> issues, since commit 4c289425752f in mv-ddr-marvell started using
> Asynchronous Mode where Synchronous Mode was used previously.
> ---
>   drivers/ddr/marvell/a38x/mv_ddr_plat.c | 19 ++++++++-----------
>   1 file changed, 8 insertions(+), 11 deletions(-)
> 
> diff --git a/drivers/ddr/marvell/a38x/mv_ddr_plat.c b/drivers/ddr/marvell/a38x/mv_ddr_plat.c
> index faafc86ea2..7c7bce73a3 100644
> --- a/drivers/ddr/marvell/a38x/mv_ddr_plat.c
> +++ b/drivers/ddr/marvell/a38x/mv_ddr_plat.c
> @@ -167,8 +167,6 @@ static u16 a38x_vco_freq_per_sar_ref_clk_40_mhz[] = {
>   };
>   
>   
> -static u32 async_mode_at_tf;
> -
>   static u32 dq_bit_map_2_phy_pin[] = {
>   	1, 0, 2, 6, 9, 8, 3, 7,	/* 0 */
>   	8, 9, 1, 7, 2, 6, 3, 0,	/* 1 */
> @@ -734,7 +732,8 @@ static int ddr3_tip_a38x_set_divider(u8 dev_num, u32 if_id,
>   	u32 divider = 0;
>   	u32 sar_val, ref_clk_satr;
>   	u32 async_val;
> -	u32 freq = mv_ddr_freq_get(frequency);
> +	u32 cpu_freq;
> +	u32 ddr_freq = mv_ddr_freq_get(frequency);
>   
>   	if (if_id != 0) {
>   		DEBUG_TRAINING_ACCESS(DEBUG_LEVEL_ERROR,
> @@ -751,11 +750,14 @@ static int ddr3_tip_a38x_set_divider(u8 dev_num, u32 if_id,
>   	ref_clk_satr = reg_read(DEVICE_SAMPLE_AT_RESET2_REG);
>   	if (((ref_clk_satr >> DEVICE_SAMPLE_AT_RESET2_REG_REFCLK_OFFSET) & 0x1) ==
>   	    DEVICE_SAMPLE_AT_RESET2_REG_REFCLK_25MHZ)
> -		divider = a38x_vco_freq_per_sar_ref_clk_25_mhz[sar_val] / freq;
> +		cpu_freq = a38x_vco_freq_per_sar_ref_clk_25_mhz[sar_val];
>   	else
> -		divider = a38x_vco_freq_per_sar_ref_clk_40_mhz[sar_val] / freq;
> +		cpu_freq = a38x_vco_freq_per_sar_ref_clk_40_mhz[sar_val];
> +
> +	divider = cpu_freq / ddr_freq;
>   
> -	if ((async_mode_at_tf == 1) && (freq > 400)) {
> +	if (((cpu_freq % ddr_freq != 0) || (divider != 2 && divider != 3)) &&
> +	    (ddr_freq > 400)) {
>   		/* Set async mode */
>   		dunit_write(0x20220, 0x1000, 0x1000);
>   		dunit_write(0xe42f4, 0x200, 0x200);
> @@ -869,8 +871,6 @@ int ddr3_tip_ext_write(u32 dev_num, u32 if_id, u32 reg_addr,
>   
>   int mv_ddr_early_init(void)
>   {
> -	struct mv_ddr_topology_map *tm = mv_ddr_topology_map_get();
> -
>   	/* FIXME: change this configuration per ddr type
>   	 * configure a380 and a390 to work with receiver odt timing
>   	 * the odt_config is defined:
> @@ -882,9 +882,6 @@ int mv_ddr_early_init(void)
>   
>   	mv_ddr_sw_db_init(0, 0);
>   
> -	if (tm->interface_params[0].memory_freq != MV_DDR_FREQ_SAR)
> -		async_mode_at_tf = 1;
> -
>   	return MV_OK;
>   }
>   
> 

Viele Grüße,
Stefan Roese

On 1/4/22 15:57, Marek Behún wrote:
> From: Marek Behún <marek.behun@nic.cz>
> 
> Before commit 4c289425752f ("mv_ddr: a38x: add support for ddr async
> mode"), Asynchornous Mode was only used when the CPU Subsystem Clock
> Options[4:0] field in the SAR1 register was set to value 0x13: CPU at
> 2 GHz and DDR at 933 MHz.
> 
> Then commit 4c289425752f ("mv_ddr: a38x: add support for ddr async
> mode") added support for Asynchornous Modes with frequencies other than
> 933 MHz (but at least 467 MHz), but the code it added to check for
> whether Asynchornous Mode should be used is wrong: it checks whether the
> frequency setting in board DDR topology map is set to value other than
> MV_DDR_FREQ_SAR.
> 
> Thus boards which define a specific value, greater than 400 MHz, for DDR
> frequency in their board topology (e.g. Turris Omnia defines
> MV_DDR_FREQ_800), are incorrectly put into Asynchornous Mode after that
> commit.
> 
> The A38x Functional Specification, section 10.12 DRAM Clocking, says:
>    In Synchornous mode, the DRAM and CPU clocks are edge aligned and run
>    in 1:2 or 1:3 CPU to DRAM frequency ratios.
> 
> Change the check for whether Asynchornous Mode should be used according
> to this explanation in Functional Specification.
> 
> Signed-off-by: Marek Behún <marek.behun@nic.cz>

Applied to u-boot-marvell/master

Thanks,
Stefan

> ---
> A PR was also created for mv-ddr-marvell:
>    https://github.com/MarvellEmbeddedProcessors/mv-ddr-marvell/pull/35
> 
> Please test this. It is possible this commit will fix DDR training
> issues, since commit 4c289425752f in mv-ddr-marvell started using
> Asynchronous Mode where Synchronous Mode was used previously.
> ---
>   drivers/ddr/marvell/a38x/mv_ddr_plat.c | 19 ++++++++-----------
>   1 file changed, 8 insertions(+), 11 deletions(-)
> 
> diff --git a/drivers/ddr/marvell/a38x/mv_ddr_plat.c b/drivers/ddr/marvell/a38x/mv_ddr_plat.c
> index faafc86ea2..7c7bce73a3 100644
> --- a/drivers/ddr/marvell/a38x/mv_ddr_plat.c
> +++ b/drivers/ddr/marvell/a38x/mv_ddr_plat.c
> @@ -167,8 +167,6 @@ static u16 a38x_vco_freq_per_sar_ref_clk_40_mhz[] = {
>   };
>   
>   
> -static u32 async_mode_at_tf;
> -
>   static u32 dq_bit_map_2_phy_pin[] = {
>   	1, 0, 2, 6, 9, 8, 3, 7,	/* 0 */
>   	8, 9, 1, 7, 2, 6, 3, 0,	/* 1 */
> @@ -734,7 +732,8 @@ static int ddr3_tip_a38x_set_divider(u8 dev_num, u32 if_id,
>   	u32 divider = 0;
>   	u32 sar_val, ref_clk_satr;
>   	u32 async_val;
> -	u32 freq = mv_ddr_freq_get(frequency);
> +	u32 cpu_freq;
> +	u32 ddr_freq = mv_ddr_freq_get(frequency);
>   
>   	if (if_id != 0) {
>   		DEBUG_TRAINING_ACCESS(DEBUG_LEVEL_ERROR,
> @@ -751,11 +750,14 @@ static int ddr3_tip_a38x_set_divider(u8 dev_num, u32 if_id,
>   	ref_clk_satr = reg_read(DEVICE_SAMPLE_AT_RESET2_REG);
>   	if (((ref_clk_satr >> DEVICE_SAMPLE_AT_RESET2_REG_REFCLK_OFFSET) & 0x1) ==
>   	    DEVICE_SAMPLE_AT_RESET2_REG_REFCLK_25MHZ)
> -		divider = a38x_vco_freq_per_sar_ref_clk_25_mhz[sar_val] / freq;
> +		cpu_freq = a38x_vco_freq_per_sar_ref_clk_25_mhz[sar_val];
>   	else
> -		divider = a38x_vco_freq_per_sar_ref_clk_40_mhz[sar_val] / freq;
> +		cpu_freq = a38x_vco_freq_per_sar_ref_clk_40_mhz[sar_val];
> +
> +	divider = cpu_freq / ddr_freq;
>   
> -	if ((async_mode_at_tf == 1) && (freq > 400)) {
> +	if (((cpu_freq % ddr_freq != 0) || (divider != 2 && divider != 3)) &&
> +	    (ddr_freq > 400)) {
>   		/* Set async mode */
>   		dunit_write(0x20220, 0x1000, 0x1000);
>   		dunit_write(0xe42f4, 0x200, 0x200);
> @@ -869,8 +871,6 @@ int ddr3_tip_ext_write(u32 dev_num, u32 if_id, u32 reg_addr,
>   
>   int mv_ddr_early_init(void)
>   {
> -	struct mv_ddr_topology_map *tm = mv_ddr_topology_map_get();
> -
>   	/* FIXME: change this configuration per ddr type
>   	 * configure a380 and a390 to work with receiver odt timing
>   	 * the odt_config is defined:
> @@ -882,9 +882,6 @@ int mv_ddr_early_init(void)
>   
>   	mv_ddr_sw_db_init(0, 0);
>   
> -	if (tm->interface_params[0].memory_freq != MV_DDR_FREQ_SAR)
> -		async_mode_at_tf = 1;
> -
>   	return MV_OK;
>   }
>   
> 

Viele Grüße,
Stefan Roese