| Message ID | 20231109140958.1093235-1-joao.goncalves@toradex.com |
|---|---|
| State | RFC |
| Delegated to: | Tom Rini |
| Headers | show |
| Series | [RFC,1/1] arm: mach-k3: Enable icache on am625 to boot faster | expand |
On 11:09-20231109, Joao Paulo Goncalves wrote: > Enable the am625 instruction cache on SPL and U-boot earlier for the A53 > to execute code a bit faster. For normal boot flow, it was possible to > gain about 2 seconds on boot time. > > Signed-off-by: Joao Paulo Goncalves <joao.goncalves@toradex.com> > --- > Hello all, > > We are trying to optimize boot time on our AM62 devices and on TI community > forum [1] someone advised to enable ICACHE for am625. We enabled it, tested and got > around 2 seconds faster boot on storage media and about 5 seconds with DFU > for downloading images from A53 SPL. However, we don't know if this is the correct > solution and want more comments on this or why the ICACHE was not enabled by TI in > the first place. > > [1] https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1217839/am625-boot-time-between-a53-spl-and-a53-u-boot > > Regards, > Joao Paulo Goncalves > > arch/arm/mach-k3/am625_init.c | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/arch/arm/mach-k3/am625_init.c b/arch/arm/mach-k3/am625_init.c > index 8fa36f7b91..d665d07b0b 100644 > --- a/arch/arm/mach-k3/am625_init.c > +++ b/arch/arm/mach-k3/am625_init.c > @@ -210,6 +210,9 @@ void board_init_f(ulong dummy) > panic("DRAM init failed: %d\n", ret); > } > spl_enable_dcache(); > + > + if (!IS_ENABLED(CONFIG_CPU_V7R) && !IS_ENABLED(CONFIG_SYS_ICACHE_OFF)) > + icache_enable(); > } > > u32 spl_mmc_boot_mode(struct mmc *mmc, const u32 boot_device) > -- > 2.34.1 Just noticed that icache_enable was missed for all of k3 as well - I suspect this is a miss rather than anything consciously done. btw icache_enable is already wrapped correctly under CONFIG_SYS_ICACHE_OFF in cache_v8.c
On 11/9/23 11:37 AM, Nishanth Menon wrote: > On 11:09-20231109, Joao Paulo Goncalves wrote: >> Enable the am625 instruction cache on SPL and U-boot earlier for the A53 >> to execute code a bit faster. For normal boot flow, it was possible to >> gain about 2 seconds on boot time. >> >> Signed-off-by: Joao Paulo Goncalves <joao.goncalves@toradex.com> >> --- >> Hello all, >> >> We are trying to optimize boot time on our AM62 devices and on TI community >> forum [1] someone advised to enable ICACHE for am625. We enabled it, tested and got >> around 2 seconds faster boot on storage media and about 5 seconds with DFU >> for downloading images from A53 SPL. However, we don't know if this is the correct >> solution and want more comments on this or why the ICACHE was not enabled by TI in >> the first place. >> >> [1] https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1217839/am625-boot-time-between-a53-spl-and-a53-u-boot >> >> Regards, >> Joao Paulo Goncalves >> >> arch/arm/mach-k3/am625_init.c | 3 +++ >> 1 file changed, 3 insertions(+) >> >> diff --git a/arch/arm/mach-k3/am625_init.c b/arch/arm/mach-k3/am625_init.c >> index 8fa36f7b91..d665d07b0b 100644 >> --- a/arch/arm/mach-k3/am625_init.c >> +++ b/arch/arm/mach-k3/am625_init.c >> @@ -210,6 +210,9 @@ void board_init_f(ulong dummy) >> panic("DRAM init failed: %d\n", ret); >> } >> spl_enable_dcache(); >> + >> + if (!IS_ENABLED(CONFIG_CPU_V7R) && !IS_ENABLED(CONFIG_SYS_ICACHE_OFF)) >> + icache_enable(); >> } >> >> u32 spl_mmc_boot_mode(struct mmc *mmc, const u32 boot_device) >> -- >> 2.34.1 > > Just noticed that icache_enable was missed for all of k3 as well - I > suspect this is a miss rather than anything consciously done. btw > icache_enable is already wrapped correctly under CONFIG_SYS_ICACHE_OFF in > cache_v8.c > For U-Boot proper, both I/D caches are enabled early in board_init_r(). Oddly the SPL version of this function does not do that. Some boards seem to work around this by adding it manually, others must have missed this. Maybe due to armv7/start.S doing this for us already, but not armv8/start.S.. We enable Icache in SPL for our CPU_V7R case inside setup_mpu_regions()(??) We also enable Dcache in spl_enable_dcache(), maybe this should be renamed spl_enable_cache() and it call enable_caches() which enables both. Just be careful to invalidate the icache when enabling after relocation. This would be temporary, better solution would be to add it to the SPL common board_init_r() to match the U-Boot one, that way all these platforms wouldn't need to do it in their current inconsistent ways. Andrew
On Thu, Nov 09, 2023 at 12:44:56PM -0600, Andrew Davis wrote: > On 11/9/23 11:37 AM, Nishanth Menon wrote: > > On 11:09-20231109, Joao Paulo Goncalves wrote: > > > Enable the am625 instruction cache on SPL and U-boot earlier for the A53 > > > to execute code a bit faster. For normal boot flow, it was possible to > > > gain about 2 seconds on boot time. > > > > > > Signed-off-by: Joao Paulo Goncalves <joao.goncalves@toradex.com> > > > --- > > > Hello all, > > > > > > We are trying to optimize boot time on our AM62 devices and on TI community > > > forum [1] someone advised to enable ICACHE for am625. We enabled it, tested and got > > > around 2 seconds faster boot on storage media and about 5 seconds with DFU > > > for downloading images from A53 SPL. However, we don't know if this is the correct > > > solution and want more comments on this or why the ICACHE was not enabled by TI in > > > the first place. > > > > > > [1] https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1217839/am625-boot-time-between-a53-spl-and-a53-u-boot > > > > > > Regards, > > > Joao Paulo Goncalves > > > > > > arch/arm/mach-k3/am625_init.c | 3 +++ > > > 1 file changed, 3 insertions(+) > > > > > > diff --git a/arch/arm/mach-k3/am625_init.c b/arch/arm/mach-k3/am625_init.c > > > index 8fa36f7b91..d665d07b0b 100644 > > > --- a/arch/arm/mach-k3/am625_init.c > > > +++ b/arch/arm/mach-k3/am625_init.c > > > @@ -210,6 +210,9 @@ void board_init_f(ulong dummy) > > > panic("DRAM init failed: %d\n", ret); > > > } > > > spl_enable_dcache(); > > > + > > > + if (!IS_ENABLED(CONFIG_CPU_V7R) && !IS_ENABLED(CONFIG_SYS_ICACHE_OFF)) > > > + icache_enable(); > > > } > > > u32 spl_mmc_boot_mode(struct mmc *mmc, const u32 boot_device) > > > -- > > > 2.34.1 > > > > Just noticed that icache_enable was missed for all of k3 as well - I > > suspect this is a miss rather than anything consciously done. btw > > icache_enable is already wrapped correctly under CONFIG_SYS_ICACHE_OFF in > > cache_v8.c > > > > For U-Boot proper, both I/D caches are enabled early in board_init_r(). > Oddly the SPL version of this function does not do that. Some boards seem > to work around this by adding it manually, others must have missed this. > Maybe due to armv7/start.S doing this for us already, but not armv8/start.S.. > > We enable Icache in SPL for our CPU_V7R case inside setup_mpu_regions()(??) > We also enable Dcache in spl_enable_dcache(), maybe this should be > renamed spl_enable_cache() and it call enable_caches() which enables > both. Just be careful to invalidate the icache when enabling after > relocation. > > This would be temporary, better solution would be to add it to the > SPL common board_init_r() to match the U-Boot one, that way all these > platforms wouldn't need to do it in their current inconsistent ways. I will say that part of the reason we don't have a more high-level "enable all the caches in SPL on ARMv7" is that I never could figure out why on earth it locked up my am335x GP EVM, I believe. So yes, I'd be quite happy to see the mach-k3 case cleaned up and made generic to enable (and then disable/flush/enable in full). But we can't easily make it higher-level than that.
On Thu, Nov 09, 2023 at 01:51:19PM -0500, Tom Rini wrote: > On Thu, Nov 09, 2023 at 12:44:56PM -0600, Andrew Davis wrote: > > On 11/9/23 11:37 AM, Nishanth Menon wrote: > > > On 11:09-20231109, Joao Paulo Goncalves wrote: > > > > Enable the am625 instruction cache on SPL and U-boot earlier for the A53 > > > > to execute code a bit faster. For normal boot flow, it was possible to > > > > gain about 2 seconds on boot time. > > > > > > > > Signed-off-by: Joao Paulo Goncalves <joao.goncalves@toradex.com> > > > > --- > > > > Hello all, > > > > > > > > We are trying to optimize boot time on our AM62 devices and on TI community > > > > forum [1] someone advised to enable ICACHE for am625. We enabled it, tested and got > > > > around 2 seconds faster boot on storage media and about 5 seconds with DFU > > > > for downloading images from A53 SPL. However, we don't know if this is the correct > > > > solution and want more comments on this or why the ICACHE was not enabled by TI in > > > > the first place. > > > > > > > > [1] https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1217839/am625-boot-time-between-a53-spl-and-a53-u-boot > > > > > > > > Regards, > > > > Joao Paulo Goncalves > > > > > > > > arch/arm/mach-k3/am625_init.c | 3 +++ > > > > 1 file changed, 3 insertions(+) > > > > > > > > diff --git a/arch/arm/mach-k3/am625_init.c b/arch/arm/mach-k3/am625_init.c > > > > index 8fa36f7b91..d665d07b0b 100644 > > > > --- a/arch/arm/mach-k3/am625_init.c > > > > +++ b/arch/arm/mach-k3/am625_init.c > > > > @@ -210,6 +210,9 @@ void board_init_f(ulong dummy) > > > > panic("DRAM init failed: %d\n", ret); > > > > } > > > > spl_enable_dcache(); > > > > + > > > > + if (!IS_ENABLED(CONFIG_CPU_V7R) && !IS_ENABLED(CONFIG_SYS_ICACHE_OFF)) > > > > + icache_enable(); > > > > } > > > > u32 spl_mmc_boot_mode(struct mmc *mmc, const u32 boot_device) > > > > -- > > > > 2.34.1 > > > > > > Just noticed that icache_enable was missed for all of k3 as well - I > > > suspect this is a miss rather than anything consciously done. btw ... > > We enable Icache in SPL for our CPU_V7R case inside setup_mpu_regions()(??) > > We also enable Dcache in spl_enable_dcache(), maybe this should be > > renamed spl_enable_cache() and it call enable_caches() which enables > > both. Just be careful to invalidate the icache when enabling after > > relocation. > > > > This would be temporary, better solution would be to add it to the > > SPL common board_init_r() to match the U-Boot one, that way all these > > platforms wouldn't need to do it in their current inconsistent ways. > > I will say that part of the reason we don't have a more high-level > "enable all the caches in SPL on ARMv7" is that I never could figure out > why on earth it locked up my am335x GP EVM, I believe. So yes, I'd be > quite happy to see the mach-k3 case cleaned up and made generic to > enable (and then disable/flush/enable in full). But we can't easily make > it higher-level than that. So the plan would be to enable the cache for SPL in mach-k3 and not just for AM625, correct? Andrew: is this something you could do or should we send a patch ourselves? We can test only TI AM62. Francesco
On 09:59-20231110, Francesco Dolcini wrote: > On Thu, Nov 09, 2023 at 01:51:19PM -0500, Tom Rini wrote: > > On Thu, Nov 09, 2023 at 12:44:56PM -0600, Andrew Davis wrote: > > > On 11/9/23 11:37 AM, Nishanth Menon wrote: > > > > On 11:09-20231109, Joao Paulo Goncalves wrote: > > > > > Enable the am625 instruction cache on SPL and U-boot earlier for the A53 > > > > > to execute code a bit faster. For normal boot flow, it was possible to > > > > > gain about 2 seconds on boot time. > > > > > > > > > > Signed-off-by: Joao Paulo Goncalves <joao.goncalves@toradex.com> > > > > > --- > > > > > Hello all, > > > > > > > > > > We are trying to optimize boot time on our AM62 devices and on TI community > > > > > forum [1] someone advised to enable ICACHE for am625. We enabled it, tested and got > > > > > around 2 seconds faster boot on storage media and about 5 seconds with DFU > > > > > for downloading images from A53 SPL. However, we don't know if this is the correct > > > > > solution and want more comments on this or why the ICACHE was not enabled by TI in > > > > > the first place. > > > > > > > > > > [1] https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1217839/am625-boot-time-between-a53-spl-and-a53-u-boot > > > > > > > > > > Regards, > > > > > Joao Paulo Goncalves > > > > > > > > > > arch/arm/mach-k3/am625_init.c | 3 +++ > > > > > 1 file changed, 3 insertions(+) > > > > > > > > > > diff --git a/arch/arm/mach-k3/am625_init.c b/arch/arm/mach-k3/am625_init.c > > > > > index 8fa36f7b91..d665d07b0b 100644 > > > > > --- a/arch/arm/mach-k3/am625_init.c > > > > > +++ b/arch/arm/mach-k3/am625_init.c > > > > > @@ -210,6 +210,9 @@ void board_init_f(ulong dummy) > > > > > panic("DRAM init failed: %d\n", ret); > > > > > } > > > > > spl_enable_dcache(); > > > > > + > > > > > + if (!IS_ENABLED(CONFIG_CPU_V7R) && !IS_ENABLED(CONFIG_SYS_ICACHE_OFF)) > > > > > + icache_enable(); > > > > > } > > > > > u32 spl_mmc_boot_mode(struct mmc *mmc, const u32 boot_device) > > > > > -- > > > > > 2.34.1 > > > > > > > > Just noticed that icache_enable was missed for all of k3 as well - I > > > > suspect this is a miss rather than anything consciously done. btw > ... > > > We enable Icache in SPL for our CPU_V7R case inside setup_mpu_regions()(??) > > > We also enable Dcache in spl_enable_dcache(), maybe this should be > > > renamed spl_enable_cache() and it call enable_caches() which enables > > > both. Just be careful to invalidate the icache when enabling after > > > relocation. > > > > > > This would be temporary, better solution would be to add it to the > > > SPL common board_init_r() to match the U-Boot one, that way all these > > > platforms wouldn't need to do it in their current inconsistent ways. > > > > I will say that part of the reason we don't have a more high-level > > "enable all the caches in SPL on ARMv7" is that I never could figure out > > why on earth it locked up my am335x GP EVM, I believe. So yes, I'd be > > quite happy to see the mach-k3 case cleaned up and made generic to > > enable (and then disable/flush/enable in full). But we can't easily make > > it higher-level than that. > > So the plan would be to enable the cache for SPL in mach-k3 and not just > for AM625, correct? Yes. > > Andrew: is this something you could do or should we send a patch > ourselves? We can test only TI AM62. > That would be much appreciated, we can help test the rest of TI K3 platforms.
diff --git a/arch/arm/mach-k3/am625_init.c b/arch/arm/mach-k3/am625_init.c index 8fa36f7b91..d665d07b0b 100644 --- a/arch/arm/mach-k3/am625_init.c +++ b/arch/arm/mach-k3/am625_init.c @@ -210,6 +210,9 @@ void board_init_f(ulong dummy) panic("DRAM init failed: %d\n", ret); } spl_enable_dcache(); + + if (!IS_ENABLED(CONFIG_CPU_V7R) && !IS_ENABLED(CONFIG_SYS_ICACHE_OFF)) + icache_enable(); } u32 spl_mmc_boot_mode(struct mmc *mmc, const u32 boot_device)
Enable the am625 instruction cache on SPL and U-boot earlier for the A53 to execute code a bit faster. For normal boot flow, it was possible to gain about 2 seconds on boot time. Signed-off-by: Joao Paulo Goncalves <joao.goncalves@toradex.com> --- Hello all, We are trying to optimize boot time on our AM62 devices and on TI community forum [1] someone advised to enable ICACHE for am625. We enabled it, tested and got around 2 seconds faster boot on storage media and about 5 seconds with DFU for downloading images from A53 SPL. However, we don't know if this is the correct solution and want more comments on this or why the ICACHE was not enabled by TI in the first place. [1] https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1217839/am625-boot-time-between-a53-spl-and-a53-u-boot Regards, Joao Paulo Goncalves arch/arm/mach-k3/am625_init.c | 3 +++ 1 file changed, 3 insertions(+)