Message ID | 1686527657-366116-1-git-send-email-ganboing@gmail.com |
---|---|
State | Accepted |
Commit | 28ff3f16c432f53670d4f3867f003e0a86a5e1e6 |
Delegated to: | Andes |
Headers | show |
Series | [RESEND,v2] riscv: setup per-hart stack earlier | expand |
> From: Bo Gan <ganboing@gmail.com> > Sent: Monday, June 12, 2023 7:54 AM > To: u-boot@lists.denx.de > Cc: Bo Gan <ganboing@gmail.com>; Rick Jian-Zhi Chen(陳建志) <rick@andestech.com>; Leo Yu-Chi Liang(梁育齊) <ycliang@andestech.com>; Sean Anderson <seanga2@gmail.com>; Bin Meng <bmeng.cn@gmail.com>; Lukas Auer <lukas.auer@aisec.fraunhofer.de> > Subject: [RESEND PATCH v2] riscv: setup per-hart stack earlier > > Harts need to use per-hart stack before any function call, even if that function is a simple one. When the callee uses stack for register save/ restore, especially RA, if nested call, concurrent access by multiple harts on the same stack will cause data-race. > > This patch sets up SP before `board_init_f_alloc_reserve`. A side effect of this is that the memory layout has changed as the following: > > +----------------+ +----------------+ <----- SPL_STACK/ > | ...... | | hart 0 stack | SYS_INIT_SP_ADDR > | malloc_base | +----------------+ > +----------------+ | hart 1 stack | > | GD | +----------------+ If not SMP, N=1 > +----------------+ | ...... | > | hart 0 stack | +----------------+ > +----------------+ ==> | hart N-1 stack| > | hart 1 stack | +----------------+ > +----------------+ | ...... | > | ...... | | malloc_base | > +----------------+ +----------------+ > | hart N-1 stack| | GD | > +----------------+ +----------------+ > | | | | > > Signed-off-by: Bo Gan <ganboing@gmail.com> > Cc: Rick Chen <rick@andestech.com> > Cc: Leo <ycliang@andestech.com> > Cc: Sean Anderson <seanga2@gmail.com> > Cc: Bin Meng <bmeng.cn@gmail.com> > Cc: Lukas Auer <lukas.auer@aisec.fraunhofer.de> > --- > > v2: > - Fixed macro CONFIG_SYS_INIT_SP_ADDR -> SYS_INIT_SP_ADDR > - Tested SPL with VisionFive 2 board > --- > arch/riscv/cpu/start.S | 37 ++++++++++++++++++++++++------------- > 1 file changed, 24 insertions(+), 13 deletions(-) Reviewed-by: Rick Chen <rick@andestech.com>
On Sun, Jun 11, 2023 at 04:54:17PM -0700, Bo Gan wrote: > Harts need to use per-hart stack before any function call, even if that > function is a simple one. When the callee uses stack for register save/ > restore, especially RA, if nested call, concurrent access by multiple > harts on the same stack will cause data-race. > > This patch sets up SP before `board_init_f_alloc_reserve`. A side effect > of this is that the memory layout has changed as the following: > > +----------------+ +----------------+ <----- SPL_STACK/ > | ...... | | hart 0 stack | SYS_INIT_SP_ADDR > | malloc_base | +----------------+ > +----------------+ | hart 1 stack | > | GD | +----------------+ If not SMP, N=1 > +----------------+ | ...... | > | hart 0 stack | +----------------+ > +----------------+ ==> | hart N-1 stack| > | hart 1 stack | +----------------+ > +----------------+ | ...... | > | ...... | | malloc_base | > +----------------+ +----------------+ > | hart N-1 stack| | GD | > +----------------+ +----------------+ > | | | | > > Signed-off-by: Bo Gan <ganboing@gmail.com> > Cc: Rick Chen <rick@andestech.com> > Cc: Leo <ycliang@andestech.com> > Cc: Sean Anderson <seanga2@gmail.com> > Cc: Bin Meng <bmeng.cn@gmail.com> > Cc: Lukas Auer <lukas.auer@aisec.fraunhofer.de> > Reviewed-by: Rick Chen <rick@andestech.com> > --- > > v2: > - Fixed macro CONFIG_SYS_INIT_SP_ADDR -> SYS_INIT_SP_ADDR > - Tested SPL with VisionFive 2 board > --- > arch/riscv/cpu/start.S | 37 ++++++++++++++++++++++++------------- > 1 file changed, 24 insertions(+), 13 deletions(-) Reviewed-by: Leo Yu-Chi Liang <ycliang@andestech.com>
diff --git a/arch/riscv/cpu/start.S b/arch/riscv/cpu/start.S index dad22bf..59d58a5 100644 --- a/arch/riscv/cpu/start.S +++ b/arch/riscv/cpu/start.S @@ -91,16 +91,35 @@ _start: * Set stackpointer in internal/ex RAM to call board_init_f */ call_board_init_f: - li t0, -16 #if defined(CONFIG_SPL_BUILD) && defined(CONFIG_SPL_STACK) - li t1, CONFIG_SPL_STACK + li t0, CONFIG_SPL_STACK #else - li t1, SYS_INIT_SP_ADDR + li t0, SYS_INIT_SP_ADDR #endif - and sp, t1, t0 /* force 16 byte alignment */ + and t0, t0, -16 /* force 16 byte alignment */ + + /* setup stack */ +#if CONFIG_IS_ENABLED(SMP) + /* tp: hart id */ + slli t1, tp, CONFIG_STACK_SIZE_SHIFT + sub sp, t0, t1 +#else + mv sp, t0 +#endif +/* + * Now sp points to the right stack belonging to current CPU. + * It's essential before any function call, otherwise, we get data-race. + */ call_board_init_f_0: - mv a0, sp + /* find top of reserve space */ +#if CONFIG_IS_ENABLED(SMP) + li t1, CONFIG_NR_CPUS +#else + li t1, 1 +#endif + slli t1, t1, CONFIG_STACK_SIZE_SHIFT + sub a0, t0, t1 /* t1 -> size of all CPU stacks */ jal board_init_f_alloc_reserve /* @@ -109,14 +128,6 @@ call_board_init_f_0: */ mv s0, a0 - /* setup stack */ -#if CONFIG_IS_ENABLED(SMP) - /* tp: hart id */ - slli t0, tp, CONFIG_STACK_SIZE_SHIFT - sub sp, a0, t0 -#else - mv sp, a0 -#endif /* Configure proprietary settings and customized CSRs of harts */ call_harts_early_init:
Harts need to use per-hart stack before any function call, even if that function is a simple one. When the callee uses stack for register save/ restore, especially RA, if nested call, concurrent access by multiple harts on the same stack will cause data-race. This patch sets up SP before `board_init_f_alloc_reserve`. A side effect of this is that the memory layout has changed as the following: +----------------+ +----------------+ <----- SPL_STACK/ | ...... | | hart 0 stack | SYS_INIT_SP_ADDR | malloc_base | +----------------+ +----------------+ | hart 1 stack | | GD | +----------------+ If not SMP, N=1 +----------------+ | ...... | | hart 0 stack | +----------------+ +----------------+ ==> | hart N-1 stack| | hart 1 stack | +----------------+ +----------------+ | ...... | | ...... | | malloc_base | +----------------+ +----------------+ | hart N-1 stack| | GD | +----------------+ +----------------+ | | | | Signed-off-by: Bo Gan <ganboing@gmail.com> Cc: Rick Chen <rick@andestech.com> Cc: Leo <ycliang@andestech.com> Cc: Sean Anderson <seanga2@gmail.com> Cc: Bin Meng <bmeng.cn@gmail.com> Cc: Lukas Auer <lukas.auer@aisec.fraunhofer.de> --- v2: - Fixed macro CONFIG_SYS_INIT_SP_ADDR -> SYS_INIT_SP_ADDR - Tested SPL with VisionFive 2 board --- arch/riscv/cpu/start.S | 37 ++++++++++++++++++++++++------------- 1 file changed, 24 insertions(+), 13 deletions(-)