From patchwork Sun May 21 06:20:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bo Gan X-Patchwork-Id: 1784121 X-Patchwork-Delegate: uboot@andestech.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20221208 header.b=YB7+Eh2i; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QP9Sh3mKGz20PY for ; Sun, 21 May 2023 16:21:06 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 5F349846C1; Sun, 21 May 2023 08:20:57 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="YB7+Eh2i"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id F1754846C6; Sun, 21 May 2023 08:20:55 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 7A7FC8468A for ; Sun, 21 May 2023 08:20:53 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=ganboing@gmail.com Received: by mail-pj1-x102b.google.com with SMTP id 98e67ed59e1d1-2537909d28cso1708693a91.0 for ; Sat, 20 May 2023 23:20:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1684650051; x=1687242051; h=message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=X02x6r00qz2hqwWXfAA9EaGwUD0NTXWnGpv1dwuZ6OY=; b=YB7+Eh2ix7gtmr71iGRSejioeZXB+NirOuf3M1LUdo7EOQBLEE90/KhxGeuqTu9U9B 60ZACmd5NBwt7R5tKS9AN7LUNB0CPyWbp6JbdrrIzBszgURM5sjfpsB9OVePddgvv74W Kj3M3rF4P5WyYj4lLmKkIv1Azvn44A2ab5rHd6ABULHy4AqbF3G7ddfi5oo/0S5R/n95 yDdj1SFx8e/M6sriG8F9SIrYy6x4vLGg0syCuyq+4PLi+xmN2ObKrlMsvJBgn/lusLxA /B8wm5YjDxAumGtvh9DJFSVfxUCssuq1phM0E6Ts0L/TyDglt3etAeq/MBFh2FMXetu1 GzFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684650051; x=1687242051; h=message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=X02x6r00qz2hqwWXfAA9EaGwUD0NTXWnGpv1dwuZ6OY=; b=lzU//jzcwj1IDNBaZWLdFbiIMtFICWUs4NLENaP1Y+3AdorfqBNH7yxow2jUDLSyeb W+IBUyb48sX0/K1gKIAQrN5osxau9fHZXJVbZgLJUIIL4l2mvL5wfXX/kgTfIcobayXk 1y+MtSDNj8/BqKNbMvNiePM7TechbntOnuU1FTEzEFzepmhLMy15mD0s+216y4YoT/Ul veoHJdobke0rOM7jaPS01i8WnajE2cZEZFZbIbqZDt28asg+ABE7xi5CIqpNI62GCROj CBhNurTeVx0PJWUakW6j2do4Qw2s6I5WxqFjKO4yzCujAX25Y/4hnqLEo5lvpntQgM4b 06cw== X-Gm-Message-State: AC+VfDz78e6YyDxs1Xvdva/19L4wEm0Wk9EIKvbuf/tnUEMYb6sIEwAn RZ9jLl4m6SroDj+c9jYF7yr7x+fm5flhUQ== X-Google-Smtp-Source: ACHHUZ6Wkvm7Ld9Kw/rfxrCdZeIG0GIKBSv/rni5GaE6Acez5S2Vp/UdH3PbbG4ODpZRiL8CtrSgZg== X-Received: by 2002:a17:90b:1d03:b0:24d:fba9:80e9 with SMTP id on3-20020a17090b1d0300b0024dfba980e9mr7694765pjb.23.1684650051337; Sat, 20 May 2023 23:20:51 -0700 (PDT) Received: from riscv.airy.home ([172.92.174.136]) by smtp.gmail.com with ESMTPSA id p1-20020a17090b010100b00250aa8ef89csm2105759pjz.18.2023.05.20.23.20.50 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 20 May 2023 23:20:50 -0700 (PDT) From: Bo Gan To: u-boot@lists.denx.de Cc: Bo Gan , Rick Chen , Leo , Sean Anderson , Bin Meng , Lukas Auer Subject: [PATCH v2] riscv: setup per-hart stack earlier Date: Sat, 20 May 2023 23:20:44 -0700 Message-Id: <1684650044-313122-1-git-send-email-ganboing@gmail.com> X-Mailer: git-send-email 2.7.4 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Harts need to use per-hart stack before any function call, even if that function is a simple one. When the callee uses stack for register save/ restore, especially RA, if nested call, concurrent access by multiple harts on the same stack will cause data-race. This patch sets up SP before `board_init_f_alloc_reserve`. A side effect of this is that the memory layout has changed as the following: +----------------+ +----------------+ <----- SPL_STACK/ | ...... | | hart 0 stack | SYS_INIT_SP_ADDR | malloc_base | +----------------+ +----------------+ | hart 1 stack | | GD | +----------------+ If not SMP, N=1 +----------------+ | ...... | | hart 0 stack | +----------------+ +----------------+ ==> | hart N-1 stack| | hart 1 stack | +----------------+ +----------------+ | ...... | | ...... | | malloc_base | +----------------+ +----------------+ | hart N-1 stack| | GD | +----------------+ +----------------+ | | | | Signed-off-by: Bo Gan Cc: Rick Chen Cc: Leo Cc: Sean Anderson Cc: Bin Meng Cc: Lukas Auer --- v2: - Fixed macro CONFIG_SYS_INIT_SP_ADDR -> SYS_INIT_SP_ADDR - Tested SPL with VisionFive 2 board --- arch/riscv/cpu/start.S | 37 ++++++++++++++++++++++++------------- 1 file changed, 24 insertions(+), 13 deletions(-) diff --git a/arch/riscv/cpu/start.S b/arch/riscv/cpu/start.S index dad22bf..59d58a5 100644 --- a/arch/riscv/cpu/start.S +++ b/arch/riscv/cpu/start.S @@ -91,16 +91,35 @@ _start: * Set stackpointer in internal/ex RAM to call board_init_f */ call_board_init_f: - li t0, -16 #if defined(CONFIG_SPL_BUILD) && defined(CONFIG_SPL_STACK) - li t1, CONFIG_SPL_STACK + li t0, CONFIG_SPL_STACK #else - li t1, SYS_INIT_SP_ADDR + li t0, SYS_INIT_SP_ADDR #endif - and sp, t1, t0 /* force 16 byte alignment */ + and t0, t0, -16 /* force 16 byte alignment */ + + /* setup stack */ +#if CONFIG_IS_ENABLED(SMP) + /* tp: hart id */ + slli t1, tp, CONFIG_STACK_SIZE_SHIFT + sub sp, t0, t1 +#else + mv sp, t0 +#endif +/* + * Now sp points to the right stack belonging to current CPU. + * It's essential before any function call, otherwise, we get data-race. + */ call_board_init_f_0: - mv a0, sp + /* find top of reserve space */ +#if CONFIG_IS_ENABLED(SMP) + li t1, CONFIG_NR_CPUS +#else + li t1, 1 +#endif + slli t1, t1, CONFIG_STACK_SIZE_SHIFT + sub a0, t0, t1 /* t1 -> size of all CPU stacks */ jal board_init_f_alloc_reserve /* @@ -109,14 +128,6 @@ call_board_init_f_0: */ mv s0, a0 - /* setup stack */ -#if CONFIG_IS_ENABLED(SMP) - /* tp: hart id */ - slli t0, tp, CONFIG_STACK_SIZE_SHIFT - sub sp, a0, t0 -#else - mv sp, a0 -#endif /* Configure proprietary settings and customized CSRs of harts */ call_harts_early_init: