From patchwork Sun Jun 11 23:54:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bo Gan X-Patchwork-Id: 1793730 X-Patchwork-Delegate: uboot@andestech.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20221208 header.b=kntw0XBw; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QfWv01h9Yz20X6 for ; Mon, 12 Jun 2023 09:56:43 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 2E53F8477A; Mon, 12 Jun 2023 01:56:38 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="kntw0XBw"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id C76E7847BA; Mon, 12 Jun 2023 01:56:36 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 08D9F84704 for ; Mon, 12 Jun 2023 01:56:34 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=ganboing@gmail.com Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-652a6bf4e6aso2950828b3a.2 for ; Sun, 11 Jun 2023 16:56:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686527792; x=1689119792; h=message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=X02x6r00qz2hqwWXfAA9EaGwUD0NTXWnGpv1dwuZ6OY=; b=kntw0XBwhQZXrTF4nrCE1NgWixgwzFGIjY70HTagS7krfmwGJf3tr6Yc91jw6nNsdO o0jr89rB+T++HI3Rn1EbEeE37WhfALMYw/0KzyC9p1sNgfOePkdJmJ5VDMOm0NA+71iP 9uodJJQDhFTZWRLsaBcEmgAn6Y1HHC/cbY8zTX1WCGwnedcYISJPUZIjFmzJOeiMCC+d 3ppm0JVpJOZmUHclCUOuMdnQj+W1Zg45253BaUm0GFYjHLGHdBVtW/i+5JBZQgzV5EG3 3SobrAsJz08yjicxjbzs1/aOnmcWFfdpRfKBlJAGDBKMad31AOe3H9SmAw4R9I+1M0QH 7J0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686527792; x=1689119792; h=message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=X02x6r00qz2hqwWXfAA9EaGwUD0NTXWnGpv1dwuZ6OY=; b=LnTUmG2pcmjsXd3Ew6DYZOYXknFQhPKb0k61o1ORzqnUrW5abuKN6BY7zRAbhWf28M iKkFv1z3a2b73+RfF40FWlIwVztegnHIx+aJPLusglc+j1H03/ceZJFAZMyEetyQ73bZ 3B1F2uaXIHBoUaIqOQvtwb4xzf3cmYHBFpGB0KyxofOw26TJpNKQf+0ch5yCzDGfDdTM 1GxQiBciizmyftm7sxP33v8BR17yf+AST5MqXtBr73BkqJkT88GZPxC/gutxE3Rqzxm7 wpHSkK1NbefxBYftiLxlnI9Od7Nh7CKcFwnHYMMED3xgo2yfxmbbNKz48kCXD27SsEf1 OsMA== X-Gm-Message-State: AC+VfDz1BkDKbdnUYsnfe3Qa3nsNJPPtVb3lFFF+bdFUp9Zc7BftRsqu tyi5GeQOCXAtvAsKxy6ULGws9DSUbkYXeA== X-Google-Smtp-Source: ACHHUZ5CrrbgRgnIoOak+t5mfUcMEidl0tTR3+lBngY19yF/Wqjf6qG8ICEddN1tMCp7huVnP3AxUQ== X-Received: by 2002:a05:6a00:b8c:b0:645:fc7b:63db with SMTP id g12-20020a056a000b8c00b00645fc7b63dbmr9905236pfj.20.1686527791866; Sun, 11 Jun 2023 16:56:31 -0700 (PDT) Received: from riscv.airy.home ([172.92.174.136]) by smtp.gmail.com with ESMTPSA id 23-20020aa79157000000b0063b806b111csm5751079pfi.169.2023.06.11.16.56.30 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 11 Jun 2023 16:56:31 -0700 (PDT) From: Bo Gan To: u-boot@lists.denx.de Cc: Bo Gan , Rick Chen , Leo , Sean Anderson , Bin Meng , Lukas Auer Subject: [RESEND PATCH v2] riscv: setup per-hart stack earlier Date: Sun, 11 Jun 2023 16:54:17 -0700 Message-Id: <1686527657-366116-1-git-send-email-ganboing@gmail.com> X-Mailer: git-send-email 2.7.4 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Harts need to use per-hart stack before any function call, even if that function is a simple one. When the callee uses stack for register save/ restore, especially RA, if nested call, concurrent access by multiple harts on the same stack will cause data-race. This patch sets up SP before `board_init_f_alloc_reserve`. A side effect of this is that the memory layout has changed as the following: +----------------+ +----------------+ <----- SPL_STACK/ | ...... | | hart 0 stack | SYS_INIT_SP_ADDR | malloc_base | +----------------+ +----------------+ | hart 1 stack | | GD | +----------------+ If not SMP, N=1 +----------------+ | ...... | | hart 0 stack | +----------------+ +----------------+ ==> | hart N-1 stack| | hart 1 stack | +----------------+ +----------------+ | ...... | | ...... | | malloc_base | +----------------+ +----------------+ | hart N-1 stack| | GD | +----------------+ +----------------+ | | | | Signed-off-by: Bo Gan Cc: Rick Chen Cc: Leo Cc: Sean Anderson Cc: Bin Meng Cc: Lukas Auer Reviewed-by: Rick Chen Reviewed-by: Leo Yu-Chi Liang --- v2: - Fixed macro CONFIG_SYS_INIT_SP_ADDR -> SYS_INIT_SP_ADDR - Tested SPL with VisionFive 2 board --- arch/riscv/cpu/start.S | 37 ++++++++++++++++++++++++------------- 1 file changed, 24 insertions(+), 13 deletions(-) diff --git a/arch/riscv/cpu/start.S b/arch/riscv/cpu/start.S index dad22bf..59d58a5 100644 --- a/arch/riscv/cpu/start.S +++ b/arch/riscv/cpu/start.S @@ -91,16 +91,35 @@ _start: * Set stackpointer in internal/ex RAM to call board_init_f */ call_board_init_f: - li t0, -16 #if defined(CONFIG_SPL_BUILD) && defined(CONFIG_SPL_STACK) - li t1, CONFIG_SPL_STACK + li t0, CONFIG_SPL_STACK #else - li t1, SYS_INIT_SP_ADDR + li t0, SYS_INIT_SP_ADDR #endif - and sp, t1, t0 /* force 16 byte alignment */ + and t0, t0, -16 /* force 16 byte alignment */ + + /* setup stack */ +#if CONFIG_IS_ENABLED(SMP) + /* tp: hart id */ + slli t1, tp, CONFIG_STACK_SIZE_SHIFT + sub sp, t0, t1 +#else + mv sp, t0 +#endif +/* + * Now sp points to the right stack belonging to current CPU. + * It's essential before any function call, otherwise, we get data-race. + */ call_board_init_f_0: - mv a0, sp + /* find top of reserve space */ +#if CONFIG_IS_ENABLED(SMP) + li t1, CONFIG_NR_CPUS +#else + li t1, 1 +#endif + slli t1, t1, CONFIG_STACK_SIZE_SHIFT + sub a0, t0, t1 /* t1 -> size of all CPU stacks */ jal board_init_f_alloc_reserve /* @@ -109,14 +128,6 @@ call_board_init_f_0: */ mv s0, a0 - /* setup stack */ -#if CONFIG_IS_ENABLED(SMP) - /* tp: hart id */ - slli t0, tp, CONFIG_STACK_SIZE_SHIFT - sub sp, a0, t0 -#else - mv sp, a0 -#endif /* Configure proprietary settings and customized CSRs of harts */ call_harts_early_init: