From patchwork Tue May 9 01:45:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bo Gan X-Patchwork-Id: 1778771 X-Patchwork-Delegate: uboot@andestech.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20221208 header.b=jsI/2vm2; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QFqQX1pYxz214c for ; Tue, 9 May 2023 17:23:48 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 43BF584788; Tue, 9 May 2023 09:23:44 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="jsI/2vm2"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 2720C85FA5; Tue, 9 May 2023 03:46:08 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 9F091846C6 for ; Tue, 9 May 2023 03:46:05 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=ganboing@gmail.com Received: by mail-pl1-x633.google.com with SMTP id d9443c01a7336-1ab1b79d3a7so36013955ad.3 for ; Mon, 08 May 2023 18:46:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1683596763; x=1686188763; h=message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5MJmelUKgxFAYXstVo56GQNS62ZfNbsBFyd/QUlO6gw=; b=jsI/2vm2QdZlRLZyl7TYzzlvTrlYdrR+XPdqkxoxGBfXjnbI/zOL9AYhufMHx8MNau ZoUFvB1fEEJRvfim3Z8n79ONBt1fwACmDD0fpUiONaJS29Jp49rTRYDt4+j3G7w0YiT/ 4VV7sDLrtMNdAcLCtO+zsZHtcFiEUPIhlh7KSvbRT/vi777C3IejtfssaRJCCF50z+HP L4n8JnRxGcbXkzHDvz9OZEw0SxKcHJJ8YCEGMRuJ0ZRshJJjEjYwTV++dJGWT1/cSpgQ luuY6EspRwhAGsCnVYXxOVR0ydYH+hJ3bbs1yCTxq5426zOJ++McU21dqp26TrejEYGS hYlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683596763; x=1686188763; h=message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5MJmelUKgxFAYXstVo56GQNS62ZfNbsBFyd/QUlO6gw=; b=G3nSdKuR3QFPnLUxEK9UehkDcnMUo+aCUYBGg9odJ3i9q1UzwVaqZdXWLYZ9qx/gjJ 7o5/d196TOyYo2NBa39PteAt+WgNQUhPa956GOPfs0TULu0iazc8MlWVu7Z0WYwEYaMh BQAyTfrOMEOejiTWqgbKU1cMdhc9by56nVdPrnNSCK10XyhGxr5r4HoLcExzK1j3Dmg0 u8B8DfBL/VRlloFEGhaDzHYc21MM8X/bpJ3paQOVVsI+nJY4BVCz1Bb2f2X1wKJm/v4u BhKRgLf2cQmNbQzedqPwRnSy2lAM2JRTZPrlltY/ptuE1DIm5d7egSSGPJl4kK1Io+Uv UkHA== X-Gm-Message-State: AC+VfDzwo005EE87Go782VFr2k6gdaJhCC/WW1qbBVp2SNYMzdlMgpGb ZGJV8lZIsF7dTA9aC6gpKDeQyIopeh1Zlw== X-Google-Smtp-Source: ACHHUZ4vWZ7ZiflHkSRGiq/dmP0yA83/CgNp7r8veuxw88mj7MgK1h/RropFwI30iLTbBDnLDgIcrQ== X-Received: by 2002:a17:902:ec84:b0:1ac:5405:f38e with SMTP id x4-20020a170902ec8400b001ac5405f38emr11724419plg.2.1683596763415; Mon, 08 May 2023 18:46:03 -0700 (PDT) Received: from m91p.airy.home ([172.92.174.136]) by smtp.gmail.com with ESMTPSA id i2-20020a170902c94200b001aae909cfbbsm144018pla.119.2023.05.08.18.46.02 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 08 May 2023 18:46:02 -0700 (PDT) From: Bo Gan To: u-boot@lists.denx.de Cc: Bo Gan , Rick Chen , Leo Subject: [PATCH] riscv: setup per-hart stack earlier Date: Mon, 8 May 2023 18:45:41 -0700 Message-Id: <1683596741-2429007-1-git-send-email-ganboing@gmail.com> X-Mailer: git-send-email 2.7.4 X-Mailman-Approved-At: Tue, 09 May 2023 09:23:41 +0200 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Harts need to use per-hart stack before any function call, even if that function is a simple one. When the callee uses stack for register save/ restore, especially RA, if nested call, concurrent access by multiple harts on the same stack will cause data-race. This patch sets up SP before `board_init_f_alloc_reserve`. A side effect of this is that the memory layout has changed as the following: +----------------+ +----------------+ <----- SPL_STACK/ | ...... | | hart 0 stack | SYS_INIT_SP_ADDR | malloc_base | +----------------+ +----------------+ | hart 1 stack | | GD | +----------------+ If not SMP, N=1 +----------------+ | ...... | | hart 0 stack | +----------------+ +----------------+ ==> | hart N-1 stack| | hart 1 stack | +----------------+ +----------------+ | ...... | | ...... | | malloc_base | +----------------+ +----------------+ | hart N-1 stack| | GD | +----------------+ +----------------+ | | | | Signed-off-by: Bo Gan Cc: Rick Chen Cc: Leo --- arch/riscv/cpu/start.S | 37 ++++++++++++++++++++++++------------- 1 file changed, 24 insertions(+), 13 deletions(-) diff --git a/arch/riscv/cpu/start.S b/arch/riscv/cpu/start.S index dad22bf..90015c2 100644 --- a/arch/riscv/cpu/start.S +++ b/arch/riscv/cpu/start.S @@ -91,16 +91,35 @@ _start: * Set stackpointer in internal/ex RAM to call board_init_f */ call_board_init_f: - li t0, -16 #if defined(CONFIG_SPL_BUILD) && defined(CONFIG_SPL_STACK) - li t1, CONFIG_SPL_STACK + li t0, CONFIG_SPL_STACK #else - li t1, SYS_INIT_SP_ADDR + li t0, CONFIG_SYS_INIT_SP_ADDR #endif - and sp, t1, t0 /* force 16 byte alignment */ + and t0, t0, -16 /* force 16 byte alignment */ + + /* setup stack */ +#if CONFIG_IS_ENABLED(SMP) + /* tp: hart id */ + slli t1, tp, CONFIG_STACK_SIZE_SHIFT + sub sp, t0, t1 +#else + mv sp, t0 +#endif +/* + * Now sp points to the right stack belonging to current CPU. + * It's essential before any function call, otherwise, we get data-race. + */ call_board_init_f_0: - mv a0, sp + /* find top of reserve space */ +#if CONFIG_IS_ENABLED(SMP) + li t1, CONFIG_NR_CPUS +#else + li t1, 1 +#endif + slli t1, t1, CONFIG_STACK_SIZE_SHIFT + sub a0, t0, t1 /* t1 -> size of all CPU stacks */ jal board_init_f_alloc_reserve /* @@ -109,14 +128,6 @@ call_board_init_f_0: */ mv s0, a0 - /* setup stack */ -#if CONFIG_IS_ENABLED(SMP) - /* tp: hart id */ - slli t0, tp, CONFIG_STACK_SIZE_SHIFT - sub sp, a0, t0 -#else - mv sp, a0 -#endif /* Configure proprietary settings and customized CSRs of harts */ call_harts_early_init: