From patchwork Tue Dec 3 21:39:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukas Auer X-Patchwork-Id: 1203854 X-Patchwork-Delegate: uboot@andestech.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=aisec.fraunhofer.de Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 47SG6551VJz9s4Y for ; Wed, 4 Dec 2019 08:56:45 +1100 (AEDT) Received: by phobos.denx.de (Postfix, from userid 109) id 22D5B81815; Tue, 3 Dec 2019 22:54:46 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mail.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from phobos.denx.de (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id AAF11816C0; Tue, 3 Dec 2019 22:42:08 +0100 (CET) Authentication-Results: mail.denx.de; dmarc=none (p=none dis=none) header.from=aisec.fraunhofer.de Authentication-Results: mail.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Received: by phobos.denx.de (Postfix, from userid 109) id 800ED80668; Tue, 3 Dec 2019 22:40:29 +0100 (CET) Received: from mail-edgeKA27.fraunhofer.de (mail-edgeka27.fraunhofer.de [153.96.1.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 4063F808B6 for ; Tue, 3 Dec 2019 22:40:25 +0100 (CET) Authentication-Results: mail.denx.de; dmarc=none (p=none dis=none) header.from=aisec.fraunhofer.de Authentication-Results: mail.denx.de; spf=none smtp.mailfrom=lukas.auer@aisec.fraunhofer.de X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2HDBACT1eZd/xoHYZllHQEBAQkBEQUFAYF+giCBQQEfPAqNJKNLCQEBAQEBAQEBAQcBLwEBhEACgg4kOBMCEAEBBQEBAQEBBQQCAmmFQ4YoBicLAUEFEFFXBg4FgyKCdwGzCzOIe4FICQGBLIc9hFkPgUw/gRE2gxuEEoYlBI0KgleeYgeBRG1uBIVKjxEMG4JBjAGLYpANmFoCBAIEBQIVgWkjgVgzGiSDO1ARFIxyF44kQAEBMQGPPYExgRABAQ X-IPAS-Result: A2HDBACT1eZd/xoHYZllHQEBAQkBEQUFAYF+giCBQQEfPAqNJKNLCQEBAQEBAQEBAQcBLwEBhEACgg4kOBMCEAEBBQEBAQEBBQQCAmmFQ4YoBicLAUEFEFFXBg4FgyKCdwGzCzOIe4FICQGBLIc9hFkPgUw/gRE2gxuEEoYlBI0KgleeYgeBRG1uBIVKjxEMG4JBjAGLYpANmFoCBAIEBQIVgWkjgVgzGiSDO1ARFIxyF44kQAEBMQGPPYExgRABAQ X-IronPort-AV: E=Sophos;i="5.69,275,1571695200"; d="scan'208";a="18789691" Received: from mail-mtas26.fraunhofer.de ([153.97.7.26]) by mail-edgeKA27.fraunhofer.de with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Dec 2019 22:40:24 +0100 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0ATAgCT1eZdfRBhWMBlHAEBAQEBBwEBEQEEBAEBgX6DYAEBHzwKjSSjSwkBAwEBAQEBBwEvAQGEQAKCMTgTAhABAQQBAQECAQUEFAEBFjqFSoVTAwMnCwFBBRBRVwYOBYMignizCzOIe4FICQGBLIc9hGiBTD+BETaDG4QShiUEjQqCV55iB4FEbW4EhUqPEQwbgkGMAYtikA2YWgIEAgQFAhWBaSGBWTMaJIM7UBEUjHIXjiRAAQEBMAGPPYExgRABAQ X-IronPort-AV: E=Sophos;i="5.69,275,1571695200"; d="scan'208";a="98933309" Received: from fgdemucivp01ltm.xch.fraunhofer.de (HELO FGDEMUCIMP12EXC.ads.fraunhofer.de) ([192.88.97.16]) by mail-mtaS26.fraunhofer.de with ESMTP/TLS/AES256-SHA; 03 Dec 2019 22:40:21 +0100 Received: from xch-onprem-07.ads.fraunhofer.de (10.225.16.45) by FGDEMUCIMP12EXC.ads.fraunhofer.de (10.80.232.43) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 3 Dec 2019 22:40:21 +0100 Received: from localhost.localdomain (10.225.16.62) by xch-onprem-07.ads.fraunhofer.de (10.225.16.45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1779.2; Tue, 3 Dec 2019 22:40:20 +0100 From: Lukas Auer To: Subject: [PATCH 1/4] spl: opensbi: specify main hart as preferred boot hart Date: Tue, 3 Dec 2019 22:39:53 +0100 Message-ID: <20191203213956.24339-2-lukas.auer@aisec.fraunhofer.de> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191203213956.24339-1-lukas.auer@aisec.fraunhofer.de> References: <20191203213956.24339-1-lukas.auer@aisec.fraunhofer.de> MIME-Version: 1.0 X-Originating-IP: [10.225.16.62] X-ClientProxiedBy: xch-onprem-05.ads.fraunhofer.de (10.225.16.54) To xch-onprem-07.ads.fraunhofer.de (10.225.16.45) X-TM-AS-Product-Ver: SMEX-11.0.0.4179-8.200.1013-25080.006 X-TM-AS-Result: No--4.290200-8.000000-31 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.26 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.101.4 at mail.denx.de X-Virus-Status: Clean OpenSBI uses a relocation lottery to determine the hart to relocate OpenSBI to its link address. In the U-Boot SPL boot flow, the main hart schedules the secondary harts to enter OpenSBI before doing so itself. One of the secondary harts will therefore always be the winner of the relocation lottery. This is problematic if the link address ranges of OpenSBI and U-Boot SPL overlap. OpenSBI will be relocated and therefore overwrite U-Boot SPL while some harts may still run it, leading to code corruption. Avoid this problem by specifying the main hart as the preferred boot hart to perform the OpenSBI relocation. The main hart will be the last hart to enter OpenSBI, relocation can therefore occur safely. The boot hart field was added to version 2 of the OpenSBI FW_DYNAMIC info structure. The header file include/opensbi.h is synchronized with include/sbi/fw_dynamic.h from the OpenSBI project to update the info structure. The header file is recent as of commit 7a13beb21326 ("firmware: Add preferred boot HART field in struct fw_dynamic_info"). Reported-by: Rick Chen Suggested-by: Anup Patel Signed-off-by: Lukas Auer Reviewed-by: Rick Chen Tested-by: Rick Chen Reviewed-by: Anup Patel --- common/spl/spl_opensbi.c | 1 + include/opensbi.h | 18 +++++++++++++++++- 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/common/spl/spl_opensbi.c b/common/spl/spl_opensbi.c index a6b4480ed2..79ee7edcf9 100644 --- a/common/spl/spl_opensbi.c +++ b/common/spl/spl_opensbi.c @@ -69,6 +69,7 @@ void spl_invoke_opensbi(struct spl_image_info *spl_image) opensbi_info.next_addr = uboot_entry; opensbi_info.next_mode = FW_DYNAMIC_INFO_NEXT_MODE_S; opensbi_info.options = SBI_SCRATCH_NO_BOOT_PRINTS; + opensbi_info.boot_hart = gd->arch.boot_hart; opensbi_entry = (void (*)(ulong, ulong, ulong))spl_image->entry_point; invalidate_icache_all(); diff --git a/include/opensbi.h b/include/opensbi.h index 9f1d62e7dd..d812cc8ccd 100644 --- a/include/opensbi.h +++ b/include/opensbi.h @@ -11,7 +11,7 @@ #define FW_DYNAMIC_INFO_MAGIC_VALUE 0x4942534f /** Maximum supported info version */ -#define FW_DYNAMIC_INFO_VERSION 0x1 +#define FW_DYNAMIC_INFO_VERSION 0x2 /** Possible next mode values */ #define FW_DYNAMIC_INFO_NEXT_MODE_U 0x0 @@ -35,6 +35,22 @@ struct fw_dynamic_info { unsigned long next_mode; /** Options for OpenSBI library */ unsigned long options; + /** + * Preferred boot HART id + * + * It is possible that the previous booting stage uses same link + * address as the FW_DYNAMIC firmware. In this case, the relocation + * lottery mechanism can potentially overwrite the previous booting + * stage while other HARTs are still running in the previous booting + * stage leading to boot-time crash. To avoid this boot-time crash, + * the previous booting stage can specify last HART that will jump + * to the FW_DYNAMIC firmware as the preferred boot HART. + * + * To avoid specifying a preferred boot HART, the previous booting + * stage can set it to -1UL which will force the FW_DYNAMIC firmware + * to use the relocation lottery mechanism. + */ + unsigned long boot_hart; } __packed; #endif From patchwork Tue Dec 3 21:39:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukas Auer X-Patchwork-Id: 1203847 X-Patchwork-Delegate: uboot@andestech.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=aisec.fraunhofer.de Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 47SFmK3fZ9z9s4Y for ; Wed, 4 Dec 2019 08:41:21 +1100 (AEDT) Received: by phobos.denx.de (Postfix, from userid 109) id CB7F381669; Tue, 3 Dec 2019 22:41:18 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mail.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from phobos.denx.de (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id AB5EC80668; Tue, 3 Dec 2019 22:40:30 +0100 (CET) Authentication-Results: mail.denx.de; dmarc=none (p=none dis=none) header.from=aisec.fraunhofer.de Authentication-Results: mail.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Received: by phobos.denx.de (Postfix, from userid 109) id 3F5228162F; Tue, 3 Dec 2019 22:40:27 +0100 (CET) Received: from mail-edgeKA27.fraunhofer.de (mail-edgeka27.fraunhofer.de [153.96.1.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id B53FC80668 for ; Tue, 3 Dec 2019 22:40:24 +0100 (CET) Authentication-Results: mail.denx.de; dmarc=none (p=none dis=none) header.from=aisec.fraunhofer.de Authentication-Results: mail.denx.de; spf=none smtp.mailfrom=lukas.auer@aisec.fraunhofer.de X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2FJAwCT1eZd/xoBYJllHAEBAQEBBwEBEQEEBAEBgX6CIIFBAR88Co0ko0sJAQEBAQEBAQEBBwEvAQGEQAKCDiQ4EwIQAQEFAQEBAQEFBAICaYVDhigGJwsBRhBRVwYOBYMigncBswsziHuBSAkBgSyHPWaDcw+BTD+BETaDG4o3BI9hnmIHgURtbgSFSo8RDBuCQYduhBMsizaoZwIEAgQFAhWBaSOBWDMaJIM7UBEUjHIXjiRAAQExAZBugRABAQ X-IPAS-Result: A2FJAwCT1eZd/xoBYJllHAEBAQEBBwEBEQEEBAEBgX6CIIFBAR88Co0ko0sJAQEBAQEBAQEBBwEvAQGEQAKCDiQ4EwIQAQEFAQEBAQEFBAICaYVDhigGJwsBRhBRVwYOBYMigncBswsziHuBSAkBgSyHPWaDcw+BTD+BETaDG4o3BI9hnmIHgURtbgSFSo8RDBuCQYduhBMsizaoZwIEAgQFAhWBaSOBWDMaJIM7UBEUjHIXjiRAAQExAZBugRABAQ X-IronPort-AV: E=Sophos;i="5.69,275,1571695200"; d="scan'208";a="18789689" Received: from mail-mtaka26.fraunhofer.de ([153.96.1.26]) by mail-edgeKA27.fraunhofer.de with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Dec 2019 22:40:24 +0100 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0ATAgCT1eZdfRBhWMBlHAEBAQEBBwEBEQEEBAEBgX6DYAEBHzwKjSSjSwkBAwEBAQEBBwEvAQGEQAKCMTgTAhABAQQBAQECAQUEFAEBFjqFSoVTAwMnCwFGEFFXBg4FgyKCeLMLM4h7gUgJAYEshz1mhAKBTD+BETaDG4o3BI9hnmIHgURtbgSFSo8RDBuCQYduhBMsizaoZwIEAgQFAhWBaSGBWTMaJIM7UBEUjHIXjiRAAQEBMAGQboEQAQE X-IronPort-AV: E=Sophos;i="5.69,275,1571695200"; d="scan'208";a="67490453" Received: from fgdemucivp01ltm.xch.fraunhofer.de (HELO FGDEMUCIMP11EXC.ads.fraunhofer.de) ([192.88.97.16]) by mail-mtaKA26.fraunhofer.de with ESMTP/TLS/AES256-SHA; 03 Dec 2019 22:40:22 +0100 Received: from xch-onprem-07.ads.fraunhofer.de (10.225.16.45) by FGDEMUCIMP11EXC.ads.fraunhofer.de (10.80.232.42) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 3 Dec 2019 22:40:21 +0100 Received: from localhost.localdomain (10.225.16.62) by xch-onprem-07.ads.fraunhofer.de (10.225.16.45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1779.2; Tue, 3 Dec 2019 22:40:21 +0100 From: Lukas Auer To: Subject: [PATCH 2/4] riscv: add functions for reading the IPI status Date: Tue, 3 Dec 2019 22:39:54 +0100 Message-ID: <20191203213956.24339-3-lukas.auer@aisec.fraunhofer.de> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191203213956.24339-1-lukas.auer@aisec.fraunhofer.de> References: <20191203213956.24339-1-lukas.auer@aisec.fraunhofer.de> MIME-Version: 1.0 X-Originating-IP: [10.225.16.62] X-ClientProxiedBy: xch-onprem-05.ads.fraunhofer.de (10.225.16.54) To xch-onprem-07.ads.fraunhofer.de (10.225.16.45) X-TM-AS-Product-Ver: SMEX-11.0.0.4179-8.200.1013-25080.006 X-TM-AS-Result: No--8.703000-8.000000-31 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.26 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.101.4 at mail.denx.de X-Virus-Status: Clean Add the function riscv_get_ipi() for reading the pending status of IPIs. The supported controllers are Andes' Platform Level Interrupt Controller (PLIC), the Supervisor Binary Interface (SBI), and SiFive's Core Local Interruptor (CLINT). Signed-off-by: Lukas Auer --- I do not have access to the datasheet of the Andes PLIC. The riscv_clear_ipi() implementation seems to read the IPI status from the claim register before writing back the results to clear them. Based on this, I also used the claim register. Rick, please let me know if that is ok or if I should use the pending register instead. arch/riscv/lib/andes_plic.c | 9 +++++++++ arch/riscv/lib/sbi_ipi.c | 11 +++++++++++ arch/riscv/lib/sifive_clint.c | 9 +++++++++ arch/riscv/lib/smp.c | 12 ++++++++++++ 4 files changed, 41 insertions(+) diff --git a/arch/riscv/lib/andes_plic.c b/arch/riscv/lib/andes_plic.c index 28568e4e2b..731ac3a148 100644 --- a/arch/riscv/lib/andes_plic.c +++ b/arch/riscv/lib/andes_plic.c @@ -114,6 +114,15 @@ int riscv_clear_ipi(int hart) return 0; } +int riscv_get_ipi(int hart, int *pending) +{ + PLIC_BASE_GET(); + + *pending = !!readl((void __iomem *)CLAIM_REG(gd->arch.plic, hart)); + + return 0; +} + static const struct udevice_id andes_plic_ids[] = { { .compatible = "riscv,plic1", .data = RISCV_SYSCON_PLIC }, { } diff --git a/arch/riscv/lib/sbi_ipi.c b/arch/riscv/lib/sbi_ipi.c index 170346da68..9a698ce74e 100644 --- a/arch/riscv/lib/sbi_ipi.c +++ b/arch/riscv/lib/sbi_ipi.c @@ -23,3 +23,14 @@ int riscv_clear_ipi(int hart) return 0; } + +int riscv_get_ipi(int hart, int *pending) +{ + /* + * The SBI does not support reading the IPI status. We always return 0 + * to indicate that no IPI is pending. + */ + *pending = 0; + + return 0; +} diff --git a/arch/riscv/lib/sifive_clint.c b/arch/riscv/lib/sifive_clint.c index d24e0d585b..d7899d16d7 100644 --- a/arch/riscv/lib/sifive_clint.c +++ b/arch/riscv/lib/sifive_clint.c @@ -71,6 +71,15 @@ int riscv_clear_ipi(int hart) return 0; } +int riscv_get_ipi(int hart, int *pending) +{ + CLINT_BASE_GET(); + + *pending = readl((void __iomem *)MSIP_REG(gd->arch.clint, hart)); + + return 0; +} + static const struct udevice_id sifive_clint_ids[] = { { .compatible = "riscv,clint0", .data = RISCV_SYSCON_CLINT }, { } diff --git a/arch/riscv/lib/smp.c b/arch/riscv/lib/smp.c index cc66f15567..6ff0de4b74 100644 --- a/arch/riscv/lib/smp.c +++ b/arch/riscv/lib/smp.c @@ -31,6 +31,18 @@ extern int riscv_send_ipi(int hart); */ extern int riscv_clear_ipi(int hart); +/** + * riscv_get_ipi() - Get status of inter-processor interrupt (IPI) + * + * Platform code must provide this function. + * + * @hart: Hart ID of hart to be checked + * @pending: Pointer to variable with result of the check, + * 1 if IPI is pending, 0 otherwise + * @return 0 if OK, -ve on error + */ +extern int riscv_get_ipi(int hart, int *pending); + static int send_ipi_many(struct ipi_data *ipi) { ofnode node, cpus; From patchwork Tue Dec 3 21:39:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukas Auer X-Patchwork-Id: 1203848 X-Patchwork-Delegate: uboot@andestech.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=aisec.fraunhofer.de Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 47SFnz63s8z9s4Y for ; Wed, 4 Dec 2019 08:42:47 +1100 (AEDT) Received: by phobos.denx.de (Postfix, from userid 109) id 2C1618172A; Tue, 3 Dec 2019 22:42:45 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mail.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from phobos.denx.de (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 9AA79816AC; Tue, 3 Dec 2019 22:41:12 +0100 (CET) Authentication-Results: mail.denx.de; dmarc=none (p=none dis=none) header.from=aisec.fraunhofer.de Authentication-Results: mail.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Received: by phobos.denx.de (Postfix, from userid 109) id C0E57815E1; Tue, 3 Dec 2019 22:40:27 +0100 (CET) Received: from mail-edgeKA27.fraunhofer.de (mail-edgeka27.fraunhofer.de [153.96.1.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 7C48A80661 for ; Tue, 3 Dec 2019 22:40:24 +0100 (CET) Authentication-Results: mail.denx.de; dmarc=none (p=none dis=none) header.from=aisec.fraunhofer.de Authentication-Results: mail.denx.de; spf=none smtp.mailfrom=lukas.auer@aisec.fraunhofer.de X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2GFBACT1eZd/xoHYZllHAEBAQEBBwEBEQEEBAEBgX4Cgh6BQQEfPAqNJKNLCQEBAQEBAQEBAQcBLwEBhEACgg4kOBMCEAEBBQEBAQEBBQQCAmmFQ0IBDAGFWAYnCwFGEFFXBg4FgyKCdwGzCzOIe4FICQGBLAGHPGaDcw+BTD+BR4MbijcEjSSCPYhElh4HgURtbgSFSo8RDBuOQotiqGcCBAIEBQIVgWkjgVgzGiSDO1ARFIxyFxWOD0ABATEBhimKRYEQAQE X-IPAS-Result: A2GFBACT1eZd/xoHYZllHAEBAQEBBwEBEQEEBAEBgX4Cgh6BQQEfPAqNJKNLCQEBAQEBAQEBAQcBLwEBhEACgg4kOBMCEAEBBQEBAQEBBQQCAmmFQ0IBDAGFWAYnCwFGEFFXBg4FgyKCdwGzCzOIe4FICQGBLAGHPGaDcw+BTD+BR4MbijcEjSSCPYhElh4HgURtbgSFSo8RDBuOQotiqGcCBAIEBQIVgWkjgVgzGiSDO1ARFIxyFxWOD0ABATEBhimKRYEQAQE X-IronPort-AV: E=Sophos;i="5.69,275,1571695200"; d="scan'208";a="18789692" Received: from mail-mtas26.fraunhofer.de ([153.97.7.26]) by mail-edgeKA27.fraunhofer.de with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Dec 2019 22:40:24 +0100 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0DcAgCT1eZdfRBhWMBlHAEBAQEBBwEBEQEEBAEBgX4Cg14BAR88Co0ko0sJAQMBAQEBAQcBLwEBhEACgjE4EwIQAQEEAQEBAgEFBBQBARY6hUpCAQwBhQMDAycLAUYQUVcGDgWDIoJ4swsziHuBSAkBgSwBhzxmhAKBTD+BR4MbijcEjSSCPYhElh4HgURtbgSFSo8RDBuOQotiqGcCBAIEBQIVgWkhgVkzGiSDO1ARFIxyFxWOD0ABAQEwAYYpikWBEAEB X-IronPort-AV: E=Sophos;i="5.69,275,1571695200"; d="scan'208";a="98933314" Received: from fgdemucivp01ltm.xch.fraunhofer.de (HELO FGDEMUCIMP12EXC.ads.fraunhofer.de) ([192.88.97.16]) by mail-mtaS26.fraunhofer.de with ESMTP/TLS/AES256-SHA; 03 Dec 2019 22:40:22 +0100 Received: from xch-onprem-07.ads.fraunhofer.de (10.225.16.45) by FGDEMUCIMP12EXC.ads.fraunhofer.de (10.80.232.43) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 3 Dec 2019 22:40:22 +0100 Received: from localhost.localdomain (10.225.16.62) by xch-onprem-07.ads.fraunhofer.de (10.225.16.45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1779.2; Tue, 3 Dec 2019 22:40:21 +0100 From: Lukas Auer To: Subject: [PATCH 3/4] riscv: add option to wait for ack from secondary harts in smp functions Date: Tue, 3 Dec 2019 22:39:55 +0100 Message-ID: <20191203213956.24339-4-lukas.auer@aisec.fraunhofer.de> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191203213956.24339-1-lukas.auer@aisec.fraunhofer.de> References: <20191203213956.24339-1-lukas.auer@aisec.fraunhofer.de> MIME-Version: 1.0 X-Originating-IP: [10.225.16.62] X-ClientProxiedBy: xch-onprem-05.ads.fraunhofer.de (10.225.16.54) To xch-onprem-07.ads.fraunhofer.de (10.225.16.45) X-TM-AS-Product-Ver: SMEX-11.0.0.4179-8.200.1013-25080.006 X-TM-AS-Result: No-2.256400-8.000000-31 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.26 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marcus Comstedt Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.101.4 at mail.denx.de X-Virus-Status: Clean Add a wait option to smp_call_function() to wait for the secondary harts to acknowledge the call-function request. The request is considered to be acknowledged once each secondary hart has cleared the corresponding IPI. As part of the call-function request, the secondary harts invalidate the instruction cache after clearing the IPI. This adds a delay between acknowledgment (clear IPI) and fulfillment (call function) of the request. We want to use the acknowledgment to be able to judge when the request has been completed. Remove the delay by clearing the IPI after cache invalidation and just before calling the function from the request. Signed-off-by: Lukas Auer Reviewed-by: Rick Chen Tested-by: Rick Chen Reviewed-by: Anup Patel --- arch/riscv/cpu/start.S | 2 ++ arch/riscv/include/asm/smp.h | 3 ++- arch/riscv/lib/bootm.c | 2 +- arch/riscv/lib/smp.c | 31 ++++++++++++++++++++++--------- arch/riscv/lib/spl.c | 2 +- common/spl/spl_opensbi.c | 2 +- 6 files changed, 29 insertions(+), 13 deletions(-) diff --git a/arch/riscv/cpu/start.S b/arch/riscv/cpu/start.S index 0a2ce6d691..60631638dd 100644 --- a/arch/riscv/cpu/start.S +++ b/arch/riscv/cpu/start.S @@ -197,6 +197,7 @@ spl_secondary_hart_stack_gd_setup: la a0, secondary_hart_relocate mv a1, s0 mv a2, s0 + mv a3, zero jal smp_call_function /* hang if relocation of secondary harts has failed */ @@ -337,6 +338,7 @@ relocate_secondary_harts: mv a1, s2 mv a2, s3 + mv a3, zero jal smp_call_function /* hang if relocation of secondary harts has failed */ diff --git a/arch/riscv/include/asm/smp.h b/arch/riscv/include/asm/smp.h index bc863fdbaf..74de92ed13 100644 --- a/arch/riscv/include/asm/smp.h +++ b/arch/riscv/include/asm/smp.h @@ -46,8 +46,9 @@ void handle_ipi(ulong hart); * @addr: Address of function * @arg0: First argument of function * @arg1: Second argument of function + * @wait: Wait for harts to acknowledge request * @return 0 if OK, -ve on error */ -int smp_call_function(ulong addr, ulong arg0, ulong arg1); +int smp_call_function(ulong addr, ulong arg0, ulong arg1, int wait); #endif diff --git a/arch/riscv/lib/bootm.c b/arch/riscv/lib/bootm.c index efbd3e23e7..e96137a50c 100644 --- a/arch/riscv/lib/bootm.c +++ b/arch/riscv/lib/bootm.c @@ -99,7 +99,7 @@ static void boot_jump_linux(bootm_headers_t *images, int flag) if (IMAGE_ENABLE_OF_LIBFDT && images->ft_len) { #ifdef CONFIG_SMP ret = smp_call_function(images->ep, - (ulong)images->ft_addr, 0); + (ulong)images->ft_addr, 0, 0); if (ret) hang(); #endif diff --git a/arch/riscv/lib/smp.c b/arch/riscv/lib/smp.c index 6ff0de4b74..d575e904c2 100644 --- a/arch/riscv/lib/smp.c +++ b/arch/riscv/lib/smp.c @@ -43,11 +43,11 @@ extern int riscv_clear_ipi(int hart); */ extern int riscv_get_ipi(int hart, int *pending); -static int send_ipi_many(struct ipi_data *ipi) +static int send_ipi_many(struct ipi_data *ipi, int wait) { ofnode node, cpus; u32 reg; - int ret; + int ret, pending; cpus = ofnode_path("/cpus"); if (!ofnode_valid(cpus)) { @@ -90,6 +90,15 @@ static int send_ipi_many(struct ipi_data *ipi) pr_err("Cannot send IPI to hart %d\n", reg); return ret; } + + if (wait) { + pending = 1; + while (pending) { + ret = riscv_get_ipi(reg, &pending); + if (ret) + return ret; + } + } } return 0; @@ -103,21 +112,25 @@ void handle_ipi(ulong hart) if (hart >= CONFIG_NR_CPUS) return; + __smp_mb(); + + smp_function = (void (*)(ulong, ulong, ulong))gd->arch.ipi[hart].addr; + invalidate_icache_all(); + + /* + * Clear the IPI to acknowledge the request before jumping to the + * requested function. + */ ret = riscv_clear_ipi(hart); if (ret) { pr_err("Cannot clear IPI of hart %ld\n", hart); return; } - __smp_mb(); - - smp_function = (void (*)(ulong, ulong, ulong))gd->arch.ipi[hart].addr; - invalidate_icache_all(); - smp_function(hart, gd->arch.ipi[hart].arg0, gd->arch.ipi[hart].arg1); } -int smp_call_function(ulong addr, ulong arg0, ulong arg1) +int smp_call_function(ulong addr, ulong arg0, ulong arg1, int wait) { int ret = 0; struct ipi_data ipi; @@ -126,7 +139,7 @@ int smp_call_function(ulong addr, ulong arg0, ulong arg1) ipi.arg0 = arg0; ipi.arg1 = arg1; - ret = send_ipi_many(&ipi); + ret = send_ipi_many(&ipi, wait); return ret; } diff --git a/arch/riscv/lib/spl.c b/arch/riscv/lib/spl.c index bea8695987..fd3c2efc5b 100644 --- a/arch/riscv/lib/spl.c +++ b/arch/riscv/lib/spl.c @@ -40,7 +40,7 @@ void __noreturn jump_to_image_no_args(struct spl_image_info *spl_image) debug("image entry point: 0x%lX\n", spl_image->entry_point); #ifdef CONFIG_SMP - ret = smp_call_function(spl_image->entry_point, (ulong)fdt_blob, 0); + ret = smp_call_function(spl_image->entry_point, (ulong)fdt_blob, 0, 0); if (ret) hang(); #endif diff --git a/common/spl/spl_opensbi.c b/common/spl/spl_opensbi.c index 79ee7edcf9..91a411a3db 100644 --- a/common/spl/spl_opensbi.c +++ b/common/spl/spl_opensbi.c @@ -77,7 +77,7 @@ void spl_invoke_opensbi(struct spl_image_info *spl_image) #ifdef CONFIG_SMP ret = smp_call_function((ulong)spl_image->entry_point, (ulong)spl_image->fdt_addr, - (ulong)&opensbi_info); + (ulong)&opensbi_info, 0); if (ret) hang(); #endif From patchwork Tue Dec 3 21:39:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukas Auer X-Patchwork-Id: 1203853 X-Patchwork-Delegate: uboot@andestech.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=aisec.fraunhofer.de Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 47SG4D20QJz9s4Y for ; Wed, 4 Dec 2019 08:55:08 +1100 (AEDT) Received: by phobos.denx.de (Postfix, from userid 109) id 88FD4816F8; Tue, 3 Dec 2019 22:54:11 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mail.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from phobos.denx.de (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id C998781678; Tue, 3 Dec 2019 22:41:43 +0100 (CET) Authentication-Results: mail.denx.de; dmarc=none (p=none dis=none) header.from=aisec.fraunhofer.de Authentication-Results: mail.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Received: by phobos.denx.de (Postfix, from userid 109) id 427BF80668; Tue, 3 Dec 2019 22:40:29 +0100 (CET) Received: from mail-edgeKA27.fraunhofer.de (mail-edgeka27.fraunhofer.de [153.96.1.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id EFC908089E for ; Tue, 3 Dec 2019 22:40:24 +0100 (CET) Authentication-Results: mail.denx.de; dmarc=none (p=none dis=none) header.from=aisec.fraunhofer.de Authentication-Results: mail.denx.de; spf=none smtp.mailfrom=lukas.auer@aisec.fraunhofer.de X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2FJAwCT1eZd/xoHYZllHAEBAQEBBwEBEQEEBAEBgX6CIIFBAR88Co0ko0sJAQEBAQEBAQEBBwEvAQGEQAKCDiQ4EwIQAQEFAQEBAQEFBAICaYVDhigGJwsBRhBRVwYOBYMigncBswsziHuBSAkBgSyHPYRZD4FMP4FHgxuEEoEEhSEEjQqCV55iB4FEbW4EhUqPEQwbjkKLYqhnAgQCBAUCFYFpI4FYMxokgztQERSMchcVjg9AAQExAY89gTGBEAEB X-IPAS-Result: A2FJAwCT1eZd/xoHYZllHAEBAQEBBwEBEQEEBAEBgX6CIIFBAR88Co0ko0sJAQEBAQEBAQEBBwEvAQGEQAKCDiQ4EwIQAQEFAQEBAQEFBAICaYVDhigGJwsBRhBRVwYOBYMigncBswsziHuBSAkBgSyHPYRZD4FMP4FHgxuEEoEEhSEEjQqCV55iB4FEbW4EhUqPEQwbjkKLYqhnAgQCBAUCFYFpI4FYMxokgztQERSMchcVjg9AAQExAY89gTGBEAEB X-IronPort-AV: E=Sophos;i="5.69,275,1571695200"; d="scan'208";a="18789690" Received: from mail-mtas26.fraunhofer.de ([153.97.7.26]) by mail-edgeKA27.fraunhofer.de with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Dec 2019 22:40:24 +0100 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0ATAgCT1eZdfRBhWMBlHAEBAQEBBwEBEQEEBAEBgX6DYAEBHzwKjSSjSwkBAwEBAQEBBwEvAQGEQAKCMTgTAhABAQQBAQECAQUEFAEBFjqFSoVTAwMnCwFGEFFXBg4FgyKCeLMLM4h7gUgJAYEshz2EaIFMP4FHgxuEEoEEhSEEjQqCV55iB4FEbW4EhUqPEQwbjkKLYqhnAgQCBAUCFYFpIYFZMxokgztQERSMchcVjg9AAQEBMAGPPYExgRABAQ X-IronPort-AV: E=Sophos;i="5.69,275,1571695200"; d="scan'208";a="98933315" Received: from fgdemucivp01ltm.xch.fraunhofer.de (HELO FGDEMUCIMP12EXC.ads.fraunhofer.de) ([192.88.97.16]) by mail-mtaS26.fraunhofer.de with ESMTP/TLS/AES256-SHA; 03 Dec 2019 22:40:23 +0100 Received: from xch-onprem-07.ads.fraunhofer.de (10.225.16.45) by FGDEMUCIMP12EXC.ads.fraunhofer.de (10.80.232.43) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 3 Dec 2019 22:40:22 +0100 Received: from localhost.localdomain (10.225.16.62) by xch-onprem-07.ads.fraunhofer.de (10.225.16.45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1779.2; Tue, 3 Dec 2019 22:40:22 +0100 From: Lukas Auer To: Subject: [PATCH 4/4] spl: opensbi: wait for ack from secondary harts before entering OpenSBI Date: Tue, 3 Dec 2019 22:39:56 +0100 Message-ID: <20191203213956.24339-5-lukas.auer@aisec.fraunhofer.de> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191203213956.24339-1-lukas.auer@aisec.fraunhofer.de> References: <20191203213956.24339-1-lukas.auer@aisec.fraunhofer.de> MIME-Version: 1.0 X-Originating-IP: [10.225.16.62] X-ClientProxiedBy: xch-onprem-05.ads.fraunhofer.de (10.225.16.54) To xch-onprem-07.ads.fraunhofer.de (10.225.16.45) X-TM-AS-Product-Ver: SMEX-11.0.0.4179-8.200.1013-25080.006 X-TM-AS-Result: No--3.774700-8.000000-31 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.26 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.101.4 at mail.denx.de X-Virus-Status: Clean At the start, OpenSBI relocates itself to its link address. If the link address ranges of U-Boot SPL and OpenSBI overlap, the relocation can lead to code corruption if a hart is still running U-Boot SPL during relocation. To avoid this problem, the main hart is specified as the preferred boot hart to perform the relocation. This fixes the code corruption problems based on the assumption that since the main hart schedules the secondary harts to enter OpenSBI, it will be the last to enter OpenSBI. However it was reported that this assumption is not always correct. To make sure the assumption always holds true, wait for all secondary harts to acknowledge the call-function request before entering OpenSBI on the main hart. Reported-by: Rick Chen Signed-off-by: Lukas Auer Reviewed-by: Rick Chen Tested-by: Rick Chen Reviewed-by: Anup Patel --- common/spl/spl_opensbi.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/common/spl/spl_opensbi.c b/common/spl/spl_opensbi.c index 91a411a3db..5ea59d423f 100644 --- a/common/spl/spl_opensbi.c +++ b/common/spl/spl_opensbi.c @@ -75,9 +75,19 @@ void spl_invoke_opensbi(struct spl_image_info *spl_image) invalidate_icache_all(); #ifdef CONFIG_SMP + /* + * Start OpenSBI on all secondary harts and wait for acknowledgment. + * + * OpenSBI first relocates itself to its link address. This is done by + * the main hart. To make sure no hart is still running U-Boot SPL + * during relocation, we wait for all secondary harts to acknowledge + * the call-function request before entering OpenSBI on the main hart. + * Otherwise, code corruption can occur if the link address ranges of + * U-Boot SPL and OpenSBI overlap. + */ ret = smp_call_function((ulong)spl_image->entry_point, (ulong)spl_image->fdt_addr, - (ulong)&opensbi_info, 0); + (ulong)&opensbi_info, 1); if (ret) hang(); #endif