From patchwork Thu Nov 30 12:41:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu-Chien Peter Lin X-Patchwork-Id: 1870105 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=bombadil.20210309 header.b=vsBwbSGV; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.infradead.org (client-ip=2607:7c80:54:3::133; helo=bombadil.infradead.org; envelope-from=opensbi-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org; receiver=patchwork.ozlabs.org) Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SgwpJ0YNKz23nw for ; Thu, 30 Nov 2023 23:43:03 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=L2Df9iCyjWU+g9fDBBa5ZIm8mqyOJ+pMx2N+ocAH8yI=; b=vsBwbSGVE661nY ogedWavenihRG6GNmUcpQm7U9zCHsKUmF8lS99I9gajIHGgTjol97IwgIhiel+BpEE/w01oo2ecNO h9Zwg65Nm5EafVKo0rkAHd8+QNxltI7+gp9esXPnm8mU5k5oSX3spr3WIUBetbh+3Bd82vpESL8E1 AAz+ff7EqGb5MEyDp2RreL6v9CZyA6MAd2OormcIhFrKUYrp91o/eXhcdDu4BSn2bksMnpsMunlRE z004GpLIba0sjOFsPXK/oQoPEKv2Zk2ZooCdDQZ/vLztIzR0to/gcg0m82CykjIlpbV1pZIC3IyQB 0/bRXB9pAyccMlno+LCg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1r8gNX-00AtDA-1k; Thu, 30 Nov 2023 12:42:51 +0000 Received: from 60-248-80-70.hinet-ip.hinet.net ([60.248.80.70] helo=Atcsqr.andestech.com) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1r8gNR-00AtAN-0l for opensbi@lists.infradead.org; Thu, 30 Nov 2023 12:42:48 +0000 Received: from mail.andestech.com (ATCPCS16.andestech.com [10.0.1.222]) by Atcsqr.andestech.com with ESMTP id 3AUCgSTa055911; Thu, 30 Nov 2023 20:42:28 +0800 (+08) (envelope-from peterlin@andestech.com) Received: from swlinux02.andestech.com (10.0.15.183) by ATCPCS16.andestech.com (10.0.1.222) with Microsoft SMTP Server id 14.3.498.0; Thu, 30 Nov 2023 20:42:24 +0800 From: Yu Chien Peter Lin To: CC: , , , , , , , , , Yu Chien Peter Lin , Randolph Subject: [PATCH v4 01/15] lib: ipi: Adjust Andes PLICSW to single-bit-per-hart scheme Date: Thu, 30 Nov 2023 20:41:59 +0800 Message-ID: <20231130124213.2590640-2-peterlin@andestech.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231130124213.2590640-1-peterlin@andestech.com> References: <20231130124213.2590640-1-peterlin@andestech.com> MIME-Version: 1.0 X-Originating-IP: [10.0.15.183] X-DNSRBL: X-SPAM-SOURCE-CHECK: pass X-MAIL: Atcsqr.andestech.com 3AUCgSTa055911 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231130_044245_760235_69F83478 X-CRM114-Status: GOOD ( 17.04 ) X-Spam-Score: 0.4 (/) X-Spam-Report: Spam detection software, running on the system "bombadil.infradead.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: From: Leo Yu-Chi Liang The old scheme doesn't allow sending hart0 self-IPI as the corresponding bit on pending register is hardwired to 0, this could lead to unhandle IPIs on SMP systems, esp. on single-core. Content analysis details: (0.4 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 TVD_RCVD_IP Message was received from an IP address -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.4 RDNS_DYNAMIC Delivered to internal network by host with dynamic-looking rDNS X-BeenThere: opensbi@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "opensbi" Errors-To: opensbi-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org From: Leo Yu-Chi Liang The old scheme doesn't allow sending hart0 self-IPI as the corresponding bit on pending register is hardwired to 0, this could lead to unhandle IPIs on SMP systems, esp. on single-core. Furthermore, the limitation of old scheme is 8-core, instead of reserving source hart information, we assign bit (x + 1) as the enable and pending bit of hartx, this also expands the bootable hart number. The following diagram shows the enable bits of the new scheme on 32-core Andes platform. Pending regs: 0x1000 x---0---0---0---0------0---0 Pending hart ID: 0 1 2 3 ... 30 31 Interrupt ID: 0 1 2 3 4 ... 31 32 | | | | | | | Enable regs: 0x2000 x---1---0---0---0-...--0---0---> hart0 | | | | | | | 0x2080 x---0---1---0---0-...--0---0---> hart1 | | | | | | | 0x2100 x---0---0---1---0-...--0---0---> hart2 | | | | | | | 0x2180 x---0---0---0---1-...--0---0---> hart3 . . . . . . . . . . . . . . . . . . . . . 0x2f00 x---0---0---0---0-...--1---0---> hart30 | | | | | | | 0x2f80 x---0---0---0---0-...--0---1---> hart31 <-------- word 0 -------><--- word 1 ---> To send IPI to hart0, for example, another hart (including hart0 itself) will set bit 1 of first word on the pending register. We also fix indentation in andes_plicsw.h along with this patch. Fixes: ce7c490719ed ("lib: utils/ipi: Add Andes fdt ipi driver support") Signed-off-by: Leo Yu-Chi Liang Reviewed-by: Yu Chien Peter Lin Reviewed-by: Randolph Reported-by: Lad Prabhakar Link: https://lists.infradead.org/pipermail/opensbi/2023-October/005665.html Reviewed-by: Lad Prabhakar Tested-by: Lad Prabhakar Reviewed-by: Anup Patel --- Changes v2 -> v3: - New patch Changes v3 -> v4: - Include Prabhakar's RB & TB tags --- include/sbi_utils/ipi/andes_plicsw.h | 23 +++--- lib/utils/ipi/andes_plicsw.c | 104 ++++++++++----------------- 2 files changed, 46 insertions(+), 81 deletions(-) diff --git a/include/sbi_utils/ipi/andes_plicsw.h b/include/sbi_utils/ipi/andes_plicsw.h index e93cda0..0d18444 100644 --- a/include/sbi_utils/ipi/andes_plicsw.h +++ b/include/sbi_utils/ipi/andes_plicsw.h @@ -13,30 +13,23 @@ #ifndef _IPI_ANDES_PLICSW_H_ #define _IPI_ANDES_PLICSW_H_ -#define PLICSW_PRIORITY_BASE 0x4 +#define PLICSW_PRIORITY_BASE 0x4 -#define PLICSW_PENDING_BASE 0x1000 -#define PLICSW_PENDING_STRIDE 0x8 +#define PLICSW_PENDING_BASE 0x1000 -#define PLICSW_ENABLE_BASE 0x2000 -#define PLICSW_ENABLE_STRIDE 0x80 +#define PLICSW_ENABLE_BASE 0x2000 +#define PLICSW_ENABLE_STRIDE 0x80 -#define PLICSW_CONTEXT_BASE 0x200000 -#define PLICSW_CONTEXT_STRIDE 0x1000 -#define PLICSW_CONTEXT_CLAIM 0x4 +#define PLICSW_CONTEXT_BASE 0x200000 +#define PLICSW_CONTEXT_STRIDE 0x1000 +#define PLICSW_CONTEXT_CLAIM 0x4 -#define PLICSW_HART_MASK 0x01010101 - -#define PLICSW_HART_MAX_NR 8 - -#define PLICSW_REGION_ALIGN 0x1000 +#define PLICSW_REGION_ALIGN 0x1000 struct plicsw_data { unsigned long addr; unsigned long size; uint32_t hart_count; - /* hart id to source id table */ - uint32_t source_id[PLICSW_HART_MAX_NR]; }; int plicsw_warm_ipi_init(void); diff --git a/lib/utils/ipi/andes_plicsw.c b/lib/utils/ipi/andes_plicsw.c index 5693efb..413ac20 100644 --- a/lib/utils/ipi/andes_plicsw.c +++ b/lib/utils/ipi/andes_plicsw.c @@ -18,77 +18,45 @@ struct plicsw_data plicsw; -static inline void plicsw_claim(void) +static void plicsw_ipi_send(u32 hart_index) { - u32 hartid = current_hartid(); + ulong pending_reg; + u32 interrupt_id, word_index, pending_bit; + u32 target_hart = sbi_hartindex_to_hartid(hart_index); - if (plicsw.hart_count <= hartid) + if (plicsw.hart_count <= target_hart) ebreak(); - plicsw.source_id[hartid] = - readl((void *)plicsw.addr + PLICSW_CONTEXT_BASE + - PLICSW_CONTEXT_CLAIM + PLICSW_CONTEXT_STRIDE * hartid); -} - -static inline void plicsw_complete(void) -{ - u32 hartid = current_hartid(); - u32 source = plicsw.source_id[hartid]; - - writel(source, (void *)plicsw.addr + PLICSW_CONTEXT_BASE + - PLICSW_CONTEXT_CLAIM + - PLICSW_CONTEXT_STRIDE * hartid); -} - -static inline void plic_sw_pending(u32 target_hart) -{ /* - * The pending array registers are w1s type. - * IPI pending array mapping as following: - * - * Pending array start address: base + 0x1000 - * --------------------------------- - * | hart3 | hart2 | hart1 | hart0 | - * --------------------------------- - * Each hartX can send IPI to another hart by setting the - * bitY to its own region (see the below). - * - * In each hartX region: - * <---------- PICSW_PENDING_STRIDE --------> - * | bit7 | ... | bit3 | bit2 | bit1 | bit0 | - * ------------------------------------------ - * The bitY of hartX region indicates that hartX sends an - * IPI to hartY. + * We assign a single bit for each hart. + * Bit 0 is hardwired to 0, thus unavailable. + * Bit(X+1) indicates that IPI is sent to hartX. */ - u32 hartid = current_hartid(); - u32 word_index = hartid / 4; - u32 per_hart_offset = PLICSW_PENDING_STRIDE * hartid; - u32 val = 1 << target_hart << per_hart_offset; + interrupt_id = target_hart + 1; + word_index = interrupt_id / 32; + pending_bit = interrupt_id % 32; + pending_reg = plicsw.addr + PLICSW_PENDING_BASE + word_index * 4; - writel(val, (void *)plicsw.addr + PLICSW_PENDING_BASE + word_index * 4); + /* Set target hart's mip.MSIP */ + writel_relaxed(BIT(pending_bit), (void *)pending_reg); } -static void plicsw_ipi_send(u32 hart_index) +static void plicsw_ipi_clear(u32 hart_index) { u32 target_hart = sbi_hartindex_to_hartid(hart_index); + ulong reg = plicsw.addr + PLICSW_CONTEXT_BASE + PLICSW_CONTEXT_CLAIM + + PLICSW_CONTEXT_STRIDE * target_hart; if (plicsw.hart_count <= target_hart) ebreak(); - /* Set PLICSW IPI */ - plic_sw_pending(target_hart); -} + /* Claim */ + u32 source = readl((void *)reg); -static void plicsw_ipi_clear(u32 hart_index) -{ - u32 target_hart = sbi_hartindex_to_hartid(hart_index); - - if (plicsw.hart_count <= target_hart) - ebreak(); + /* A successful claim will clear mip.MSIP */ - /* Clear PLICSW IPI */ - plicsw_claim(); - plicsw_complete(); + /* Complete */ + writel(source, (void *)reg); } static struct sbi_ipi_device plicsw_ipi = { @@ -110,22 +78,26 @@ int plicsw_warm_ipi_init(void) int plicsw_cold_ipi_init(struct plicsw_data *plicsw) { int rc; + u32 interrupt_id, word_index, enable_bit; + ulong enable_reg, priority_reg; /* Setup source priority */ - uint32_t *priority = (void *)plicsw->addr + PLICSW_PRIORITY_BASE; - - for (int i = 0; i < plicsw->hart_count; i++) - writel(1, &priority[i]); - - /* Setup target enable */ - uint32_t enable_mask = PLICSW_HART_MASK; + for (int i = 0; i < plicsw->hart_count; i++) { + priority_reg = plicsw->addr + PLICSW_PRIORITY_BASE + i * 4; + writel(1, (void *)priority_reg); + } + /* + * Setup enable for each hart, skip non-existent interrupt ID 0 + * which is hardwired to 0. + */ for (int i = 0; i < plicsw->hart_count; i++) { - uint32_t *enable = (void *)plicsw->addr + PLICSW_ENABLE_BASE + - PLICSW_ENABLE_STRIDE * i; - writel(enable_mask, enable); - writel(enable_mask, enable + 1); - enable_mask <<= 1; + interrupt_id = i + 1; + word_index = interrupt_id / 32; + enable_bit = interrupt_id % 32; + enable_reg = plicsw->addr + PLICSW_ENABLE_BASE + + PLICSW_ENABLE_STRIDE * i + 4 * word_index; + writel(BIT(enable_bit), (void *)enable_reg); } /* Add PLICSW region to the root domain */