From patchwork Wed Oct 23 13:21:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 2001082 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=bombadil.20210309 header.b=2ke8yvze; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linux.dev header.i=@linux.dev header.a=rsa-sha256 header.s=key1 header.b=Px1Hzl1g; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.infradead.org (client-ip=2607:7c80:54:3::133; helo=bombadil.infradead.org; envelope-from=kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org; receiver=patchwork.ozlabs.org) Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XYV8Y5klMz1xx0 for ; Thu, 24 Oct 2024 00:22:37 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=QEOJfEN8oOttOWQjqOQGC2vjGOc3MsDMyQAsUIb7lm4=; b=2ke8yvzewDyvyR Q2PIwCVmpYb+/DkZ9js39kg46u82WSdGXOoE+AZ3upKI66FUqP7lvEiBWc/4HAqLGtWvwPjG3HiOg kmgS6aOxZ0lJcCXfyhEji6VYAx6VChTTUq+ZpY5DTWACBRIuUcO+nyjQEb6izJS/BBpVN6Jn//NVW AJbJHDT6ls/t2I8Ln3k8CN33Sk5m8myf6lpzcdC52wjPGhXCFwmiGPBsxzOTtplNUSiuzBj3JesWF 26wXLvfb9PVX2rbW1ikyBjZGFl6wdS1V8Soligz+x+TH2zlAL5vzLeXEU32o7NryCV5solqPylFIa 29vF7Y2rtBzxJqNYhXQA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t3bJs-0000000EWfi-2LHk; Wed, 23 Oct 2024 13:22:36 +0000 Received: from out-178.mta0.migadu.com ([91.218.175.178]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1t3bIz-0000000EWTi-3nFO for kvm-riscv@lists.infradead.org; Wed, 23 Oct 2024 13:21:43 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1729689699; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cgbOwzQMc+kT3Z3Q0ZRUFULWs2irjvHdzSBuM71eWkA=; b=Px1Hzl1g/+LfykaXLvfKi/Q5H4a9cxu4Y09Q8bOiHxdQLRulSY0CEfWMWiCgDwsY86tLX+ Xb4hLvZViQVV8G22QsYzbBEJIzUWGvi4ldLpc9HHBJQSR11Oq9c/zMiaASLUFShQ9Jo8S9 MoP4nX+6zLozg9m+Q6gywE7JM9eqKzg= From: Andrew Jones To: kvm@vger.kernel.org, kvm-riscv@lists.infradead.org Cc: atishp@rivosinc.com, jamestiotio@gmail.com Subject: [kvm-unit-tests PATCH 1/4] riscv: Bump NR_CPUS to 256 Date: Wed, 23 Oct 2024 15:21:32 +0200 Message-ID: <20241023132130.118073-7-andrew.jones@linux.dev> In-Reply-To: <20241023132130.118073-6-andrew.jones@linux.dev> References: <20241023132130.118073-6-andrew.jones@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241023_062142_111810_98DB5D1F X-CRM114-Status: GOOD ( 10.53 ) X-Spam-Score: -2.8 (--) X-Spam-Report: Spam detection software, running on the system "bombadil.infradead.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Besides a bit more memory used for the .bss section, where there are NR_CPUS sized arrays, and a tiny bit more stack used by functions with cpumasks on their stacks, then there's no harm in bumping NR [...] Content analysis details: (-2.8 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [91.218.175.178 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] 0.0 RCVD_IN_VALIDITY_CERTIFIED_BLOCKED RBL: ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. [91.218.175.178 listed in sa-trusted.bondedsender.org] 0.0 RCVD_IN_VALIDITY_SAFE_BLOCKED RBL: ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. [91.218.175.178 listed in sa-accredit.habeas.com] 0.0 RCVD_IN_VALIDITY_RPBL_BLOCKED RBL: ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. [91.218.175.178 listed in bl.score.senderscore.com] X-BeenThere: kvm-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "kvm-riscv" Errors-To: kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org Besides a bit more memory used for the .bss section, where there are NR_CPUS sized arrays, and a tiny bit more stack used by functions with cpumasks on their stacks, then there's no harm in bumping NR_CPUS. Bump it to 256, which should cover us for quite a while. Signed-off-by: Andrew Jones --- lib/riscv/asm/setup.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/riscv/asm/setup.h b/lib/riscv/asm/setup.h index a13159bfe395..a031ebe7d762 100644 --- a/lib/riscv/asm/setup.h +++ b/lib/riscv/asm/setup.h @@ -4,7 +4,7 @@ #include #include -#define NR_CPUS 16 +#define NR_CPUS 256 extern struct thread_info cpus[NR_CPUS]; extern int nr_cpus; extern uint64_t timebase_frequency; From patchwork Wed Oct 23 13:21:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 2001083 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=bombadil.20210309 header.b=B6JB6iJe; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linux.dev header.i=@linux.dev header.a=rsa-sha256 header.s=key1 header.b=FdSutKbr; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.infradead.org (client-ip=2607:7c80:54:3::133; helo=bombadil.infradead.org; envelope-from=kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org; receiver=patchwork.ozlabs.org) Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XYV8Z4BY7z1xwf for ; Thu, 24 Oct 2024 00:22:38 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Y95mlLUDuwnTUO1ZFWa7oCZK/WsSyI9VDCVQhqSXvQc=; b=B6JB6iJeZO0IAd s5b8SPd6cn9kHsePeQ7sHlxOVA6o4fotENxNUrxKWKYzZ48cWA+FkuRF+vWFIwD+RvAiKA6ruDCde jRvHahH5ZxwcYhiYFXms/Y6l21e1c6cd6mJrPVrkPsovd1NpY8Dh+fpXuuSswHK4xerqVgh1BBytD ZlkDYkanth/Aqu/ZEHEuxsKPCBUxx5UMtLfQKXtw9b54DC/GPZzTgIhlCrVxCPWdLG+R3bmPT3dUJ jcb9vQwAV1dS7naN+65459hr1Mw2MqL+cvShY/9TEFr2p7UT/CPAKsHRXl+UyhMkc9wLaFbEskaTz mf2GzQ3CM06RISUEta7Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t3bJt-0000000EWg8-05G6; Wed, 23 Oct 2024 13:22:37 +0000 Received: from out-184.mta0.migadu.com ([2001:41d0:1004:224b::b8]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1t3bJ3-0000000EWUO-0tzg for kvm-riscv@lists.infradead.org; Wed, 23 Oct 2024 13:21:46 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1729689702; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=v4n6iXUzuE7MoPSEF0ddVeFcFHlzBfHTUGSIQ/G9qu4=; b=FdSutKbr+n7M6TgBthyAaF8qcxESYQIgxvYeGZ+VnbE5FGkNMTj03ppIQK0dBEhU0SJX8c GcO3or5UYcOlpvUf9pjn4G14pI3zsdoptmz/+ffm7JvRD46wRwZaEflil76u3t8Ru/4uV4 WzTzSm5NZStcGTGjB2xES9P1eBCwaYU= From: Andrew Jones To: kvm@vger.kernel.org, kvm-riscv@lists.infradead.org Cc: atishp@rivosinc.com, jamestiotio@gmail.com Subject: [kvm-unit-tests PATCH 2/4] riscv: Filter unmanaged harts from present mask Date: Wed, 23 Oct 2024 15:21:33 +0200 Message-ID: <20241023132130.118073-8-andrew.jones@linux.dev> In-Reply-To: <20241023132130.118073-6-andrew.jones@linux.dev> References: <20241023132130.118073-6-andrew.jones@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241023_062145_410378_510F087F X-CRM114-Status: UNSURE ( 9.49 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -2.1 (--) X-Spam-Report: Spam detection software, running on the system "bombadil.infradead.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: We use SBI to manage harts and SBI may have a different idea of which harts it should manage than our hardware description. Filter out all harts which fail an SBI HSM status call from the present mask [...] Content analysis details: (-2.1 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-BeenThere: kvm-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "kvm-riscv" Errors-To: kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org We use SBI to manage harts and SBI may have a different idea of which harts it should manage than our hardware description. Filter out all harts which fail an SBI HSM status call from the present mask to ensure we don't try to use them. Signed-off-by: Andrew Jones --- lib/riscv/setup.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/lib/riscv/setup.c b/lib/riscv/setup.c index f347ad6352d7..211945448b0f 100644 --- a/lib/riscv/setup.c +++ b/lib/riscv/setup.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include @@ -51,7 +52,9 @@ static void cpu_set_fdt(int fdtnode __unused, u64 regval, void *info __unused) cpus[cpu].cpu = cpu; cpus[cpu].hartid = regval; - set_cpu_present(cpu, true); + + if (!sbi_hart_get_status(cpus[cpu].hartid).error) + set_cpu_present(cpu, true); } static void cpu_init_acpi(void) @@ -61,7 +64,7 @@ static void cpu_init_acpi(void) static void cpu_init(void) { - int ret; + int ret, me; nr_cpus = 0; if (dt_available()) { @@ -71,7 +74,9 @@ static void cpu_init(void) cpu_init_acpi(); } - set_cpu_online(hartid_to_cpu(csr_read(CSR_SSCRATCH)), true); + me = hartid_to_cpu(csr_read(CSR_SSCRATCH)); + assert(cpu_present(me)); + set_cpu_online(me, true); cpu0_calls_idle = true; } From patchwork Wed Oct 23 13:21:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 2001084 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=bombadil.20210309 header.b=wA3+KD2S; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linux.dev header.i=@linux.dev header.a=rsa-sha256 header.s=key1 header.b=puXWbn4x; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.infradead.org (client-ip=2607:7c80:54:3::133; helo=bombadil.infradead.org; envelope-from=kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org; receiver=patchwork.ozlabs.org) Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XYV8Z59lHz1xx0 for ; Thu, 24 Oct 2024 00:22:38 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=bV2zigBgf3/h94w8u2TYGp/yWHip0EfF1Fr9H3M2KBk=; b=wA3+KD2SqJFm2k C1vnmqWBObNojlEA/xhog+uC35ke4BnMxXEGe9KpBmhKfeGrH1TEUY4zVqqlL/uEfPfNkAvin25gl fBhUhY85FYDD5kDZuERVniZZ6HWLAgGCQHAfI3IhzraHBnq8dSxjyq2deuiKnJeDH/T6L+wrkhWJU AuxcadfBp7qDc7tz/wyqZo4i0v31yAEy/iWXksGdXBNDS+W8oW2jJlHDAl1QFJ/mLoMV5YywXb/oN 9dAMH+I4Bc1qa3hAz1apnb28dJY+qVTIAv8HvRt4myUoAui4YnfCnTSfUHTLd1Mf5ATz3m+I7TXN6 gnrFO4wICz2bkQQRyyzw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t3bJt-0000000EWgX-1avs; Wed, 23 Oct 2024 13:22:37 +0000 Received: from out-183.mta1.migadu.com ([2001:41d0:203:375::b7]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1t3bJ5-0000000EWUl-37LG for kvm-riscv@lists.infradead.org; Wed, 23 Oct 2024 13:21:49 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1729689705; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=b5ln1rtoJpJgmzh9v/JiEP0bJ5A/bzOlNsyfFTmukE8=; b=puXWbn4xIhdKnF680eXYpH/Za5XvN8PIQP1A+r4RVPvZLOXs+bUNevUYLEIxJRdF6zb/76 YFSi7aYuhCD6TWaHY7FzEFkxgl20VPJdsL0le6JkO4osu1xFcilwK2XaN3YOjjkfoDvAat GGNVSx/KORH+vr2TCIhJL4DlzMHAP24= From: Andrew Jones To: kvm@vger.kernel.org, kvm-riscv@lists.infradead.org Cc: atishp@rivosinc.com, jamestiotio@gmail.com Subject: [kvm-unit-tests PATCH 3/4] riscv: Fix secondary_entry Date: Wed, 23 Oct 2024 15:21:34 +0200 Message-ID: <20241023132130.118073-9-andrew.jones@linux.dev> In-Reply-To: <20241023132130.118073-6-andrew.jones@linux.dev> References: <20241023132130.118073-6-andrew.jones@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241023_062147_955431_73AD8D75 X-CRM114-Status: UNSURE ( 9.64 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -2.1 (--) X-Spam-Report: Spam detection software, running on the system "bombadil.infradead.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: The last few instructions of secondary_entry had the right concept, but were the totally wrong implementation. Without setting ra, then, when the boot function doesn't stay in an infinite loop, like d [...] Content analysis details: (-2.1 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-BeenThere: kvm-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "kvm-riscv" Errors-To: kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org The last few instructions of secondary_entry had the right concept, but were the totally wrong implementation. Without setting ra, then, when the boot function doesn't stay in an infinite loop, like do_idle() would, we'd go off into the weeds when trying to return from it. Make sure we set ra to come back to where we can then call do_idle() instead. The bug was found by inspection since nobody is calling smp_boot_secondary() with anything other than do_idle() at this time. Fixes: 9c92b28e6b7b ("riscv: Add SMP support") Signed-off-by: Andrew Jones --- riscv/cstart.S | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/riscv/cstart.S b/riscv/cstart.S index bae1d2f5b4d5..687173706d83 100644 --- a/riscv/cstart.S +++ b/riscv/cstart.S @@ -154,9 +154,9 @@ secondary_entry: mv a0, sp call secondary_cinit addi sp, sp, SECONDARY_DATA_SIZE - jr a0 - la a0, do_idle - jr a0 + jalr ra, a0 + call do_idle + j . /* unreachable */ /* * Save context to address in a0. From patchwork Wed Oct 23 13:21:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 2001085 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=bombadil.20210309 header.b=Z5DDiEJ3; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linux.dev header.i=@linux.dev header.a=rsa-sha256 header.s=key1 header.b=BQBr5S2s; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.infradead.org (client-ip=2607:7c80:54:3::133; helo=bombadil.infradead.org; envelope-from=kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org; receiver=patchwork.ozlabs.org) Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XYV8b1rGHz1xx1 for ; Thu, 24 Oct 2024 00:22:39 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=bW4uRKLduGePRoO52fMj9W8C0farQnAVthlO54UrSv4=; b=Z5DDiEJ3V2V9nM tVsykD9qoZM4jX90d1iCvPNJsKMLnwdRasYP0/xp/+8XAlk9u7H1FOD/xueu70r2esJnJr9Fq32vi odej+oFN5Hruky9zJz6tWC2kixmx5WYXZGHcZysGQLuuXOPdUuBSpK9L0ltt/nzcENgjM/HbfAOyQ y66dNHzmvrVPacuaoTio7xb5lYCNFn+EWgdDaloQdqFKw+QQ+W/ba8oZvrxhpBjQyOJGk48ujcWL+ MLqGivEjU79iZP0idT8y/B02gjxPqnKOIojQm8inzitedFTUIQbEruxuGwPacbkaVO3xxv0+J2Tp9 3Wuw7dFTjw4ttxfZUqIg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t3bJt-0000000EWgo-3Q47; Wed, 23 Oct 2024 13:22:37 +0000 Received: from out-171.mta0.migadu.com ([2001:41d0:1004:224b::ab]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1t3bJA-0000000EWWX-2ABs for kvm-riscv@lists.infradead.org; Wed, 23 Oct 2024 13:21:54 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1729689709; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Nh3KjWy9iMLDSdx8xjwI08CQw6sxDmd7hGwJRrvc7UY=; b=BQBr5S2scTAX50tMufJcEaRHKgZpnuxvgRQu9C1Jd1mV8mWocl9ZFdfJmp/IyoVLry3WUp ULv8gkcqYuPVLaCrF4SAGTTSPCmXj6GadHS9EKMk1T3TjiSXachK7D9a+CHsvYJA9clEs9 8G11q7tXV5VQmWqGBzPRjjjBKQ/JBgU= From: Andrew Jones To: kvm@vger.kernel.org, kvm-riscv@lists.infradead.org Cc: atishp@rivosinc.com, jamestiotio@gmail.com Subject: [kvm-unit-tests PATCH 4/4] riscv: Rework smp_boot_secondary Date: Wed, 23 Oct 2024 15:21:35 +0200 Message-ID: <20241023132130.118073-10-andrew.jones@linux.dev> In-Reply-To: <20241023132130.118073-6-andrew.jones@linux.dev> References: <20241023132130.118073-6-andrew.jones@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241023_062152_944061_9AE1EDF1 X-CRM114-Status: GOOD ( 12.14 ) X-Spam-Score: -2.1 (--) X-Spam-Report: Spam detection software, running on the system "bombadil.infradead.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Use HSM status to determine when a secondary should be started. Also save the stack pointer so a secondary may be stopped and started again without leaking old stacks. Signed-off-by: Andrew Jones --- lib/riscv/asm/processor.h | 1 + lib/riscv/smp.c | 49 +++++++++++++++++++++++++++ riscv/cstart.S | 1 + 3 files changed, 36 insertion [...] Content analysis details: (-2.1 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-BeenThere: kvm-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "kvm-riscv" Errors-To: kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org Use HSM status to determine when a secondary should be started. Also save the stack pointer so a secondary may be stopped and started again without leaking old stacks. Signed-off-by: Andrew Jones --- lib/riscv/asm/processor.h | 1 + lib/riscv/smp.c | 49 +++++++++++++++++++++++++++------------ riscv/cstart.S | 1 + 3 files changed, 36 insertions(+), 15 deletions(-) diff --git a/lib/riscv/asm/processor.h b/lib/riscv/asm/processor.h index 4063255a3475..401042724cee 100644 --- a/lib/riscv/asm/processor.h +++ b/lib/riscv/asm/processor.h @@ -13,6 +13,7 @@ struct thread_info { int cpu; unsigned long hartid; unsigned long isa[1]; + unsigned long sp; exception_fn exception_handlers[EXCEPTION_CAUSE_MAX]; exception_fn interrupt_handlers[INTERRUPT_CAUSE_MAX]; }; diff --git a/lib/riscv/smp.c b/lib/riscv/smp.c index eb7061abfe7f..e92f83e1310d 100644 --- a/lib/riscv/smp.c +++ b/lib/riscv/smp.c @@ -19,8 +19,6 @@ cpumask_t cpu_present_mask; cpumask_t cpu_online_mask; cpumask_t cpu_idle_mask; -static cpumask_t cpu_started; - secondary_func_t secondary_cinit(struct secondary_data *data) { struct thread_info *info; @@ -37,27 +35,40 @@ secondary_func_t secondary_cinit(struct secondary_data *data) static void __smp_boot_secondary(int cpu, secondary_func_t func) { - struct secondary_data *sp = alloc_pages(1) + SZ_8K - 16; - phys_addr_t sp_phys; + void *sp_mem = (void *)cpus[cpu].sp; + struct secondary_data *data; struct sbiret ret; - sp -= sizeof(struct secondary_data); - sp->satp = csr_read(CSR_SATP); - sp->stvec = csr_read(CSR_STVEC); - sp->func = func; + if (!sp_mem) { + phys_addr_t sp_phys; + + sp_mem = alloc_pages(1) + SZ_8K - 16; + sp_phys = virt_to_phys(sp_mem); + cpus[cpu].sp = __pa(sp_phys); - sp_phys = virt_to_phys(sp); - assert(sp_phys == __pa(sp_phys)); + assert(sp_phys == cpus[cpu].sp); + } - ret = sbi_hart_start(cpus[cpu].hartid, (unsigned long)&secondary_entry, __pa(sp_phys)); + sp_mem -= sizeof(struct secondary_data); + data = (struct secondary_data *)sp_mem; + data->satp = csr_read(CSR_SATP); + data->stvec = csr_read(CSR_STVEC); + data->func = func; + + ret = sbi_hart_start(cpus[cpu].hartid, (unsigned long)&secondary_entry, cpus[cpu].sp); assert(ret.error == SBI_SUCCESS); } void smp_boot_secondary(int cpu, void (*func)(void)) { - int ret = cpumask_test_and_set_cpu(cpu, &cpu_started); + struct sbiret ret; - assert_msg(!ret, "CPU%d already boot once", cpu); + do { + ret = sbi_hart_get_status(cpus[cpu].hartid); + assert(!ret.error); + } while (ret.value == SBI_EXT_HSM_STOP_PENDING); + + assert_msg(ret.value == SBI_EXT_HSM_STOPPED, "CPU%d is not stopped", cpu); __smp_boot_secondary(cpu, func); while (!cpu_online(cpu)) @@ -66,10 +77,18 @@ void smp_boot_secondary(int cpu, void (*func)(void)) void smp_boot_secondary_nofail(int cpu, void (*func)(void)) { - int ret = cpumask_test_and_set_cpu(cpu, &cpu_started); + struct sbiret ret; + + do { + ret = sbi_hart_get_status(cpus[cpu].hartid); + assert(!ret.error); + } while (ret.value == SBI_EXT_HSM_STOP_PENDING); - if (!ret) + if (ret.value == SBI_EXT_HSM_STOPPED) __smp_boot_secondary(cpu, func); + else + assert_msg(ret.value == SBI_EXT_HSM_START_PENDING || ret.value == SBI_EXT_HSM_STARTED, + "CPU%d is in an unexpected state %ld", cpu, ret.value); while (!cpu_online(cpu)) smp_wait_for_event(); diff --git a/riscv/cstart.S b/riscv/cstart.S index 687173706d83..b7ee9b9c96b3 100644 --- a/riscv/cstart.S +++ b/riscv/cstart.S @@ -149,6 +149,7 @@ secondary_entry: csrw CSR_SSCRATCH, a0 mv sp, a1 mv fp, zero + addi sp, sp, -SECONDARY_DATA_SIZE REG_L a0, SECONDARY_STVEC(sp) csrw CSR_STVEC, a0 mv a0, sp