From patchwork Fri Apr 26 10:08:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 1928103 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=bombadil.20210309 header.b=RY7SQHsN; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ventanamicro.com header.i=@ventanamicro.com header.a=rsa-sha256 header.s=google header.b=kDMXuvqc; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.infradead.org (client-ip=2607:7c80:54:3::133; helo=bombadil.infradead.org; envelope-from=kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org; receiver=patchwork.ozlabs.org) Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VQpMk0c1Dz23ts for ; Fri, 26 Apr 2024 20:08:31 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=pX32g9yWqhYNk8I7mKGMyoB3V4TLLAT/9OTFIyIssII=; b=RY7SQHsNk57seO ENZDVvIok9eliLtoL/WHQbpC0iqMFFqlmccOhtV70dpEWd46ty4ruyd89XxaeL4eDEvZPNz7RTbk+ HNSfIkIhNu5llFgjJ3BuYpiiFFWLE2MVaR285Fzl12+CpAeaT3f4FIAZPlUvX1ylNsO/mU+NcfL4U LNb7xedzG154g3vmy06WQpdfM4mnKdpscps0ErItWLYoxbgTr6iHvL8R/yA1MtSCKToQbwfbX4sDZ IE/9LJrWX21300PiwpkOByP+yBdoIxhV9OJKKWTn1xPC32HvUSLOVO5I8V7UzRcf4taJV6ijpDgtq XZ/IG3uGSWA5oGouUgLQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s0IVJ-0000000C3pX-1ZuM; Fri, 26 Apr 2024 10:08:29 +0000 Received: from mail-wr1-x433.google.com ([2a00:1450:4864:20::433]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s0IVD-0000000C3j4-1ZCo for kvm-riscv@lists.infradead.org; Fri, 26 Apr 2024 10:08:26 +0000 Received: by mail-wr1-x433.google.com with SMTP id ffacd0b85a97d-344047ac7e4so1731385f8f.0 for ; Fri, 26 Apr 2024 03:08:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; t=1714126101; x=1714730901; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=hbx1On+TTYqeoURSkZkCrdGwttFNEWRGbinjc3HE+Hk=; b=kDMXuvqcsoMIIjrDlPVQAU3aAbyjio8vrqOhzyiXCtIfcxtlPQGCIBc9nfMV59alSG LxME5tmAJoTB9pFnkKPMxWw4buQ6GSx1+TtD5moqeFw5hUcqAypAuWb3bKy0I4v84fJH PQmbHOylGu+LbMfWNGuSvEOaP6e77vaIqd2qhnoo61eaDVZVNtTLqkXzFCYYuEKOk9/k CByd7T1feRUqGiGk0BBGC5mDWip4WF+0YmaRwGYW3aRoi/ekr51xoocE4naDiF35qpkj 8QJW0fchDLUBOZ0kiAVxvtd1Zxig9cj1IU1Gk9c1Sxfix3euR0NvrnUly1Sb+/rHnnqw Ks0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714126101; x=1714730901; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=hbx1On+TTYqeoURSkZkCrdGwttFNEWRGbinjc3HE+Hk=; b=lH4OMZiVk3XoAOD7qY8gxyw9FjwRO05/BVRa70zJlEjkfzrsUKNNZhxN+FaU6VU0yx EXim8fcLt68R6B7qkMIuv7GqM3SPz51m3Ix/6uTDdpgrV/fnmzgP1HqfCeDY9YCqBoBf 5doyuTOlEnpmr2UHIkU1F8j2nRtR8g3Mz+xhUsJ/3o42lsCoq6DCI8Zq5q23G8wC1dF9 5Crg8+RKCom9A90WaMWzgA1tm8HT7CyzlpudfQ7xf/JLjHCYwVWoBV1xN1mXnK7grCJU ELeOXu37x0wMh8mHB4Kx1tHRSMHMEAz0zr0pqlJl+Oa7Ohj+tNuwh85tvi2WHCopBEmY WDoA== X-Forwarded-Encrypted: i=1; AJvYcCX1pHIwT+DOLvmHHQ3cIUpDi/Tj/slpY03WhZvjO8ISbmRp9bOZlowuBmLyW2ydxNZtgS84Auw7uIYfzPhN0F1kSlpLOfoAC9JG6lcnZw== X-Gm-Message-State: AOJu0Yw/XOsvunPOxawfrq555MKyMr+H13REQK4NtSF8ndC0GO2WPBEP I7izhShDPCstR0on3Z+O5ZUyAQRfatuWQ0RS2T3scKqr6YFeYNLyGUVXpanpn0w= X-Google-Smtp-Source: AGHT+IFq1PBLGeh1M4r+k/wSmDqxEHo8VdDIM9MYkH3T6k1cJcTZz/q7QYNfgMeHr+ScXMDq7gOpIw== X-Received: by 2002:a5d:6b0d:0:b0:34c:4c67:c798 with SMTP id v13-20020a5d6b0d000000b0034c4c67c798mr1551849wrw.4.1714126101328; Fri, 26 Apr 2024 03:08:21 -0700 (PDT) Received: from localhost (2001-1ae9-1c2-4c00-20f-c6b4-1e57-7965.ip6.tmcz.cz. [2001:1ae9:1c2:4c00:20f:c6b4:1e57:7965]) by smtp.gmail.com with ESMTPSA id j13-20020a056000124d00b0034b7906c716sm8823766wrx.106.2024.04.26.03.08.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Apr 2024 03:08:20 -0700 (PDT) From: Andrew Jones To: linux-riscv@lists.infradead.org, kvm-riscv@lists.infradead.org, devicetree@vger.kernel.org Cc: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, conor.dooley@microchip.com, anup@brainfault.org, atishp@atishpatra.org, robh@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, christoph.muellner@vrull.eu, heiko@sntech.de, charlie@rivosinc.com, David.Laight@ACULAB.COM, parri.andrea@gmail.com, luxu.kernel@bytedance.com Subject: [PATCH v3 0/6] riscv: Apply Zawrs when available Date: Fri, 26 Apr 2024 12:08:20 +0200 Message-ID: <20240426100820.14762-8-ajones@ventanamicro.com> X-Mailer: git-send-email 2.44.0 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240426_030823_466407_F417A8D9 X-CRM114-Status: GOOD ( 23.01 ) X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "bombadil.infradead.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Zawrs provides two instructions (wrs.nto and wrs.sto), where both are meant to allow the hart to enter a low-power state while waiting on a store to a memory location. The instructions also both wait [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:433 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.0 PP_MIME_FAKE_ASCII_TEXT BODY: MIME text/plain claims to be ASCII but isn't X-BeenThere: kvm-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "kvm-riscv" Errors-To: kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org Zawrs provides two instructions (wrs.nto and wrs.sto), where both are meant to allow the hart to enter a low-power state while waiting on a store to a memory location. The instructions also both wait an implementation-defined "short" duration (unless the implementation terminates the stall for another reason). The difference is that while wrs.sto will terminate when the duration elapses, wrs.nto, depending on configuration, will either just keep waiting or an ILL exception will be raised. Linux will use wrs.nto, so if platforms have an implementation which falls in the "just keep waiting" category (which is not expected), then it should _not_ advertise Zawrs in the hardware description. Like wfi (and with the same {m,h}status bits to configure it), when wrs.nto is configured to raise exceptions it's expected that the higher privilege level will see the instruction was a wait instruction, do something, and then resume execution following the instruction. For example, KVM does configure exceptions for wfi (hstatus.VTW=1) and therefore also for wrs.nto. KVM does this for wfi since it's better to allow other tasks to be scheduled while a VCPU waits for an interrupt. For waits such as those where wrs.nto/sto would be used, which are typically locks, it is also a good idea for KVM to be involved, as it can attempt to schedule the lock holding VCPU. This series starts with Christoph's addition of the riscv smp_cond_load_relaxed function which applies wrs.sto when available. That patch has been reworked to use wrs.nto and to use the same approach as Arm for the wait loop, since we can't have arbitrary C code between the load-reserved and the wrs. Then, hwprobe support is added (since the instructions are also usable from usermode), and finally KVM is taught about wrs.nto, allowing guests to see and use the Zawrs extension. We still don't have test results from hardware, and it's not possible to prove that using Zawrs is a win when testing on QEMU, not even when oversubscribing VCPUs to guests. However, it is possible to use KVM selftests to force a scenario where we can prove Zawrs does its job and does it well. [4] is a test which does this and, on my machine, without Zawrs it takes 16 seconds to complete and with Zawrs it takes 0.25 seconds. This series is also available here [1]. In order to use QEMU for testing a build with [2] is needed. In order to enable guests to use Zawrs with KVM using kvmtool, the branch at [3] may be used. [1] https://github.com/jones-drew/linux/commits/riscv/zawrs-v3/ [2] https://lore.kernel.org/all/20240312152901.512001-2-ajones@ventanamicro.com/ [3] https://github.com/jones-drew/kvmtool/commits/riscv/zawrs/ [4] https://github.com/jones-drew/linux/commit/cb2beccebcece10881db842ed69bdd5715cfab5d Thanks, drew v3: - Moved comment about expected termination from the DT binding text to a code comment. v2: - Added DT bindings patch with additional Linux specifications due to wrs.nto potentially never terminating, as suggested by Palmer - Added patch to share pause insn definition - Rework main Zawrs support patch to use Arm approach (which is also the approach that Andrea Parri suggested) - Dropped the riscv implementation of smp_cond_load_acquire(). afaict, the generic implementation, which will use the riscv implementation of smp_cond_load_relaxed() is sufficient for riscv. - The rework was large enough (IMO) to drop Heiko's s-o-b and to add myself as a co-developer Andrew Jones (5): riscv: Provide a definition for 'pause' dt-bindings: riscv: Add Zawrs ISA extension description riscv: hwprobe: export Zawrs ISA extension KVM: riscv: Support guest wrs.nto KVM: riscv: selftests: Add Zawrs extension to get-reg-list test Christoph Müllner (1): riscv: Add Zawrs support for spinlocks Documentation/arch/riscv/hwprobe.rst | 4 ++ .../devicetree/bindings/riscv/extensions.yaml | 7 +++ arch/riscv/Kconfig | 20 ++++--- arch/riscv/Makefile | 3 - arch/riscv/include/asm/barrier.h | 45 +++++++++----- arch/riscv/include/asm/cmpxchg.h | 58 +++++++++++++++++++ arch/riscv/include/asm/hwcap.h | 1 + arch/riscv/include/asm/insn-def.h | 4 ++ arch/riscv/include/asm/kvm_host.h | 1 + arch/riscv/include/asm/vdso/processor.h | 8 +-- arch/riscv/include/uapi/asm/hwprobe.h | 1 + arch/riscv/include/uapi/asm/kvm.h | 1 + arch/riscv/kernel/cpufeature.c | 1 + arch/riscv/kernel/sys_hwprobe.c | 1 + arch/riscv/kvm/vcpu.c | 1 + arch/riscv/kvm/vcpu_insn.c | 15 +++++ arch/riscv/kvm/vcpu_onereg.c | 2 + .../selftests/kvm/riscv/get-reg-list.c | 4 ++ 18 files changed, 146 insertions(+), 31 deletions(-)