From patchwork Sat Feb 17 00:57:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Atish Kumar Patra X-Patchwork-Id: 1900411 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=bombadil.20210309 header.b=Zi/teGTI; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=v+sA2JCm; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.infradead.org (client-ip=2607:7c80:54:3::133; helo=bombadil.infradead.org; envelope-from=kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org; receiver=patchwork.ozlabs.org) Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TcBtD662gz23j9 for ; Sat, 17 Feb 2024 13:03:48 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=Jb8gyPIl+dGQIO3PFJCTZvHOE4H36xH6NqE2LSfMo38=; b=Zi/teGTIeL3Q0y yUjp5/+HdWj/M/V7yFnt4rGgih90Ak/NrrT3sTcxY4M++Zx59jrHGC2TRauQXa6T6/+AzS0xN3sP2 TIGgtMbIi5jWN05PMv+g2vY2HkTaWDYszmMgLXItf7mISyL/JxevJh6eL/JLL03uamDJv/YRF1kYp HsQxJmcReHb93Q6jbsRlNVjHiDQgQzrkm616cokJJE7XAnvC+RnGfbgyeD7O1C658TM9pUR7JXXwi HK2nR1Ep7X+pfnQiJRy9h7iUJw6AvbSTMO786EjuT+Q58IQezE6pND9Xyhht3XK4oDLdszkDJ+6OE gxKnDtHAWWuurj3C7stw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rbA3P-00000004S8u-0o69; Sat, 17 Feb 2024 02:03:47 +0000 Received: from mail-oo1-xc34.google.com ([2607:f8b0:4864:20::c34]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rb91z-00000004HrL-0Cxg for kvm-riscv@lists.infradead.org; Sat, 17 Feb 2024 00:58:16 +0000 Received: by mail-oo1-xc34.google.com with SMTP id 006d021491bc7-58e256505f7so1021735eaf.3 for ; Fri, 16 Feb 2024 16:58:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1708131494; x=1708736294; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=IzVo2l1UeODXlFr9ZspobMSeOz6FqRsDzZRtwqz0aks=; b=v+sA2JCmo+ngz22Bbhg7bOZHskQscykdJcHj6r0xpfHpzfxJ6silKUk2eHAIql02qR Ucp40J2aF5Fi7YcOwHYIJlQaB7BG43ZuVtW8BKapkU8KjR6JmttBcn2Eyc+x1eMIYI6Z 9YGP7xuWY8lMw80SezQfTpnbqOvAxd8TM2H/eF1BAHpNihhUdIf5tp9HdcjoU4IQBhjn X5tlkbPN/FXq5rN/uCp8dmY9USd05wR+4rkko81kDN9Nt5fBT16LGgIA2bO2RdQc4Gvw cnQnBBNdXcf42s4+mm+u4SrqZzBsXiXjdp5HIgDIU+AdWtYREV9jYQBRujpMNXu4HPUs emnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708131494; x=1708736294; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=IzVo2l1UeODXlFr9ZspobMSeOz6FqRsDzZRtwqz0aks=; b=eHx5wTVn6To1IZo8Cjs9isVwzeT1cKARwd1xZB+kD9MJkGDzsn416vm0BCMvKuiCdg M1oOP0/nDLP9DMAnYaDZ2HO/4sUeOBCXh6jIAtJ5EUekJP0wVRUwhM1g9cW53C0QFZ+B HTDjcuSexTmcAmR85x5/qOx+pIbaM5NwD7H2utXxlkm7C31R8WEn6AdLQLzwMfb+ujgj zgNQalKwzDJT6x2VnqmvIv1FNk02D8PCDKG2LhsKNHMAlvPIbl/LZIfN4fweRCEaT0OG 31p6/n9wp84SNMIeuqNtjlyDW2IzdHqZqhMfPnZry3g0NDScUPrjj118iDRapaq55qpr rq1A== X-Forwarded-Encrypted: i=1; AJvYcCW8Lwg22Yg7IzejshRtUfDAntzJ2qvAO8rnGAAsqHcGsq4ezquhiKbBfclX2zUj2+ZcpXpTtnpH5+Na/Tcq7QBgvrQA/zo814tAMWHRDw== X-Gm-Message-State: AOJu0Ywb5C4zLF6/e3ZhwcyWu16rYBigGbLZQ4+H48sd9unHncIcBB56 QhzRbyjb8huaDkoE9KcHWs/hNrObVxvebAAufIIhAgbUpr1GNtGGx1O89wTWpcc= X-Google-Smtp-Source: AGHT+IF2qj5VG2aWGKmy+Z/F9xqjxzKMZfdoF+ZXES4kZiPGRVxWB31P4WLI9MkML3OCwmckEUxPAA== X-Received: by 2002:a05:6358:659b:b0:178:fd13:d6e4 with SMTP id x27-20020a056358659b00b00178fd13d6e4mr7493295rwh.25.1708131493668; Fri, 16 Feb 2024 16:58:13 -0800 (PST) Received: from atishp.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id d188-20020a6336c5000000b005dc89957e06sm487655pga.71.2024.02.16.16.58.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 16:58:13 -0800 (PST) From: Atish Patra To: linux-kernel@vger.kernel.org Cc: Adrian Hunter , Alexander Shishkin , Alexandre Ghiti , Andrew Jones , Anup Patel , Arnaldo Carvalho de Melo , Atish Patra , Christian Brauner , =?utf-8?b?Q2zDqW1lbnQgTMOpZ2Vy?= , Conor Dooley , devicetree@vger.kernel.org, Evan Green , Guo Ren , Heiko Stuebner , Ian Rogers , Ingo Molnar , James Clark , Jing Zhang , Jiri Olsa , Ji Sheng Teoh , John Garry , Jonathan Corbet , Kan Liang , Krzysztof Kozlowski , kvm-riscv@lists.infradead.org, kvm@vger.kernel.org, Ley Foon Tan , linux-doc@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Rutland , Namhyung Kim , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Rob Herring , Samuel Holland , Weilin Wang , Will Deacon , kaiwenxue1@gmail.com, Yang Jihong Subject: [PATCH RFC 00/20] Add Counter delegation ISA extension support Date: Fri, 16 Feb 2024 16:57:18 -0800 Message-Id: <20240217005738.3744121-1-atishp@rivosinc.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240216_165815_155090_06529A34 X-CRM114-Status: GOOD ( 33.46 ) X-Spam-Score: -0.0 (/) X-Spam-Report: Spam detection software, running on the system "bombadil.infradead.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: This series adds the counter delegation extension support. It is based on very early PoC work done by Kevin Xue and mostly rewritten after that. The counter delegation ISA extension(Smcdeleg/Ssccfg) a [...] Content analysis details: (-0.0 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:c34 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 T_SCC_BODY_TEXT_LINE No description available. X-BeenThere: kvm-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "kvm-riscv" Errors-To: kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org This series adds the counter delegation extension support. It is based on very early PoC work done by Kevin Xue and mostly rewritten after that. The counter delegation ISA extension(Smcdeleg/Ssccfg) actually depends on multiple ISA extensions. 1. S[m|s]csrind : The indirect CSR extension[1] which defines additional 5 ([M|S|VS]IREG2-[M|S|VS]IREG6) register to address size limitation of RISC-V CSR address space. 2. Smstateen: The stateen bit[60] controls the access to the registers indirectly via the above indirect registers. 3. Smcdeleg/Ssccfg: The counter delegation extensions[2] The counter delegation extension allows Supervisor mode to program the hpmevent and hpmcounters directly without needing the assistance from the M-mode via SBI calls. This results in a faster perf profiling and very few traps. This extension also introduces a scountinhibit CSR which allows to stop/start any counter directly from the S-mode. As the counter delegation extension potentially can have more than 100 CSRs, the specification leverages the indirect CSR extension to save the precious CSR address range. Due to the dependency of these extensions, the following extensions must be enabled in qemu to use the counter delegation feature in S-mode. "smstateen=true,sscofpmf=true,ssccfg=true,smcdeleg=true,smcsrind=true,sscsrind=true" When we access the counters directly in S-mode, we also need to solve the following problems. 1. Event to counter mapping 2. Event encoding discovery The RISC-V ISA doesn't define any standard either for event encoding or the event to counter mapping rules. Until now, the SBI PMU implementation relies on device tree binding[3] to discover the event to counter mapping in RISC-V platform in the firmware. The SBI PMU specification[4] defines event encoding for standard perf events as well. Thus, the kernel can query the appropriate counter for an given event from the firmware. However, the kernel doesn't need any firmware interaction for hardware counters if counter delegation is available in the hardware. Thus, the driver needs to discover the above mappings/encodings by itself without any assistance from firmware. One of the options considered was to extend the PMU DT parsing support to kernel as well. However, that requires additional support in ACPI based system. It also needs more infrastructure in the virtualization as well. This patch series solves the above problem #1 by extending the perf tool in a way so that event json file can specify the counter constraints of each event and that can be passed to the driver to choose the best counter for a given event. The perf stat metric series[5] from Weilin already extend the perf tool to parse "Counter" property to specify the hardware counter restriction. I have included the patch from Weilin in this series for verification purposes only. I will rebase as that series evolves. This series extends that support by converting comma separated string to a bitmap. The counter constraint bitmap is passed to the perf driver via newly introduced "counterid_mask" property set in "config2". Even though, this is a generic perf tool change, this should not affect any other architecture if "counterid_mask" is not mapped. @Weilin: Please let me know if there is a better way to solve the problem I described. The problem #2 is solved by defining a architecture specific override function that will replace the perf standard event encoding with an encoding specified in the json file with the same event name. The alternate solution considered was to specify the encodings in the driver. However, these encodings are vendor specific in absence of an ISA guidelines and will become unmanageable with so many RISC-V vendors touching the driver for their encoding. The override is only required when counter delegation is available in the platform which is detected at the runtime. The SBI PMU (current implementation) doesn't require any override as it defines the standard event encoding. The hwprobe syscall defined for RISC-V is used for this detection in this series. A sysfs based property can be explored to do the same but we may require hwprobe in future given the churn of extensions in RISC-V. That's why, I went with hwprobe. Let me know if anybody thinks that's a bad idea. The perf tool also hook allows RISC-V ISA platform vendors to define their encoding for any standard perf or ISA event. I have tried to cover all the use cases that I am aware of (stat, record, top). Please let me know if I have missed any particular use case where architecture hook must be invoked. I am also open to any other idea to solve the above said problem. PATCH organization: PATCH 1 is from the perf metric series[5] PATCH 2-5 defines and implements the indirect CSR extension. PATCH 6-10 defines the other required ISA extensions. PATCH 11 just an overall restructure of the RISC-V PMU driver. PATCH 12-14 implements the counter delegation extension and new perf tool plumbings to solve #1 and #2. PATCH 15-16 improves the perf tool support to solve #1 and #2. PATCH 17 adds a perf json file for qemu virt machine. PATCH 18-20 adds hwprobe mechanism to enable perf to detect if platform supports delegation extensions. There is no change in process to run perf stat/record and will continue to work as it is as long as the relevant extensions have been enabled in Qemu. However, the perf tool needs to be recompiled with as it requires new kenrel headers. The Qemu patches can be found here: https://github.com/atishp04/qemu/tree/counter_delegation_rfc The opensbi patch can be found here: https://github.com/atishp04/opensbi/tree/counter_delegation_v1 The Linux kernel patches can be found here: https://github.com/atishp04/linux/tree/counter_delegation_rfc [1] https://github.com/riscv/riscv-indirect-csr-access [2] https://github.com/riscv/riscv-smcdeleg-ssccfg [3] https://www.kernel.org/doc/Documentation/devicetree/bindings/perf/riscv%2Cpmu.yaml [4] https://github.com/riscv-non-isa/riscv-sbi-doc/blob/master/src/ext-pmu.adoc [5] https://lore.kernel.org/all/20240209031441.943012-4-weilin.wang@intel.com/ Atish Patra (17): RISC-V: Add Sxcsrind ISA extension definition and parsing dt-bindings: riscv: add Sxcsrind ISA extension description RISC-V: Define indirect CSR access helpers RISC-V: Add Ssccfg ISA extension definition and parsing dt-bindings: riscv: add Ssccfg ISA extension description RISC-V: Add Smcntrpmf extension parsing dt-bindings: riscv: add Smcntrpmf ISA extension description RISC-V: perf: Restructure the SBI PMU code RISC-V: perf: Modify the counter discovery mechanism RISC-V: perf: Implement supervisor counter delegation support RISC-V: perf: Use config2 for event to counter mapping tools/perf: Add arch hooks to override perf standard events tools/perf: Pass the Counter constraint values in the pmu events perf: Add json file for virt machine supported events tools arch uapi: Sync the uinstd.h header file for RISC-V RISC-V: Add hwprobe support for Counter delegation extensions tools/perf: Detect if platform supports counter delegation Kaiwen Xue (2): RISC-V: Add Sxcsrind ISA extension CSR definitions RISC-V: Add Sscfg extension CSR definition Weilin Wang (1): perf pmu-events: Add functions in jevent.py to parse counter and event info for hardware aware grouping Documentation/arch/riscv/hwprobe.rst | 10 + .../devicetree/bindings/riscv/extensions.yaml | 34 + MAINTAINERS | 4 +- arch/riscv/include/asm/csr.h | 47 ++ arch/riscv/include/asm/csr_ind.h | 42 ++ arch/riscv/include/asm/hwcap.h | 5 + arch/riscv/include/asm/sbi.h | 2 +- arch/riscv/include/uapi/asm/hwprobe.h | 4 + arch/riscv/kernel/cpufeature.c | 5 + arch/riscv/kernel/sys_hwprobe.c | 3 + arch/riscv/kvm/vcpu_pmu.c | 2 +- drivers/perf/Kconfig | 16 +- drivers/perf/Makefile | 4 +- .../perf/{riscv_pmu.c => riscv_pmu_common.c} | 0 .../perf/{riscv_pmu_sbi.c => riscv_pmu_dev.c} | 654 ++++++++++++++---- include/linux/perf/riscv_pmu.h | 13 +- tools/arch/riscv/include/uapi/asm/unistd.h | 14 +- tools/perf/arch/riscv/util/Build | 2 + tools/perf/arch/riscv/util/evlist.c | 60 ++ tools/perf/arch/riscv/util/pmu.c | 41 ++ tools/perf/arch/riscv/util/pmu.h | 11 + tools/perf/builtin-record.c | 3 + tools/perf/builtin-stat.c | 2 + tools/perf/builtin-top.c | 3 + .../pmu-events/arch/riscv/arch-standard.json | 10 + tools/perf/pmu-events/arch/riscv/mapfile.csv | 1 + .../pmu-events/arch/riscv/qemu/virt/cpu.json | 30 + .../arch/riscv/qemu/virt/firmware.json | 68 ++ tools/perf/pmu-events/jevents.py | 186 ++++- tools/perf/pmu-events/pmu-events.h | 25 +- tools/perf/util/evlist.c | 6 + tools/perf/util/evlist.h | 6 + 32 files changed, 1167 insertions(+), 146 deletions(-) create mode 100644 arch/riscv/include/asm/csr_ind.h rename drivers/perf/{riscv_pmu.c => riscv_pmu_common.c} (100%) rename drivers/perf/{riscv_pmu_sbi.c => riscv_pmu_dev.c} (61%) create mode 100644 tools/perf/arch/riscv/util/evlist.c create mode 100644 tools/perf/arch/riscv/util/pmu.c create mode 100644 tools/perf/arch/riscv/util/pmu.h create mode 100644 tools/perf/pmu-events/arch/riscv/arch-standard.json create mode 100644 tools/perf/pmu-events/arch/riscv/qemu/virt/cpu.json create mode 100644 tools/perf/pmu-events/arch/riscv/qemu/virt/firmware.json --- 2.34.1