From patchwork Wed Jul 20 11:03:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1658480 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=oZZgfFOU; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Lnt9M4KyDz9s1l for ; Wed, 20 Jul 2022 21:03:34 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4Lnt9K43ssz3bk8 for ; Wed, 20 Jul 2022 21:03:33 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=oZZgfFOU; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::531; helo=mail-pg1-x531.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=oZZgfFOU; dkim-atps=neutral Received: from mail-pg1-x531.google.com (mail-pg1-x531.google.com [IPv6:2607:f8b0:4864:20::531]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4Lnt916KQgz2xjg for ; Wed, 20 Jul 2022 21:03:15 +1000 (AEST) Received: by mail-pg1-x531.google.com with SMTP id f65so16055599pgc.12 for ; Wed, 20 Jul 2022 04:03:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=OS9YmbIgBAwgeLHvR4rL+gxtjmJopxjNVCtZdsYgwCI=; b=oZZgfFOUfvkuPB6a6jHw97HxL9AMcYyNS3yrt4NH7H1xmCbut9nOq1EHrO+XasSvVj nIj5fB4ui2w8Js098ZjYC6ERDjckQASpmcAHAvzZmfL/YDENGN5qWZjDBq34h1KJx5Fz nNeQHiRMFgj9/sPwn0i+lJkCQmDYo+0rT9XqXAz2TIAiXeyXQXmUjzLQe+nzxULNtIUT tvNoiypIufbnougnLMclsuJy7SqLHbOIeJYfhFqlS6kZp5kPzc1ZiAHZ8bWNNRNCdxLu nPHINSPUUJAMztbncDvMPjREcYe0A5VjbAJfg9z5bLEm/WRA/DUm3Ij6J1BCYc8MPjJp IIaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=OS9YmbIgBAwgeLHvR4rL+gxtjmJopxjNVCtZdsYgwCI=; b=ftwzAXFCN2Gh49fDU7fmm+NXiGQ5lOJRnsSJt0wYgZbMjWC+S0tAmD+Becn3Uz0tNy yUHlmQYOkhkK8bXusj4hHZzMzKbT3qQ50/3D0NEBHOnCLu/tuF+2C+x1r1aalR90Ygq+ LoPB9obF9xBSBf2xL5wcSpKSZcErBQhI7IPZoKxqPMRypb+xs9RYc4Alvr0AldhdcNQi zPV0wG2s2P9I+454PGzmq09HqaP7+PXP5rf7K5mOFBkW3zFAHoFvpymm3qpqx4q9Hxbp DPAhFk048VToYVTv8KPNS1d8qomkJ7z/SoxiJmLBmkaYcHvSJ3qWH4dpns1/BrQeukkk ZMUA== X-Gm-Message-State: AJIora/gKvTu9RJwjtCIaxvOEKabtKL1W1EmScnv65WujWZx+RMYJxEQ WrBH9rKXQJZbls3Gp3BT2pPduGvjpSE= X-Google-Smtp-Source: AGRyM1uqfwREkhXKh6353Py0ljXpNsFyvKnNmrOY7qqAYW/jvtgx7CBmnps4WAXIaRj87yxNVacXkg== X-Received: by 2002:a63:87:0:b0:40c:b1f6:a879 with SMTP id 129-20020a630087000000b0040cb1f6a879mr33261947pga.236.1658314992194; Wed, 20 Jul 2022 04:03:12 -0700 (PDT) Received: from bobo.ozlabs.ibm.com (27-33-251-27.static.tpgi.com.au. [27.33.251.27]) by smtp.gmail.com with ESMTPSA id p9-20020aa79e89000000b00528a097aeffsm13238652pfq.118.2022.07.20.04.03.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Jul 2022 04:03:11 -0700 (PDT) From: Nicholas Piggin To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 1/2] powerpc: add ISA v3.0 / v3.1 wait opcode macro Date: Wed, 20 Jul 2022 21:03:01 +1000 Message-Id: <20220720110302.684113-1-npiggin@gmail.com> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicholas Piggin Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" The wait instruction encoding changed between ISA v2.07 and ISA v3.0. In v3.1 the instruction gained a new field. Update the PPC_WAIT macro to the current encoding. Rename the older incompatible one with a _v203 suffix as it was introduced in v2.03 (the WC field was introduced in v2.07 but the kernel only uses WC=0). Signed-off-by: Nicholas Piggin --- v2: Update naming, patch changelog and title. arch/powerpc/include/asm/ppc-opcode.h | 7 +++++-- arch/powerpc/kernel/idle_book3e.S | 2 +- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/arch/powerpc/include/asm/ppc-opcode.h b/arch/powerpc/include/asm/ppc-opcode.h index 89beabf5325c..856555d69d93 100644 --- a/arch/powerpc/include/asm/ppc-opcode.h +++ b/arch/powerpc/include/asm/ppc-opcode.h @@ -331,6 +331,7 @@ #define __PPC_XSP(s) ((((s) & 0x1e) | (((s) >> 5) & 0x1)) << 21) #define __PPC_XTP(s) __PPC_XSP(s) #define __PPC_T_TLB(t) (((t) & 0x3) << 21) +#define __PPC_PL(p) (((p) & 0x3) << 16) #define __PPC_WC(w) (((w) & 0x3) << 21) #define __PPC_WS(w) (((w) & 0x1f) << 11) #define __PPC_SH(s) __PPC_WS(s) @@ -398,7 +399,8 @@ #define PPC_RAW_RFDI (0x4c00004e) #define PPC_RAW_RFMCI (0x4c00004c) #define PPC_RAW_TLBILX(t, a, b) (0x7c000024 | __PPC_T_TLB(t) | __PPC_RA0(a) | __PPC_RB(b)) -#define PPC_RAW_WAIT(w) (0x7c00007c | __PPC_WC(w)) +#define PPC_RAW_WAIT_v203 (0x7c00007c) +#define PPC_RAW_WAIT(w, p) (0x7c00003c | __PPC_WC(w) | __PPC_PL(p)) #define PPC_RAW_TLBIE(lp, a) (0x7c000264 | ___PPC_RB(a) | ___PPC_RS(lp)) #define PPC_RAW_TLBIE_5(rb, rs, ric, prs, r) \ (0x7c000264 | ___PPC_RB(rb) | ___PPC_RS(rs) | ___PPC_RIC(ric) | ___PPC_PRS(prs) | ___PPC_R(r)) @@ -613,7 +615,8 @@ #define PPC_TLBILX_ALL(a, b) PPC_TLBILX(0, a, b) #define PPC_TLBILX_PID(a, b) PPC_TLBILX(1, a, b) #define PPC_TLBILX_VA(a, b) PPC_TLBILX(3, a, b) -#define PPC_WAIT(w) stringify_in_c(.long PPC_RAW_WAIT(w)) +#define PPC_WAIT_v203(w) stringify_in_c(.long PPC_RAW_WAIT_v203) +#define PPC_WAIT(w, p) stringify_in_c(.long PPC_RAW_WAIT(w, p)) #define PPC_TLBIE(lp, a) stringify_in_c(.long PPC_RAW_TLBIE(lp, a)) #define PPC_TLBIE_5(rb, rs, ric, prs, r) \ stringify_in_c(.long PPC_RAW_TLBIE_5(rb, rs, ric, prs, r)) diff --git a/arch/powerpc/kernel/idle_book3e.S b/arch/powerpc/kernel/idle_book3e.S index cc008de58b05..6447de51ea71 100644 --- a/arch/powerpc/kernel/idle_book3e.S +++ b/arch/powerpc/kernel/idle_book3e.S @@ -77,7 +77,7 @@ _GLOBAL(\name) .macro BOOK3E_IDLE_LOOP 1: - PPC_WAIT(0) + PPC_WAIT_v203 b 1b .endm From patchwork Wed Jul 20 11:03:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1658481 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=kwwnuXXd; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LntB05wvTz9s1l for ; Wed, 20 Jul 2022 21:04:08 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4LntB04jZ5z3c1b for ; Wed, 20 Jul 2022 21:04:08 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=kwwnuXXd; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::42c; helo=mail-pf1-x42c.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=kwwnuXXd; dkim-atps=neutral Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4Lnt9200gGz2xjm for ; Wed, 20 Jul 2022 21:03:17 +1000 (AEST) Received: by mail-pf1-x42c.google.com with SMTP id 70so16220491pfx.1 for ; Wed, 20 Jul 2022 04:03:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kBi+NdAzq7wuL4GEH5aN8bOZ61bpd6rksyx869TB+hI=; b=kwwnuXXdiUEkd9KzV9z9mS6fmmSYKdYN+eKzxLYyXtII9bokEQ19SP1S+wnu8R3noJ 4SZVwi1DDKPyDwreHnouvgJF1euKfiOlMVHm8VO2NEk/I1CG/c1trr7DdzD+Ik7/2oQ1 V0/r3OT50LSJwFpIQHYfnOCuNeBX3sI7TlVRNoMxMRwlQA2S3eaBqJcLBcnQPLREvron Fq1JQIDKql9CsRPgW6TQ+Sykl316r4Ge8oLU0g92kVAzfIUHIK9YVRWXVKbxJH2lOGXk tMrGpBmtfUr8mC01ysXzvyJ2dArJ7FUGxja9TqPzYgW4rv1ZLMOi1ST2U4XuP1pWtDln U44A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kBi+NdAzq7wuL4GEH5aN8bOZ61bpd6rksyx869TB+hI=; b=qw0Ghd9LhVPvZZFHw3VxjrjOl5y4dZ8bpQAVAJT3lUC1kBKpUrXly/ORGhWmx1Mnb3 tTAmMnBQC0XflpjawHAP/EPJUtRjXP2CaBzx/BVH0pEfmUGdonLYvYpcemjp4Bufo3Uh k0Bza3EMXpH7fPL4uuVl5hiG+8AOQ9Qegwutenb4lj0kuq8pPIavnfV7u+0MyI85VoPE qJn5dbboFq42l6FAPvhNX0HIphxE1ZCFQ4PhypFkxoWk2pBoMfGz/3gx7Ur/lHdaUELK jBihtUsTGt+23ydPzfS9mJ9Ba5lnsEkajQI5qu8pIi/HL6qG3GsxtJ/wf8XCz1t3U4Z6 AmQg== X-Gm-Message-State: AJIora+HeR8dgQJLOtcHsSEMdvSBjuL6pU5pa2OjDKgK0QeUKFnMoHDW fJNpxsvxwxSTOqEigKI/UFcOO0cqqI4= X-Google-Smtp-Source: AGRyM1svqswZGmnXH7rfjn++mkftbOmkTg8N3JAH2sjhrEEE0zY+tAeR7j1GCgoo6lOKPQ/LspR6MQ== X-Received: by 2002:aa7:80d3:0:b0:52b:9237:a355 with SMTP id a19-20020aa780d3000000b0052b9237a355mr6196528pfn.73.1658314994724; Wed, 20 Jul 2022 04:03:14 -0700 (PDT) Received: from bobo.ozlabs.ibm.com (27-33-251-27.static.tpgi.com.au. [27.33.251.27]) by smtp.gmail.com with ESMTPSA id p9-20020aa79e89000000b00528a097aeffsm13238652pfq.118.2022.07.20.04.03.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Jul 2022 04:03:14 -0700 (PDT) From: Nicholas Piggin To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 2/2] powerpc/64s: Make POWER10 and later use pause_short in cpu_relax loops Date: Wed, 20 Jul 2022 21:03:02 +1000 Message-Id: <20220720110302.684113-2-npiggin@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220720110302.684113-1-npiggin@gmail.com> References: <20220720110302.684113-1-npiggin@gmail.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicholas Piggin Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" We want to move away from using SMT prioroty updates for cpu_relax, and use a 'wait' instruction which is similar to x86. As well as being a much better fit for what everybody else uses and tests with, priority nops are stateful which is nasty (interrupts have to consider they might be taken at a different priority), and they're expensive to execute, similar to a mtSPR which can effect other threads in the pipe. Signed-off-by: Nicholas Piggin --- Unfortunately qemu TCG does not emulate pause_short properly and will cause hangs. I have a patch for it but not merged yet. But if we tune qspinlock code it would be best to do it with this patch. Thanks, Nick arch/powerpc/include/asm/processor.h | 30 +++++++++++++++++++---- arch/powerpc/include/asm/vdso/processor.h | 10 +++++++- 2 files changed, 34 insertions(+), 6 deletions(-) diff --git a/arch/powerpc/include/asm/processor.h b/arch/powerpc/include/asm/processor.h index fdfaae194ddd..61f16515cbe0 100644 --- a/arch/powerpc/include/asm/processor.h +++ b/arch/powerpc/include/asm/processor.h @@ -355,11 +355,31 @@ static inline unsigned long __pack_fe01(unsigned int fpmode) #ifdef CONFIG_PPC64 -#define spin_begin() HMT_low() - -#define spin_cpu_relax() barrier() - -#define spin_end() HMT_medium() +#define spin_begin() \ +do { \ + asm volatile(ASM_FTR_IFCLR( \ + "or 1,1,1", /* HMT_LOW */ \ + "nop",/* POWER10 onward uses pause_short (wait 2,0) */ \ + %0) :: "i" (CPU_FTR_ARCH_31) : "memory"); \ +} while (0) + +#define spin_cpu_relax() \ +do { \ + asm volatile(ASM_FTR_IFCLR( \ + /* Pre-POWER10 uses low ; medium priority nops */ \ + "nop", \ + /* POWER10 onward uses pause_short (wait 2,0) */ \ + PPC_WAIT_BOOKS(2, 0), \ + %0) :: "i" (CPU_FTR_ARCH_31) : "memory"); \ +} while (0) + +#define spin_end() \ +do { \ + asm volatile(ASM_FTR_IFCLR( \ + "or 2,2,2", /* HMT_MEDIUM */ \ + "nop",/* POWER10 onward uses pause_short (wait 2,0) */ \ + %0) :: "i" (CPU_FTR_ARCH_31) : "memory"); \ +} while (0) #endif diff --git a/arch/powerpc/include/asm/vdso/processor.h b/arch/powerpc/include/asm/vdso/processor.h index 8d79f994b4aa..1116230ebb08 100644 --- a/arch/powerpc/include/asm/vdso/processor.h +++ b/arch/powerpc/include/asm/vdso/processor.h @@ -22,7 +22,15 @@ #endif #ifdef CONFIG_PPC64 -#define cpu_relax() do { HMT_low(); HMT_medium(); barrier(); } while (0) +#define cpu_relax() \ +do { \ + asm volatile(ASM_FTR_IFCLR( \ + /* Pre-POWER10 uses low ; medium priority nops */ \ + "or 1,1,1 ; or 2,2,2", \ + /* POWER10 onward uses pause_short (wait 2,0) */ \ + PPC_WAIT_BOOKS(2, 0), \ + %0) :: "i" (CPU_FTR_ARCH_31) : "memory"); \ +} while (0) #else #define cpu_relax() barrier() #endif