From patchwork Sun Mar 28 06:30:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 1459242 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=sparclinux-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=bBnpmp+6; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4F7QqY2plGz9sR4 for ; Sun, 28 Mar 2021 17:32:25 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231131AbhC1Gbx (ORCPT ); Sun, 28 Mar 2021 02:31:53 -0400 Received: from mail.kernel.org ([198.145.29.99]:56702 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229577AbhC1GbV (ORCPT ); Sun, 28 Mar 2021 02:31:21 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 1EADE61981; Sun, 28 Mar 2021 06:31:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1616913080; bh=ZlgshpaCoA6zGi17OQe5KtXtUjgmxHsHhFE58+U5sOg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bBnpmp+6pVikaOh/qNNe9vmCla3ob2E2Pd8bBeyZl96U5fbR/nWaoBMz1OwMu08Te SO+RatKDGFM4fqWsxeNoeSNzfAfeWQzF1xA/fPk5pMWmI1TVS/h197n9j1TRSq6xZI m2XkAcw1QKjqyxSKvjRVLn0rund1OxBiepTiPWuXuh4TdZpROEKCnj9+A2IkwdLZj3 CGhHAvFJGUfPrimw3QmF6vglQQ4DiOvxXO4br7hgKoTUDo5rGhNiOJwUEu2xer9idk S20CUGeGlv7MXGp6fp/FA4yq+yPvzztNFE3O6Mo5oJ8yj9WbdAALPXN9xHBcitlIla 7HSMwZUxZCWDg== From: guoren@kernel.org To: guoren@kernel.org Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-arch@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-xtensa@linux-xtensa.org, openrisc@lists.librecores.org, sparclinux@vger.kernel.org, Guo Ren , Peter Zijlstra , Will Deacon , Ingo Molnar , Waiman Long , Arnd Bergmann , Anup Patel Subject: [PATCH v5 1/7] locking/qspinlock: Add ARCH_USE_QUEUED_SPINLOCKS_XCHG32 Date: Sun, 28 Mar 2021 06:30:22 +0000 Message-Id: <1616913028-83376-2-git-send-email-guoren@kernel.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1616913028-83376-1-git-send-email-guoren@kernel.org> References: <1616913028-83376-1-git-send-email-guoren@kernel.org> Precedence: bulk List-ID: X-Mailing-List: sparclinux@vger.kernel.org From: Guo Ren Some architectures don't have sub-word swap atomic instruction, they only have the full word's one. The sub-word swap only improve the performance when: NR_CPUS < 16K * 0- 7: locked byte * 8: pending * 9-15: not used * 16-17: tail index * 18-31: tail cpu (+1) The 9-15 bits are wasted to use xchg16 in xchg_tail. Please let architecture select xchg16/xchg32 to implement xchg_tail. Signed-off-by: Guo Ren Cc: Peter Zijlstra Cc: Will Deacon Cc: Ingo Molnar Cc: Waiman Long Cc: Arnd Bergmann Cc: Anup Patel --- kernel/Kconfig.locks | 3 +++ kernel/locking/qspinlock.c | 46 +++++++++++++++++++++----------------- 2 files changed, 28 insertions(+), 21 deletions(-) diff --git a/kernel/Kconfig.locks b/kernel/Kconfig.locks index 3de8fd11873b..d02f1261f73f 100644 --- a/kernel/Kconfig.locks +++ b/kernel/Kconfig.locks @@ -239,6 +239,9 @@ config LOCK_SPIN_ON_OWNER config ARCH_USE_QUEUED_SPINLOCKS bool +config ARCH_USE_QUEUED_SPINLOCKS_XCHG32 + bool + config QUEUED_SPINLOCKS def_bool y if ARCH_USE_QUEUED_SPINLOCKS depends on SMP diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c index cbff6ba53d56..4bfaa969bd15 100644 --- a/kernel/locking/qspinlock.c +++ b/kernel/locking/qspinlock.c @@ -163,26 +163,6 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock) WRITE_ONCE(lock->locked_pending, _Q_LOCKED_VAL); } -/* - * xchg_tail - Put in the new queue tail code word & retrieve previous one - * @lock : Pointer to queued spinlock structure - * @tail : The new queue tail code word - * Return: The previous queue tail code word - * - * xchg(lock, tail), which heads an address dependency - * - * p,*,* -> n,*,* ; prev = xchg(lock, node) - */ -static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail) -{ - /* - * We can use relaxed semantics since the caller ensures that the - * MCS node is properly initialized before updating the tail. - */ - return (u32)xchg_relaxed(&lock->tail, - tail >> _Q_TAIL_OFFSET) << _Q_TAIL_OFFSET; -} - #else /* _Q_PENDING_BITS == 8 */ /** @@ -206,6 +186,30 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock) { atomic_add(-_Q_PENDING_VAL + _Q_LOCKED_VAL, &lock->val); } +#endif /* _Q_PENDING_BITS == 8 */ + +#if _Q_PENDING_BITS == 8 && !defined(CONFIG_ARCH_USE_QUEUED_SPINLOCKS_XCHG32) +/* + * xchg_tail - Put in the new queue tail code word & retrieve previous one + * @lock : Pointer to queued spinlock structure + * @tail : The new queue tail code word + * Return: The previous queue tail code word + * + * xchg(lock, tail), which heads an address dependency + * + * p,*,* -> n,*,* ; prev = xchg(lock, node) + */ +static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail) +{ + /* + * We can use relaxed semantics since the caller ensures that the + * MCS node is properly initialized before updating the tail. + */ + return (u32)xchg_relaxed(&lock->tail, + tail >> _Q_TAIL_OFFSET) << _Q_TAIL_OFFSET; +} + +#else /** * xchg_tail - Put in the new queue tail code word & retrieve previous one @@ -236,7 +240,7 @@ static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail) } return old; } -#endif /* _Q_PENDING_BITS == 8 */ +#endif /** * queued_fetch_set_pending_acquire - fetch the whole lock value and set pending From patchwork Sun Mar 28 06:30:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 1459245 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=sparclinux-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=A0adv1QS; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4F7Qqb3QRcz9sWS for ; Sun, 28 Mar 2021 17:32:27 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231150AbhC1Gby (ORCPT ); Sun, 28 Mar 2021 02:31:54 -0400 Received: from mail.kernel.org ([198.145.29.99]:56740 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229485AbhC1GbZ (ORCPT ); Sun, 28 Mar 2021 02:31:25 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 2B9A76198F; Sun, 28 Mar 2021 06:31:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1616913084; bh=RHIlCNf2j+Ok+QoigsRs+Vcz1qgiomFS7YnCplj9JuE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=A0adv1QSOjJetRSxFJOCIhqf786u8FpE9GL+suXuO7xAirjxM0RoLpgzIbWh2WAj5 uj7H1dDw8/ZjNARTmCR+P8rd5Iq6ScTBSMh8mVeEbbG/tKV2sS188Ds9+/xobj8cUC EibrcghgWbPSsA+uIgUohyqc3ejFXck2oTwUzJcysxFMiEzIezMlG5EHaUdQm8bN3P G0dLchaLk0h9/WWWjZGIhg0BGaVdgb/5FdjmHa9Zbm9XOGJ7zqG9MlL/BfAbNqBW/N vfXwm+7+XgrunPvivrLA4LkMUMh4p0YIbP24um+x1fk8I5nX6bTgJTVheiGpmMVqg8 wAN+Vnc39IEiQ== From: guoren@kernel.org To: guoren@kernel.org Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-arch@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-xtensa@linux-xtensa.org, openrisc@lists.librecores.org, sparclinux@vger.kernel.org, Michael Clark , Guo Ren , Peter Zijlstra , Anup Patel , Arnd Bergmann , Palmer Dabbelt Subject: [PATCH v5 2/7] riscv: Convert custom spinlock/rwlock to generic qspinlock/qrwlock Date: Sun, 28 Mar 2021 06:30:23 +0000 Message-Id: <1616913028-83376-3-git-send-email-guoren@kernel.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1616913028-83376-1-git-send-email-guoren@kernel.org> References: <1616913028-83376-1-git-send-email-guoren@kernel.org> Precedence: bulk List-ID: X-Mailing-List: sparclinux@vger.kernel.org From: Michael Clark Update the RISC-V port to use the generic qspinlock and qrwlock. This patch requires support for xchg_xtail for full-word which are added by a previous patch: Guo added select ARCH_USE_QUEUED_SPINLOCKS_XCHG32 in Kconfig Guo fixed up compile error which made by below include sequence: +#include +#include Signed-off-by: Michael Clark Co-developed-by: Guo Ren Tested-by: Guo Ren Signed-off-by: Guo Ren Link: https://lore.kernel.org/linux-riscv/20190211043829.30096-3-michaeljclark@mac.com/ Cc: Peter Zijlstra Cc: Anup Patel Cc: Arnd Bergmann Cc: Palmer Dabbelt --- arch/riscv/Kconfig | 3 + arch/riscv/include/asm/Kbuild | 3 + arch/riscv/include/asm/spinlock.h | 126 +----------------------- arch/riscv/include/asm/spinlock_types.h | 15 +-- 4 files changed, 11 insertions(+), 136 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 87d7b52f278f..67cc65ba1ea1 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -33,6 +33,9 @@ config RISCV select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU select ARCH_WANT_FRAME_POINTERS select ARCH_WANT_HUGE_PMD_SHARE if 64BIT + select ARCH_USE_QUEUED_RWLOCKS + select ARCH_USE_QUEUED_SPINLOCKS + select ARCH_USE_QUEUED_SPINLOCKS_XCHG32 select CLONE_BACKWARDS select CLINT_TIMER if !MMU select COMMON_CLK diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild index 445ccc97305a..750c1056b90f 100644 --- a/arch/riscv/include/asm/Kbuild +++ b/arch/riscv/include/asm/Kbuild @@ -3,5 +3,8 @@ generic-y += early_ioremap.h generic-y += extable.h generic-y += flat.h generic-y += kvm_para.h +generic-y += mcs_spinlock.h +generic-y += qrwlock.h +generic-y += qspinlock.h generic-y += user.h generic-y += vmlinux.lds.h diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h index f4f7fa1b7ca8..a557de67a425 100644 --- a/arch/riscv/include/asm/spinlock.h +++ b/arch/riscv/include/asm/spinlock.h @@ -7,129 +7,7 @@ #ifndef _ASM_RISCV_SPINLOCK_H #define _ASM_RISCV_SPINLOCK_H -#include -#include -#include - -/* - * Simple spin lock operations. These provide no fairness guarantees. - */ - -/* FIXME: Replace this with a ticket lock, like MIPS. */ - -#define arch_spin_is_locked(x) (READ_ONCE((x)->lock) != 0) - -static inline void arch_spin_unlock(arch_spinlock_t *lock) -{ - smp_store_release(&lock->lock, 0); -} - -static inline int arch_spin_trylock(arch_spinlock_t *lock) -{ - int tmp = 1, busy; - - __asm__ __volatile__ ( - " amoswap.w %0, %2, %1\n" - RISCV_ACQUIRE_BARRIER - : "=r" (busy), "+A" (lock->lock) - : "r" (tmp) - : "memory"); - - return !busy; -} - -static inline void arch_spin_lock(arch_spinlock_t *lock) -{ - while (1) { - if (arch_spin_is_locked(lock)) - continue; - - if (arch_spin_trylock(lock)) - break; - } -} - -/***********************************************************/ - -static inline void arch_read_lock(arch_rwlock_t *lock) -{ - int tmp; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bltz %1, 1b\n" - " addi %1, %1, 1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - : "+A" (lock->lock), "=&r" (tmp) - :: "memory"); -} - -static inline void arch_write_lock(arch_rwlock_t *lock) -{ - int tmp; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bnez %1, 1b\n" - " li %1, -1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - : "+A" (lock->lock), "=&r" (tmp) - :: "memory"); -} - -static inline int arch_read_trylock(arch_rwlock_t *lock) -{ - int busy; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bltz %1, 1f\n" - " addi %1, %1, 1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - "1:\n" - : "+A" (lock->lock), "=&r" (busy) - :: "memory"); - - return !busy; -} - -static inline int arch_write_trylock(arch_rwlock_t *lock) -{ - int busy; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bnez %1, 1f\n" - " li %1, -1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - "1:\n" - : "+A" (lock->lock), "=&r" (busy) - :: "memory"); - - return !busy; -} - -static inline void arch_read_unlock(arch_rwlock_t *lock) -{ - __asm__ __volatile__( - RISCV_RELEASE_BARRIER - " amoadd.w x0, %1, %0\n" - : "+A" (lock->lock) - : "r" (-1) - : "memory"); -} - -static inline void arch_write_unlock(arch_rwlock_t *lock) -{ - smp_store_release(&lock->lock, 0); -} +#include +#include #endif /* _ASM_RISCV_SPINLOCK_H */ diff --git a/arch/riscv/include/asm/spinlock_types.h b/arch/riscv/include/asm/spinlock_types.h index f398e7638dd6..d033a973f287 100644 --- a/arch/riscv/include/asm/spinlock_types.h +++ b/arch/riscv/include/asm/spinlock_types.h @@ -6,20 +6,11 @@ #ifndef _ASM_RISCV_SPINLOCK_TYPES_H #define _ASM_RISCV_SPINLOCK_TYPES_H -#ifndef __LINUX_SPINLOCK_TYPES_H +#if !defined(__LINUX_SPINLOCK_TYPES_H) && !defined(_ASM_RISCV_SPINLOCK_H) # error "please don't include this file directly" #endif -typedef struct { - volatile unsigned int lock; -} arch_spinlock_t; - -#define __ARCH_SPIN_LOCK_UNLOCKED { 0 } - -typedef struct { - volatile unsigned int lock; -} arch_rwlock_t; - -#define __ARCH_RW_LOCK_UNLOCKED { 0 } +#include +#include #endif /* _ASM_RISCV_SPINLOCK_TYPES_H */ From patchwork Sun Mar 28 06:30:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 1459244 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=sparclinux-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=FAgaTJnJ; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4F7QqZ5DYSz9sWR for ; Sun, 28 Mar 2021 17:32:26 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231177AbhC1Gby (ORCPT ); Sun, 28 Mar 2021 02:31:54 -0400 Received: from mail.kernel.org ([198.145.29.99]:56770 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230144AbhC1Gb2 (ORCPT ); Sun, 28 Mar 2021 02:31:28 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 331216197C; Sun, 28 Mar 2021 06:31:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1616913088; bh=aYDkJQnGePK/+lkF8hDtexkWagQ4SRB6bigpJhVfgAg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FAgaTJnJOmusUtmymyBNZXGK1BUiB+LA57rvQXmHTnMcTLIWlPLGv0PPjuf3tTRRr 2M7sjdWDdwVjcO7G7yxZCrx9l+ZUtk+0DDg1pMt3NRHZMphNHPFG0G7hgysB/LzNXL Qk+qoQKbLwUilgC6hb7zxfcsvrM9A3HaNy/c8TSQG4x2zdRyPx3pAS67nZoGmcISIM 93imAG0DvPoKRNOZTqu4k854xndY6bJpFXoFuhV/ny4XCRhKJ0GsX6mARj0YJylwLR a8OiaIYEGfIi7XB3XgVeDPDpsGXg7Tm6jCxNm6emiN9+uFIgg5DJNngRhzrFEk3zLp l+WH2kaYZxeFQ== From: guoren@kernel.org To: guoren@kernel.org Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-arch@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-xtensa@linux-xtensa.org, openrisc@lists.librecores.org, sparclinux@vger.kernel.org, Guo Ren , Waiman Long , Peter Zijlstra , Will Deacon , Arnd Bergmann Subject: [PATCH v5 3/7] csky: Convert custom spinlock/rwlock to generic qspinlock/qrwlock Date: Sun, 28 Mar 2021 06:30:24 +0000 Message-Id: <1616913028-83376-4-git-send-email-guoren@kernel.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1616913028-83376-1-git-send-email-guoren@kernel.org> References: <1616913028-83376-1-git-send-email-guoren@kernel.org> Precedence: bulk List-ID: X-Mailing-List: sparclinux@vger.kernel.org From: Guo Ren Update the C-SKY port to use the generic qspinlock and qrwlock. C-SKY only support ldex.w/stex.w with word(double word) size & align access. So it must select XCHG32 to let qspinlock only use word atomic xchg_tail. Signed-off-by: Guo Ren Cc: Waiman Long Cc: Peter Zijlstra Cc: Will Deacon Cc: Arnd Bergmann --- arch/csky/Kconfig | 2 + arch/csky/include/asm/Kbuild | 2 + arch/csky/include/asm/spinlock.h | 82 +------------------------- arch/csky/include/asm/spinlock_types.h | 16 +---- 4 files changed, 6 insertions(+), 96 deletions(-) diff --git a/arch/csky/Kconfig b/arch/csky/Kconfig index 34e91224adc3..5910eb6ddde2 100644 --- a/arch/csky/Kconfig +++ b/arch/csky/Kconfig @@ -8,6 +8,8 @@ config CSKY select ARCH_HAS_SYNC_DMA_FOR_DEVICE select ARCH_USE_BUILTIN_BSWAP select ARCH_USE_QUEUED_RWLOCKS + select ARCH_USE_QUEUED_SPINLOCKS + select ARCH_USE_QUEUED_SPINLOCKS_XCHG32 select ARCH_WANT_FRAME_POINTERS if !CPU_CK610 select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT select COMMON_CLK diff --git a/arch/csky/include/asm/Kbuild b/arch/csky/include/asm/Kbuild index cc24bb8e539f..2a2d09963bb9 100644 --- a/arch/csky/include/asm/Kbuild +++ b/arch/csky/include/asm/Kbuild @@ -2,6 +2,8 @@ generic-y += asm-offsets.h generic-y += gpio.h generic-y += kvm_para.h +generic-y += mcs_spinlock.h generic-y += qrwlock.h +generic-y += qspinlock.h generic-y += user.h generic-y += vmlinux.lds.h diff --git a/arch/csky/include/asm/spinlock.h b/arch/csky/include/asm/spinlock.h index 69f5aa249c5f..fcff36753c25 100644 --- a/arch/csky/include/asm/spinlock.h +++ b/arch/csky/include/asm/spinlock.h @@ -3,87 +3,7 @@ #ifndef __ASM_CSKY_SPINLOCK_H #define __ASM_CSKY_SPINLOCK_H -#include -#include - -/* - * Ticket-based spin-locking. - */ -static inline void arch_spin_lock(arch_spinlock_t *lock) -{ - arch_spinlock_t lockval; - u32 ticket_next = 1 << TICKET_NEXT; - u32 *p = &lock->lock; - u32 tmp; - - asm volatile ( - "1: ldex.w %0, (%2) \n" - " mov %1, %0 \n" - " add %0, %3 \n" - " stex.w %0, (%2) \n" - " bez %0, 1b \n" - : "=&r" (tmp), "=&r" (lockval) - : "r"(p), "r"(ticket_next) - : "cc"); - - while (lockval.tickets.next != lockval.tickets.owner) - lockval.tickets.owner = READ_ONCE(lock->tickets.owner); - - smp_mb(); -} - -static inline int arch_spin_trylock(arch_spinlock_t *lock) -{ - u32 tmp, contended, res; - u32 ticket_next = 1 << TICKET_NEXT; - u32 *p = &lock->lock; - - do { - asm volatile ( - " ldex.w %0, (%3) \n" - " movi %2, 1 \n" - " rotli %1, %0, 16 \n" - " cmpne %1, %0 \n" - " bt 1f \n" - " movi %2, 0 \n" - " add %0, %0, %4 \n" - " stex.w %0, (%3) \n" - "1: \n" - : "=&r" (res), "=&r" (tmp), "=&r" (contended) - : "r"(p), "r"(ticket_next) - : "cc"); - } while (!res); - - if (!contended) - smp_mb(); - - return !contended; -} - -static inline void arch_spin_unlock(arch_spinlock_t *lock) -{ - smp_mb(); - WRITE_ONCE(lock->tickets.owner, lock->tickets.owner + 1); -} - -static inline int arch_spin_value_unlocked(arch_spinlock_t lock) -{ - return lock.tickets.owner == lock.tickets.next; -} - -static inline int arch_spin_is_locked(arch_spinlock_t *lock) -{ - return !arch_spin_value_unlocked(READ_ONCE(*lock)); -} - -static inline int arch_spin_is_contended(arch_spinlock_t *lock) -{ - struct __raw_tickets tickets = READ_ONCE(lock->tickets); - - return (tickets.next - tickets.owner) > 1; -} -#define arch_spin_is_contended arch_spin_is_contended - +#include #include #endif /* __ASM_CSKY_SPINLOCK_H */ diff --git a/arch/csky/include/asm/spinlock_types.h b/arch/csky/include/asm/spinlock_types.h index 8ff0f6ff3a00..757594760e65 100644 --- a/arch/csky/include/asm/spinlock_types.h +++ b/arch/csky/include/asm/spinlock_types.h @@ -7,21 +7,7 @@ # error "please don't include this file directly" #endif -#define TICKET_NEXT 16 - -typedef struct { - union { - u32 lock; - struct __raw_tickets { - /* little endian */ - u16 owner; - u16 next; - } tickets; - }; -} arch_spinlock_t; - -#define __ARCH_SPIN_LOCK_UNLOCKED { { 0 } } - +#include #include #endif /* __ASM_CSKY_SPINLOCK_TYPES_H */ From patchwork Sun Mar 28 06:30:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 1459247 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=sparclinux-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=oCRMri96; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4F7Qqd119wz9sWS for ; Sun, 28 Mar 2021 17:32:29 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231195AbhC1Gbz (ORCPT ); Sun, 28 Mar 2021 02:31:55 -0400 Received: from mail.kernel.org ([198.145.29.99]:56816 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230493AbhC1Gbc (ORCPT ); Sun, 28 Mar 2021 02:31:32 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 1BAC76191A; Sun, 28 Mar 2021 06:31:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1616913092; bh=QzKnPWoBnPHqdJHKCVpbj1BpTP2hO0zRu8JrhjYU+j0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oCRMri96fe/VV+ky4tT+TYgNgYdWWJzEgeXfW1Tvd/axc2XqcW9wem1Jg6EWGepv2 QjpcK6XDPVTYzlAGc9cBpGJNs76gqhECTTGkJoQEDk+3zXn3vooA/aHijGpPJOqIjA KaNg4iVmEpWEVc+v+reBWXGLdp/9vX3UPt6ESFnyGi8C3C85wjhdaSIOc2rHkvEjGU q/dKoB5wnBsLvoh7s8j/ZofobGS9h+hH0KO9HHec613wxtdvFuKWgCppj0bO1HPk6w BYMnDjrYcXcZ5Cd+KQpevDg06rlgh2/Yp6Q3kReMZDAa8puVTZfEmFJKn+m05xwAZ0 BuEJgjqjL8urw== From: guoren@kernel.org To: guoren@kernel.org Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-arch@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-xtensa@linux-xtensa.org, openrisc@lists.librecores.org, sparclinux@vger.kernel.org, Guo Ren , Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras Subject: [PATCH v5 4/7] powerpc/qspinlock: Add ARCH_USE_QUEUED_SPINLOCKS_XCHG32 Date: Sun, 28 Mar 2021 06:30:25 +0000 Message-Id: <1616913028-83376-5-git-send-email-guoren@kernel.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1616913028-83376-1-git-send-email-guoren@kernel.org> References: <1616913028-83376-1-git-send-email-guoren@kernel.org> Precedence: bulk List-ID: X-Mailing-List: sparclinux@vger.kernel.org From: Guo Ren We don't have native hw xchg16 instruction, so let qspinlock generic code to deal with it. Using the full-word atomic xchg instructions implement xchg16 has the semantic risk for atomic operations. This patch cancels the dependency of on qspinlock generic code on architecture's xchg16. Signed-off-by: Guo Ren Cc: Michael Ellerman Cc: Benjamin Herrenschmidt Cc: Paul Mackerras --- arch/powerpc/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 386ae12d8523..69ec4ade6521 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -151,6 +151,7 @@ config PPC select ARCH_USE_CMPXCHG_LOCKREF if PPC64 select ARCH_USE_QUEUED_RWLOCKS if PPC_QUEUED_SPINLOCKS select ARCH_USE_QUEUED_SPINLOCKS if PPC_QUEUED_SPINLOCKS + select ARCH_USE_QUEUED_SPINLOCKS_XCHG32 if PPC_QUEUED_SPINLOCKS select ARCH_WANT_IPC_PARSE_VERSION select ARCH_WANT_IRQS_OFF_ACTIVATE_MM select ARCH_WANT_LD_ORPHAN_WARN From patchwork Sun Mar 28 06:30:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 1459246 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=sparclinux-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=Wf0uvxCQ; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4F7Qqc5lD5z9sWR for ; Sun, 28 Mar 2021 17:32:28 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231206AbhC1Gbz (ORCPT ); Sun, 28 Mar 2021 02:31:55 -0400 Received: from mail.kernel.org ([198.145.29.99]:56844 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230495AbhC1Gbg (ORCPT ); Sun, 28 Mar 2021 02:31:36 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id C9D806197C; Sun, 28 Mar 2021 06:31:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1616913096; bh=Bg9d2w1WAX5KFLAnbnlQNHWx+c16ctVJTJF6+EgKbfQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Wf0uvxCQIWoezH2Q//Fy+0JLf/n/JQM1fgGTIy+LNlK2ke3/GfMHR3INbhImyBKub PYoWfqkNgd/16507YS2V+9oXcBGnlB31xMCYPq4i/42w/2S/VMWUBfiWhHpcJilL8F 3Ug35ynRWtusIYmXztSQi1aR21Tlyp2sgtUPmJYRmC8gtc/K5PXnkUmkkt1jKh3C9F TyjzDm/lKbQBpjG+hHUIJbOwh7gFQOtovJe5DUrQIogSJfInGNEiZgiDba+jknWCpx 2kwyOHYg5WsCDiNR70IA5XNcpFv4vfIXOBFYCliK9/b+BMC46SrVQb0cTyEhy0/v1K h7HNGGnm6y0Eg== From: guoren@kernel.org To: guoren@kernel.org Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-arch@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-xtensa@linux-xtensa.org, openrisc@lists.librecores.org, sparclinux@vger.kernel.org, Guo Ren , Arnd Bergmann , Jonas Bonn , Stefan Kristiansson , Stafford Horne Subject: [PATCH v5 5/7] openrisc: qspinlock: Add ARCH_USE_QUEUED_SPINLOCKS_XCHG32 Date: Sun, 28 Mar 2021 06:30:26 +0000 Message-Id: <1616913028-83376-6-git-send-email-guoren@kernel.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1616913028-83376-1-git-send-email-guoren@kernel.org> References: <1616913028-83376-1-git-send-email-guoren@kernel.org> Precedence: bulk List-ID: X-Mailing-List: sparclinux@vger.kernel.org From: Guo Ren We don't have native hw xchg16 instruction, so let qspinlock generic code to deal with it. Using the full-word atomic xchg instructions implement xchg16 has the semantic risk for atomic operations. This patch cancels the dependency of on qspinlock generic code on architecture's xchg16. Signed-off-by: Guo Ren Cc: Arnd Bergmann Cc: Jonas Bonn Cc: Stefan Kristiansson Cc: Stafford Horne Cc: openrisc@lists.librecores.org --- arch/openrisc/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/openrisc/Kconfig b/arch/openrisc/Kconfig index 591acc5990dc..b299e409429f 100644 --- a/arch/openrisc/Kconfig +++ b/arch/openrisc/Kconfig @@ -33,6 +33,7 @@ config OPENRISC select OR1K_PIC select CPU_NO_EFFICIENT_FFS if !OPENRISC_HAVE_INST_FF1 select ARCH_USE_QUEUED_SPINLOCKS + select ARCH_USE_QUEUED_SPINLOCKS_XCHG32 select ARCH_USE_QUEUED_RWLOCKS select OMPIC if SMP select ARCH_WANT_FRAME_POINTERS From patchwork Sun Mar 28 06:30:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 1459249 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=sparclinux-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=RIkij08x; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4F7Qqg0xptz9sWX for ; Sun, 28 Mar 2021 17:32:31 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231218AbhC1Gb4 (ORCPT ); Sun, 28 Mar 2021 02:31:56 -0400 Received: from mail.kernel.org ([198.145.29.99]:56878 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230502AbhC1Gbj (ORCPT ); Sun, 28 Mar 2021 02:31:39 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id AFCCB61999; Sun, 28 Mar 2021 06:31:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1616913099; bh=O2KxmIPB+fHLZecH69jIXhtq++wb564lj/icyjeUAfo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RIkij08xlS8DB8+yEsaHfhkJ4frVk8aqcXZKUjc8aZRsRg14dDqPdSr8TSjbqYWar YZa7xXiDmepsbU0aY2OQB+g9ZqgA6O0epYM/s/Lq/CGXTTUZR/qeGjvktEkSawSenY 6uspD6vEI92SJzbXoQjxXYH9frX6UTrSuMhM5XRheYpddQnE2p8LoyPEDQ5x0vhDc+ aPIyENu/RTIQJgeveEE/LmORCiHcAjMknUD4XEEpdndhKUefcId80TspfCdlvafStJ 6CIxmOlQadSXN+7398RrpGZOq23MCSKdY+hW6nlsOKJHvtdBV75Hz84ouhtYXGQH+u 8EGq83B9rHGOg== From: guoren@kernel.org To: guoren@kernel.org Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-arch@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-xtensa@linux-xtensa.org, openrisc@lists.librecores.org, sparclinux@vger.kernel.org, Guo Ren , Arnd Bergmann , "David S . Miller" , Rob Gardner Subject: [PATCH v5 6/7] sparc: qspinlock: Add ARCH_USE_QUEUED_SPINLOCKS_XCHG32 Date: Sun, 28 Mar 2021 06:30:27 +0000 Message-Id: <1616913028-83376-7-git-send-email-guoren@kernel.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1616913028-83376-1-git-send-email-guoren@kernel.org> References: <1616913028-83376-1-git-send-email-guoren@kernel.org> Precedence: bulk List-ID: X-Mailing-List: sparclinux@vger.kernel.org From: Guo Ren We don't have native hw xchg16 instruction, so let qspinlock generic code to deal with it. Using the full-word atomic xchg instructions implement xchg16 has the semantic risk for atomic operations. This patch cancels the dependency of on qspinlock generic code on architecture's xchg16. Signed-off-by: Guo Ren Cc: Arnd Bergmann Cc: David S. Miller Cc: Rob Gardner --- arch/sparc/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig index 164a5254c91c..1079fe3f058c 100644 --- a/arch/sparc/Kconfig +++ b/arch/sparc/Kconfig @@ -91,6 +91,7 @@ config SPARC64 select HAVE_REGS_AND_STACK_ACCESS_API select ARCH_USE_QUEUED_RWLOCKS select ARCH_USE_QUEUED_SPINLOCKS + select ARCH_USE_QUEUED_SPINLOCKS_XCHG32 select GENERIC_TIME_VSYSCALL select ARCH_CLOCKSOURCE_DATA select ARCH_HAS_PTE_SPECIAL From patchwork Sun Mar 28 06:30:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 1459248 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=sparclinux-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=K1i+Vgp+; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4F7Qqf1T1Xz9sWg for ; Sun, 28 Mar 2021 17:32:30 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231235AbhC1Gb4 (ORCPT ); Sun, 28 Mar 2021 02:31:56 -0400 Received: from mail.kernel.org ([198.145.29.99]:56946 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230512AbhC1Gbm (ORCPT ); Sun, 28 Mar 2021 02:31:42 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id A4715619A0; Sun, 28 Mar 2021 06:31:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1616913102; bh=sUpQ42V7/BeYJXo8xKilrJhdJzUrsLncMVt8E/NbmTI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=K1i+Vgp+d0Nt/rDzju12lI4q66EYLkSy5JZ076QiXlZZAxL4Zz1kplBNEVKnDnzeL XUtkjJuzzqCLW32Oy7ysq/RMmkHMZuPYez5Wyxk2SMMxQoDGAf6hQweBSn8yFBqUCW ilcp7x5OfSZ83pAAh9tjiC72g4z2cXm2BDzOz97kdkTOZQbGOFPWZ2VNnDrQitQwTS PSxomOd2/RHIlHwBLl4fVNVs7Ua1PUFN+gK1ipO1IHCQiFh6FVawbp4ISyuFZ29+sQ ZJw6ShzgANhvydDXfVdd/XWVF3Es2P54W9xEgMHObmbb/odl0xgFm9ORAQlNoAWC4v O2brtoUrLFkBQ== From: guoren@kernel.org To: guoren@kernel.org Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-arch@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-xtensa@linux-xtensa.org, openrisc@lists.librecores.org, sparclinux@vger.kernel.org, Guo Ren , Arnd Bergmann , Chris Zankel , Max Filippov Subject: [PATCH v5 7/7] xtensa: qspinlock: Add ARCH_USE_QUEUED_SPINLOCKS_XCHG32 Date: Sun, 28 Mar 2021 06:30:28 +0000 Message-Id: <1616913028-83376-8-git-send-email-guoren@kernel.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1616913028-83376-1-git-send-email-guoren@kernel.org> References: <1616913028-83376-1-git-send-email-guoren@kernel.org> Precedence: bulk List-ID: X-Mailing-List: sparclinux@vger.kernel.org From: Guo Ren We don't have native hw xchg16 instruction, so let qspinlock generic code to deal with it. Using the full-word atomic xchg instructions implement xchg16 has the semantic risk for atomic operations. This patch cancels the dependency of on qspinlock generic code on architecture's xchg16. Signed-off-by: Guo Ren Cc: Arnd Bergmann Cc: Chris Zankel Cc: Max Filippov --- arch/xtensa/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/xtensa/Kconfig b/arch/xtensa/Kconfig index 9ad6b7b82707..f19d780638f7 100644 --- a/arch/xtensa/Kconfig +++ b/arch/xtensa/Kconfig @@ -9,6 +9,7 @@ config XTENSA select ARCH_HAS_DMA_SET_UNCACHED if MMU select ARCH_USE_QUEUED_RWLOCKS select ARCH_USE_QUEUED_SPINLOCKS + select ARCH_USE_QUEUED_SPINLOCKS_XCHG32 select ARCH_WANT_FRAME_POINTERS select ARCH_WANT_IPC_PARSE_VERSION select BUILDTIME_TABLE_SORT