From patchwork Sun Aug 9 06:28:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Max Filippov X-Patchwork-Id: 1342573 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=uclibc-ng.org (client-ip=2a00:1828:2000:679::23; helo=helium.openadk.org; envelope-from=devel-bounces@uclibc-ng.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=C6Taq+xJ; dkim-atps=neutral Received: from helium.openadk.org (helium.openadk.org [IPv6:2a00:1828:2000:679::23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4BPTh01jm5z9sR4 for ; Sun, 9 Aug 2020 16:28:47 +1000 (AEST) Received: from helium.openadk.org (localhost [IPv6:::1]) by helium.openadk.org (Postfix) with ESMTP id 64AA3100A1; Sun, 9 Aug 2020 08:28:41 +0200 (CEST) X-Original-To: devel@uclibc-ng.org Delivered-To: devel@helium.openadk.org Received: from mail-pj1-f67.google.com (mail-pj1-f67.google.com [209.85.216.67]) by helium.openadk.org (Postfix) with ESMTPS id 225CB100A1 for ; Sun, 9 Aug 2020 08:28:38 +0200 (CEST) Received: by mail-pj1-f67.google.com with SMTP id 2so3086786pjx.5 for ; Sat, 08 Aug 2020 23:28:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=UcsVBKwWWQoOnYcE5guMpxp9La3HrdAMHJ30d+tHcW4=; b=C6Taq+xJt1/hoOBDvVos0npCaF2v5HXfP0bKv7+ooW8sgVCXYVuJIs304ixGfzfbRO LU+wGTVT+LBBHmed59SbT3V//m8LRKjZ+MK1bG3YAwuTxV/C22uuBGMwW9j300awzjGF nR2Jex7wIpIu16WcWp8WA81HCVE5UuTVccl0P6sGXi343zq4OrejLY0JVQt8Vd2tbsfu GOMsrq4WYscroBqrsa0CkILvE4cZyL8cZn9h1iR15w84HF1snL0i7K0ovkz6idE78vkw jcPQa28vZ4modSxAZE5viL6wFOHRIBb1nbWk8uF7iA4gwuXiIhDXJB6syIfJeqO4P/Nn qtog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=UcsVBKwWWQoOnYcE5guMpxp9La3HrdAMHJ30d+tHcW4=; b=CUeMqsCjdRuHBKuwfR5UHFwTkjaDwF3vJiXC61KiPs7uvx84stEDmubT1+VbNoHYCB gfogARCZEs2Hm3C6TSUjCakYv8GMvOzpyVQUowPhbo8rctYH2KbCmNnZxOrum0JQy43m rBLV6hapYLdHoogHV82rg1lzStrn0dcEKPd43+OtKTq9Kk4RMhKyYkjtc5BspCuj5hMN ojirC0GYeY6OYk72T2VmT5+ANKI9N9aAMnW5BYbVX+UFJO3pnmmSqQb3lE3X5LLiurSH M54Y4TyyFpxFrC3KD2hstmM0ERR6lERSqNyg8ttjT8q2lsZL3EKoda9UksHkPFXhA7Yb k8zw== X-Gm-Message-State: AOAM531NRWBI28msLLHkVTzKoXkq8LfZlDj8viV+Da/haM3zpsD823uN qOZwR5M7HxF3iQD8s0AkGGSLABmg X-Google-Smtp-Source: ABdhPJz04iAVggg10MwAJyT2UGoNEgoJ6x93hfYYjQL4+MZ/QgrIRLQXq4VwN1p+34ajBhC4/LxesQ== X-Received: by 2002:a17:902:7b85:: with SMTP id w5mr18805364pll.22.1596954516391; Sat, 08 Aug 2020 23:28:36 -0700 (PDT) Received: from octofox.metropolis ([2607:fb90:4e2a:9849:ec4c:126e:c341:8919]) by smtp.gmail.com with ESMTPSA id k21sm16559021pgl.0.2020.08.08.23.28.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 08 Aug 2020 23:28:35 -0700 (PDT) From: Max Filippov To: devel@uclibc-ng.org Date: Sat, 8 Aug 2020 23:28:17 -0700 Message-Id: <20200809062817.19107-1-jcmvbkbc@gmail.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Subject: [uclibc-ng-devel] [PATCH] xtensa: add exclusive access support X-BeenThere: devel@uclibc-ng.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: uClibc-ng Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-xtensa@linux-xtensa.org Errors-To: devel-bounces@uclibc-ng.org Sender: "devel" Add XCHAL definitions for S32C1I and EXCLUSIVE options to xtensa-config.h, include it in places that implement atomic operations and add implementations with exclusive access option opcodes. Signed-off-by: Max Filippov --- libc/sysdeps/linux/xtensa/bits/atomic.h | 128 ++++++++++++++++++ .../sysdeps/linux/xtensa/bits/xtensa-config.h | 6 + .../linuxthreads/sysdeps/xtensa/pt-machine.h | 56 ++++++++ .../nptl/sysdeps/xtensa/pthread_spin_lock.S | 16 +++ .../sysdeps/xtensa/pthread_spin_trylock.S | 17 +++ 5 files changed, 223 insertions(+) diff --git a/libc/sysdeps/linux/xtensa/bits/atomic.h b/libc/sysdeps/linux/xtensa/bits/atomic.h index efc027d1ac25..18b809998054 100644 --- a/libc/sysdeps/linux/xtensa/bits/atomic.h +++ b/libc/sysdeps/linux/xtensa/bits/atomic.h @@ -18,6 +18,7 @@ #ifndef _BITS_ATOMIC_H #define _BITS_ATOMIC_H 1 +#include #include typedef int32_t atomic32_t; @@ -50,6 +51,128 @@ typedef uintmax_t uatomic_max_t; #define __arch_compare_and_exchange_bool_16_rel(mem, newval, oldval) \ (abort (), 0) +#if XCHAL_HAVE_EXCLUSIVE + +/* Atomically store NEWVAL in *MEM if *MEM is equal to OLDVAL. + Return the old *MEM value. */ + +#define __arch_compare_and_exchange_val_32_acq(mem, newval, oldval) \ + ({__typeof__(*(mem)) __tmp, __value; \ + __asm__ __volatile__( \ + " memw \n" \ + "1: l32ex %0, %2 \n" \ + " bne %0, %4, 2f \n" \ + " mov %1, %3 \n" \ + " s32ex %1, %2 \n" \ + " getex %1 \n" \ + " beqz %1, 1b \n" \ + " memw \n" \ + "2: \n" \ + : "=&a" (__value), "=&a" (__tmp) \ + : "a" (mem), "a" (newval), "a" (oldval) \ + : "memory" ); \ + __value; \ + }) + +/* Atomically store NEWVAL in *MEM if *MEM is equal to OLDVAL. + Return zero if *MEM was changed or non-zero if no exchange happened. */ + +#define __arch_compare_and_exchange_bool_32_acq(mem, newval, oldval) \ + ({__typeof__(*(mem)) __tmp, __value; \ + __asm__ __volatile__( \ + " memw \n" \ + "1: l32ex %0, %2 \n" \ + " sub %0, %4, %0 \n" \ + " bnez %0, 2f \n" \ + " mov %1, %3 \n" \ + " s32ex %1, %2 \n" \ + " getex %1 \n" \ + " beqz %1, 1b \n" \ + " movi %0, 0 \n" \ + " memw \n" \ + "2: \n" \ + : "=&a" (__value), "=&a" (__tmp) \ + : "a" (mem), "a" (newval), "a" (oldval) \ + : "memory" ); \ + __value != 0; \ + }) + +/* Store NEWVALUE in *MEM and return the old value. */ + +#define __arch_exchange_32_acq(mem, newval) \ + ({__typeof__(*(mem)) __tmp, __value; \ + __asm__ __volatile__( \ + " memw \n" \ + "1: l32ex %0, %2 \n" \ + " mov %1, %3 \n" \ + " s32ex %1, %2 \n" \ + " getex %1 \n" \ + " beqz %1, 1b \n" \ + " memw \n" \ + : "=&a" (__value), "=&a" (__tmp) \ + : "a" (mem), "a" (newval) \ + : "memory" ); \ + __value; \ + }) + +/* Add VALUE to *MEM and return the old value of *MEM. */ + +#define __arch_atomic_exchange_and_add_32(mem, value) \ + ({__typeof__(*(mem)) __tmp, __value; \ + __asm__ __volatile__( \ + " memw \n" \ + "1: l32ex %0, %2 \n" \ + " add %1, %0, %3 \n" \ + " s32ex %1, %2 \n" \ + " getex %1 \n" \ + " beqz %1, 1b \n" \ + " memw \n" \ + : "=&a" (__value), "=&a" (__tmp) \ + : "a" (mem), "a" (value) \ + : "memory" ); \ + __value; \ + }) + +/* Subtract VALUE from *MEM and return the old value of *MEM. */ + +#define __arch_atomic_exchange_and_sub_32(mem, value) \ + ({__typeof__(*(mem)) __tmp, __value; \ + __asm__ __volatile__( \ + " memw \n" \ + "1: l32ex %0, %2 \n" \ + " sub %1, %0, %3 \n" \ + " s32ex %1, %2 \n" \ + " getex %1 \n" \ + " beqz %1, 1b \n" \ + " memw \n" \ + : "=&a" (__value), "=&a" (__tmp) \ + : "a" (mem), "a" (value) \ + : "memory" ); \ + __tmp; \ + }) + +/* Decrement *MEM if it is > 0, and return the old value. */ + +#define __arch_atomic_decrement_if_positive_32(mem) \ + ({__typeof__(*(mem)) __tmp, __value; \ + __asm__ __volatile__( \ + " memw \n" \ + "1: l32ex %0, %2 \n" \ + " blti %0, 1, 2f \n" \ + " addi %1, %0, -1 \n" \ + " s32ex %1, %2 \n" \ + " getex %1 \n" \ + " beqz %1, 1b \n" \ + " memw \n" \ + "2: \n" \ + : "=&a" (__value), "=&a" (__tmp) \ + : "a" (mem) \ + : "memory" ); \ + __value; \ + }) + +#elif XCHAL_HAVE_S32C1I + /* Atomically store NEWVAL in *MEM if *MEM is equal to OLDVAL. Return the old *MEM value. */ @@ -156,6 +279,11 @@ typedef uintmax_t uatomic_max_t; __value; \ }) +#else + +#error No hardware atomic operations + +#endif /* These are the preferred public interfaces: */ diff --git a/libc/sysdeps/linux/xtensa/bits/xtensa-config.h b/libc/sysdeps/linux/xtensa/bits/xtensa-config.h index 2e60af936e6e..b99928b1e058 100644 --- a/libc/sysdeps/linux/xtensa/bits/xtensa-config.h +++ b/libc/sysdeps/linux/xtensa/bits/xtensa-config.h @@ -43,4 +43,10 @@ #undef XCHAL_NUM_AREGS #define XCHAL_NUM_AREGS 64 +#undef XCHAL_HAVE_S32C1I +#define XCHAL_HAVE_S32C1I 1 + +#undef XCHAL_HAVE_EXCLUSIVE +#define XCHAL_HAVE_EXCLUSIVE 0 + #endif /* !XTENSA_CONFIG_H */ diff --git a/libpthread/linuxthreads/sysdeps/xtensa/pt-machine.h b/libpthread/linuxthreads/sysdeps/xtensa/pt-machine.h index 82d9b540c611..0b7f58b635f3 100644 --- a/libpthread/linuxthreads/sysdeps/xtensa/pt-machine.h +++ b/libpthread/linuxthreads/sysdeps/xtensa/pt-machine.h @@ -21,6 +21,7 @@ #ifndef _PT_MACHINE_H #define _PT_MACHINE_H 1 +#include #include #include @@ -34,6 +35,55 @@ extern long int testandset (int *spinlock); extern int __compare_and_swap (long int *p, long int oldval, long int newval); +#if XCHAL_HAVE_EXCLUSIVE + +/* Spinlock implementation; required. */ +PT_EI long int +testandset (int *spinlock) +{ + unsigned long tmp; + __asm__ volatile ( +" memw \n" +"1: l32ex %0, %1 \n" +" bnez %0, 2f \n" +" movi %0, 1 \n" +" s32ex %0, %1 \n" +" getex %0 \n" +" beqz %0, 1b \n" +" movi %0, 0 \n" +" memw \n" +"2: \n" + : "=&a" (tmp) + : "a" (spinlock) + : "memory" + ); + return tmp; +} + +PT_EI int +__compare_and_swap (long int *p, long int oldval, long int newval) +{ + unsigned long tmp; + unsigned long value; + __asm__ volatile ( +" memw \n" +"1: l32ex %0, %2 \n" +" bne %0, %4, 2f \n" +" mov %1, %3 \n" +" s32ex %1, %2 \n" +" getex %1 \n" +" beqz %1, 1b \n" +" memw \n" +"2: \n" + : "=&a" (tmp), "=&a" (value) + : "a" (p), "a" (newval), "a" (oldval) + : "memory" ); + + return tmp == oldval; +} + +#elif XCHAL_HAVE_S32C1I + /* Spinlock implementation; required. */ PT_EI long int testandset (int *spinlock) @@ -71,6 +121,12 @@ __compare_and_swap (long int *p, long int oldval, long int newval) return tmp == oldval; } +#else + +#error No hardware atomic operations + +#endif + /* Get some notion of the current stack. Need not be exactly the top of the stack, just something somewhere in the current frame. */ #define CURRENT_STACK_FRAME __builtin_frame_address (0) diff --git a/libpthread/nptl/sysdeps/xtensa/pthread_spin_lock.S b/libpthread/nptl/sysdeps/xtensa/pthread_spin_lock.S index 3386afae9e58..3faac36da12b 100644 --- a/libpthread/nptl/sysdeps/xtensa/pthread_spin_lock.S +++ b/libpthread/nptl/sysdeps/xtensa/pthread_spin_lock.S @@ -15,16 +15,32 @@ License along with the GNU C Library; see the file COPYING.LIB. If not, see .  */ +#include #include .text ENTRY (pthread_spin_lock) +#if XCHAL_HAVE_EXCLUSIVE + memw +1: l32ex a3, a2 + bnez a3, 1b + movi a3, 1 + s32ex a3, a2 + getex a3 + beqz a3, 1b + memw +#elif XCHAL_HAVE_S32C1I movi a3, 0 wsr a3, scompare1 movi a3, 1 1: s32c1i a3, a2, 0 bnez a3, 1b +#else + +#error No hardware atomic operations + +#endif movi a2, 0 abi_ret diff --git a/libpthread/nptl/sysdeps/xtensa/pthread_spin_trylock.S b/libpthread/nptl/sysdeps/xtensa/pthread_spin_trylock.S index 72b2dda92310..0669682ec9bd 100644 --- a/libpthread/nptl/sysdeps/xtensa/pthread_spin_trylock.S +++ b/libpthread/nptl/sysdeps/xtensa/pthread_spin_trylock.S @@ -17,15 +17,32 @@ #define _ERRNO_H 1 #include +#include #include .text ENTRY (pthread_spin_trylock) +#if XCHAL_HAVE_EXCLUSIVE + memw + l32ex a3, a2 + bnez a3, 1f + movi a3, 1 + s32ex a3, a2 + getex a3 + addi a3, a3, -1 + memw +1: +#elif XCHAL_HAVE_S32C1I movi a3, 0 wsr a3, scompare1 movi a3, 1 s32c1i a3, a2, 0 +#else + +#error No hardware atomic operations + +#endif movi a2, EBUSY moveqz a2, a3, a3