From patchwork Sun Nov 27 22:57:20 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Modra X-Patchwork-Id: 127905 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) by ozlabs.org (Postfix) with SMTP id 5728FB6FA0 for ; Mon, 28 Nov 2011 09:57:47 +1100 (EST) Received: (qmail 7403 invoked by alias); 27 Nov 2011 22:57:45 -0000 Received: (qmail 7394 invoked by uid 22791); 27 Nov 2011 22:57:43 -0000 X-SWARE-Spam-Status: No, hits=-2.6 required=5.0 tests=AWL, BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_LOW X-Spam-Check-By: sourceware.org Received: from mail-iy0-f175.google.com (HELO mail-iy0-f175.google.com) (209.85.210.175) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Sun, 27 Nov 2011 22:57:27 +0000 Received: by iahk25 with SMTP id k25so8762025iah.20 for ; Sun, 27 Nov 2011 14:57:27 -0800 (PST) Received: by 10.42.154.7 with SMTP id o7mr22638987icw.48.1322434646923; Sun, 27 Nov 2011 14:57:26 -0800 (PST) Received: from bubble.grove.modra.org ([115.187.252.19]) by mx.google.com with ESMTPS id jm11sm37444878ibb.1.2011.11.27.14.57.24 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 27 Nov 2011 14:57:26 -0800 (PST) Received: by bubble.grove.modra.org (Postfix, from userid 1000) id 08066170C2BF; Mon, 28 Nov 2011 09:27:20 +1030 (CST) Date: Mon, 28 Nov 2011 09:27:20 +1030 From: Alan Modra To: Jakub Jelinek Cc: gcc-patches@gcc.gnu.org Subject: Use atomics in libgomp mutex Message-ID: <20111127225720.GB7827@bubble.grove.modra.org> Mail-Followup-To: Jakub Jelinek , gcc-patches@gcc.gnu.org References: <20111125000328.GD5085@bubble.grove.modra.org> <20111125073839.GN27242@tyan-ft48-01.lab.bos.redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20111125073839.GN27242@tyan-ft48-01.lab.bos.redhat.com> User-Agent: Mutt/1.5.20 (2009-06-14) X-IsSubscribed: yes Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org This is the mutex part. Depends on http://gcc.gnu.org/ml/gcc-patches/2011-11/msg02454.html for MEMMODEL_* values. * config/linux/mutex.h: Use atomic rather than sync builtins. * config/linux/mutex.c: Likewise. Comment. Use -1 for waiting state. * config/linux/omp-lock.h: Comment fix. * config/linux/arm/mutex.h: Delete. * config/linux/powerpc/mutex.h: Delete. * config/linux/ia64/mutex.h: Delete. * config/linux/mips/mutex.h: Delete. Index: libgomp/config/linux/mutex.h =================================================================== --- libgomp/config/linux/mutex.h (revision 181718) +++ libgomp/config/linux/mutex.h (working copy) @@ -33,39 +33,35 @@ typedef int gomp_mutex_t; #define GOMP_MUTEX_INIT_0 1 -static inline void gomp_mutex_init (gomp_mutex_t *mutex) +extern void gomp_mutex_lock_slow (gomp_mutex_t *mutex, int); +extern void gomp_mutex_unlock_slow (gomp_mutex_t *mutex); + +static inline void +gomp_mutex_init (gomp_mutex_t *mutex) { *mutex = 0; } -extern void gomp_mutex_lock_slow (gomp_mutex_t *mutex, int); -static inline void gomp_mutex_lock (gomp_mutex_t *mutex) +static inline void +gomp_mutex_destroy (gomp_mutex_t *mutex) { - int oldval = __sync_val_compare_and_swap (mutex, 0, 1); - if (__builtin_expect (oldval, 0)) - gomp_mutex_lock_slow (mutex, oldval); } -extern void gomp_mutex_unlock_slow (gomp_mutex_t *mutex); -static inline void gomp_mutex_unlock (gomp_mutex_t *mutex) +static inline void +gomp_mutex_lock (gomp_mutex_t *mutex) { - /* Warning: By definition __sync_lock_test_and_set() does not have - proper memory barrier semantics for a mutex unlock operation. - However, this default implementation is written assuming that it - does, which is true for some targets. - - Targets that require additional memory barriers before - __sync_lock_test_and_set to achieve the release semantics of - mutex unlock, are encouraged to include - "config/linux/ia64/mutex.h" in a target specific mutex.h instead - of using this file. */ - int val = __sync_lock_test_and_set (mutex, 0); - if (__builtin_expect (val > 1, 0)) - gomp_mutex_unlock_slow (mutex); + int oldval = 0; + __atomic_compare_exchange_4 (mutex, &oldval, 1, false, + MEMMODEL_ACQUIRE, MEMMODEL_RELAXED); + if (__builtin_expect (oldval != 0, 0)) + gomp_mutex_lock_slow (mutex, oldval); } -static inline void gomp_mutex_destroy (gomp_mutex_t *mutex) +static inline void +gomp_mutex_unlock (gomp_mutex_t *mutex) { + int wait = __atomic_exchange_4 (mutex, 0, MEMMODEL_RELEASE); + if (__builtin_expect (wait < 0, 0)) + gomp_mutex_unlock_slow (mutex); } - #endif /* GOMP_MUTEX_H */ Index: libgomp/config/linux/mutex.c =================================================================== --- libgomp/config/linux/mutex.c (revision 181718) +++ libgomp/config/linux/mutex.c (working copy) @@ -34,25 +34,33 @@ long int gomp_futex_wait = FUTEX_WAIT | void gomp_mutex_lock_slow (gomp_mutex_t *mutex, int oldval) { + /* First loop spins a while. */ while (oldval == 1) { if (do_spin (mutex, 1)) { - oldval = __sync_lock_test_and_set (mutex, 2); + /* Spin timeout, nothing changed. Set waiting flag. */ + oldval = __atomic_exchange_4 (mutex, -1, MEMMODEL_ACQUIRE); if (oldval == 0) return; - futex_wait (mutex, 2); + futex_wait (mutex, -1); break; } else { - oldval = __sync_val_compare_and_swap (mutex, 0, 1); + /* Something changed. If now unlocked, we're good to go. */ + oldval = 0; + __atomic_compare_exchange_4 (mutex, &oldval, 1, false, + MEMMODEL_ACQUIRE, MEMMODEL_RELAXED); if (oldval == 0) return; } } - while ((oldval = __sync_lock_test_and_set (mutex, 2))) - do_wait (mutex, 2); + + /* Second loop waits until mutex is unlocked. We always exit this + loop with wait flag set, so next unlock will awaken a thread. */ + while ((oldval = __atomic_exchange_4 (mutex, -1, MEMMODEL_ACQUIRE))) + do_wait (mutex, -1); } void Index: libgomp/config/linux/omp-lock.h =================================================================== --- libgomp/config/linux/omp-lock.h (revision 181718) +++ libgomp/config/linux/omp-lock.h (working copy) @@ -3,8 +3,8 @@ structures without polluting the namespace. When using the Linux futex primitive, non-recursive locks require - only one int. Recursive locks require we identify the owning task - and so require one int and a pointer. */ + one int. Recursive locks require we identify the owning task + and so require in addition one int and a pointer. */ typedef int omp_lock_t; typedef struct { int lock, count; void *owner; } omp_nest_lock_t; Index: libgomp/config/linux/arm/mutex.h =================================================================== --- libgomp/config/linux/arm/mutex.h (revision 181718) +++ libgomp/config/linux/arm/mutex.h (working copy) @@ -1,28 +0,0 @@ -/* Copyright (C) 2010 Free Software Foundation, Inc. - Contributed by ARM Ltd. - - This file is part of the GNU OpenMP Library (libgomp). - - Libgomp is free software; you can redistribute it and/or modify it - under the terms of the GNU General Public License as published by - the Free Software Foundation; either version 3, or (at your option) - any later version. - - Libgomp is distributed in the hope that it will be useful, but WITHOUT ANY - WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS - FOR A PARTICULAR PURPOSE. See the GNU General Public License for - more details. - - Under Section 7 of GPL version 3, you are granted additional - permissions described in the GCC Runtime Library Exception, version - 3.1, as published by the Free Software Foundation. - - You should have received a copy of the GNU General Public License and - a copy of the GCC Runtime Library Exception along with this program; - see the files COPYING3 and COPYING.RUNTIME respectively. If not, see - . */ - -/* ARM needs the same correct usage of __sync_synchronize and - __sync_lock_test_and_set as ia64. So we just use its mutex.h. */ - -#include "config/linux/ia64/mutex.h" Index: libgomp/config/linux/powerpc/mutex.h =================================================================== --- libgomp/config/linux/powerpc/mutex.h (revision 181718) +++ libgomp/config/linux/powerpc/mutex.h (working copy) @@ -1,2 +0,0 @@ -/* On PowerPC __sync_lock_test_and_set isn't a full barrier. */ -#include "config/linux/ia64/mutex.h" Index: libgomp/config/linux/ia64/mutex.h =================================================================== --- libgomp/config/linux/ia64/mutex.h (revision 181718) +++ libgomp/config/linux/ia64/mutex.h (working copy) @@ -1,66 +0,0 @@ -/* Copyright (C) 2005, 2008, 2009, 2011 Free Software Foundation, Inc. - Contributed by Richard Henderson . - - This file is part of the GNU OpenMP Library (libgomp). - - Libgomp is free software; you can redistribute it and/or modify it - under the terms of the GNU General Public License as published by - the Free Software Foundation; either version 3, or (at your option) - any later version. - - Libgomp is distributed in the hope that it will be useful, but WITHOUT ANY - WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS - FOR A PARTICULAR PURPOSE. See the GNU General Public License for - more details. - - Under Section 7 of GPL version 3, you are granted additional - permissions described in the GCC Runtime Library Exception, version - 3.1, as published by the Free Software Foundation. - - You should have received a copy of the GNU General Public License and - a copy of the GCC Runtime Library Exception along with this program; - see the files COPYING3 and COPYING.RUNTIME respectively. If not, see - . */ - -/* This is a Linux specific implementation of a mutex synchronization - mechanism for libgomp. This type is private to the library. This - implementation uses atomic instructions and the futex syscall. */ - -#ifndef GOMP_MUTEX_H -#define GOMP_MUTEX_H 1 - -typedef int gomp_mutex_t; - -#define GOMP_MUTEX_INIT_0 1 - -static inline void gomp_mutex_init (gomp_mutex_t *mutex) -{ - *mutex = 0; -} - -extern void gomp_mutex_lock_slow (gomp_mutex_t *mutex, int); -static inline void gomp_mutex_lock (gomp_mutex_t *mutex) -{ - int oldval = __sync_val_compare_and_swap (mutex, 0, 1); - if (__builtin_expect (oldval, 0)) - gomp_mutex_lock_slow (mutex, oldval); -} - -extern void gomp_mutex_unlock_slow (gomp_mutex_t *mutex); - -/* IA64 needs a __sync_synchronize call before __sync_lock_test_and_set - because __sync_lock_test_and_set is not a full memory fence. */ -static inline void gomp_mutex_unlock (gomp_mutex_t *mutex) -{ - int val; - __sync_synchronize (); - val = __sync_lock_test_and_set (mutex, 0); - if (__builtin_expect (val > 1, 0)) - gomp_mutex_unlock_slow (mutex); -} - -static inline void gomp_mutex_destroy (gomp_mutex_t *mutex) -{ -} - -#endif /* GOMP_MUTEX_H */ Index: libgomp/config/linux/mips/mutex.h =================================================================== --- libgomp/config/linux/mips/mutex.h (revision 181718) +++ libgomp/config/linux/mips/mutex.h (working copy) @@ -1,27 +0,0 @@ -/* Copyright (C) 2009 Free Software Foundation, Inc. - - This file is part of the GNU OpenMP Library (libgomp). - - Libgomp is free software; you can redistribute it and/or modify it - under the terms of the GNU General Public License as published by - the Free Software Foundation; either version 3, or (at your option) - any later version. - - Libgomp is distributed in the hope that it will be useful, but WITHOUT ANY - WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS - FOR A PARTICULAR PURPOSE. See the GNU General Public License for - more details. - - Under Section 7 of GPL version 3, you are granted additional - permissions described in the GCC Runtime Library Exception, version - 3.1, as published by the Free Software Foundation. - - You should have received a copy of the GNU General Public License and - a copy of the GCC Runtime Library Exception along with this program; - see the files COPYING3 and COPYING.RUNTIME respectively. If not, see - . */ - -/* MIPS needs the same correct usage of __sync_synchronize and - __sync_lock_test_and_set as ia64. So we just use its mutex.h. */ - -#include "config/linux/ia64/mutex.h"