From patchwork Fri Jun 26 19:44:58 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 488978 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id B2F3F140285 for ; Sat, 27 Jun 2015 05:45:23 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=sourceware.org header.i=@sourceware.org header.b=o8/LsHIT; dkim-atps=neutral DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:message-id:date:from:mime-version:to:subject :content-type:content-transfer-encoding; q=dns; s=default; b=c7j g7gtWSjDUtSEBfZnt7Ck+tt2GUa6xBDcOetsc3h28SQvOXt89vVSaJb9GzfddPpM 56n4vo9hrnjM24JqUuwcPrItba5gccI61tcIOVu5qpSWL/W9jcRyk3AL0WvaxigH sVqTEDcyiQOvVMW+sUj8LJre8aGzQxGBfIvLPz/A= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:message-id:date:from:mime-version:to:subject :content-type:content-transfer-encoding; s=default; bh=Tj/G9Se4B 4qm41fspNlqHnUhqEI=; b=o8/LsHITHSpyz/BsKdih1ORU06m+kpE3kE3ZJcw5d a7nEr8bgz0l3e8+jFADnvdtZKtFB8lKm5pBLIotVL4SaLYqdaLuqtHls6q1sTB6A 99OAC/kHGhAIjUOHqrZvKS/YPSwFT/nrXgJGF9mYa/YyTG1XEDvBuFhGKcUa0Ay1 qY= Received: (qmail 90939 invoked by alias); 26 Jun 2015 19:45:15 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 90914 invoked by uid 89); 26 Jun 2015 19:45:14 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-0.9 required=5.0 tests=AWL, BAYES_50, RCVD_IN_DNSWL_LOW, SPF_PASS autolearn=ham version=3.3.2 X-HELO: mail-yk0-f182.google.com X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:content-type:content-transfer-encoding; bh=/7lPdOArHTknIBU2Ed1nYB9bpRNzLIzFy8PeWLnCxVs=; b=Kt2N6xinl8Vh+3ImV8WeUKpxJZB9v1hZl0ysrV6nplw9tZo1EV0MqKZsVDx8epKgvS OuKp4GXi4ehsXjRiTETB9/zFA8GJth7lW1qADUFIqev4Y8YvnUQFSzO5ScUQelF7d3zo uqBnB1oJr4P9KFwXQrGymXhVxHpzTAu2ZHU2NnhAVs7vudv3zi7t4a9FfrHfxJCcQcS+ Omm6cDK0jLVqOK4vn3YBQVMfSafCvk1Lqr527lBQ4KB2Z5j8yN7RtfjRD2Uro0L48pEL EvqpEUbUGTlZjER6OzpqcM6SKScIzIq2cyqbTq6WMLbKxnFkbeccH9dT8f+TidRl3tJ6 /Sww== X-Gm-Message-State: ALoCoQkzrzkrH+nMFu6JbEuJg8+eizRVs/9B+zrGZEcTAIHkuqT60v14odHqfq9QOFkejKgPgtgs X-Received: by 10.13.228.71 with SMTP id n68mr4116216ywe.165.1435347901376; Fri, 26 Jun 2015 12:45:01 -0700 (PDT) Message-ID: <558DABBA.9090600@linaro.org> Date: Fri, 26 Jun 2015 16:44:58 -0300 From: Adhemerval Zanella User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: GNU C Library Subject: [PATCH 03/08] nptl: x86_64: Fix Race conditions in pthread cancellation (BZ#12683) This patches adds the x86_64 modification required for the BZ#12683 fix. It basically removes the enable_asynccancel/disable_asynccancel function usage on code used on x86_64, provide a arch-specific symbol that contains global markers to be used in SIGCANCEL handler, and remove x86_64 assembly pthread conditional and semaphore code to use default one (that already contains the cancel call fixes). It assumes that the pthread_cond_{timed}wait assembly implementation are removed. --- * sysdeps/unix/sysv/linux/x86_64/cancellation.S: Remove file. * sysdeps/unix/sysv/linux/x86_64/libc-cancellation.S: Remove file. * sysdeps/unix/sysv/linux/x86_64/librt-cancellation.S: Remove file. * sysdeps/unix/sysv/linux/x86_64/lowlevellock.h (lll_wait_tid): Use cancellable futex wait call. * sysdeps/unix/sysv/linux/x86_64/sysdep-cancel.h (PSEUDO): Likewise. * sysdeps/unix/sysv/linux/x86_64/syscall_cancel.S: New file. * sysdeps/unix/sysv/linux/x86_64/sysdep-cancel.h (PSEUDO): Redefine to call __syscall_cancel function for cancellable syscalls. (CENABLE): Remove definition. (CDISABLE): Likewise. (__pthread_get_ip): Add implementation. * sysdeps/unix/sysv/linux/x86_64/sysdep.h (SYSCALL_CANCEL_ERROR): New define. (SYSCALL_CANCEL_ERRNO): Likewise. * sysdeps/x86_64/nptl/tcb-offsets.sym [TCB_CANCELING_BITMASK]: Remove. * sysdeps/x86_64/nptl/tls.h (THREAD_ATOMIC_BIT_SET): Remove macro. --- diff --git a/sysdeps/unix/sysv/linux/x86_64/cancellation.S b/sysdeps/unix/sysv/linux/x86_64/cancellation.S deleted file mode 100644 index 4c34beb..0000000 --- a/sysdeps/unix/sysv/linux/x86_64/cancellation.S +++ /dev/null @@ -1,117 +0,0 @@ -/* Copyright (C) 2009-2015 Free Software Foundation, Inc. - This file is part of the GNU C Library. - Contributed by Ulrich Drepper , 2009. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - . */ - -#include -#include -#include -#include "lowlevellock.h" - -#if IS_IN (libpthread) -# if defined SHARED && !defined NO_HIDDEN -# define __pthread_unwind __GI___pthread_unwind -# endif -#else -# ifndef SHARED - .weak __pthread_unwind -# endif -#endif - - -#ifdef __ASSUME_PRIVATE_FUTEX -# define LOAD_PRIVATE_FUTEX_WAIT(reg) \ - movl $(FUTEX_WAIT | FUTEX_PRIVATE_FLAG), reg -#else -# if FUTEX_WAIT == 0 -# define LOAD_PRIVATE_FUTEX_WAIT(reg) \ - movl %fs:PRIVATE_FUTEX, reg -# else -# define LOAD_PRIVATE_FUTEX_WAIT(reg) \ - movl %fs:PRIVATE_FUTEX, reg ; \ - orl $FUTEX_WAIT, reg -# endif -#endif - -/* It is crucial that the functions in this file don't modify registers - other than %rax and %r11. The syscall wrapper code depends on this - because it doesn't explicitly save the other registers which hold - relevant values. */ - .text - - .hidden __pthread_enable_asynccancel -ENTRY(__pthread_enable_asynccancel) - movl %fs:CANCELHANDLING, %eax -2: movl %eax, %r11d - orl $TCB_CANCELTYPE_BITMASK, %r11d - cmpl %eax, %r11d - je 1f - - lock - cmpxchgl %r11d, %fs:CANCELHANDLING - jnz 2b - - andl $(TCB_CANCELSTATE_BITMASK|TCB_CANCELTYPE_BITMASK|TCB_CANCELED_BITMASK|TCB_EXITING_BITMASK|TCB_CANCEL_RESTMASK|TCB_TERMINATED_BITMASK), %r11d - cmpl $(TCB_CANCELTYPE_BITMASK|TCB_CANCELED_BITMASK), %r11d - je 3f - -1: ret - -3: subq $8, %rsp - cfi_adjust_cfa_offset(8) - LP_OP(mov) $TCB_PTHREAD_CANCELED, %fs:RESULT - lock - orl $TCB_EXITING_BITMASK, %fs:CANCELHANDLING - mov %fs:CLEANUP_JMP_BUF, %RDI_LP -#ifdef SHARED - call __pthread_unwind@PLT -#else - call __pthread_unwind -#endif - hlt -END(__pthread_enable_asynccancel) - - - .hidden __pthread_disable_asynccancel -ENTRY(__pthread_disable_asynccancel) - testl $TCB_CANCELTYPE_BITMASK, %edi - jnz 1f - - movl %fs:CANCELHANDLING, %eax -2: movl %eax, %r11d - andl $~TCB_CANCELTYPE_BITMASK, %r11d - lock - cmpxchgl %r11d, %fs:CANCELHANDLING - jnz 2b - - movl %r11d, %eax -3: andl $(TCB_CANCELING_BITMASK|TCB_CANCELED_BITMASK), %eax - cmpl $TCB_CANCELING_BITMASK, %eax - je 4f -1: ret - - /* Performance doesn't matter in this loop. We will - delay until the thread is canceled. And we will unlikely - enter the loop twice. */ -4: mov %fs:0, %RDI_LP - movl $__NR_futex, %eax - xorq %r10, %r10 - addq $CANCELHANDLING, %rdi - LOAD_PRIVATE_FUTEX_WAIT (%esi) - syscall - movl %fs:CANCELHANDLING, %eax - jmp 3b -END(__pthread_disable_asynccancel) diff --git a/sysdeps/unix/sysv/linux/x86_64/libc-cancellation.S b/sysdeps/unix/sysv/linux/x86_64/libc-cancellation.S deleted file mode 100644 index dbb39f3..0000000 --- a/sysdeps/unix/sysv/linux/x86_64/libc-cancellation.S +++ /dev/null @@ -1,21 +0,0 @@ -/* Copyright (C) 2009-2015 Free Software Foundation, Inc. - This file is part of the GNU C Library. - Contributed by Ulrich Drepper , 2009. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - . */ - -#define __pthread_enable_asynccancel __libc_enable_asynccancel -#define __pthread_disable_asynccancel __libc_disable_asynccancel -#include "cancellation.S" diff --git a/sysdeps/unix/sysv/linux/x86_64/librt-cancellation.S b/sysdeps/unix/sysv/linux/x86_64/librt-cancellation.S deleted file mode 100644 index 40b2986..0000000 --- a/sysdeps/unix/sysv/linux/x86_64/librt-cancellation.S +++ /dev/null @@ -1,21 +0,0 @@ -/* Copyright (C) 2009-2015 Free Software Foundation, Inc. - This file is part of the GNU C Library. - Contributed by Ulrich Drepper , 2009. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - . */ - -#define __pthread_enable_asynccancel __librt_enable_asynccancel -#define __pthread_disable_asynccancel __librt_disable_asynccancel -#include "cancellation.S" diff --git a/sysdeps/unix/sysv/linux/x86_64/lowlevellock.h b/sysdeps/unix/sysv/linux/x86_64/lowlevellock.h index 573b48c..4ef8d58 100644 --- a/sysdeps/unix/sysv/linux/x86_64/lowlevellock.h +++ b/sysdeps/unix/sysv/linux/x86_64/lowlevellock.h @@ -309,10 +309,10 @@ extern int __lll_timedlock_elision (int *futex, short *adapt_count, afterwards. The kernel up to version 3.16.3 does not use the private futex operations for futex wake-up when the clone terminates. */ #define lll_wait_tid(tid) \ - do { \ - __typeof (tid) __tid; \ - while ((__tid = (tid)) != 0) \ - lll_futex_wait (&(tid), __tid, LLL_SHARED);\ + do { \ + __typeof (tid) __tid; \ + while ((__tid = (tid)) != 0) \ + lll_futex_wait_cancel (&(tid), __tid, LLL_SHARED); \ } while (0) extern int __lll_timedwait_tid (int *, const struct timespec *) diff --git a/sysdeps/unix/sysv/linux/x86_64/syscall_cancel.S b/sysdeps/unix/sysv/linux/x86_64/syscall_cancel.S new file mode 100644 index 0000000..3c44069 --- /dev/null +++ b/sysdeps/unix/sysv/linux/x86_64/syscall_cancel.S @@ -0,0 +1,62 @@ +/* Cancellable syscall wrapper - x86_64 version. + Copyright (C) 2014 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +/* long int [rax] __syscall_cancel_arch (int *cancelhandling [%rdi], + long int nr [%rsi], + long int arg1 [%rdx], + long int arg2 [%rcx], + long int arg3 [%r8], + long int arg4 [%r9], + long int arg5 [SP+8], + long int arg6 [SP+16]) */ + +ENTRY (__syscall_cancel_arch) + + .globl __syscall_cancel_arch_start + .type __syscall_cancel_arch_start,@function +__syscall_cancel_arch_start: + + /* if (*cancelhandling & CANCELED_BITMASK) + __syscall_do_cancel() */ + mov (%rdi),%eax + testb $4, (%rdi) + jne __syscall_do_cancel + + /* Issue a 6 argument syscall, the nr [%rax] being the syscall + number. */ + mov %rdi,%r11 + mov %rsi,%rax + mov %rdx,%rdi + mov %rcx,%rsi + mov %r8,%rdx + mov %r9,%r10 + mov 8(%rsp),%r8 + mov 16(%rsp),%r9 + mov %r11,8(%rsp) + syscall + + .globl __syscall_cancel_arch_end + .type __syscall_cancel_arch_end,@function +__syscall_cancel_arch_end: + + ret + +END (__syscall_cancel_arch) +libc_hidden_def (__syscall_cancel_arch) diff --git a/sysdeps/unix/sysv/linux/x86_64/sysdep-cancel.h b/sysdeps/unix/sysv/linux/x86_64/sysdep-cancel.h index 6436ff0..a216364 100644 --- a/sysdeps/unix/sysv/linux/x86_64/sysdep-cancel.h +++ b/sysdeps/unix/sysv/linux/x86_64/sysdep-cancel.h @@ -24,53 +24,43 @@ #if IS_IN (libc) || IS_IN (libpthread) || IS_IN (librt) -/* The code to disable cancellation depends on the fact that the called - functions are special. They don't modify registers other than %rax - and %r11 if they return. Therefore we don't have to preserve other - registers around these calls. */ +# if IS_IN (libc) +# define JMP_SYSCALL_CANCEL HIDDEN_JUMPTARGET(__syscall_cancel) +# else +# define JMP_SYSCALL_CANCEL __syscall_cancel@plt +# endif + # undef PSEUDO # define PSEUDO(name, syscall_name, args) \ .text; \ ENTRY (name) \ SINGLE_THREAD_P; \ jne L(pseudo_cancel); \ - .type __##syscall_name##_nocancel,@function; \ - .globl __##syscall_name##_nocancel; \ - __##syscall_name##_nocancel: \ DO_CALL (syscall_name, args); \ cmpq $-4095, %rax; \ jae SYSCALL_ERROR_LABEL; \ ret; \ - .size __##syscall_name##_nocancel,.-__##syscall_name##_nocancel; \ L(pseudo_cancel): \ - /* We always have to align the stack before calling a function. */ \ - subq $8, %rsp; cfi_adjust_cfa_offset (8); \ - CENABLE \ - /* The return value from CENABLE is argument for CDISABLE. */ \ - movq %rax, (%rsp); \ - DO_CALL (syscall_name, args); \ - movq (%rsp), %rdi; \ - /* Save %rax since it's the error code from the syscall. */ \ - movq %rax, %rdx; \ - CDISABLE \ - movq %rdx, %rax; \ - addq $8,%rsp; cfi_adjust_cfa_offset (-8); \ - cmpq $-4095, %rax; \ - jae SYSCALL_ERROR_LABEL - + subq $24, %rsp; \ + cfi_def_cfa_offset (32); \ + movq %r9, (%rsp); \ + movq %r8, %r9; \ + movq %rcx, %r8; \ + movq %rdx, %rcx; \ + movq %rsi, %rdx; \ + movq %rdi, %rsi; \ + lea SYS_ify (syscall_name), %edi; \ + call JMP_SYSCALL_CANCEL; \ + cfi_def_cfa_offset (8); \ + addq $24, %rsp; \ + cmpq $-4095, %rax; \ + jae SYSCALL_ERROR_LABEL; # if IS_IN (libpthread) -# define CENABLE call __pthread_enable_asynccancel; -# define CDISABLE call __pthread_disable_asynccancel; # define __local_multiple_threads __pthread_multiple_threads # elif IS_IN (libc) -# define CENABLE call __libc_enable_asynccancel; -# define CDISABLE call __libc_disable_asynccancel; # define __local_multiple_threads __libc_multiple_threads -# elif IS_IN (librt) -# define CENABLE call __librt_enable_asynccancel; -# define CDISABLE call __librt_disable_asynccancel; -# else +# elif !IS_IN (librt) # error Unsupported library # endif @@ -78,7 +68,7 @@ # ifndef __ASSEMBLER__ extern int __local_multiple_threads attribute_hidden; # define SINGLE_THREAD_P \ - __builtin_expect (__local_multiple_threads == 0, 1) + __builtin_expect (__local_multiple_threads == 0, 1) # else # define SINGLE_THREAD_P cmpl $0, __local_multiple_threads(%rip) # endif @@ -87,18 +77,13 @@ extern int __local_multiple_threads attribute_hidden; # ifndef __ASSEMBLER__ # define SINGLE_THREAD_P \ - __builtin_expect (THREAD_GETMEM (THREAD_SELF, \ - header.multiple_threads) == 0, 1) + __builtin_expect (THREAD_GETMEM (THREAD_SELF, \ + header.multiple_threads) == 0, 1) # else # define SINGLE_THREAD_P cmpl $0, %fs:MULTIPLE_THREADS_OFFSET -# endif +# endif /* __ASSEMBLER */ -# endif - -#elif !defined __ASSEMBLER__ - -# define SINGLE_THREAD_P (1) -# define NO_CANCELLATION 1 +# endif /* IS_IN (libpthread) || IS_IN (libc) */ #endif @@ -106,4 +91,10 @@ extern int __local_multiple_threads attribute_hidden; # define RTLD_SINGLE_THREAD_P \ __builtin_expect (THREAD_GETMEM (THREAD_SELF, \ header.multiple_threads) == 0, 1) + +static inline +long int __pthread_get_ip (const ucontext_t *uc) +{ + return (long int)uc->uc_mcontext.gregs[REG_RIP]; +} #endif diff --git a/sysdeps/unix/sysv/linux/x86_64/sysdep.h b/sysdeps/unix/sysv/linux/x86_64/sysdep.h index 5a62cce..e4059f0 100644 --- a/sysdeps/unix/sysv/linux/x86_64/sysdep.h +++ b/sysdeps/unix/sysv/linux/x86_64/sysdep.h @@ -252,6 +252,14 @@ # undef INTERNAL_SYSCALL_ERRNO # define INTERNAL_SYSCALL_ERRNO(val, err) (-(val)) +# undef SYSCALL_CANCEL_ERROR +# define SYSCALL_CANCEL_ERROR(__val) \ + ((unsigned long int) (long int) (__val) >= -4095L) + +# undef SYSCALL_CANCEL_ERRNO +# define SYSCALL_CANCEL_ERRNO(__val) \ + (-(__val)) + /* List of system calls which are supported as vsyscalls. */ # define HAVE_CLOCK_GETTIME_VSYSCALL 1 # define HAVE_GETTIMEOFDAY_VSYSCALL 1 diff --git a/sysdeps/x86_64/nptl/tcb-offsets.sym b/sysdeps/x86_64/nptl/tcb-offsets.sym index 729d1da..026c9de 100644 --- a/sysdeps/x86_64/nptl/tcb-offsets.sym +++ b/sysdeps/x86_64/nptl/tcb-offsets.sym @@ -21,7 +21,6 @@ RTLD_SAVESPACE_SSE offsetof (tcbhead_t, rtld_savespace_sse) -- Not strictly offsets, but these values are also used in the TCB. TCB_CANCELSTATE_BITMASK CANCELSTATE_BITMASK TCB_CANCELTYPE_BITMASK CANCELTYPE_BITMASK -TCB_CANCELING_BITMASK CANCELING_BITMASK TCB_CANCELED_BITMASK CANCELED_BITMASK TCB_EXITING_BITMASK EXITING_BITMASK TCB_CANCEL_RESTMASK CANCEL_RESTMASK diff --git a/sysdeps/x86_64/nptl/tls.h b/sysdeps/x86_64/nptl/tls.h index d7543c6..0e277ec 100644 --- a/sysdeps/x86_64/nptl/tls.h +++ b/sysdeps/x86_64/nptl/tls.h @@ -325,17 +325,6 @@ typedef struct abort (); }) -/* Atomic set bit. */ -# define THREAD_ATOMIC_BIT_SET(descr, member, bit) \ - (void) ({ if (sizeof ((descr)->member) == 4) \ - asm volatile (LOCK_PREFIX "orl %1, %%fs:%P0" \ - :: "i" (offsetof (struct pthread, member)), \ - "ir" (1 << (bit))); \ - else \ - /* Not necessary for other sizes in the moment. */ \ - abort (); }) - - # define CALL_THREAD_FCT(descr) \ ({ void *__res; \ asm volatile ("movq %%fs:%P2, %%rdi\n\t" \