| Message ID | 20210918040157.608691-1-rodgert@appliantology.com |
|---|---|
| State | New |
| Headers | show |
| Series | libstdc++: Implement C++20 atomic<shared_ptr> and atomic<weak_ptr> | expand |
On Sat, 18 Sept 2021 at 05:12, Thomas Rodgers <rodgert@appliantology.com> wrote: > From: Thomas Rodgers <rodgert@twrodgers.com> > > Let's try this one instead. > > Signed-off-by: Thomas Rodgers <trodgers@redhat.com> > If you're doing DCO "Signed-off-by" commits you don't need FSF copyright notices in the new tests. I no longer put any copyright notices in the tests, because it's largely pointless. They're usually not interesting and don't do anything that anybody is going to want to steal to incorporate into non-GPL code. Your new 20_util/shared_ptr/atomic/atomic_shared_ptr.cc is non-trivial, and maybe interesting, but the two that just check feature test macros are not worth putting licence headers and copyright notices on. > libstdc++-v3/ChangeLog: > * acinclude.m4: Update ABI version. > * config/abi/pre/gnu.ver (GLIBCXX_3.4.21): Do not match new > _Sp_locker > constructor. > (GLIBCXX_3.4.30): Export _Sp_locker::_M_wait/_M_notify and new > constructor. > * include/bits/shared_ptr_atomic.h: define > __cpp_lib_atomic_shared_ptr > feature test macro. > (_Sp_locker::_Sp_locker(const void*, bool): New constructor. > (_Sp_locker::_M_wait()), _Sp_locker::_M_notify()): New methods. > (_Sp_impl): New type. > (atomic<shared_ptr<_Tp>>): New partial template specialization. > (atomic<weak_ptr<_Tp>>): New partial template specialization. > * include/std/version: define __cpp_lib_atomic_shared_ptr feature > test macro. > * doc/xml/manual/abi.xml: New ABI version. > * src/c++11/Makefile.am: Compile src/c++11/shared_ptr.cc > -std=gnu++20. > * src/c++11/Makefile.in: Regenerate. > * src/c++11/shared_ptr.cc (_Sp_locker::_Sp_locker(const void*, > bool), > _Sp_locker::_M_wait(), _Sp_locker::_M_notify(): Implement. > * testsuite/20_util/shared_ptr/atomic/4.cc: New test. > * testsuite/20_util/shared_ptr/atomic/5.cc: Likewise. > * testsuite/20_util/shared_ptr/atomic/atomic_shared_ptr.cc: > Likewise. > * testuite/util/testsuite_abi.cc: New ABI version. > --- > libstdc++-v3/acinclude.m4 | 2 +- > libstdc++-v3/config/abi/pre/gnu.ver | 12 +- > libstdc++-v3/configure | 2 +- > libstdc++-v3/doc/xml/manual/abi.xml | 1 + > libstdc++-v3/include/bits/shared_ptr_atomic.h | 309 ++++++++++++++++++ > libstdc++-v3/include/std/version | 1 + > libstdc++-v3/src/c++11/Makefile.am | 6 + > libstdc++-v3/src/c++11/Makefile.in | 6 + > libstdc++-v3/src/c++11/shared_ptr.cc | 86 ++++- > .../testsuite/20_util/shared_ptr/atomic/4.cc | 28 ++ > .../testsuite/20_util/shared_ptr/atomic/5.cc | 28 ++ > .../shared_ptr/atomic/atomic_shared_ptr.cc | 159 +++++++++ > libstdc++-v3/testsuite/util/testsuite_abi.cc | 3 +- > 13 files changed, 637 insertions(+), 6 deletions(-) > create mode 100644 libstdc++-v3/testsuite/20_util/shared_ptr/atomic/4.cc > create mode 100644 libstdc++-v3/testsuite/20_util/shared_ptr/atomic/5.cc > create mode 100644 > libstdc++-v3/testsuite/20_util/shared_ptr/atomic/atomic_shared_ptr.cc > > [...] > + void > + store(value_type __r, memory_order __o = memory_order_seq_cst) > noexcept > + { _M_impl._M_store(move(__r), __o); } > Every move(x) needs to be qualified as std::move(x) to avoid ADL. > > + > +// { dg-options "-std=gnu++2a" } > +// { dg-do run { target c++2a } } > gnu++20 and c++20 has been OK for a while, there's no need to use the 2a forms. Other than that, the patch looks OK, but I have a modified version based on your patch just using atomic ops, no mutexes, which I'll post shortly.
On Fri, 14 Jan 2022 at 22:40, Jonathan Wakely <jwakely@redhat.com> wrote: > > > On Sat, 18 Sept 2021 at 05:12, Thomas Rodgers <rodgert@appliantology.com> > wrote: > >> From: Thomas Rodgers <rodgert@twrodgers.com> >> >> Let's try this one instead. >> >> Signed-off-by: Thomas Rodgers <trodgers@redhat.com> >> > > If you're doing DCO "Signed-off-by" commits you don't need FSF copyright > notices in the new tests. > > I no longer put any copyright notices in the tests, because it's largely > pointless. They're usually not interesting and don't do anything that > anybody is going to want to steal to incorporate into non-GPL code. Your > new 20_util/shared_ptr/atomic/atomic_shared_ptr.cc is non-trivial, and > maybe interesting, but the two that just check feature test macros are not > worth putting licence headers and copyright notices on. > > > >> libstdc++-v3/ChangeLog: >> * acinclude.m4: Update ABI version. >> * config/abi/pre/gnu.ver (GLIBCXX_3.4.21): Do not match new >> _Sp_locker >> constructor. >> (GLIBCXX_3.4.30): Export _Sp_locker::_M_wait/_M_notify and new >> constructor. >> * include/bits/shared_ptr_atomic.h: define >> __cpp_lib_atomic_shared_ptr >> feature test macro. >> (_Sp_locker::_Sp_locker(const void*, bool): New constructor. >> (_Sp_locker::_M_wait()), _Sp_locker::_M_notify()): New methods. >> (_Sp_impl): New type. >> (atomic<shared_ptr<_Tp>>): New partial template specialization. >> (atomic<weak_ptr<_Tp>>): New partial template specialization. >> * include/std/version: define __cpp_lib_atomic_shared_ptr feature >> test macro. >> * doc/xml/manual/abi.xml: New ABI version. >> * src/c++11/Makefile.am: Compile src/c++11/shared_ptr.cc >> -std=gnu++20. >> * src/c++11/Makefile.in: Regenerate. >> * src/c++11/shared_ptr.cc (_Sp_locker::_Sp_locker(const void*, >> bool), >> _Sp_locker::_M_wait(), _Sp_locker::_M_notify(): Implement. >> * testsuite/20_util/shared_ptr/atomic/4.cc: New test. >> * testsuite/20_util/shared_ptr/atomic/5.cc: Likewise. >> * testsuite/20_util/shared_ptr/atomic/atomic_shared_ptr.cc: >> Likewise. >> * testuite/util/testsuite_abi.cc: New ABI version. >> --- >> libstdc++-v3/acinclude.m4 | 2 +- >> libstdc++-v3/config/abi/pre/gnu.ver | 12 +- >> libstdc++-v3/configure | 2 +- >> libstdc++-v3/doc/xml/manual/abi.xml | 1 + >> libstdc++-v3/include/bits/shared_ptr_atomic.h | 309 ++++++++++++++++++ >> libstdc++-v3/include/std/version | 1 + >> libstdc++-v3/src/c++11/Makefile.am | 6 + >> libstdc++-v3/src/c++11/Makefile.in | 6 + >> libstdc++-v3/src/c++11/shared_ptr.cc | 86 ++++- >> .../testsuite/20_util/shared_ptr/atomic/4.cc | 28 ++ >> .../testsuite/20_util/shared_ptr/atomic/5.cc | 28 ++ >> .../shared_ptr/atomic/atomic_shared_ptr.cc | 159 +++++++++ >> libstdc++-v3/testsuite/util/testsuite_abi.cc | 3 +- >> 13 files changed, 637 insertions(+), 6 deletions(-) >> create mode 100644 libstdc++-v3/testsuite/20_util/shared_ptr/atomic/4.cc >> create mode 100644 libstdc++-v3/testsuite/20_util/shared_ptr/atomic/5.cc >> create mode 100644 >> libstdc++-v3/testsuite/20_util/shared_ptr/atomic/atomic_shared_ptr.cc >> >> > [...] > > >> + void >> + store(value_type __r, memory_order __o = memory_order_seq_cst) >> noexcept >> + { _M_impl._M_store(move(__r), __o); } >> > > Every move(x) needs to be qualified as std::move(x) to avoid ADL. > > > >> >> + >> +// { dg-options "-std=gnu++2a" } >> +// { dg-do run { target c++2a } } >> > > gnu++20 and c++20 has been OK for a while, there's no need to use the 2a > forms. > > Other than that, the patch looks OK, but I have a modified version based > on your patch just using atomic ops, no mutexes, which I'll post shortly. > Here's the finished version of that patch, which is all inline in the header, and performs faster too. Tested powerpc64le-linux and x86_64-linux. Pushed to trunk. commit 2ac0649d7bf3eacbf92add1ec2b54045c401a4c2 Author: Jonathan Wakely <jwakely@redhat.com> Date: Sun Jan 16 20:47:09 2022 libstdc++: Implement C++20 atomic<shared_ptr> and atomic<weak_ptr> This adds another piece of C++20, the std::atomic specializations for std::shared_ptr and std::weak_ptr. The new _Sp_atomic type mimics the structure of shared_ptr<T> and weak_ptr<T>, holding a T* pointer (the one returned by get() on a shared_ptr/weak ptr) and a _Sp_counted_base<>* pointer to the ref-counted control block. For _Sp_atomic the low bit of the control block pointer is used as a lock bit, to ensure only one thread will access the object at a time. The pointer is actually stored as a uintptr_t to avoid accidental dereferences of the pointer when unlocked (which would be a race) or when locked (which would dereference the wrong pointer value due to the low bit being set). To get a raw pointer to the control block, the lock must be acquired. Converting between a _Sp_atomic and a shared_ptr or weak_ptr requires manually adjusting the T* and _Sp_counted_base<>* members of the shared/weak ptr, instead of going through the public API. This must be done carefully to ensure that any change in the number of owners is reflected in a ref-count update. Co-authored-by: Thomas Rodgers <trodgers@redhat.com> Signed-off-by: Thomas Rodgers <trodgers@redhat.com> libstdc++-v3/ChangeLog: * include/bits/shared_ptr_atomic.h (__cpp_lib_atomic_shared_ptr): New macro. (_Sp_atomic): New class template. (atomic<shared_ptr<T>>, atomic<weak_ptr<T>>): New partial specializations. * include/bits/shared_ptr_base.h (__shared_count, __weak_count) (__shared_ptr, __weak_ptr): Declare _Sp_atomic as a friend. * include/std/version (__cpp_lib_atomic_shared_ptr): New macro. * testsuite/20_util/shared_ptr/atomic/atomic_shared_ptr.cc: New test. * testsuite/20_util/weak_ptr/atomic_weak_ptr.cc: New test. diff --git a/libstdc++-v3/include/bits/shared_ptr_atomic.h b/libstdc++-v3/include/bits/shared_ptr_atomic.h index 0e1c289c0a9..900499bae32 100644 --- a/libstdc++-v3/include/bits/shared_ptr_atomic.h +++ b/libstdc++-v3/include/bits/shared_ptr_atomic.h @@ -327,6 +327,461 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION } /// @} +#if __cplusplus >= 202002L +# define __cpp_lib_atomic_shared_ptr 201711L + template<typename _Tp> + class atomic; + + template<typename _Up> + static constexpr bool __is_shared_ptr = false; + template<typename _Up> + static constexpr bool __is_shared_ptr<shared_ptr<_Up>> = true; + + template<typename _Tp> + class _Sp_atomic + { + using value_type = _Tp; + + friend class atomic<_Tp>; + + // An atomic version of __shared_count<> and __weak_count<>. + // Stores a _Sp_counted_base<>* but uses the LSB as a lock. + struct _Atomic_count + { + // Either __shared_count<> or __weak_count<> + using __count_type = decltype(_Tp::_M_refcount); + + // _Sp_counted_base<>* + using pointer = decltype(__count_type::_M_pi); + + // Ensure we can use the LSB as the lock bit. + static_assert(alignof(remove_pointer_t<pointer>) > 1); + + _Atomic_count() : _M_val(0) { } + + explicit + _Atomic_count(__count_type&& __c) noexcept + : _M_val(reinterpret_cast<uintptr_t>(__c._M_pi)) + { + __c._M_pi = nullptr; + } + + ~_Atomic_count() + { + auto __val = _M_val.load(memory_order_relaxed); + __glibcxx_assert(!(__val & _S_lock_bit)); + if (auto __pi = reinterpret_cast<pointer>(__val)) + { + if constexpr (__is_shared_ptr<_Tp>) + __pi->_M_release(); + else + __pi->_M_weak_release(); + } + } + + _Atomic_count(const _Atomic_count&) = delete; + _Atomic_count& operator=(const _Atomic_count&) = delete; + + // Precondition: Caller does not hold lock! + // Returns the raw pointer value without the lock bit set. + pointer + lock(memory_order __o) const noexcept + { + // To acquire the lock we flip the LSB from 0 to 1. + + auto __current = _M_val.load(memory_order_relaxed); + while (__current & _S_lock_bit) + { + __detail::__thread_relax(); + __current = _M_val.load(memory_order_relaxed); + } + + while (!_M_val.compare_exchange_strong(__current, + __current | _S_lock_bit, + __o, + memory_order_relaxed)) + { + __detail::__thread_relax(); + __current = __current & ~_S_lock_bit; + } + return reinterpret_cast<pointer>(__current); + } + + // Precondition: caller holds lock! + void + unlock(memory_order __o) const noexcept + { + _M_val.fetch_sub(1, __o); + } + + // Swaps the values of *this and __c, and unlocks *this. + // Precondition: caller holds lock! + void + _M_swap_unlock(__count_type& __c, memory_order __o) noexcept + { + if (__o != memory_order_seq_cst) + __o = memory_order_release; + auto __x = reinterpret_cast<uintptr_t>(__c._M_pi); + __x = _M_val.exchange(__x, __o); + __c._M_pi = reinterpret_cast<pointer>(__x & ~_S_lock_bit); + } + +#if __cpp_lib_atomic_wait + // Precondition: caller holds lock! + void + _M_wait_unlock(memory_order __o) const noexcept + { + auto __v = _M_val.fetch_sub(1, memory_order_relaxed); + _M_val.wait(__v & ~_S_lock_bit, __o); + } + + void + notify_one() noexcept + { + _M_val.notify_one(); + } + + void + notify_all() noexcept + { + _M_val.notify_all(); + } +#endif + + private: + mutable __atomic_base<uintptr_t> _M_val{0}; + static constexpr uintptr_t _S_lock_bit{1}; + }; + + typename _Tp::element_type* _M_ptr; + _Atomic_count _M_refcount; + + static _Atomic_count::pointer + _S_add_ref(_Atomic_count::pointer __p) + { + if (__p) + { + if constexpr (__is_shared_ptr<_Tp>) + __p->_M_add_ref_copy(); + else + __p->_M_weak_add_ref(); + } + return __p; + } + + constexpr _Sp_atomic() noexcept = default; + + explicit + _Sp_atomic(value_type __r) noexcept + : _M_ptr(__r._M_ptr), _M_refcount(std::move(__r._M_refcount)) + { } + + ~_Sp_atomic() = default; + + _Sp_atomic(const _Sp_atomic&) = delete; + void operator=(const _Sp_atomic&) = delete; + + value_type + load(memory_order __o) const noexcept + { + __glibcxx_assert(__o != memory_order_release + && __o != memory_order_acq_rel); + // Ensure that the correct value of _M_ptr is visible after locking., + // by upgrading relaxed or consume to acquire. + if (__o != memory_order_seq_cst) + __o = memory_order_acquire; + + value_type __ret; + auto __pi = _M_refcount.lock(__o); + __ret._M_ptr = _M_ptr; + __ret._M_refcount._M_pi = _S_add_ref(__pi); + _M_refcount.unlock(memory_order_relaxed); + return __ret; + } + + void + swap(value_type& __r, memory_order __o) noexcept + { + _M_refcount.lock(memory_order_acquire); + std::swap(_M_ptr, __r._M_ptr); + _M_refcount._M_swap_unlock(__r._M_refcount, __o); + } + + bool + compare_exchange_strong(value_type& __expected, value_type __desired, + memory_order __o, memory_order __o2) noexcept + { + bool __result = true; + auto __pi = _M_refcount.lock(memory_order_acquire); + if (_M_ptr == __expected._M_ptr + && __pi == __expected._M_refcount._M_pi) + { + _M_ptr = __desired._M_ptr; + _M_refcount._M_swap_unlock(__desired._M_refcount, __o); + } + else + { + _Tp __sink = std::move(__expected); + __expected._M_ptr = _M_ptr; + __expected._M_refcount._M_pi = _S_add_ref(__pi); + _M_refcount.unlock(__o2); + __result = false; + } + return __result; + } + +#if __cpp_lib_atomic_wait + void + wait(value_type __old, memory_order __o) const noexcept + { + auto __pi = _M_refcount.lock(memory_order_acquire); + if (_M_ptr == __old._M_ptr && __pi == __old._M_refcount._M_pi) + _M_refcount._M_wait_unlock(__o); + else + _M_refcount.unlock(memory_order_relaxed); + } + + void + notify_one() noexcept + { + _M_refcount.notify_one(); + } + + void + notify_all() noexcept + { + _M_refcount.notify_all(); + } +#endif + }; + + template<typename _Tp> + class atomic<shared_ptr<_Tp>> + { + public: + using value_type = shared_ptr<_Tp>; + + static constexpr bool is_always_lock_free = false; + + bool + is_lock_free() const noexcept + { return false; } + + constexpr atomic() noexcept = default; + + atomic(shared_ptr<_Tp> __r) noexcept + : _M_impl(std::move(__r)) + { } + + atomic(const atomic&) = delete; + void operator=(const atomic&) = delete; + + shared_ptr<_Tp> + load(memory_order __o = memory_order_seq_cst) const noexcept + { return _M_impl.load(__o); } + + operator shared_ptr<_Tp>() const noexcept + { return _M_impl.load(memory_order_seq_cst); } + + void + store(shared_ptr<_Tp> __desired, + memory_order __o = memory_order_seq_cst) noexcept + { _M_impl.swap(__desired, __o); } + + void + operator=(shared_ptr<_Tp> __desired) noexcept + { _M_impl.swap(__desired, memory_order_seq_cst); } + + shared_ptr<_Tp> + exchange(shared_ptr<_Tp> __desired, + memory_order __o = memory_order_seq_cst) noexcept + { + _M_impl.swap(__desired, __o); + return __desired; + } + + bool + compare_exchange_strong(shared_ptr<_Tp>& __expected, + shared_ptr<_Tp> __desired, + memory_order __o, memory_order __o2) noexcept + { + return _M_impl.compare_exchange_strong(__expected, __desired, __o, __o2); + } + + bool + compare_exchange_strong(value_type& __expected, value_type __desired, + memory_order __o = memory_order_seq_cst) noexcept + { + memory_order __o2; + switch (__o) + { + case memory_order_acq_rel: + __o2 = memory_order_acquire; + break; + case memory_order_release: + __o2 = memory_order_relaxed; + break; + default: + __o2 = __o; + } + return compare_exchange_strong(__expected, std::move(__desired), + __o, __o2); + } + + bool + compare_exchange_weak(value_type& __expected, value_type __desired, + memory_order __o, memory_order __o2) noexcept + { + return compare_exchange_strong(__expected, std::move(__desired), + __o, __o2); + } + + bool + compare_exchange_weak(value_type& __expected, value_type __desired, + memory_order __o = memory_order_seq_cst) noexcept + { + return compare_exchange_strong(__expected, std::move(__desired), __o); + } + +#if __cpp_lib_atomic_wait + void + wait(value_type __old, + memory_order __o = memory_order_seq_cst) const noexcept + { + _M_impl.wait(std::move(__old), __o); + } + + void + notify_one() noexcept + { + _M_impl.notify_one(); + } + + void + notify_all() noexcept + { + _M_impl.notify_all(); + } +#endif + + private: + _Sp_atomic<shared_ptr<_Tp>> _M_impl; + }; + + template<typename _Tp> + class atomic<weak_ptr<_Tp>> + { + public: + using value_type = weak_ptr<_Tp>; + + static constexpr bool is_always_lock_free = false; + + bool + is_lock_free() const noexcept + { return false; } + + constexpr atomic() noexcept = default; + + atomic(weak_ptr<_Tp> __r) noexcept + : _M_impl(move(__r)) + { } + + atomic(const atomic&) = delete; + void operator=(const atomic&) = delete; + + weak_ptr<_Tp> + load(memory_order __o = memory_order_seq_cst) const noexcept + { return _M_impl.load(__o); } + + operator weak_ptr<_Tp>() const noexcept + { return _M_impl.load(memory_order_seq_cst); } + + void + store(weak_ptr<_Tp> __desired, + memory_order __o = memory_order_seq_cst) noexcept + { _M_impl.swap(__desired, __o); } + + void + operator=(weak_ptr<_Tp> __desired) noexcept + { _M_impl.swap(__desired, memory_order_seq_cst); } + + weak_ptr<_Tp> + exchange(weak_ptr<_Tp> __desired, + memory_order __o = memory_order_seq_cst) noexcept + { + _M_impl.swap(__desired, __o); + return __desired; + } + + bool + compare_exchange_strong(weak_ptr<_Tp>& __expected, + weak_ptr<_Tp> __desired, + memory_order __o, memory_order __o2) noexcept + { + return _M_impl.compare_exchange_strong(__expected, __desired, __o, __o2); + } + + bool + compare_exchange_strong(value_type& __expected, value_type __desired, + memory_order __o = memory_order_seq_cst) noexcept + { + memory_order __o2; + switch (__o) + { + case memory_order_acq_rel: + __o2 = memory_order_acquire; + break; + case memory_order_release: + __o2 = memory_order_relaxed; + break; + default: + __o2 = __o; + } + return compare_exchange_strong(__expected, std::move(__desired), + __o, __o2); + } + + bool + compare_exchange_weak(value_type& __expected, value_type __desired, + memory_order __o, memory_order __o2) noexcept + { + return compare_exchange_strong(__expected, std::move(__desired), + __o, __o2); + } + + bool + compare_exchange_weak(value_type& __expected, value_type __desired, + memory_order __o = memory_order_seq_cst) noexcept + { + return compare_exchange_strong(__expected, std::move(__desired), __o); + } + +#if __cpp_lib_atomic_wait + void + wait(value_type __old, + memory_order __o = memory_order_seq_cst) const noexcept + { + _M_impl.wait(std::move(__old), __o); + } + + void + notify_one() noexcept + { + _M_impl.notify_one(); + } + + void + notify_all() noexcept + { + _M_impl.notify_all(); + } +#endif + + private: + _Sp_atomic<weak_ptr<_Tp>> _M_impl; + }; +#endif // C++20 + /// @} relates shared_ptr /// @} group pointer_abstractions diff --git a/libstdc++-v3/include/bits/shared_ptr_base.h b/libstdc++-v3/include/bits/shared_ptr_base.h index 9e80aabcda8..5b8f84b65be 100644 --- a/libstdc++-v3/include/bits/shared_ptr_base.h +++ b/libstdc++-v3/include/bits/shared_ptr_base.h @@ -409,6 +409,10 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION template<_Lock_policy _Lp = __default_lock_policy> class __shared_count; +#if __cplusplus >= 202002L + template<typename> + class _Sp_atomic; +#endif // Counted ptr with no deleter or allocator support template<typename _Ptr, _Lock_policy _Lp> @@ -1121,6 +1125,9 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION private: friend class __weak_count<_Lp>; +#if __cplusplus >= 202002L + template<typename> friend class _Sp_atomic; +#endif _Sp_counted_base<_Lp>* _M_pi; }; @@ -1218,6 +1225,9 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION private: friend class __shared_count<_Lp>; +#if __cplusplus >= 202002L + template<typename> friend class _Sp_atomic; +#endif _Sp_counted_base<_Lp>* _M_pi; }; @@ -1765,6 +1775,10 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION template<typename _Del, typename _Tp1> friend _Del* get_deleter(const shared_ptr<_Tp1>&) noexcept; +#if __cplusplus >= 202002L + friend _Sp_atomic<shared_ptr<_Tp>>; +#endif + element_type* _M_ptr; // Contained pointer. __shared_count<_Lp> _M_refcount; // Reference counter. }; @@ -2097,6 +2111,9 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION template<typename _Tp1, _Lock_policy _Lp1> friend class __weak_ptr; friend class __enable_shared_from_this<_Tp, _Lp>; friend class enable_shared_from_this<_Tp>; +#if __cplusplus >= 202002L + friend _Sp_atomic<weak_ptr<_Tp>>; +#endif element_type* _M_ptr; // Contained pointer. __weak_count<_Lp> _M_refcount; // Reference counter. diff --git a/libstdc++-v3/include/std/version b/libstdc++-v3/include/std/version index a8b792e9b0f..7bd32f67f19 100644 --- a/libstdc++-v3/include/std/version +++ b/libstdc++-v3/include/std/version @@ -215,6 +215,7 @@ #if _GLIBCXX_HOSTED #define __cpp_lib_array_constexpr 201811L #define __cpp_lib_assume_aligned 201811L +#define __cpp_lib_atomic_shared_ptr 201711L #if defined _GLIBCXX_HAS_GTHREADS || defined _GLIBCXX_HAVE_LINUX_FUTEX # define __cpp_lib_atomic_wait 201907L # if __cpp_aligned_new diff --git a/libstdc++-v3/testsuite/20_util/shared_ptr/atomic/atomic_shared_ptr.cc b/libstdc++-v3/testsuite/20_util/shared_ptr/atomic/atomic_shared_ptr.cc new file mode 100644 index 00000000000..725e7ba5031 --- /dev/null +++ b/libstdc++-v3/testsuite/20_util/shared_ptr/atomic/atomic_shared_ptr.cc @@ -0,0 +1,150 @@ +// { dg-options "-std=gnu++20" } +// { dg-do run { target c++20 } } +// { dg-require-effective-target gthreads } +// { dg-additional-options "-pthread" { target pthread } } +// { dg-add-options libatomic } + +#include <memory> + +#ifndef __cpp_lib_atomic_shared_ptr +# error "Feature-test macro for atomic<shared_ptr<T>> missing in <memory>" +#elif __cpp_lib_atomic_shared_ptr != 201711L +# error "Feature-test macro for atomic<shared_ptr<T>> has wrong value in <memory>" +#endif + +#include <thread> + +#include <testsuite_hooks.h> + +void +test_is_lock_free() +{ + using test_type = std::atomic<std::shared_ptr<int>>; + static_assert( test_type::is_always_lock_free == false ); + + test_type p; + VERIFY( p.is_lock_free() == false ); +} + +void +test_atomic_shared_ptr() +{ + struct A { int a; int b; }; + + auto a = std::make_shared<A>( 0, 42 ); + using ptr_t = std::shared_ptr<A>; + { + std::atomic<ptr_t> p{ }; + VERIFY( p.load().get() == nullptr ); + } + + std::atomic<ptr_t> p{ a }; + VERIFY( p.load().get() == a.get() ); + auto b = std::make_shared<A>( 42, 0 ); + p.store(b); + VERIFY( p.load().get() != a.get() ); + VERIFY( p.load().get() == b.get() ); + p.exchange(a); + VERIFY( p.load().get() != b.get() ); + VERIFY( p.load().get() == a.get() ); + + { + ptr_t aa{ a }; + VERIFY( p.compare_exchange_strong(aa, b, + std::memory_order_seq_cst, + std::memory_order_seq_cst) == true ); + ptr_t bb{ a }; + VERIFY( p.compare_exchange_strong(bb, b, + std::memory_order_seq_cst, + std::memory_order_seq_cst) == false ); + VERIFY( bb.get() == b.get() ); + } + + { + ptr_t bb{ b }; + VERIFY( p.compare_exchange_weak(bb, a, + std::memory_order_seq_cst, + std::memory_order_seq_cst) == true ); + ptr_t aa{ b }; + VERIFY( p.compare_exchange_weak(aa, a, + std::memory_order_seq_cst, + std::memory_order_seq_cst) == false ); + VERIFY( aa.get() == a.get() ); + } +} + +void +test_wait_notify() +{ + std::atomic<std::shared_ptr<int>> p; + std::shared_ptr<int> a = std::make_shared<int>(); + std::shared_ptr<int> b = std::make_shared<int>(); + + p.store(a); + p.wait(b); + std::thread t([&] + { + p.store(b); + p.notify_one(); + }); + p.wait(a); + t.join(); +} + +int counter = 0; + +void +test_counting() +{ + struct X + { + ~X() { ++counter; } + }; + + { + std::atomic<std::shared_ptr<X>> p{ std::make_shared<X>() }; + std::shared_ptr<X> a = p.load(); + VERIFY( a.use_count() == 2 ); // p, a + p.store({}); + VERIFY( a.use_count() == 1 ); // a + p.store(a); + VERIFY( a.use_count() == 2 ); // p, a + std::shared_ptr<X> b = std::make_shared<X>(); + std::shared_ptr<X> c = p.exchange(b); + VERIFY( a.use_count() == 2 ); // a, c + VERIFY( c == a ); + VERIFY( b.use_count() == 2 ); // p, b + std::atomic<std::shared_ptr<X>> p2{a}; + VERIFY( a.use_count() == 3 ); // p2, a, c + VERIFY( p2.compare_exchange_strong(a, b) ); + VERIFY( a.use_count() == 2 ); // a, c + VERIFY( b.use_count() == 3 ); // p, p2, b + VERIFY ( ! p2.compare_exchange_strong(a, b) ); + VERIFY( a == b ); + VERIFY( a.use_count() == 4 ); // p, p2, a, b + VERIFY( b.use_count() == 4 ); + VERIFY( c.use_count() == 1 ); // c + VERIFY( p.compare_exchange_weak(b, c) ); + VERIFY( b.use_count() == 3 ); // p2, a, b + VERIFY( c.use_count() == 2 ); // p, c + VERIFY( ! p.compare_exchange_weak(a, b) ); + VERIFY( a == c ); + VERIFY( a.use_count() == 3 ); // p, a, c + VERIFY( b.use_count() == 2 ); // p2, b + VERIFY( c.use_count() == 3 ); // p, a, c + a.reset(); + b.reset(); + c.reset(); + VERIFY( counter == 0 ); + } + VERIFY( counter == 2 ); +} + +int +main() +{ + test_is_lock_free(); + test_atomic_shared_ptr(); + test_wait_notify(); + test_counting(); +} diff --git a/libstdc++-v3/testsuite/20_util/weak_ptr/atomic_weak_ptr.cc b/libstdc++-v3/testsuite/20_util/weak_ptr/atomic_weak_ptr.cc new file mode 100644 index 00000000000..e394e5562b9 --- /dev/null +++ b/libstdc++-v3/testsuite/20_util/weak_ptr/atomic_weak_ptr.cc @@ -0,0 +1,95 @@ +// { dg-options "-std=gnu++20" } +// { dg-do run { target c++20 } } +// { dg-require-effective-target gthreads } +// { dg-additional-options "-pthread" { target pthread } } +// { dg-add-options libatomic } + +#include <memory> +#include <thread> +#include <testsuite_hooks.h> + +void +test_is_lock_free() +{ + using test_type = std::atomic<std::weak_ptr<int>>; + static_assert( test_type::is_always_lock_free == false ); + + test_type p; + VERIFY( p.is_lock_free() == false ); +} + +void +test_atomic_weak_ptr() +{ + struct A { int a; int b; }; + + auto a = std::make_shared<A>( 0, 42 ); + using ptr_t = std::weak_ptr<A>; + ptr_t wa{ a }; + { + std::atomic<ptr_t> p{ }; + VERIFY( p.load().lock().get() == nullptr ); + } + + std::atomic<ptr_t> p{ wa }; + VERIFY( p.load().lock().get() == a.get() ); + + auto b = std::make_shared<A>( 42, 0 ); + ptr_t wb{ b }; + p.store(wb); + VERIFY( p.load().lock().get() != a.get() ); + VERIFY( p.load().lock().get() == b.get() ); + p.exchange(wa); + VERIFY( p.load().lock().get() != b.get() ); + VERIFY( p.load().lock().get() == a.get() ); + + { + ptr_t aa{ a }; + VERIFY( p.compare_exchange_strong(aa, b, + std::memory_order_seq_cst, + std::memory_order_seq_cst) == true ); + ptr_t bb{ a }; + VERIFY( p.compare_exchange_strong(bb, b, + std::memory_order_seq_cst, + std::memory_order_seq_cst) == false ); + VERIFY( bb.lock().get() == b.get() ); + } + + { + ptr_t bb{ b }; + VERIFY( p.compare_exchange_weak(bb, a, + std::memory_order_seq_cst, + std::memory_order_seq_cst) == true ); + ptr_t aa{ b }; + VERIFY( p.compare_exchange_weak(aa, a, + std::memory_order_seq_cst, + std::memory_order_seq_cst) == false ); + VERIFY( aa.lock().get() == a.get() ); + } +} + +void +test_wait_notify() +{ + std::atomic<std::weak_ptr<int>> p; + std::weak_ptr<int> a = std::make_shared<int>(); + std::weak_ptr<int> b = std::make_shared<int>(); + + p.store(a); + p.wait(b); + std::thread t([&] + { + p.store(b); + p.notify_one(); + }); + p.wait(a); + t.join(); +} + +int +main() +{ + test_is_lock_free(); + test_atomic_weak_ptr(); + test_wait_notify(); +}
On 1/17/22 01:12, Jonathan Wakely via Libstdc++ wrote: > Here's the finished version of that patch, which is all inline in the > header, and performs faster too. > > Tested powerpc64le-linux and x86_64-linux. > Pushed to trunk. FYI, recent Clang still requires typename in two places to compile <memory> with -std=c++20, > diff --git a/libstdc++-v3/include/bits/shared_ptr_atomic.h b/libstdc++-v3/include/bits/shared_ptr_atomic.h > index 900499bae32..50aa46370ca 100644 > --- a/libstdc++-v3/include/bits/shared_ptr_atomic.h > +++ b/libstdc++-v3/include/bits/shared_ptr_atomic.h > @@ -456,8 +456,8 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION > typename _Tp::element_type* _M_ptr; > _Atomic_count _M_refcount; > > - static _Atomic_count::pointer > - _S_add_ref(_Atomic_count::pointer __p) > + static typename _Atomic_count::pointer > + _S_add_ref(typename _Atomic_count::pointer __p) > { > if (__p) > {
On Mon, 17 Jan 2022 at 09:39, Stephan Bergmann <sbergman@redhat.com> wrote: > On 1/17/22 01:12, Jonathan Wakely via Libstdc++ wrote: > > Here's the finished version of that patch, which is all inline in the > > header, and performs faster too. > > > > Tested powerpc64le-linux and x86_64-linux. > > Pushed to trunk. > > FYI, recent Clang still requires typename in two places to compile > <memory> with -std=c++20, > > > diff --git a/libstdc++-v3/include/bits/shared_ptr_atomic.h > b/libstdc++-v3/include/bits/shared_ptr_atomic.h > > index 900499bae32..50aa46370ca 100644 > > --- a/libstdc++-v3/include/bits/shared_ptr_atomic.h > > +++ b/libstdc++-v3/include/bits/shared_ptr_atomic.h > > @@ -456,8 +456,8 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION > > typename _Tp::element_type* _M_ptr; > > _Atomic_count _M_refcount; > > > > - static _Atomic_count::pointer > > - _S_add_ref(_Atomic_count::pointer __p) > > + static typename _Atomic_count::pointer > > + _S_add_ref(typename _Atomic_count::pointer __p) > > { > > if (__p) > > { > Sigh, yes, I already noticed this myself. Clang is wrong (P0634R0 gives an example *very* similar to this as motivation for the change), but we need to keep writing the code in C++17 style if we want to be compatible with released versions of Clang. Fixed by the attached patch, tested x86_64-linux and pushed to trunk. commit a923345c72aef9ebb6b32f81f4825acc11085803 Author: Jonathan Wakely <jwakely@redhat.com> Date: Mon Jan 17 09:42:35 2022 libstdc++: Add 'typename' to dependent types in atomic<shared_ptr<T>> libstdc++-v3/ChangeLog: * include/bits/shared_ptr_atomic.h (_Sp_atomic): Add typename to qualified-id for dependent type. diff --git a/libstdc++-v3/include/bits/shared_ptr_atomic.h b/libstdc++-v3/include/bits/shared_ptr_atomic.h index 900499bae32..50aa46370ca 100644 --- a/libstdc++-v3/include/bits/shared_ptr_atomic.h +++ b/libstdc++-v3/include/bits/shared_ptr_atomic.h @@ -456,8 +456,8 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION typename _Tp::element_type* _M_ptr; _Atomic_count _M_refcount; - static _Atomic_count::pointer - _S_add_ref(_Atomic_count::pointer __p) + static typename _Atomic_count::pointer + _S_add_ref(typename _Atomic_count::pointer __p) { if (__p) {
diff --git a/libstdc++-v3/acinclude.m4 b/libstdc++-v3/acinclude.m4 index 90ecc4a87a2..30a4abb98b3 100644 --- a/libstdc++-v3/acinclude.m4 +++ b/libstdc++-v3/acinclude.m4 @@ -3798,7 +3798,7 @@ changequote([,])dnl fi # For libtool versioning info, format is CURRENT:REVISION:AGE -libtool_VERSION=6:29:0 +libtool_VERSION=6:30:0 # Everything parsed; figure out what files and settings to use. case $enable_symvers in diff --git a/libstdc++-v3/config/abi/pre/gnu.ver b/libstdc++-v3/config/abi/pre/gnu.ver index 5323c7f0604..727afd2d488 100644 --- a/libstdc++-v3/config/abi/pre/gnu.ver +++ b/libstdc++-v3/config/abi/pre/gnu.ver @@ -1705,8 +1705,9 @@ GLIBCXX_3.4.21 { # std::ctype_base::blank _ZNSt10ctype_base5blankE; - # std::_Sp_locker::* - _ZNSt10_Sp_locker[CD]*; + # std::_Sp_locker:: constructors and destructors + _ZNSt10_Sp_lockerC*[^b]; + _ZNSt10_Sp_lockerD*; # std::notify_all_at_thread_exit _ZSt25notify_all_at_thread_exitRSt18condition_variableSt11unique_lockISt5mutexE; @@ -2397,6 +2398,13 @@ GLIBCXX_3.4.29 { } GLIBCXX_3.4.28; +GLIBCXX_3.4.30 { + # std::_Sp_locker:: wait/notify support + _ZNSt10_Sp_lockerC*[b]; + _ZNSt10_Sp_locker7_M_waitEv; + _ZNSt10_Sp_locker9_M_notifyEv; +} GLIBCXX_3.4.29; + # Symbols in the support library (libsupc++) have their own tag. CXXABI_1.3 { diff --git a/libstdc++-v3/configure b/libstdc++-v3/configure index 13d52eb0c0e..67ee6db1bea 100755 --- a/libstdc++-v3/configure +++ b/libstdc++-v3/configure @@ -74684,7 +74684,7 @@ $as_echo "$as_me: WARNING: === Symbol versioning will be disabled." >&2;} fi # For libtool versioning info, format is CURRENT:REVISION:AGE -libtool_VERSION=6:29:0 +libtool_VERSION=6:30:0 # Everything parsed; figure out what files and settings to use. case $enable_symvers in diff --git a/libstdc++-v3/doc/xml/manual/abi.xml b/libstdc++-v3/doc/xml/manual/abi.xml index c2c0c028a8b..10bef12c768 100644 --- a/libstdc++-v3/doc/xml/manual/abi.xml +++ b/libstdc++-v3/doc/xml/manual/abi.xml @@ -348,6 +348,7 @@ compatible. <listitem><para>GCC 9.3.0: GLIBCXX_3.4.28, CXXABI_1.3.12</para></listitem> <listitem><para>GCC 10.1.0: GLIBCXX_3.4.28, CXXABI_1.3.12</para></listitem> <listitem><para>GCC 11.1.0: GLIBCXX_3.4.29, CXXABI_1.3.13</para></listitem> + <listitem><para>GCC 12.1.0: GLIBCXX_3.4.30, CXXABI_1.3.13</para></listitem> </itemizedlist> </listitem> diff --git a/libstdc++-v3/include/bits/shared_ptr_atomic.h b/libstdc++-v3/include/bits/shared_ptr_atomic.h index 6e94d83c46d..2aec3adac7c 100644 --- a/libstdc++-v3/include/bits/shared_ptr_atomic.h +++ b/libstdc++-v3/include/bits/shared_ptr_atomic.h @@ -32,6 +32,10 @@ #include <bits/atomic_base.h> +#if __cplusplus > 201703L +# define __cpp_lib_atomic_shared_ptr 201711L +#endif + namespace std _GLIBCXX_VISIBILITY(default) { _GLIBCXX_BEGIN_NAMESPACE_VERSION @@ -55,6 +59,17 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION _Sp_locker(const void*, const void*) noexcept; ~_Sp_locker(); +#if __cpp_lib_atomic_shared_ptr + // called only by notifiers, does not acquire a lock + _Sp_locker(const void*, bool) noexcept; + + void + _M_wait() noexcept; + + void + _M_notify() noexcept; +#endif + private: unsigned char _M_key1; unsigned char _M_key2; @@ -327,6 +342,300 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION } /// @} +#if __cpp_lib_atomic_shared_ptr + template<typename _Tp> + struct _Sp_impl + { + using value_type = _Tp; + + static constexpr bool + _M_is_always_lock_free = false; + + static bool + _M_is_lock_free() noexcept + { return false; } + + constexpr _Sp_impl() noexcept = default; + _Sp_impl(value_type __r) noexcept + : _M_p(move(__r)) + { } + + _Sp_impl(const _Sp_impl&) = delete; + void operator=(const _Sp_impl&) = delete; + + value_type + _M_load(memory_order) const noexcept + { + _Sp_locker __lock{&_M_p}; + return _M_p; + } + + void + _M_store(value_type __r, memory_order) noexcept + { + _Sp_locker __lock{&_M_p}; + _M_p.swap(__r); // use swap so that *__p not destroyed while lock held + } + + value_type + _M_exchange(value_type __r, memory_order) noexcept + { + _Sp_locker __lock{&_M_p}; + _M_p.swap(__r); + return __r; + } + + template<typename _Pred> + bool + _M_compare_exchange_strong(value_type& __v, value_type __w, + _Pred __pred, + memory_order, memory_order) noexcept + { + value_type __x; // goes out of scope after __lock + _Sp_locker __lock{&_M_p, &__v}; + if (__pred(_M_p, __v)) + { + __x = move(_M_p); + _M_p = move(__w); + return true; + } + __x = move(__v); + __v = _M_p; + return false; + } + + template<typename _Pred> + void + _M_wait(value_type __old, + _Pred __pred, memory_order) const noexcept + { + _Sp_locker __lock(&_M_p); + while (__pred(_M_p, __old)) + __lock._M_wait(); + } + + void + _M_notify() noexcept + { + _Sp_locker __lock(&_M_p, true); + __lock._M_notify(); + } + + private: + value_type _M_p; + }; + + template<typename _Tp> + class atomic<shared_ptr<_Tp>> + { + using __impl_t = _Sp_impl<shared_ptr<_Tp>>; + + public: + using value_type = __impl_t::value_type; + + static constexpr bool + is_always_lock_free = __impl_t::_M_is_always_lock_free; + + bool + is_lock_free() const noexcept + { return __impl_t::_M_is_lock_free(); } + + constexpr atomic() noexcept = default; + atomic(value_type __r) noexcept + : _M_impl(move(__r)) + { } + + atomic(const atomic&) = delete; + void operator=(const atomic&) = delete; + + value_type + load(memory_order __o = memory_order_seq_cst) const noexcept + { return _M_impl._M_load(__o); } + + operator shared_ptr<_Tp>() const noexcept + { return load(); } + + void + store(value_type __r, memory_order __o = memory_order_seq_cst) noexcept + { _M_impl._M_store(move(__r), __o); } + + void operator=(value_type __r) noexcept + { store(move(__r)); } + + value_type + exchange(value_type __r, memory_order __o = memory_order_seq_cst) noexcept + { return _M_impl._M_exchange(move(__r), __o); } + + bool + compare_exchange_strong(value_type& __v, value_type __w, + memory_order, memory_order) noexcept + { + return _M_impl._M_compare_exchange_strong(__v, move(__w), + [](const auto& __a, const auto& __b) { return _S_compare(__a, __b); }, + memory_order_seq_cst, memory_order_seq_cst); + } + + bool + compare_exchange_strong(value_type& __v, value_type __w, + memory_order __o = memory_order_seq_cst) noexcept + { + return _M_impl._M_compare_exchange_strong(__v, move(__w), + [](const auto& __a, const auto& __b) { return _S_compare(__a, __b); }, + __o, __o); + } + + bool + compare_exchange_weak(value_type& __v, value_type __w, + memory_order, memory_order) noexcept + { + return _M_impl._M_compare_exchange_strong(__v, move(__w), + [](const auto& __a, const auto& __b) { return _S_compare(__a, __b); }, + memory_order_seq_cst, memory_order_seq_cst); + } + + bool + compare_exchange_weak(value_type& __v, value_type __w, + memory_order __o = memory_order_seq_cst) noexcept + { + return _M_impl._M_compare_exchange_strong(__v, move(__w), + [](const auto& __a, const auto& __b) { return _S_compare(__a, __b); }, + __o, __o); + } + + void + wait(value_type __old, memory_order __o = memory_order_seq_cst) const noexcept + { + _M_impl._M_wait(move(__old), + [](const auto& __a, const auto& __b) { return _S_compare(__a, __b); }, + __o); + } + + void + notify_one() noexcept + { _M_impl._M_notify(); } + + void + notify_all() noexcept + { _M_impl._M_notify(); } + + private: + __impl_t _M_impl; + + static bool + _S_compare(const value_type& __a, const value_type& __b) noexcept + { + owner_less<value_type> __less; + return __a == __b && !__less(__a, __b) && !__less(__b, __a); + } + }; + + template<typename _Tp> + class atomic<weak_ptr<_Tp>> + { + using __impl_t = _Sp_impl<weak_ptr<_Tp>>; + + public: + using value_type = __impl_t::value_type; + + static constexpr bool + is_always_lock_free = __impl_t::_M_is_always_lock_free; + + bool + is_lock_free() const noexcept + { return __impl_t::_M_is_lock_free(); } + + constexpr atomic() noexcept = default; + atomic(value_type __r) noexcept + : _M_impl(move(__r)) + { } + + atomic(const atomic&) = delete; + void operator=(const atomic&) = delete; + + value_type + load(memory_order __o = memory_order_seq_cst) const noexcept + { return _M_impl._M_load(__o); } + + operator weak_ptr<_Tp>() const noexcept + { return load(); } + + void + store(value_type __r, memory_order __o = memory_order_seq_cst) noexcept + { _M_impl._M_store(move(__r), __o); } + + void operator=(value_type __r) noexcept + { store(move(__r)); } + + value_type + exchange(value_type __r, memory_order __o = memory_order_seq_cst) noexcept + { return _M_impl._M_exchange(move(__r), __o); } + + bool + compare_exchange_strong(value_type& __v, value_type __w, + memory_order, memory_order) noexcept + { + return _M_impl._M_compare_exchange_strong(__v, move(__w), + [](const auto& __a, const auto& __b) { return _S_compare(__a, __b); }, + memory_order_seq_cst, memory_order_seq_cst); + } + + bool + compare_exchange_strong(value_type& __v, value_type __w, + memory_order __o = memory_order_seq_cst) noexcept + { + return _M_impl._M_compare_exchange_strong(__v, move(__w), + [](const auto& __a, const auto& __b) { return _S_compare(__a, __b); }, + __o, __o); + } + + bool + compare_exchange_weak(value_type& __v, value_type __w, + memory_order, memory_order) noexcept + { + return _M_impl._M_compare_exchange_strong(__v, move(__w), + [](const auto& __a, const auto& __b) { return _S_compare(__a, __b); }, + memory_order_seq_cst, memory_order_seq_cst); + } + + bool + compare_exchange_weak(value_type& __v, value_type __w, + memory_order __o = memory_order_seq_cst) noexcept + { + return _M_impl._M_compare_exchange_strong(__v, move(__w), + [](const auto& __a, const auto& __b) { return _S_compare(__a, __b); }, + __o, __o); + } + + void + wait(value_type __old, memory_order __o = memory_order_seq_cst) const noexcept + { + _M_impl._M_wait(move(__old), + [](const auto& __a, const auto& __b) { return _S_compare(__a, __b); }, + __o); + } + + void + notify_one() noexcept + { _M_impl._M_notify(); } + + void + notify_all() noexcept + { _M_impl._M_notify(); } + + private: + __impl_t _M_impl; + + static bool + _S_compare(const value_type& __a, const value_type& __b) noexcept + { + owner_less<value_type> __less; + const auto& __sa = __a.lock(); + const auto& __sb = __b.lock(); + return __sa == __sb && !__less(__a, __b) && !__less(__b, __a); + } + }; +#endif // __cpp_lib_atomic_shared_ptr + /// @} relates shared_ptr /// @} group pointer_abstractions diff --git a/libstdc++-v3/include/std/version b/libstdc++-v3/include/std/version index f41004b5911..37e5f0e5175 100644 --- a/libstdc++-v3/include/std/version +++ b/libstdc++-v3/include/std/version @@ -179,6 +179,7 @@ #define __cpp_lib_atomic_flag_test 201907L #define __cpp_lib_atomic_float 201711L #define __cpp_lib_atomic_ref 201806L +#define __cpp_lib_atomic_shared_ptr 201711L #define __cpp_lib_atomic_value_initialization 201911L #define __cpp_lib_bitops 201907L #define __cpp_lib_bounded_array_traits 201902L diff --git a/libstdc++-v3/src/c++11/Makefile.am b/libstdc++-v3/src/c++11/Makefile.am index a26903db6bc..72ab7ac22fe 100644 --- a/libstdc++-v3/src/c++11/Makefile.am +++ b/libstdc++-v3/src/c++11/Makefile.am @@ -136,6 +136,12 @@ limits.lo: limits.cc limits.o: limits.cc $(CXXCOMPILE) -fchar8_t -c $< +# Use -std=gnu++20 for shared_ptr.cc +shared_ptr.lo: shared_ptr.cc + $(LTCXXCOMPILE) -std=gnu++20 -c $< +shared_ptr.o: shared_ptr.cc + $(CXXCOMPILE) -std=gnu++20 -c $< + if ENABLE_DUAL_ABI # Rewrite the type info for __ios_failure. rewrite_ios_failure_typeinfo = sed -e '/^_*_ZTISt13__ios_failure:/,/_ZTVN10__cxxabiv120__si_class_type_infoE/s/_ZTVN10__cxxabiv120__si_class_type_infoE/_ZTVSt19__iosfail_type_info/' diff --git a/libstdc++-v3/src/c++11/shared_ptr.cc b/libstdc++-v3/src/c++11/shared_ptr.cc index 4678fbeffe2..144273e6cea 100644 --- a/libstdc++-v3/src/c++11/shared_ptr.cc +++ b/libstdc++-v3/src/c++11/shared_ptr.cc @@ -22,12 +22,56 @@ // see the files COPYING3 and COPYING.RUNTIME respectively. If not, see // <http://www.gnu.org/licenses/>. +#include <cerrno> #include <memory> #include "mutex_pool.h" namespace __gnu_internal _GLIBCXX_VISIBILITY(hidden) { + struct __condvar + { + __condvar() noexcept + { +#ifndef __GTHREAD_COND_INIT + __GTHREAD_COND_INIT_FUNCTION(&_M_cond); +#endif + } + + ~__condvar() + { + int __e __attribute__((__unused__)) = __gthread_cond_destroy(&_M_cond); + __glibcxx_assert(__e != EBUSY); // threads are still blocked + } + + __condvar(const __condvar&) = delete; + __condvar& operator=(const __condvar&) = delete; + + void + _M_wait(__gnu_cxx::__mutex& __m) noexcept + { + int __e __attribute__((__unused__)) + = __gthread_cond_wait(&_M_cond, __m.gthread_mutex()); + __glibcxx_assert(__e == 0); + } + + void + _M_notify() noexcept + { + // we do a broadcast here because multiple threads may multiplex a wait + // against the same __condvar + int __e __attribute__((__unused__)) = __gthread_cond_broadcast(&_M_cond); + __glibcxx_assert(__e == 0); + } + + private: +#ifdef __GTHREAD_COND_INIT + __gthread_cond_t _M_cond = __GTHREAD_COND_INIT; +#else + __gthread_cond_t _M_cond; +#endif + }; + /* Returns different instances of __mutex depending on the passed index * in order to limit contention. */ @@ -39,6 +83,18 @@ namespace __gnu_internal _GLIBCXX_VISIBILITY(hidden) static M m[mask + 1]; return m[i]; } + + /* Returns different instances of __condvar depending on the passed index + * in order to limit contention + */ + __condvar& + get_condvar(unsigned char i) + { + // increase alignment to put each condvar on a separate cache line + struct alignas(64) CV : __condvar { }; + static CV cv[mask + 1]; + return cv[i]; + } } namespace std _GLIBCXX_VISIBILITY(default) @@ -90,10 +146,38 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION if (_M_key1 != __gnu_internal::invalid) { __gnu_internal::get_mutex(_M_key1).unlock(); - if (_M_key2 != _M_key1) +#if __cpp_lib_atomic_shared_ptr + if ((_M_key2 != _M_key1) && (_M_key2 != __gnu_internal::invalid)) +#else + if ((_M_key2 != _M_key1)) +#endif __gnu_internal::get_mutex(_M_key2).unlock(); } } + +#if __cpp_lib_atomic_shared_ptr + _Sp_locker::_Sp_locker(const void* p, bool) noexcept + { + _M_key1 = key(p); + _M_key2 = __gnu_internal::invalid; + } + + void + _Sp_locker::_M_wait() noexcept + { + __glibcxx_assert(_M_key1 == _M_key2); // can't hold two locks + __gnu_cxx::__mutex& m = __gnu_internal::get_mutex(_M_key1); + __gnu_internal::get_condvar(_M_key1)._M_wait(m); + } + + void + _Sp_locker::_M_notify() noexcept + { + __glibcxx_assert(_M_key2 == __gnu_internal::invalid); // can't hold a lock + __gnu_internal::get_condvar(_M_key1)._M_notify(); + } +#endif + #endif bool diff --git a/libstdc++-v3/testsuite/20_util/shared_ptr/atomic/4.cc b/libstdc++-v3/testsuite/20_util/shared_ptr/atomic/4.cc new file mode 100644 index 00000000000..aa9fce9e235 --- /dev/null +++ b/libstdc++-v3/testsuite/20_util/shared_ptr/atomic/4.cc @@ -0,0 +1,28 @@ +// Copyright (C) 2021 Free Software Foundation, Inc. +// +// This file is part of the GNU ISO C++ Library. This library is free +// software; you can redistribute it and/or modify it under the +// terms of the GNU General Public License as published by the +// Free Software Foundation; either version 3, or (at your option) +// any later version. + +// This library is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License along +// with this library; see the file COPYING3. If not see +// <http://www.gnu.org/licenses/>. + +// { dg-options "-std=gnu++2a" } +// { dg-do compile { target c++2a } } +// { dg-require-effective-target gthreads } + +#include <memory> + +#ifndef __cpp_lib_atomic_shared_ptr +# error "Feature-test macro for atomic<shared_ptr<T>> missing in <memory>" +#elif __cpp_lib_atomic_shared_ptr != 201711L +# error "Feature-test macro for atomic<shared_ptr<T>> has wrong value in <memory>" +#endif diff --git a/libstdc++-v3/testsuite/20_util/shared_ptr/atomic/5.cc b/libstdc++-v3/testsuite/20_util/shared_ptr/atomic/5.cc new file mode 100644 index 00000000000..5f8791f78b5 --- /dev/null +++ b/libstdc++-v3/testsuite/20_util/shared_ptr/atomic/5.cc @@ -0,0 +1,28 @@ +// Copyright (C) 2021 Free Software Foundation, Inc. +// +// This file is part of the GNU ISO C++ Library. This library is free +// software; you can redistribute it and/or modify it under the +// terms of the GNU General Public License as published by the +// Free Software Foundation; either version 3, or (at your option) +// any later version. + +// This library is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License along +// with this library; see the file COPYING3. If not see +// <http://www.gnu.org/licenses/>. + +// { dg-options "-std=gnu++2a" } +// { dg-do compile { target c++2a } } +// { dg-require-effective-target gthreads } + +#include <version> + +#ifndef __cpp_lib_atomic_shared_ptr +# error "Feature-test macro for atomic<shared_ptr<T>> missing in <version>" +#elif __cpp_lib_atomic_shared_ptr != 201711L +# error "Feature-test macro for atomic<shared_ptr<T>> has wrong value in <version>" +#endif diff --git a/libstdc++-v3/testsuite/20_util/shared_ptr/atomic/atomic_shared_ptr.cc b/libstdc++-v3/testsuite/20_util/shared_ptr/atomic/atomic_shared_ptr.cc new file mode 100644 index 00000000000..7a34148a974 --- /dev/null +++ b/libstdc++-v3/testsuite/20_util/shared_ptr/atomic/atomic_shared_ptr.cc @@ -0,0 +1,159 @@ +// Copyright (C) 2020-2021 Free Software Foundation, Inc. +// +// This file is part of the GNU ISO C++ Library. This library is free +// software; you can redistribute it and/or modify it under the +// terms of the GNU General Public License as published by the +// Free Software Foundation; either version 3, or (at your option) +// any later version. + +// This library is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. + +// You should have received a copy of the GNU General Public License along +// with this library; see the file COPYING3. If not see +// <http://www.gnu.org/licenses/>. + +// { dg-options "-std=gnu++2a" } +// { dg-do run { target c++2a } } +// { dg-additional-options "-pthread" { target pthread } } +// { dg-require-gthreads "" } + +#include <memory> +#include <thread> + +#include <testsuite_hooks.h> + +template<typename Tp> + void + test_is_lock_free() + { + using test_type = std::atomic<Tp>; + VERIFY( test_type::is_always_lock_free == false ); + + test_type p; + VERIFY( p.is_lock_free() == false ); + } + +struct A { int a; int b; }; + +template<typename Tp, typename Ta> + void + test_wait_notify(Tp& p, const Ta& a, const Ta& b) + { + p.store(a); + p.wait(b); + std::thread t([&] + { + p.store(b); + p.notify_one(); + }); + p.wait(a); + t.join(); + } + +void +test_atomic_shared_ptr() +{ + test_is_lock_free<std::shared_ptr<int>>(); + auto a = std::make_shared<A>( 0, 42 ); + using ptr_t = std::shared_ptr<A>; + { + std::atomic<ptr_t> p{ }; + VERIFY( p.load().get() == nullptr ); + } + + std::atomic<ptr_t> p{ a }; + VERIFY( p.load().get() == a.get() ); + auto b = std::make_shared<A>( 42, 0 ); + p.store(b); + VERIFY( p.load().get() != a.get() ); + VERIFY( p.load().get() == b.get() ); + p.exchange(a); + VERIFY( p.load().get() != b.get() ); + VERIFY( p.load().get() == a.get() ); + + { + ptr_t aa{ a }; + VERIFY( p.compare_exchange_strong(aa, b, + std::memory_order_seq_cst, + std::memory_order_seq_cst) == true ); + ptr_t bb{ a }; + VERIFY( p.compare_exchange_strong(bb, b, + std::memory_order_seq_cst, + std::memory_order_seq_cst) == false ); + VERIFY( bb.get() == b.get() ); + } + + { + ptr_t bb{ b }; + VERIFY( p.compare_exchange_weak(bb, a, + std::memory_order_seq_cst, + std::memory_order_seq_cst) == true ); + ptr_t aa{ b }; + VERIFY( p.compare_exchange_weak(aa, a, + std::memory_order_seq_cst, + std::memory_order_seq_cst) == false ); + VERIFY( aa.get() == a.get() ); + } + test_wait_notify(p, a, b); +} + +void +test_atomic_weak_ptr() +{ + test_is_lock_free<std::weak_ptr<int>>(); + auto a = std::make_shared<A>( 0, 42 ); + using ptr_t = std::weak_ptr<A>; + ptr_t wa{ a }; + { + std::atomic<ptr_t> p{ }; + VERIFY( p.load().lock().get() == nullptr ); + } + + std::atomic<ptr_t> p{ wa }; + VERIFY( p.load().lock().get() == a.get() ); + + auto b = std::make_shared<A>( 42, 0 ); + ptr_t wb{ b }; + p.store(wb); + VERIFY( p.load().lock().get() != a.get() ); + VERIFY( p.load().lock().get() == b.get() ); + p.exchange(wa); + VERIFY( p.load().lock().get() != b.get() ); + VERIFY( p.load().lock().get() == a.get() ); + + { + ptr_t aa{ a }; + VERIFY( p.compare_exchange_strong(aa, b, + std::memory_order_seq_cst, + std::memory_order_seq_cst) == true ); + ptr_t bb{ a }; + VERIFY( p.compare_exchange_strong(bb, b, + std::memory_order_seq_cst, + std::memory_order_seq_cst) == false ); + VERIFY( bb.lock().get() == b.get() ); + } + + { + ptr_t bb{ b }; + VERIFY( p.compare_exchange_weak(bb, a, + std::memory_order_seq_cst, + std::memory_order_seq_cst) == true ); + ptr_t aa{ b }; + VERIFY( p.compare_exchange_weak(aa, a, + std::memory_order_seq_cst, + std::memory_order_seq_cst) == false ); + VERIFY( aa.lock().get() == a.get() ); + } + test_wait_notify(p, wa, wb); +} + +int +main() +{ + test_atomic_shared_ptr(); + test_atomic_weak_ptr(); + return 0; +} diff --git a/libstdc++-v3/testsuite/util/testsuite_abi.cc b/libstdc++-v3/testsuite/util/testsuite_abi.cc index 3af5dc593c2..1ca7da4fcd0 100644 --- a/libstdc++-v3/testsuite/util/testsuite_abi.cc +++ b/libstdc++-v3/testsuite/util/testsuite_abi.cc @@ -210,6 +210,7 @@ check_version(symbol& test, bool added) known_versions.push_back("GLIBCXX_3.4.27"); known_versions.push_back("GLIBCXX_3.4.28"); known_versions.push_back("GLIBCXX_3.4.29"); + known_versions.push_back("GLIBCXX_3.4.30"); known_versions.push_back("GLIBCXX_LDBL_3.4.29"); known_versions.push_back("GLIBCXX_IEEE128_3.4.29"); known_versions.push_back("CXXABI_1.3"); @@ -245,7 +246,7 @@ check_version(symbol& test, bool added) test.version_status = symbol::incompatible; // Check that added symbols are added in the latest pre-release version. - bool latestp = (test.version_name == "GLIBCXX_3.4.29" + bool latestp = (test.version_name == "GLIBCXX_3.4.30" // XXX remove next 3 lines when baselines have been regenerated // to include {IEEE128,LDBL} symbols: || test.version_name == "GLIBCXX_LDBL_3.4.29"