Skip to content

Commit

Permalink
Rollup merge of #98383 - m-ou-se:remove-memory-order-restrictions, r=…
Browse files Browse the repository at this point in the history 
…joshtriplett

Remove restrictions on compare-exchange memory ordering.

We currently don't allow the failure memory ordering of compare-exchange operations to be stronger than the success ordering, as was the case in C++11 when its memory model was copied to Rust. However, this restriction was lifted in C++17 as part of [p0418r2](https://wg21.link/p0418r2). It's time  we lift the restriction too.

| Success | Failure | Before | After |
|---------|---------|--------|-------|
| Relaxed | Relaxed | ✔️ | ✔️ |
| Relaxed | Acquire | ❌                | ✔️ |
| Relaxed | SeqCst  | ❌                | ✔️ |
| Acquire | Relaxed | ✔️ | ✔️ |
| Acquire | Acquire | ✔️ | ✔️ |
| Acquire | SeqCst  | ❌                | ✔️ |
| Release | Relaxed | ✔️ | ✔️ |
| Release | Acquire | ❌                | ✔️ |
| Release | SeqCst  | ❌                | ✔️ |
| AcqRel  | Relaxed | ✔️ | ✔️ |
| AcqRel  | Acquire | ✔️ | ✔️ |
| AcqRel  | SeqCst  | ❌                | ✔️ |
| SeqCst  | Relaxed | ✔️ | ✔️ |
| SeqCst  | Acquire | ✔️ | ✔️ |
| SeqCst  | SeqCst  | ✔️ | ✔️ |
| \*      | Release | ❌                | ❌                |
| \*      | AcqRel  | ❌                | ❌                |

Fixes #68464
  • Loading branch information
@JohnTitor
JohnTitor authored Jul 17, 2022
2 parents 263edd4 + a898f41 commit 796bc7c
Showing 1 changed file with 35 additions and 30 deletions.
65 changes: 35 additions & 30 deletions library/core/src/sync/atomic.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -581,8 +581,7 @@ impl AtomicBool {
/// `failure` describes the required ordering for the load operation that takes place when
/// the comparison fails. Using [`Acquire`] as success ordering makes the store part
/// of this operation [`Relaxed`], and using [`Release`] makes the successful load
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`]
/// and must be equivalent to or weaker than the success ordering.
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`].
///
/// **Note:** This method is only available on platforms that support atomic
/// operations on `u8`.
Expand Down Expand Up @@ -640,8 +639,7 @@ impl AtomicBool {
/// `failure` describes the required ordering for the load operation that takes place when
/// the comparison fails. Using [`Acquire`] as success ordering makes the store part
/// of this operation [`Relaxed`], and using [`Release`] makes the successful load
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`]
/// and must be equivalent to or weaker than the success ordering.
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`].
///
/// **Note:** This method is only available on platforms that support atomic
/// operations on `u8`.
Expand Down Expand Up @@ -941,8 +939,7 @@ impl AtomicBool {
/// Using [`Acquire`] as success ordering makes the store part of this
/// operation [`Relaxed`], and using [`Release`] makes the final successful
/// load [`Relaxed`]. The (failed) load ordering can only be [`SeqCst`],
/// [`Acquire`] or [`Relaxed`] and must be equivalent to or weaker than the
/// success ordering.
/// [`Acquire`] or [`Relaxed`].
///
/// **Note:** This method is only available on platforms that support atomic
/// operations on `u8`.
Expand Down Expand Up @@ -1301,8 +1298,7 @@ impl<T> AtomicPtr<T> {
/// `failure` describes the required ordering for the load operation that takes place when
/// the comparison fails. Using [`Acquire`] as success ordering makes the store part
/// of this operation [`Relaxed`], and using [`Release`] makes the successful load
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`]
/// and must be equivalent to or weaker than the success ordering.
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`].
///
/// **Note:** This method is only available on platforms that support atomic
/// operations on pointers.
Expand Down Expand Up @@ -1347,8 +1343,7 @@ impl<T> AtomicPtr<T> {
/// `failure` describes the required ordering for the load operation that takes place when
/// the comparison fails. Using [`Acquire`] as success ordering makes the store part
/// of this operation [`Relaxed`], and using [`Release`] makes the successful load
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`]
/// and must be equivalent to or weaker than the success ordering.
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`].
///
/// **Note:** This method is only available on platforms that support atomic
/// operations on pointers.
Expand Down Expand Up @@ -1404,8 +1399,7 @@ impl<T> AtomicPtr<T> {
/// Using [`Acquire`] as success ordering makes the store part of this
/// operation [`Relaxed`], and using [`Release`] makes the final successful
/// load [`Relaxed`]. The (failed) load ordering can only be [`SeqCst`],
/// [`Acquire`] or [`Relaxed`] and must be equivalent to or weaker than the
/// success ordering.
/// [`Acquire`] or [`Relaxed`].
///
/// **Note:** This method is only available on platforms that support atomic
/// operations on pointers.
Expand Down Expand Up @@ -2227,8 +2221,7 @@ macro_rules! atomic_int {
/// `failure` describes the required ordering for the load operation that takes place when
/// the comparison fails. Using [`Acquire`] as success ordering makes the store part
/// of this operation [`Relaxed`], and using [`Release`] makes the successful load
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`]
/// and must be equivalent to or weaker than the success ordering.
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`].
///
/// **Note**: This method is only available on platforms that support atomic operations on
#[doc = concat!("[`", $s_int_type, "`].")]
Expand Down Expand Up @@ -2279,8 +2272,7 @@ macro_rules! atomic_int {
/// `failure` describes the required ordering for the load operation that takes place when
/// the comparison fails. Using [`Acquire`] as success ordering makes the store part
/// of this operation [`Relaxed`], and using [`Release`] makes the successful load
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`]
/// and must be equivalent to or weaker than the success ordering.
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`].
///
/// **Note**: This method is only available on platforms that support atomic operations on
#[doc = concat!("[`", $s_int_type, "`].")]
Expand Down Expand Up @@ -2517,8 +2509,7 @@ macro_rules! atomic_int {
///
/// Using [`Acquire`] as success ordering makes the store part
/// of this operation [`Relaxed`], and using [`Release`] makes the final successful load
/// [`Relaxed`]. The (failed) load ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`]
/// and must be equivalent to or weaker than the success ordering.
/// [`Relaxed`]. The (failed) load ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`].
///
/// **Note**: This method is only available on platforms that support atomic operations on
#[doc = concat!("[`", $s_int_type, "`].")]
Expand Down Expand Up @@ -3035,22 +3026,29 @@ unsafe fn atomic_compare_exchange<T: Copy>(
let (val, ok) = unsafe {
match (success, failure) {
(Relaxed, Relaxed) => intrinsics::atomic_cxchg_relaxed_relaxed(dst, old, new),
//(Relaxed, Acquire) => intrinsics::atomic_cxchg_relaxed_acquire(dst, old, new),
//(Relaxed, SeqCst) => intrinsics::atomic_cxchg_relaxed_seqcst(dst, old, new),
#[cfg(not(bootstrap))]
(Relaxed, Acquire) => intrinsics::atomic_cxchg_relaxed_acquire(dst, old, new),
#[cfg(not(bootstrap))]
(Relaxed, SeqCst) => intrinsics::atomic_cxchg_relaxed_seqcst(dst, old, new),
(Acquire, Relaxed) => intrinsics::atomic_cxchg_acquire_relaxed(dst, old, new),
(Acquire, Acquire) => intrinsics::atomic_cxchg_acquire_acquire(dst, old, new),
//(Acquire, SeqCst) => intrinsics::atomic_cxchg_acquire_seqcst(dst, old, new),
#[cfg(not(bootstrap))]
(Acquire, SeqCst) => intrinsics::atomic_cxchg_acquire_seqcst(dst, old, new),
(Release, Relaxed) => intrinsics::atomic_cxchg_release_relaxed(dst, old, new),
//(Release, Acquire) => intrinsics::atomic_cxchg_release_acquire(dst, old, new),
//(Release, SeqCst) => intrinsics::atomic_cxchg_release_seqcst(dst, old, new),
#[cfg(not(bootstrap))]
(Release, Acquire) => intrinsics::atomic_cxchg_release_acquire(dst, old, new),
#[cfg(not(bootstrap))]
(Release, SeqCst) => intrinsics::atomic_cxchg_release_seqcst(dst, old, new),
(AcqRel, Relaxed) => intrinsics::atomic_cxchg_acqrel_relaxed(dst, old, new),
(AcqRel, Acquire) => intrinsics::atomic_cxchg_acqrel_acquire(dst, old, new),
//(AcqRel, SeqCst) => intrinsics::atomic_cxchg_acqrel_seqcst(dst, old, new),
#[cfg(not(bootstrap))]
(AcqRel, SeqCst) => intrinsics::atomic_cxchg_acqrel_seqcst(dst, old, new),
(SeqCst, Relaxed) => intrinsics::atomic_cxchg_seqcst_relaxed(dst, old, new),
(SeqCst, Acquire) => intrinsics::atomic_cxchg_seqcst_acquire(dst, old, new),
(SeqCst, SeqCst) => intrinsics::atomic_cxchg_seqcst_seqcst(dst, old, new),
(_, AcqRel) => panic!("there is no such thing as an acquire-release failure ordering"),
(_, Release) => panic!("there is no such thing as a release failure ordering"),
#[cfg(bootstrap)]
_ => panic!("a failure ordering can't be stronger than a success ordering"),
}
};
Expand All @@ -3070,22 +3068,29 @@ unsafe fn atomic_compare_exchange_weak<T: Copy>(
let (val, ok) = unsafe {
match (success, failure) {
(Relaxed, Relaxed) => intrinsics::atomic_cxchgweak_relaxed_relaxed(dst, old, new),
//(Relaxed, Acquire) => intrinsics::atomic_cxchgweak_relaxed_acquire(dst, old, new),
//(Relaxed, SeqCst) => intrinsics::atomic_cxchgweak_relaxed_seqcst(dst, old, new),
#[cfg(not(bootstrap))]
(Relaxed, Acquire) => intrinsics::atomic_cxchgweak_relaxed_acquire(dst, old, new),
#[cfg(not(bootstrap))]
(Relaxed, SeqCst) => intrinsics::atomic_cxchgweak_relaxed_seqcst(dst, old, new),
(Acquire, Relaxed) => intrinsics::atomic_cxchgweak_acquire_relaxed(dst, old, new),
(Acquire, Acquire) => intrinsics::atomic_cxchgweak_acquire_acquire(dst, old, new),
//(Acquire, SeqCst) => intrinsics::atomic_cxchgweak_acquire_seqcst(dst, old, new),
#[cfg(not(bootstrap))]
(Acquire, SeqCst) => intrinsics::atomic_cxchgweak_acquire_seqcst(dst, old, new),
(Release, Relaxed) => intrinsics::atomic_cxchgweak_release_relaxed(dst, old, new),
//(Release, Acquire) => intrinsics::atomic_cxchgweak_release_acquire(dst, old, new),
//(Release, SeqCst) => intrinsics::atomic_cxchgweak_release_seqcst(dst, old, new),
#[cfg(not(bootstrap))]
(Release, Acquire) => intrinsics::atomic_cxchgweak_release_acquire(dst, old, new),
#[cfg(not(bootstrap))]
(Release, SeqCst) => intrinsics::atomic_cxchgweak_release_seqcst(dst, old, new),
(AcqRel, Relaxed) => intrinsics::atomic_cxchgweak_acqrel_relaxed(dst, old, new),
(AcqRel, Acquire) => intrinsics::atomic_cxchgweak_acqrel_acquire(dst, old, new),
//(AcqRel, SeqCst) => intrinsics::atomic_cxchgweak_acqrel_seqcst(dst, old, new),
#[cfg(not(bootstrap))]
(AcqRel, SeqCst) => intrinsics::atomic_cxchgweak_acqrel_seqcst(dst, old, new),
(SeqCst, Relaxed) => intrinsics::atomic_cxchgweak_seqcst_relaxed(dst, old, new),
(SeqCst, Acquire) => intrinsics::atomic_cxchgweak_seqcst_acquire(dst, old, new),
(SeqCst, SeqCst) => intrinsics::atomic_cxchgweak_seqcst_seqcst(dst, old, new),
(_, AcqRel) => panic!("there is no such thing as an acquire-release failure ordering"),
(_, Release) => panic!("there is no such thing as a release failure ordering"),
#[cfg(bootstrap)]
_ => panic!("a failure ordering can't be stronger than a success ordering"),
}
};
Expand Down

0 comments on commit 796bc7c

Please sign in to comment.