Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix fence on non-x86 arch and miri #16

Merged
merged 1 commit into from
Jul 17, 2022
Merged

Fix fence on non-x86 arch and miri #16

merged 1 commit into from
Jul 17, 2022

Conversation

taiki-e
Copy link
Collaborator

@taiki-e taiki-e commented Jul 17, 2022

The problem seems to be that the original author of this code confused fence in the x86 hardware memory model with atomic fence in the C++ memory model. (On x86, lock cmpxchg; mov (load from memory) is fine. See also https://www.cl.cam.ac.uk/~pes20/cpp/cpp0xmappings.html. On C++ memory model and many architectures, fence for load should be load; fence)

Fixes bevyengine/bevy#5164
FYI @cbeuw

@taiki-e
Copy link
Collaborator Author

taiki-e commented Jul 17, 2022

At least crossbeam and event-listener also have the same issue, but fixing them is probably more complex...

@taiki-e taiki-e merged commit 54df36a into master Jul 17, 2022
@taiki-e taiki-e deleted the fence branch July 17, 2022 13:33
@sbarral
Copy link

sbarral commented Jul 19, 2022

I feel a bit uncomfortable with this commit.

Admittedly, I don't know what exactly is the role of the fence here. This fence does not exist in Dmitry Vyukov's original implementation of the queue, so I guess it was added as part of the modifications that ensure that this queue is linearisable (unlike the original queue).

That being said, if the cross-platform solution is indeed to place the load before the fence (this, I do not know) then I am pretty sure that the intel specialization that uses a lock operation instead of an mfence should also keep the load before.

I did look at https://www.cl.cam.ac.uk/~pes20/cpp/cpp0xmappings.html but could not see where it states that lock + mov (in this order) is equivalent to mov + mfence. In fact, the latest GCC does use the lock optimization and definitely preserves the order, i.e. mov + lock (see this godbolt: https://godbolt.org/z/o3rYdTvYv).

taiki-e added a commit that referenced this pull request Jul 20, 2022
taiki-e added a commit that referenced this pull request Jul 20, 2022
@@ -461,7 +464,11 @@ fn full_fence() {
// x86 platforms is going to optimize this away.
let a = AtomicUsize::new(0);
let _ = a.compare_exchange(0, 1, Ordering::SeqCst, Ordering::SeqCst);
// On x86, `lock cmpxchg; mov` is fine. See also https://www.cl.cam.ac.uk/~pes20/cpp/cpp0xmappings.html.
load_op()
Copy link

@RalfJung RalfJung Jul 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, this is still Rust code -- so if Miri complains when running this branch of the code (which I suspect it will, since a SC RMW before a load cannot replace a fence after a load), then this code is still wrong.

When you write Rust code, the hardware memory model is all but irrelevant for program correctness. Only the Rust memory model counts.

EDIT: Oh I see this got reverted in #18.

@@ -461,7 +464,11 @@ fn full_fence() {
// x86 platforms is going to optimize this away.
Copy link

@RalfJung RalfJung Jul 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fact that you are hoping that "sane" compilers for particular targets are going to treat the memory model differently, is a big red flag. The memory model is target-independent, and a whole bunch of optimizations run on this code (including its use of atomics) before any target-specific concerns are applied.

Inline assembly is the only correct choice here.

EDIT: Oh I see this got reverted in #18.

@RalfJung
Copy link

That being said, if the cross-platform solution is indeed to place the load before the fence (this, I do not know) then I am pretty sure that the intel specialization that uses a lock operation instead of an mfence should also keep the load before.

I would usually expect that to be the case -- a relaxed load followed by an acquire-or-stronger fence can induce a synchronization edge. But I don't know the context for this particular code.

Does something break, or perf go down badly, if the fence is moved after the load?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

world::World doctests sometimes hangs with Miri
3 participants