miri no longer builds after rust-lang/rust#95624

[rust-highfive]

Hello, this is your friendly neighborhood mergebot.
After merging PR #95624, I observed that the tool miri no longer builds.
A follow-up PR to the repository https://github.com/rust-lang/miri is needed to fix the fallout.

cc @Dylan-DPC, do you think you would have time to do the follow-up work?
If so, that would be great!

[RalfJung]

#95035 makes this somewhat tricky, Miri will have to support more ways of using futex. (It supported enough for the thread parker implementation, but now it seems we are using more of that API.)

[RalfJung]

We support FUTEX_WAIT and FUTEX_WAKE, but not FUTEX_WAIT_BITSET.

@m-ou-se what would you say is a good place to learn about futex semantics, ideally suitable as a reference documentation that one can follow if one wants to do a reimplementation in an interpreter?

[m-ou-se]

The man page for the Linux syscall is the best documentation/reference: https://man7.org/linux/man-pages/man2/futex.2.html

FUTEX_WAIT_BITSET shouldn't be hard to implement. I can work on Miri support for it. :)

[RalfJung]

That would be great. :)

Best to start with ./rustup-toolchain 306ba8357fb36212b7d30efb9eb9e41659ac1445 (rather than the usual ./rustup-toolchain HEAD) as otherwise you will have to also deal with other breakage that happened after the futex change.

[m-ou-se]

This adds support for FUTEX_WAIT_BITSET and FUTEX_WAKE_BITSET in Miri: rust-lang/miri#2054

[RalfJung]

The next trouble is #95469 -- somehow this broke printing to stdout in Miri on Windows targets, which now just loops infinitely. @ChrisDenton any idea what might be going on?

[ChrisDenton]

I made a quick summary in #95759 but still need to investigate properly. Short version is that in some cases we're passing asynchronous handles to child processes, which is unsound when read synchronously (but mostly seemed to work in practice).

Though I'm not actually sure why that should make it loop infinitely since it's calling std::sys::abort_internal?

[RalfJung]

Note that this happens even on a Linux host interpreting Windows target MIR. So it might be a different problem?

[RalfJung]

The odd thing is, these tests do not even read/write a file (Miri does not support that on Windows targets). They just write to stderr/stdout. If NtRead/WriteFile were ever executed, Miri would stop immediately since it does not support those operations.

[ChrisDenton]

Oh right! Hm... I'm not really sure why that be. The changes are limited to reading and writing from a handle.

[ChrisDenton]

I had another look and as far I can tell there's no code path that doesn't call either NtReadFile or NtWriteFile. The only thing of note it does before that is call c::IO_STATUS_BLOCK::default().

Just to be clear, you're totally certain it was that PR that caused the issue?

[RalfJung]

Yeah I bisected it to this PR in rust-lang/miri#2056. (I can double-check later.)

[ChrisDenton]

Then I'll admit to being stumped. 😕

[RalfJung]

Does the Windows stdout/stderr go through a "handle" or is it a separate code path?

[ChrisDenton]

Ah, I did recently change piped stdout/stderr to use a loop (in #95467). But it also uses new functions to do that (WriteFIleEx and SleepEx).

[RalfJung]

But Miri works fine on bbe9d27 which includes that PR...

[RalfJung]

Oh, I think it's pretty simple -- NtWriteFile is declared via compat_fn, and in Miri that function is just not available. Then what happens is that it panics, which triggers printing to stderr, so it tries calling NtWriteFile again, and the loop continues.

It doesn't make a lot of sense for NtWriteFile to be a compat_fn when the "fallback" branch consists in an infinite loop on any write to stdout/stderr...

[ChrisDenton]

Oh! Sorry! I did that without even thinking and it didn't even occur to me look at that again. The fallback should probably just abort or return an error. I'll make a PR.

[RalfJung]

Uff, that compat stuff is using some nasty things which Miri would all have to support. :/

Is there any chance these functions could be called like regular 'extern fn', without all that compat craziness? I doubt we are making stdin/stdout optional so for all intents and purposes those functions have to exist for Rust programs to run on Windows.

[ChrisDenton]

The problem is these Nt functions really should be loaded at runtime. I guess we could technically do it at load time but we'd at least need to wait until the raw_dylib feature is stabilized because we can't necessarily count on users having the right import library.

[RalfJung]

An alternative might be a hack like this: #95775.
Unfortunately I don't have the time to implement wild linker stuff in Miri for a platform that Miri only barely supports anyway...

[RalfJung]

Miri CI finally passes again. 🎉

This took a lot more time than I actually have, so I think next time Windows breaks in obscure ways I will have to just disable the tests for that platform and hope someone else can maintain that code.