epoll ICE on tokio test #3858
Comments
|
@rustbot claim |
|
A quick fix would be removing the unwrap and only use the epoll fd if it still exists, but I think it is better to investigate why the assumption is violated. |
|
This is the test in tokio that ICE'd: #[test]
fn fs_shutdown_before_started() {
let rt = rt();
let _enter = rt.enter();
rt.shutdown_timeout(Duration::from_secs(1000));
let err: std::io::Error = Handle::current()
.block_on(fs::read_to_string("Cargo.toml"))
.unwrap_err();
assert_eq!(err.kind(), std::io::ErrorKind::Other);
let inner_err = err.get_ref().expect("no inner error");
assert_eq!(inner_err.to_string(), "background task failed");
}This is the reproducible I wrote to be tested in miri: use tokio::fs;
use tokio::runtime::Handle;
use tokio::time::Duration;
fn main() {
let rt = tokio::runtime::Builder::new_current_thread().enable_all().build().unwrap();
let _enter = rt.enter();
rt.shutdown_timeout(Duration::from_secs(1000));
let err: std::io::Error =
Handle::current().block_on(fs::read_to_string("Cargo.toml")).unwrap_err();
assert_eq!(err.kind(), std::io::ErrorKind::Other);
let inner_err = err.get_ref().expect("no inner error");
assert_eq!(inner_err.to_string(), "background task failed");
}The tokio test still ICE on latest nightly @Darksonn Could you help to take a look at this? I am not sure if my reproducible is equivalent to the original tokio test. |
|
It might be kind of random if it depends on the exact scheduler behavior. A particular Miri version is deterministic, but it is not deterministic across multiple versions. Different nightlies means you explore different interleavings. On the nightly where your example managed to cause the ICE, does it also cause the ICE with You can try running The goal should be to get a reproducer that doesn't use tokio, but directly uses epoll. But that is probably best done by actually understanding what happens, and then writing code to specifically hit that corner case. This is a Miri issue so I doubt tokio people will have a good idea of how to get to the bottom of this (but I may be wrong ofc :D ). It needs someone who understands the internal Miri implementation. Try adding more and more tracing to Miri until you understand where that invariant gets broken. |
|
Why does EpollEventInterest even store an FD number to begin with? That's a relatively fragile identifier that we should generally never use inside Miri. Why not store a (Weak)FileDescriptionRef? |
|
For instance, one could use |
RalfJung
added
C-bug
|
Thanks for the suggestions.
Yes, the ICE still happened after I added that flag.
For the test I have written, all seeds range from 0 to 63 won't ICE. I think it might help to check if I somewhat got the |
Good point, it is better to use I willl add a test for this. |
|
I see there are two fd numbers in |
|
I think the |
|
Okay, that field should then have a comment saying that is is only used to report the FD number back to the user in case of an event. |
|
Ok, now the test can ICE by adding |
Fix tokio test ICE Fixes rust-lang#3858 It turned out that the issue mentioned [here](rust-lang/miri#3858 (comment)) is the exact cause of ICE. So in this PR, I changed the type of ``EpollEventInterest::epfd`` from ``i32`` to ``WeakFileDescriptionRef``.
Miri ICE on this Tokio test: https://github.com/tokio-rs/tokio/blob/27539ae3bdfd08b7503f4919fe13299537cc16fc/tokio/tests/rt_handle_block_on.rs#L146. It panics on the unwrap in this line.
I am currently working on getting a reproducible. This
unwrapwill fail when we try to retrieve an epoll instance that is closed from fd table. It supposed to be guarded by theupgradeofweak_epoll_interest, because epoll file description is the only one that holds a strong ref toEpollInterest, so when the epoll instance is closed, the upgrade ofweak_epoll_interestshould fail. This ICE is a case whereweak_epoll_interestupgrade succeed, but the epoll file descriptor is closed.Full trace
Version:
The text was updated successfully, but these errors were encountered: