Consider exposing simulation's types in Tokio #1845
Comments
|
Thanks @davidbarsky for capturing this, I have written out a few more details on how we could integrate Simulation Testing for Tokiotokio-rs/simulation is an initial POC bringing deterministic simulation The way it currently works is by providing an #[async_trait]
pub trait Environment: Unpin + Sized + Clone + Send + 'static {
type TcpStream: TcpStream + Send + 'static + Unpin;
type TcpListener: TcpListener + Send + 'static + Unpin;
/// Spawn a task on the runtime provided by this [`Environment`].
fn spawn<F>(&self, future: F)
where
F: Future<Output = ()> + Send + 'static;
/// Return the time now according to the executor.
fn now(&self) -> time::Instant;
/// Returns a delay future which completes after the provided instant.
fn delay(&self, deadline: time::Instant) -> tokio_timer::Delay;
/// Returns a delay future which completes at some time from now.
fn delay_from(&self, from_now: time::Duration) -> tokio_timer::Delay {
let now = self.now();
self.delay(now + from_now)
}
/// Creates a timeout future which which will execute T until the timeout elapses.
fn timeout<T>(&self, value: T, timeout: time::Duration) -> tokio_timer::Timeout<T>;
/// Binds and returns a listener which can be used to listen for new connections.
async fn bind<A>(&self, addr: A) -> io::Result<Self::TcpListener>
where
A: Into<net::SocketAddr> + Send + Sync;
/// Connects to the specified addr, returning a [`TcpStream`] which can be
/// used to send and receive bytes.
///
/// [`TcpStream`]:`TcpStream`
async fn connect<A>(&self, addr: A) -> io::Result<Self::TcpStream>
where
A: Into<net::SocketAddr> + Send + Sync;
}The desired outcome of this is an application which is generic over sources of However, while this can be made to work in some cases, there are 3rd party ProposalWe propose adding simulation support directly to Tokio via a feature flag. This There are types which will need to be mocked when the feature flag is enabled. MocksSchedulingWhen the feature flag is enabled, all runtime creation will use the By disabling the remote queue, it appears that the Each spawned task will be wrapped in a top-level task which will provide Tasks which are spawned via `tokio::spawn` will automatically inherit the The task metadata will be optional to begin with, but in the future we would #[tokio::simulation]
fn simulation(seed: u64) {
tokio::simulation::spawn_process(
"us-east-1a", "rack-1", "server-1.svc.cluster.local", spawn_server()
);
tokio::simulation::spawn_process(
"us-east-1b", "rack-2", "server-2.svc.cluster.local", spawn_server()
);
tokio::simulation::spawn_process(
"us-east-1c", "rack-3", "server-3.svc.cluster.local", spawn_server()
);
tokio::spawn(tokio::simulation::clog_connections("us-east-1c", "us-east-1b"));
verify_linearizability().await;
}TimeThe mock clock will be advanced automatically by the executor whenever
NetworkingWhen the feature flag is enabled, the pub struct SimulatedTcpStream {
tx: mpsc::Sender<Bytes>,
rx: mpsc::Receiver<Bytes>,
staged: Option<Bytes>,
shutdown: bool,
local_addr: net::SocketAddr,
peer_addr: net::SocketAddr,
}
Disk IOLike networking, disk IO will be done using an in-memory mock filesystem. The This is not needed initially, and has not been explored yet with FaultsBeyond providing a deterministic environment for executing simulations, it's We can take advantage of this global state to also support an API for fault async fn swizzle_clog_fault() {
loop {
let connections: Vec<_> = tokio::simulation::get_connections();
for connection in connections {
connection.clog();
warn!("clogging connection {:?}", connection);
tokio::timer::delay_from(time::Duration::from_secs(10)).await;
}
warn!("waiting 30s before unclogging connections {:?}", connection);
tokio::timer::delay_from(time::Duration::from_secs(30)).await;
for connection in connections {
connection.unclog();
warn!("unclogging connection {:?}", connection);
tokio::timer::delay_from(time::Duration::from_secs(10)).await;
}
}
}Some of the faults that will be supported:
|
|
This is a fantastic write-up @gardnervickers! I am super excited to start seeing some of this stuff being used. I think overall the proposal is very very solid. The one area I am a bit fuzzy on is how we integrate with tokio directly. It may make sense to explore providing an enumed TcpStream that matches the tokio api 1-1 that lives within the simulation crate. This way we could move away from the giant |
|
I can't speak to the details of the tokio integration, but in general this seems like a great direction. It certainly would be very valuable for tokio to have good support for in-memory testing (or every developer or protocol implementor would need to separately implement their own). Will try to follow along! |
This patchset introduces a new scheduler which supports mock time. Also, `Registration` is moved behind the io handle. The IO driver and handle have been moved around to support easier mocking.
This changeset adds initial support for Simulation testing inside of Tokio. Simulated time and TCP* networking are supported.
Darksonn
added
A-tokio
|
Since simulation has been archived, I'll close this. |
Despite the relatively limited use that tokio-rs/simulation has seen so far, @gardnervickers has been able to reproduce several complex bugs in H2 and Tonic. To make
tokio-rs/simulationmore accessible to more people, I think it's worth opening a discussion on re-exporting simulation's networking types from Tokio. I believe (and I might be misquoting @gardnervickers!) that the desired end state for this feature would enable a library like Hyper to be deterministically simulated just by changing which feature flags are enabled within Tokio.To enable this, I think that Tokio will need to make the following changes:
tokio::runtime::Builder::clockto allow for setting of other hooks in Tokio as to enable deterministic simulation.FaultyTcpStreamthrough a rename and re-export under the name of the non-simulated type (In this case,tokio::net::TcpStream) if a feature flag enablingsimulationis toggled.A discussion between @gardnervickers, @carllerche, and I covering how this would be implemented. It's slightly messy, I apologize about that.
The text was updated successfully, but these errors were encountered: