CompleteIo

A thread-per-core Rust IO drivers and async runtime backed by IOCP/io_uring/kqueue. The name comes from "completion-based IO".

This repository is a fork of compio.

Why not Compio?

The project has different goals:

• provide low overhead IO drivers that preallocate memory on initialization

• drivers API accepts non-'static IO buffers

  • drivers don't own buffers
  • buffers are required to be Unpin

• bias towards io_uring API to achieve zero-cost abstraction on Linux:

  • fixed size submission queue for operations
  • external runtime could submit an external queue of not yet queued operations as a single batch
  • timers are exposed as Timeout operation and use suspend-aware CLOCK_BOOTTIME clock source when it's available
  • file descriptors could be registered to remove refcounting overhead in io_uring
  • Separate implementations of nonvectored and vectored Recv/Send and RecvFrom/SendTo - to exclude msghdr related overhead
  • Use send/recv syscalls/opcodes over read/write to remove the associated overhead
  • Add Read/Write operations for nonseekable files

• Async runtime is an example runtime to test implementation of drivers

Quick start

With runtime feature enabled, we can use the high level APIs to perform fs & net IO.

use completeio::{fs::File, task::block_on};

let buffer = block_on(async {
    let file = File::open("Cargo.toml").unwrap();
    let (read, buffer) = file.read_to_end_at(Vec::with_capacity(1024), 0).await;
    let read = read.unwrap();
    assert_eq!(read, buffer.len());
    String::from_utf8(buffer).unwrap()
});
println!("{}", buffer);

While you can also control the low-level driver manually:

use arrayvec::ArrayVec;
use std::collections::VecDeque;
use completeio::{
    buf::IntoInner,
    driver::{AsRawFd, Driver, Entry, CompleteIo},
    fs::File,
    op::ReadAt,
};

let mut driver = Driver::new().unwrap();
let file = File::open("Cargo.toml").unwrap();
// Attach the `RawFd` to driver first.
//
// The result is `Fd` attached to the driver.
// It's possible to register file descriptor instead of attaching.
let fd = driver.attach(file.as_raw_fd()).unwrap();

// Create IO operation and push it to the driver's submission queue.
let mut op = ReadAt::new(fd, 0, Vec::with_capacity(4096));
// We don't pass operation ownership to the driver and have to keep operations
// until they will be finished
let mut ops = VecDeque::from([(&mut op, 0).into()]);
driver.push_queue(&mut ops);

// Submit the queued operations and wait for IO completed.
let mut entries = ArrayVec::<Entry, 1>::new();
unsafe {
    driver
        .submit(None, &mut entries)
        .unwrap();
}
let entry = entries.drain(..).next().unwrap();
assert_eq!(entry.user_data(), 0);

// Resize the buffer by return value.
let n = entry.into_result().unwrap();
let mut buffer = op.into_inner();
unsafe {
    buffer.set_len(n);
}

println!("{}", String::from_utf8(buffer).unwrap());