Performance Improvements

[mempirate]

This discussion contains a list of all the performance improvements we're thinking about and their results (if applied)

• Using FuturesUnordered with manual Future implementation instead of JoinSet for pending requests in the reply driver:

  • msg-rs/msg-socket/src/rep/driver.rs

    Line 23 in 380ff65

    pending_requests: FuturesUnordered<PendingRequest>,

  • Granted, we also stopped allocating the future with Box::pin, which will have an impact as well
  • Resulted in ~10% speedup in the req_res_localhost benchmark
• Single-Writer Principle
  • We can look into using SPSC channel implementations instead of Tokio's MPSC in some situations. This will decrease contention / synchronization overhead.
• Different channel implementations
  • We make heavy use of Tokio's channels. Should test other channels like Kanal or Flume.
• Reducing syscall overhead
  • Framed and underlying AsyncRead + AsyncWrite IO objects that represent network streams (like TcpStream) are used a lot in msg-rs. They are a useful abstraction, but we should be careful in calling poll_flush or poll_write too often, as this will incur a lot of syscall overhead. There is a trade-off between throughput and latency here. For example, calling poll_flush after every start_send will have the lowest latency but incur a high througput penalty. On the other hand, waiting for the backpressure_boundary to be hit (usually 8KiB), will result in much higher throughput but also higher latency. To give the user some control, every socket should include a flush_interval option that determines the rate at which we should flush our buffers and make the syscall.
  • Resulted in ~200% speedup in the pubsub benchmark
• Reducing TCP latency for small messages
  • For sending small (<MTU) sized messages over TCP with low latency, make sure to disable Nagle's algorithm by setting set_nodelay(true) on your socket abstraction.