Improve MultiplexingStream v2/v3 throughput ~2.3x - #1119
Merged
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR targets the throughput gap reported in #505 by reducing per-frame overhead and improving buffering/flow-control behavior in MultiplexingStream v2/v3, and adds a benchmark to measure bulk-transfer throughput across protocol versions.
Changes:
- Increase the default per-channel receiving window size to reduce flow-control stalls on fast transports.
- Reduce frame fragmentation and ACK allocation overhead (pre-reserve frame space; avoid
Sequence<byte>in hot ACK path; improveSequence<T>segment sizing). - Add a BenchmarkDotNet benchmark that measures single-channel bulk-transfer throughput over loopback TCP across protocol v1/v2/v3.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| test/Nerdbank.Streams.Benchmark/Program.cs | Registers the new multiplexing throughput benchmark with the benchmark switcher. |
| test/Nerdbank.Streams.Benchmark/MultiplexingStreamBenchmark.cs | Adds a loopback-TCP bulk transfer benchmark for MultiplexingStream protocol v1/v2/v3. |
| src/Nerdbank.Streams/Sequence`1.cs | Ensures MinimumSpanLength is honored even when callers provide no size hint, improving stream read efficiency. |
| src/Nerdbank.Streams/MultiplexingStream.Options.cs | Updates default window size and expands remarks to explain throughput/memory implications. |
| src/Nerdbank.Streams/MultiplexingStream.Formatters.cs | Adds small fixed buffer writer for hot ACK serialization and pre-reserves contiguous space to avoid frame fragmentation. |
| src/Nerdbank.Streams/MultiplexingStream.cs | Introduces tuned transport PipeOptions and applies them to CreateAsync transport writer creation. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Protocol v2/v3 transferred bulk data far slower than v1 (issue #505). Measured over loopback TCP (512 MB, one channel): v1 ~1670 MB/s, v2 ~208 MB/s, v3 ~202 MB/s. After these changes v2 ~476 MB/s and v3 ~465 MB/s. Four independent bottlenecks, in order of measured impact: 1. Flow control window stalls. Profiling showed the sender blocked ~64% of elapsed time waiting for window capacity: ~8,400 stalls of ~170us each, where the 170us is the software round trip for a ContentProcessed frame (about four thread pool wakeups), not network latency. The 100KB default window allowed only five 20KB frames in flight. Raised the default to 50 frames (~1MB). This is a limit rather than an allocation, so channels that carry little data still cost little memory. 2. Frame fragmentation in the msgpack write path. MessagePackWriter's constructor calls PipeWriter.GetMemory(0), writes the frame prologue into whatever small buffer it gets, then Ensure(payload) forces a Commit and a fresh segment. The prologue and payload thus landed in separate pipe segments, so UsePipeWriter issued a separate stream write for each. Reserve the whole frame up front instead, as V1Formatter already did. 3. SerializeContentProcessed allocated a whole ArrayPool<byte> per call. Sequence<byte>'s default constructor calls ArrayPool<byte>.Create(), and this method runs once per frame of content received. Use a small fixed-size buffer writer instead. 4. The transport pipe used PipeOptions.Default, whose 64KB pause threshold is barely three frames, making the frame producer ping-pong with the socket copy loop. Use options scaled to the frame size. Also, Sequence<T>.GetSegment ignored MinimumSpanLength when given no size hint, so MessagePackStreamReader always read from the transport in 4KB chunks despite asking for larger buffers. Honor MinimumSpanLength for the array pool path, without ever shrinking below the pool's own default. Adds a BenchmarkDotNet benchmark covering bulk transfer across all three protocol versions. Closes #505 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
AArnott
force-pushed
the
perf/mxstream-throughput
branch
from
July 26, 2026 19:38
c83530d to
1a6a506
Compare
This was referenced Sep 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #505.
Investigated #505 empirically (built a loopback-TCP throughput harness, reproduced the ~7x v1-vs-v2/v3 gap, then profiled with
dotnet-traceand custom instrumentation rather than guessing). Found and fixed four independent bottlenecks:MessagePackWriter's ctor doesGetMemory(0), so prologue and payload landed in separate pipe segments ⇒ one socket write each. V1 never had this.SerializeContentProcessedallocated an entireArrayPool<byte>per ACK (new Sequence<byte>()→ArrayPool.Create())FixedSizeBufferWriterPipeOptions.Default(64 KB pause ≈ 3 frames)Plus:
Sequence<T>.GetSegmentignoredMinimumSpanLengthwith no size hint, forcing 4 KB socket reads.Results (512 MB, 1 channel, loopback TCP): v2 208 → 476 MB/s, v3 202 → 465 MB/s (~2.3x). v1 unchanged. Wire-compatible — identical msgpack bytes, only buffer management changed.
Notably rejected: I built a flush-coalescing optimization, but it caused an intermittent stranded-bytes hang in the full suite and only bought 2–4%, so I removed it rather than ship the hazard. Also reverted: inlining awaiters (-17%) and finer ACK granularity (-9%).
Validated: 0 warnings, and 3 consecutive clean full-suite runs (879 tests) before the window change plus 2 after.
Note on the window size default
Item 1 is the only behavioral change visible to consumers:
Options.DefaultChannelReceivingWindowSizegoes from5 * FramePayloadMaxLength(100 KB) to50 * FramePayloadMaxLength(~1 MB). It is a limit, not an allocation — memory is consumed on demand — so channels that carry little data still cost little memory. But it does raise the ceiling for consumers that open many channels, so it's worth a deliberate look.A benchmark covering bulk transfer across all three protocol versions is added to
Nerdbank.Streams.Benchmark.