Repository navigation
Artery: inbound lanes for parallel inbound message processing (ships at 1) - #8356
Conversation
… dispatch Fans ordinary-stream messages across N bounded per-lane channels within one accepted connection, keyed by recipient path hash so per-recipient ordering is preserved. Ships with inbound-lanes = 1, which keeps the existing fused single-lane pipeline and materializes no lane machinery.
|
Ran the C.1 lane-scaling sweep against this branch (plus the #8355 CLI flags) on the quiet 9900X box: RemotePingPong Echo (plateau median across 15-30 clients, warm sweeps):
Oneway:
Reading: echo is flat within run-to-run noise across 1/2/4/8 lanes — the whole band sits inside the ~1.9% between-run drift envelope, ordering is non-monotonic, and lanes=4 lands slightly below lanes=1. Oneway shows a clean, reproducible jump at lanes=2 and then saturates completely; 4 and 8 lanes add nothing over 2. That asymmetry is the diagnosis rather than a problem: echo actually runs higher than oneway (~1.82M vs ~1.25M) because echo drives two outbound encode islands while oneway drives one, so the binding constraint is the single outbound encode island (this build has no outbound lanes) plus toy messages being too cheap to deserialize for the inbound station to matter beyond the first doubling. The oneway gain proves the lane machinery genuinely parallelizes when the inbound side is the loaded one, which clears the partition seams of the pre-registered "flat curve means we added a serialized station" suspicion. Bottom line for this PR: shipping default stays 1 (lowest latency for round-trip traffic, matching Pekko's own caveat), and |
| // Ships at 1 (single-lane, fused pipeline -- byte-identical to pre-lanes processing) | ||
| // until the scaling gates pass -- see ArterySettings.InboundLanes's remarks. | ||
| settings.InboundLanes.Should().Be(1); | ||
| settings.InboundLaneBufferSize.Should().Be(4096); |
There was a problem hiding this comment.
Buffer size == msg count here, not bytes.
| int maxLargeFrameLength, | ||
| Akka.Serialization.Serialization serialization, | ||
| int inboundLanes = 1, | ||
| int inboundLaneBufferSize = 0, |
There was a problem hiding this comment.
0 == no buffer size limitation
| // match a real wire SerializerId, so the defensive checks below simply never trigger | ||
| // rather than throwing. | ||
| // | ||
| // LOAD-BEARING INVARIANT: lane mode classifies a frame as "control" by comparing its |
There was a problem hiding this comment.
for future reference
| Push(_stage.Out, _pending.Dequeue()); | ||
| } | ||
| else if (IsClosed(_stage.In)) | ||
| else if (IsClosed(_stage.In) && !_writeInFlight) |
There was a problem hiding this comment.
don't complete the stage if there are still pending writes
1 participant
Adds inbound lanes to Artery's ordinary stream. When
akka.remote.artery.advanced.inbound-lanesis raised above 1, each accepted connection fans user messages out across N bounded queues, each drained by its own consumer task, so payload deserialization, actor-ref resolution, and dispatch run in parallel. The lane is picked by a stable hash of the recipient path, so all traffic to the same actor stays on one lane and arrives in send order.ActorSelectionMessages hash on the selection's own target path elements, since their envelope recipient is always the remote root guardian. A newinbound-lane-buffer-sizesetting (default 4096) caps each lane's queue; a full lane backpressures the connection instead of dropping or growing unbounded.The default is 1, where none of the lane machinery is materialized and the inbound pipeline is unchanged from what's on
devtoday. Control-stream traffic (handshake, heartbeats, system messages) is unaffected at any setting. We'll consider flipping the default to Pekko's 4 after the lane-scaling runs on a quiet box.Verification at the lanes=1 default (RemotePingPong
--arteryecho, 9900X): warm plateau 1.80-1.88M msgs/s, peak 1.875M vs the 1.843M same-nightdevanchor, zero drops. Full Artery suite green on three consecutive runs, plus MNTR sanity (NewRemoteActorSpec,SunnyWeatherSpec) over the Artery transport.