Repository navigation
Client/transport: one recv per small chunk, ManagedSession lock fast path, write coalescing - #4737
Conversation
ReceiveChunkAsync read the 8-byte chunk header and then the body, two receive calls per chunk. On loopback, receive syscalls are a large share of the client CPU per small request. The transport now waits for data with a small staging buffer (8 KB, allocated on the first receive, not from the BufferManager budget). One receive returns the header and, when it is small and already there, the whole chunk; bytes of following chunks are kept for the next call. A chunk larger than the staging buffer is completed with a receive into the rented buffer, as before. The receive buffer is still rented only after the header has been validated, so an idle peer holds no budget; message type and size checks are unchanged. A cancelled receive keeps the bytes that already arrived. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…achine Every service call was an async wrapper around the inner session call: await the reader lock, then await the call, one more async state machine box per request on top of the inner session's. When services are available and no writer holds or waits for the lock, the reader lock is now taken synchronously (AsyncReaderWriterLock TryEnterReader) and released by a cached continuation on the inner call's task, registered before the caller's await, so the lock is free again when the caller resumes (unless the call completes while the continuation is being registered; the release then runs on the pool). Otherwise the call waits for the lock as before, so reconnect and failover still block new calls and wait for the calls in flight. A synchronous exception from the inner call is still reported through the returned task. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The channel writer sent queued writes one by one, one socket send each. With several requests in flight on a channel (a shared client channel, or a server answering concurrent requests) writes queue up behind the one being sent. When more writes for the same transport are queued, the writer now takes up to 16 of them and sends their segments with one vectored send, if the transport carries a byte stream (TcpByteTransport, through the admission lease on the server, opts in with the internal IUaSCChunkCoalescing). Each write is then reported on its own, in order, with its size; a failed send is reported for every write in it, and their buffers are returned. Writes still never start on the caller's stack, and a single writer still sends one batch at a time, so chunk order is unchanged. Transports that frame each send (WebSocket, capture decorators) are not coalesced. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
TCP receive cancellation can lose stream bytes, and net48 cannot cancel the new staged receive as tested.
4 open findings
What changed in this PR
Optimizes OPC UA TCP transport, ManagedSession locking, and batched writes while extending performance tooling and regression tests.
Changes:
- Adds staged TCP receives and vectored write coalescing.
- Adds a synchronous ManagedSession reader-lock fast path.
- Extends transport tests and shared-channel benchmarks.
| File | Description |
|---|---|
tests/Opc.Ua.Core.Tests/Stack/Transport/UaSCBinaryChannelWriteQueueTests.cs |
Tests coalesced writes and failures. |
tests/Opc.Ua.Core.Tests/Stack/Transport/TcpByteTransportTests.cs |
Tests staged TCP receives. |
tests/Opc.Ua.Client.Tests/Utils/AsyncReaderWriterLockTests.cs |
Tests reader-lock fast path. |
src/Opc.Ua.Core/Stack/Tcp/UaScConnectionAdmission.cs |
Forwards coalescing capability. |
src/Opc.Ua.Core/Stack/Tcp/UaSCBinaryChannel.cs |
Coalesces queued writes. |
src/Opc.Ua.Core/Stack/Tcp/TcpByteTransport.cs |
Adds staged receiving and coalescing support. |
src/Opc.Ua.Core/Stack/Tcp/IUaSCByteTransport.cs |
Defines coalescing capability. |
src/Opc.Ua.Client/Utils/AsyncReaderWriterLock.cs |
Adds nonblocking reader acquisition. |
src/Opc.Ua.Client/Session/Reconnect/ConnectionStateMachine.cs |
Exposes service availability. |
src/Opc.Ua.Client/Session/ManagedSession.Services.cs |
Uses the lock fast path for services. |
perf/ServerLoadHarness/Program.cs |
Adds shared-channel sessions. |
perf/README.md |
Documents shared-client mode. |
perf/ab.sh |
Adds candidate-build selection. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Code coverage✅ Coverage gate passed.
Uncovered changed lines
Coverage is above the recorded baseline - consider ratcheting Thresholds live in |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #4737 +/- ##
==========================================
+ Coverage 82.84% 82.86% +0.01%
==========================================
Files 2268 2268
Lines 346153 346275 +122
Branches 60358 60381 +23
==========================================
+ Hits 286783 286949 +166
+ Misses 39271 39218 -53
- Partials 20099 20108 +9
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Review follow-ups: - A receive that fails while reading the body of a chunk larger than the staging buffer has consumed part of the chunk, which goes back with the rented buffer. The transport now remembers that and fails later receives with BadTcpInternalError instead of reading body bytes as the next chunk header. Cancellation while the bytes are in the staging buffer still resumes. - On .NET Framework the staging receive takes no token; the bytes it returned are now kept before the cancellation is observed. The partial-header cancellation tests run where ReceiveAsync is cancellable (.NET 5+). - Tests send their whole payload (SendAsync may send less), and the harness wraps the connect-parallel line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the receive guard Patch coverage follow-up: CreateSession/ActivateSession/CloseSession passthrough cases; a service call waiting while a writer holds the service lock; a call that fails before the inner call starts faulting the returned task and releasing the reader lock; an unexpected exception in a coalesced send reported as BadTcpInternalError for every write in it; a second concurrent ReceiveChunkAsync rejected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>



Follow-ups from the "Not changed" list of #4720 (client per-request allocation and CPU): three changes that the paired A/B shows a gain for, plus a write-up of the fourth (discarded
ResponseHeader), which is not changed.Results
Paired A/B, each change against the commit before it: 4 alternating rounds, medians, 20 sessions, 24-core machine shared with other sessions. Client = the harness process, server = the reference server child. Allocated bytes per operation are deterministic; throughput and latency carry the run-to-run noise noted below.
≈: within run-to-run noise (about ±5% for CPU per op here).
¹ Rounds ranged from −42% to +32%; the inflight-4 runs were better in all four rounds.
Changes
TcpByteTransportreceive:ReceiveChunkAsyncread the 8-byte chunk header and then the body, tworecvcalls per chunk. It now waits for data with a small staging buffer (8 KB, allocated on the first receive, not from theBufferManagerbudget). One receive returns the header and, when the chunk is small and already there, the whole chunk; bytes of following chunks are kept for the next call. A larger chunk is completed with a receive into the rented buffer, as before.BadTcpInternalErrorrather than read body bytes as a header. Before this PR such a failure lost the header and body bytes silently. The channel cancels receives only when it closes.ManagedSessionservice calls: each call was an async wrapper (await the reader lock, await the inner call), one more state machine box per request.AsyncReaderWriterLock.TryEnterReader). It is released by a cached continuation on the inner call's task, registered before the caller's await, so the lock is free again when the caller resumes.UaSCUaBinaryChannel): when more writes for the same transport are queued behind the one being sent, the writer takes up to 16 of them and sends their segments in one vectored send.TcpByteTransport, also through the server's admission lease, via the internalIUaSCChunkCoalescing). WebSocket and capture decorators keep one send per chunk.Behaviour notes
BufferManagerbudget. The previous transport held an 8-byte header array.ManagedSessionservice calls: if the inner call completes while the release continuation is being registered, the reader lock is released on the thread pool shortly after, instead of before the caller resumes.ReceiveChunkAsynccalls on oneTcpByteTransportnow throwInvalidOperationException. The channel has a single receive loop, and interleaved receives on a stream were never valid.Not changed: discarded
ResponseHeaderEvery generated structure's
Initialize()constructs its structure-typed fields:new ResponseHeader()(112 B),new RequestHeader()(104 B). The decoders always create the instance (new T()or the activator) and then assign every field inDecode, so the default is thrown away. That is 4.6% of the client's 2.4 KB per Read and about 3% on the server, which discards aRequestHeaderper decoded request and aResponseHeaderper response it constructs.PublishResponsealso discards a defaultNotificationMessage. Options considered:get => m_x ??= new X()):IsEqual/GetHashCodecompare fields, so a never-read default (null) would no longer equal an explicit default. Two threads reading an unset header could get different instances. Pooled instances reset throughInitialize()anyway.Class/DerivedClasstemplates assign every field inDecode, not optional-field or union types. A derived type's constructor must chain to a base built by an older generator in another assembly, which lacks it (the same problemForeignDerivedPooledExtensionClassworks around). It also needs a new public marker type inOpc.Ua.Types.IDecodermember in all decoders. It also changes behaviour when a caller decodes into an object whose header it still references.Each touches every generated type for about 100 B per message, so this is left as is.
Harness
--client shared: classic sessions whose secure channels come from oneClientChannelManager.--connect-parallel, default 1). ConcurrentSession.CreateAsynccalls on one manager fail withBadSecureChannelClosed, a separate bug.ab.sh:CANDIDATE=selects the candidate build.Testing
TcpByteTransportTests: chunks sent together, a chunk split across sends (inside the next header), a chunk larger than the staging buffer, cancellation keeping staged bytes.UaSCBinaryChannelWriteQueueTests: coalesced in order (1 + 16 + 16 + 7 sends), framing transport not coalesced, a failed batch reported for every write and later writes still sent.AsyncReaderWriterLockTests:TryEnterReaderagainst a writer, the reader held until the operation completes and released on failure and on a completed operation, the awaiter resuming after the release.ClientBaseTests.ActivityTraceFlags_Metrics_ShouldRecordMetricsOnly, a meter-listener flake fixed on master by Tests: scope MeterListeners to the meter under test (fix flaky ClientBaseTests metrics test) #4708; passes 3/3 in isolation); Client 2768; Sessions 851 (379 skipped); Server 8398; Subscriptions 643.UA.slnx: clean (0 warnings, 0 errors);dotnet format whitespaceclean.🤖 Generated with Claude Code