Skip to content

feat(networking): add Linux io_uring transport - #10846

Draft
ReubenBond wants to merge 78 commits into
dotnet:mainfrom
ReubenBond:rb-10074-message-oriented-transports
Draft

ReubenBond wants to merge 78 commits into
dotnet:mainfrom
ReubenBond:rb-10074-message-oriented-transports

Conversation

@ReubenBond

@ReubenBond ReubenBond commented Aug 27, 2026 •

Copy link
Copy Markdown
Member

Depends on #11379.

Includes the allocation fixes from #11416.

Problem

The message-oriented transport merged in #10074 provides the batching and buffer-lifetime boundaries needed for a native completion transport. Its socket implementation still routes operations through the general-purpose .NET async socket engine.

Solution

  • Add an opt-in Linux io_uring transport using ORLEANS_USE_IO_URING=1 or the Orleans.Connections.Transport.Sockets.UseIoUring AppContext switch.
  • Use bounded, sharded single-issuer rings, generation-tagged operation slots, and direct SQ/CQ access.
  • Keep small sends on their receive ring, attempt nonblocking sends for pinned contiguous buffers below 16 KiB, and create a separate large-send engine lazily.
  • Support incremental provided-buffer receives, adaptive multishot promotion, large non-loopback zero-copy sends, and optional NAPI polling.
  • Retain pending-operation ownership during cancellation, FIN, partial startup, and disposal. Socket shutdown interrupts I/O, but descriptor and buffer release wait until the operations retire.
  • Preserve TCP keepalive, endpoint options, and TLS behavior.

Integration

The branch is stacked on #11379 at e5724550f65343ef57faed89437dfed85e7ae8dc, which includes the measured main (b084e035b8ebcc7933f09515d09f2bf9fb7da776). The updated io_uring head, including the serializer allocation fixes, is 05c9d2d1bc88e92542784965d19d416b694b1eb9.

Measured comparison

The table below is the pre-allocation-fix comparison, pinned to main b084e035b, PR 11379 24f2ae508, and io_uring 6e7a169dd. It uses .NET SDK 10.0.401/runtime 10.0.12, Release, Linux 7.0, and an Intel i7-11850H. Client and silo ran in separate processes pinned to disjoint groups of four physical cores plus SMT siblings. Throughput uses 250 concurrent calls over plaintext TCP loopback; latency uses one outstanding call. Figures are medians of three fresh-process runs, with rotated variant order, 20-second warmup, and 30-second measurement.

Workload Main calls/s PR 11379 calls/s Stacked io_uring calls/s
Empty completion 352,606 321,988 372,051
1 KiB request 340,863 324,052 344,055
16 KiB request 86,404 85,077 158,985
64 KiB request 50,092 49,663 53,646
Integer result 326,413 277,558 288,500
1 KiB echo 303,866 258,816 263,871
64 KiB echo 26,496 28,149 29,968
Latency p50 Main PR 11379 Stacked io_uring
Empty completion 24.25 us 24.98 us 23.01 us
Integer result 24.07 us 31.24 us 28.35 us
Workload Main CPU us/call / RSS MiB PR 11379 CPU us/call / RSS MiB io_uring CPU us/call / RSS MiB
Empty completion 31.37 / 208.44 31.79 / 229.96 30.91 / 236.26
Integer result 31.79 / 204.67 44.67 / 240.89 45.91 / 245.56
16 KiB request 137.62 / 289.72 140.27 / 305.97 73.95 / 348.29

CPU includes client+silo user/system time. RSS is their combined mean resident memory, including shared pages twice and excluding kernel-only socket/ring memory. The shared laptop retained its powersave governor and desktop processes; small-payload results vary, so a modest median lead is not a guarantee. The 81 runs completed 411,153,176 measured calls without failures, and live ring descriptors verified io_uring activation in both processes.

In this pre-fix comparison, the stacked transport improves 16 KiB throughput by 86.9% over its PR 11379 base. Small typed results remain slower and consume more CPU than main: this comparison does not claim an across-the-board improvement. Completion-only calls additionally expose 232 managed allocation bytes/call on the PR base. The fixes in #11416 remove this cache-hit allocation and reuse cleared default-admission traversal state; their focused warmed-up regressions require zero allocation.

These figures are pinned to 24f2ae508/6e7a169dd. Later dependency updates are not silently substituted into the results. Earlier measurements against different source revisions have been superseded.

Post-fix follow-up

A separate three-run comparison uses main b084e035b, the refreshed PR 11379 base e5724550f, allocation-fix branch 6704d279c, and io_uring 05c9d2d1b, with the same harness, core affinity, warmup, and measurement durations.

Workload Main PR 11379 base Allocation fixes io_uring + fixes
Empty completion calls/s 318,036 307,200 347,661 372,388
Integer-result calls/s 343,220 267,966 279,207 287,073
Integer-result p50 24.34 us 32.55 us 32.13 us 29.46 us
Empty completion client allocation 592.07 B/call 824.07 B/call 592.07 B/call 592.07 B/call
Empty completion CPU / combined RSS 32.37 us/call / 207.94 MiB 32.97 / 227.92 31.40 / 228.02 30.61 / 234.20

io_uring's empty-call throughput is 17.1% above main in this batch: all three io_uring runs (338k-382k calls/s) exceed all three main runs (293k-321k). The allocation fix improves empty-call throughput by 13.2% over its dependency and removes the additional 232 bytes/call. Small typed results remain below main; the improvement is not across-the-board. All 36 follow-up runs completed 235,988,455 calls without failures.

Compatibility

The implementation requires little-endian 64-bit Linux, kernel 6.1 or later, and liburing.so.2; multishot incremental receives require Linux 6.12 or later. The transport remains opt-in.

This PR remains draft while its dependency, physical-NIC validation, and the native dependency strategy are finalized. Loopback avoids zero-copy send, so local loopback measurements do not validate physical-NIC zero-copy or NAPI behavior.

Copilot AI lite review requested due to automatic review settings August 27, 2026 00:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends Orleans’ new message-oriented networking stack with a TCP transport option to enable a Linux io_uring engine, updates the surrounding infrastructure (TLS, testing transports, listeners/connectors, and instrumentation), and tightens message lifetime management by disposing dropped/unhandled messages.

Changes:

  • Introduces new message-transport primitives/connectors/listeners (TCP sockets, Unix domain sockets, in-memory testing transport) and wires them into default silo/client/test hosting.
  • Refactors TLS configuration from connection-pipeline middleware to message-transport middleware and moves TLS types under Orleans.Connections.Transport.Security.
  • Improves message lifetime handling by disposing dropped/rejected/unhandled Message instances and updates related tests/benchmarks/docs.
Show a summary per file
File Description
test/TestInfrastructure/Orleans.TestingHost.Tests/InMemoryTransportTests.cs Adds coverage for in-memory message-transport behaviors (registration/cancellation/pipe completion).
test/Orleans.Serialization.UnitTests/PooledBufferTests.cs Updates enumerator test scope/docs after buffer enumeration changes.
test/Orleans.Runtime.Tests/TransportTests/UnixSocketTransportTests.cs Adjusts Unix-socket transport cluster settings (GatewayPerSilo).
test/Orleans.Core.Tests/Membership/ClusterHealthMonitorTests.cs Updates connection test scaffolding to new message-transport-based Connection construction.
test/Orleans.Connections.Security.Tests/TlsClientAuthenticationOptionsTests.cs Updates namespace to new TLS transport security location.
test/Orleans.Connections.Security.Tests/Orleans.Connections.Security.Tests.csproj Removes project reference to removed Orleans.Connections.Security project.
test/Grains/BenchmarkGrains/Ping/PayloadGrain.cs Adds benchmark grain for payload-based ping scenarios.
test/Grains/BenchmarkGrainInterfaces/Ping/IPayloadGrain.cs Adds payload grain interface for benchmarks.
test/Benchmarks/Serialization/ComplexTypeBenchmarks.cs Updates serializer construction after shared memory pool changes.
test/Benchmarks/Program.cs Adds stable/process ping benchmark entry points.
test/Benchmarks/Ping/PingBenchmark.cs Adds tunable run/warmup parameters for ping benchmarks.
test/Benchmarks/Ping/ConcurrentLoadGenerator.cs Adds configurable warmup block count to load generator.
src/Orleans.TestingHost/UnixSocketTransport/UnixSocketConnectionOptions.cs Removes legacy memory-pool factory and shared networking dependency.
src/Orleans.TestingHost/UnixSocketTransport/UnixSocketConnectionListenerFactory.cs Deletes legacy ASP.NET Core IConnectionListenerFactory implementation.
src/Orleans.TestingHost/UnixSocketTransport/UnixSocketConnectionListener.cs Deletes legacy IConnectionListener accept loop implementation.
src/Orleans.TestingHost/UnixSocketTransport/UnixSocketConnectionFactory.cs Deletes legacy ASP.NET Core IConnectionFactory implementation.
src/Orleans.TestingHost/UnixSocketTransport/UnixSocketConnectionExtensions.cs Rewires Unix socket transport registration to message-transport connector/listeners.
src/Orleans.TestingHost/UnixSocketTransport/UnixDomainSocketMessageTransportListener.cs Adds Unix domain socket message-transport listener implementation.
src/Orleans.TestingHost/UnixSocketTransport/UnixDomainSocketMessageTransportConnector.cs Adds Unix domain socket message-transport connector implementation.
src/Orleans.TestingHost/TestClusterOptions.cs Changes TestCluster default transport from InMemory to TCP sockets.
src/Orleans.TestingHost/TestClusterHostFactory.cs Adds logging dependency usage in TestingHost factory.
src/Orleans.TestingHost/Orleans.TestingHost.csproj Adds required package references for new transport implementations.
src/Orleans.TestingHost/InMemoryTransport/InMemoryTransportHostingExtensions.cs Adds DI helpers to register in-memory message transport for tests.
src/Orleans.TestingHost/InMemoryTransport/InMemoryTransportConnection.cs Deletes legacy TransportConnection-based in-memory transport.
src/Orleans.Serialization/Serializers/CodecProvider.cs Whitespace-only comment tidy-up.
src/Orleans.Serialization/Serializer.cs Whitespace-only formatting tidy-up.
src/Orleans.Serialization/Buffers/Reader.cs Whitespace-only XML doc formatting tidy-up.
src/Orleans.Runtime/Networking/SiloConnectionListener.cs Refactors silo listener to consume message-transport listeners and middleware.
src/Orleans.Runtime/Networking/GatewayConnectionListener.cs Refactors gateway listener to consume message-transport listeners and middleware.
src/Orleans.Runtime/Messaging/MessageCenter.cs Disposes messages on drop/reject paths to prevent leaks and factors rejection response sending.
src/Orleans.Runtime/Hosting/SiloTlsHostingExtensions.ISiloBuilder.cs Replaces prior TLS hosting extensions with transport-middleware-based TLS configuration.
src/Orleans.Runtime/Hosting/DefaultSiloServices.cs Registers default TCP message transports/listeners and supporting services/instruments.
src/Orleans.Runtime/Core/InsideRuntimeClient.cs Disposes responses with no callback to avoid leaks.
src/Orleans.Runtime/Configuration/SiloConnectionOptions.cs Removes legacy connection-pipeline configuration options.
src/Orleans.Runtime/Catalog/ActivationData.cs Disposes expired messages on dispatch drop path.
src/Orleans.Core/Utils/StripedMpscBuffer.cs Enables nullable for file (prep for new transport usage).
src/Orleans.Core/Utils/SingleWaiterAutoResetEvent.cs Adds a single-waiter auto-reset event for transport/runtime synchronization.
src/Orleans.Core/Runtime/OutsideRuntimeClient.cs Disposes unsupported/unhandled messages and responses with no callback.
src/Orleans.Core/Runtime/InvokableObjectManager.cs Disposes messages dropped due to GC/expiry.
src/Orleans.Core/Runtime/IMessageReceiver.cs Introduces internal receiver abstraction for message dispatch.
src/Orleans.Core/Orleans.Core.csproj Enables unsafe code and adds InternalsVisibleTo for new components.
src/Orleans.Core/Networking/Transport/WriteRequest.cs Adds write-request abstraction for message transports.
src/Orleans.Core/Networking/Transport/TlsMessageTransportConnectorMiddleware.cs Adds middleware wrappers to apply TLS to connectors/listeners.
src/Orleans.Core/Networking/Transport/Sockets/TcpMessageTransportConnector.cs Adds TCP connector (with UseLinuxIoUring option) for message transports.
src/Orleans.Core/Networking/Transport/Sockets/SocketsLog.cs Adds socket transport logging helpers.
src/Orleans.Core/Networking/Transport/Sockets/SocketSender.cs Adds socket send helper for multi-buffer send operations.
src/Orleans.Core/Networking/Transport/Sockets/SocketReceiver.cs Adds socket receive helper for multi-buffer receive operations.
src/Orleans.Core/Networking/Transport/Sockets/SocketOperationResult.cs Adds socket operation result struct.
src/Orleans.Core/Networking/Transport/Sockets/SocketExtensions.cs Adds socket fast-path helper (Windows loopback fast path).
src/Orleans.Core/Networking/Transport/Sockets/SocketConnectionException.cs Adds transport-specific connection exception type.
src/Orleans.Core/Networking/Transport/Sockets/SocketAwaitableEventArgs.cs Adds ValueTask-based socket awaitable event args.
src/Orleans.Core/Networking/Transport/Sockets/CorrelationIdGenerator.cs Adds correlation id generator under new socket transport namespace.
src/Orleans.Core/Networking/Transport/Sockets/BufferExtensions.cs Adds memory-to-array helpers for socket buffer lists.
src/Orleans.Core/Networking/Transport/Sockets/AddressInUseException.cs Adds transport-specific address-in-use exception.
src/Orleans.Core/Networking/Transport/Security/TlsServerAuthenticationOptions.cs Adds TLS server authentication options wrapper under new namespace.
src/Orleans.Core/Networking/Transport/Security/TlsMessageTransportListener.cs Adds TLS listener decorator for message transports.
src/Orleans.Core/Networking/Transport/Security/TlsMessageTransportConnector.cs Adds TLS connector decorator for message transports.
src/Orleans.Core/Networking/Transport/Security/TlsConnectionFeature.cs Adds TLS feature implementations for transports.
src/Orleans.Core/Networking/Transport/Security/TlsClientAuthenticationOptions.cs Adds TLS client authentication options wrapper under new namespace.
src/Orleans.Core/Networking/Transport/Security/ServerTlsMessageTransport.cs Adds server-authenticated TLS message transport wrapper.
src/Orleans.Core/Networking/Transport/Security/RemoteCertificateMode.cs Adds remote certificate mode enum under new namespace.
src/Orleans.Core/Networking/Transport/Security/ITlsHandshakeFeature.cs Adds TLS handshake feature interface under new namespace.
src/Orleans.Core/Networking/Transport/Security/ITlsConnectionFeature.cs Adds TLS connection feature interface under new namespace.
src/Orleans.Core/Networking/Transport/Security/ITlsApplicationProtocolFeature.cs Adds TLS application protocol feature interface under new namespace.
src/Orleans.Core/Networking/Transport/Security/ClientTlsMessageTransport.cs Adds client-authenticated TLS message transport wrapper.
src/Orleans.Core/Networking/Transport/Security/CertificateLoader.cs Adds certificate loading/validation helpers under new namespace.
src/Orleans.Core/Networking/Transport/ReadRequest.cs Adds read-request abstraction for message transports.
src/Orleans.Core/Networking/Transport/NetworkTransportBase.cs Adds base class for message transports with default feature collection.
src/Orleans.Core/Networking/Transport/MessageTransportListener.cs Adds listener abstraction for message transports.
src/Orleans.Core/Networking/Transport/MessageTransport.cs Adds core connector/listener/middleware abstractions for message transports.
src/Orleans.Core/Networking/Transport/IFeatureCollection.cs Adds typed feature-collection contract for transports.
src/Orleans.Core/Networking/Transport/FeatureCollection.cs Adds feature-collection implementation for transports.
src/Orleans.Core/Networking/Transport/ConnectionResetException.cs Adds connection reset exception type.
src/Orleans.Core/Networking/Transport/ConnectionEndPointFeature.cs Adds endpoint feature contract for transports.
src/Orleans.Core/Networking/Transport/ConnectionAbortedException.cs Adds connection aborted/closed exception types.
src/Orleans.Core/Networking/SocketDirection.cs Removes legacy connection direction enum file (moved/renamed).
src/Orleans.Core/Networking/Shared/TransportConnection.cs Removes legacy Kestrel/ConnectionContext-based transport connection base.
src/Orleans.Core/Networking/Shared/SocketsTrace.cs Removes legacy socket trace wrapper.
src/Orleans.Core/Networking/Shared/SocketSenderReceiverBase.cs Removes legacy socket sender/receiver base.
src/Orleans.Core/Networking/Shared/SocketSender.cs Removes legacy socket sender implementation.
src/Orleans.Core/Networking/Shared/SocketSchedulers.cs Removes legacy socket scheduler helper.
src/Orleans.Core/Networking/Shared/SocketReceiver.cs Removes legacy socket receiver implementation.
src/Orleans.Core/Networking/Shared/SocketExtensions.cs Removes legacy socket extensions.
src/Orleans.Core/Networking/Shared/SocketConnectionOptions.cs Removes legacy socket connection options.
src/Orleans.Core/Networking/Shared/SocketConnectionListenerFactory.cs Removes legacy listener factory.
src/Orleans.Core/Networking/Shared/SocketConnectionListener.cs Removes legacy listener.
src/Orleans.Core/Networking/Shared/SocketConnectionFactory.cs Removes legacy connection factory.
src/Orleans.Core/Networking/Shared/SocketAwaitableEventArgs.cs Removes legacy awaitable socket args.
src/Orleans.Core/Networking/Shared/SharedMemoryPool.cs Removes legacy shared memory pool.
src/Orleans.Core/Networking/Shared/MemoryPoolSlab.cs Removes legacy slab memory pool component.
src/Orleans.Core/Networking/Shared/MemoryPoolBlock.cs Removes legacy slab memory pool component.
src/Orleans.Core/Networking/Shared/KestrelMemoryPool.cs Removes legacy Kestrel memory pool wrapper.
src/Orleans.Core/Networking/Shared/ISocketsTrace.cs Removes legacy socket trace interface.
src/Orleans.Core/Networking/Shared/IOQueue.cs Removes legacy IO queue scheduler.
src/Orleans.Core/Networking/Shared/DuplexPipe.cs Removes legacy duplex pipe helper.
src/Orleans.Core/Networking/Shared/CorrelationIdGenerator.cs Removes legacy correlation id generator location.
src/Orleans.Core/Networking/Shared/BufferExtensions.cs Removes legacy buffer extensions.
src/Orleans.Core/Networking/MessageWriteRequest.cs Adds message write batching/framing request abstraction for transport layer.
src/Orleans.Core/Networking/MessageHandlerShared.cs Adds shared pools for serializers/read/write requests across connections.
src/Orleans.Core/Networking/IUnderlyingTransportFeature.cs Removes legacy underlying-transport feature tied to IDuplexPipe.
src/Orleans.Core/Networking/CorrelationIdGenerator.cs Adds correlation id generator in new networking stack location.
src/Orleans.Core/Networking/ConnectionTrace.cs Adds connection trace diagnostic/logger wrapper.
src/Orleans.Core/Networking/ConnectionShared.cs Refactors shared connection state and adds lazy MessageHandlerShared access.
src/Orleans.Core/Networking/ConnectionOptions.cs Adds CloseConnectionTimeout option.
src/Orleans.Core/Networking/ConnectionManager.cs Updates to renamed RunAsync entry point.
src/Orleans.Core/Networking/ConnectionLogScope.cs Updates connection log scope content/shape.
src/Orleans.Core/Networking/ConnectionFrameHelper.cs Updates framing helper implementation and documentation.
src/Orleans.Core/Networking/ConnectionFactory.cs Refactors connection factory to use message transport connector + middleware pipeline.
src/Orleans.Core/Networking/ConnectionDirection.cs Reintroduces/moves connection direction and adds transport protocol feature types.
src/Orleans.Core/Networking/ConnectionBuilderDelegates.cs Removes legacy builder delegate storage tied to ASP.NET Core connections.
src/Orleans.Core/Networking/ClientOutboundConnectionFactory.cs Refactors client outbound connection creation to message transport model.
src/Orleans.Core/Networking/ClientOutboundConnection.cs Refactors outbound connection implementation to message transport model and updates metrics/cleanup.
src/Orleans.Core/Networking/ClientConnectionOptions.cs Removes legacy client connection options tied to ASP.NET Core connections.
src/Orleans.Core/Messaging/MessageFactory.cs Preserves TTL correctly via helper rather than direct assignment.
src/Orleans.Core/Messaging/ClientMessageCenter.cs Disposes messages when client is not running / after rejection.
src/Orleans.Core/Messaging/CachingSiloAddressCodec.cs Nullable/cleanup changes around inner reader/writer creation and timestamp usage.
src/Orleans.Core/Hosting/ClientTlsHostingExtensions.IClientBuilder.cs Moves TLS client hosting extensions to new transport middleware model and adds config validation.
src/Orleans.Core.Abstractions/Runtime/IMessageReceiverCache.cs Adds public cache interface for message receiver resolution.
src/Orleans.Connections.Security/Security/TlsServerAuthenticationOptions.cs Deletes old TLS option wrapper (moved to Orleans.Core).
src/Orleans.Connections.Security/Security/TlsOptions.cs Deletes old TLS options type (moved to Orleans.Core).
src/Orleans.Connections.Security/Security/TlsDuplexPipe.cs Deletes old TLS duplex pipe adapter.
src/Orleans.Connections.Security/Security/TlsConnectionFeature.cs Deletes old TLS feature implementation (moved).
src/Orleans.Connections.Security/Security/TlsClientAuthenticationOptions.cs Deletes old TLS client options wrapper (moved).
src/Orleans.Connections.Security/Security/RemoteCertificateMode.cs Deletes old enum (moved).
src/Orleans.Connections.Security/Security/OrleansApplicationProtocol.cs Deletes old application protocol constant.
src/Orleans.Connections.Security/Security/MemoryPoolExtensions.cs Deletes old memory pool helpers.
src/Orleans.Connections.Security/Security/ITlsHandshakeFeature.cs Deletes old TLS handshake feature interface (moved).
src/Orleans.Connections.Security/Security/ITlsConnectionFeature.cs Deletes old TLS connection feature interface (moved).
src/Orleans.Connections.Security/Security/ITlsApplicationProtocolFeature.cs Deletes old TLS app protocol feature interface (moved).
src/Orleans.Connections.Security/Security/DuplexPipeStreamAdapter.cs Deletes old duplex pipe stream adapter.
src/Orleans.Connections.Security/Security/CertificateLoader.cs Deletes old certificate loader (moved).
src/Orleans.Connections.Security/Orleans.Connections.Security.csproj Removes legacy Microsoft.Orleans.Connections.Security project.
src/Orleans.Connections.Security/Hosting/HostingExtensions.cs Removes legacy connection-builder TLS middleware extensions.
src/api/Orleans.Runtime/Orleans.Runtime.cs Updates public API surface: removes SiloConnectionOptions, adds SiloTlsHostingExtensions.
src/api/Orleans.Core.Abstractions/Orleans.Core.Abstractions.cs Updates public API surface to include IMessageReceiverCache.
samples/TransportLayerSecurity/TLS.Server/TLS.Server.csproj Removes package dependency on removed TLS package.
samples/TransportLayerSecurity/TLS.Client/TLS.Client.csproj Removes package dependency on removed TLS package.
samples/TransportLayerSecurity/Directory.Packages.props Removes package version for removed TLS package.
Orleans.slnx Removes removed TLS project from solution.
docs/site/src/content/docs/resources/nuget-packages.md Removes package listing for removed TLS NuGet package.
docs/site/src/content/docs/host/snippets/transport-layer-security/csharp/SiloExample/SiloExample.csproj Updates snippet project to reference source projects instead of removed package.
docs/site/src/content/docs/host/snippets/transport-layer-security/csharp/SiloExample/Program.cs Updates TLS snippet namespaces for new TLS transport security location.
docs/site/src/content/docs/host/snippets/transport-layer-security/csharp/ClientExample/Program.cs Updates TLS snippet namespaces for new TLS transport security location.
docs/site/src/content/docs/host/snippets/transport-layer-security/csharp/ClientExample/ClientExample.csproj Updates snippet project to reference source projects instead of removed package.
docs/site/src/content/docs/host/connection-middleware.md Updates documentation from connection middleware to message transport middleware model.

Review details

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

  • Files reviewed: 178/185 changed files
  • Comments generated: 3
  • Review effort level: Lite

Comment thread src/Orleans.Core/Networking/Transport/Sockets/TcpMessageTransportConnector.cs Outdated
Comment thread src/Orleans.Core/Networking/ConnectionFrameHelper.cs Outdated
Copilot AI review requested due to automatic review settings August 27, 2026 02:50
@ReubenBond
ReubenBond force-pushed the rb-10074-message-oriented-transports branch from ff76b0d to cb0869a Compare August 27, 2026 02:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 179/188 changed files
  • Comments generated: 1
  • Review effort level: Lite

@github-actions

github-actions Bot commented Aug 27, 2026 •

Copy link
Copy Markdown
Contributor

Code coverage

Metric Pull request
Lines 82.20% (119,491 / 145,367)
Branches 72.03% (35,838 / 49,753)

Report-only conclusion: current-main baseline stale.

The newest successful coverage run tested dfd30e9, not current main ec48d17.

Coverage combines every CI test matrix job, including providers, CodeGen, .NET 8/10, Linux, Windows, and macOS, using canonical physical source and branch identities.

The comparison remains report-only while normal line and branch variance is calibrated.

Coverage details

Copilot AI review requested due to automatic review settings August 28, 2026 01:09
@ReubenBond
ReubenBond force-pushed the rb-10074-message-oriented-transports branch from cb0869a to 60f0ff3 Compare August 28, 2026 01:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

Review tier: Lite
Findings: 3 Medium severity · 1 Low severity

New issues introduced by this change (2)
Severity Finding
Medium severity src/​Orleans.Core/​Networking/​Transport/​Sockets/​TcpMessageTransportListener.cs — IsValid only checks Enabled, but CreateListenSocket/BindAsync dereference…
Low severity src/​Orleans.Core/​Networking/​Transport/​MessageTransport.cs — The <returns> docs for EnqueueRead and EnqueueWrite appear to be swapped (they refer to…
Pre-existing issues (2)
Severity Finding
Medium severity src/​Orleans.TestingHost/​UnixSocketTransport/​UnixDomainSocketMessageTransportListener.cs — Unix domain socket endpoints are filesystem entries which can persist after the listener socket is… View comment
Medium severity src/​Orleans.TestingHost/​UnixSocketTransport/​UnixDomainSocketMessageTransportConnector.cs — The connect awaitable uses a TaskCompletionSource without RunContinuationsAsynchronously, so… View comment
Issues resolved since last review (2)
Severity Finding
Medium severity src/​Orleans.Core/​Networking/​ConnectionFrameHelper.cs — WriteFrameWithPrefixingWriter now always allocates an ArrayBufferWriter and copies the payload into… View resolved comment
Medium severity src/​Orleans.Core/​Networking/​Transport/​Sockets/​TcpMessageTransportConnector.cs — The connect awaitable uses a TaskCompletionSource without RunContinuationsAsynchronously, so… View resolved comment
Suppressed comments (1)

src/Orleans.Core/Networking/Transport/Sockets/TcpMessageTransportListener.cs:168

  • TcpMessageTransportListener allocates a CancellationTokenSource (_closingCts) but never disposes it. This can leak registrations/resources over time (especially in tests which create many listeners). UnixDomainSocketMessageTransportListener already disposes its CTS; this listener should too.

Comment thread src/Orleans.Core/Networking/Transport/Sockets/TcpMessageTransportListener.cs Outdated
Comment thread src/Orleans.Core/Networking/Transport/MessageTransport.cs
@ReubenBond
ReubenBond force-pushed the rb-10074-message-oriented-transports branch from 60f0ff3 to 2745a13 Compare August 28, 2026 01:22
Copilot AI review requested due to automatic review settings August 28, 2026 05:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

Review tier: Lite
Findings: 4 Medium severity · 1 Low severity

New issues introduced by this change (1)
Severity Finding
Medium severity src/​Orleans.Runtime/​Hosting/​SiloTlsHostingExtensions.ISiloBuilder.cs — configureOptions is applied to named TlsOptions via AddOptions(...).Configure(configureOptions)…
Pre-existing issues (4)
Severity Finding
Medium severity src/​Orleans.Core/​Networking/​Transport/​Sockets/​TcpMessageTransportListener.cs — IsValid only checks Enabled, but CreateListenSocket/BindAsync dereference… View comment
Medium severity src/​Orleans.TestingHost/​UnixSocketTransport/​UnixDomainSocketMessageTransportListener.cs — Unix domain socket endpoints are filesystem entries which can persist after the listener socket is… View comment
Medium severity src/​Orleans.TestingHost/​UnixSocketTransport/​UnixDomainSocketMessageTransportConnector.cs — The connect awaitable uses a TaskCompletionSource without RunContinuationsAsynchronously, so… View comment
Low severity src/​Orleans.Core/​Networking/​Transport/​MessageTransport.cs — The <returns> docs for EnqueueRead and EnqueueWrite appear to be swapped (they refer to… View comment
Suppressed comments (4)

src/Orleans.Core/Networking/Transport/MessageTransport.cs:30

  • XML doc return description for EnqueueRead says "write request" but this method enqueues a read request, which is confusing for API consumers.
    src/Orleans.Runtime/Hosting/SiloTlsHostingExtensions.ISiloBuilder.cs:118
  • configureOptions is applied to named TlsOptions via AddOptions(...).Configure(configureOptions) and to all options via services.Configure(configureOptions), so the same action will run twice for the named "gateway" options. If configureOptions isn't idempotent, this can lead to duplicated configuration.
    src/Orleans.TestingHost/UnixSocketTransport/UnixDomainSocketMessageTransportListener.cs:70
  • Unix domain socket endpoints leave a filesystem entry at the configured path. BindAsync currently doesn't attempt to remove an existing socket file, so a stale path from a previous run can cause AddressAlreadyInUse failures (especially in repeated local test runs). Consider deleting any existing file at the path before binding.
    src/Orleans.Core/Networking/Transport/MessageTransport.cs:37
  • XML doc return description for EnqueueWrite says "read request" but this method enqueues a write request, which is confusing for API consumers.

Copilot AI review requested due to automatic review settings August 28, 2026 10:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

Review tier: Lite
Findings: 2 High severity

New issues introduced by this change (2)
Severity Finding
High severity src/​Orleans.Runtime/​Hosting/​SiloTlsHostingExtensions.ISiloBuilder.cs — SiloTlsHostingExtensions references ClientTlsHostingExtensions.TlsOptionsValidator, but that…
High severity src/​Orleans.Core/​Networking/​Transport/​Security/​ClientTlsMessageTransport.cs — The local variable cert is inferred as non-nullable X509Certificate2, but it is then reassigned…
Issues resolved since last review (5)
Severity Finding
Medium severity src/​Orleans.Runtime/​Hosting/​SiloTlsHostingExtensions.ISiloBuilder.cs — configureOptions is applied to named TlsOptions via AddOptions(...).Configure(configureOptions)… View resolved comment
Low severity src/​Orleans.Core/​Networking/​Transport/​MessageTransport.cs — The <returns> docs for EnqueueRead and EnqueueWrite appear to be swapped (they refer to… View resolved comment
Medium severity src/​Orleans.Core/​Networking/​Transport/​Sockets/​TcpMessageTransportListener.cs — IsValid only checks Enabled, but CreateListenSocket/BindAsync dereference… View resolved comment
Medium severity src/​Orleans.TestingHost/​UnixSocketTransport/​UnixDomainSocketMessageTransportListener.cs — Unix domain socket endpoints are filesystem entries which can persist after the listener socket is… View resolved comment
Medium severity src/​Orleans.TestingHost/​UnixSocketTransport/​UnixDomainSocketMessageTransportConnector.cs — The connect awaitable uses a TaskCompletionSource without RunContinuationsAsynchronously, so… View resolved comment

Comment thread src/Orleans.Core/Networking/Transport/Security/ClientTlsMessageTransport.cs Outdated
Copilot AI review requested due to automatic review settings August 28, 2026 10:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

Review tier: Lite
Findings: None

Issues resolved since last review (2)
Severity Finding
High severity src/​Orleans.Core/​Networking/​Transport/​Security/​ClientTlsMessageTransport.cs — The local variable cert is inferred as non-nullable X509Certificate2, but it is then reassigned… View resolved comment
High severity src/​Orleans.Runtime/​Hosting/​SiloTlsHostingExtensions.ISiloBuilder.cs — SiloTlsHostingExtensions references ClientTlsHostingExtensions.TlsOptionsValidator, but that… View resolved comment
Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Orleans.Core/Networking/ClientOutboundConnection.cs:91

  • The exception filter when ((error = exception) is null) is always false (the caught exception is never null), so this catch block is effectively unreachable and error will never be set on failure. This prevents passing the actual failure exception to OnConnectionTerminated.
    src/Orleans.Core/Networking/Transport/Sockets/SocketsLog.cs:10
  • This file is under .../Transport/Sockets/ and is consumed by other socket transport types (for example TcpMessageTransportConnector.cs:11 uses namespace Orleans.Connections.Transport.Sockets;), but SocketsLog is declared in Orleans.Connections.Sockets. The mismatched namespace makes discovery and usage inconsistent with the rest of the socket transport code.

Copilot AI review requested due to automatic review settings September 2, 2026 19:04
@ReubenBond
ReubenBond force-pushed the rb-10074-message-oriented-transports branch from 5615925 to ac5d41e Compare September 2, 2026 19:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The new TLS options surface has a concrete XML-doc issue and an API design inconsistency around client certificate selector nullability which should be resolved before merging.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Lite
Findings: 1 Medium severity · 2 Low severity

New issues introduced by this change (3)
Severity Finding
Medium severity src/​Orleans.Core/​Networking/​Transport/​Security/​TlsOptions.cs — TlsOptions.LocalClientCertificateSelector is declared as returning a non-null X509Certificate2, but…
Low severity src/​Orleans.Core/​Networking/​Transport/​Security/​TlsOptions.cs — The XML doc comment has a duplicated <summary> tag and an orphaned description about an "enabled"…
Low severity test/​Orleans.Serialization.UnitTests/​PooledBufferTests.cs — This test name still uses "SliceEnumerators" even though it now only validates the SpanEnumerator.…

Comment thread src/Orleans.Core/Networking/Transport/Security/TlsOptions.cs Outdated
Comment thread src/Orleans.Core/Networking/Transport/Security/TlsOptions.cs Outdated
Comment thread test/Orleans.Serialization.UnitTests/PooledBufferTests.cs Outdated
Copilot AI review requested due to automatic review settings September 2, 2026 21:57
@ReubenBond
ReubenBond force-pushed the rb-10074-message-oriented-transports branch from ac5d41e to f64ce56 Compare September 2, 2026 21:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

It introduces a new low-level transport (including io_uring) and performs a broad networking/TLS refactor touching core runtime behavior and resource lifetimes.

Review tier: Lite
Findings: 1 Low severity

New issues introduced by this change (1)
Severity Finding
Low severity src/​Orleans.Core/​Networking/​Transport/​MessageTransportListener.cs — The XML docs for BindAsync indicate it returns "The bound endpoint configuration", but the method…
Issues resolved since last review (3)
Severity Finding
Low severity test/​Orleans.Serialization.UnitTests/​PooledBufferTests.cs — This test name still uses "SliceEnumerators" even though it now only validates the SpanEnumerator.… View resolved comment
Low severity src/​Orleans.Core/​Networking/​Transport/​Security/​TlsOptions.cs — The XML doc comment has a duplicated <summary> tag and an orphaned description about an "enabled"… View resolved comment
Medium severity src/​Orleans.Core/​Networking/​Transport/​Security/​TlsOptions.cs — TlsOptions.LocalClientCertificateSelector is declared as returning a non-null X509Certificate2, but… View resolved comment

Comment thread src/Orleans.Core/Networking/Transport/MessageTransportListener.cs
ReubenBond and others added 26 commits October 6, 2026 17:41
Reuse the CQ head already loaded while locating each completion and use liburing's cached ring mask, preserving immediate slot release while removing redundant shared-memory reads.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Check the completion queue after submission and avoid submit_and_wait when inline completions are already available. This reduced io_uring_enter calls by roughly 37% under traced empty-message load and improved throughput across all measured payloads.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Route sub-4 KiB sends through the receive engine so inline completions can enqueue follow-up I/O without a cross-engine eventfd wake. Keep larger sends on a second engine to preserve parallel CQ draining.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Allow ArcBufferWriter to link externally filled pinned pages and extend incrementally received tails while preserving page reference counting and slice lifetimes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Add an opt-in adaptive receiver using multishot recv, incremental provided-buffer rings, and copy-free ArcBuffer page adoption. Start with one-shot receives, promote after sustained large frames, and demote only at a quiescent request boundary.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Use IORING_SEND_VECTORIZED for scatter batches of at least 32 KiB on Linux 6.17 and later, avoiding the live msghdr while retaining SENDMSG for smaller batches.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Request zero-copy usage notifications and disable zero-copy per sender after four consecutive copied or fallback sends, avoiding repeated notification overhead on unsupported paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Allow physical-NIC deployments to opt into per-ring NAPI busy polling using ORLEANS_IO_URING_NAPI_BUSY_POLL_US while leaving the default interrupt-driven path unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Vectorized sends improved isolated 64 KiB throughput but regressed 16 KiB batched traffic and could strand partial sends during benchmark shutdown. Retain the robust SENDMSG paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Store provided-buffer metadata contiguously and avoid locked updates for counters with a single writer or existing lock protection.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Preserve framed-length reporting and correct the engine scope after replaying the io_uring stack over the latest message transport refactor.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Pass the current xUnit cancellation token through the rebased io_uring lifecycle tests to satisfy main's analyzer rules and keep timeouts responsive.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Attempt nonblocking sends directly for pinned contiguous buffers below 16 KiB, falling back to io_uring when the socket would block.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Keep small sends on one engine, create the large-send engine lazily, and promote sustained sub-1 KiB receives to copy-free multishot on supported kernels.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Reserve the reusable io_uring sender state around optimistic native sends and verify disposed senders reject further use.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Keep sender lifecycle ownership through io_uring fallback and capture native send errors before the reusable operation becomes idle.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Integrate the io-only series with upstream 1cc211b. Keep upstream keepalive configuration, CreateAndStartAsync startup cleanup, and transport disposal contracts while retaining the complete io_uring engine and optimizations.

Read FIN interrupts writes using socket shutdown without disposing io_uring descriptors before send completions and zero-copy notifications retire. Add regression coverage for descriptor ownership, all receive modes, and connector/listener startup selection. Capture Closed tokens before disposal and document Linux transport prerequisites and switches.

Rebase omissions: 0539394 is superseded by upstream nullable certificate selection and unconditional ValidateCertificate handling. 2b06962 removes a temporary Unix listener implementation which is already absent upstream; preserve the current upstream listener and lifecycle tests. The obsolete Unix dependency-only portions of e383242 were likewise omitted, undoing the transient stale-file cleanup from 2130baa without changing the final upstream Unix implementation. No io_uring optimizations were dropped.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Read an initialized published serializer service before constructing another graph scope. Keep admission and fault checks, and isolate the allocating factory closure on the cold path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Borrow a cleared, bounded per-thread visited set without caching admission decisions. Preserve provider-specific overrides and graph boundaries while removing 176 bytes from each warmed default-service lookup.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings October 7, 2026 01:02
@ReubenBond
ReubenBond force-pushed the rb-10074-message-oriented-transports branch from 6e7a169 to 05c9d2d Compare October 7, 2026 01:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Critical framing and SIGPIPE issues remain unresolved, and reference metadata dependency scanning is incomplete.

Review effort: Lite
Findings: 5 High severity · 1 Medium severity

Open (6)

Comment on lines +101 to 104
if (!_hasFrameLengths)
{
Span<byte> scratch = stackalloc byte[Message.LENGTH_HEADER_SIZE];
var lengthBytes = bufferReader.Peek(in scratch);
Comment on lines +1343 to +1348
internal override void PrepareSubmission(LinuxIoUringEngine.IoUringSubmission* submission)
{
if (WaitForNotification)
{
submission->IoPriority |= SendZeroCopyReportUsage;
}
Comment on lines +248 to +251
internal static ISerializableTypeDescription? CreateSerializableTypeDescription(IGeneratorServices services, INamedTypeSymbol symbol, bool inspectReferenceMetadata = false)
=> CreateSerializableTypeDescription(services.Compilation, services.LibraryTypes, services.Options, symbol, inspectReferenceMetadata);

internal static ISerializableTypeDescription? CreateSerializableTypeDescription(Compilation compilation, LibraryTypes libraryTypes, CodeGeneratorOptions options, INamedTypeSymbol symbol)
internal static ISerializableTypeDescription? CreateSerializableTypeDescription(Compilation compilation, LibraryTypes libraryTypes, CodeGeneratorOptions options, INamedTypeSymbol symbol, bool inspectReferenceMetadata = false)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants