Repository navigation
feat(networking): add Linux io_uring transport - #10846
ReubenBond wants to merge 78 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR extends Orleans’ new message-oriented networking stack with a TCP transport option to enable a Linux io_uring engine, updates the surrounding infrastructure (TLS, testing transports, listeners/connectors, and instrumentation), and tightens message lifetime management by disposing dropped/unhandled messages.
Changes:
- Introduces new message-transport primitives/connectors/listeners (TCP sockets, Unix domain sockets, in-memory testing transport) and wires them into default silo/client/test hosting.
- Refactors TLS configuration from connection-pipeline middleware to message-transport middleware and moves TLS types under
Orleans.Connections.Transport.Security. - Improves message lifetime handling by disposing dropped/rejected/unhandled
Messageinstances and updates related tests/benchmarks/docs.
Show a summary per file
| File | Description |
|---|---|
| test/TestInfrastructure/Orleans.TestingHost.Tests/InMemoryTransportTests.cs | Adds coverage for in-memory message-transport behaviors (registration/cancellation/pipe completion). |
| test/Orleans.Serialization.UnitTests/PooledBufferTests.cs | Updates enumerator test scope/docs after buffer enumeration changes. |
| test/Orleans.Runtime.Tests/TransportTests/UnixSocketTransportTests.cs | Adjusts Unix-socket transport cluster settings (GatewayPerSilo). |
| test/Orleans.Core.Tests/Membership/ClusterHealthMonitorTests.cs | Updates connection test scaffolding to new message-transport-based Connection construction. |
| test/Orleans.Connections.Security.Tests/TlsClientAuthenticationOptionsTests.cs | Updates namespace to new TLS transport security location. |
| test/Orleans.Connections.Security.Tests/Orleans.Connections.Security.Tests.csproj | Removes project reference to removed Orleans.Connections.Security project. |
| test/Grains/BenchmarkGrains/Ping/PayloadGrain.cs | Adds benchmark grain for payload-based ping scenarios. |
| test/Grains/BenchmarkGrainInterfaces/Ping/IPayloadGrain.cs | Adds payload grain interface for benchmarks. |
| test/Benchmarks/Serialization/ComplexTypeBenchmarks.cs | Updates serializer construction after shared memory pool changes. |
| test/Benchmarks/Program.cs | Adds stable/process ping benchmark entry points. |
| test/Benchmarks/Ping/PingBenchmark.cs | Adds tunable run/warmup parameters for ping benchmarks. |
| test/Benchmarks/Ping/ConcurrentLoadGenerator.cs | Adds configurable warmup block count to load generator. |
| src/Orleans.TestingHost/UnixSocketTransport/UnixSocketConnectionOptions.cs | Removes legacy memory-pool factory and shared networking dependency. |
| src/Orleans.TestingHost/UnixSocketTransport/UnixSocketConnectionListenerFactory.cs | Deletes legacy ASP.NET Core IConnectionListenerFactory implementation. |
| src/Orleans.TestingHost/UnixSocketTransport/UnixSocketConnectionListener.cs | Deletes legacy IConnectionListener accept loop implementation. |
| src/Orleans.TestingHost/UnixSocketTransport/UnixSocketConnectionFactory.cs | Deletes legacy ASP.NET Core IConnectionFactory implementation. |
| src/Orleans.TestingHost/UnixSocketTransport/UnixSocketConnectionExtensions.cs | Rewires Unix socket transport registration to message-transport connector/listeners. |
| src/Orleans.TestingHost/UnixSocketTransport/UnixDomainSocketMessageTransportListener.cs | Adds Unix domain socket message-transport listener implementation. |
| src/Orleans.TestingHost/UnixSocketTransport/UnixDomainSocketMessageTransportConnector.cs | Adds Unix domain socket message-transport connector implementation. |
| src/Orleans.TestingHost/TestClusterOptions.cs | Changes TestCluster default transport from InMemory to TCP sockets. |
| src/Orleans.TestingHost/TestClusterHostFactory.cs | Adds logging dependency usage in TestingHost factory. |
| src/Orleans.TestingHost/Orleans.TestingHost.csproj | Adds required package references for new transport implementations. |
| src/Orleans.TestingHost/InMemoryTransport/InMemoryTransportHostingExtensions.cs | Adds DI helpers to register in-memory message transport for tests. |
| src/Orleans.TestingHost/InMemoryTransport/InMemoryTransportConnection.cs | Deletes legacy TransportConnection-based in-memory transport. |
| src/Orleans.Serialization/Serializers/CodecProvider.cs | Whitespace-only comment tidy-up. |
| src/Orleans.Serialization/Serializer.cs | Whitespace-only formatting tidy-up. |
| src/Orleans.Serialization/Buffers/Reader.cs | Whitespace-only XML doc formatting tidy-up. |
| src/Orleans.Runtime/Networking/SiloConnectionListener.cs | Refactors silo listener to consume message-transport listeners and middleware. |
| src/Orleans.Runtime/Networking/GatewayConnectionListener.cs | Refactors gateway listener to consume message-transport listeners and middleware. |
| src/Orleans.Runtime/Messaging/MessageCenter.cs | Disposes messages on drop/reject paths to prevent leaks and factors rejection response sending. |
| src/Orleans.Runtime/Hosting/SiloTlsHostingExtensions.ISiloBuilder.cs | Replaces prior TLS hosting extensions with transport-middleware-based TLS configuration. |
| src/Orleans.Runtime/Hosting/DefaultSiloServices.cs | Registers default TCP message transports/listeners and supporting services/instruments. |
| src/Orleans.Runtime/Core/InsideRuntimeClient.cs | Disposes responses with no callback to avoid leaks. |
| src/Orleans.Runtime/Configuration/SiloConnectionOptions.cs | Removes legacy connection-pipeline configuration options. |
| src/Orleans.Runtime/Catalog/ActivationData.cs | Disposes expired messages on dispatch drop path. |
| src/Orleans.Core/Utils/StripedMpscBuffer.cs | Enables nullable for file (prep for new transport usage). |
| src/Orleans.Core/Utils/SingleWaiterAutoResetEvent.cs | Adds a single-waiter auto-reset event for transport/runtime synchronization. |
| src/Orleans.Core/Runtime/OutsideRuntimeClient.cs | Disposes unsupported/unhandled messages and responses with no callback. |
| src/Orleans.Core/Runtime/InvokableObjectManager.cs | Disposes messages dropped due to GC/expiry. |
| src/Orleans.Core/Runtime/IMessageReceiver.cs | Introduces internal receiver abstraction for message dispatch. |
| src/Orleans.Core/Orleans.Core.csproj | Enables unsafe code and adds InternalsVisibleTo for new components. |
| src/Orleans.Core/Networking/Transport/WriteRequest.cs | Adds write-request abstraction for message transports. |
| src/Orleans.Core/Networking/Transport/TlsMessageTransportConnectorMiddleware.cs | Adds middleware wrappers to apply TLS to connectors/listeners. |
| src/Orleans.Core/Networking/Transport/Sockets/TcpMessageTransportConnector.cs | Adds TCP connector (with UseLinuxIoUring option) for message transports. |
| src/Orleans.Core/Networking/Transport/Sockets/SocketsLog.cs | Adds socket transport logging helpers. |
| src/Orleans.Core/Networking/Transport/Sockets/SocketSender.cs | Adds socket send helper for multi-buffer send operations. |
| src/Orleans.Core/Networking/Transport/Sockets/SocketReceiver.cs | Adds socket receive helper for multi-buffer receive operations. |
| src/Orleans.Core/Networking/Transport/Sockets/SocketOperationResult.cs | Adds socket operation result struct. |
| src/Orleans.Core/Networking/Transport/Sockets/SocketExtensions.cs | Adds socket fast-path helper (Windows loopback fast path). |
| src/Orleans.Core/Networking/Transport/Sockets/SocketConnectionException.cs | Adds transport-specific connection exception type. |
| src/Orleans.Core/Networking/Transport/Sockets/SocketAwaitableEventArgs.cs | Adds ValueTask-based socket awaitable event args. |
| src/Orleans.Core/Networking/Transport/Sockets/CorrelationIdGenerator.cs | Adds correlation id generator under new socket transport namespace. |
| src/Orleans.Core/Networking/Transport/Sockets/BufferExtensions.cs | Adds memory-to-array helpers for socket buffer lists. |
| src/Orleans.Core/Networking/Transport/Sockets/AddressInUseException.cs | Adds transport-specific address-in-use exception. |
| src/Orleans.Core/Networking/Transport/Security/TlsServerAuthenticationOptions.cs | Adds TLS server authentication options wrapper under new namespace. |
| src/Orleans.Core/Networking/Transport/Security/TlsMessageTransportListener.cs | Adds TLS listener decorator for message transports. |
| src/Orleans.Core/Networking/Transport/Security/TlsMessageTransportConnector.cs | Adds TLS connector decorator for message transports. |
| src/Orleans.Core/Networking/Transport/Security/TlsConnectionFeature.cs | Adds TLS feature implementations for transports. |
| src/Orleans.Core/Networking/Transport/Security/TlsClientAuthenticationOptions.cs | Adds TLS client authentication options wrapper under new namespace. |
| src/Orleans.Core/Networking/Transport/Security/ServerTlsMessageTransport.cs | Adds server-authenticated TLS message transport wrapper. |
| src/Orleans.Core/Networking/Transport/Security/RemoteCertificateMode.cs | Adds remote certificate mode enum under new namespace. |
| src/Orleans.Core/Networking/Transport/Security/ITlsHandshakeFeature.cs | Adds TLS handshake feature interface under new namespace. |
| src/Orleans.Core/Networking/Transport/Security/ITlsConnectionFeature.cs | Adds TLS connection feature interface under new namespace. |
| src/Orleans.Core/Networking/Transport/Security/ITlsApplicationProtocolFeature.cs | Adds TLS application protocol feature interface under new namespace. |
| src/Orleans.Core/Networking/Transport/Security/ClientTlsMessageTransport.cs | Adds client-authenticated TLS message transport wrapper. |
| src/Orleans.Core/Networking/Transport/Security/CertificateLoader.cs | Adds certificate loading/validation helpers under new namespace. |
| src/Orleans.Core/Networking/Transport/ReadRequest.cs | Adds read-request abstraction for message transports. |
| src/Orleans.Core/Networking/Transport/NetworkTransportBase.cs | Adds base class for message transports with default feature collection. |
| src/Orleans.Core/Networking/Transport/MessageTransportListener.cs | Adds listener abstraction for message transports. |
| src/Orleans.Core/Networking/Transport/MessageTransport.cs | Adds core connector/listener/middleware abstractions for message transports. |
| src/Orleans.Core/Networking/Transport/IFeatureCollection.cs | Adds typed feature-collection contract for transports. |
| src/Orleans.Core/Networking/Transport/FeatureCollection.cs | Adds feature-collection implementation for transports. |
| src/Orleans.Core/Networking/Transport/ConnectionResetException.cs | Adds connection reset exception type. |
| src/Orleans.Core/Networking/Transport/ConnectionEndPointFeature.cs | Adds endpoint feature contract for transports. |
| src/Orleans.Core/Networking/Transport/ConnectionAbortedException.cs | Adds connection aborted/closed exception types. |
| src/Orleans.Core/Networking/SocketDirection.cs | Removes legacy connection direction enum file (moved/renamed). |
| src/Orleans.Core/Networking/Shared/TransportConnection.cs | Removes legacy Kestrel/ConnectionContext-based transport connection base. |
| src/Orleans.Core/Networking/Shared/SocketsTrace.cs | Removes legacy socket trace wrapper. |
| src/Orleans.Core/Networking/Shared/SocketSenderReceiverBase.cs | Removes legacy socket sender/receiver base. |
| src/Orleans.Core/Networking/Shared/SocketSender.cs | Removes legacy socket sender implementation. |
| src/Orleans.Core/Networking/Shared/SocketSchedulers.cs | Removes legacy socket scheduler helper. |
| src/Orleans.Core/Networking/Shared/SocketReceiver.cs | Removes legacy socket receiver implementation. |
| src/Orleans.Core/Networking/Shared/SocketExtensions.cs | Removes legacy socket extensions. |
| src/Orleans.Core/Networking/Shared/SocketConnectionOptions.cs | Removes legacy socket connection options. |
| src/Orleans.Core/Networking/Shared/SocketConnectionListenerFactory.cs | Removes legacy listener factory. |
| src/Orleans.Core/Networking/Shared/SocketConnectionListener.cs | Removes legacy listener. |
| src/Orleans.Core/Networking/Shared/SocketConnectionFactory.cs | Removes legacy connection factory. |
| src/Orleans.Core/Networking/Shared/SocketAwaitableEventArgs.cs | Removes legacy awaitable socket args. |
| src/Orleans.Core/Networking/Shared/SharedMemoryPool.cs | Removes legacy shared memory pool. |
| src/Orleans.Core/Networking/Shared/MemoryPoolSlab.cs | Removes legacy slab memory pool component. |
| src/Orleans.Core/Networking/Shared/MemoryPoolBlock.cs | Removes legacy slab memory pool component. |
| src/Orleans.Core/Networking/Shared/KestrelMemoryPool.cs | Removes legacy Kestrel memory pool wrapper. |
| src/Orleans.Core/Networking/Shared/ISocketsTrace.cs | Removes legacy socket trace interface. |
| src/Orleans.Core/Networking/Shared/IOQueue.cs | Removes legacy IO queue scheduler. |
| src/Orleans.Core/Networking/Shared/DuplexPipe.cs | Removes legacy duplex pipe helper. |
| src/Orleans.Core/Networking/Shared/CorrelationIdGenerator.cs | Removes legacy correlation id generator location. |
| src/Orleans.Core/Networking/Shared/BufferExtensions.cs | Removes legacy buffer extensions. |
| src/Orleans.Core/Networking/MessageWriteRequest.cs | Adds message write batching/framing request abstraction for transport layer. |
| src/Orleans.Core/Networking/MessageHandlerShared.cs | Adds shared pools for serializers/read/write requests across connections. |
| src/Orleans.Core/Networking/IUnderlyingTransportFeature.cs | Removes legacy underlying-transport feature tied to IDuplexPipe. |
| src/Orleans.Core/Networking/CorrelationIdGenerator.cs | Adds correlation id generator in new networking stack location. |
| src/Orleans.Core/Networking/ConnectionTrace.cs | Adds connection trace diagnostic/logger wrapper. |
| src/Orleans.Core/Networking/ConnectionShared.cs | Refactors shared connection state and adds lazy MessageHandlerShared access. |
| src/Orleans.Core/Networking/ConnectionOptions.cs | Adds CloseConnectionTimeout option. |
| src/Orleans.Core/Networking/ConnectionManager.cs | Updates to renamed RunAsync entry point. |
| src/Orleans.Core/Networking/ConnectionLogScope.cs | Updates connection log scope content/shape. |
| src/Orleans.Core/Networking/ConnectionFrameHelper.cs | Updates framing helper implementation and documentation. |
| src/Orleans.Core/Networking/ConnectionFactory.cs | Refactors connection factory to use message transport connector + middleware pipeline. |
| src/Orleans.Core/Networking/ConnectionDirection.cs | Reintroduces/moves connection direction and adds transport protocol feature types. |
| src/Orleans.Core/Networking/ConnectionBuilderDelegates.cs | Removes legacy builder delegate storage tied to ASP.NET Core connections. |
| src/Orleans.Core/Networking/ClientOutboundConnectionFactory.cs | Refactors client outbound connection creation to message transport model. |
| src/Orleans.Core/Networking/ClientOutboundConnection.cs | Refactors outbound connection implementation to message transport model and updates metrics/cleanup. |
| src/Orleans.Core/Networking/ClientConnectionOptions.cs | Removes legacy client connection options tied to ASP.NET Core connections. |
| src/Orleans.Core/Messaging/MessageFactory.cs | Preserves TTL correctly via helper rather than direct assignment. |
| src/Orleans.Core/Messaging/ClientMessageCenter.cs | Disposes messages when client is not running / after rejection. |
| src/Orleans.Core/Messaging/CachingSiloAddressCodec.cs | Nullable/cleanup changes around inner reader/writer creation and timestamp usage. |
| src/Orleans.Core/Hosting/ClientTlsHostingExtensions.IClientBuilder.cs | Moves TLS client hosting extensions to new transport middleware model and adds config validation. |
| src/Orleans.Core.Abstractions/Runtime/IMessageReceiverCache.cs | Adds public cache interface for message receiver resolution. |
| src/Orleans.Connections.Security/Security/TlsServerAuthenticationOptions.cs | Deletes old TLS option wrapper (moved to Orleans.Core). |
| src/Orleans.Connections.Security/Security/TlsOptions.cs | Deletes old TLS options type (moved to Orleans.Core). |
| src/Orleans.Connections.Security/Security/TlsDuplexPipe.cs | Deletes old TLS duplex pipe adapter. |
| src/Orleans.Connections.Security/Security/TlsConnectionFeature.cs | Deletes old TLS feature implementation (moved). |
| src/Orleans.Connections.Security/Security/TlsClientAuthenticationOptions.cs | Deletes old TLS client options wrapper (moved). |
| src/Orleans.Connections.Security/Security/RemoteCertificateMode.cs | Deletes old enum (moved). |
| src/Orleans.Connections.Security/Security/OrleansApplicationProtocol.cs | Deletes old application protocol constant. |
| src/Orleans.Connections.Security/Security/MemoryPoolExtensions.cs | Deletes old memory pool helpers. |
| src/Orleans.Connections.Security/Security/ITlsHandshakeFeature.cs | Deletes old TLS handshake feature interface (moved). |
| src/Orleans.Connections.Security/Security/ITlsConnectionFeature.cs | Deletes old TLS connection feature interface (moved). |
| src/Orleans.Connections.Security/Security/ITlsApplicationProtocolFeature.cs | Deletes old TLS app protocol feature interface (moved). |
| src/Orleans.Connections.Security/Security/DuplexPipeStreamAdapter.cs | Deletes old duplex pipe stream adapter. |
| src/Orleans.Connections.Security/Security/CertificateLoader.cs | Deletes old certificate loader (moved). |
| src/Orleans.Connections.Security/Orleans.Connections.Security.csproj | Removes legacy Microsoft.Orleans.Connections.Security project. |
| src/Orleans.Connections.Security/Hosting/HostingExtensions.cs | Removes legacy connection-builder TLS middleware extensions. |
| src/api/Orleans.Runtime/Orleans.Runtime.cs | Updates public API surface: removes SiloConnectionOptions, adds SiloTlsHostingExtensions. |
| src/api/Orleans.Core.Abstractions/Orleans.Core.Abstractions.cs | Updates public API surface to include IMessageReceiverCache. |
| samples/TransportLayerSecurity/TLS.Server/TLS.Server.csproj | Removes package dependency on removed TLS package. |
| samples/TransportLayerSecurity/TLS.Client/TLS.Client.csproj | Removes package dependency on removed TLS package. |
| samples/TransportLayerSecurity/Directory.Packages.props | Removes package version for removed TLS package. |
| Orleans.slnx | Removes removed TLS project from solution. |
| docs/site/src/content/docs/resources/nuget-packages.md | Removes package listing for removed TLS NuGet package. |
| docs/site/src/content/docs/host/snippets/transport-layer-security/csharp/SiloExample/SiloExample.csproj | Updates snippet project to reference source projects instead of removed package. |
| docs/site/src/content/docs/host/snippets/transport-layer-security/csharp/SiloExample/Program.cs | Updates TLS snippet namespaces for new TLS transport security location. |
| docs/site/src/content/docs/host/snippets/transport-layer-security/csharp/ClientExample/Program.cs | Updates TLS snippet namespaces for new TLS transport security location. |
| docs/site/src/content/docs/host/snippets/transport-layer-security/csharp/ClientExample/ClientExample.csproj | Updates snippet project to reference source projects instead of removed package. |
| docs/site/src/content/docs/host/connection-middleware.md | Updates documentation from connection middleware to message transport middleware model. |
Review details
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
- Files reviewed: 178/185 changed files
- Comments generated: 3
- Review effort level: Lite
ff76b0d to
cb0869a
Compare
Code coverage
Report-only conclusion: current-main baseline stale. The newest successful coverage run tested dfd30e9, not current main ec48d17. Coverage combines every CI test matrix job, including providers, CodeGen, .NET 8/10, Linux, Windows, and macOS, using canonical physical source and branch identities. The comparison remains report-only while normal line and branch variance is calibrated. Coverage details |
cb0869a to
60f0ff3
Compare
There was a problem hiding this comment.
Copilot review overview
Review tier: Lite
Findings: 3
New issues introduced by this change (2)
| Severity | Finding |
|---|---|
src/Orleans.Core/Networking/Transport/Sockets/TcpMessageTransportListener.cs — IsValid only checks Enabled, but CreateListenSocket/BindAsync dereference… |
|
src/Orleans.Core/Networking/Transport/MessageTransport.cs — The <returns> docs for EnqueueRead and EnqueueWrite appear to be swapped (they refer to… |
Pre-existing issues (2)
| Severity | Finding |
|---|---|
src/Orleans.TestingHost/UnixSocketTransport/UnixDomainSocketMessageTransportListener.cs — Unix domain socket endpoints are filesystem entries which can persist after the listener socket is… View comment |
|
src/Orleans.TestingHost/UnixSocketTransport/UnixDomainSocketMessageTransportConnector.cs — The connect awaitable uses a TaskCompletionSource without RunContinuationsAsynchronously, so… View comment |
Issues resolved since last review (2)
| Severity | Finding |
|---|---|
src/Orleans.Core/Networking/ConnectionFrameHelper.cs — WriteFrameWithPrefixingWriter now always allocates an ArrayBufferWriter and copies the payload into… View resolved comment |
|
src/Orleans.Core/Networking/Transport/Sockets/TcpMessageTransportConnector.cs — The connect awaitable uses a TaskCompletionSource without RunContinuationsAsynchronously, so… View resolved comment |
Suppressed comments (1)
src/Orleans.Core/Networking/Transport/Sockets/TcpMessageTransportListener.cs:168
TcpMessageTransportListenerallocates aCancellationTokenSource(_closingCts) but never disposes it. This can leak registrations/resources over time (especially in tests which create many listeners).UnixDomainSocketMessageTransportListeneralready disposes its CTS; this listener should too.
60f0ff3 to
2745a13
Compare
There was a problem hiding this comment.
Copilot review overview
Review tier: Lite
Findings: 4
New issues introduced by this change (1)
| Severity | Finding |
|---|---|
src/Orleans.Runtime/Hosting/SiloTlsHostingExtensions.ISiloBuilder.cs — configureOptions is applied to named TlsOptions via AddOptions(...).Configure(configureOptions)… |
Pre-existing issues (4)
| Severity | Finding |
|---|---|
src/Orleans.Core/Networking/Transport/Sockets/TcpMessageTransportListener.cs — IsValid only checks Enabled, but CreateListenSocket/BindAsync dereference… View comment |
|
src/Orleans.TestingHost/UnixSocketTransport/UnixDomainSocketMessageTransportListener.cs — Unix domain socket endpoints are filesystem entries which can persist after the listener socket is… View comment |
|
src/Orleans.TestingHost/UnixSocketTransport/UnixDomainSocketMessageTransportConnector.cs — The connect awaitable uses a TaskCompletionSource without RunContinuationsAsynchronously, so… View comment |
|
src/Orleans.Core/Networking/Transport/MessageTransport.cs — The <returns> docs for EnqueueRead and EnqueueWrite appear to be swapped (they refer to… View comment |
Suppressed comments (4)
src/Orleans.Core/Networking/Transport/MessageTransport.cs:30
- XML doc return description for EnqueueRead says "write request" but this method enqueues a read request, which is confusing for API consumers.
src/Orleans.Runtime/Hosting/SiloTlsHostingExtensions.ISiloBuilder.cs:118 - configureOptions is applied to named TlsOptions via AddOptions(...).Configure(configureOptions) and to all options via services.Configure(configureOptions), so the same action will run twice for the named "gateway" options. If configureOptions isn't idempotent, this can lead to duplicated configuration.
src/Orleans.TestingHost/UnixSocketTransport/UnixDomainSocketMessageTransportListener.cs:70 - Unix domain socket endpoints leave a filesystem entry at the configured path. BindAsync currently doesn't attempt to remove an existing socket file, so a stale path from a previous run can cause AddressAlreadyInUse failures (especially in repeated local test runs). Consider deleting any existing file at the path before binding.
src/Orleans.Core/Networking/Transport/MessageTransport.cs:37 - XML doc return description for EnqueueWrite says "read request" but this method enqueues a write request, which is confusing for API consumers.
There was a problem hiding this comment.
Copilot review overview
Review tier: Lite
Findings: 2
New issues introduced by this change (2)
| Severity | Finding |
|---|---|
src/Orleans.Runtime/Hosting/SiloTlsHostingExtensions.ISiloBuilder.cs — SiloTlsHostingExtensions references ClientTlsHostingExtensions.TlsOptionsValidator, but that… |
|
src/Orleans.Core/Networking/Transport/Security/ClientTlsMessageTransport.cs — The local variable cert is inferred as non-nullable X509Certificate2, but it is then reassigned… |
Issues resolved since last review (5)
| Severity | Finding |
|---|---|
src/Orleans.Runtime/Hosting/SiloTlsHostingExtensions.ISiloBuilder.cs — configureOptions is applied to named TlsOptions via AddOptions(...).Configure(configureOptions)… View resolved comment |
|
src/Orleans.Core/Networking/Transport/MessageTransport.cs — The <returns> docs for EnqueueRead and EnqueueWrite appear to be swapped (they refer to… View resolved comment |
|
src/Orleans.Core/Networking/Transport/Sockets/TcpMessageTransportListener.cs — IsValid only checks Enabled, but CreateListenSocket/BindAsync dereference… View resolved comment |
|
src/Orleans.TestingHost/UnixSocketTransport/UnixDomainSocketMessageTransportListener.cs — Unix domain socket endpoints are filesystem entries which can persist after the listener socket is… View resolved comment |
|
src/Orleans.TestingHost/UnixSocketTransport/UnixDomainSocketMessageTransportConnector.cs — The connect awaitable uses a TaskCompletionSource without RunContinuationsAsynchronously, so… View resolved comment |
There was a problem hiding this comment.
Copilot review overview
Review tier: Lite
Findings: None
Issues resolved since last review (2)
| Severity | Finding |
|---|---|
src/Orleans.Core/Networking/Transport/Security/ClientTlsMessageTransport.cs — The local variable cert is inferred as non-nullable X509Certificate2, but it is then reassigned… View resolved comment |
|
src/Orleans.Runtime/Hosting/SiloTlsHostingExtensions.ISiloBuilder.cs — SiloTlsHostingExtensions references ClientTlsHostingExtensions.TlsOptionsValidator, but that… View resolved comment |
Suppressed comments (2)
Previously missed (2) — in code that hasn't changed since the last review.
src/Orleans.Core/Networking/ClientOutboundConnection.cs:91
- The exception filter
when ((error = exception) is null)is always false (the caught exception is never null), so this catch block is effectively unreachable anderrorwill never be set on failure. This prevents passing the actual failure exception to OnConnectionTerminated.
src/Orleans.Core/Networking/Transport/Sockets/SocketsLog.cs:10 - This file is under
.../Transport/Sockets/and is consumed by other socket transport types (for exampleTcpMessageTransportConnector.cs:11usesnamespace Orleans.Connections.Transport.Sockets;), butSocketsLogis declared inOrleans.Connections.Sockets. The mismatched namespace makes discovery and usage inconsistent with the rest of the socket transport code.
5615925 to
ac5d41e
Compare
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The new TLS options surface has a concrete XML-doc issue and an API design inconsistency around client certificate selector nullability which should be resolved before merging.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Lite
Findings: 1
New issues introduced by this change (3)
| Severity | Finding |
|---|---|
src/Orleans.Core/Networking/Transport/Security/TlsOptions.cs — TlsOptions.LocalClientCertificateSelector is declared as returning a non-null X509Certificate2, but… |
|
src/Orleans.Core/Networking/Transport/Security/TlsOptions.cs — The XML doc comment has a duplicated <summary> tag and an orphaned description about an "enabled"… |
|
test/Orleans.Serialization.UnitTests/PooledBufferTests.cs — This test name still uses "SliceEnumerators" even though it now only validates the SpanEnumerator.… |
ac5d41e to
f64ce56
Compare
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
It introduces a new low-level transport (including io_uring) and performs a broad networking/TLS refactor touching core runtime behavior and resource lifetimes.
Review tier: Lite
Findings: 1
New issues introduced by this change (1)
| Severity | Finding |
|---|---|
src/Orleans.Core/Networking/Transport/MessageTransportListener.cs — The XML docs for BindAsync indicate it returns "The bound endpoint configuration", but the method… |
Issues resolved since last review (3)
| Severity | Finding |
|---|---|
test/Orleans.Serialization.UnitTests/PooledBufferTests.cs — This test name still uses "SliceEnumerators" even though it now only validates the SpanEnumerator.… View resolved comment |
|
src/Orleans.Core/Networking/Transport/Security/TlsOptions.cs — The XML doc comment has a duplicated <summary> tag and an orphaned description about an "enabled"… View resolved comment |
|
src/Orleans.Core/Networking/Transport/Security/TlsOptions.cs — TlsOptions.LocalClientCertificateSelector is declared as returning a non-null X509Certificate2, but… View resolved comment |
Reuse the CQ head already loaded while locating each completion and use liburing's cached ring mask, preserving immediate slot release while removing redundant shared-memory reads. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Check the completion queue after submission and avoid submit_and_wait when inline completions are already available. This reduced io_uring_enter calls by roughly 37% under traced empty-message load and improved throughput across all measured payloads. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Route sub-4 KiB sends through the receive engine so inline completions can enqueue follow-up I/O without a cross-engine eventfd wake. Keep larger sends on a second engine to preserve parallel CQ draining. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Allow ArcBufferWriter to link externally filled pinned pages and extend incrementally received tails while preserving page reference counting and slice lifetimes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Add an opt-in adaptive receiver using multishot recv, incremental provided-buffer rings, and copy-free ArcBuffer page adoption. Start with one-shot receives, promote after sustained large frames, and demote only at a quiescent request boundary. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Use IORING_SEND_VECTORIZED for scatter batches of at least 32 KiB on Linux 6.17 and later, avoiding the live msghdr while retaining SENDMSG for smaller batches. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Request zero-copy usage notifications and disable zero-copy per sender after four consecutive copied or fallback sends, avoiding repeated notification overhead on unsupported paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Allow physical-NIC deployments to opt into per-ring NAPI busy polling using ORLEANS_IO_URING_NAPI_BUSY_POLL_US while leaving the default interrupt-driven path unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Vectorized sends improved isolated 64 KiB throughput but regressed 16 KiB batched traffic and could strand partial sends during benchmark shutdown. Retain the robust SENDMSG paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Store provided-buffer metadata contiguously and avoid locked updates for counters with a single writer or existing lock protection. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Preserve framed-length reporting and correct the engine scope after replaying the io_uring stack over the latest message transport refactor. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Pass the current xUnit cancellation token through the rebased io_uring lifecycle tests to satisfy main's analyzer rules and keep timeouts responsive. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Attempt nonblocking sends directly for pinned contiguous buffers below 16 KiB, falling back to io_uring when the socket would block. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Keep small sends on one engine, create the large-send engine lazily, and promote sustained sub-1 KiB receives to copy-free multishot on supported kernels. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Reserve the reusable io_uring sender state around optimistic native sends and verify disposed senders reject further use. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Keep sender lifecycle ownership through io_uring fallback and capture native send errors before the reusable operation becomes idle. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9dc3f7d8-8a24-4942-b7bf-1537605e1661
Integrate the io-only series with upstream 1cc211b. Keep upstream keepalive configuration, CreateAndStartAsync startup cleanup, and transport disposal contracts while retaining the complete io_uring engine and optimizations. Read FIN interrupts writes using socket shutdown without disposing io_uring descriptors before send completions and zero-copy notifications retire. Add regression coverage for descriptor ownership, all receive modes, and connector/listener startup selection. Capture Closed tokens before disposal and document Linux transport prerequisites and switches. Rebase omissions: 0539394 is superseded by upstream nullable certificate selection and unconditional ValidateCertificate handling. 2b06962 removes a temporary Unix listener implementation which is already absent upstream; preserve the current upstream listener and lifecycle tests. The obsolete Unix dependency-only portions of e383242 were likewise omitted, undoing the transient stale-file cleanup from 2130baa without changing the final upstream Unix implementation. No io_uring optimizations were dropped. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Read an initialized published serializer service before constructing another graph scope. Keep admission and fault checks, and isolate the allocating factory closure on the cold path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Borrow a cleared, bounded per-thread visited set without caching admission decisions. Preserve provider-specific overrides and graph boundaries while removing 176 bytes from each warmed default-service lookup. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
6e7a169 to
05c9d2d
Compare
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Critical framing and SIGPIPE issues remain unresolved, and reference metadata dependency scanning is incomplete.
Review effort: Lite
Findings: 5
Open (6)
Avoid payload-length guard after frame header consumption · New Set MSG_NOSIGNAL for native send submissions · New Handle io_uring CQ overflow to prevent stuck operations Skip io_uring buffer ring header when indexing entries Cancel pending receives before freeing the provided-buffer ring Enable reference metadata inspection for dependency scans · New
| if (!_hasFrameLengths) | ||
| { | ||
| Span<byte> scratch = stackalloc byte[Message.LENGTH_HEADER_SIZE]; | ||
| var lengthBytes = bufferReader.Peek(in scratch); |
| internal override void PrepareSubmission(LinuxIoUringEngine.IoUringSubmission* submission) | ||
| { | ||
| if (WaitForNotification) | ||
| { | ||
| submission->IoPriority |= SendZeroCopyReportUsage; | ||
| } |
| internal static ISerializableTypeDescription? CreateSerializableTypeDescription(IGeneratorServices services, INamedTypeSymbol symbol, bool inspectReferenceMetadata = false) | ||
| => CreateSerializableTypeDescription(services.Compilation, services.LibraryTypes, services.Options, symbol, inspectReferenceMetadata); | ||
|
|
||
| internal static ISerializableTypeDescription? CreateSerializableTypeDescription(Compilation compilation, LibraryTypes libraryTypes, CodeGeneratorOptions options, INamedTypeSymbol symbol) | ||
| internal static ISerializableTypeDescription? CreateSerializableTypeDescription(Compilation compilation, LibraryTypes libraryTypes, CodeGeneratorOptions options, INamedTypeSymbol symbol, bool inspectReferenceMetadata = false) |





Depends on #11379.
Includes the allocation fixes from #11416.
Problem
The message-oriented transport merged in #10074 provides the batching and buffer-lifetime boundaries needed for a native completion transport. Its socket implementation still routes operations through the general-purpose .NET async socket engine.
Solution
io_uringtransport usingORLEANS_USE_IO_URING=1or theOrleans.Connections.Transport.Sockets.UseIoUringAppContext switch.Integration
The branch is stacked on #11379 at
e5724550f65343ef57faed89437dfed85e7ae8dc, which includes the measured main (b084e035b8ebcc7933f09515d09f2bf9fb7da776). The updated io_uring head, including the serializer allocation fixes, is05c9d2d1bc88e92542784965d19d416b694b1eb9.Measured comparison
The table below is the pre-allocation-fix comparison, pinned to main
b084e035b, PR 1137924f2ae508, and io_uring6e7a169dd. It uses .NET SDK 10.0.401/runtime 10.0.12, Release, Linux 7.0, and an Intel i7-11850H. Client and silo ran in separate processes pinned to disjoint groups of four physical cores plus SMT siblings. Throughput uses 250 concurrent calls over plaintext TCP loopback; latency uses one outstanding call. Figures are medians of three fresh-process runs, with rotated variant order, 20-second warmup, and 30-second measurement.CPU includes client+silo user/system time. RSS is their combined mean resident memory, including shared pages twice and excluding kernel-only socket/ring memory. The shared laptop retained its powersave governor and desktop processes; small-payload results vary, so a modest median lead is not a guarantee. The 81 runs completed 411,153,176 measured calls without failures, and live ring descriptors verified io_uring activation in both processes.
In this pre-fix comparison, the stacked transport improves 16 KiB throughput by 86.9% over its PR 11379 base. Small typed results remain slower and consume more CPU than main: this comparison does not claim an across-the-board improvement. Completion-only calls additionally expose 232 managed allocation bytes/call on the PR base. The fixes in #11416 remove this cache-hit allocation and reuse cleared default-admission traversal state; their focused warmed-up regressions require zero allocation.
These figures are pinned to
24f2ae508/6e7a169dd. Later dependency updates are not silently substituted into the results. Earlier measurements against different source revisions have been superseded.Post-fix follow-up
A separate three-run comparison uses main
b084e035b, the refreshed PR 11379 basee5724550f, allocation-fix branch6704d279c, and io_uring05c9d2d1b, with the same harness, core affinity, warmup, and measurement durations.io_uring's empty-call throughput is 17.1% above main in this batch: all three io_uring runs (338k-382k calls/s) exceed all three main runs (293k-321k). The allocation fix improves empty-call throughput by 13.2% over its dependency and removes the additional 232 bytes/call. Small typed results remain below main; the improvement is not across-the-board. All 36 follow-up runs completed 235,988,455 calls without failures.
Compatibility
The implementation requires little-endian 64-bit Linux, kernel 6.1 or later, and
liburing.so.2; multishot incremental receives require Linux 6.12 or later. The transport remains opt-in.This PR remains draft while its dependency, physical-NIC validation, and the native dependency strategy are finalized. Loopback avoids zero-copy send, so local loopback measurements do not validate physical-NIC zero-copy or NAPI behavior.