Skip to content

fix(nativeaot): generate self-writing RPC responses - #11379

Draft
ReubenBond wants to merge 42 commits into
dotnet:mainfrom
ReubenBond:rb-nativeaot-rpc-responses
Draft

ReubenBond wants to merge 42 commits into
dotnet:mainfrom
ReubenBond:rb-nativeaot-rpc-responses

Conversation

@ReubenBond

@ReubenBond ReubenBond commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

Fixes #11375.

Native RPC response handling selected pooled codecs and copiers by closing open generic implementations at runtime. Generate concrete, non-generic response holders and closed construction graphs for source-known method results and proxy dependencies.

The generator emits identical invocation and registration code for managed and NativeAOT publishing. Closed factories participate through the common provider resolution pipeline, with application registration precedence selecting services in either execution mode. Full emitted-source parity is covered directly, including result models, collections, primitive responses, completion responses, and explicit serializer contexts.

Generated invokables implement IResponseInvokable, isolate mutable results before incoming filters resume, and rent a successful holder with provider-owned dependencies. Incoming filters can transform the response envelope and nested references; the runtime copies their selected result for delivery. The invocation context owns every superseded filter response until completion, supports restoring earlier selections, and releases each unselected wrapper exactly once. Unfiltered calls transfer the invocation-time copy directly, and expired or one-way requests release the owned response. IRawResponseWriter writes the existing raw protocol directly before the sender queries a response codec. Closed IRawResponseReader registrations reconstruct holders from the wire result type. Primitive codecs, reference cycles, shared identity, pool reset, completion responses, exceptions, and custom implementation dispatch retain their established semantics.

Inferred defaults use the existing metadata selector and complete constructor dependency descriptors. Admission compares matching implementation identities, and closed factories supply executable services. Automatic metadata construction retains its ordinary dependency-injection boundary throughout the call chain; inferred default leaves participate in an existing closed transaction or start a transaction as a closed root. Typed and untyped codec/copier lookups use the actual closed service contract for admission. This preserves host startup dependencies such as the migration context's SerializerSessionPool while retaining transactional rollback for explicit graphs.

Concrete factories retain admission checks independently of which alias registration wins. Metadata bridges propagate child requirements, and type-dispatch wrappers follow the winning alias. Inferred codecs and copiers also yield to selected converter contracts, preserving custom array conversion priority. Closed generic source models specialize canonical constructor contracts, including generated activators marked with [GeneratedActivatorConstructor]. Explicit registrations retain their declared behavior. Closed canonical arrays, reference/value tuples, collection surrogates, value/base serializer aliases, and referenced generated activators supply the known construction graph.

The combined inferred response graph admits up to 1,024 closed executable types. RPC candidate and holder admission count that executable graph; explicit serializer contexts retain their separate metadata traversal limits. Aggregate failures report ORLEANS0116 in both managed and native builds while retaining holder declarations. Argument admission failures use argument-specific ORLEANS0116 guidance to register the argument's codec, copier, and serialization-service dependencies, including for one-way contracts.

Rebased onto main at b084e035b8ebcc7933f09515d09f2bf9fb7da776, containing #11382. Explicit serializer contexts and inferred RPC response graphs share the finalized graph collector, constructor inspection, provider-owned construction scopes, and metadata registration contracts. The message-oriented transport and documentation allowlist fix from #11401 are included. NativeAOT targets .NET 10+, with managed .NET 8 compatibility retained. Native coverage uses the shared project and centralized runner from #11367. Converter lookup consumes surrogate metadata captured during initialization, and .NET 10 multidimensional array allocation uses the concrete array type.

Published reconciled head: dd8576836fc2b247b37a44791666695aee5c051f. The rebase preserves concurrently published response-lease and rooted-materialization fixes. Draft remains held for fresh CI and final human review.

Microsoft Reviewers: Open in CodeFlow

Copilot AI balanced review requested due to automatic review settings October 1, 2026 17:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Registration ordering, an inert public generation attribute, and partial dictionary comparer support introduce unresolved behavioral problems.

Review effort: Balanced
Findings: 3 Medium severity

Open (3)
What changed in this PR

Adds NativeAOT-safe, closed RPC response serializer/copier graphs and supporting static serializer-context infrastructure.

Changes:

  • Generates closed response factories and strict diagnostics for concrete RPC results.
  • Adds transactional factory resolution, explicit serializer contexts, and response codecs/copiers.
  • Adds unit, source-generator, and NativeAOT smoke coverage.
File Description
test/​Orleans.Serialization.UnitTests/​RpcResponseTests.cs Tests response codecs, copiers, factories, and invokables.
test/​Orleans.Serialization.UnitTests/​Orleans.Serialization.UnitTests.csproj Links shared RPC smoke contracts.
test/​Orleans.NativeAotSmoke/​RpcResponsesModels.cs Defines smoke RPC contracts and payload.
test/​Orleans.NativeAotSmoke/​RpcResponses.smoke.json Configures strict RPC smoke diagnostics.
test/​Orleans.NativeAotSmoke/​RpcResponses.cs Runs the native response smoke scenario.
test/​Orleans.NativeAotSmoke/​RpcResponses.Contracts.cs Implements shared response serialization checks.
test/​Orleans.NativeAotSmoke/​Orleans.NativeAotSmoke.csproj Defines the NativeAOT smoke project.
test/​Orleans.NativeAotSmoke/​NativeAotEnvironment.cs Verifies execution without dynamic code.
test/​Orleans.NativeAotSmoke/​Factories.cs Runs static factory smoke checks.
test/​Orleans.NativeAotSmoke/​Factories.Contracts.cs Tests cyclic factory publication and rollback.
test/​Orleans.CodeGenerator.Tests/​StaticSerializerFactoryTests.cs Exposes factory contracts as unit tests.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestGrainWithMultipleInterfaces.verified.cs Updates generated response-factory snapshot.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestGrainWithDifferentKeyTypes.verified.cs Updates generated response-factory snapshot.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestGrainMethodAnnotatedWithResponseTimeout.verified.cs Updates generated response-factory snapshot.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestGrainMethodAnnotatedWithInvokableBaseType.verified.cs Updates generated response-factory snapshot.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestGrainComplexGrain.verified.cs Updates complex-result factory snapshot.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestClassWithGenerateMethodSerializersAnnotation.verified.cs Updates generated factory snapshot.
test/​Orleans.CodeGenerator.Tests/​snapshots/​OrleansSourceGeneratorTests.TestBasicGrain.verified.cs Updates basic response-factory snapshot.
test/​Orleans.CodeGenerator.Tests/​OrleansSourceGeneratorTests.cs Tests factory generation and diagnostics.
test/​Orleans.CodeGenerator.Tests/​Orleans.CodeGenerator.Tests.csproj Links shared factory contracts.
src/​Orleans.Serialization/​TypeSystem/​SerializerContextTypeResolver.cs Resolves only context-registered types.
src/​Orleans.Serialization/​Serializers/​CodecProvider.cs Adds static factory lookup and transactional caching.
src/​Orleans.Serialization/​SerializerContext.cs Introduces the serializer-context base class.
src/​Orleans.Serialization/​Invocation/​Response.cs Adds concrete response codec/copier implementations.
src/​Orleans.Serialization/​Hosting/​ServiceCollectionExtensions.cs Adds serializer-context DI registration.
src/​Orleans.Serialization/​Hosting/​SerializerBuilderExtensions.cs Adds builder-level context registration.
src/​Orleans.Serialization/​GeneratedCodeHelpers/​OrleansGeneratedCodeHelper.cs Resolves statically registered services.
src/​Orleans.Serialization/​Configuration/​TypeManifestOptions.cs Adds closed serializer and service factories.
src/​Orleans.Serialization/​Codecs/​DictionaryCodec.cs Adds a default-comparer constructor.
src/​Orleans.Serialization.Abstractions/​Orleans.Serialization.Abstractions.csproj Marks supported targets as AOT-compatible.
src/​Orleans.Serialization.Abstractions/​GenerateSerializerContextAttribute.cs Introduces serializer-context generation metadata.
src/​Orleans.CodeGenerator/​SourceGeneratorOptionsParser.cs Parses response-factory validation options.
src/​Orleans.CodeGenerator/​SerializerFactoryGenerator.cs Builds closed serialization dependency graphs.
src/​Orleans.CodeGenerator/​RpcResponseGenerator.cs Generates closed RPC response factories.
src/​Orleans.CodeGenerator/​Resources.resx Adds the response-factory diagnostic text.
src/​Orleans.CodeGenerator/​OrleansSourceGenerator.cs Integrates response generation into the pipeline.
src/​Orleans.CodeGenerator/​Diagnostics/​DiagnosticRuleId.cs Defines diagnostic ORLEANS0116.
src/​Orleans.CodeGenerator/​build/​Microsoft.Orleans.CodeGenerator.props Exposes AOT validation build properties.
src/​Orleans.CodeGenerator/​AnalyzerReleases.Unshipped.md Records the new diagnostic.
src/​api/​Orleans.Serialization/​Orleans.Serialization.cs Updates the generated serialization API surface.
src/​api/​Orleans.Serialization.Abstractions/​Orleans.Serialization.Abstractions.cs Updates the abstractions API surface.
docs/​site/​src/​content/​docs/​implementation/​serialization.md Documents closed RPC response factories.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/Orleans.Serialization.Abstractions/GenerateSerializerContextAttribute.cs Outdated
Comment thread src/Orleans.Serialization/Codecs/DictionaryCodec.cs
Comment thread src/Orleans.Serialization/Hosting/ServiceCollectionExtensions.cs
Copilot AI balanced review requested due to automatic review settings October 1, 2026 18:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

NativeAOT default response copying remains broken, and the serializer-context attribute lacks generator implementation.

Review effort: Balanced
Findings: 1 High severity · 3 Medium severity

Open (4)

Comment thread src/Orleans.Serialization/Serializers/CodecProvider.cs Outdated
Copilot AI balanced review requested due to automatic review settings October 1, 2026 18:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Native response serialization omits completion and exception codecs, context generation is not implemented, and serializer setup is order-dependent.

Review effort: Balanced
Findings: 1 High severity · 3 Medium severity

Open (4)
Resolved since last review (1)

Comment thread src/Orleans.CodeGenerator/RpcResponseGenerator.cs Outdated
Copilot AI balanced review requested due to automatic review settings October 1, 2026 18:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Serializer setup is order-dependent, the context attribute has no generator implementation, and special response codecs remain unregistered.

Review effort: Balanced
Findings: 1 High severity · 2 Medium severity

Open (3)
Resolved since last review (1)

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Code coverage

Metric Pull request
Lines 83.44% (119,267 / 142,945)
Branches 73.12% (35,783 / 48,940)

Report-only conclusion: current-main baseline stale.

The newest successful coverage run tested dfd30e9, not current main ec48d17.

Coverage combines every CI test matrix job, including providers, CodeGen, .NET 8/10, Linux, Windows, and macOS, using canonical physical source and branch identities.

The comparison remains report-only while normal line and branch variance is calibrated.

Coverage details

Copilot AI balanced review requested due to automatic review settings October 1, 2026 19:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Registration order can omit default services, and strict NativeAOT graphs cannot serialize completed or exception responses.

Review effort: Balanced
Findings: 1 High severity · 1 Medium severity

Open (2)
Resolved since last review (1)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The strict generated response context cannot serialize completed or exception responses despite registering the polymorphic Response codec.

Review effort: Balanced
Findings: 1 High severity

Open (1)
Resolved since last review (1)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Completion-only RPC interfaces do not generate the required NativeAOT response and CompletedResponse registrations.

Review effort: Balanced
Findings: 1 High severity

Open (1)
Resolved since last review (1)

Comment thread src/Orleans.CodeGenerator/RpcResponseGenerator.cs Outdated
Copilot AI balanced review requested due to automatic review settings October 1, 2026 20:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

It substantially changes serializer resolution, concurrent factory publication, public APIs, and NativeAOT code generation across the runtime.

Review effort: Balanced
Findings: None

Resolved since last review (1)

Copilot AI balanced review requested due to automatic review settings October 1, 2026 23:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Response generation incorrectly includes static interface helpers, which can cause unrelated NativeAOT build failures.

Review effort: Balanced
Findings: 1 High severity

Open (1)

Comment thread src/Orleans.CodeGenerator/RpcResponseGenerator.cs Outdated
@ReubenBond
ReubenBond force-pushed the rb-nativeaot-rpc-responses branch from a3ca5c3 to c032496 Compare October 6, 2026 20:11
Copilot AI balanced review requested due to automatic review settings October 6, 2026 20:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

ORLEANS0116 currently gives incorrect guidance for argument-graph and aggregate failures.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
Resolved since last review (3)

Comment thread src/Orleans.CodeGenerator/RpcResponseGenerator.cs Outdated
Copilot AI balanced review requested due to automatic review settings October 6, 2026 20:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Generated API surfaces for additional affected packable projects are missing.

Review effort: Balanced
Findings: 1 High severity

Open (1)
Resolved since last review (1)

Comment thread src/Orleans.CodeGenerator/InvokableGenerator.cs
Copilot AI balanced review requested due to automatic review settings October 6, 2026 21:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Inferred parent response graphs can bypass explicitly registered result codec and copier factories.

Review effort: Balanced
Findings: None

Resolved since last review (1)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Explicit child factories do not invalidate inferred parent services

src/​Orleans.Serialization/​Serializers/​CodecProvider.cs:708

An explicit closed dependency factory does not make an inferred parent graph ineligible. For example, AddSerializer<int>(customCodec, customCopier) replaces the generated IFieldCodec<int>/IDeepCopier<int> services, but the generated Response<int> factory constructs PooledResponseCodec<int, Int32Codec> and PooledResponseCopier<int, ShallowCopier<int>> from concrete canonical services. This loop treats the explicit dependency as satisfied, so GetCodec<Response<int>>() still selects that default and response serialization/copying bypasses the application's explicit factories. The holder's IsSupported check only forces the invocation fallback; it does not prevent the fallback Response<int> codec/copier from using the canonical children. Decline the inferred parent whenever a required canonical dependency has been replaced by an explicit service, unless the parent construction actually consumes that explicit service contract.

Copilot AI balanced review requested due to automatic review settings October 6, 2026 22:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Response generation crashes for metadata-only compilations that contain referenced proxy interfaces but no syntax trees.

Review effort: Balanced
Findings: 1 High severity

Open (1)

Comment thread src/Orleans.CodeGenerator/RpcResponseGenerator.cs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Default admission can incorrectly reject executable closed graphs when an open generic candidate’s constraints do not match.

Review effort: Balanced
Findings: 1 High severity

Open (1)
Resolved since last review (1)

Comment thread src/Orleans.Serialization/Serializers/CodecProvider.cs Outdated
Record canonical service aliases independently of registration order and declare the exact services consumed by inferred factories. Preserve contract-aware parents and custom metadata precedence.

Emit RPC registrations unconditionally and retain managed/native/framework-symbol source parity.
@ReubenBond

Copy link
Copy Markdown
Member Author

Addressed the previously missed "Explicit child factories do not invalidate inferred parent services" feedback in e572455. Canonical implementation aliases now retain their service identity in either registration order, and generated dependency descriptors name the services each factory actually consumes. Parents which use canonical children yield to explicit child contracts; parents which consume those contracts directly remain admitted. This preserves custom metadata selection and applies to codec, copier, and activator aliases. RPC registrations are now emitted unconditionally, with parity checked for managed/native publishing and with/without NET5_0_OR_GREATER.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

It broadly changes generated serialization graphs, response ownership, filtering, pooling, and wire transport and remains held for fresh CI and final human review.

Review effort: Balanced
Findings: 1 High severity

Open (1)

Share candidate constraint validation, source-known argument metadata, immutable response plans, and construction-driven registration emission. Remove redundant manifest maps and pair each closed factory with its service identity.

Centralize response copy/dispose ownership and raw reader adaptation while retaining typed/runtime dispatch, filter isolation, graph publication, and native-safe factory selection.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Successful remote sends currently abandon generated pooled response holders without disposing them.

Review effort: Balanced
Findings: 1 High severity

Open (1)
Resolved since last review (1)

Comment on lines +414 to +415
SendResponse(message, ownedResponse);
ownedResponse = null;

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NativeAOT full-silo startup cannot activate closed PooledResponseCopier<bool>

2 participants