feat(codegen): support custom grain-call return types

[ReubenBond]

Proposal

Allow grain-call return types to register the invokable base used by generated proxies without adding return-type-specific logic to Orleans. InvokableBaseTypeAttribute can now be applied to the return type or at assembly scope, so an adapter assembly can connect independently owned return types, proxy bases, and invokable bases.

Resolution semantics

Mappings are resolved deterministically with exact closed registrations before open generic registrations, then by source precedence: method override, return-type registration, assembly adapter, and proxy default. A compilation-scoped resolver is shared by the analyzer, generator, and incremental interface model. Discovery sorts current and referenced assemblies, coalesces identical registrations, and reports stable diagnostics for conflicts, attempts to replace built-ins, inaccessible or incompatible bases, generic arity and constraint failures, invalid return-value initializer signatures, and bases which generated requests cannot construct.

Generics and compatibility

Open generic mappings are normalized and constructed from the grain method return type after arity and recursively substituted constraint validation. Existing proxy defaults and method-level overrides retain their behavior, including ReturnValueProxy initialization and generated-activator construction. Invokable bases must expose an accessible parameterless constructor or a generator-compatible accessible [GeneratedActivatorConstructor] path. The adapter mechanism remains in the serialization/code-generation layer and adds no Orleans Core or provider dependency.

Microsoft Reviewers: Open in CodeFlow

[Copilot]

Pull request overview

This PR extends Orleans’ code-generation/analyzer pipeline to support custom grain-call return types by allowing InvokableBaseTypeAttribute registrations on return types and at assembly scope, and centralizing deterministic mapping resolution in a shared resolver used by the analyzer, generator, and model extraction.

Changes:

• Expanded InvokableBaseTypeAttribute usage targets (including assembly and struct) and updated public API surface accordingly.
• Introduced a shared InvokableBaseTypeResolver and integrated it into codegen (proxy/invokable generation) and analyzer validation, including new diagnostics for invalid mappings.
• Added unit tests covering custom return types, precedence rules, referenced adapter discovery/coalescing, and deterministic diagnostics.

Show a summary per file

File	Description
test/Orleans.CodeGenerator.Tests/CustomReturnTypeTests.cs	Adds codegen tests for custom return types, precedence, adapter assembly discovery, and diagnostics stability.
test/Orleans.Analyzers.Tests/GrainInterfaceMethodReturnTypeDiagnosticAnalyzerTest.cs	Adds analyzer test ensuring registered custom return types are accepted.
src/Orleans.Serialization.Abstractions/Annotations.cs	Expands InvokableBaseTypeAttribute applicability to return types and assemblies.
src/Orleans.CodeGenerator/ProxyGenerationContext.cs	Wires shared resolver into proxy generation context and mapping extraction.
src/Orleans.CodeGenerator/Orleans.CodeGenerator.csproj	Links the shared resolver into the code generator project.
src/Orleans.CodeGenerator/Model/ProxyInterfaceModelExtractor.cs	Uses the shared resolver to extract invokable base type mappings for the incremental model.
src/Orleans.CodeGenerator/InvokableGenerator.cs	Switches invokable base selection to resolver-based resolution and surfaces mapping diagnostics.
src/Orleans.CodeGenerator/Diagnostics/InvokableBaseTypeMappingDiagnostic.cs	Adds a generator diagnostic wrapper for invalid invokable base type mappings (ORLEANS0111).
src/Orleans.CodeGenerator/Diagnostics/DiagnosticRuleId.cs	Registers new diagnostic ID ORLEANS0111.
src/Orleans.CodeGenerator/AnalyzerReleases.Unshipped.md	Documents new ORLEANS0111 diagnostic.
src/Orleans.CodeGenerator.Shared/InvokableBaseTypeResolver.cs	Adds the shared resolver: discovery, precedence, construction/validation, and deterministic diagnostics.
src/Orleans.Analyzers/Orleans.Analyzers.csproj	Links the shared resolver into the analyzers project.
src/Orleans.Analyzers/GrainInterfaceMethodReturnTypeDiagnosticAnalyzer.cs	Updates analyzer to validate return types via resolver and adds ORLEANS0026 for invalid mappings.
src/Orleans.Analyzers/AnalyzerReleases.Unshipped.md	Documents new ORLEANS0026 diagnostic.
src/api/Orleans.Serialization.Abstractions/Orleans.Serialization.Abstractions.cs	Updates API surface file to reflect expanded attribute usage targets.

Review details

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

• Files reviewed: 15/15 changed files
• Comments generated: 2
• Review effort level: Lite

[Copilot]

Review details

Suppressed comments (4)

Previously missed (2) — in code that hasn't changed since the last review.

src/Orleans.Analyzers/GrainInterfaceMethodReturnTypeDiagnosticAnalyzer.cs:18

• Diagnostic Title still says "Grain interfaces methods must return a compatible type" while the message now reports "registered grain-call return type". Updating the Title to match the new meaning (and fix the grammar: interface vs interfaces) will make the diagnostic clearer in IDE error lists.

This issue also appears on line 87 of the same file.

        public const string DiagnosticId = "ORLEANS0009";
        public const string Title = "Grain interfaces methods must return a compatible type";
        public const string MessageFormat = "Grain interface methods must return a registered grain-call return type";
        public const string Category = "Usage";

src/Orleans.CodeGenerator.Shared/InvokableBaseTypeResolver.cs:133

• When InvokableBaseTypeAttribute is applied directly to a return type, GetReturnTypeMappings currently trusts the attribute’s returnType constructor argument without validating it matches the annotated type. That allows a return type to accidentally (or intentionally) register mappings for unrelated types, which is surprising given the attribute target. Consider filtering these to only mappings whose ReturnType.OriginalDefinition matches the annotated type’s OriginalDefinition.

        var builder = ImmutableArray.CreateBuilder<Mapping>();
        AddMappings(named.OriginalDefinition.GetAttributes(), MappingKind.ReturnType, builder);
        return Sort(builder);
    }

src/Orleans.CodeGenerator/InvokableGenerator.cs:262

• Avoid branching on ResolverDiagnostic.Message text (StartsWith("No invokable base type is registered")). This is brittle (message wording changes/localization) and couples the generator to a specific string; it also duplicates the same check in the analyzer. Prefer returning a structured diagnostic kind from TryResolve (eg, NotRegistered vs InvalidMapping) or adding a property on ResolverDiagnostic so callers can branch reliably.

        if (resolverDiagnostic is not null
            && !resolverDiagnostic.Message.StartsWith("No invokable base type is registered", StringComparison.Ordinal))
        {
            throw new OrleansGeneratorDiagnosticAnalysisException(
                InvokableBaseTypeMappingDiagnostic.CreateDiagnostic(resolverDiagnostic));
        }

src/Orleans.Analyzers/GrainInterfaceMethodReturnTypeDiagnosticAnalyzer.cs:92

• The analyzer filters mapping failures by checking whether ResolverDiagnostic.Message starts with a specific string. This is fragile and duplicates generator logic. Consider having InvokableBaseTypeResolver return a structured result (eg, enum/flag on ResolverDiagnostic indicating NotRegistered vs InvalidMapping) so the analyzer can decide which diagnostic to report without depending on message text.

                if (diagnostic is not null
                    && !diagnostic.Message.StartsWith("No invokable base type is registered", StringComparison.Ordinal))
                {
                    mappingDiagnostic ??= diagnostic;
                }
            }

• Files reviewed: 15/15 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

Review details

Suppressed comments (4)

Previously missed (2) — in code that hasn't changed since the last review.

src/Orleans.CodeGenerator.Shared/InvokableBaseTypeResolver.cs:15

• The "No invokable base type is registered" prefix is relied on by both the analyzer and generator via StartsWith checks. Centralizing that prefix in a single constant reduces fragility and avoids string-literal drift between components.

This issue also appears on line 67 of the same file.

    internal const string InvokableBaseTypeAttributeMetadataName = "Orleans.InvokableBaseTypeAttribute";
    internal const string DefaultInvokableBaseTypeAttributeMetadataName = "Orleans.DefaultInvokableBaseTypeAttribute";
    internal const string ReturnValueProxyAttributeMetadataName = "Orleans.Invocation.ReturnValueProxyAttribute";
    internal const string GeneratedActivatorConstructorAttributeMetadataName = "Orleans.GeneratedActivatorConstructorAttribute";

src/Orleans.Analyzers/GrainInterfaceMethodReturnTypeDiagnosticAnalyzer.cs:88

• This path distinguishes "not registered" vs "invalid mapping" by matching a hard-coded message prefix. Using a shared prefix constant from InvokableBaseTypeResolver reduces the chance of subtle behavior changes if the resolver message text is edited.

                if (diagnostic is not null
                    && !diagnostic.Message.StartsWith("No invokable base type is registered", StringComparison.Ordinal))

src/Orleans.CodeGenerator.Shared/InvokableBaseTypeResolver.cs:69

• This diagnostic text is later pattern-matched by other components. Using a shared prefix constant (and formatting the message from it) makes that coupling explicit and easier to keep consistent.

        diagnostic = new ResolverDiagnostic(
            $"No invokable base type is registered for return type '{Display(returnType)}' and proxy base '{Display(proxyBaseType)}'.",
            method.Locations.FirstOrDefault());

src/Orleans.CodeGenerator/InvokableGenerator.cs:258

• This logic depends on a string prefix to distinguish "not registered" from "invalid mapping" diagnostics. To avoid brittle string literals drifting across projects, use a shared prefix constant from InvokableBaseTypeResolver.

        if (resolverDiagnostic is not null
            && !resolverDiagnostic.Message.StartsWith("No invokable base type is registered", StringComparison.Ordinal))

• Files reviewed: 15/15 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

Review details

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

src/Orleans.Analyzers/GrainInterfaceMethodReturnTypeDiagnosticAnalyzer.cs:91

• This analyzer distinguishes “unregistered return type” from “invalid mapping” via diagnostic.Message.StartsWith("No invokable base type is registered" ...). Message-text matching is brittle for control flow (string changes/localization) and couples the analyzer to the resolver’s wording. Prefer a typed discriminator from the resolver (e.g., an enum/bool on ResolverDiagnostic) or a null diagnostic for the “not registered” case.

                if (diagnostic is not null
                    && !diagnostic.Message.StartsWith("No invokable base type is registered", StringComparison.Ordinal))
                {
                    mappingDiagnostic ??= diagnostic;
                }

src/Orleans.CodeGenerator/InvokableGenerator.cs:262

• This code branches on resolverDiagnostic.Message.StartsWith("No invokable base type is registered" ...) to distinguish “unregistered return type” from “invalid mapping”. Using string matching for control flow is brittle (message text changes/localization will break behavior) and duplicates the message contract across components. Consider giving the resolver a typed discriminator (e.g., ResolverDiagnosticKind / IsMissingMapping) or returning diagnostic = null for the “not registered” case, then key off that instead.

        if (resolverDiagnostic is not null
            && !resolverDiagnostic.Message.StartsWith("No invokable base type is registered", StringComparison.Ordinal))
        {
            throw new OrleansGeneratorDiagnosticAnalysisException(
                InvokableBaseTypeMappingDiagnostic.CreateDiagnostic(resolverDiagnostic));
        }

• Files reviewed: 15/15 changed files
• Comments generated: 1
• Review effort level: Lite

[Copilot]

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

src/Orleans.Analyzers/GrainInterfaceMethodReturnTypeDiagnosticAnalyzer.cs:93

• The analyzer returns as soon as any proxy base type resolves, but [GenerateMethodSerializers] supports AllowMultiple=true and an interface can have multiple proxy bases (directly or via inheritance). If a return type is registered for one proxy base but missing for another, the analyzer will not report a diagnostic even though code generation for the other proxy base would fail. Track whether all proxy base types resolve and only return early when they all do.

            foreach (var proxyBaseType in proxyBaseTypes)
            {
                if (resolver.TryResolve(proxyBaseType, symbol, out _, out var diagnostic))
                {
                    return;

• Files reviewed: 15/15 changed files
• Comments generated: 1
• Review effort level: Lite

[Copilot]

Review details

• Files reviewed: 19/19 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

Review details

• Files reviewed: 22/22 changed files
• Comments generated: 1
• Review effort level: Lite