Repository navigation
fix(runtime): support scoped cancellation acknowledgments - #11241
ReubenBond wants to merge 5 commits into
Conversation
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
The runtime changes span multiple invocation paths and require final human review.
Review tier: Lite
Findings: None
What changed in this PR
Adds opt-in, per-invocation cancellation acknowledgements while preserving existing global behavior.
Changes:
- Propagates scoped cancellation policy through invocation and callback pipelines.
- Adds scope, callback, filter, shutdown, and request-reuse tests.
- Adds focused CI coverage.
| File | Summary |
|---|---|
test/Orleans.Runtime.Tests/CancellationTests/ScopedCancellationRpcTests.cs |
Tests scoped cancellation across generated proxies, filters, reuse, and shutdown. |
test/Orleans.Runtime.Tests/CancellationTests/CancellationAcknowledgementScopeTests.cs |
Tests scope nesting and execution-context restoration. |
test/Orleans.Runtime.Tests/CallbackDataTests.cs |
Tests per-invocation callback acknowledgement behavior. |
src/Orleans.Runtime/Core/InsideRuntimeClient.cs |
Carries acknowledgement policy for silo calls. |
src/Orleans.Core/Runtime/OutsideRuntimeClient.cs |
Carries acknowledgement policy for client calls. |
src/Orleans.Core/Runtime/OutgoingCallInvoker.cs |
Preserves policy through outgoing filters. |
src/Orleans.Core/Runtime/IRuntimeClient.cs |
Extends the request dispatch contract. |
src/Orleans.Core/Runtime/GrainReferenceRuntime.cs |
Captures and propagates invocation policy. |
src/Orleans.Core/Runtime/CancellationAcknowledgementScope.cs |
Implements scoped policy management. |
src/Orleans.Core/Runtime/CallbackData.cs |
Applies per-callback cancellation policy. |
.github/workflows/rpc-cancellation.yml |
Runs focused cancellation tests across frameworks. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Code coverage
Report-only conclusion: mixed. The current-main baseline is commit Coverage combines every CI test matrix job, including providers, CodeGen, .NET 8/10, Linux, Windows, and macOS, using canonical physical source and branch identities. The comparison remains report-only while normal line and branch variance is calibrated. Coverage details |
|
Closing per the user's direction. The consuming features will use ordinary cancellation and bounded local attempts, fan-out, message sizes, and retry pacing. Local concurrency limits describe active local attempts; they do not require tracking remote execution after a local timeout. The per-invocation acknowledgement policy and extra permit-retention machinery will be removed from the consuming PRs. This PR is retired without merging. |
Problem
Runtime components which bound their own local waits need to retain the underlying caller RPC after signaling cancellation. With the default
WaitForCancellationAcknowledgement = false, cancellation completes and unregisters that RPC immediately, allowing caller-side admission capacity to be released while the callee is still executing.Solution
Add the internal
Orleans.Runtime.CancellationAcknowledgementScope.Enter()opt-in around creation of a generated-proxy task.GrainReferenceRuntimesnapshots and suppresses the scope before outgoing filters run, then explicitly carries the choice through the invocation pipeline intoCallbackData. The callback combines this choice with the global acknowledgement policy.Cancellation signals the callee while the opted-in RPC remains pending until a response, cancellation acknowledgement, response timeout, target failure, or local host shutdown. The configured response timeout and timeout-cancellation policy remain authoritative. Callers place their independently bounded local wait outside the invocation scope.
The policy lives in per-invocation runtime state, preserving generated request metadata, serialization aliases, and request reuse. Asynchronous outgoing filters retain the original invocation's choice, and additional RPCs initiated by those filters select their own policy. Scope disposal restores the prior execution-context value, including nested scopes and synchronous exceptions. Ordinary unscoped calls preserve the existing execution context without writing ambient state.
Rationale and scope
This is an independent shared runtime prerequisite for the split of #10236, alongside lifecycle prerequisite #11237 and manifest retrieval #11239. Feature call sites remain in their owning PRs. The isolatable runtime-and-regressions chain is
dbdbcb348baa6cc0b21cdde4c8cd6fb47163f3cb,2c7153dd7da980704640f8c01ae8efe08fe8ad80, and test-fixture correction2133d59c381972c857a2392f2e32510699428bbb, based on main025bf9ed87ae9636ca56ec9527e97b30e7171456.Admission accounting tracks observable pending caller requests. Remote execution can continue after a runtime timeout; bounded physical remote execution additionally depends on transport delivery and callee cooperation.
Microsoft Reviewers: Open in CodeFlow