Repository navigation
refactor(engine): share ORT session ownership - #2122
Merged
Merged
Conversation
Convert the remaining primary, draft, and EAGLE-3 session holders to Arc and give persistent decode runners a structural shared-session owner. Remove the synthetic static session borrow while keeping bindings, allocators, values, KV, and graph-capture state worker-local. Preserve the concurrent-run capability signal and W=1 behavior, add lifecycle/parity coverage, and update the session concurrency phase status without enabling multi-worker execution. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Copilot <223556219+Copilot@users.noreply.github.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2122 +/- ##
==========================================
+ Coverage 80.29% 80.70% +0.40%
==========================================
Files 426 429 +3
Lines 205244 215150 +9906
Branches 205244 215150 +9906
==========================================
+ Hits 164801 173636 +8835
- Misses 34792 35742 +950
- Partials 5651 5772 +121
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Arc<Session>ownership'staticsession borrowIoBinding, allocator, deviceValue, KV/cache, and graph-capture state runner/worker-localSession::supports_concurrent_run()as a capability signal and add stable-address, lifecycle, and W=1 continuation parity coverageSESSION_CONCURRENCY.mdonly for this completed prerequisiteScope
This PR is a shared immutable ORT ownership prerequisite only. It does not enable N>1 workers, concurrent session runs, new server worker configuration, or concurrent use of providers/configurations that decline
supports_concurrent_run().Validation
cargo test -p onnx-genai-engine --libcargo test -p onnx-genai-engine --lib --features native-backendcargo test -p onnx-genai-engine --test multi_session --test canonical_execution_parity --test chained_proposer_realcargo test -p onnx-genai-ortcargo test -p onnx-genai-server --libcargo check -p onnx-genai-server --all-targetscargo clippy -p onnx-genai-ort -p onnx-genai-engine -p onnx-genai-server --all-targets -- -D warningscargo clippy -p onnx-genai-engine -p onnx-genai-server --all-targets --features native-backend -- -D warningscargo fmt --all -- --check