Skip to content

feat(router): bound asynchronous donor inventory lookups [D11] - #18029

Closed
yifjiang wants to merge 1 commit into
codex/kv-reuse-d10-source-eligibilityfrom
codex/kv-reuse-d11-bounded-lookups
Closed

yifjiang wants to merge 1 commit into
codex/kv-reuse-d10-source-eligibilityfrom
codex/kv-reuse-d11-bounded-lookups

Conversation

@yifjiang

@yifjiang yifjiang commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Owned prompt copies, deadlines, blocking-pool saturation and departure handling; query-only paths never book decode capacity.

Review slice D11 of #16748; part of #16747 and #16746. This draft targets codex/kv-reuse-d10-source-eligibility for a complete incremental review diff of 485 additions plus deletions, including tests, documentation and extraction glue.

Extraction details:

  • Host and GPU candidate helper methods move here with their first production caller. Asynchronous query/deadline/semaphore ownership and all CPU lookup regressions are preserved.

Reuse remains experimental and default off. Retained inventory is advisory, complete compatible rank parts must precede readiness, and native operations retain ownership until quiescence. Supported GPU activation also requires the companion KVCR/native slices and the declared deployment profile.

The lower-tier lookup helpers compile with their optional standalone-selection consumers. D12 additionally exposes the GPU helpers to their default unit tests; no test function is gated or skipped by this correction.

Validation

All configured pre-commit hooks passed for this exact slice on allocated SLURM. Stack-level CPU regression coverage is summarized on D31; each draft still requires its own applicable CI in the Checks tab. Historical combined-head and hardware results retain their original source scope.

@devin-ai-integration

devin-ai-integration Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

✅ Dynamo PR CI passed — run 41941643671 (attempt 2) on 7e7acd1c3c

Gate checks: ✅ backend-status-check · ✅ deploy-status-check · ✅ dynamo-status-check

Posted automatically by Devin for run 41941643671. Updated on every full-CI run of this PR.

@yifjiang
yifjiang force-pushed the codex/kv-reuse-d11-bounded-lookups branch from 930f359 to 87510bb Compare November 4, 2026 08:16
@yifjiang
yifjiang force-pushed the codex/kv-reuse-d10-source-eligibility branch from 27d640b to 60e8ce6 Compare November 4, 2026 08:16
@yifjiang
yifjiang force-pushed the codex/kv-reuse-d11-bounded-lookups branch 2 times, most recently from 66e8c36 to fded18a Compare November 4, 2026 20:39
@copy-pr-bot

copy-pr-bot Bot commented Nov 4, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

yifjiang commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test fded18a

Signed-off-by: Yifan Jiang <19356972+yifjiang@users.noreply.github.com>
@yifjiang
yifjiang force-pushed the codex/kv-reuse-d11-bounded-lookups branch from fded18a to 7e7acd1 Compare November 5, 2026 00:24
@yifjiang

yifjiang commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 7e7acd1

@yifjiang yifjiang closed this Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant