Repository navigation
feat(decisions): qualify causal vLLM scoring runtimes - #17132
biswapanda wants to merge 1 commit into
Conversation
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
❌ Dynamo PR CI failed — run 39853608393 (attempt 1) on
|
| Other | Jobs |
|---|---|
| dynamo-runtime | ❌ 1 ✅ 6 |
Failure details
1 job failed: mypy reports 3 type errors in the new vLLM decision modules, all from an Optional[str] (str | None) value used where a str is required.
❌ dynamo-runtime / mypy: 3 errors in vllm/decision_heads.py, vllm/decision_execution.py
Job: dynamo-runtime / mypy · Failed step: Run mypy (components) · Logs: gh run view --job 120094591595 -R ai-dynamo/dynamo --log-failed
components/src/dynamo/vllm/decision_heads.py:124: error: Unsupported operand types for in ("str | None" and "Generator[str, None, None]") [operator]
components/src/dynamo/vllm/decision_heads.py:148: error: Argument 1 to "Path" has incompatible type "str | None"; expected "str | PathLike[str]" [arg-type]
components/src/dynamo/vllm/decision_execution.py:166: error: Argument 1 to "Path" has incompatible type "str | None"; expected "str | PathLike[str]" [arg-type]
Found 3 errors in 2 files (checked 967 source files)
mypy fails on 3 errors in decision_heads.py (lines 124, 148) and decision_execution.py (line 166). In each one, a value typed str | None is used in an in check against a generator or passed to Path(). Narrowing the value (a None check or assert) before use, or tightening its type, should clear them.
For agents
{"pr": 17132, "run_id": 39853608393, "run_attempt": 1, "head_sha": "33484d269e221608ef78f0944df5e32b33bf82c7", "failures": [{"job": "dynamo-runtime / mypy", "job_id": 120094591595, "failed_step": "Run mypy (components)", "signature": "mypy: 3 errors in components/src/dynamo/vllm/decision_heads.py, decision_execution.py (str | None)", "tests": [], "log_cmd": "gh run view --job 120094591595 -R ai-dynamo/dynamo --log-failed"}]}Posted automatically by Devin for run 39853608393. Updated on every full-CI run of this PR.
Summary
Part of #15961; architecture #15843 and executor DEP #15967, with causal execution context #15274. Slice V3 of V8, stacked on its immediate parent.
Validate supported vLLM causal runtimes, effective configuration and tokenizer labels before publishing a capability; include deterministic CPU tests.
This parent-relative diff is +813 / -0 = 813 changed lines, including tests and intermediate scaffolding. The complete eight-slice stack preserves the original #17122 source tree exactly. Narrow staging annotations and capability guards are removed in final activation.
Compatibility and dependencies
Public contracts remain OpenAI
POST /v1/decisionsand JevPOST /v1/systemone. Nonvext.format, SGLang-native Decisions wire schema, HTTP proxy, or new public route is introduced. Preserve typed values, question order, accounting, revision fencing, placement, admission, deadlines, and cancellation.The existing core train through #15857 remains the base. Separately owned #16757 is the production-image upgrade prerequisite. This stack changes no container, dependency pin, Cargo manifest, or lockfile; frontend crates remain published
dynamo-decisions =0.1.0anddynamo-protocols =8.2.0.Specialized adapters target pinned upstream
9da5f2d3103df460dfca53df4ab4884def5c935f/0.32.1rc1.dev172+g9da5f2d31, not an implied upgrade of the released 0.32.0 causal profile. Worker initialization does not imply release qualification or capability publication.Validation
157 combined causal/head CPU helper tests passed; no skips or CUDA/engine construction attempts. Causal capability publication is wired in the next slice.
Applicable repository-pinned pre-commit hooks and whitespace checks passed. All new commits are signed and DCO-signed-off. Historical GPU campaigns predate the final source and are not evidence for these heads. Live-model, released-image, distributed-lifecycle, and CI release gates remain open in #17122. No GPU campaign or benchmark was launched for this split.
Stack navigation
Merge in the order below; each PR targets its immediate parent. V4 enables causal vLLM execution; V6/V7 initialize native workers without advertising them, and V8 enables native request dispatch and capability publication together. The stack begins at #15857, not main.