Skip to content

feat(decisions): qualify causal vLLM scoring runtimes - #17132

Draft
biswapanda wants to merge 1 commit into
bis/vllm-decisions-v2-head-executorfrom
bis/vllm-decisions-v3-causal-readiness
Draft

biswapanda wants to merge 1 commit into
bis/vllm-decisions-v2-head-executorfrom
bis/vllm-decisions-v3-causal-readiness

Conversation

@biswapanda

@biswapanda biswapanda commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Part of #15961; architecture #15843 and executor DEP #15967, with causal execution context #15274. Slice V3 of V8, stacked on its immediate parent.

Validate supported vLLM causal runtimes, effective configuration and tokenizer labels before publishing a capability; include deterministic CPU tests.

This parent-relative diff is +813 / -0 = 813 changed lines, including tests and intermediate scaffolding. The complete eight-slice stack preserves the original #17122 source tree exactly. Narrow staging annotations and capability guards are removed in final activation.

Compatibility and dependencies

Public contracts remain OpenAI POST /v1/decisions and Jev POST /v1/systemone. No nvext.format, SGLang-native Decisions wire schema, HTTP proxy, or new public route is introduced. Preserve typed values, question order, accounting, revision fencing, placement, admission, deadlines, and cancellation.

The existing core train through #15857 remains the base. Separately owned #16757 is the production-image upgrade prerequisite. This stack changes no container, dependency pin, Cargo manifest, or lockfile; frontend crates remain published dynamo-decisions =0.1.0 and dynamo-protocols =8.2.0.

Specialized adapters target pinned upstream 9da5f2d3103df460dfca53df4ab4884def5c935f / 0.32.1rc1.dev172+g9da5f2d31, not an implied upgrade of the released 0.32.0 causal profile. Worker initialization does not imply release qualification or capability publication.

Validation

157 combined causal/head CPU helper tests passed; no skips or CUDA/engine construction attempts. Causal capability publication is wired in the next slice.

Applicable repository-pinned pre-commit hooks and whitespace checks passed. All new commits are signed and DCO-signed-off. Historical GPU campaigns predate the final source and are not evidence for these heads. Live-model, released-image, distributed-lifecycle, and CI release gates remain open in #17122. No GPU campaign or benchmark was launched for this split.

Stack navigation

Merge in the order below; each PR targets its immediate parent. V4 enables causal vLLM execution; V6/V7 initialize native workers without advertising them, and V8 enables native request dispatch and capability publication together. The stack begins at #15857, not main.

Order PR Scope Additions Deletions Total
V1 #17130 Execution contracts +1,145 -3 1,148
V2 #17131 Native head adapter +1,356 -0 1,356
V3 #17132 Causal runtime readiness +813 -0 813
V4 #17133 Causal serving and discovery +745 -140 885
V5 #17134 Native read adapters +1,476 -0 1,476
V6 #17135 Head worker integration +814 -6 820
V7 #17136 Read worker integration +644 -4 648
V8 #17122 Native activation and HTTP SDK tests +1,307 -17 1,324
flowchart TB
  D7["#15857 decision core"] --> V1["#17130 contracts"]
  V1 --> V2["#17131 head adapter"]
  V2 --> V3["#17132 causal readiness"]
  V3 --> V4["#17133 causal serving"]
  V4 --> V5["#17134 read adapters"]
  V5 --> V6["#17135 head workers"]
  V6 --> V7["#17136 read workers"]
  V7 --> V8["#17122 native activation"]
Loading

Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor

❌ Dynamo PR CI failed — run 39853608393 (attempt 1) on 33484d269e

Gate checks: ✅ backend-status-check · ✅ deploy-status-check · ❌ dynamo-status-check

Other Jobs
dynamo-runtime ❌ 1 ✅ 6

Failure details

1 job failed: mypy reports 3 type errors in the new vLLM decision modules, all from an Optional[str] (str | None) value used where a str is required.

❌ dynamo-runtime / mypy: 3 errors in vllm/decision_heads.py, vllm/decision_execution.py

Job: dynamo-runtime / mypy · Failed step: Run mypy (components) · Logs: gh run view --job 120094591595 -R ai-dynamo/dynamo --log-failed

components/src/dynamo/vllm/decision_heads.py:124: error: Unsupported operand types for in ("str | None" and "Generator[str, None, None]")  [operator]
components/src/dynamo/vllm/decision_heads.py:148: error: Argument 1 to "Path" has incompatible type "str | None"; expected "str | PathLike[str]"  [arg-type]
components/src/dynamo/vllm/decision_execution.py:166: error: Argument 1 to "Path" has incompatible type "str | None"; expected "str | PathLike[str]"  [arg-type]
Found 3 errors in 2 files (checked 967 source files)

mypy fails on 3 errors in decision_heads.py (lines 124, 148) and decision_execution.py (line 166). In each one, a value typed str | None is used in an in check against a generator or passed to Path(). Narrowing the value (a None check or assert) before use, or tightening its type, should clear them.

For agents
{"pr": 17132, "run_id": 39853608393, "run_attempt": 1, "head_sha": "33484d269e221608ef78f0944df5e32b33bf82c7", "failures": [{"job": "dynamo-runtime / mypy", "job_id": 120094591595, "failed_step": "Run mypy (components)", "signature": "mypy: 3 errors in components/src/dynamo/vllm/decision_heads.py, decision_execution.py (str | None)", "tests": [], "log_cmd": "gh run view --job 120094591595 -R ai-dynamo/dynamo --log-failed"}]}

Posted automatically by Devin for run 39853608393. Updated on every full-CI run of this PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend::vllm Relates to the vllm backend feat size/XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant