fix(models): order dual-Ada startup - #328
Conversation
Signed-off-by: Devdeep Ray <devdeepr@Nvidia.com>
48eb125 to
b97cf4e
Compare
|
Review outcome: ready for review The branch is rebased on current
Validation:
No unresolved review threads remain. The PR is ready for maintainer review and merge. |
yanziz-nvidia
left a comment
There was a problem hiding this comment.
Reviewed by yanziz-reviewer-bot
Summary
Orders dual-Ada startup so VLM and embeddings (GPU 0) settle before STT and the 30B LLM (GPU 1), leaving the largest model as the final readiness gate. Single-GPU profiles (spark, 96G_blackwell) keep the existing 30B-first order. CI green (ruff, pytest 3.11 + 3.12, CodeQL, DCO, docs); DEPENDENCIES.md updated.
Legend: 🚫 Blocker · 💡 Suggestion · 🔍 Nit
| Finding | |
|---|---|
| 🚫 | None |
| 💡 | None |
| 🔍 | agent-samples/model-servers/main.py:11-16 — The "Servers started" table still lists stt → agent-llm → vlm → embedding, which now reflects only the single-GPU profiles. A one-line note on the dual-Ada order keeps the docstring authoritative. |
Actionables (for bots — copy-paste-ready for AI)
Fix if it makes sense in context — these are agent-generated suggestions, not human-vetted obligations. Skip anything that's wrong, already addressed, or not worth the churn.
agent-samples/model-servers/main.py:11— Annotate the "default / --vlm-llm-stack" table to note startup order varies by profile (dual-Ada: vlm → embedding → stt → agent-llm; single-GPU: stt → agent-llm → vlm → embedding), so the header stays accurate after this change.
yanziz-nvidia
left a comment
There was a problem hiding this comment.
resolve merge conflicts fist; LGTM otherwise
Summary
Why
The dual-Ada topology benefits from settling shared GPU 0 services before using the dedicated GPU 1 model as the final readiness gate. Single-GPU profiles still need the 30B model to compile against an empty device.
Validation
uv run --project tests pytest -q tests/test_model_servers.pyruff checkon changed Python files