feat(vlm): default to Cosmos3 Nano Reasoner - #315
Conversation
b1e3811 to
daccf77
Compare
|
Review outcome: changes required before merge. Finding:
CI on the current PR head is green (all reported checks passed), but it ran against the old base and does not clear the current merge conflict. |
daccf77 to
934f886
Compare
Signed-off-by: wenxind <wenxind@nvidia.com>
934f886 to
48d0754
Compare
Signed-off-by: Wenxin Ding <wenxind@nvidia.com>
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| flush=True, | ||
| ) | ||
| proc = subprocess.Popen(argv, start_new_session=True) | ||
| existing = False |
| ) | ||
| proc = subprocess.Popen(argv, start_new_session=True) | ||
| existing = False | ||
| running = False |
| if not remove_container(container_name): | ||
| log.error("Unable to remove stale container %s", container_name) | ||
| sys.exit(1) | ||
| existing = False |
| log.error("Unable to remove stale container %s", container_name) | ||
| sys.exit(1) | ||
| existing = False | ||
| running = False |
yanziz-nvidia
left a comment
yanziz-nvidia
left a comment
There was a problem hiding this comment.
Reviewed by yanziz-reviewer-bot
Summary
Adds a cosmos3_nano_reasoner client preset and makes it the default across local and hosted VLM profiles, bumps vlm-server's vLLM floor to >=0.23.0, moves shipped YAML configs to vllm_image: 26.07-py3 + async_scheduling: true, and adds a fingerprint-based lifecycle manager that replaces stale persistent runtimes on config change. CI green (CodeQL, DCO, lint, lock-check, SPDX, pytest 3.11/3.12, docs); DEPENDENCIES.md, changelog, and docs updated.
Legend: 🚫 Blocker · 💡 Suggestion · 🔍 Nit
| Finding | |
|---|---|
| 🚫 | None |
| 💡 | tests/test_vllm_docker.py:344 — test_pid_on_port_returns_none_when_tools_missing patches xr_ai_vllm._docker.subprocess.check_output, but pid_on_port is imported from xr_ai_vllm._process (line 26). The mock never intercepts, so the assert passes on a real check_output against a quiet port rather than on the tool-missing fallback. |
| 🔍 | None |
Actionables (for bots — copy-paste-ready for AI)
Fix if it makes sense in context — these are agent-generated suggestions, not human-vetted obligations. Skip anything that's wrong, already addressed, or not worth the churn.
tests/test_vllm_docker.py:344— Retarget the patch to"xr_ai_vllm._process.subprocess.check_output"so theFileNotFoundErrorside effect reachespid_on_portin its current module.
3 participants
Summary
cosmos3_nano_reasonerclient preset while retainingcosmos_vlmunchanged as the Cosmos-Reason1 compatibility optionnvidia/Cosmos3-Nano, NVIDIA's standard vLLM Reasoner path, with--async-scheduling, vLLM >= 0.23, and the 26.07 NGC vLLM imagenvidia/cosmos3-nano-reasonerReasoner, not Generator
nvidia/Cosmos3-Nanois NVIDIA's unified local checkpoint name. NVIDIA documents that standardvllm serve nvidia/Cosmos3-Nano --async-schedulingloads only the text-output Reasoner. The separate Generator requires vLLM-Omni plus both--omniand--model-class-name Cosmos3OmniDiffusersPipeline; xr-ai does not enable either flag.A CPU regression test executes the shipped VLM wrapper config and asserts that the Reasoner command includes
--async-schedulingwhile excluding--omni,--model-class-name, andCosmos3OmniDiffusersPipeline. It also checks all local profiles and both hosted-NIM profiles use the intended Reasoner identifiers.References:
Compatibility
Cosmos-Reason1 remains available: configure
model: nvidia/Cosmos-Reason1-7Bon vlm-server and selectpreset:cosmos_vlminmodels.yaml. Its existingenable_thinking=falsewire behavior is unchanged.Validation
uv lock --directory ai-services/vlm-servergit diff --checkThe GPU smoke test now targets Cosmos3 Nano Reasoner and the 26.07 image, but was not run locally because it requires the model/GPU runtime.