test(serve): add aggregated_spec_decoding launch scenario

[nv-tusharma]

Implements the OPS-7567 launch scenario as part of OPS-7562 (Automate Remaining P0 Launch Scenarios).

Adds an aggregated_spec_decoding serve config (agg_spec_decoding.sh, Meta-Llama-3.1-8B + EAGLE3 draft, gpu_1).

Profiling (measured, not a placeholder): the base model is HF-gated, so it was profiled against the non-gated, weight-identical NousResearch/Meta-Llama-3.1-8B-Instruct mirror. Measured peak with a 1 GiB KV cap is ~18.6 GiB (8B weights + EAGLE3 draft + capped KV); profiled_vram_gib(20.0) is a small safe ceiling over that measurement. The --kv-cache-memory-bytes cap (via build_vllm_gpu_mem_args + requested_vllm_kv_cache_bytes(1 GiB)) makes the footprint GPU-size-independent, so it runs in the VRAM-aware parallel gpu_1 stage. Also predownloads the EAGLE3 draft (CI workers run HF_HUB_OFFLINE=True, and only the base model is auto-registered).

Marked nightly (heavy e2e launch scenario). Validated green in the parallel stage — aggregated_spec_decoding-2 PASSED (120s): https://github.com/ai-dynamo/dynamo/actions/runs/29369837585/job/87211038649

Linear: https://linear.app/nvidia/issue/OPS-7567

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Added a speculative decoding test configuration using Llama-3.1-8B and an EAGLE3 draft model.
  • Added configurable GPU memory limits and support for passing additional worker options.
  • Added configurable system port handling for speculative decoding workers.

• Tests

  • Added nightly and pre-merge coverage for speculative decoding, including model download requirements and representative chat requests.

[datadog-official]

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 40.33% (+6.88%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: e5001d0 | Docs | Give us feedback!}

[devin-ai-integration]

Devin Review found 1 potential issue.

[coderabbitai]

Walkthrough

Changes

Aggregated speculative decoding

Layer / File(s)	Summary
Launch argument and resource wiring examples/backends/vllm/launch/agg_spec_decoding.sh	The script derives KV-cache memory arguments, supports a configurable system port, and forwards extra arguments to dynamo.vllm.
Serve test registration tests/serve/test_vllm.py	Adds a GPU-1 nightly speculative decoding configuration using Llama-3.1-8B, EAGLE3, bounded KV cache, a 900-second timeout, and constrained requests.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly and concisely matches the main change: adding the aggregated_spec_decoding serve scenario.
Description check	✅ Passed	The description is detailed and covers the change well, with only minor template deviations like a missing reviewer-start section and nonstandard issue formatting.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

examples/backends/vllm/launch/agg_spec_decoding.sh (1)

14-14: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Make the generated GPU-memory arguments authoritative.

build_vllm_gpu_mem_args already emits --gpu-memory-utilization 0.01 with the KV-cache cap, while Line 40 always passes --gpu-memory-utilization 0.8 first. Avoid relying on vLLM’s duplicate-option behavior: use the default only when the helper returns no arguments, and pass the resulting options as an argv array. This also avoids the unquoted expansion flagged by ShellCheck; simply quoting $GPU_MEM_ARGS would incorrectly collapse multiple options into one argument.

Suggested fix

-GPU_MEM_ARGS=$(build_vllm_gpu_mem_args)
+GPU_MEM_ARGS_TEXT="$(build_vllm_gpu_mem_args)"
+GPU_MEM_ARGS=()
+if [[ -n "$GPU_MEM_ARGS_TEXT" ]]; then
+    read -r -a GPU_MEM_ARGS <<< "$GPU_MEM_ARGS_TEXT"
+else
+    GPU_MEM_ARGS=(--gpu-memory-utilization 0.8)
+fi

...
-    --gpu-memory-utilization 0.8 $GPU_MEM_ARGS "$@" &
+    "${GPU_MEM_ARGS[@]}" "$@" &

Also applies to: 40-40

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/backends/vllm/launch/agg_spec_decoding.sh` at line 14, Update the
GPU memory option setup around GPU_MEM_ARGS and the vLLM launch command so
build_vllm_gpu_mem_args remains authoritative: use the 0.8 default only when it
returns no arguments, store the resulting options in an argv array, and expand
that array safely without an unconditional earlier --gpu-memory-utilization
value or unquoted string expansion.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/serve/test_vllm.py`:
- Around line 163-166: Update the scheduling comment for the Llama-3.1-8B
speculative-decoding scenario near the test configuration to state that it runs
in nightly CI. Remove the stale instruction about temporarily using pre_merge
and switching to nightly, while preserving the HF token and VRAM availability
notes.

---

Nitpick comments:
In `@examples/backends/vllm/launch/agg_spec_decoding.sh`:
- Line 14: Update the GPU memory option setup around GPU_MEM_ARGS and the vLLM
launch command so build_vllm_gpu_mem_args remains authoritative: use the 0.8
default only when it returns no arguments, store the resulting options in an
argv array, and expand that array safely without an unconditional earlier
--gpu-memory-utilization value or unquoted string expansion.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d9ad92de-5c13-4242-b42c-2ad9ebed3f56

📥 Commits

Reviewing files that changed from the base of the PR and between c6eeb95 and 947f2c4.

📒 Files selected for processing (2)

• examples/backends/vllm/launch/agg_spec_decoding.sh
• tests/serve/test_vllm.py