feat: add experimental DeepSeek-V4-Flash + V4-Pro vLLM agg recipes

[biswapanda]

Overview

Adds two experimental aggregated vLLM serving recipes for the DeepSeek-V4 series, sharing a single dsv4 reference container:

• recipes/deepseek-v4-flash/ — DeepSeek-V4-Flash (284B / 13B active), 4x B200, DP=4 + EP.
• recipes/deepseek-v4-pro/ — DeepSeek-V4-Pro (1.6T / 49B active, 1M context), 8x B200, TP=8 + EP.

Both follow the existing recipe layout (recipes/qwen3-235b-a22b-fp8, recipes/llama-3-70b/vllm/agg) and are listed under Experimental Recipes in recipes/README.md — functional, not yet performance-tuned, not benchmarked, and depend on a custom container build.

Smoke-tested on a 4x B200 node (Flash, namespace bis-dsv4) and an 8x B200 node (Pro). Reasoning (reasoning_content) and tool calling (tool_calls) verified end-to-end via /v1/chat/completions against both.

Recipe layout (per recipe)

Each recipe directory contains:

• container/Dockerfile.dsv4 — reference container build that overlays the Dynamo runtime (wheels, nats, etcd, NIXL, UCX, dynamo.vllm worker) on top of vllm/vllm-openai:deepseekv4-cu130. The two Dockerfiles are duplicates so each recipe is self-contained; the produced image is identical and is reusable across both recipes (the model is selected at runtime via --model).

  docker build -f recipes/deepseek-v4-pro/container/Dockerfile.dsv4 \
    -t <your-registry>/vllm-dsv4:<tag> .
  

• container/README.md — build flow, build args (DYNAMO_SRC_IMAGE, DSV4_BASE_IMAGE), troubleshooting.
• model-cache/model-cache.yaml — ReadWriteMany PVC. 700Gi for Flash, 1500Gi for Pro.
• model-cache/model-download.yaml — HuggingFace download Job, reads hf-token-secret.
• vllm/agg/vllm-dgd.yaml — DynamoGraphDeployment.
• README.md — prerequisites, Quick Start, recipe details with flag table, reasoning + tool calling verification curls, notes.

DGD shapes

	Flash	Pro
Parallelism	--tensor-parallel-size 1 --data-parallel-size 4 --enable-expert-parallel	--tensor-parallel-size 8 --enable-expert-parallel
GPUs	4x B200 (single node, 4 of 8)	8x B200 (fills one node)
KV cache	--kv-cache-dtype fp8 --block-size 256	same
Indexer	--attention-config '{"use_fp4_indexer_cache":true}'	same
Cudagraphs	'{"cudagraph_mode":"FULL_AND_PIECEWISE","custom_ops":["all"]}'	'{"mode":0,"cudagraph_mode":"FULL_DECODE_ONLY"}' (more conservative for the 1.6T model; matches the upstream V4-Pro example)
Tokenizer / parsers	--tokenizer-mode deepseek_v4, --dyn-reasoning-parser deepseek_v4, --dyn-tool-call-parser deepseek_v4	same
Startup probe	60 min (failureThreshold: 360)	90 min (failureThreshold: 540) — sized for the larger TP=8 weight load
VLLM_ENGINE_READY_TIMEOUT_S	3600	5400

Both worker pods run with HF_HUB_OFFLINE=1 (weights served from the PVC, never contacting the HF Hub), and the manifests ship with the placeholder nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag so users build the real image from their recipe's container/Dockerfile.dsv4.

Why TP=8 for Pro vs DP=4 for Flash?

Pro is ~5.5x larger than Flash on disk (~865 GB vs ~160 GB). With FP4+FP8 mixed weights, Pro does not fit in 4 ranks at typical batch shapes, so the upstream tested shape is TP=8 across all 8 GPUs of one node. Expert Parallel is still enabled on top of TP — TP shards the dense (attention/router/norm) weights, EP shards the experts.

Pro-specific reasoning modes

Per the model card, V4-Pro exposes three reasoning effort levels. The recipe surfaces them via chat_template_kwargs:

• Non-think — {} — fast intuitive responses
• Think High — {"thinking":true,"reasoning_effort":"high"} — explicit chain-of-thought
• Think Max — {"thinking":true,"reasoning_effort":"max"} — maximum reasoning depth (model card recommends --max-model-len >= 393216)

Index update

recipes/README.md adds DeepSeek-V4-Flash and DeepSeek-V4-Pro rows under Experimental Recipes, both linking to their container/Dockerfile.dsv4.

Where should the reviewer start?

• recipes/deepseek-v4-pro/README.md and recipes/deepseek-v4-flash/README.md — user-facing flow.
• recipes/deepseek-v4-pro/vllm/agg/vllm-dgd.yaml — the larger, more conservative DGD; flag rationale in the README.
• recipes/deepseek-v4-pro/container/Dockerfile.dsv4 (identical to Flash's) — reference image build.

Status / caveats

• Experimental. Not performance-tuned, no perf.yaml, no benchmark numbers.
• Requires custom container build. DeepSeek-V4 is not in a stock vLLM release yet; users build from recipes/<recipe>/container/Dockerfile.dsv4.
• Slow first launch. Up to ~60 min (Flash) / ~90 min (Pro) for weight load + FlashInfer autotune + cudagraph warmup; startup probes are sized accordingly.
• Scope. Aggregated single-node only. No disagg/P-D variant yet. No multi-node Pro variant.

Testing

Deployed both recipes via kubectl apply -f vllm/agg/vllm-dgd.yaml:

• kubectl get dynamographdeployment → state: successful, All resources are ready (both).
• GET /v1/models returns the expected model id.
• POST /v1/chat/completions with a reasoning prompt → message.reasoning_content populated, clean message.content.
• POST /v1/chat/completions with tool definitions → structured message.tool_calls, finish_reason: "tool_calls".
• For Pro: verified all three reasoning effort modes via chat_template_kwargs.

Related

• Depends on the DeepSeek-V4 parser PR (reasoning + tool-call support) — the --dyn-reasoning-parser deepseek_v4 and --dyn-tool-call-parser deepseek_v4 flags require that change.
• Upstream vLLM dsv4 image: vllm-project/vllm#40760.

[dmitry-tokarev-nv]

/ok to test ed842e1

[biswapanda]

/ok to test 98d8494