feat(recipes): add Nemotron 3.5 Lightning RTX PRO 6000 aggregate DSpark recipe

[scheckerNV]

Overview

Adds agg-rtxpro6000-dspark — the first RTX PRO 6000 Blackwell (sm_120) variant in the
Nemotron 3.5 Lightning family. Existing variants cover H100, H200, B200, and GB200 only.

Derived from agg-h100-dspark. Six lines changed: node selector, DGD name, label, ConfigMap
name and reference, header comment. No engine flags touched.

NEW  recipes/nemotron-3.5-lightning/vllm/agg-rtxpro6000-dspark/deploy.yaml
MOD  recipes/nemotron-3.5-lightning/vllm/README.md    (+5)
MOD  recipes/nemotron-3.5-lightning/perf/README.md   (+14)

Results

Benchmarked on the family's shared trace
(64k_400_90kv_agent_new_noschedule_short_15perc.jsonl) at concurrency 20 with 3,541
requests — the same shape as the published H100 DSpark row.

Recipe	Conc	Valid	Err	Output tok/s/GPU	User tok/s p50	TTFT p50
agg-h100-dspark	20	3,411	130	1,878	93.9	239 ms
agg-b200-dspark-bf16	24	3,411	130	2,471	103.0	207 ms
agg-rtxpro6000-dspark	20	3,526	15	776	53.4	543 ms

~41% of H100 throughput, consistent with the GDDR7-vs-HBM3 bandwidth ratio for a
decode-bound workload.

Two notes on comparability, both also in perf/README.md:

• 15 errors rather than 130. This deployment resolves max_seq_len to 1048576, so the
  long trace requests that exceed a 262144-token context on the other SKUs completed here.
• Synthetic AL is n/m. The manifest inherits 3.69 from the H100 recipe; I did not
  re-derive it on sm_120. Affects synthetic benchmarking runs only, not serving.

Why NVFP4 rather than BF16

B200 and GB200 moved to BF16 in #13370. That result does not transfer to sm_120. Measured on
identical hardware, same trace, cold cache, only the checkpoint differing:

	NVFP4	BF16
Output tok/s	252	76
User tok/s p50	85	24
ITL p50	11.8 ms	41.2 ms
TTFT p50	11,227 ms	11,547 ms

TTFT is unchanged — prefill is compute-bound and precision-insensitive. Inter-token latency
is 3.5× worse under BF16 because decode is memory-bandwidth-bound and BF16 weights are 4×
larger. RTX PRO 6000 uses GDDR7 rather than HBM, so weight size dominates decode in a way it
does not on B200.

DSpark was also measured against DFlash on the same hardware: 1.86× throughput, better on
every metric.

Best-practices compliance

Checked against nim-turbo-recipe-best-practices (MR !7).

Passes: v1beta1, no internal cluster references, no taints/tolerations, no hardcoded
namespace, 2-space indentation, user/group 0, IPC_LOCK capabilities, frontend 1 replica,
imagePullPolicy: IfNotPresent, same image across frontend and workers, commented
synthetic-AL block present.

Deviations, all inherited from agg-h100-dspark and identical across the family: node
affinity for GPU type, explicit probes, imagePullSecrets, unpinned image digest, frontend
mounting model-cache, no DYN_ROUTER_MODE, sharedMemorySize: 40Gi vs 64Gi, no
ephemeral-disk or CPU requests, and the family's vllm-agg-<sku>-<specdec> naming.

I matched the siblings rather than diverge from them; happy to conform whenever the family
does.

One note for whoever owns the standard, not a blocker here: rule 4 (no GPU-type node affinity
in the base recipe) is currently the only thing distinguishing SKU variants in this family —
agg-h100-dspark and agg-h200-dspark differ by six lines, of which one is the node label.
Worth a decision at some point for SKU-differentiated families.

Not included: perf/perf.yaml (no sibling in this family ships one) and Fern docs (happy
to write one, would need the pattern).

Testing

Deployed and benchmarked on AWS g7e.8xlarge (1× RTX PRO 6000 Blackwell, 96 GB, 32 vCPU,
256 GiB), single-node Kubernetes, Dynamo Platform 1.3.0. Smoke-tested via
/v1/chat/completions, then the full shared-trace run above. Raw aiperf exports available.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

👋 Hi scheckerNV! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀