Use mtp3 for 1k8k, 1k1k gb200 fp8 disagg low latency configs

[trevor-m]

Summary by CodeRabbit

• Updates
  • Optimized GB200 FP8 low-latency recipe configurations (1K1K and 1K8K deployment profiles) with enhanced speculative execution parameters and refined resource allocation settings to improve performance and throughput characteristics.

[coderabbitai]

Caution

Review failed

The pull request is closed.

📝 Walkthrough

Walkthrough

Two GB200 FP8 low-latency MTP recipe configurations are updated with renamed versions, increased speculative inference parameters (num-steps and num-draft-tokens), and adjusted memory allocation settings in the 1k1k variant.

Changes

Cohort / File(s)	Summary
GB200 FP8 Low-Latency MTP Recipes recipes/gb200-fp8/1k1k/low-latency-mtp.yaml, recipes/gb200-fp8/1k8k/low-latency-mtp.yaml	Recipe versions renamed to v3; speculative-num-steps increased from 2 to 3; speculative-num-draft-tokens increased from 3 to 4 in both prefill and decode sections. Additionally, 1k1k recipe adjusts mem-fraction-static to 0.95 and max-running-requests to 128.

Estimated Code Review Effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly Related PRs

• Add low-latency FP8 disagg with MTP for GB200 #116: Modifies the same GB200 FP8 recipe files and adjusts matching speculative inference and memory configuration parameters.
• Update MTP recipe with multiple draft steps #78: Updates MTP recipe speculative parameters (speculative-num-steps and speculative-num-draft-tokens) in similar prefill/decode configurations.

Suggested Reviewers

• ishandhanani
• kyleliang-nv

Poem

🐰 Three hops for speculative dreams,
Tokens draft in bountiful streams,
Memory fractions tuned just right,
Recipes gleaming, polished bright! ✨

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch trevor-m/mtp3

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}