configs for gb300-fp8-no-mtp

[gracehonv]

Summary by CodeRabbit

• New Features

  • Added GB300 FP8 deployment profiles for low-latency, mid-throughput, and max-throughput modes covering 1k/8k sequence variants.
  • Included performance/benchmark presets and CUDA-graph tuning options for prefill and decode phases.

• Chores

  • Added comprehensive environment and resource configuration sets to support FP8 inference on GB300 hardware.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

📝 Walkthrough

Walkthrough

Six new YAML deployment profiles were added for FP8 inference on GB300 hardware, covering low-latency, mid, and max-throughput targets for two model sizes (1k1k and 8k1k). Each file encodes resources, environment hooks, sglang_config, and benchmarking parameters.

Changes

Cohort / File(s)	Summary
1k1k FP8 Deployment Profiles recipies/gb300-fp8/1k1k/stp/low-latency.yaml, recipies/gb300-fp8/1k1k/stp/max.yaml, recipies/gb300-fp8/1k1k/stp/mid.yaml	Added three new configs for 1k1k FP8: resource allocation (prefill/decode nodes, workers, GPUs), backend env hooks, detailed sglang_config for prefill/decode (model, TP/DP/EP, kv-cache dtype, attention backend, disaggregation, DeepEP/MoE), CUDA-graph and benchmark settings.
8k1k FP8 Deployment Profiles recipies/gb300-fp8/8k1k/stp/low-latency.yaml, recipies/gb300-fp8/8k1k/stp/max.yaml, recipies/gb300-fp8/8k1k/stp/mid.yaml	Added three new configs for 8k1k FP8: same structure as 1k1k set but tuned for larger model (resource counts, parallelism, memory/disaggregation, DeepEP/MoE options), plus CUDA-graph and benchmark parameters.

Sequence Diagram(s)

(Skipped — changes are configuration files without new multi-component control flow requiring sequence visualization.)

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Suggested reviewers

• ishandhanani

Poem

🐰 I hopped through YAML rows and leaves,
Tucked FP8 dreams beneath GB300 eaves.
Low-latency whispers, throughput sings—
Six new paths for model wings. 🥕✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Title check	❓ Inconclusive	The PR title 'configs for gb300-fp8-no-mtp' is vague and generic, using non-descriptive phrasing that doesn't clearly convey the specific changes made.	Consider using a more descriptive title that clearly indicates the main change, such as 'Add GB300 FP8 configuration files for low-latency, mid, and max throughput profiles' to better reflect the actual additions.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 5

🤖 Fix all issues with AI agents

In `@recipies/gb300-fp8/1k1k/stp/max.yaml`:
- Around line 1-3: The comment header at the top of the recipe incorrectly reads
"GB200"; update that comment to reference "GB300" so it matches the
configuration name "gb300-1k1k-fp8-max" (keep the rest of the header text
unchanged) — locate the top-of-file comment and change "GB200" to "GB300".

In `@recipies/gb300-fp8/1k1k/stp/mid.yaml`:
- Around line 1-2: Update the top comment to match the actual config: replace
the incorrect header "GB200 FP8 Max Throughput Configuration" with a correct
description reflecting GB300 and mid-tier throughput (for example "GB300 FP8 Mid
Throughput Configuration"), ensuring it aligns with the config name field name:
"gb300-1k1k-fp8-mid".

In `@recipies/gb300-fp8/8k1k/stp/low-latency.yaml`:
- Around line 118-123: The benchmark block has isl set to 8102 (benchmark: isl:
8102) which likely is a typo compared to other 8k1k configs that use 8192;
verify the intended target and if it's a mistake, update the isl value to 8192
in the same benchmark section (and ensure req_rate, osl and concurrencies remain
unchanged), and run any config validation or tests to confirm the change; if
8102 was intentional, add an inline comment explaining why it differs from the
8192 baseline.

In `@recipies/gb300-fp8/8k1k/stp/max.yaml`:
- Around line 1-3: The top-line comment is inconsistent with the configuration
name/gpu_type: update the header comment "# GB200 FP8 Max Throughput
Configuration" to reflect "GB300" so it matches the configuration name
"gb300-8k1k-fp8-max" (or alternatively rename the configuration/name/gpu_type to
GB200 if that was intended); ensure the header text and the symbol name
"gb300-8k1k-fp8-max" (and any gpu_type field) are consistent.

In `@recipies/gb300-fp8/8k1k/stp/mid.yaml`:
- Around line 1-3: The comment header is incorrect: replace "GB200 FP8 Max
Throughput Configuration" with a header that matches this recipe (GB300,
mid-throughput). Update the top-of-file comment to something like "GB300 FP8 Mid
Throughput Configuration" so it aligns with the name field "gb300-8k1k-fp8-mid"
and the file's intended throughput tier.

🧹 Nitpick comments (1)

recipies/gb300-fp8/8k1k/stp/low-latency.yaml (1)

61-116: Configuration uses different parameter naming style.

This file uses tensor-parallel-size, data-parallel-size, expert-parallel-size while other configurations in this PR use the shorter tp-size, dp-size, ep-size style. Both likely work, but consider standardizing for maintainability across configurations.