[Cookbook] Add Laguna-S-2.1 by Jiminator · Pull Request #31918 · sgl-project/sglang

Jiminator · 2026-07-21T09:02:37Z

Summary

Test plan

mint validate passes in docs_new/
Cells render correct commands; URL-hash nav persists across reload
NEW badge shows on Laguna-S-2.1 only (stripped from XS-2.1 and M.1)
Poolside homepage card links to Laguna-S-2.1
Fill verified: true + benchmark numbers after measurement

🤖 Generated with Claude Code

CI States

Latest PR Test (Base): ✅ Run #29850723346
Latest PR Test (Extra): ❌ Run #29850721918

118B hybrid-SWA MoE (8B active, 1M context) — config-driven page with H200 / B300 / GB300 × BF16 / FP8 / NVFP4 / INT4 × low-latency (DFlash) / high-throughput. All cells marked unverified; benchmarks pending. Strips NEW from Laguna-XS-2.1 and Laguna-M.1 siblings. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- Mark all 6 H200 cells verified (sglang 0.5.15.post1, 2026-07-20) - Add --sampling-defaults openai to all cells (generation_config top_k=0 causes HTTP 400 on non-greedy requests without it) - Add --mem-fraction-static 0.80 to H200 BF16 high-throughput (default/ 0.88 OOM at cuda-graph capture / NCCL alloc at high concurrency) - Confirm SGLANG_SHARED_EXPERT_TP1=1 for FP8 only (INT4 shared expert stays bf16, no flag needed) - Fill H200 accuracy: GSM8K 93.18–95.00%; AIME25 FP8 67.29% / INT4 67.71% (BF16 null pending higher max_tokens re-run due to truncation) - Document FP8 DFlash draft rope_theta issue and BF16 reasoning-length caveat in Configuration Tips Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- Mark GB300 BF16/FP8/INT4 cells verified (sglang dev build, 2026-07-20) - Add SGLANG_SHARED_EXPERT_TP1=1 to GB300 FP8 cells (confirmed required at TP=4 on S-2.1, unlike XS-2.1 where TP=4 works without it) - Fill GB300 accuracy: GSM8K 93.3–95.2% across BF16/FP8/INT4 - GB300 AIME25 left null (sgl-eval --thinking sends wrong key for Laguna) - NVFP4 on GB300: verified:false — degenerate output (repetition loop), both LL and HT; documented in config header and Configuration Tips - Fix DFlash draft rope_theta note to cover NVFP4 draft as well Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Measured on 8xB300 TP=8, sglang 0.5.15.post1 (v0.5.15.post1-cu130 image), sgl-eval gsm8k full 1319 questions, greedy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…4 benchmark stubs Remove all header paragraphs containing benchmark run conclusions and TP-sizing rationale from config and benchmarks files. Drop per-cell GSM8K inline notes (numbers live in the benchmarks data). Remove GB300 NVFP4 stub entries from benchmarks; config cells remain verified:false. Trim MDX BF16 memory and NVFP4 tips to drop internal debugging details. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Removed outdated comments regarding sampling defaults and FP8/NVFP4 DFlash drafts.

B300 GSM8K benchmarks were added in a prior commit; update the corresponding config cells from verified:false to verified:true. NVFP4 cells remain unverified. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add config.defaultSelection support to _deployment.jsx (non-breaking; other configs without the field continue to use cells[0]). Set the default for Laguna-S-2.1 to B300 FP8 low-latency. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…s[0] Move B300 FP8 low-latency cell to cells[0] so the deployment panel opens on it by default. Revert _deployment.jsx to its original form. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Measured on 4xGB300 TP=4, stock sglang 0.5.15, sgl-eval gsm8k full 1319 questions, greedy. Supersedes the earlier dev-build degenerate-quality observation, which does not reproduce on 0.5.15 with identical weights. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

gemini-code-assist

Code Review

This pull request adds deployment documentation and configurations for poolside's Laguna-S-2.1 model, including a new MDX cookbook, benchmark results, and SGLang deployment cells. Feedback points out a contradiction in the verification status of NVFP4 on B300 and GB300 across the configuration and benchmark files, and a discrepancy in the recommended Docker image tags (dev vs. latest) between the installation guide and the configuration.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

gemini-code-assist · 2026-07-21T09:04:11Z

+      match: { hw: "b300", variant: "default", quant: "nvfp4", strategy: "high-throughput", nodes: "single" },
+      verified: false,
+      env: [],


There is a contradiction regarding the verification status of NVFP4 on B300 and GB300:

In laguna-s21.jsx (lines 307, 321, 443, 457), these cells are marked as verified: false, and a comment on line 373 states: // NVFP4 cells: unverified — degenerate output observed on GB300.

In Laguna-S-2.1.mdx (line 142), it states: NVFP4 is not yet verified on GB300 or B300; pending resolution upstream.

However, in laguna-s21-benchmarks.jsx (lines 71-81 and 124-134), these same cells are marked as verified: true and have high accuracy numbers (e.g., GSM8K 94.54% - 95.30%).

Please align these files to consistently reflect the actual verification status of NVFP4 on B300 and GB300.

Resolved: NVFP4 is now verified on both B300 and GB300 with full GSM8K runs on stock sglang 0.5.15 (B300 94.54/95.30, GB300 94.69/94.16); verified flags, cell comments, and prose are aligned across all three files.

gemini-code-assist · 2026-07-21T09:04:11Z

+```bash Command
+docker pull lmsysorg/sglang:dev
+```


The manual installation step specifies pulling lmsysorg/sglang:dev, but the configuration in laguna-s21.jsx specifies lmsysorg/sglang:latest for H200 hardware. To avoid version mismatch or unnecessary image downloads for H200 users, consider aligning the documentation or the configuration.

Resolved: the manual install step now pulls lmsysorg/sglang:latest, matching the H200 config.

Benchmarks file already has verified:true + real GSM8K numbers for all four NVFP4 cells. Sync config and remove the stale "not yet verified" prose from the MDX Configuration Tips. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…25_pct to null in benchmarks

Add bench_serving results (random ISL=8192/OSL=1024, sglang 0.5.15.post1, 8×H200 TP=8). Low-latency cells: conc=1 and conc=16. High-throughput cells: conc=1024. Also restore FP8/INT4 AIME25 accuracy values lost in a prior rebase. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

gemini-code-assist · 2026-07-21T16:47:54Z

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

30 problems x 16 repeats, temp 1.0, top_p 0.95, 128 threads, thinking via enable_thinking-patched sgl-eval. FP8 66.88, INT4 68.33, NVFP4 66.46; BF16 44.58 at max_tokens 131072 (7.7% samples still truncate). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The draft repos now ship flat rope_theta configs; the local-patch instruction only applies to checkpoints cached before the fix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-authored-by: Zijie Xia <zijie.xia@radixark.ai> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>

zijiexia and others added 11 commits July 20, 2026 17:40

Add B300 GSM8K benchmark values for Laguna-S-2.1

367d60e

Measured on 8xB300 TP=8, sglang 0.5.15.post1 (v0.5.15.post1-cu130 image), sgl-eval gsm8k full 1319 questions, greedy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Clean up comments in laguna-s21.jsx

93dd748

Removed outdated comments regarding sampling defaults and FP8/NVFP4 DFlash drafts.

[Cookbook] Laguna-S-2.1: mark B300 BF16/FP8/INT4 cells as verified

f9352b4

B300 GSM8K benchmarks were added in a prior commit; update the corresponding config cells from verified:false to verified:true. NVFP4 cells remain unverified. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

[Cookbook] Laguna-S-2.1: set default to B300 FP8 low-latency via cell…

e6f84bf

…s[0] Move B300 FP8 low-latency cell to cells[0] so the deployment panel opens on it by default. Revert _deployment.jsx to its original form. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

[Cookbook] Laguna-S-2.1: drop --sampling-defaults openai from all cells

8fb7e99

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

github-actions Bot added the documentation Improvements or additions to documentation label Jul 21, 2026

Jiminator changed the title ~~[Cookbook] Add poolside/Laguna-S-2.1~~ [Cookbook] Add _ Jul 21, 2026

gemini-code-assist Bot reviewed Jul 21, 2026

View reviewed changes

mintlify Bot deployed to staging - docs_new July 21, 2026 09:33

View deployment

[Cookbook] Laguna-S-2.1: update Docker pull command to use latest tag

abf2156

mintlify Bot deployed to staging - docs_new July 21, 2026 09:38

View deployment

[Cookbook] Laguna-S-2.1: update sglang_version to 0.5.15 and set aime…

d0df906

…25_pct to null in benchmarks

mintlify Bot deployed to staging - docs_new July 21, 2026 09:42

View deployment

mintlify Bot deployed to staging - docs_new July 21, 2026 09:47

View deployment

remove wrong aime25

099b1ea

mintlify Bot deployed to staging - docs_new July 21, 2026 10:04

View deployment

Jiminator marked this pull request as ready for review July 21, 2026 16:47

Jiminator requested review from JustinTong0323, sogalin, wisclmy0611 and zijiexia as code owners July 21, 2026 16:47

mintlify Bot deployed to staging - docs_new July 21, 2026 16:50

View deployment

Jiminator and others added 3 commits July 21, 2026 16:50

Update DFlash rope_theta note: fixed upstream 2026-07-21

bbbc3c0

The draft repos now ship flat rope_theta configs; the local-patch instruction only applies to checkpoints cached before the fix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

update gb300 metrics

e31520a

update h200

5e79e1a

zijiexia changed the title ~~[Cookbook] Add _~~ [Cookbook] Add Laguna-S-2.1 Jul 21, 2026

mintlify Bot deployed to staging - docs_new July 21, 2026 16:55

View deployment

zijiexia approved these changes Jul 21, 2026

View reviewed changes

zijiexia enabled auto-merge (squash) July 21, 2026 17:00

zijiexia merged commit 9a6c960 into main Jul 21, 2026
86 of 90 checks passed

zijiexia deleted the cookbook-migrate-laguna-s21 branch July 21, 2026 17:01

This was referenced Jul 21, 2026

[CI Monitor] Daily Report - 2026-07-21 bingxche/sglang-ci-bot#142

Open

[CI Monitor] Daily Report - 2026-07-22 bingxche/sglang-ci-bot#143

Open

robbiemu mentioned this pull request Jul 22, 2026

[FlashInfer] Plan wrappers from layer-local attention metadata #32062

Open

4 of 5 tasks

iaroslav-reflection mentioned this pull request Jul 27, 2026

[Bug] poolside_v1 reasoning parser doesn't separate reasoning for Laguna-S-2.1 (thinking-on template default) #32536

Closed

Navigation Menu

Repository navigation

[Cookbook] Add Laguna-S-2.1 - #31918

[Cookbook] Add Laguna-S-2.1#31918
zijiexia merged 20 commits into
mainfrom
cookbook-migrate-laguna-s21

Jiminator commented Jul 21, 2026 •

edited by github-actions Bot

Loading

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot Jul 21, 2026

Uh oh!

Jiminator Jul 21, 2026

Uh oh!

gemini-code-assist Bot Jul 21, 2026

Uh oh!

Jiminator Jul 21, 2026

Uh oh!

gemini-code-assist Bot commented Jul 21, 2026

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Jiminator commented Jul 21, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

CI States

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

Jiminator Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

Jiminator Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot commented Jul 21, 2026

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Jiminator commented Jul 21, 2026 •

edited by github-actions Bot

Loading