Skip to content

fix: support DeepSeek-V4-Flash-0731 reasoning effort - #33145

Closed
ormandj wants to merge 1 commit into
sgl-project:mainfrom
ormandj:fix/dsv4-0731-reasoning-effort
Closed

ormandj wants to merge 1 commit into
sgl-project:mainfrom
ormandj:fix/dsv4-0731-reasoning-effort

Conversation

@ormandj

@ormandj ormandj commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Motivation

DeepSeek-V4-Flash-0731/encoding/encoding_dsv4.py defines three reasoning-effort tiers. SGLang currently preserves the earlier DeepSeek-V4 contract, where high adds no prefix and max adds the "Absolute maximum" prefix. Applying the newer mapping to every DeepSeek-V4 checkpoint would change existing behavior.

Checkpoint family low high max
Earlier DeepSeek-V4 filtered to the default no prefix "Absolute maximum" prefix
DeepSeek-V4-Flash-0731 no prefix "Absolute maximum" prefix "Beyond maximum" prefix

Modifications

  • Select the newer mapping with SGLang's existing checkpoint_bundles_dspark_draft() helper. The HF config carries no encoding-version field, so this uses the bundled dspark_* config keys as the proxy; any DeepSeek-V4 checkpoint carrying those keys selects the newer mapping.
  • Add the 0731 low, high, and max prompts from the bundled encoder.
  • Preserve the existing earlier-checkpoint mapping.
  • Add unit coverage for checkpoint detection, serving-layer filtering, all three newer tiers, invalid values, and legacy compatibility.

Accuracy Tests

  • uv run --no-project python -m pytest -q test/registered/unit/entrypoints/openai/test_serving_chat.py: 86 passed and 30 subtests passed.
  • On a TP2 server using DeepSeek-V4-Flash-0731, a one-message "Hello" request with max_tokens=8 produced prompt-token counts of 5, 84, and 97 for low, high, and max, respectively.

Speed Tests and Profiling

Not run. This changes which prompt prefix is emitted.

Checklist

AI was used to assist with implementation.


CI States

Latest PR Test (Base): ❌ Run #30674188412
Latest PR Test (Extra): ❌ Run #30674188407

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Jul 31, 2026
@ormandj

ormandj commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

@mmangkad @JustinTong0323 @b8zhong @AliceChenyy, could you review this alongside #33140? The functional distinction is checkpoint selection: this version uses the existing checkpoint_bundles_dspark_draft() config signal instead of a model-path substring. It also updates the SGLANG_DSV4_REASONING_EFFORT documentation and covers request filtering, invalid values, and legacy behavior.

@JustinTong0323

Copy link
Copy Markdown
Collaborator

Thanks @ormandj. The DeepSeek V4 reasoning-effort work from this PR has been merged into #33140, with co-author attribution preserved there. Closing this PR in favor of #33140.

Leoyzen added a commit to Leoyzen/sglang that referenced this pull request Aug 8, 2026
…project#32035 sgl-project#33656 sgl-project#32183 sgl-project#33145)

Applied PRs (latest from GitHub):
  sgl-project#33288  Indexer logits OOM fix
  sgl-project#30393  HiCache packed/sidecar draft caches
  sgl-project#31170  DPA prefix_affinity load balancing
  sgl-project#33795  DSpark compact ragged-verify CUDA graph JIT race
  sgl-project#32467  C128 plan-kernel warp barrier
  sgl-project#33865  DSpark x prefill CP unblock
  sgl-project#30371  SWA state pool sizing (storage page)
  sgl-project#33358  FlashMLA norm-rope K-tokens-per-block ILP
  sgl-project#33872  num_draft_tokens clamp + extend_len==0 skip (supersede sgl-project#32183)
  sgl-project#34002  Sidecar backup vacuously-successful fix (replaces sgl-project#33656, with tests)
  sgl-project#33862  Reclaim redundant host mirrors after storage backup
  sgl-project#31315  Avoid repeated Mooncake gets after stale hits
  sgl-project#32327  Q8KV8 sparse MLA prefill backend (flashmla_sparse_q8)
  sgl-project#31668  Fix sidecar pool life-time (use-after-free on prefetch abort)
  sgl-project#31195  TP0 verify-token-budget broadcast (adapted to get_schedule() API)

Dropped (per user request or superseded):
  sgl-project#32771  IndexCache C4 top-k reuse — has bug
  sgl-project#32035  DSpark C128 online compressor — has bug
  sgl-project#33656  Superseded by sgl-project#34002 (same fix + unit tests)
  sgl-project#32183  Superseded by sgl-project#33872 (included in supersede PR)
  sgl-project#33145  Base f01f706 already has superior reasoning-effort profile system

Conflicts resolved:
  sgl-project#31195: adapted to base get_schedule().disable_overlap_schedule API
  sgl-project#32327: path remapped jit_kernel/ -> kernels/jit/ and kernels/ops/attention/
  sgl-project#31668: applied cleanly on top of sgl-project#30393+sgl-project#34002+sgl-project#33862 modifications
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants