Skip to content

[Fix] Qwen3-ASR config: set thinker_config before super().__init__#24187

Merged
AgainstEntropy merged 2 commits into
sgl-project:mainfrom
AgainstEntropy:fix/qwen3-asr-config-init-order
May 12, 2026
Merged

[Fix] Qwen3-ASR config: set thinker_config before super().__init__#24187
AgainstEntropy merged 2 commits into
sgl-project:mainfrom
AgainstEntropy:fix/qwen3-asr-config-init-order

Conversation

@AgainstEntropy
Copy link
Copy Markdown
Collaborator

@AgainstEntropy AgainstEntropy commented Apr 30, 2026

Motivation

Launching any Qwen3-ASR model crashes during config init under huggingface_hub >= 1.12:

File "python/sglang/srt/configs/qwen3_asr.py", line 151, in __init__
    super().__init__(**kwargs)
File "huggingface_hub/dataclasses.py", line 276, in init_with_validate
    cls.validate(self)
File "transformers/configuration_utils.py", line 446, in validate_token_ids
    text_config = self.get_text_config(decoder=True)
File "python/sglang/srt/configs/qwen3_asr.py", line 164, in get_text_config
    return self.thinker_config.text_config
AttributeError: 'Qwen3ASRConfig' object has no attribute 'thinker_config'

PretrainedConfig.__init__ is wrapped by huggingface_hub.dataclasses.init_with_validate, which runs validate_token_idsget_text_config(decoder=True) during base init. Our get_text_config reads self.thinker_config.text_config, but self.thinker_config is only set after super().__init__ returns. Result: every Qwen3-ASR launch hard-fails before model load.

This was introduced in #22073 (Qwen3-ASR support) and started crashing once huggingface_hub 1.12+ began running validation inside the base __init__. The current sglang pyproject.toml already requires huggingface_hub >= 1.0, so users hit this on any reasonably recent install.

Registered unittest for this model is covered here: #22731

Modifications

python/sglang/srt/configs/qwen3_asr.py — set self.thinker_config before calling super().__init__(**kwargs), so the base validator finds the attribute it needs. One-line move; no behavior change for callers that already construct successfully on older huggingface_hub.

Accuracy Tests

Speed Tests and Profiling

Test plan

Verified locally on H200 with transformers 5.5.4 + huggingface_hub 1.12.2:

  • Without this fix: python -m sglang.launch_server --model-path Qwen/Qwen3-ASR-0.6B and the existing test/manual/models/test_qwen3_asr.py suite both crash at server startup with the AttributeError above (every test ERRORs in setUpClass).
  • With this fix: server launches cleanly; all 19 tests in test/manual/models/test_qwen3_asr.py run end-to-end (HTTP + WebSocket paths), passing or failing on their own merits.

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

PretrainedConfig.__init__ runs through
huggingface_hub.dataclasses.init_with_validate, which invokes
validate_token_ids → get_text_config → self.thinker_config.text_config
during base init. With huggingface_hub >= 1.12 this hard-fails any
Qwen3-ASR launch:

  AttributeError: 'Qwen3ASRConfig' object has no attribute 'thinker_config'

Set self.thinker_config before super().__init__ so the base validator
finds it.
@gemini-code-assist
Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@AgainstEntropy
Copy link
Copy Markdown
Collaborator Author

/tag-and-rerun-ci

@AgainstEntropy
Copy link
Copy Markdown
Collaborator Author

/rerun-failed-ci

@AgainstEntropy AgainstEntropy requested a review from mickqian May 2, 2026 07:53
@vf827393
Copy link
Copy Markdown

vf827393 commented May 7, 2026

Same issue encountered with v0.5.11

@AgainstEntropy AgainstEntropy enabled auto-merge (squash) May 10, 2026 20:16
@AgainstEntropy AgainstEntropy merged commit 0f3932c into sgl-project:main May 12, 2026
257 of 280 checks passed
SpencerGarnets added a commit to ai-blaise/optimization-playground that referenced this pull request May 12, 2026
…ack)

Brings in upstream sgl-project/sglang main commits since
096ad02 (merge base, Laguna-XS.2 model support).
Total: 28 upstream commits composed.

Custom-stack files preserved intact (entirely-ours, byte-identical to
origin/main):
  - Blackwell CuTe kernel suite (warp_decode_cute, g1_attention_cute,
    gated_norm_cute, layersplit_cute, fused_store_index_cache)
  - TurboQuant 2.5-bit dense KV cache path
  - HIGGS 2-bit dense KV cache path (with split-K decode)
  - NVFP4 IndexCache dispatcher (active gate)
  - quantization_config_dispatch (HF-config-driven runtime routing)
  - All custom server-args flags and runtime methods preserved

Verification:
  - 200+ merged Python files compile cleanly
  - Dispatcher symbol presence verified
  - HIGGS pool / TurboQuant pool classes present at expected lines
  - compressed_tensors_w4a4_nvfp4_moe imports clean
  - All custom server-args flags present (enable_higgs_dense_2bit_kv_cache,
    enable_turboquant_dense_kv_cache, turboquant_dense_kv_preset,
    indexer_quantization_declared, higgs_mla_decode_num_splits, etc.)

Manual-merged shared files (auto-merge gave broken/mixed output; cleaned
up post-merge):
  - python/sglang/srt/disaggregation/mooncake/conn.py: upstream's PR#24932
    refactored maybe_send_extra into a state-types-loop. Replayed our
    LayerSplit NSA state-index-length-mismatch check inside the SWA/NSA
    branch of the new loop body.
  - sgl-kernel/python/sgl_kernel/__init__.py: upstream's PR#23449 (Apple
    Silicon Metal kernel) wrapped the entire module body in
    `if darwin/arm64: from sgl_kernel.metal import * else: ...`. The
    auto-merge duplicated the file body; rewrote cleanly with upstream's
    structure and re-injected our `g1_gate_forward`,
    `warp_decode_cute_moe_forward`, and
    `warp_decode_cute_moe_packed_forward` imports plus `g1_gate_forward`
    in _DEBUG_EXPORT_NAMES.
  - python/sglang/srt/managers/scheduler_output_processor_mixin.py: line
    628 still referenced `result.num_accepted_drafts` (renamed by PR
    sgl-project#25038 to `num_correct_drafts`). Renamed in place.
  - python/sglang/srt/observability/scheduler_metrics_mixin.py: a block
    around the spec-decode logging path had mixed old/new names from
    auto-merge (lines 553/557/560). Renamed `spec_num_accepted_tokens`
    -> `spec_num_accept_tokens` and local `num_accepted_drafts` ->
    `num_correct_drafts` to match the rest of the file.
  - test/test_smc_info.py: stub Req mock used the old field names
    `spec_accepted_drafts` and `update_spec_acceptance_histogram`.
    Renamed to `spec_num_correct_drafts` and
    `update_spec_correct_drafts_histogram` per PR sgl-project#24081.

Auto-merge cleanly integrated upstream changes to:
  - server_args.py (new fields: prefill_only_disable_kv_cache,
    weight_loader_drop_cache_after_load, prefill_delayer_queue_min_ratio,
    prefill_delayer_max_delay_ms, speculative_draft_window_size, etc.)
  - mem_cache/memory_pool.py (new NoOpMHATokenToKVPool)
  - model_executor/model_runner_kv_cache_mixin.py (NoOpMHATokenToKVPool
    pool factory + _validate_prefill_only_disable_kv_cache_pool_family)
  - layers/attention/nsa_backend.py (spec rename
    num_accepted_drafts -> num_correct_drafts;
    num_accepted_tokens -> num_accept_tokens)
  - layers/attention/nsa/nsa_indexer.py (new _apply_q_scale_and_softmax_scale
    compile method; torch.mm replaces deep_gemm wrapper)
  - 28+ disaggregation/spec/runner files with mostly clean
    upstream-side-only integration.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

----- upstream commit subjects (28) -----
fd3eb77 [Cookbook]: add Laguna-XS.2 (Poolside) (sgl-project#24730)
6be1a45 Fix swa component host hit (sgl-project#25085)
693f497 [NPU] use causal_conv1d_update_v2 for performance (sgl-project#24595)
1efe9e2 [Bug Fix] Reject incompatible combination of --disable-cuda-graph-padding and --enable-torch-compile (sgl-project#23903)
8d27ce7 Optimize uvicorn startup command (sgl-project#25041)
b35fd5f [fix] skip legacy minicpmv conv template for MiniCPM-V 4.6 (sgl-project#24998)
7582237 [Tiny Fix] Disable BCG when inner layer_model unresolved (sgl-project#25021)
ca3bc05 Deepseek-v4-Pro share expert tp1 (sgl-project#24949)
a72d3ae [Spec] Multi-layer mamba scatter cleanup; fix positional call bug (sgl-project#25030)
7128533 Revert "Migrate Intel CPU cases to the test/registered." (sgl-project#25044)
1f985c5 [Spec] Rename `accepted_indices` -> `accept_indices`; drop `_token_id` suffix per Rule 5 (sgl-project#25038)
ecf5d84 Migrate Intel CPU cases to the test/registered. (sgl-project#22670)
d7f4761 [PD] Refactor hybrid state transfer (sgl-project#24932)
91907b7 [UnifiedTree]: Fix Unified HiCache tombstone lock release replay (sgl-project#24972)
4ad63ad [Spec] Rename `accepted_drafts` -> `correct_drafts` for unambiguous naming (sgl-project#24081)
6bfb365 [PD] Rate limit prefill inflight polling warnings (sgl-project#24967)
6bb79c1 [Linear Attn] Add CUSTOM enum and plugin extensibility for kernel backends (sgl-project#24937)
cfc41d5 Fix kimi k2.5 mla eagle + dp attention (sgl-project#25033)
0f3932c [Fix] Qwen3-ASR config: set thinker_config before super().__init__ (sgl-project#24187)
f526e3f [Spec] Mamba scatter cleanup; fix multi-layer positional bug; dflash naming (sgl-project#25029)
10375a1 [NIXL][XPU] Fix uint64 overflow for mismatched P/D TP sizes (e.g. prefill_tp=1, decode_tp=2) (sgl-project#24648)
0a37d24 [diffusion] hardware: support sage attention backend on MUSA (attn backend, 21/N) (sgl-project#24752)
5495026 [HiCache] feat: default storage prefetch timeout (sgl-project#23309)
186eb42 Feat: Support SWA (Sliding Window Attention) for EAGLE-3 drafter (sgl-project#24664)
a75b79e Feat: Support newer EAGLE-3 drafters (sgl-project#24663)
f3a8189 [Spec] Internal rename per N2 v2 naming rule (sgl-project#25014)
bfc2eda [MUSA] Use MUSA-optimized operators in piecewise CUDA graph (sgl-project#23633)
74d70af [Apple Silicon] Add Metal kernel support in sgl-kernel (sgl-project#23449)
@AgainstEntropy AgainstEntropy deleted the fix/qwen3-asr-config-init-order branch May 26, 2026 22:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants