Repository navigation

Performance 5/7: Qwen3.5 and Qwen3-Omni - #970

Merged

merceod merged 518 commits into

mainfrom

perf/split-5-language

Nov 3, 2026

Merged

Performance 5/7: Qwen3.5 and Qwen3-Omni#970
merceod merged 518 commits into
mainfrom
perf/split-5-language

This pull request is big! We're only showing the most recent 250 commits

Commits on Oct 30, 2026
- Merge perf/split-1-shared (#858, fd7211dcd..f3de7486c) into perf/split-2-speech-language
  
  dghosef
  and
  claude
  committed
  8266de2
- API server: the queue gauge counts inputs not being prepared, pooled or inline (R3.1, R3.2)
  
  dghosef
  and
  claude
  committed
  f6c8cd3
- API server: validate the class PREPROCESS_WORKERS value; fix a stale comment (R3.3, R3.4)
  
  dghosef
  and
  claude
  committed
  898034a
- Merge perf/split-2-speech-language (#859, with #858's fd7211dcd..f3de7486c) into perf/split-3-generation
  
  dghosef
  and
  claude
  committed
  ebe1478
- API server: one positive-int check for preprocess_workers; fix the startup comment (R4.2, R4.3)
  
  dghosef
  and
  claude
  committed
  403ec8f
- API server: without a pool, the queue gauge's count is the deque's length (R4.4)
  
  dghosef
  and
  claude
  committed
  42f8527
- Merge origin/main into perf/split-1-shared
  
  dghosef
  and
  claude
  committed
  5286136
- Docs: say which MSTAR_PY_SWITCH_INTERVAL_SEC values the worker ignores
  
  dghosef
  and
  claude
  committed
  6e67694
- Docs: point api_switch_interval_s to the workers' switch-interval variable
  
  dghosef
  and
  claude
  committed
  bcd00f6
- Merge origin/main into perf/split-1-shared
  
  dghosef
  and
  claude
  committed
  ad5f644
- Tests: pin the pooled queue gauge once settled, and its count during a hand-off (R4.1)
  
  dghosef
  and
  claude
  committed
  9e93d0c
- Merge perf/split-3-generation (#860, with main 8a2095f81) into perf/split-4-shared
  
  dghosef
  and
  claude
  committed
  fe699ff
- Merge perf/split-1-shared (#858, with origin/main fadeb2759) into perf/split-2-speech-language
  
  dghosef
  and
  claude
  committed
  f4df456
- Ragged attention: update a test docstring for unique op keys
  
  dghosef
  and
  claude
  committed
  ed31270
- Docs: say which outputs a stop check may withhold
  
  dghosef
  and
  claude
  committed
  e2c8d1b
- Worker, engine: the stop check's host outputs are Mappings in every signature
  
  dghosef
  and
  claude
  committed
  2ed1437
- Tests: one idle poll for the API server's stand-in transports; drop dead ones
  
  dghosef
  and
  claude
  committed
  797577a
- PiecewisePackedConfig: attn_key is resolved when the region declares, not by rewriting declare_step
  
  dghosef
  and
  claude
  committed
  11a5d33
- Engine: a forward gets a lease_before_step region's runner only on a step that leased it
  
  dghosef
  and
  claude
  committed
  219098f
- Device loop-back is its own declaration, DeviceLoopback, not a TokenStop field
  
  dghosef
  and
  claude
  committed
  6634770
- Stop check: the step closing a reply that ended at prefill withholds the loop-back signal too
  
  dghosef
  and
  claude
  committed
  a643d39
- Qwen3-Omni audio encoder: a step whose rows all failed batches without raising (R1.18)
  
  dghosef
  and
  claude
  committed
  e7e41f4
- Chatterbox T3, Higgs Audio, Qwen3-ASR: a stop token sampled at prefill ends the reply
  
  dghosef
  and
  claude
  committed
  bb96fe8
- Qwen3.5: a captured decode step stages no rotary tables, whatever its rows bring
  
  dghosef
  and
  claude
  committed
  f659715
- Qwen3-Omni audio encoder on the shared AuT encoder (R1.1, R1.10, R1.11, R1.5)
  
  dghosef
  and
  claude
  committed
  f1702e9
- mstar::fused_add_rms_norm: the CUDA kernel takes FlashInfer's fused path only when x, residual and weight share a dtype
  
  dghosef
  and
  claude
  committed
  c7fc739
- One cuDNN benchmark scope, beside the engine's precision scope
  
  dghosef
  and
  claude
  committed
  56eca1e
- Qwen3-Omni audio encoder: the capture check reads the attention resource's ceiling (R1.2)
  
  dghosef
  and
  claude
  committed
  5d7dd05
- Qwen3-Omni: the audio encoder is the AuT node Qwen3-ASR runs, packed and captured
  
  dghosef
  and
  claude
  committed
  7cd33ba
- Ragged attention: a one-row-per-step region's ceiling is max_segments_per_step
  
  dghosef
  and
  claude
  committed
  17fa86f
- AuT encoder: clamp an fp16 residual stream as HF does; Omni's audio attention follows the engine dtype (R1.8)
  
  dghosef
  and
  claude
  committed
  fc1912f
- Qwen3.5: the vision node boots on the image processor alone
  
  dghosef
  and
  claude
  committed
  6f082fe
- Tests: the refused video processor is read into a name (ruff B018)
  
  dghosef
  and
  claude
  committed
  dae9a7a
- Qwen3-Omni: drop the audio_seqlens tensor nothing reads (R1.5)
  
  dghosef
  and
  claude
  committed
  98fb426
- Qwen3.5 vision: the compile regions warm at start-up; the docs say when model code still compiles
  
  dghosef
  and
  claude
  committed
  422d385
- Prefill ladder override: one checked override point in ARNodeSubmodule; Omni keeps its default ladder (R1.3)
  
  dghosef
  and
  claude
  committed
  271e51b
- configs/qwen3_5_9b_dp2.yaml: the 9B prefill ladder (R1.4)
  
  dghosef
  and
  claude
  committed
  bbed279
- Qwen3-Omni Talker on DecoderLayer and run_layers; Thinker layers start the residual stream through forward_residual (R1.7)
  
  dghosef
  and
  claude
  committed
  15f7a20
- test_multimodal_ordering: run the Qwen3-Omni cases in CPU Core (R1.9)
  
  dghosef
  and
  claude
  committed
  14bff61
- Accelerator graph runner: `_stage`'s docstring says what a replay's padded tail holds (R1.13)
  
  dghosef
  and
  claude
  committed
  2ba511c
- Qwen3-Omni Thinker: one mechanism for the Talker masks' host sync; the vision prefill's mask writes don't sync (R1.15)
  
  dghosef
  and
  claude
  committed
  8249d90
- Qwen3-Omni Thinker decode: padding rows take the broadcast positions; decode rows skip unused reads (R1.14)
  
  dghosef
  and
  claude
  committed
  98691fc
- Tests: Qwen3-Omni's audio encoder loads no transformers config to check
  
  dghosef
  and
  claude
  committed
  e260779
- Tests: pin the decode-row interleave, multi-window encoder parity and the replay's windows (R1.17)
  
  dghosef
  and
  claude
  committed
  e9749cf
- ARNodeSubmodule: check a ladder's entries before hashing them (R1.3)
  
  dghosef
  and
  claude
  committed
  ec07abd
- test_trust_remote_code: the Qwen3-Omni audio encoder reads no HF config (R1.1)
  
  dghosef
  and
  claude
  committed
  ce8da17
- Kokoro: say that cuDNN benchmarking covers both captured halves
  
  dghosef
  and
  claude
  committed
  32c0f21
- Kokoro: put the fused BiLSTM in the module tree
  
  dghosef
  and
  claude
  committed
  9aa2036
- Split speech configs: run their two workers under MPS
  
  dghosef
  and
  claude
  committed
  ef563bf
- One residual code predictor for Qwen3-Omni and Qwen3-TTS
  
  dghosef
  and
  claude
  committed
  fb3440e
- Qwen3-TTS test: name the row-output test after what it checks
  
  dghosef
  and
  claude
  committed
  1934aec
- Docs: describe the Qwen3-Omni 2-GPU PD config and OmniVoice's token buckets
  
  dghosef
  and
  claude
  committed
  ee582fe
- OmniVoice: check the packed runner at boot, drop the unused plan dtype
  
  dghosef
  and
  claude
  committed
  b8f4ecf
- OmniVoice test: compare the silence trim with pydub at full scale
  
  dghosef
  and
  claude
  committed
  cad69d3
- Code predictor: check the sampled tokens with is_contiguous
  
  dghosef
  and
  claude
  committed
  54037d8
- AuT frontend: pad only the last convolution launch (R1.11)
  
  dghosef
  and
  claude
  committed
  3c03ce5
- Merge perf/split-1-shared (#858) into perf/split-2-speech-language
  
  dghosef
  and
  claude
  committed
  bfb1b69
- Docs: serving keys for the speech split configs and the switch interval
  
  dghosef
  and
  claude
  committed
  0dc1f40
- Docs: Qwen3-Omni's audio encoder is the AuT node
  
  dghosef
  and
  claude
  committed
  19150a2
- Merge origin/main into perf/split-1-shared
  
  dghosef
  and
  claude
  committed
  da5ef36
- Qwen3-TTS, Omnilingual, ragged attention: comments name the new owners
  
  dghosef
  and
  claude
  committed
  1cf20c7
- Comments and docs: a shard_over_sp=False ragged resource splits its heads over TP only
  
  dghosef
  and
  claude
  committed
  509189b
- Merge perf/split-1-shared (#858, with origin/main b433a24a5) into perf/split-2-speech-language
  
  dghosef
  and
  claude
  committed
  70f8eb6
- docs/models.rst: the Omni audio encoder's eager and sub-second cases as the code has them (R2.1)
  
  dghosef
  and
  claude
  committed
  09a6b65
- Head sharding: one per-resource switch, shard_over_sp; NodeSubmodule.splits_heads goes
  
  dghosef
  and
  claude
  committed
  5514453
- Qwen3-Omni Thinker prefill capture: padding rows take the broadcast positions (R2.2)
  
  dghosef
  and
  claude
  committed
  ad54171
- GLM-5.2 prefill ladder through ARNodeSubmodule's override; the base declares the ladder (R2.3)
  
  dghosef
  and
  claude
  committed
  f2ef681
- AuT encoder: the frontend takes the step's clips as a list; Omni and Qwen3-ASR stop padding them (R2.4)
  
  dghosef
  and
  claude
  committed
  1c49bdc
- AuT encoder: one checkpoint loader, `AuTEncoder.from_checkpoint`, for Qwen3-Omni and Qwen3-ASR (R2.5)
  
  dghosef
  and
  claude
  committed
  3bb81c9
- Merge origin/main into perf/split-1-shared
  
  dghosef
  and
  claude
  committed
  d28d18d
- AuT frontend: pad the last convolution launch to a multiple of 32 chunks, not a power of two (R2.6)
  
  dghosef
  and
  claude
  committed
  7fb675b
- AuT encoder: pack a step only up to the largest captured token count
  
  dghosef
  and
  claude
  committed
  d27bc7b
- Merge origin/main into perf/split-1-shared
  
  dghosef
  and
  claude
  committed
  a908f31
- AuT encoder: count tokens by the checkpoint's own chunk, for both models
  
  dghosef
  and
  claude
  committed
  7444428
- Ragged attention: refuse a per-request ceiling on a resource sized per step
  
  dghosef
  and
  claude
  committed
  3fa13b4
- Test: the engine hands a forward only the regions its step leased
  
  dghosef
  and
  claude
  committed
  9086836
- Cosmos3: take cudnn_benchmark from fp32_precision, drop the wan_vae re-export
  
  dghosef
  and
  claude
  committed
  14823f4
- Qwen3-Omni: drop audio_seqlens, which the audio encoder no longer reads
  
  dghosef
  and
  claude
  committed
  66d9334
- AuT encoder node: move it beside the encoder, one loader and spec for both models
  
  dghosef
  and
  claude
  committed
  61eb015
- Docs: a Qwen3-Omni audio step over 4096 tokens runs its clips one by one
  
  dghosef
  and
  claude
  committed
  39f4de3
- AuT frontend: pad the last convolution launch to the smaller of a power of two and a multiple of 32 (R3.1)
  
  dghosef
  and
  claude
  committed
  45a43d3
- ARNodeSubmodule: narrow the PREFILL_TOKEN_BUCKETS comment to what holds (R3.2)
  
  dghosef
  and
  claude
  committed
  a35f495
- Merge perf/split-4-shared (final) into perf/split-5-language
  
  dghosef
  and
  claude
  committed
  5e4fa5b
- Name the TP x SP head-sharding test for the kinds it covers; reflow a docs paragraph
  
  dghosef
  and
  claude
  committed
  50401ad
- AuT encoder: say a step holds up to 4096 tokens; import its loader at the top
  
  dghosef
  and
  claude
  committed
  8c77576
- Merge perf/split-1-shared (#858, with origin/main 18b0859b2) into perf/split-2-speech-language
  
  dghosef
  and
  claude
  committed
  2fe99aa
- Qwen3.5 9B DP2 config: give the GDN pool prefix snapshot slots (RM.1)
  
  dghosef
  and
  claude
  committed
  8cdbd83
- Merge perf/split-2-speech-language (#859, with #858 and origin/main 18b0859b2) into perf/split-3-generation
  
  dghosef
  and
  claude
  committed
  965c8eb
- Wan 2.2: keep the dit's heads whole over SP with shard_over_sp=False
  
  dghosef
  and
  claude
  committed
  070437f
- Merge origin/main into perf/split-1-shared
  
  dghosef
  and
  claude
  committed
  78c53ac
- BAGEL: run a guided window as its own walk on one node too
  
  dghosef
  and
  claude
  committed
  60d4a68
- Say the api_switch_interval_s numbers predate #849's output-path rewrite
  
  dghosef
  and
  claude
  committed
  02c4517
- Drop the head-sharding paragraph from the submodule attributes section
  
  dghosef
  and
  claude
  committed
  82d9fb5
- Drop the TP x SP head-sharding test that main's sizing already passes
  
  dghosef
  and
  claude
  committed
  ac2cd28
- BAGEL: hold guidance KV only for the branches a request's window runs
  
  dghosef
  and
  claude
  committed
  25cb21e
- Drop the shard_over_sp sizing test, which main's sizing already passes
  
  dghosef
  and
  claude
  committed
  bb15c48
- BAGEL: comments state the contract instead of the code they replaced
  
  dghosef
  and
  claude
  committed
  60b297b
- BAGEL docs: give the measured output shift of guided text-to-image
  
  dghosef
  and
  claude
  committed
  8b7f8bf
- test_kv_admission: name the CFG walk BAGEL defines
  
  dghosef
  and
  claude
  committed
  cd0bd5b
- Merge perf/split-1-shared (#858, with origin/main f83fcaeb3) into perf/split-2-speech-language
  
  dghosef
  and
  claude
  committed
  0791582
- Cosmos3 and LingBot decode through wan_vae.FastDecoder and decode_dtype
  
  dghosef
  and
  claude
  committed
  0ff325d
- Wan 2.2: drop the FFN GELU epilogue, which has no measurement of its own
  
  dghosef
  and
  claude
  committed
  beef216
- Qwen3 text encoder: refuse a row without text_len instead of never replaying
  
  dghosef
  and
  claude
  committed
  da89d47
- Image output knobs: an explicit null is the default, and only integers pass intake
  
  dghosef
  and
  claude
  committed
  0b9c597
- V-JEPA2: say the B=1 guard routes eager steps only, and drop "as before" comments
  
  dghosef
  and
  claude
  committed
  6d3d9df
- wan_vae.decode_dtype: say that an override of 0 forces bf16
  
  dghosef
  and
  claude
  committed
  a699496
- Merge origin/main into perf/split-1-shared
  
  dghosef
  and
  claude
  committed
  0cc8902
- Higgs Audio: declare text_output so inline decode tokens skip the per-row postprocess
  
  dghosef
  and
  claude
  committed
  c6d9efd
- Qwen3.5: say the API_SWITCH_INTERVAL_S gain predates #849
  
  dghosef
  and
  claude
  committed
  d1bb3d5
- Conductor: build its instance table from WorkerGraph.replica_ranks
  
  dghosef
  and
  claude
  committed
  bbe03b6
- Image output knobs: serve the whole numbers main served
  
  dghosef
  and
  claude
  committed
  4e25b96
- BAGEL: build only the window walks a deployment can run
  
  dghosef
  and
  claude
  committed
  db74664
- BAGEL: say why a split LLM still publishes both branch labels
  
  dghosef
  and
  claude
  committed
  e436e80
- BAGEL and Cosmos3: drop loop-counter leftovers from comments and names
  
  dghosef
  and
  claude
  committed
  359e9e8
- Merge perf/split-2-speech-language (#859, with #858 and origin/main f83fcaeb3) into perf/split-3-generation
  
  dghosef
  and
  claude
  committed
  a18e325
- Merge perf/split-1-shared (#858, with origin/main 3e57b37b6) into perf/split-2-speech-language
  
  dghosef
  and
  claude
  committed
  6e5ba4d
- Image output: check only the chosen format's compression knob
  
  dghosef
  and
  claude
  committed
  3d83149
- BAGEL: say LLM-disaggregated, not split, in the branch-label comment
  
  dghosef
  and
  claude
  committed
  3dd63ee
- TP read-failure test: derive the loop-back hooks from device_loopback = None
  
  dghosef
  and
  claude
  committed
  62efad1
- TP read-failure test: drop the stale "emit" from the stand-in's comment
  
  dghosef
  and
  claude
  committed
  a91f25b
- Merge perf/split-2-speech-language (#859, with #858 and origin/main 3e57b37b6) into perf/split-3-generation
  
  dghosef
  and
  claude
  committed
  1a60e50
- Merge perf/split-3-generation (#860 head 1a60e5036) into perf/split-4-shared
  
  dghosef
  and
  claude
  committed
  ed24d03
- Merge-up fixes: DeviceLoopback in the withhold check, chunk-sink stand-in poll, postprocess_s on inline failure
  
  dghosef
  and
  claude
  committed
  a988617
- Withhold check: the loop-back signal is exempt only on its own walk
  
  dghosef
  and
  claude
  committed
  7ed07c4
- Merge origin/main (bb0e0a8d7: #962, #941, #944) into perf/split-1-shared
  
  dghosef
  and
  claude
  committed
  0e495ae
- Merge perf/split-1-shared (#858, with origin/main bb0e0a8d7) into perf/split-2-speech-language
  
  dghosef
  and
  claude
  committed
  1b44fda
- Merge perf/split-2-speech-language (#859, with #858 and origin/main bb0e0a8d7) into perf/split-3-generation
  
  dghosef
  and
  claude
  committed
  3a1ff3b
- Merge perf/split-3-generation (#860 head 3a1ff3bc8, main bb0e0a8d7) into perf/split-4-shared
  
  dghosef
  and
  claude
  committed
  886c20c
- Withhold check test: the loop-back exemption is scoped to its walk
  
  dghosef
  and
  claude
  committed
  d5cde83
- Merge perf/split-4-shared (#969 head d5cde83a4, #860 3a1ff3bc8) into perf/split-5-language
  
  dghosef
  and
  claude
  committed
  9226c90
- Qwen3-ASR/Omni AuT node: pin the replay's windows and packed-clip parity on the shared node
  
  dghosef
  and
  claude
  committed
  44c871a
- Merge origin/main (c96e02b7d: #968, #856, #778, #876) into perf/split-1-shared
  
  dghosef
  and
  claude
  committed
  7ee5eb4
- Merge perf/split-1-shared (#858, with origin/main c96e02b7d) into perf/split-2-speech-language
  
  dghosef
  and
  claude
  committed
  45226de
- Merge perf/split-2-speech-language (#859, with #858 and origin/main c96e02b7d) into perf/split-3-generation
  
  dghosef
  and
  claude
  committed
  bde1253
- Merge perf/split-3-generation (#860 head bde1253ed) into perf/split-4-shared
  
  dghosef
  and
  claude
  committed
  00374f0
- Merge perf/split-4-shared (#969 head 00374f0a8, #860 bde1253ed) into perf/split-5-language
  
  dghosef
  and
  claude
  committed
  21989d6
Commits on Oct 31, 2026
- Attention: select_last_hidden is one eager boundary, so prefill buckets don't recompile its frame
  
  dghosef
  and
  claude
  committed
  b53c500
- Merge origin/main (9f84f18cd: #965) into perf/split-1-shared
  
  dghosef
  and
  claude
  committed
  15f8249
- Merge perf/split-1-shared (#858, with origin/main 9f84f18cd) into perf/split-2-speech-language
  
  dghosef
  and
  claude
  committed
  bd9aa67
- Merge perf/split-2-speech-language (#859, with #858 and origin/main 9f84f18cd) into perf/split-3-generation
  
  dghosef
  and
  claude
  committed
  d4bd60c
- BAGEL takes its prefill's last rows through select_last_hidden; test docstrings
  
  dghosef
  and
  claude
  committed
  5d0bab7
- Merge perf/split-4-shared (#969 head 5d0bab728) into perf/split-5-language
  
  dghosef
  and
  claude
  committed
  86be385
- Merge perf/split-3-generation (#860 head d4bd60cf9) into perf/split-4-shared
  
  dghosef
  and
  claude
  committed
  86d313c
- Merge perf/split-4-shared (#969 head 86d313ccf, #860 d4bd60cf9) into perf/split-5-language
  
  dghosef
  and
  claude
  committed
  cea7d20
- Merge perf/split-4-shared (#969 head acc31f22d) into perf/split-5-language
  
  dghosef
  and
  claude
  committed
  df7ae34
- GDN prefill run: state the zeroed-tail precondition the mixed wrapper also meets
  
  dghosef
  and
  claude
  committed
  acc31f2
- OmniVoice: key the backbone's step outputs by the engine's row ids
  
  dghosef
  and
  claude
  committed
  099f4ea
- Merge perf/split-2-speech-language (#859, the OmniVoice request-id fix) into perf/split-3-generation
  
  dghosef
  and
  claude
  committed
  85575a0
- Qwen3-Omni 2-GPU config: keep the 8192 Thinker ladder off the 80 GB default
  
  dghosef
  and
  claude
  committed
  ef6b293
- Merge perf/split-4-shared (#969 head 49c9a4008, #860 85575a08b) into perf/split-5-language
  
  dghosef
  and
  claude
  committed
  e6f27fb
- Merge perf/split-3-generation (#860 head 85575a08b) into perf/split-4-shared
  
  dghosef
  and
  claude
  committed
  49c9a40
- Qwen3-Omni 2-GPU config: don't claim the remaining captures fit 80 GB
  
  dghosef
  and
  claude
  committed
  9372dc5
- Merge main (c829a71cc) into perf/split-1-shared
  
  dghosef
  and
  claude
  committed
  42bb972
- Merge perf/split-1-shared (#858, with origin/main c829a71cc) into perf/split-2-speech-language
  
  dghosef
  and
  claude
  committed
  a9da761
- Merge perf/split-2-speech-language (#859, with origin/main c829a71cc) into perf/split-3-generation
  
  dghosef
  and
  claude
  committed
  4e410aa
- Merge perf/split-3-generation (#860 head 4e410aafd) into perf/split-4-shared
  
  dghosef
  and
  claude
  committed
  8f9766e
- Merge perf/split-4-shared (#969 head 8f9766e6b, with origin/main c829a71cc) into perf/split-5-language
  
  dghosef
  and
  claude
  committed
  05db3b7
Commits on Nov 1, 2026
- Merge main (ad0a21eca) into perf/split-1-shared
  
  dghosef
  and
  claude
  committed
  bff76e2
- Pin video decode threads to the CPUs the plan leaves under pinning
  
  dghosef
  and
  claude
  committed
  7de9cc6
- Merge perf/split-1-shared (#858, with origin/main ad0a21eca) into perf/split-2-speech-language
  
  dghosef
  and
  claude
  committed
  576d05f
- Merge perf/split-2-speech-language (#859, with origin/main ad0a21eca) into perf/split-3-generation
  
  dghosef
  and
  claude
  committed
  5a7b055
- Ragged attention: a wrapper skips replanning the layout it last planned
  
  dghosef
  and
  claude
  committed
  0022bbe
- Wan2.2 docs: date the attention and CFG-parallel timings as before #963
  
  dghosef
  and
  claude
  committed
  e1d3def
- Merge perf/split-3-generation (#860, with origin/main ad0a21eca) into perf/split-4-shared
  
  dghosef
  and
  claude
  committed
  7b67849
- Conductor: leave the idle wait to its own change
  
  dghosef
  and
  claude
  committed
  a5c0e9e
- API server tests: the failed-wait test pins its back-off; stand-in waits as main has them
  
  dghosef
  and
  claude
  committed
  3d7a23f
- Preprocess pool: at most the CPUs the plan leaves unpinned
  
  dghosef
  and
  claude
  committed
  f1a1493
- Data worker: drain routes before an input only when it prepares here
  
  dghosef
  and
  claude
  committed
  e2a8352
- Merge perf/split-4-shared (#969, with origin/main ad0a21eca) into perf/split-5-language
  
  dghosef
  and
  claude
  committed
  3e49609
- Qwen3.5 vision: compile the block loop, so a step past the captured buckets runs compiled
  
  dghosef
  and
  claude
  committed
  bd48d76
- API server: drop the api_switch_interval_s knob
  
  dghosef
  and
  claude
  committed
  bffb067
- Drop the GIL switch-interval knob's model defaults (the knob is removed in #858)
  
  dghosef
  and
  claude
  committed
  a071d24
- Tests: the TP2 publish test's KV managers hold one rank's head
  
  dghosef
  and
  claude
  committed
  641e06c
- API server: drop warmup_requests' concurrency option
  
  dghosef
  and
  claude
  committed
  a25694a
- Merge remote-tracking branch 'origin/main' into land/858
  dghosef
  committed
  34fbf94
- Qwen3.5 vision: a request too large for a replay gets an encoder step of its own
  
  dghosef
  and
  claude
  committed
  3f8d4af
- Merge origin/main (165b403b7: #794, #950) into perf/split-1-shared
  
  dghosef
  and
  claude
  committed
  03f7758
- Tests: Canary's decode cap leaves the prefill's token in the budget
  
  dghosef
  and
  claude
  committed
  a4b5fca
- Docs: last_prefill_flag for a prompt prefilled over several passes; GLM-5.2 test comment names the loop cap
  
  dghosef
  and
  claude
  committed
  f991121
- Qwen3.5 vision split: review cleanups
  
  dghosef
  and
  claude
  committed
  b0cbacd
- Merge c2/859-video: Qwen3.5 vision compiles its block loop, and a request too large for a replay gets an encoder step of its own
  
  dghosef
  and
  claude
  committed
  a3e2053
- Merge #858's final head (f99112154) into #859
  
  dghosef
  and
  claude
  committed
  5007896
- Merge origin/main (160b4bc6d, #858 squash) into #859
  
  dghosef
  and
  claude
  committed
  db7eccd
- Chatterbox, Higgs Audio tests: a stop id at prefill ends the partition
  
  dghosef
  and
  claude
  committed
  3b47e16
- Orpheus test: the stop check's fake info names its walk
  
  dghosef
  and
  claude
  committed
  51b9502
- Merge main (3e98682a8: #920 KV-aware routing) into perf/split-2-speech-language
  
  dghosef
  and
  claude
  committed
  c26159f
- Merge c3/859 into perf/split-3-generation
  dghosef
  committed
  fd37bb9
- Merge c3/860 into perf/split-4-shared
  dghosef
  committed
  4c6d88b
- Merge c3/969 into perf/split-5-language
  dghosef
  committed
  60629b0
- Qwen3.5 9B DP2 config: the replica pick follows #920's router
  
  dghosef
  and
  claude
  committed
  0bb4c9b
- Review wording: the vision split's capture-only scope, Qwen3.5 configs' start-up compile, Qwen3-Omni's encoder header
  
  dghosef
  and
  claude
  committed
  5d159d2
- Merge c3/859 into perf/split-3-generation
  
  dghosef
  and
  claude
  committed
  22b4d4f
- Wan2.2 docs: date the timings by the DiT step they measured, not a PR number
  
  dghosef
  and
  claude
  committed
  706f986
- Wan2.2 docs: an earlier version of the DiT block, not an earlier DiT step
  
  dghosef
  and
  claude
  committed
  96a4c16
- Merge c3/860 into perf/split-4-shared
  
  dghosef
  and
  claude
  committed
  d130853
- Engine: a capped deployment holds cuDNN's capture search to its share
  
  dghosef
  and
  claude
  committed
  70ee3c9
- Docs: the preprocess pool's CPU cap does not count a prepare's FFmpeg threads
  
  dghosef
  and
  claude
  committed
  20d02e1
- Chunk policy: leave the longer primed first chunk to #977
  
  dghosef
  and
  claude
  committed
  cdda4d5
- Merge c3/969 into perf/split-5-language
  
  dghosef
  and
  claude
  committed
  faeff0c
- ViT patch embed: the conv timing in its comment is Qwen3.5's
  
  dghosef
  and
  claude
  committed
  b3db573
- Engine: _search_cap's docstring separates a skipped search try from a failed sizing
  
  dghosef
  and
  claude
  committed
  ba1f4c1
- Qwen3.5-9B DP2 config: say the vision regions compile at start-up
  
  dghosef
  and
  claude
  committed
  ee79f9f
- Engine: capture_cudnn_benchmark's search runs in capture sizing, held to a capped deployment's share
  
  dghosef
  and
  claude
  committed
  4e385bc
- Fish S2 Pro: capture the decode buckets uncompiled, as on main
  
  dghosef
  and
  claude
  committed
  f48f70c
- Fish S2 Pro: TF32 codec decode is the model's default
  
  dghosef
  and
  claude
  committed
  e1e4d81
- Code predictor: open the depth loop with position 0 alone
  
  dghosef
  and
  claude
  committed
  a725282
- Qwen3-TTS: drop the 24- and 48-row decode buckets
  
  dghosef
  and
  claude
  committed
  fe5f304
- run_layers: the residual stream starts from its own copy
  
  dghosef
  and
  claude
  committed
  218c36f
- OmniVoice silence test: pydub is the oracle, installed with dev
  
  dghosef
  and
  claude
  committed
  1911f1c
- Code predictor: each Talker keeps its own embedding-sum dtype
  
  dghosef
  and
  claude
  committed
  3be750e
- Merge c3/859 into perf/split-3-generation
  
  dghosef
  and
  claude
  committed
  6c42b33
- Merge c3/860 into perf/split-4-shared
  dghosef
  committed
  8fd51ac
- Merge c3/969 into perf/split-5-language
  
  dghosef
  and
  claude
  committed
  c9cccc5
- Review wording: Qwen3-TTS's embedding sum, the cuDNN search in capture sizing; audioop-lts for pydub on 3.13
  
  dghosef
  and
  claude
  committed
  41a030c
- Merge c3/859 into perf/split-3-generation
  
  dghosef
  and
  claude
  committed
  0695600
- Merge c3/860 into perf/split-4-shared
  
  dghosef
  and
  claude
  committed
  b883052
- Merge c3/969 into perf/split-5-language
  
  dghosef
  and
  claude
  committed
  ecba1a4
- Merge origin/main (41c77c98c) into perf/split-2-speech-language
  
  dghosef
  and
  claude
  committed
  a55cfcf
- TokenStop: withholding outputs and the runtime check refuse each other
  
  dghosef
  and
  claude
  committed
  24bf74f
Commits on Nov 2, 2026
- test_knobs: take main's version back now that ingraph_decode_rope stays
  
  dghosef
  and
  claude
  committed
  387b08b
- Merge perf/split-2-speech-language (#859, 387b08b86) into perf/split-3-generation
  dghosef
  committed
  c6a2e09
- Wan2.2 CFG-parallel: the scaffold keeps ragged_for, docs say a rank runs its branch over the batch
  
  dghosef
  and
  claude
  committed
  dcc5b22
- BAGEL docs: quote the guided text-to-image PSNR and the configuration of each number
  
  dghosef
  and
  claude
  committed
  746a790
- wan22 extra: install flashinfer-python for the default ragged attention
  
  dghosef
  and
  claude
  committed
  96e4723
- Wan2.2: leave CFG-parallel to the cfg_size axis
  
  dghosef
  and
  claude
  committed
  5aaa217
- Z-Image-Turbo: capture batch sizes 1, 2, 4, 8; every size up to 8 is a commented opt-in
  
  dghosef
  and
  claude
  committed
  66538d4
- V-JEPA2-AC test: a rollout request's stamped clip shape keys the encoder capture
  
  dghosef
  and
  claude
  committed
  005d27a
- Merge perf/split-3-generation into fix/860
  
  dghosef
  and
  claude
  committed
  9dddee7
- Merge perf/split-3-generation (#860, 9dddee7a6) into perf/split-4-shared
  dghosef
  committed
  91d6ba3
- Merge perf/split-4-shared (#969, 91d6ba347) into perf/split-5-language
  dghosef
  committed
  bb70e92
- Qwen3.5-9B configs: keep main's 2176 bucket for a mixed step
  
  dghosef
  and
  claude
  committed
  3155847
- Merge main (#859 squash 696065149) into #860
  
  dghosef
  and
  claude
  committed
  6df580a
- Merge perf/split-3-generation (#860, 6df580a1d: main with #859) into perf/split-4-shared
  
  dghosef
  and
  claude
  committed
  7a2232d
- Merge main (fdfa1239a, the #860 squash) into perf/split-4-shared
  
  dghosef
  and
  claude
  committed
  d0dfd90
- Sampler RNG offset test: both captured kernels, each against a draw it can match
  
  dghosef
  and
  claude
  committed
  2fdfd95
- Data worker: refuse a bad preprocess_workers before the thread starts
  
  dghosef
  and
  claude
  committed
  a951da5
- Merge perf/split-4-shared (#969, a951da51c) into perf/split-5-language
  
  dghosef
  and
  claude
  committed
  0fac73a
- Merge origin/main (eceea1bee) into perf/split-4-shared
  
  dghosef
  and
  claude
  committed
  3e09745
- TokenStepNode: copy only the outputs the per-request stop check reads
  
  dghosef
  and
  claude
  committed
  a94779a
- Check every shipped prefill ladder, not only Qwen3.5's
  
  dghosef
  and
  claude
  committed
  3692210
- Merge perf/split-4-shared (#969, a94779ae3) into perf/split-5-language
  
  dghosef
  and
  claude
  committed
  85eb7c6
- Ladder test: pin the config-to-model plumbing, drop the Qwen3.5 repeat
  
  dghosef
  and
  claude
  committed
  0efb536
- Ladder test: say what the plumbing check covers, and why it fails
  
  dghosef
  and
  claude
  committed
  a3798f4
- Ladder test: read the Thinker's prefill capture by walk
  
  dghosef
  and
  claude
  committed
  f35812f
- Ladder test: dispatch on the model class, and name what it checks
  
  dghosef
  and
  claude
  committed
  7c9b2f3
Commits on Nov 3, 2026
- Merge origin/main (f3bce74692) into perf/split-5-language
  
  dghosef
  and
  claude
  committed
  1c6ac24
- AuT encoder: stage the trim indices with to_device_nonblocking
  
  dghosef
  and
  claude
  committed
  39d3f72
- Configs: drop prefill rungs no step reaches, and the restated speculation default
  
  dghosef
  and
  claude
  committed
  77834bc
- Thinker: start the residual stream from a copy of the input embeddings
  
  dghosef
  and
  claude
  committed
  4b9400d
- Qwen3.5: warn when MSTAR_QWEN35_PREFILL_TOKEN_BUCKETS is still set
  
  dghosef
  and
  claude
  committed
  849de9e

Commits

Commits on Oct 30, 2026

Commits on Oct 31, 2026