Repository navigation

cake_gdn: deterministic chunked GDN training kernels (--linear-attention-backend cake) for the head-sharded Qwen GDN layer - #3597

Open

yyihuang wants to merge 35 commits into

radixark:mainfrom

yyihuang:feat/qwen-gdn-deterministic-backend

Open

cake_gdn: deterministic chunked GDN training kernels (--linear-attention-backend cake) for the head-sharded Qwen GDN layer#3597
yyihuang wants to merge 35 commits into
radixark:mainfrom
yyihuang:feat/qwen-gdn-deterministic-backend

Commits on Sep 22, 2026
- Qwen GDN: add the generated deterministic chunked GDN training kernels (SM100a/SM103a)
  
  averyhNV
  and
  claude
  committed
  949f4e3
- Qwen GDN: head-sharded unified GDN core and the 'loom' deterministic backend
  
  averyhNV
  and
  claude
  committed
  9aa01cf
- Qwen GDN: format with black/isort; bf16 modules in the TP/CP parity test; tests/fast/models package
  
  averyhNV
  and
  claude
  committed
  906a9d1
- Qwen GDN: build the fla CP context inside the head-sharded core when only cp_group is set
  
  averyhNV
  and
  claude
  committed
  96b09d6
- Qwen GDN e2e: skip the visual tower in the post-update weight equality check
  
  averyhNV
  and
  claude
  committed
  4434cc8
- Qwen GDN e2e: record/replay mode for training-side bit determinism
  
  averyhNV
  and
  claude
  committed
  9d75fb0
- Qwen GDN e2e: dump per-rank train data in replay mode
  
  averyhNV
  and
  claude
  committed
  a6d7601
- Qwen GDN loom backend: prebuild all kernel stages in parallel on first use
  
  averyhNV
  and
  claude
  committed
  eda2814
- Qwen GDN tests: register in the existing stage-c-8-gpu-b200 suite
  
  averyhNV
  and
  claude
  committed
  48d3a00
Commits on Oct 9, 2026
- Merge origin/main into feat/qwen-gdn-deterministic-backend
  
  averyhNV
  and
  claude
  committed
  6a849af
- loom GDN: run the short conv on fla's Triton kernels so the layer's gradients are reproducible
  
  averyhNV
  and
  claude
  committed
  7020a92
- Merge origin/main (B200 CI scheduling by minimum GPU count) into feat/qwen-gdn-deterministic-backend
  
  averyhNV
  and
  claude
  committed
  c5d7db9
- ci: register the loom GDN B200 tests on the 4-GPU stage with num_gpus=4
  
  averyhNV
  and
  claude
  committed
  9437f0b
- loom GDN: one chunk table per micro-batch, keep the WY/state tensors on the tape
  
  averyhNV
  and
  claude
  committed
  2cc3aa0
- tests: create the GDN wiring test's autograd leaves outside the profiled region
  
  averyhNV
  and
  claude
  committed
  af4b3e3
Commits on Oct 10, 2026
Commits on Oct 11, 2026
Commits on Oct 12, 2026
- gdn_chunk_train: round-11 kernels (loom 1852a7e39eb) — eight-warp WY pullback on a padded-pitch pool with stmatrix.x4 tiles and packed f32x2 epilogues
  
  averyhNV
  and
  claude
  committed
  7fb04f9
Commits on Oct 13, 2026
- gdn_chunk_train: round-13 kernels (loom 9a9653333cc) — forward WY on a padded-pitch pool with trace-time address folding and a line-aligned cp.async plan
  
  averyhNV
  and
  claude
  committed
  40ea59b
Commits on Oct 14, 2026
- gdn_chunk_train: launch the split-plan forward scan (round-10 host dispatch)
  
  averyhNV
  and
  claude
  committed
  0993335
Commits on Oct 15, 2026

Commits

Commits on Sep 22, 2026

Commits on Oct 9, 2026

Commits on Oct 10, 2026

Commits on Oct 11, 2026

Commits on Oct 12, 2026

Commits on Oct 13, 2026

Commits on Oct 14, 2026

Commits on Oct 15, 2026