Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .squad/agents/deckard/history.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,3 +81,9 @@ After #978 split-K and #981 block SkipRMSNorm landed, the post-#981 base-floor r

- New implementation target: topology-gated capacity-policy fix for V2-Lite's additive attention-mask builder so CUDA graph capture can engage on MoE without regressing GLM-5.2 logical-width masks.
- Expected surface is executor geometry/build tests plus capture eligibility, not CUDA kernels; Rachael review and Wallace byte-identity/perf validation are required.

## 2026-08-18T03:15Z — V2-Lite classifier lockout episode recorded

- Authored the first V2-Lite additive-mask capacity classifier for CUDA graph capture; the target topology and GLM-5.2 exclusion were directionally correct.
- Rachael rejected the artifact for two safety blockers: non-capacity Attention could be blessed, and a root graph-output mask escape was not rejected. Strict reviewer lockout was honored, so Deckard did not revise the rejected artifact.
- Wallace owned the revision, tightened both cases with negative tests, and Rachael approved; #1171 merged as `bc1e97ff`.
6 changes: 6 additions & 0 deletions .squad/agents/leon/history.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,9 @@ tok/s**, capture fully engaged (1 segment / 0 seams).
- Rachael's initial 🔴 on the silent golden move triggered reviewer lockout; Leon did not revise the rejected artifact. Lockout cleared on merge.
- Final outcome: V2-Lite MoE E2E is unblocked; correctness is gated by a native-CUDA/f64-justified golden rather than CPU bit-identity.


## 2026-08-18T03:15Z — V2-Lite `_d1` planner fix landed; Engine long-context follow-up

- PR #1181 landed as `c9c7f64c`, fixing the V2-Lite additive-mask query-axis workspace-planner under-resolution by deriving exact `_d1` shape through deterministic mask-cone producers.
- Rachael approved the planner path as exact, bounded, and fail-closed; Wallace's final real-model A/B showed the combined classifier+planner unlock is byte-identical over 320 tokens and 1.79× faster under capture.
- Follow-up assigned: long-context `Engine::generate` Attention workspace under-plan on node 38 (requires 33288 bytes vs prepared 16904 around ~320 tokens), reproducing in eager and capture and therefore not capture-specific.
6 changes: 6 additions & 0 deletions .squad/agents/luv/history.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,3 +84,9 @@ Fresh adversarial review of `nxrt/mlas-avx2-layernorm` @ `72e02cd92c` after the

- Rebuilt QMoE gate/up occupancy on #1150 baseline as `ONNX_GENAI_QMOE_OCC` default-OFF, symmetric int4 only; `(256,6)` launch bounds are spill-free and cut live gate/up kernel ~10.6%.
- Byte-identity is green and V2-Lite E2E converts modestly (~+1%, 56.6→57.2 tok/s); down/grouped paths are already saturated or different-lane, so no remaining QMoE occupancy vein.

## 2026-08-18T03:15Z — QMoE gate/up occupancy lever merged (#1167)

- PR #1167 landed on `main` as `651901a3`: `ONNX_GENAI_QMOE_OCC` remains default-OFF and uses launch-bounds/occupancy tuning for QMoE gate/up activation.
- Evidence recorded: byte-identical QMoE suite and V2-Lite golden streams, registers about 54→40, achieved occupancy about 43%→64%, isolated gate/up kernel about −10.6%, and modest ~+1% E2E on V2-Lite.
- Rachael approved with a scope wording note: the flag is not literally limited to symmetric int4, even though default-OFF and affine smoke tests made that non-blocking.
6 changes: 6 additions & 0 deletions .squad/agents/rachael/history.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,3 +80,9 @@ All five red CI jobs trace to pre-existing `onnx-genai-server` failure on `main`
- Established precedent: int4 GEMV/QMoE CPU bit-identity is not the oracle when CPU and CUDA use different f32 accumulation orders; use f64-bounded evidence plus deterministic backend output and explicit golden rationale.
- Final merge: PR #1150 landed on `main` at squash `e075a715`; reviewer lockout was honored throughout.


## 2026-08-18T03:15Z — QMoE/classifier/planner review gates closed

- Approved Luv's default-OFF QMoE occupancy lever (#1167) with a non-blocking scope caveat.
- Rejected Deckard's first V2-Lite mask classifier for two safety blockers and enforced reviewer lockout; re-approved Wallace's tightened revision after present-KV and root graph-output negative tests.
- Approved Leon's `_d1` additive-mask query-axis workspace-planner fix (#1181) as exact and fail-closed; Wallace's hardware A/B then closed the DeepSeek MoE performance mandate at 1.79× capture/eager with 0.000% divergence.
36 changes: 36 additions & 0 deletions .squad/agents/wallace/history.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,43 @@
Approved PR #365 after four rounds, including recursive scanning and uniform structural identity.

Full pre-compaction history in `history-archive.md`.

## 2026-08-17T16:25Z — PR #1134 validation and ORT-gap measurement

- Measured native base decode ahead of ORT-CUDA on int4 Phi-4-mini (1.33×), qwen7b (1.13×), and qwen14b (1.80×), with mandatory caveat that native used captured decode while the ORT harness was eager. Profiling showed int4 MatMulNBits GEMV dominates decode (~77% GPU time).
- Independently validated Luv's GEMV pipeline: qwen14b 157.39→166.22 tok/s (+5.6%), qwen7b 306.04→312.80 tok/s (+2.2%), and pipe ON/OFF greedy streams were identical over 300 tokens per model.
- Outcome: validation green; PR #1134 merged.

## 2026-08-17T17:10Z — Independent validation: gateup-vec (PR #1137), GREEN

- Validated `e54cae31` in a fresh detached worktree (did not reuse Luv's GPU3); pinned idle GPU5/6; 5 process invocations per config, measurement only.
- Byte-identity (decisive gate): 0.000% divergence over 300 greedy tokens on both qwen2.5-14b and qwen2.5-7b → GREEN.
- Perf: qwen7b +1.4% clean (313.55 vs 309.24, no overlap); qwen14b break-even within noise (164.75 vs 165.37, median −0.4%). Luv's +0.82%/14b did NOT reproduce — 14b variance (~±3 tok/s) exceeds a sub-1% effect; big-model decode is memory-latency-bound, so −7.4% ALU ops has limited E2E leverage. Recommended not advertising a headline 14b number. Merged as `70cc06ad`.

## 2026-08-17T18:05Z — Independent validation: gateup-occ (PR #1139), GREEN

- Validated `squad/gateup-rms-stage @ 11a01fae` in a fresh detached worktree `.worktrees/wallace-gateup-occ` (did NOT reuse Luv's worktree/GPU); pinned idle H200 GPU5 (14b) / GPU6 (7b); measurement only. Interleaved A/B, 5 rounds (OCC=1 then OCC=0 back-to-back, `--steady --tokens 128 --warmups 1 --runs 3`) to control host/thermal drift.
- Byte-identity (decisive gate): 0.000% divergence over 300 greedy tokens on both qwen2.5-14b and qwen2.5-7b → GREEN.
- Perf: 14b OCC=1/OCC=0 median 171.19/166.82 = 1.026x (+2.6%), every round OCC=1 wins (HELD 5/5, min-OCC1 168.46 > max-OCC0 168.29) — confirms Luv's +2.4%. 7b 313.76/313.88 = 0.9996 (~flat, ±0.2%, no regression — 7b GEMV not occupancy-bound at that size).
- Verdict GREEN / MERGE-ELIGIBLE — byte-identical both models, +2.6% on 14b, no 7b regression. Did NOT run ncu (optional; regs 32/82% is Luv's isolated claim) — E2E + byte-identity both pass decisively. Merged as `0636a759`.

## 2026-08-17T18:40Z — Independent validation: gateup-preperm safety GREEN, perf NO-GO

- Validated `squad/gateup-preperm @ 6629f0aa`: byte-identity 0.000% over 300 greedy tokens on both qwen2.5-14b and qwen2.5-7b. Perf did not reproduce: 14b +0.27% median with 2/5 regressions and widened variance; 7b flat. Recommended no default-ON; coordinator shelved/not merged.

- 2026-08-17T21:40Z — Assigned ORT CUDA-graph fairness follow-up to verify native-vs-ORT decode claims under equal graph-capture conditions.
## 2026-08-17T22:20Z — ORT CUDA-graph fairness verdict

- Completed the native-vs-ORT CUDA-graph fairness study. Fixed ORT auto-discovery by pinning `ONNX_GENAI_ORT_LIB` to CUDA ORT and using `ONNX_GENAI_EP_FALLBACK=1` for shape nodes.
- Found stock ORT CUDA EP cannot capture a replayable graph on these int4 genai decode models; graph-vs-graph is therefore an honest capability gap.
- Eager-vs-eager medians: native 1.28× ORT on qwen14b only via on-GPU argmax, but 0.76× on qwen7b and 0.86× on Phi; kernel-only native is 0.75–0.87× ORT. Verdict: native's production lead is graph-capture + device-sampling architecture, not per-kernel speed.
## 2026-08-18T01:35Z — V2-Lite MoE measurement + graph-capture scope

- Measured corrected DeepSeek-V2-Lite int4 MoE: native CUDA eager median ~55.6 tok/s; graph flag currently no-ops because capture declines on `attention_mask_consumers_are_capacity_aware`; ORT CUDA lacks QMoE kernels and falls back to CPU experts at ~0.20 tok/s.
- Scoped V2-Lite graph-capture unlock as GO: topology-gated capacity policy for additive-mask-builder → capacity-form `Attention[3]`, with GLM-5.2 negative guard intact; post-implementation Wallace owns byte-identity/perf A/B.

## 2026-08-18T03:15Z — V2-Lite graph-capture classifier revised and final A/B green

- Under reviewer lockout, revised Deckard's rejected V2-Lite additive-mask capacity classifier by requiring present KV inputs and rejecting root graph-output mask escapes; Rachael re-approved and #1171 merged as `bc1e97ff`.
- After Leon's `_d1` planner fix, measured the real 27-layer DeepSeek-V2-Lite int4 QMoE artifact: capture ON vs eager OFF was byte-identical over 320 tokens and median 101.80 vs 56.94 tok/s (1.79×), with every capture run faster than every eager run.
- Flagged a separate long-context Engine Attention workspace under-plan for Leon; it reproduces with graph capture disabled and is not a classifier/capture regression.
Loading
Loading