justinchuby · Jul 31, 2026 · Jul 31, 2026
diff --git a/.squad/agents/cohaagen/history-archive.md b/.squad/agents/cohaagen/history-archive.md
@@ -0,0 +1,36 @@
+# cohaagen — History Archive
+
+_Archived by Scribe round 9, 2026-07-31T10:24:07Z._
+
+## 2026-07-29T00:45:00+0000 — PR #380 re-review
+- Re-reviewed Melina's encoder-decoder fixture correction for issue #377 and approved PR #380, merged as `47c3331d`.
+- Ran the CLI ORT E2E gate (23/23); metadata/I/O-detection reviews require that gate alongside engine/native unit tests.
+
+## 2026-07-30T09:16:00Z — 7B native CUDA perf findings
+
+- The Foundry baseline reports native CUDA ahead of ORT; 7B tracing localized o_proj at 19.5% of kernel time.
+- Reverted the two-way o_proj split-K gate after a repeatable 0.59% 7B regression; do not retry that lever without a new higher-split kernel experiment.
+
+## 2026-07-30T13:36:00Z — Issue #63 increment 2 delivered; #87 assessment complete
+
+- PR #444 merged: wired GPU weight pager into live decode hot-path with `ONNX_GENAI_WEIGHT_OFFLOAD=1`. Added `CudaWeightResidency` LRU device-page cache, env-gated device policy, extended lazy boundary to `com.microsoft::QMoE` + `MatMulNBits`.
+- Test: 0.6B native CUDA on 2MiB budget: 32-token decode 2.79 → 2.30 tok/s (1.21× slowdown); tokens identical; paging counters confirm page-ins/evictions.
+- Issue #87: assessed async H2D prefetch infrastructure — all async/copy-stream machinery exists and is GPU-tested. Missing only live wiring (residency currently synchronous). Inc1 = switch to async, Inc2 = double-buffered look-ahead. Unblocked, awaiting signal.
+
+## 2026-07-30T15:20:00Z — PR #480 merged (CUDA CausalConvWithState + GBQ coverage)
+
+- PR #480 merged (Melina APPROVED): NVRTC `CausalConvWithState` kernel (fp32/fp16/bf16, f32-accum to match ORT/CPU) + `GatherBlockQuantized` coverage declaration (#67). Oracle chain CUDA->CPU EP->ORT; advertised ops 161 -> 163. Closed the registered-but-undeclared GBQ coverage-of-coverage hole.
+- Empirical #67 audit: classic transformer decode is 100% covered on CUDA; control-flow (If/Loop/Scan) is executor-handled and must NOT be added to the EP; remaining real gaps are the Qwen3.5 hybrid family (CausalConvWithState landed, LinearAttention next).
+- Follow-up (safe-to-defer, fail-closed): GBQ bits=4 odd-blocks-per-row. In flight: LinearAttention (rank-2 gap), PR pending.
+
+## 2026-07-30T21:15:00Z — PR #484 + #525 merged (CUDA LinearAttention; #67 coverage-polish)
+
+- PR #484 merged: CUDA LinearAttention (Gated DeltaNet) kernel — per-thread f32-register-column state; 4/4 parity; qwen3.5 hybrid node placement 0→18/18/24. Coordinator resolved a merge conflict with #480 in kernels/mod.rs/provider.rs/docs (union of ops).
+- PR #525 merged (Melina APPROVED): #67 coverage-polish — RotaryEmbedding com.microsoft + fixed dtype-check bug (Int64 position_ids compared vs float); Bool NonZero on both EPs via `to_dense_bool`; GatherBlockQuantized odd-blocks-per-row LOUD fail-closed gate + honest doc softening. Op counts unchanged (no new op names).
+- Qwen3.5 hybrid recurrent op set now fully CUDA-covered (CausalConvWithState #480 + LinearAttention #484 + RoPE-contrib/NonZero #525).
+
+## 2026-07-31T00:25:00Z — PR #529 merged (qwen3.5-0.8b hybrid 100% CUDA placement)
+
+- PR #529 merged: qwen3.5-0.8b hybrid places 100% on CUDA — split package embedding.onnx 24 nodes + text.onnx 1265 = 1289 nodes, 0 declines (after #480/#484/#525). Regression lock `qwen35_0_8b_placement_lock`.
+- E2e decode still BLOCKED on the loader: `Engine::from_dir` rejects the 3-onnx split; `from_pipeline_dir` refuses during vision `smart_resize` admission. Parity harness `qwen35_0_8b_hybrid_native_cuda_e2e` graceful-skips until fixed.
+- In flight (cohaagen-4): loader-unblock — admit the text-only split hybrid for decode, flip the e2e parity harness active.
diff --git a/.squad/agents/cohaagen/history.md b/.squad/agents/cohaagen/history.md
@@ -1,47 +1,25 @@
 # cohaagen — History
 
-## 2026-07-29T00:45:00+0000 — PR #380 re-review
-- Re-reviewed Melina's encoder-decoder fixture correction for issue #377 and approved PR #380, merged as `47c3331d`.
-- Ran the CLI ORT E2E gate (23/23); metadata/I/O-detection reviews require that gate alongside engine/native unit tests.
+## Role
+CUDA performance and kernel specialist. Owns CUDA EP op-coverage (#67), weight-offload (#63/#87), kernel tuning (o_proj split-K, CausalConvWithState, LinearAttention), and decode profiling.
 
-## 2026-07-30T09:16:00Z — 7B native CUDA perf findings
-
-- The Foundry baseline reports native CUDA ahead of ORT; 7B tracing localized o_proj at 19.5% of kernel time.
-- Reverted the two-way o_proj split-K gate after a repeatable 0.59% 7B regression; do not retry that lever without a new higher-split kernel experiment.
-
-## 2026-07-30T13:36:00Z — Issue #63 increment 2 delivered; #87 assessment complete
-
-- PR #444 merged: wired GPU weight pager into live decode hot-path with `ONNX_GENAI_WEIGHT_OFFLOAD=1`. Added `CudaWeightResidency` LRU device-page cache, env-gated device policy, extended lazy boundary to `com.microsoft::QMoE` + `MatMulNBits`.
-- Test: 0.6B native CUDA on 2MiB budget: 32-token decode 2.79 → 2.30 tok/s (1.21× slowdown); tokens identical; paging counters confirm page-ins/evictions.
-- Issue #87: assessed async H2D prefetch infrastructure — all async/copy-stream machinery exists and is GPU-tested. Missing only live wiring (residency currently synchronous). Inc1 = switch to async, Inc2 = double-buffered look-ahead. Unblocked, awaiting signal.
-
-## 2026-07-30T15:20:00Z — PR #480 merged (CUDA CausalConvWithState + GBQ coverage)
-
-- PR #480 merged (Melina APPROVED): NVRTC `CausalConvWithState` kernel (fp32/fp16/bf16, f32-accum to match ORT/CPU) + `GatherBlockQuantized` coverage declaration (#67). Oracle chain CUDA->CPU EP->ORT; advertised ops 161 -> 163. Closed the registered-but-undeclared GBQ coverage-of-coverage hole.
-- Empirical #67 audit: classic transformer decode is 100% covered on CUDA; control-flow (If/Loop/Scan) is executor-handled and must NOT be added to the EP; remaining real gaps are the Qwen3.5 hybrid family (CausalConvWithState landed, LinearAttention next).
-- Follow-up (safe-to-defer, fail-closed): GBQ bits=4 odd-blocks-per-row. In flight: LinearAttention (rank-2 gap), PR pending.
-
-## 2026-07-30T21:15:00Z — PR #484 + #525 merged (CUDA LinearAttention; #67 coverage-polish)
-
-- PR #484 merged: CUDA LinearAttention (Gated DeltaNet) kernel — per-thread f32-register-column state; 4/4 parity; qwen3.5 hybrid node placement 0→18/18/24. Coordinator resolved a merge conflict with #480 in kernels/mod.rs/provider.rs/docs (union of ops).
-- PR #525 merged (Melina APPROVED): #67 coverage-polish — RotaryEmbedding com.microsoft + fixed dtype-check bug (Int64 position_ids compared vs float); Bool NonZero on both EPs via `to_dense_bool`; GatherBlockQuantized odd-blocks-per-row LOUD fail-closed gate + honest doc softening. Op counts unchanged (no new op names).
-- Qwen3.5 hybrid recurrent op set now fully CUDA-covered (CausalConvWithState #480 + LinearAttention #484 + RoPE-contrib/NonZero #525).
-- In flight: hybrid-e2e.
-
-## 2026-07-31T00:25:00Z — PR #529 merged (qwen3.5-0.8b hybrid 100% CUDA placement)
-
-- PR #529 merged: qwen3.5-0.8b hybrid places 100% on CUDA — split package embedding.onnx 24 nodes + text.onnx 1265 = 1289 nodes, 0 declines (after #480/#484/#525). Regression lock `qwen35_0_8b_placement_lock`.
-- E2e decode still BLOCKED on the loader: `Engine::from_dir` rejects the 3-onnx split; `from_pipeline_dir` refuses during vision `smart_resize` admission. Parity harness `qwen35_0_8b_hybrid_native_cuda_e2e` graceful-skips until fixed.
-- In flight (cohaagen-4): loader-unblock — admit the text-only split hybrid for decode, flip the e2e parity harness active.
+_Entries before 2026-07-31T03:03:15Z archived to `history-archive.md` (Scribe round 9). Archived: PR #380 re-review; 7B perf (o_proj split-K revert); #63 inc2 (#444 weight pager); #480 (CausalConvWithState+GBQ); #484+#525 (LinearAttention+RoPE-contrib/NonZero); #529 (qwen3.5 100% CUDA placement)._
 
 ## 2026-07-31T03:03:15Z — PR #535 merged: text-only decode pipeline synthesis (hybrid loader unblock)
 
-- PR #535 merged: unblocks the split VLM package whose image preprocessing is unrepresentable (`smart_resize`). New `GenAiConfigError::UnrepresentablePreprocessing` (distinct from `IncompletePipeline`) → `to_strict_text_only_pipeline_metadata` synthesizes an embedding→decoder AR pipeline with NO vision component (positions rank-3 `linear_increment`, decoder `inputs_embeds`). Modality-driven, NOT a model-name case. Resolves the symbolic leading (batch) axis in `decode/values.rs`. Flips the `qwen35_0_8b_hybrid_native_cuda_e2e` parity harness active (paved the way for Mary's #543 landmark).
-- Weight-offload chain (#63): #444 first increment merged — `page_lazy_weight` dispatch-layer seam + `CudaWeightResidency` LRU, gated behind `ONNX_GENAI_WEIGHT_OFFLOAD=1`, token-identical, ~1.21× slowdown at 2 MiB. NEXT LEVER: #87 async page-in (PLAN-ONLY, awaiting Justin green-light).
-- Guardrail proven: o_proj 2-way split-K (K_SPLIT=2) REGRESSES the 7B o_proj GEMV (−0.59%, repeatable) — do NOT re-try; reduction tax > sub-wave grid-fill. A K_SPLIT>2 new kernel with its own A/B is the future candidate.
+- PR #535 merged: unblocks the split VLM package whose image preprocessing is unrepresentable (`smart_resize`). New `GenAiConfigError::UnrepresentablePreprocessing` → `to_strict_text_only_pipeline_metadata` synthesizes an embedding→decoder AR pipeline with NO vision component. Modality-driven, NOT a model-name case. Flips the `qwen35_0_8b_hybrid_native_cuda_e2e` parity harness active.
+- Weight-offload chain (#63): #444 first increment merged — `page_lazy_weight` dispatch-layer seam + `CudaWeightResidency` LRU, gated behind `ONNX_GENAI_WEIGHT_OFFLOAD=1`. NEXT: #87 async page-in (PLAN-ONLY, awaiting Justin green-light).
+- Guardrail proven: o_proj 2-way split-K (K_SPLIT=2) REGRESSES the 7B o_proj GEMV (−0.59%) — **do NOT re-try**. A K_SPLIT>2 new kernel is the future candidate.
 
 ## 2026-07-31T08:48:28Z — PR #544 async page-in MERGED; #552 observability MERGED; GQA capture validated
 
 - PR #544 merged (Harry test fix; Melina RC→APPROVE): async `cuMemcpyHtoDAsync`+`compute_wait_fence` page-in; eviction re-serializes (WAR); ~1% win at 2 MiB only; byte-identical all budgets. `ONNX_GENAI_WEIGHT_OFFLOAD_ASYNC_PAGEIN` default ON.
 - PR #552 merged (Lori APPROVE): `NativeDecodeSession::load_with_resolved_io` — genai_config decoders now emit `cuda_graph captures/fallbacks/fallback_report` + `--trace` capture-reject reasons. `profile_native` simple path now fully observable.
 - GQA capture default-on validated (verification-only, no code change): Qwen2.5-0.5B int4 GQA — ON 2.14× eager, byte-identical, zero declines. Standing question answered. Gemma-4-E2B unmeasurable: stale HF export + GAP-3. Capture default-on for multi-component deferred until mary's native pipeline + re-export.
+
+## 2026-07-31T10:24:07Z — 27B decode roofline profile (profile-only, no code changed)
+
+- Profiled Qwen3.6-27B int4 native-CUDA resident decode on H200. Steady-state: 168 ms/tok (~35× off roofline).
+- Root cause: `Scan` (48 LinearAttention blocks/step) = **56.5%** of decode step — structurally un-capturable control-flow. MatMulNBits already at roofline (4.4 ms, 2%) — **do NOT touch for this**.
+- Capture engages (fallbacks=0); Transpose per-call sync 9.2% is next lever in Cohaagen's lane (separate from Scan fix).
+- Fix is Justin's / pipeline-control-flow area. This was a profile+identify task only; no worktree remains.
diff --git a/.squad/agents/mary/history.md b/.squad/agents/mary/history.md
@@ -29,3 +29,10 @@ _Entries before 2026-07-30T21:15 archived to `history-archive.md` (Scribe round
 - **27B offload A/B (H200):** Qwen3.6-27B int4 (497 MatMulNBits). 2 GiB budget → 6.2 GiB peak VRAM (2.9× vs 17.7 GiB resident), byte-exact at all budgets. Cliff: ≤12 GiB → 0.11 tok/s (bandwidth-bound, whole working set evicted/token). cuda_graph auto-off under offload. Action item: add `model.io` block to canonical int4-cuda package.
 - **#553 bug found (#554 MERGED, Harry APPROVED):** `NativeDecodeSession` 2nd+ generation returns garbage — conv_state/recurrent_state not zeroed between generates. Not graph-capture related. Fixed.
 - **Native pipeline keystone test:** `native_full_pipeline_parity` drives `tiny-gemma4-vlm` with native embedding + native decoder simultaneously → `[0,5,6,7] == ORT`. Closes composite-native safety gap ahead of 35B-A3B wiring.
+
+## 2026-07-31T10:24:07Z — Scan-capture increment assessed: STOP+report; right-sized workstream proposed
+
+- Reproduced 176.7 ms/tok baseline (matches Cohaagen ±5%). Confirmed `--trace` Scan seam.
+- Single-trip Scan inline is NOT an increment: (1) shared prefill+decode plan — static inline corrupts prefill; (2) control-flow structurally declined at `provider.rs:458`; (3) no child-body-fold machinery.
+- **Right-sized: Approach 1 (runtime dual-path)** — 1a correctness behind flag, 1b body enters capture. Blast radius: #443/#543 core. Reference decode tokens locked. **Awaiting Justin go-ahead.**
+- No code changed; worktree `wt-mary-scan` removed.
diff --git a/.squad/decisions-archive/2026-07.md b/.squad/decisions-archive/2026-07.md
@@ -23293,4 +23293,17 @@ narrative below). Wave: #535, #540, #541, #543 all MERGED to origin/main; contex
 for #377; native side complete via #382/#412). Held: **#534** (Harry, server contracts #481/#482,
 Melina APPROVED) targets Justin's branch `feat/genai-demo-dashboard` (PR #476), not on main.
 In flight: gemma-3n vision-optional text-only decode; #87 async weight prefetch (awaiting
-green-light); 35B-A3B routed-expert MoE representation.
+green-light); 35B-A3B routed-expert MoE representation.
+---
+
+## 2026-07-31 — Scribe consolidation (round 8)
+
+**By:** Scribe. Merged: #544 #552 #554. Wave narrative in archive.
+
+- **#544 async page-in inc1** (Cohaagen; Harry test fix; Melina RC→APPROVE): `cuMemcpyHtoDAsync`+`compute_wait_fence`; eviction re-serializes (WAR). ~1% at 2 MiB; byte-identical all budgets. `ONNX_GENAI_WEIGHT_OFFLOAD_ASYNC_PAGEIN` default ON.
+- **#552 profile_native observability** (Cohaagen; Lori APPROVE): `load_with_resolved_io` — genai_config decoders print `cuda_graph captures/fallbacks/fallback_report` + `--trace` capture-reject reasons.
+- **GQA capture validated** (Cohaagen): Qwen2.5-0.5B int4 GQA ON 2.14× eager, byte-identical, zero declines. Gemma-4-E2B: stale export (vision fn 2 vs 4 outputs) + GAP-3 (native pipeline unimplemented).
+- **27B offload A/B** (Mary, H200): Qwen3.6-27B int4, 2 GiB → 6.2 GiB peak VRAM (2.9× vs 17.7 GiB), byte-exact. Cliff: ≤12 GiB → 0.11 tok/s. cuda_graph off under offload.
+- **#553/#554 session-reuse fix** (Mary; Harry APPROVED): conv_state/recurrent_state not zeroed between generates in `NativeDecodeSession`. Not graph-capture related.
+- **Native pipeline keystone** (Mary): `native_full_pipeline_parity` — native embedding + native decoder → `[0,5,6,7] == ORT` (`tiny-gemma4-vlm`).
+In flight: #87 inc2 double-buffer; native paged-KV; 35B-A3B MoE; gemma-3n text-only.
diff --git a/.squad/decisions.md b/.squad/decisions.md
@@ -1,6 +1,6 @@
 # Decisions — live standing directives
 
-Last consolidated: 2026-07-31T08:48:28Z (Scribe round 8 — #544 async page-in inc1, #552 observability, #554 session-reuse fix, 27B offload A/B, GQA capture; round-7+MoE in archive)
+Last consolidated: 2026-07-31T10:24:07Z (Scribe round 9 — 27B decode profile Scan=56.5%, Scan-capture scoping PENDING JUSTIN, #554 session-reuse merged; round-8 in archive)
 
 Standing governance rules and constraints. Dated wave records and historical ledger updates
 are archived to `.squad/decisions-archive/2026-07.md`.
@@ -260,14 +260,21 @@ NEVER `to_raw_bytes()`. Unblocked the gemma-3n Bool audio mask.
   repeatable). Do NOT re-try that lever** (reduction tax > sub-wave grid-fill); a K_SPLIT>2 new
   kernel with its own A/B is the future candidate.
 
-## 2026-07-31 — Scribe consolidation (round 8)
+## 2026-07-31 — 27B decode profile + Scan-capture scoping (round 9)
 
-**By:** Scribe. Merged: #544 #552 #554. Wave narrative in archive.
+**By:** Scribe. Round-8 (#544/#552/#554/27B-A/B/GQA) → archive.
 
-- **#544 async page-in inc1** (Cohaagen; Harry test fix; Melina RC→APPROVE): `cuMemcpyHtoDAsync`+`compute_wait_fence`; eviction re-serializes (WAR). ~1% at 2 MiB; byte-identical all budgets. `ONNX_GENAI_WEIGHT_OFFLOAD_ASYNC_PAGEIN` default ON.
-- **#552 profile_native observability** (Cohaagen; Lori APPROVE): `load_with_resolved_io` — genai_config decoders print `cuda_graph captures/fallbacks/fallback_report` + `--trace` capture-reject reasons.
-- **GQA capture validated** (Cohaagen): Qwen2.5-0.5B int4 GQA ON 2.14× eager, byte-identical, zero declines. Gemma-4-E2B: stale export (vision fn 2 vs 4 outputs) + GAP-3 (native pipeline unimplemented).
-- **27B offload A/B** (Mary, H200): Qwen3.6-27B int4, 2 GiB → 6.2 GiB peak VRAM (2.9× vs 17.7 GiB), byte-exact. Cliff: ≤12 GiB → 0.11 tok/s. cuda_graph off under offload.
-- **#553/#554 session-reuse fix** (Mary; Harry APPROVED): conv_state/recurrent_state not zeroed between generates in `NativeDecodeSession`. Not graph-capture related.
-- **Native pipeline keystone** (Mary): `native_full_pipeline_parity` — native embedding + native decoder → `[0,5,6,7] == ORT` (`tiny-gemma4-vlm`).
+- **27B native-CUDA decode: Scan is bottleneck** (Cohaagen; profile-only): 168 ms/tok (~35× off roofline). `Scan` (48 LinearAttention blocks/step) = **56.5%**, structurally un-capturable. MatMulNBits at roofline (4.4 ms, 2%). Ceiling: **~15–30× speedup** if Scan enters capture/fuse. NOT a kernel fix.
+- **#554 MERGED** (Mary; Harry APPROVED): `DecodeCudaState.rewind(0)` re-zeroes `fixed_state_binding_range`; pure-KV models unaffected (empty range).
+
+## ⚑ PENDING JUSTIN: 27B Scan→CUDA-capture (Mary; no code changed, awaiting go-ahead)
+
+Structurally larger than an increment — blockers: (1) shared prefill+decode plan; seq=1 inline corrupts prefill. (2) Control-flow declined at `provider.rs:458`, no trip_count exemption. (3) Child bodies never fold into parent plan.
+
+**Approach 1 — runtime dual-path** (only correct+feasible path):
+- **1a** (flag-gated): inline body into parent plan alongside Scan; runtime trip_count==1 picks body. Correctness-only, no capture.
+- **1b**: body enters capture; validate captures/replays rise; assert 27B tokens byte-identical to locked reference.
+- Blast radius: #443/#543 core. Prefill MUST be validated (shared-plan is the correctness tripwire).
+
+**Baseline + locked reference tokens ready.** Awaiting Justin go-ahead.
 In flight: #87 inc2 double-buffer; native paged-KV; 35B-A3B MoE; gemma-3n text-only.