[hubert-marek]

FLA's identity-based metadata cache can skip operations during activation-checkpoint recomputation. This can misalign selective-checkpoint replay and fail backward with RuntimeError: Autograd not support dtype: Int (#3882).

Disable FLA tensor caching when apply_ac is called, for both full and selective activation checkpointing. This is a process-wide setting after AC is applied; other FLA users in that process also bypass the cache. The PR now changes only trainer/model.py.

Removed the CPU sequence-length arguments from Qwen3.5, Qwen3.8 and shared GatedDeltaNet. FLA keeps its existing GPU-to-CPU metadata preparation. On four H200s, a synthetic 237,568-token CP4 checkpointed GDN forward/backward measured 119.45 ms without CPU reuse versus 118.55 ms with it. The 0.90 ms difference was noisy (paired-bootstrap 95% interval: -0.38 to 2.37 ms). Scaling across 36 GDN layers estimates ~0.1% of a 30-second microbatch; this was an eager layer benchmark, not a full-model speedup measurement (Nebius job 5896).

Validation

• On one allocated H200 (Nebius job 5901, PyTorch 2.13.0+cu130 / FLA 0.5.2), all 19 focused model/checkpoint tests passed after removing the CPU arguments.
• All 6 cache-policy checks passed (full/selective AC, frequencies 1/2/4), including bypassing an already populated FLA cache.
• All 12 eager tiny-model cases and 8 compiled (fullgraph=false) checkpoint-backward cases passed for Qwen3.5 and Qwen3.8 with 2/4 packed sequences. Losses and embedding gradients matched no-AC references within the test tolerances. These are single-GPU checks, not a distributed production-training validation.
• Ruff, formatting and diff whitespace checks passed.

The generated summary below predates removal of the CPU sequence-length plumbing.

---

Note

Medium Risk
Changes global FLA behavior for all AC-enabled training runs and affects backward through FLA-backed layers, but the change is narrow and aligns replay with the original forward.

Overview
Disables FLA’s tensor/metadata cache whenever activation checkpointing is enabled, so the same FLA op sequence runs on the initial forward and on checkpoint recomputation.

apply_ac now imports fla.utils and sets FLA_DISABLE_TENSOR_CACHE = True before wrapping decoder layers. That is a process-wide policy for any model that uses AC (full or selective); models that never touch FLA are unaffected.

This targets backward failures such as RuntimeError: Autograd not support dtype: Int when selective AC replays layers that use FLA kernels (e.g. Qwen linear-attention / Gated DeltaNet paths).

^{Reviewed by Cursor Bugbot for commit c193e08. Bugbot is set up for automated code reviews on this repo. Configure here.}

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-21T17:15:34.793533Z	c193e08	New commits

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: abd27bd270

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit abd27bd. Configure here.}