Add native DeepSeek-V4 hybrid attention orchestration

[FDecaYed]

Add native DeepSeek-V4 hybrid attention orchestration

Summary

Wire the D1 SBHD CSA primitives into native DeepSeek-V4 attention orchestration, module specs, grouped output, MTP propagation, and indexer-loss scheduling/logging.

Scope

• Add DSv4HybridAttention and DSv4HybridSelfAttention SBHD construction and execution.
• Add the DSv4 module-spec builder without depending on later HybridModel construction. The existing models/gpt filename is a shared experimental-attention factory and does not add GPTModel construction support.
• Propagate attention/MTP layer identity needed by the indexer-loss tracker.
• Use explicit ProcessGroupCollection inputs for cross-pipeline logging behavior.
• Add per-layer CSA ratio parsing/validation and grouped output projection configuration.
• Require dsa_kernel_backend='none' for this native slice; D3 adds the fused backend.
• Rebase the required AbsorbedMLA integration on the current-main implementation instead of replacing it with a dev copy.

Non-goals

• Concrete fused CSA kernels; D3 owns them.
• HybridModel symbol allocation and construction; D4 owns them.
• Packed THD, context parallelism, CUDA-graph execution, or recipes.
• GPTModel construction support.
• Changes to ordinary DSA semantics that are not architecture-neutral and independently necessary.

Review boundary and dependencies

Requires #6400 (D1) and P6/#5944. The local branch currently carries the complete D1 prerequisite line through c463cbed9919; this will still be an independent PR targeting main.

• Own commit: 0ecb394a67d2ae2b89c20d8d489c48fa5b5b1ef9
• Exact own diff: 14 files, +1,650/-28
• Temporary carried commit boundary: c463cbed9919bcc662fd5b4282546367f1852c32

Please review the own commit only. After D1 and #5944 merge, it will be replayed directly onto then-current main.

Provenance

Reconstructed from current main plus the post-#6020 DSv4 residual, using #5795 only as behavior/test evidence. It follows the current post-#6372 CSA namespace and preserves the teacher semantics established by #5960.

Validation

Passed locally:

• clean worktree and no dependency/lock-file changes;
• signed-commit verification and required Signed-off-by trailer;
• isort 5.13.2;
• Black 26.3.1 on applicable core/test files;
• Ruff;
• Python bytecode compilation;
• git diff --check;
• focused API and obsolete-import scans.

Runtime pytest is unavailable in the bare macOS environment because PyTorch/CUDA is not installed. Focused CUDA tests and full functional CI remain required after publication.

Local artifact

D2-native-dsv4-attention-orchestration.patch, generated from the own commit only. Verify it against SHA256SUMS.

Before publication

After D1 and #5944 merge, replay only D2’s own commit onto current main, re-audit any intervening ordinary-DSA and AbsorbedMLA changes, then rerun focused/static validation and full functional CI.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.