feat(transformer): add native manifold hyper-connections

[FDecaYed]

What

Add the model-neutral native PyTorch manifold hyper-connection primitives:

• expand/reduce residual-stream operations and learned mappings;
• Sinkhorn projection and initialization controls;
• selective output-discarding recompute through CheckpointManager;
• TransformerConfig fields and compatibility validation;
• focused numerical, gradient, recompute, and serialization tests.

This slice deliberately excludes fused kernels and excludes TransformerBlock, TransformerLayer, GPTModel, HybridModel, pipeline, MTP, and recipe wiring. Those concerns belong to smaller dependent slices.

Stack

• Depends on feat(recompute): coordinate checkpoint-without-output recomputation #5935.
• Temporary review base: pull-request/5935.
• Files changed contains only the mHC primitive/config/test slice.
• Before feat(recompute): coordinate checkpoint-without-output recomputation #5935 merges, retarget and refresh this PR to main to preserve review history.

Provenance

• Reconstructs the native mHC work from [dev] feat(mHC): Add basic pytorch implementation of manifold hyper connection(mHC). #2943 and the current main proposal [Split 1/N of #3430] feat(mHC): basic pytorch implementation of manifold hyper connection #4531.
• Refactors the same implementation captured in [Tracking] DeepSeek-v4 HybridModel merge plan #5795 (commit 07c0d08).
• Original implementation by the [dev] feat(mHC): Add basic pytorch implementation of manifold hyper connection(mHC). #2943/[Split 1/N of #3430] feat(mHC): basic pytorch implementation of manifold hyper connection #4531 authors; DSv4 adaptation by @hxbai.
• Feature cutoff: 2026-07-21. GPTModel integration is intentionally excluded.

Testing

• uv run isort --check-only on all changed Python files
• uv run ruff check on all changed Python files
• uv run python -m compileall -q on all changed Python files
• Focused tests are included; local execution requires the project Torch runtime and is left to stack CI.

Stack follow-up

• feat(pipeline): carry mHC residual streams between stages #5943 contains the PP/VPP tensor-shape slice and temporarily targets this PR mirror.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[FDecaYed]

/ok to test c319ece

[FDecaYed]

Closing this draft until Round 1. Its logical successor is P2: complete fused/FP32 mHC with HybridModel and MTP integration, rebuilt after #4531 and P1 merge. It will target main directly rather than remain stacked. Tracking PR: #5795.