Skip to content

Move in-model CP layout helpers into miles_plugins/models/cp_utils.py - #1814

Merged
yueming-yuan merged 6 commits into
mainfrom
yueming/cp-layout-utils-to-plugins
Sep 22, 2026
Merged

yueming-yuan merged 6 commits into
mainfrom
yueming/cp-layout-utils-to-plugins

Conversation

@yueming-yuan

@yueming-yuan yueming-yuan commented Jul 27, 2026 •

Copy link
Copy Markdown
Collaborator

What

Pure refactor, no behavior change.

miles/backends/training_utils/cp_utils.py owns the CP helpers the training backend applies around the model — slice_with_cp, natural_to_zigzag_slice, slice_loss_masks_for_local_cp, all_gather_with_cp, allgather_cp_redistribute, get_logits_and_tokens_offset_with_cp, … All of them operate on tokens / masks / logprobs / logits outside forward(), and are called from loss.py, data.py, losses.py.

build_gdn_cp_context was the one exception. It runs inside a layer's forward, its only callers are miles_plugins/models/qwen3_next.py and qwen3_5.py, and it forced an optional flash-linear-attention dependency into a core training module:

try:
    from fla.ops.cp import build_cp_context as _fla_build_cp_context
except ImportError:
    _fla_build_cp_context = None

The zigzag ↔ contiguous-shard relayout has the same shape but the opposite problem: it lived as private helpers inside miles_plugins/models/hf_attention.py, so only models built on HuggingfaceAttention could reach it. It is not HF-attention-specific — it bridges Megatron's zigzag CP layout to the contiguous rank-local shard every fla CP operator expects, and any model with an fla linear-attention layer needs it.

Change

  • miles_plugins/models/cp_utils.py becomes the module for CP helpers that run inside a forward: build_gdn_cp_context (moved from core) plus the relayout helpers zigzag_to_packed_shard / packed_shard_to_zigzag and their internals (moved from hf_attention.py, underscore prefix dropped).
  • detect_and_setup_hybrid_cp moves the other way, out of miles_plugins/models/cp_utils.py into hf_attention.py. It is defined in terms of isinstance(module, HuggingfaceAttention), so it belongs next to that class — and moving it is what breaks the import cycle hf_attention → cp_utils → hf_attention.
  • Core cp_utils.py loses build_gdn_cp_context, the fla.ops.cp try/except, and its now-unused torch.nn import.
  • Import sites updated: qwen3_next.py, qwen3_5.py, miles/backends/megatron_utils/actor.py, tests/e2e/precision/test_hf_attention_cp_relayout.py.

Verification

  • Every relocated definition is byte-identical to its original apart from the underscore renames — checked by extracting both versions with ast and comparing source text (10/10 identical).
  • pre-commit run --all-files clean.
  • Static module-level import-graph check: no cycles involving any touched module. miles_plugins.models.cp_utils now has zero intra-repo imports; core cp_utils imports only .parallel.
  • Behavioral coverage is unchanged and comes from the existing tests/e2e/precision/test_hf_attention_cp_relayout.py round-trip test, which now imports the public names. It needs GPUs, so it runs in CI rather than locally.

Motivation

New model support adds a third model that needs the relayout, and it is not a HuggingfaceAttention subclass. Without this, it would either duplicate the permutation logic or import a private symbol across plugins. This PR stands on its own, though — it removes an optional-dependency leak from a core training module either way.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@yueming-yuan
yueming-yuan force-pushed the yueming/cp-layout-utils-to-plugins branch from bb12d56 to f8e5059 Compare July 27, 2026 15:54
@yueming-yuan
yueming-yuan force-pushed the yueming/cp-layout-utils-to-plugins branch from 55c3ac8 to e2aa3a0 Compare September 3, 2026 03:29
@yueming-yuan yueming-yuan changed the title Move in-model CP layout helpers out of training_utils into model plugins Move in-model CP layout helpers into miles_plugins/models/cp_utils.py Sep 3, 2026
miles/backends/training_utils/cp_utils.py owns the CP helpers the training
backend applies around the model: slicing data, masks, logprobs and logits.
build_gdn_cp_context was the one exception -- it runs inside a layer forward,
is used only by model plugins, and forced an optional flash-linear-attention
dependency (the fla.ops.cp try/except) into a core training module.

The zigzag <-> contiguous-shard relayout has the same shape and was private to
miles_plugins/models/hf_attention.py, so only the two models built on
HuggingfaceAttention could reach it.

Collect both in miles_plugins/models/cp_utils.py, which now holds the CP
helpers that run inside a forward. To break the resulting import cycle,
detect_and_setup_hybrid_cp moves the other way, into hf_attention.py next to
the HuggingfaceAttention class it is defined in terms of.

Pure move: every relocated definition is byte-identical apart from dropping the
underscore prefix on the four newly public relayout helpers.
@yueming-yuan
yueming-yuan force-pushed the yueming/cp-layout-utils-to-plugins branch from e2aa3a0 to 256c4eb Compare September 4, 2026 00:52
@yueming-yuan yueming-yuan added the run-ci-model-scripts Run model script smoke tests label Sep 10, 2026

@Zhichenzzz Zhichenzzz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTHuman

@yueming-yuan
yueming-yuan merged commit 90ea0b3 into main Sep 22, 2026
29 of 32 checks passed
@yueming-yuan
yueming-yuan deleted the yueming/cp-layout-utils-to-plugins branch September 22, 2026 01:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-ci-model-scripts Run model script smoke tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants