[main] [1/5] Qwen3.5 support: MTP packed-seq CP+THD fix

[wplf]

Superseded

This PR is superseded by #4495, which lands the same MTP CP+THD fix.
Closing once the replacement merges.

---

Qwen3.5 support series

This is part of a 5-PR series adding Qwen3.5-VL support, split for review clarity.

Main PRs (this series):

• [1/5] MTP packed-seq CP+THD fix — fix(mtp): use padded cu_seqlens in MTP roll for THD with CP #4495
• [2/5] FSDP DTensor Bridge checkpoint compatibility — [main] [2/5] Qwen3.5 support: FSDP DTensor Bridge checkpoint compatibility #4753
• [3/5] SharedExpertMLP meta init — [main] [3/5] Qwen3.5 support: SharedExpertMLP meta init #4754
• [4/5] Interleaved MRoPE layout — [main] [4/5] Qwen3.5 support: Interleaved MRoPE layout #4755
• [5/5] Qwen3.5-VL training example — [main] [5/5] Qwen3.5 support: Qwen3.5-VL training example #4756

Dev PRs (corresponding mirrors):

• [1/5] [Dev] fix(mtp): use padded cu_seqlens in MTP roll for THD with CP #4494
• [2/5] [dev] [2/5] Qwen3.5 support: FSDP DTensor Bridge checkpoint compatibility #4748
• [3/5] [dev] [3/5] Qwen3.5 support: SharedExpertMLP meta init #4749
• [4/5] [dev] [4/5] Qwen3.5 support: Interleaved MRoPE layout #4750
• [5/5] [dev] [5/5] Qwen3.5 support: Qwen3.5-VL training example #4751

---

Summary

• In _roll_tensor_packed_seq, prefer packed_seq_params.cu_seqlens_q_padded when available, falling back to cu_seqlens_q.

Why

Under CP+THD, the per-sample CP partition produced by tex.thd_get_partitioned_indices slices on padded boundaries (each sample padded to a multiple of 2 * cp_size). Using the unpadded cu_seqlens_q to roll the MTP target on each rank therefore misaligns boundaries and shifts tokens across sample edges on some ranks.

Padding tokens carry loss_mask=0 / label=-100, so rolling across them is a no-op for the loss but unaligned rolling on unpadded boundaries is not.

The fallback preserves behavior for callers that don't pad to a multiple of 2 * cp_size.

Notes

Mirror of #4747 (same patch, targeting main instead of dev).

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.