[2/N] Add a minimal temporal chunk callback for MiniMax-H3

[specture724]

[2/3] Add a minimal temporal chunk callback for MiniMax-H3

Part of the RFC #6872 implementation stack (2/3), stacked on #7016. GitHub can't set a fork branch as another PR's base, so this targets main like the rest of the stack; the diff includes #7016's commits until it merges, and will shrink to just this PR's 5 commits (0cfc1d1..e3dd3bb) once it does.

Summary

Expose MiniMax-H3 temporal VAE chunks through a small frames-only callback contract. Reuse the released VAE's temporal primitives (_adaptive_decode, overlap blending, and frame planning) instead of duplicating model logic or adding callback metadata.

The adapter handles isolated-frame boundaries, padding, overlap emission, rank-0 ownership, and callback-error synchronization. Peer ranks run the temporal loop without materializing a second full video. Decoded chunks are normalized through the VAE processor before reaching the consumer, matching the complete decode path.

Validation

• 132 H3 contract/chunk tests pass; Ruff passes.
• Real MiniMax-H3 FL2VA/video_vae weights from /data/models/hub, NVIDIA L20X, CUDA FP16 autocast.
• Latent [1,24,65,30,52] → 221 frames at 480×832.

Path	VAE time (avg of 2)	First callback	Callbacks / D2H
Full decode	3.395 s	—	0 / 1
Chunk callback	3.464 s	0.259 s	13 / 13
Chunk callback, 17-frame consumer batch	3.417 s	0.262 s	13 / 13

Full output and concatenated callback output are bit-exact: identical shape and SHA-256, max_abs_error=0, mean_abs_error=0, and torch.equal=True. H3's native temporal clip size already matches the 17-frame consumer batch threshold, so consumer-side batching does not change callback or D2H count here — it only matters for producers with finer-grained native chunks (e.g. Wan, [1/3]).

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[hsliuustc0106]

CI lanes select tests with pytest -m 'core_model and cpu' (.buildkite/cuda/test-merge.yml:20) and this file has no pytestmark, so the new temporal-planning coverage never runs in any lane — please add pytestmark = [pytest.mark.core_model, pytest.mark.cpu, pytest.mark.diffusion].

[hsliuustc0106]

With callback=None on a distributed VAE the all-reduce sums to 0 and this raises "callback must be supplied only on VAE group rank 0" — a misleading message for the no-callback case, and None is valid single-process. Either treat all-None as a plain full decode or narrow the message to the exactly-one-rank-0-owner contract.

[david6666666]

This ownership check is rank-local. With three VAE ranks and the sole callback on rank 1, ranks 0 and 1 raise, but rank 2 passes and enters the decoder collectives alone. Please reduce an owner-rank token so every rank makes the same decision; #6885 already has that fix and a three-rank recovery test.

[specture724]

Confirmed and fixed in 3331d3c.

The verdict was rank-local: each rank compared its own ownership against its own index. With the callback on rank 1 of three, ranks 0 and 1 disagreed and raised, while rank 2 agreed with itself, fell through, and entered the decoder collectives alone.

The all-reduce now carries which rank owns the callback, not just how many do, so every rank evaluates the same num_owners != 1 or owner_rank != 0.

Covered by a three-rank test parametrized over every rank, asserting all of them refuse. It fails on the previous code with DID NOT RAISE on exactly rank 2 — the rank that would have hung.

[david6666666]

This bypasses the num_tiles < parallel_size fallback in decode_latent(). For a small spatial input, native H3 assigns no tiles to some ranks; those ranks raise while peers wait in the gather. Please share the rank-local tiling context with the complete decode path and cover chunked decode with fewer tiles than ranks.

[specture724]

Confirmed and fixed in 3331d3c.

decode_latent_with_chunks went straight to the chunk loop and skipped the num_tiles < parallel_size fallback that decode_latent applies, so a small spatial input left some ranks tileless: they raise and leave the gather while their peers block in it.

Extracted that decision into _decode_tiling_context() and wrapped both decode paths in it, so the chunked path takes the same rank-local tiling.

Covered by two tests: fewer tiles than ranks must enter the rank-local fallback (fails on the previous code), and the normal case must keep shared tiling.

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[vllm-omni-review-bot]

This PR appears to be related to model: MinimaxH3.

Model owners: @david6666666

@specture724, please review your own changes and leave a short self-review comment describing what you checked. PRs without author self-review may not be assigned a reviewer.

Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment.

[hsliuustc0106]

[P2] Preserve isolated-frame handling before splitting decoded clips

When isolated_first_frame=True and frame_pre_padding=0, the native decoder emits one head frame (vae_ratio_t - 1 : vae_ratio_t) and removes its temporal block before splitting. This loop prepends the isolated latent but skips that extraction, shifting the emitted frames and potentially failing after publishing output. A CPU reproduction using the native temporal planning/decoding methods with a deterministic fake _adaptive_decode produced MiniMax-H3 temporal decode emitted 48/49 frames; with both isolated boundaries enabled, it completed with incorrectly ordered frames. Please restore the native head/tail extraction and emission steps and cover isolated boundaries against the native decode path.

[specture724]

Confirmed and fixed in 2629e52. Thanks for the reproduction — the frame count made it easy to locate.

The loop prepended the isolated head latent and appended the isolated tail latent to the clip, then went straight to splitting, so it never did what the released loop does with the frames that come back: publish the head frame from vae_ratio_t - 1 : vae_ratio_t and drop its temporal block, keep the tail frame and drop its block, then emit the tail after the final overlap. Every later frame therefore shifted by one block.

Restored all four steps in the native order.

For coverage I transcribed the released _decode_temporal_streaming into the test file as a parity oracle and compare the fork's concatenated chunks against it for all three boundary combinations, fed the way decode_temporal feeds it (isolated tokens split off first, then the remainder padded to a whole chunk). Two of the three fail on the previous behavior — head-only and both-boundaries — which matches what you saw.

One note while checking the plan arithmetic: the fork's chunk planning is equivalent to the released one (pseudo_tokens, pad_tokens, num_chunks all agree), so the defect was only in the post-decode extraction.

This is inert for the shipped checkpoint — frame_pre_padding is 3, so offset_frame is 0 and neither boundary is isolated — but the fork is installed structurally, so a config that enables them would have hit it.