Skip to content

[Diffusion] Add official LTX ordinary and distilled two-stage pipelines - #5583

Closed
mglyn wants to merge 22 commits into
vllm-project:mainfrom
mglyn:dev_ltx_two_stage
Closed

mglyn wants to merge 22 commits into
vllm-project:mainfrom
mglyn:dev_ltx_two_stage

Conversation

@mglyn

@mglyn mglyn commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add ordinary and full-distilled LTX two-stage T2V/I2V pipelines for LTX-2 and LTX-2.3
  • share the one-stage runtime while making phase recipes, latent transition, and phase-weight binding explicit
  • default ordinary two-stage to dynamic LoRA, with a resident mode for official BF16-aligned arithmetic
  • fix stage-one audio selection, Euler scalar arithmetic, video token layout, SP audio padding, and Ulysses cross-attention with unequal Q/K lengths
  • reject unsupported Cache-DiT and resident serialized-quant combinations instead of silently degrading
  • expand the official-reference accuracy harness to 19 one-stage, distilled, ordinary, resident, and dynamic T2V/I2V cases

Scope

This builds on the LTX guidance and precision refactor merged in #5148 and is tracked by #4985.

HQ and Res2s are intentionally excluded from this PR because they introduce a separate sampler, schedule, and phase-strength math. They should be reviewed independently after ordinary two-stage support lands.

Validation

  • 249 passed across the affected LTX, attention-config, and model-extra unit tests
  • Ulysses unequal-length cross-attention baselines pass for SP=2 and SP=4
  • all 19 accuracy cases collect successfully
  • pre-commit passes on all changed files
  • every commit carries a DCO sign-off

mglyn added 22 commits July 30, 2026 11:29
Signed-off-by: mglyn <1203789601@qq.com>
Signed-off-by: mglyn <1203789601@qq.com>
Signed-off-by: mglyn <1203789601@qq.com>
Signed-off-by: mglyn <1203789601@qq.com>
Signed-off-by: mglyn <1203789601@qq.com>
Signed-off-by: mglyn <1203789601@qq.com>
Signed-off-by: mglyn <1203789601@qq.com>
Signed-off-by: mglyn <1203789601@qq.com>
Signed-off-by: mglyn <1203789601@qq.com>
Signed-off-by: mglyn <1203789601@qq.com>
Signed-off-by: mglyn <1203789601@qq.com>
Signed-off-by: mglyn <1203789601@qq.com>
Signed-off-by: mglyn <1203789601@qq.com>
Signed-off-by: mglyn <1203789601@qq.com>
Signed-off-by: mglyn <1203789601@qq.com>
Signed-off-by: mglyn <1203789601@qq.com>
Signed-off-by: mglyn <1203789601@qq.com>
Signed-off-by: mglyn <1203789601@qq.com>
Signed-off-by: mglyn <1203789601@qq.com>
Signed-off-by: mglyn <1203789601@qq.com>
Signed-off-by: mglyn <1203789601@qq.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

@mglyn mglyn closed this Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant