Skip to content

[diffusion] model: support LTX2.3 two stage - #22182

Merged
mickqian merged 125 commits into
mainfrom
ltx-2.3-two-stage
Apr 12, 2026
Merged

mickqian merged 125 commits into
mainfrom
ltx-2.3-two-stage

Conversation

@mickqian

@mickqian mickqian commented Apr 6, 2026 •

Copy link
Copy Markdown
Collaborator
sglang generate \
  --model-path Lightricks/LTX-2.3 \
  --width 1536 --height 1024 \
  --prompt "A beautiful sunset over the ocean" \
  --pipeline-class-name LTX2TwoStagePipeline \
  --save-output

sglang:

sglang_two_stage.mp4

diffusers:

official_two_stage.mp4

difference:

  • mean_abs = 6.8681
  • PSNR = 26.7860
  • cosine = 0.992799

sglang (sp):

sglang_two_stage.mp4

Motivation

Modifications

Accuracy Tests

Speed Tests and Profiling

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

@github-actions github-actions Bot added the diffusion SGLang Diffusion label Apr 6, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements support for the LTX-2.3 model, including the two-stage pipeline, updated sampling parameters for resolution scaling, and model overlay materialization. Changes also include refinements to the denoising and latent preparation stages, along with new alignment scripts and unit tests. Review feedback identifies opportunities to improve artifact resolution by prioritizing newer versions (22b and 1.1) and suggests adding a safety check for zero-division when calculating the cross-attention gate factor.

Comment on lines +39 to +50
LTX23_DEV_CHECKPOINT_FILENAMES = (
"ltx-2.3-20b-dev.safetensors",
"ltx-2.3-22b-dev.safetensors",
)
LTX23_DISTILLED_LORA_FILENAMES = (
"ltx-2.3-20b-distilled-lora-384.safetensors",
"ltx-2.3-22b-distilled-lora-384.safetensors",
)
LTX23_SPATIAL_UPSAMPLER_FILENAMES = (
"ltx-2.3-spatial-upscaler-x2-1.0.safetensors",
"ltx-2.3-spatial-upscaler-x2-1.1.safetensors",
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The resolution order for LTX-2.3 artifacts currently prioritizes older or smaller versions (e.g., 20b over 22b, 1.0 over 1.1). This is inconsistent with the overlay_manifest.json which specifies the 22b and 1.1 versions. Prioritizing the newer/larger versions ensures that the materializer picks the intended official artifacts when multiple versions are present in the source directory.

Comment on lines +1557 to +1559
av_ca_gate_factor = (
self.av_ca_timestep_scale_multiplier / self.timestep_scale_multiplier
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Calculating av_ca_gate_factor by dividing by self.timestep_scale_multiplier without a zero-check poses a risk of ZeroDivisionError. Although this multiplier is typically non-zero (e.g., 1000), it is safer to handle the zero case or pre-calculate this factor during initialization.

        av_ca_gate_factor = (
            self.av_ca_timestep_scale_multiplier / self.timestep_scale_multiplier
            if self.timestep_scale_multiplier != 0
            else 1.0
        )

Comment on lines 49 to +50
os.path.join(model_path, "ltx-2.3-spatial-upscaler-x2-1.0.safetensors"),
os.path.join(model_path, "ltx-2.3-spatial-upscaler-x2-1.1.safetensors"),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The resolution order for spatial upsampler candidates should prioritize version 1.1 over 1.0, as 1.1 is the version listed in the overlay_manifest.json and is generally preferred for LTX-2.3.

Suggested change
os.path.join(model_path, "ltx-2.3-spatial-upscaler-x2-1.0.safetensors"),
os.path.join(model_path, "ltx-2.3-spatial-upscaler-x2-1.1.safetensors"),
os.path.join(model_path, "ltx-2.3-spatial-upscaler-x2-1.1.safetensors"),
os.path.join(model_path, "ltx-2.3-spatial-upscaler-x2-1.0.safetensors"),

Comment on lines +62 to 63
os.path.join(model_path, "ltx-2.3-20b-distilled-lora-384.safetensors"),
os.path.join(model_path, "ltx-2.3-22b-distilled-lora-384.safetensors"),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The resolution order for distilled LoRA candidates should prioritize the 22b version over 20b to align with the official LTX-2.3 artifacts specified in the manifest.

Suggested change
os.path.join(model_path, "ltx-2.3-20b-distilled-lora-384.safetensors"),
os.path.join(model_path, "ltx-2.3-22b-distilled-lora-384.safetensors"),
os.path.join(model_path, "ltx-2.3-22b-distilled-lora-384.safetensors"),
os.path.join(model_path, "ltx-2.3-20b-distilled-lora-384.safetensors"),

@mickqian

mickqian commented Apr 7, 2026

Copy link
Copy Markdown
Collaborator Author

/tag-and-rerun-ci

@github-actions github-actions Bot added the run-ci CI: run the baseline test suite on this PR label Apr 7, 2026
@mickqian
mickqian force-pushed the ltx-2.3-two-stage branch from 85a356b to b5553e1 Compare April 7, 2026 01:53
@yhyang201

Copy link
Copy Markdown
Collaborator

/rerun-failed-ci

3 similar comments
@yhyang201

Copy link
Copy Markdown
Collaborator

/rerun-failed-ci

@yhyang201

Copy link
Copy Markdown
Collaborator

/rerun-failed-ci

@yhyang201

Copy link
Copy Markdown
Collaborator

/rerun-failed-ci

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Apr 12, 2026
@mickqian
mickqian merged commit 495ef8e into main Apr 12, 2026
78 of 80 checks passed
@mickqian
mickqian deleted the ltx-2.3-two-stage branch April 12, 2026 14:16
caitengwei pushed a commit to caitengwei/sglang that referenced this pull request Jun 1, 2026
Chronostasys pushed a commit to MindLab-Research/sglang that referenced this pull request Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

diffusion SGLang Diffusion documentation Improvements or additions to documentation run-ci CI: run the baseline test suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants