[diffusion] CI: rebase LTX H100 timing references on recent CI runs

[niehen6174]

Motivation

The LTX H100 timing references sit 5-20% below what the CI pool measures, so a normal run lands at or above the limit and passes only by retrying. Over 2026-08-20 to 2026-10-06, the first attempt of each job failed the current limits 31-79% of the time. For example, the ltx_2_two_stage_t2v median-step limit is 131.0 ms (109.1 ms x 1.2), and the pool's median is 129 ms.

In the same seven weeks, LTX timings show no regression beyond the two below, which are handled separately. This PR rebases the references on the measured pool and leaves the tolerances unchanged.

• [diffusion] Name the quality tiers after what they guarantee #42370 (10-07): refinement +13-17% on all two-stage cases. [diffusion] Warm up CI cases at the request's quality level #42958 fixes it, and 10-07 is excluded from the data.
• [diffusion] Respect explicit residency over pipeline preload hints #41622 (10-03): LTX-2.5 TextEncodingStage +~850 ms, from about 90 ms to 947 ms, on every runner. The case's explicit text_encoder=component-offload now takes effect. The new reference matches this cost so current CI is unaffected; fixing it is a separate change.

Data

• Source: H100 PR CI logs of [diffusion] PRs, multimodal-gen-test-{1,2}-gpu jobs, from 2026-08-20 to 2026-10-06.
• Sample: up to 8 runs per day, 392 runs and 1,730 LTX attempts in total.
• Excluded: PRs that edit LTX code ([diffusion] refactor: scope model-specific API parameters #35613, [diffusion] Plan reversible residency before warmup #36703, [Diffusion] Make component execution options fail closed #37049, [diffusion] Measure warmup memory and layer usage per phase for residency calibration (1/4) #37916, [Diffusion] Add bounded exact conditioning cache across native models #40470, [diffusion] move kernel validation into launchers and simplify dispatch #42391).
• One sample per job: only the first attempt of each job is used. A slow attempt fails and is retried, while a fast one ends the loop, so pooling all attempts over-samples slow runs.

Weekly median of the median denoise step, relative to the seven-week median:

Case	08-17	08-24	08-31	09-07	09-14	09-21	09-28
ltx_2_two_stage_t2v	0.96	0.97	0.98	1.02	0.97	1.01	1.07
ltx_2.3_two_stage_t2v_2gpus	0.97	0.99	0.99	1.00	0.97	1.00	1.05
ltx_2_3_two_stage_ti2v_2gpus	0.99	0.95	0.98	1.00	1.00	1.01	1.06
ltx_2.3_one_stage_ti2v	0.98	0.99	0.99	0.98	0.97	1.03	1.07
ltx_2_3_hq_pipeline	0.94	0.97	1.08	0.94	1.07	1.05	1.02
ltx_2_5_diffusion_decoder_2gpus						1.03	0.98

• E2E, denoising stage and refinement move with the median step.
• The 4-7% rise from 09-27 to 10-02 went away on 10-03 and is kept in the data.
• Decoding got faster after the 08-17 week (1.34-1.43 to about 1.0). That is an improvement, and later weeks dominate the median.

Modifications

For the six LTX cases in perf_baselines/h100.json, each timing reference is set to the median over that case's window:

• expected_e2e_ms
• expected_avg_denoise_ms and expected_median_denoise_ms
• stages_ms
• denoise_step_ms

Tolerances, load references, memory references and every other case are unchanged.

LTX-2.5 uses data from 2026-09-21 onward, after decoder tiling cut its E2E from about 194 s to 10 s. Its TextEncodingStage reference uses attempts after #41622. For E2E, the earlier samples have the current text encoder time swapped in.

Case	E2E	Median step	Denoising stage	Refinement	Decoding
ltx_2_two_stage_t2v	6314 -> 6781	109 -> 129	4522 -> 5250	545 -> 636	207 -> 220
ltx_2.3_two_stage_t2v_2gpus	6350 -> 6527	146 -> 154	4492 -> 4781	589 -> 719	240 -> 320
ltx_2_3_two_stage_ti2v_2gpus	9687 -> 10886	243 -> 291	7506 -> 8958	671 -> 736	251 -> 327
ltx_2.3_one_stage_ti2v	16722 -> 18216	495 -> 556	15026 -> 16865	-	235 -> 333
ltx_2_3_hq_pipeline	15974 -> 19005	748 -> 885	11244 -> 13255	3198 -> 3216	885 -> 1292
ltx_2_5_diffusion_decoder_2gpus	10142 -> 11801	257 -> 363	3751 -> 4746	-	3382 -> 3412

LTX-2.5 TextEncodingStage: 634 -> 947. All values are in ms.

Expected effect

Replaying the same CI history against the validator's rules (all E2E, aggregate, stage and sampled-step checks):

Case	First-attempt failure	Job failure (observed -> estimated)
ltx_2_two_stage_t2v	51% -> 15%	20% -> 6%
ltx_2.3_two_stage_t2v_2gpus	38% -> 12%	11% -> 3%
ltx_2_3_two_stage_ti2v_2gpus	56% -> 20%	21% -> 5%
ltx_2.3_one_stage_ti2v	31% -> 14%	10% -> 4%
ltx_2_3_hq_pipeline	71% -> 50%	18% -> 7%
ltx_2_5_diffusion_decoder_2gpus	79% -> 34%	53% -> 19%

• The job estimate is an upper bound, because it does not replay the extra retries the new limits would allow.
• A further 3-5% margin on the references lowers the first-attempt rate by only 1-2 points. The remaining failures are tail events that the retry budget covers.
• Regression sensitivity, replaying the history slowed uniformly: for the four two-stage and one-stage cases, the first-attempt failure rate goes from 12-20% to 21-30% at +10% and to 56-60% at +20%. A +20% regression fails most jobs. A +10% regression shows up as more retries but can still pass after them.

Dependencies and follow-ups

• Merge together with or after [diffusion] Warm up CI cases at the request's quality level #42958. The refinement references exclude the [diffusion] Name the quality tiers after what they guarantee #42370 slowdown.
• ltx_2_3_hq_pipeline keeps a high first-attempt rate. Its decoding stage has a long tail (median 1.3 s, p90 5.9 s), which needs a separate fix.
• ltx_2_5_diffusion_decoder_2gpus has a bimodal denoise step: first attempts often run around 400 ms, retries around 270 ms. This and the text encoder offload are left for a separate change.

---

CI States

Latest PR Test (Base): ✅ Run #37643823235
Latest PR Test (Extra): ❌ Run #37643822735
Latest PR Test (AMD ROCm 10): ❌ Run #37643823081

[niehen6174]

/tag-and-rerun-ci