Skip to content

[CI/Build] Isolate Wan VAE fastpath tests from global RMSNorm patches - #8043

Closed
tjtanaa wants to merge 1 commit into
vllm-project:mainfrom
tjtanaa:fix/wan-vae-fastpath-test-isolation
Closed

tjtanaa wants to merge 1 commit into
vllm-project:mainfrom
tjtanaa:fix/wan-vae-fastpath-test-isolation

Conversation

@tjtanaa

@tjtanaa tjtanaa commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

Superseded for the reported CI failures: #8047 merged into main as 57d3e9fd. The original combined reproduction now passes on unmodified main (79 passed). This draft is retained for review/history; it is no longer needed to resolve those six failures.

Problem and reproduction

AMD CI reports six failures in tests/diffusion/distributed/test_wan_vae_fastpath_install.py. Reproduced locally on main 75570ee3a6572a7dc7a9463ada35a9a11c367993:

python -m pytest tests/diffusion/models/wan2_2/test_wan22_quant_config_propagation.py tests/diffusion/distributed/test_wan_vae_fastpath_install.py -q

Before: 6 failed, 73 passed (norm wrapper rejection, install reversibility, pending bias, dtype checks, and stale import). Running the installer file alone gives 1 failed, 64 passed.

Root cause

Importing Wan2.2 replaces diffusers' WanRMS_norm process-wide with Omni RMSNormVAE. These tests specifically require diffusers' original numerical semantics; the fastpath correctly rejects the substitute. The substitute test also imports the class from its old location.

Change

Load a private copy of the installed diffusers module once to obtain its original normalization class. A function-scoped monkeypatch binds that class during these tests and restores the previous binding afterward. Import RMSNormVAE from its current Wan2.2 module before fixtures execute. Production code, test assertions and tolerances are unchanged; no tests are skipped.

Validation

  • Combined reproduction command: 79 passed.
  • Same two files in reverse order: 79 passed.
  • Installer file alone: 65 passed.
  • Applicable pre-commit checks pass except a confirmed pre-existing mypy Too many arguments diagnostic at original_setattr(module, name, value) (main line 207, branch line 229). Baseline and branch each report only this diagnostic.
  • git diff --check: passed.

Tests use CPU tensors and require the repository test dependencies, no model weights. Environment: Python 3.12, torch 2.12.0+git6bbd260 / HIP 7.2.53211, diffusers 0.40.0, transformers 5.14.1, vllm 0.29.0+rocm723.

AI assistance: Codex investigated the CI failures, reproduced them locally, implemented the test isolation, ran validation and drafted this PR. Human review is pending.

Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>

tjtanaa commented Sep 23, 2026

Copy link
Copy Markdown
Member Author

Self-review (Codex-assisted): checked the posted diff at 2a8842b6, fixture teardown, original diffusers class recovery, and the moved RMSNormVAE import. Production fastpath guards and all numerical assertions remain unchanged; no skips or relaxed tolerances. Reproduced all six failures before editing; afterward 79 tests pass in each file order and 65 pass standalone. Applicable pre-commit checks pass except the same pre-existing mypy call-argument error reproduced on main. Human review is pending.

@hsliuustc0106 hsliuustc0106 added the CI/CD codes related to changes to CI/CD label Sep 24, 2026
@tjtanaa

tjtanaa commented Sep 25, 2026

Copy link
Copy Markdown
Member Author

Closing as superseded by merged #8047. Revalidated on current main 9f3f475685ba0a2c86cee64d12f7611f937d549b: both original Wan VAE reproduction files pass (included in a 116-test passing run with the worker lifecycle file). The NPU-scoped production fix resolves the reported failures without this test-isolation patch.

@tjtanaa tjtanaa closed this Sep 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI/CD codes related to changes to CI/CD

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants