Skip to content

refactor(lora): one plain PEFT adapter directory per component - #274

Merged
Rockdu merged 2 commits into
fsdp-followups/8-device-movefrom
fsdp-followups/9-lora-adapter-paths
Oct 4, 2026
Merged

Rockdu merged 2 commits into
fsdp-followups/8-device-movefrom
fsdp-followups/9-lora-adapter-paths

Conversation

@Rockdu

@Rockdu Rockdu commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

Stack 9/9 of follow-ups to #263: main ← #265 ← #266 ← #267 ← #268 ← #269 ← #271 ← #272 ← #273 ← this.

What

  • --lora-adapter-path, --ref-lora-adapter-path and --teacher-lora-adapter-path take one plain PEFT adapter directory (what save_pretrained writes) per --update-weight-target-module entry, in order, e.g. --lora-adapter-path /adapters/high_noise /adapters/low_noise for Wan2.2. Any other count is rejected in arguments.py.
  • The component-named subfolder layout and its lora_adapter_subfolder helper are gone.
  • test(fsdp): keep the caller's PYTHONPATH ...: the GPU reference-model test now puts the repository root in front of the caller's PYTHONPATH instead of replacing it (a follow-up to perf(fsdp): skip deterministic NaN fill in moves; fix reference worker PYTHONPATH #269, kept here so the PRs below keep their CI results).

Why

#263 read a multi-component adapter as one directory with a PEFT checkpoint under each component subfolder. No other tool writes or reads that layout: PEFT names its subfolders after adapters, and diffusers / the Wan2.2 community ship one file per expert. A list of standard PEFT directories keeps the loader on PEFT's own format.

Files

  • miles/utils/arguments.py: the three flags become nargs="+" lists (args.*_lora_adapter_paths); the count checks run once the component list is parsed.
  • miles/backends/fsdp_utils/model_loader.py, actor.py: each component loads its own adapter directory.
  • tests: test_reference_arguments.py (count check), test_reference_models.py and the GPU reference worker (per-component directories); tests/fast-gpu/backends/fsdp_utils/test_reference_models.py (PYTHONPATH).
  • docs: fsdp_model_roles.md, cli-reference.md.

Checklist

  • pre-commit run passes on the changed files
  • Added/updated tests for new behaviour
  • pytest -x is green — test_reference_arguments.py, test_reference_models.py locally; the GPU worker runs in CI
  • If launch flags changed, python3 train.py --help still parses — argparse nargs="+" checked standalone; the full parser runs in CI
  • If a public flag was added, it appears in the CLI reference docs
  • If an example was added, it has a real walkthrough — n/a

@Rockdu Rockdu added the run-ci-e2e Run e2e metric-regression tests on this PR label Oct 4, 2026
@Rockdu
Rockdu merged commit 71303d4 into main Oct 4, 2026
65 of 71 checks passed
OnePunchMonk pushed a commit to OnePunchMonk/miles_diffusion that referenced this pull request Oct 5, 2026
…ark#274)

* refactor(lora): take one plain PEFT adapter directory per component instead of component subfolders

* test(fsdp): keep the caller's PYTHONPATH behind the repository root in the reference worker
OnePunchMonk pushed a commit to OnePunchMonk/miles_diffusion that referenced this pull request Oct 5, 2026
OnePunchMonk pushed a commit to OnePunchMonk/miles_diffusion that referenced this pull request Oct 5, 2026
…ark#274)

* refactor(lora): take one plain PEFT adapter directory per component instead of component subfolders

* test(fsdp): keep the caller's PYTHONPATH behind the repository root in the reference worker
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-ci-basic Authorize and run only basic unlabeled CI tests run-ci-e2e Run e2e metric-regression tests on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants