[UNIDY2002]

Motivation

As a follow-up of #33684, we added unit tests for the DP and EP layouts, and we also unblocked the EPLB feature.

cc: @liusy58 @alexnails

Modifications

• Created TestWeightCacheDaemonQwen3MoeDP and TestWeightCacheDaemonQwen3MoeEP for tests
• compute_initial_expert_location_metadata on daemon startup when EPLB is enabled

Accuracy Tests

CI should pass

Speed Tests and Profiling

N/A

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

---

CI States

Latest PR Test (Base): ⏳ Run #33040672726
Latest PR Test (Extra): ⏳ Run #33040672420
Latest PR Test (AMD ROCm 7.2): ⏳ Run #33040672645

[UNIDY2002]

/tag-run-ci-label

[UNIDY2002]

/tag-and-rerun-ci extra

[UNIDY2002]

/rerun-failed-ci

[liusy58]

Thanks for your contribution! We should put together a summary table to track which combinations of args are covered by the weight cache daemon tests.

[UNIDY2002]

/rerun-failed-ci

[UNIDY2002]

/rerun-failed-ci

[UNIDY2002]

Should be fixed by #36424. Let's merge with main.

[UNIDY2002]

/rerun-failed-ci

[UNIDY2002]

/rerun-failed-ci

[github-actions]

⛔ /rerun-test requires cooldown_interval_minutes: 0 in .github/CI_PERMISSIONS.json, or write permission on the repo.

Please ask a maintainer to run this command, or use the normal CI flow.

[UNIDY2002]

(Please ignore the above comment, which was performed by my agent...)

[UNIDY2002]

Cases related to our PR are all green:

• base-a-test-cpu
  • test_weight_cache_protocol.py
  • test_server_args.py
• base-b-test-1-gpu-small
  • test_weight_cache_daemon.py (TP=1)
• extra-a-test-2-gpu-large
  • test_weight_cache_daemon.py (TP=2、DP/EP、EPLB)

The failed cases are irrelevant, and all failed for the same reason:

• AttributeError: 'DeepseekV4ForCausalLMDSpark' object has no attribute 'sample_from_anchor'

[UNIDY2002]

Fixed a very small conflict with main in server_args.py.

Upstream changed from if self.weight_cache_mode != "off" and self.enable_eplb: to if cfg.weight_cache_mode != "off" and cfg.enable_eplb:

[liusy58]

/rerun-test test/registered/model_loading/test_weight_cache_daemon.py
/rerun-test test/registered/unit/model_loader/test_weight_cache_protocol.py
/rerun-test test/registered/unit/server_args/test_server_args.py

[github-actions]

Results for /rerun-test test/registered/model_loading/test_weight_cache_daemon.py:

🚀 2-gpu-h100 (1 test): ❌ View workflow run

cd test/ && python3 registered/model_loading/test_weight_cache_daemon.py

🚀 1-gpu-5090 (1 test): ✅ View workflow run

cd test/ && python3 registered/model_loading/test_weight_cache_daemon.py

[UNIDY2002]

/rerun-test test/registered/model_loading/test_weight_cache_daemon.py

[github-actions]

Results for /rerun-test test/registered/model_loading/test_weight_cache_daemon.py:

🚀 2-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/model_loading/test_weight_cache_daemon.py

🚀 1-gpu-5090 (1 test): ✅ View workflow run

cd test/ && python3 registered/model_loading/test_weight_cache_daemon.py

[alexnails]

LGTM