[Feat] support TP for GLM-Image

[RuixiangMa]

Purpose

Close #911.

Enable tensor parallelism (TP) support for GLM-Image diffusion model.

Test Plan

Test Result

Metric	TP=1	TP=2	TP=4
Image
Time （s/img）	46.778	38.317	32.187
VRAM（GiB）	23.14	17.804	14.798

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan. Please provide the test scripts & test commands. Please state the reasons if your codes don't require additional test scripts. For test file guidelines, please check the test style doc
• The test results. Please paste the results comparison before and after, or the e2e results.
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model. Please run mkdocs serve to sync the documentation editions to ./docs.
• (Optional) Release notes update. If your change is user-facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

[hsliuustc0106]

does the tp work for AR or DiT or both?

[hsliuustc0106]

btw, I remember the AR part is not working faster as expected even in TP=1, can you help debug it？cc @JaredforReal

[RuixiangMa]

btw, I remember the AR part is not working faster as expected even in TP=1, can you help debug it？cc @JaredforReal

GLM-image AR likely support tp via vLLM. I only considered DiT， the results are also DiT.

[JaredforReal]

@RuixiangMa GLM Image is not supported in vllm main repo, we need to work on vllm-omni to speed up the AR part
The AR part already supports TP in most of its components

[wtomin]

LGTM.

[hsliuustc0106]

Blocker Scan

Category	Result
Correctness	✅ PASS
Reliability/Safety	✅ PASS
Breaking Changes	✅ PASS
Test Coverage	⚠️ NEEDS TESTS
Documentation	✅ PASS
Security	✅ PASS

OVERALL: 1 CONCERN FOUND

---

⚠️ Missing Test Coverage

This PR adds tensor parallelism support for GLM-Image but includes no test files. Other models with TP support have dedicated tests:

• tests/diffusion/models/flux2/test_flux2_transformer_tp.py
• tests/diffusion/models/z_image/test_zimage_tp_constraints.py

Consider adding a test to verify:

1. TP constraints validation (similar to test_zimage_tp_constraints.py)
2. Basic TP inference correctness

---

💡 Performance Comparison - Minor Gap

Per the performance comparison checklist for [Feature] PRs affecting performance, the PR provides:

• ✅ Latency comparison (TP=1: 46.8s → TP=4: 32.2s)
• ✅ VRAM comparison (TP=1: 23.1GiB → TP=4: 14.8GiB)
• ✅ Output consistency (same image across TP sizes)

For completeness, consider documenting:

• GPU model and CUDA version used for benchmarks
• Exact commands used

---

VERDICT: COMMENT (non-blocking - tests recommended but implementation looks correct)