Skip to content

[Perf] GLM Image#792

Closed
JaredforReal wants to merge 54 commits into
vllm-project:dev/rebase_0.14.0from
JaredforReal:perf/glm-image
Closed

[Perf] GLM Image#792
JaredforReal wants to merge 54 commits into
vllm-project:dev/rebase_0.14.0from
JaredforReal:perf/glm-image

Conversation

@JaredforReal
Copy link
Copy Markdown
Contributor

@JaredforReal JaredforReal commented Jan 15, 2026

Purpose

  • Implement GLM Image AR stage in vLLM
  • Utilize vLLM-Omni multistage to connect AR and DiT parts

Test Plan

TBD

Test Result

TBD

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

@JaredforReal
refine glm-image implementation
fc54ff9 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
implement GLM Image vllm AR
91bff06 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
init multistage
21df56b 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
revert attention_mask in GlmImageAttention forward()
b02a12a 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
init and registry
d569790 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
implement stage config and stage input processor
234e49f 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
fix image2image error
2320cf8 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
implement a pre processor func
6b9b486 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
fix image2image error
2d92e22 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
update stage config
fbb2ac8 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
Merge branch 'dev/rebase_0.14.0' into perf/glm-image
6f0e4de 
Signed-off-by: Jared Wen <w13431838023@gmail.com>
@JaredforReal
implement example offline end2end files
0e5366f 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
modify dit configs
dc8f2c2 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
fix end2end offline examples
ad0da64 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
support sub folder and model arch
974ed22 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
fix import error
f5551f2 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
tokenizer
88439b6 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
fix BaseDummyInputsBuilders
6676c96 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
tokenizer sub dir
324348c 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
fix text2image
c0877a3 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
fix i2i dummy inputs
277fd0d 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
fix gate up load weight
4883184 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
fix gate up load weight
393dfd2 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
get transformer/config.json
aa4b586 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
add glm image mrope
daa57c5 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
fix glm_image spelling
82460fa 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@david6666666 david6666666 mentioned this pull request Jan 16, 2026
Open
63 tasks
@JaredforReal
fix
e1d946f 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
fix compute_logits
468cd04 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
fix glm image stage input processors
59df495 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
fix stage input
396883c 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
fix stage input
2a36a47 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
debug
31fa960 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
diffusion temperature 1.0
1b15a94 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
end2end params temp
67ec0af 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
apply_chat_template, prepocessor text
15c6a36 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
get processor config
36bd0f7 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
debug logging
186f149 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
use processor.tokenizer
69197a4 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
use temperature 0.9 and 0.15 top_p
c22875b 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
align image_grid_thw with transformers
e37bfcb 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
fix params
ac8a81b 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
fix mrope calc
a6ae872 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
fix import
48881ed 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
add debug logging
2dda88f 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
more logs
e676c93 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
fix config
fc540f7 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
more logs
585cecd 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
correct text-to-image detection for M-RoPE position computation
9620875 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
override config detection
0dad161 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
use a straight detection of mrope_section
ed232c7 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
use get_model
eff86c1 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
cleanup: remove debug logging and simplify docstrings
e93c18b 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@JaredforReal
feat: add profiling points for stage timing analysis
ad849f0 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@david6666666 david6666666 added the high priority high priority issue, needs to be done asap label Jan 19, 2026
@JaredforReal
try implement i2i mode
44f2d30 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
@lishunyang12 lishunyang12 mentioned this pull request Feb 21, 2026
Merged
5 tasks
@JaredforReal JaredforReal deleted the perf/glm-image branch March 27, 2026 02:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hsliuustc0106 hsliuustc0106 Awaiting requested review from hsliuustc0106 hsliuustc0106 will be requested when the pull request is marked ready for review hsliuustc0106 is a code owner

Copilot code review Copilot Awaiting requested review from Copilot Copilot will automatically review once the pull request is marked ready for review

Assignees

No one assigned

Labels

high priority high priority issue, needs to be done asap

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@JaredforReal @david6666666 @hsliuustc0106