Skip to content

[Test] Fix HunyuanImage3 nightly perf startup #3819

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
TaffyOfficial wants to merge 3 commits into from
Closed

[Test] Fix HunyuanImage3 nightly perf startup #3819

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a8e4bb9
[Test] Use HunyuanImage3 DiT deploy config in perf tests
zuiho-kai May 27, 2026
e387a54
[Test] Resolve HunyuanImage3 perf deploy config path
zuiho-kai May 29, 2026
d5aac3e
[Test] Forward trust remote code to Hunyuan tokenizer
zuiho-kai May 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion tests/dfx/perf/scripts/run_benchmark.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ def _get_config_file_from_argv() -> str | None:
BENCHMARK_CONFIGS = load_configs(CONFIG_FILE_PATH)


DEPLOY_CONFIGS_DIR = Path(__file__).parent.parent / "deploy"
DEPLOY_CONFIGS_DIR = Path(__file__).resolve().parent.parent.parent.parent.parent / "vllm_omni" / "deploy"
test_params = create_unique_server_params(BENCHMARK_CONFIGS, DEPLOY_CONFIGS_DIR)
server_to_benchmark_mapping = create_test_parameter_mapping(BENCHMARK_CONFIGS)

Expand Down
23 changes: 20 additions & 3 deletions tests/dfx/perf/tests/test_hunyuan_image_tp2_fp8_cfgp2.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,29 @@
"server_type": "vllm-omni",
"server_params": {
"model": "tencent/HunyuanImage-3.0-Instruct",
"stage_config_name": "hunyuan_image3_dit.yaml",
"stage_overrides": {

@Bounty-hunter Bounty-hunter May 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is an bug(3483) that stage_overrides can't overwrite correct，please check it.

@TaffyOfficial TaffyOfficial May 27, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, pasted the comparable execution numbers below.

For the current HunyuanImage3 perf JSON baselines, all use 1024x1024, 50 steps, 10 prompts, max_concurrency=1:

config throughput_qps latency_p99 peak_memory_mb_max
tp4_fp8 0.0800 13.1227s 46838
tp2_fp8_sp2 0.0800 12.0731s 66314
tp2_fp8_cfgp2 0.1035 9.9057s 66470

Compared with tp4_fp8:

  • tp2_fp8_sp2 has 8.0% lower p99 latency, same recorded throughput, and about 41.6% higher peak memory.
  • tp2_fp8_cfgp2 has 24.5% lower p99 latency, 29.4% higher throughput, and about 41.9% higher peak memory.

I also checked the #3483 override issue. The broken path there is flat diffusion parallel overrides being left as top-level engine_args fields while diffusion reads engine_args.parallel_config. This PR is not using that flat path for the TP2 cases: it passes a full nested parallel_config through stage_overrides, and a local materialization check gives parallel_config.tensor_parallel_size=2 / parallel_config.cfg_parallel_size=2 with no flat tensor_parallel_size left in engine_args.

If we want to use flat CLI flags instead, then this PR should wait for or rebase on #3483.

"0": {
"parallel_config": {
"pipeline_parallel_size": 1,
"data_parallel_size": 1,
"tensor_parallel_size": 2,
"enable_expert_parallel": true,
"sequence_parallel_size": 1,
"ulysses_degree": 1,
"ring_degree": 1,
"cfg_parallel_size": 2,
"vae_patch_parallel_size": 1,
"use_hsdp": false,
"hsdp_shard_size": -1,
"hsdp_replicate_size": 1
}
}
},
"serve_args": {
"tensor-parallel-size": 2,
"cfg-parallel-size": 2,
"quantization": "fp8",
"distributed-executor-backend": "mp",
"enforce-eager": true,
"trust-remote-code": true,
"enable-diffusion-pipeline-profiler": true
}
},
Expand Down
23 changes: 20 additions & 3 deletions tests/dfx/perf/tests/test_hunyuan_image_tp2_fp8_sp2.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,29 @@
"server_type": "vllm-omni",
"server_params": {
"model": "tencent/HunyuanImage-3.0-Instruct",
"stage_config_name": "hunyuan_image3_dit.yaml",
"stage_overrides": {
"0": {
"parallel_config": {
"pipeline_parallel_size": 1,
"data_parallel_size": 1,
"tensor_parallel_size": 2,
"enable_expert_parallel": true,
"sequence_parallel_size": 2,
"ulysses_degree": 2,
"ring_degree": 1,
"cfg_parallel_size": 1,
"vae_patch_parallel_size": 1,
"use_hsdp": false,
"hsdp_shard_size": -1,
"hsdp_replicate_size": 1
}
}
},
"serve_args": {
"tensor-parallel-size": 2,
"usp": 2,
"quantization": "fp8",
"distributed-executor-backend": "mp",
"enforce-eager": true,
"trust-remote-code": true,
"enable-diffusion-pipeline-profiler": true
}
},
Expand Down
4 changes: 2 additions & 2 deletions tests/dfx/perf/tests/test_hunyuan_image_tp4_fp8.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@
"server_type": "vllm-omni",
"server_params": {
"model": "tencent/HunyuanImage-3.0-Instruct",
"stage_config_name": "hunyuan_image3_dit.yaml",
"serve_args": {
"tensor-parallel-size": 4,
"quantization": "fp8",
"distributed-executor-backend": "mp",
"enforce-eager": true,
"trust-remote-code": true,
"enable-diffusion-pipeline-profiler": true
}
},
Expand Down
8 changes: 6 additions & 2 deletions vllm_omni/diffusion/models/hunyuan_image3/hunyuan_image3_tokenizer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,13 @@ class Conversation:


class TokenizerWrapper:
def __init__(self, tokenizer):
def __init__(self, tokenizer, *, trust_remote_code: bool = False, revision: str | None = None):
if isinstance(tokenizer, str):
self.tokenizer = AutoTokenizer.from_pretrained(tokenizer)
self.tokenizer = AutoTokenizer.from_pretrained(
tokenizer,
revision=revision,
trust_remote_code=trust_remote_code,
)
else:
self.tokenizer = tokenizer

Expand Down
6 changes: 5 additions & 1 deletion vllm_omni/diffusion/models/hunyuan_image3/pipeline_hunyuan_image3.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -351,7 +351,11 @@ def __init__(self, od_config: OmniDiffusionConfig) -> None:
self.vae = DistributedAutoencoderKLHunyuan.from_config(self.hf_config.vae)
self.vae.use_spatial_tiling = self.od_config.vae_use_tiling
self._pipeline = None
self._tkwrapper = TokenizerWrapper(od_config.model)
self._tkwrapper = TokenizerWrapper(
od_config.model,
revision=od_config.revision,
trust_remote_code=od_config.trust_remote_code,
)
self.image_processor = HunyuanImage3ImageProcessor(self.hf_config)
self.vision_model = Siglip2VisionTransformer(self.hf_config.vit)
# self.vision_model = vision_model.vision_model
Expand Down
Loading