Skip to content

[None][test] Enable warmup request for gen_only perf sanity lanes #17098

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nvpohanh merged 4 commits into from
Aug 6, 2026
Merged

[None][test] Enable warmup request for gen_only perf sanity lanes #17098

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
7fb102f
[None][test] Enable warmup request for gen_only perf sanity lanes
erictsai-nv Jul 31, 2026
0b3f3df
[None][test] Restrict gen_only warmup to concurrency=1 and record war…
erictsai-nv Jul 31, 2026
3dbba98
Merge branch 'main' into test/gen-only-warmup
erictsai-nv Aug 3, 2026
4cf2940
Merge branch 'main' into test/gen-only-warmup
erictsai-nv Aug 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion tests/integration/defs/perf/test_perf_sanity.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -894,6 +894,7 @@ def __init__(
self.model_path = ""
self.dataset_file = client_config_data.get("dataset_file", "")
self.use_nv_sa_benchmark = client_config_data.get("use_nv_sa_benchmark", False)
self.warmup = client_config_data.get("warmup", False)
Comment thread
erictsai-nv marked this conversation as resolved.
Resolved
self.env_vars = env_vars
# spec_decoding flag is retained for DB matching (b_eos column). --ignore-eos
# is now always passed; output-length stability with spec decoding comes from
Expand Down Expand Up @@ -972,11 +973,14 @@ def _to_default_benchmark_cmd(self) -> List[str]:
str(self.concurrency * self.iterations),
"--max-concurrency",
str(self.concurrency),
"--no-test-input",
"--percentile-metrics",
"ttft,tpot,itl,e2el",
"--ignore-eos",
]
# benchmark_serving's initial single-prompt test run doubles as a warmup
# request; keep it disabled unless the lane asks for a warmup.
if not self.warmup:
benchmark_cmd.append("--no-test-input")
if dataset_path:
benchmark_cmd.append("--dataset-name")
benchmark_cmd.append("trtllm_custom")
Expand Down Expand Up @@ -2003,6 +2007,10 @@ def _parse_disagg_config_file(self, config_file_path: str, config_file: str):
"use_nv_sa_benchmark": use_nv_sa_benchmark,
"accuracy_config": accuracy_data,
"only_run_accuracy": only_run_accuracy,
# gen_only measures a single round (iterations forced to 1 above),
# so one-time costs like the cache transceiver's lazy connection
# setup would land entirely on the measured TTFT without a warmup.
"warmup": benchmark_mode == "gen_only",
}
client_config = ClientConfig(
client_config_data,
Expand Down
Loading