Skip to content

Conversation

@Wanli-Jiang
Copy link
Collaborator

@Wanli-Jiang Wanli-Jiang commented Nov 18, 2025

Stacked PRs:

CMD to launch trtllm-serve for bench:

trtllm-serve <ckpt_folder_path> \
--host 0.0.0.0 \
--port 8000 \
--backend _autodeploy \
--trust_remote_code \
--extra_llm_api_options examples/auto_deploy/nano_v3_bench.yaml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants