For TensorRT-LLM benchmarks ..whats the difference between batch_size and max_batch_size ? #1800
Unanswered
prasad-nair-amd
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
In trtllm-build tool there is an attribute --max_batch_size . What this attribute represent. Is this the same attribute as batch_size seen in other industry standard benchmarks. How to specify batch_size for a benchmark ?
Beta Was this translation helpful? Give feedback.
All reactions