renderer: Fix miselading error message#33430
renderer: Fix miselading error message#33430RishabhSaini wants to merge 1 commit intovllm-project:mainfrom
Conversation
Maximum allow total input length RendererConfig.max_length = max_model_len - (max_output_tokens + truncated_input_length) Signed-off-by: RishabhSaini <rishabhsaini01@gmail.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request aims to fix a misleading error message. While the new message is more informative, it introduces a formatting error by omitting spaces between concatenated string literals. This would result in an unreadable error message for the user. I have provided a suggestion to correct this formatting issue.
| f"The token count of your prompt ({len(token_ids)})" | ||
| f"plus request's max_tokens cannot exceed the" | ||
| f"model's context length of {self.model_config.max_model_len}. " | ||
| f"Maximum allowed input is {max_length} tokens. " | ||
| "Please reduce the input length or decrease max_tokens.", |
There was a problem hiding this comment.
The new error message is much more informative, but it has a formatting issue. The separate f-strings on lines 399 and 400 will be concatenated without a space, resulting in a malformed message containing ...prompt (X)plus... and ...exceed themodel's.... To ensure the message is readable, spaces should be added at the end of these lines.
| f"The token count of your prompt ({len(token_ids)})" | |
| f"plus request's max_tokens cannot exceed the" | |
| f"model's context length of {self.model_config.max_model_len}. " | |
| f"Maximum allowed input is {max_length} tokens. " | |
| "Please reduce the input length or decrease max_tokens.", | |
| f"The token count of your prompt ({len(token_ids)}) " | |
| f"plus request's max_tokens cannot exceed the " | |
| f"model's context length of {self.model_config.max_model_len}. " | |
| f"Maximum allowed input is {max_length} tokens. " | |
| "Please reduce the input length or decrease max_tokens.", |
|
Closing in preference of #33425 |
1 participant
Fixes: #33418