Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

max_tokens's default in docstring of InferenceClient::chat_completion's is 20 but that of TGI is 100 #2652

Closed
sadra-barikbin opened this issue Nov 3, 2024 · 1 comment · Fixed by #2653

Comments

@sadra-barikbin
Copy link

Hi there!🤗

InferenceClient::chat_completion's docstring states that max_tokens's default value is 20 but it seems that of the TGI is 100.

max_tokens (`int`, *optional*):
Maximum number of tokens allowed in the response. Defaults to 20.

https://github.com/huggingface/text-generation-inference/blob/6e3220529df5906ae586031873b7865e9923040b/router/src/lib.rs#L939

@sadra-barikbin sadra-barikbin changed the title max_new_tokens's default in docstring of InferenceClient::chat_completion's is 20 but that of TGI is 100 max_tokens's default in docstring of InferenceClient::chat_completion's is 20 but that of TGI is 100 Nov 3, 2024
@sadra-barikbin sadra-barikbin changed the title max_tokens's default in docstring of InferenceClient::chat_completion's is 20 but that of TGI is 100 max_tokens's default in docstring of InferenceClient::chat_completion's is 20 but that of TGI is 100 Nov 3, 2024
@hanouticelina
Copy link
Contributor

Hello @sadra-barikbin , thanks a lot for reporting this! 🤗 I just opened a PR #2653 to fix the docstrings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants