Support min_tokens, min_p parameters for api_server

[AllentDan]

No description provided.

[AllentDan]

The weird thing is, min_new_tokens did not work. @irexyc

[irexyc]

from lmdeploy import pipeline
pipe = pipeline('/nvme/shared/vicuna-7b-v1.5/')
pipe('hello', gen_config=GenerationConfig())
# Response(text='ಠ\\_ಠ', generate_token_len=7, input_token_len=41, session_id=8, finish_reason='stop', token_ids=[227, 181, 163, 20122, 227, 181, 163], logprobs=None, index=0)

pipe('hello', gen_config=GenerationConfig(min_new_tokens=8))
# Response(text='ಠ\\_ಠ How can I assist you today?', generate_token_len=14, input_token_len=41, session_id=7, finish_reason='stop', token_ids=[227, 181, 163, 20122, 227, 181, 163, 1128, 508, 306, 6985, 366, 9826, 29973], logprobs=None, index=0)

[AllentDan]

But if I set min_new_tokens=100 for internlm2-chat-1_8b, the result is less than 100 tokens.

[irexyc]

It only ignore eos token when length of the generated token is less than min_new_tokens. Its behavior should be consistent with transformers, except that transformers support multiple eos tokens

But If encountered stop words token, it will still stop.

[lvhan028]

Should we update completions_v1 as well?

[AllentDan]

It is a lagecy api from OpenAI. I tend to not update it.