Make skip_special_tokens a generation parameter.

[cyanic-selkie]

Hi,

I have a few models that return structured output by utilizing special tokens as delimiters. As of now, vLLM always skips special tokens during decoding. Would it be possible to add skip_special_tokens as a generation parameter?

TGI sort of supports this by giving you the option to return individual tokens with their IDs and a boolean indicating whether they are special or not.

[bingfengyiren]

I had the same problem as you.

[WoosukKwon]

@cyanic-selkie and @bingfengyiren We've just merged #1186, which adds skip_special_tokens to SamplingParams. This feature will be included in a new release which will be released very soon.