[Feature][Frontend]: Add support for `stream_options` in `ChatCompletionRequest`#5135

Merged

DarkLight1337 merged 10 commits intovllm-project:mainfrom

Etelis:feature/support-stream-options#4967

Jun 7, 2024

Commits on May 26, 2024

Add stream_options in ChatCompletionRequest
Etelis
committed

Commits on May 30, 2024

[Feature]: Support `stream_options` option (#4967)
Etelis
committed
-Implemented a validator to ensure `stream_options` can only be set if `stream` is true.
Etelis
committed
[Feature]: Add optional usage statistics to streaming responses
Etelis
committed

Commits on Jun 2, 2024

Tests added for the following scenarios:
Etelis
committed

Commits on Jun 3, 2024

Forgot to notate validator as a classmethod.
Etelis
committed