[Bugfix] Fix ResponseCreatedEvent ValidationError for json_schema format in streaming#34611
Conversation
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run You ask your reviewers to trigger select CI tests on top of Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add If you have any questions, please reach out to us on Slack at https://slack.vllm.ai. 🚀 |
There was a problem hiding this comment.
Code Review
This pull request correctly fixes a ValidationError that occurred during streaming with json_schema format. The fix, which removes the .model_dump() call when creating ResponseCreatedEvent and ResponseInProgressEvent, is appropriate and aligns the behavior with ResponseCompletedEvent. The new regression test, test_structured_output_streaming, effectively covers the specific failing scenario. The changes are well-executed and address the issue properly.
e185ce0 to
dab8ee7
Compare
|
Hi @agangadi24, the pre-commit checks have failed. Please run: uv pip install pre-commit
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
e9b2ccf to
24a6a7d
Compare
…mat in streaming Remove `.model_dump()` from `ResponsesResponse` before passing to `ResponseCreatedEvent` and `ResponseInProgressEvent`. The `.model_dump()` serialized using Python attribute names (e.g. `schema_`) but Pydantic re-validation expected JSON alias names (e.g. `schema`), causing a ValidationError when `text.format.type: json_schema` was used with streaming. This makes `ResponseCreatedEvent`/`ResponseInProgressEvent` construction consistent with `ResponseCompletedEvent`, which already passes the object directly. Signed-off-by: Anusha Gangadi <agangadi@nvidia.com>
24a6a7d to
ea6a2ed
Compare
Summary
.model_dump()fromResponsesResponsebefore passing toResponseCreatedEvent/ResponseInProgressEventduring streaming.model_dump()serialized using Python attribute names (e.g.schema_) but Pydantic re-validation expected JSON alias names (e.g.schema), causing aValidationErrorwhentext.format.type: json_schemawas used withstream=TrueResponseCompletedEvent, which already passes theResponsesResponseobject directlyTest plan
pytest tests/v1/entrypoints/openai/serving_responses/test_structured_output.py -vpytest tests/v1/entrypoints/openai/serving_responses/test_basic.py -vtest_structured_output_streamingcovers the exact failing code path (json_schema + streaming)pytest tests/entrypoints/openai/test_serving_responses.py -v