Skip to content

[Bugfix] Replace assert with ValueError for response_format validatio…#35514

Closed
antonovsergey93 wants to merge 1 commit intovllm-project:mainfrom
antonovsergey93:fix-validation-2
Closed

[Bugfix] Replace assert with ValueError for response_format validatio…#35514
antonovsergey93 wants to merge 1 commit intovllm-project:mainfrom
antonovsergey93:fix-validation-2

Conversation

@antonovsergey93
Copy link
Copy Markdown

@antonovsergey93 antonovsergey93 commented Feb 27, 2026

Purpose

When the /v1/chat/completions endpoint receives a request with response_format type json_schema but without the required json_schema field, the server crashes with an AssertionError, resulting in a 500 Internal Server Error.

Fixes #35438

This is the same class of issue addressed in #35456 for the /v1/completions endpoint

Test Plan

pytest tests/entrypoints/openai/test_chat_error.py -v

Test Result

Testing started at 14:58 ...
Launching pytest with arguments test_chat_error.py::test_json_schema_response_format_missing_schema --no-header --no-summary -q in /Users/santonov/github/vllm/tests/entrypoints/openai

============================= test session starts ==============================
collecting ... collected 1 item

test_chat_error.py::test_json_schema_response_format_missing_schema PASSED [100%]

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

…n in chat completions endpoint

Signed-off-by: Sergey Antonov <antonovsergey93@gmail.com>
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

The code changes include adding a test case to check for a validation error when the json_schema field is missing in the response_format and adding a validator to the ChatCompletionRequest class to ensure that when the response_format type is 'json_schema', the 'json_schema' field is provided. The reviewer commented that the assert statement should be replaced with a more appropriate error handling mechanism, such as raising a ValueError or a custom exception, to provide more informative error messages and prevent the server from crashing.

structured_outputs_kwargs["json"] = json_schema.json_schema
elif response_format.type == "structural_tag":
structural_tag = response_format
assert structural_tag is not None and isinstance(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The assert statement should be replaced with a more appropriate error handling mechanism, such as raising a ValueError or a custom exception, to provide more informative error messages and prevent the server from crashing. This is especially important in production environments.

                raise ValueError("structural_tag cannot be None when response_format type is 'structural_tag'")

@DarkLight1337 DarkLight1337 enabled auto-merge (squash) February 27, 2026 14:07
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Feb 27, 2026
@DarkLight1337
Copy link
Copy Markdown
Member

Sorry, #35510 came first

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working frontend ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Invalid response_format leads in 500 errors

2 participants