Skip to content

[Frontend] Add structured_outputs field to ResponsesRequest#33249

Open
MarcinGodniak wants to merge 2 commits intovllm-project:mainfrom
MarcinGodniak:feat/responses_add_structured_output
Open

[Frontend] Add structured_outputs field to ResponsesRequest#33249
MarcinGodniak wants to merge 2 commits intovllm-project:mainfrom
MarcinGodniak:feat/responses_add_structured_output

Conversation

@MarcinGodniak
Copy link
Copy Markdown

@MarcinGodniak MarcinGodniak commented Jan 28, 2026

Purpose

Expose structured_outputs as a field on ResponsesRequest to allow internal components (e.g., tool_parser) to modify structured output parameters during request processing.

Previously, structured_outputs was a local variable in to_sampling_params(), making it inaccessible to other parts of the request pipeline. This change promotes it to an instance field so it can be set/modified by tool parsers before sampling params are generated.

This aligns the Responses API with the existing behavior in /chat/completions, where structured_outputs is already available as a field.

Test Plan

pytest tests/entrypoints/openai/responses -v

Test Result

========================================================= warnings summary ==========================================================
<frozen importlib._bootstrap>:488
  <frozen importlib._bootstrap>:488: DeprecationWarning: builtin type SwigPyPacked has no __module__ attribute

<frozen importlib._bootstrap>:488
  <frozen importlib._bootstrap>:488: DeprecationWarning: builtin type SwigPyObject has no __module__ attribute

.venv/lib/python3.12/site-packages/schemathesis/generation/coverage.py:305
  /root/vllm_code/.venv/lib/python3.12/site-packages/schemathesis/generation/coverage.py:305: DeprecationWarning: jsonschema.exceptions.RefResolutionError is deprecated as of version 4.18.0. If you wish to catch potential reference resolution errors, directly catch referencing.exceptions.Unresolvable.
    ref_error: type[Exception] = jsonschema.RefResolutionError,

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================================= 61 passed, 1 skipped, 3 warnings in 639.80s (0:10:39) =======================================

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

Signed-off-by: Marcin Godniak <marcingodniak@gmail.com>

Signed-off-by: Marcin Godniak <mgodniak@amazon.com>
@github-actions
Copy link
Copy Markdown

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀

@mergify mergify bot added the frontend label Jan 28, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds the structured_outputs field to ResponsesRequest, aligning it with the ChatCompletionRequest and allowing for more flexible structured output configurations. The implementation in to_sampling_params correctly uses this new field. However, it introduces a side effect by mutating the request object. I've provided a suggestion to refactor this to avoid mutation, which improves code clarity and maintainability by making the method's behavior more predictable.

@MarcinGodniak MarcinGodniak marked this pull request as ready for review January 28, 2026 12:28
# TODO: consider supporting non harmony messages as well
previous_input_messages: list[OpenAIHarmonyMessage | dict] | None = None

structured_outputs: StructuredOutputsParams | None = Field(
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need to use this field?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My ultimate goal is to support providing grammar via the tool parser.
For chat/completions requests, this field is used to analyze the request and, when necessary, populate the StructuredOutputsParams.grammar field, which is later consumed by to_sampling_params. I already use this field in chat/completions, and it seemed natural to replicate the same approach here.

I’m open to other approaches if there’s a better way to achieve this.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For structured outputs, the Responses API has its own dedicated fields.

https://platform.openai.com/docs/guides/structured-outputs

@mergify
Copy link
Copy Markdown

mergify bot commented Feb 3, 2026

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @MarcinGodniak.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants