Skip to content

[responsesAPI][ez] add a unit test for SimpleContext logprobs#37126

Merged
houseroad merged 2 commits intovllm-project:mainfrom
qandrew:unit-test-simple-context-logprobs
Mar 16, 2026
Merged

[responsesAPI][ez] add a unit test for SimpleContext logprobs#37126
houseroad merged 2 commits intovllm-project:mainfrom
qandrew:unit-test-simple-context-logprobs

Conversation

@qandrew
Copy link
Contributor

@qandrew qandrew commented Mar 15, 2026

Purpose

this adds a unit test to make sure logprobs are outputted correctly, as it would look like this in streaming. no behavioral changes. Adding so I can be more confident in the migration in #37007.

event: response.output_text.delta
data: {"content_index":0,"delta":" 😊","item_id":"090bc9aa-4861-4e04-b2cc-aeb05bb9ab03","logprobs":[{"token":"","logprob":-2.622600959512056e-6,"top_logprobs":[{"token":"","logprob":-2.622600959512056e-6},{"token":"","logprob":-12.87500286102295},{"token":"","logprob":-17.625001907348633}]}],"output_index":1,"sequence_number":138,"type":"response.output_text.delta"}

event: response.output_text.delta
data: {"content_index":0,"delta":" How","item_id":"090bc9aa-4861-4e04-b2cc-aeb05bb9ab03","logprobs":[{"token":" How","logprob":0.0,"top_logprobs":[{"token":" How","logprob":0.0},{"token":"  \n","logprob":-17.0},{"token":" I","logprob":-18.375}]}],"output_index":1,"sequence_number":139,"type":"response.output_text.delta"}

event: response.output_text.delta
data: {"content_index":0,"delta":" can","item_id":"090bc9aa-4861-4e04-b2cc-aeb05bb9ab03","logprobs":[{"token":" can","logprob":0.0,"top_logprobs":[{"token":" can","logprob":0.0},{"token":" I","logprob":-18.5},{"token":" Can","logprob":-19.875}]}],"output_index":1,"sequence_number":140,"type":"response.output_text.delta"}

event: response.output_text.delta
data: {"content_index":0,"delta":" I","item_id":"090bc9aa-4861-4e04-b2cc-aeb05bb9ab03","logprobs":[{"token":" I","logprob":0.0,"top_logprobs":[{"token":" I","logprob":0.0},{"token":" assist","logprob":-18.875},{"token":"我","logprob":-21.125}]}],"output_index":1,"sequence_number":141,"type":"response.output_text.delta"}

Test Plan

unit test passes
python -m pytest tests/entrypoints/openai/responses/test_simple.py::test_streaming_logprobs

Signed-off-by: Andrew Xia <axia@meta.com>
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds a unit test for streaming logprobs in the responses API. The test verifies that when logprobs are requested, the streaming events contain the correct logprob data structure. My review focuses on strengthening this new test to ensure it fully validates the expected behavior. I've suggested an improvement to assert that top_logprobs are not just present but also contain data, which makes the test more robust.

Signed-off-by: Andrew Xia <axia@meta.com>
@qandrew
Copy link
Contributor Author

qandrew commented Mar 15, 2026

cc @chaunceyjiang / @houseroad could I get a quick review here?

@houseroad houseroad added the ready ONLY add when PR is ready to merge/full CI is needed label Mar 15, 2026
Copy link
Collaborator

@houseroad houseroad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@houseroad houseroad merged commit e9163b5 into vllm-project:main Mar 16, 2026
15 checks passed
Lucaskabela pushed a commit to Lucaskabela/vllm that referenced this pull request Mar 17, 2026
wendyliu235 pushed a commit to wendyliu235/vllm-public that referenced this pull request Mar 18, 2026
fxdawnn pushed a commit to fxdawnn/vllm that referenced this pull request Mar 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants