Skip to content

[gpt-oss][2/N] Support input_messages in responsesRequest #26962

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
yeqcharlotte merged 11 commits into from
Oct 27, 2025
Merged

[gpt-oss][2/N] Support input_messages in responsesRequest #26962

yeqcharlotte merged 11 commits into from
Oct 27, 2025

Conversation

@qandrew
Copy link
Contributor

@qandrew qandrew commented Oct 16, 2025 •
edited by github-actions bot
Loading

Purpose

in the RepsonsesResponse object, we already have input_messages and output_messages. We should be able to do multi_turl by allowing ResponsesRequest to have previous_input_messages as payload.

Test Plan

I tested following OpenAI's example: https://platform.openai.com/docs/guides/function-calling#function-tool-example

  1. first client call with a tool definition "What is my horoscope? I am an Aquarius."
  2. client receives the output, and runs the horoscope function
  3. second client call with function tool output
  4. verify the output looks good

Added the following as a test also.

Turn 1

curl http://localhost:20001/v1/responses   -H "Content-Type: application/json"   -N   -d '{
    "model": "/data/users/axia/checkpoints/gpt-oss-120b",
    "input": [],
    "previous_input_messages": [{
        "role": "system",
        "content": [
            {
                "type": "system_content",
                "model_identity": "You are ChatGPT, a large language model trained by OpenAI.",
                "reasoning_effort": "Medium",
                "conversation_start_date": "2025-10-06",
                "knowledge_cutoff": "2024-06",
                "channel_config": {
                    "valid_channels": [
                        "analysis",
                        "commentary",
                        "final"
                    ],
                    "channel_required": true
                }
            }
        ]
    },
    {
        "role": "developer",
        "content": [
            {
                "type": "developer_content",
                "tools": {
                    "functions": {
                        "name": "functions",
                        "tools": [
                            {
                                "name": "get_horoscope",
                                "description": "Get todays horoscope for an astrological sign.",
                                "parameters": {}
                            }
                        ]
                    }
                }
            }
        ]
    },
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "What is my horoscope? I am an Aquarius."
            }
        ]
    }],
    "temperature": 0.7,
    "max_output_tokens": 1000,
    "enable_response_messages": true, "stream": true
}'

Turn 1 output

{
    "response": {
        "id": "resp_d1bde89745cb45449c0377df8b3af921",
        "created_at": 1760654776,
        "incomplete_details": null,
        "instructions": null,
        "metadata": null,
        "model": "/data/users/axia/checkpoints/gpt-oss-120b",
        "object": "response",
        "output": [
            {
                "id": "rs_55e49b597c6640fc9a73b884701ecb09",
                "summary": [],
                "type": "reasoning",
                "content": [
                    {
                        "text": "User asks for horoscope for Aquarius. We can use function get_horoscope. Need to call it with sign Aquarius.",
                        "type": "reasoning_text"
                    }
                ],
                "encrypted_content": null,
                "status": null
            },
            {
                "arguments": "{\n  \"sign\": \"Aquarius\"\n}",
                "call_id": "call_d61059ae16db4c2f8c5a5f8b8e0cd6bd",
                "name": "get_horoscope",
                "type": "function_call",
                "id": "fc_d61059ae16db4c2f8c5a5f8b8e0cd6bd",
                "status": null
            }
        ],
        "parallel_tool_calls": true,
        "temperature": 0.7,
        "tool_choice": "auto",
        "tools": [],
        "top_p": 1.0,
        "background": false,
        "max_output_tokens": 1000,
        "max_tool_calls": null,
        "previous_response_id": null,
        "prompt": null,
        "reasoning": null,
        "service_tier": "auto",
        "status": "completed",
        "text": null,
        "top_logprobs": null,
        "truncation": "disabled",
        "usage": {
            "input_tokens": 130,
            "input_tokens_details": {
                "cached_tokens": 0,
                "input_tokens_per_turn": [
                    130
                ],
                "cached_tokens_per_turn": [
                    0
                ]
            },
            "output_tokens": 55,
            "output_tokens_details": {
                "reasoning_tokens": 30,
                "tool_output_tokens": 0,
                "output_tokens_per_turn": [
                    55
                ],
                "tool_output_tokens_per_turn": [
                    0
                ]
            },
            "total_tokens": 185
        },
        "user": null,
        "input_messages": [
            {
                "role": "system",
                "name": null,
                "content": [
                    {
                        "model_identity": "You are ChatGPT, a large language model trained by OpenAI.",
                        "reasoning_effort": "Medium",
                        "conversation_start_date": "2025-10-06",
                        "knowledge_cutoff": "2024-06",
                        "channel_config": {
                            "valid_channels": [
                                "analysis",
                                "commentary",
                                "final"
                            ],
                            "channel_required": true
                        },
                        "type": "system_content"
                    }
                ]
            },
            {
                "role": "developer",
                "name": null,
                "content": [
                    {
                        "tools": {
                            "functions": {
                                "name": "functions",
                                "tools": [
                                    {
                                        "name": "get_horoscope",
                                        "description": "Get todays horoscope for an astrological sign.",
                                        "parameters": {}
                                    }
                                ]
                            }
                        },
                        "type": "developer_content"
                    }
                ]
            },
            {
                "role": "user",
                "name": null,
                "content": [
                    {
                        "type": "text",
                        "text": "What is my horoscope? I am an Aquarius."
                    }
                ]
            }
        ],
        "output_messages": [
            {
                "role": "assistant",
                "name": null,
                "content": [
                    {
                        "type": "text",
                        "text": "User asks for horoscope for Aquarius. We can use function get_horoscope. Need to call it with sign Aquarius."
                    }
                ],
                "channel": "analysis"
            },
            {
                "role": "assistant",
                "name": null,
                "content": [
                    {
                        "type": "text",
                        "text": "{\n  \"sign\": \"Aquarius\"\n}"
                    }
                ],
                "channel": "commentary",
                "recipient": "functions.get_horoscope",
                "content_type": "<|constrain|>json"
            }
        ]
    },
    "sequence_number": 38,
    "type": "response.completed"
}

Turn 2

curl http://localhost:20001/v1/responses   -H "Content-Type: application/json"   -N   -d '{
    "model": "/data/users/axia/checkpoints/gpt-oss-120b",
    "input": [],
    "previous_input_messages": [{
        "role": "system",
        "content": [
            {
                "type": "system_content",
                "model_identity": "You are ChatGPT, a large language model trained by OpenAI.",
                "reasoning_effort": "Medium",
                "conversation_start_date": "2025-10-06",
                "knowledge_cutoff": "2024-06",
                "channel_config": {
                    "valid_channels": [
                        "analysis",
                        "commentary",
                        "final"
                    ],
                    "channel_required": true
                }
            }
        ]
    },
    {
        "role": "developer",
        "content": [
            {
                "type": "developer_content",
                "tools": {
                    "functions": {
                        "name": "functions",
                        "tools": [
                            {
                                "name": "get_horoscope",
                                "description": "Get todays horoscope for an astrological sign.",
                                "parameters": {}
                            }
                        ]
                    }
                }
            }
        ]
    },
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "What is my horoscope? I am an Aquarius."
            }
        ]
    },
    {
        "role": "assistant",
        "name": null,
        "content": [
            {
                "type": "text",
                "text": "User asks for horoscope for Aquarius. We can use function get_horoscope. Need to call it with sign Aquarius."
            }
        ],
        "channel": "analysis"
    },
    {
        "role": "assistant",
        "name": null,
        "content": [
            {
                "type": "text",
                "text": "{\n  \"sign\": \"Aquarius\"\n}"
            }
        ],
        "channel": "commentary",
        "recipient": "functions.get_horoscope",
        "content_type": "<|constrain|>json"
    },
    {
        "role": "tool",
        "name": "functions.get_horoscope",
        "content": [
            {
                "type": "text",
                "text": "{\"output\": {\"horoscope\": \"{\"sign\": \"Aquarius\"}: Next Tuesday you will befriend a baby otter.\"}}"
            }
        ],
        "content_type": "<|constrain|>json"
    }
    ],
    "temperature": 0.7,
    "max_output_tokens": 1000,
    "enable_response_messages": true, "stream": true
}'

"response": {
        "id": "resp_8d0b532ff74f443dbba13cfbda5fd1c8",
        "created_at": 1760655738,
        "incomplete_details": null,
        "instructions": null,
        "metadata": null,
        "model": "/data/users/axia/checkpoints/gpt-oss-120b",
        "object": "response",
        "output": [
            {
                "id": "rs_a21dd22daa67407586aafae9c065ffc3",
                "summary": [],
                "type": "reasoning",
                "content": [
                    {
                        "text": "The function returned a weird output: {\"horoscope\": \"{\"sign\": \"Aquarius\"}: Next Tuesday ...\"} This looks malformed. Likely the function returns an object with horoscope string. We need to parse it. The output field is a string that includes JSON-like inside. Probably the intended horoscope is \"Next Tuesday you will befriend a baby otter.\" Let's respond with that.",
                        "type": "reasoning_text"
                    }
                ],
                "encrypted_content": null,
                "status": null
            },
            {
                "id": "msg_0cf364c9e8a4439193937efa2b036449",
                "content": [
                    {
                        "annotations": [],
                        "text": "Here’s your Aquarius horoscope for today:\n\n**“Next Tuesday you will befriend a baby otter.”**",
                        "type": "output_text",
                        "logprobs": null
                    }
                ],
                "role": "assistant",
                "status": "completed",
                "type": "message"
            }
        ],
        "parallel_tool_calls": true,
        "temperature": 0.7,
        "tool_choice": "auto",
        "tools": [],
        "top_p": 1.0,
        "background": false,
        "max_output_tokens": 1000,
        "max_tool_calls": null,
        "previous_response_id": null,
        "prompt": null,
        "reasoning": null,
        "service_tier": "auto",
        "status": "completed",
        "text": null,
        "top_logprobs": null,
        "truncation": "disabled",
        "usage": {
            "input_tokens": 212,
            "input_tokens_details": {
                "cached_tokens": 176,
                "input_tokens_per_turn": [
                    212
                ],
                "cached_tokens_per_turn": [
                    176
                ]
            },
            "output_tokens": 111,
            "output_tokens_details": {
                "reasoning_tokens": 80,
                "tool_output_tokens": 0,
                "output_tokens_per_turn": [
                    111
                ],
                "tool_output_tokens_per_turn": [
                    0
                ]
            },
            "total_tokens": 323
        },
        "user": null,
        "input_messages": [
            {
                "role": "system",
                "name": null,
                "content": [
                    {
                        "model_identity": "You are ChatGPT, a large language model trained by OpenAI.",
                        "reasoning_effort": "Medium",
                        "conversation_start_date": "2025-10-06",
                        "knowledge_cutoff": "2024-06",
                        "channel_config": {
                            "valid_channels": [
                                "analysis",
                                "commentary",
                                "final"
                            ],
                            "channel_required": true
                        },
                        "type": "system_content"
                    }
                ]
            },
            {
                "role": "developer",
                "name": null,
                "content": [
                    {
                        "tools": {
                            "functions": {
                                "name": "functions",
                                "tools": [
                                    {
                                        "name": "get_horoscope",
                                        "description": "Get todays horoscope for an astrological sign.",
                                        "parameters": {}
                                    }
                                ]
                            }
                        },
                        "type": "developer_content"
                    }
                ]
            },
            {
                "role": "user",
                "name": null,
                "content": [
                    {
                        "type": "text",
                        "text": "What is my horoscope? I am an Aquarius."
                    }
                ]
            },
            {
                "role": "assistant",
                "name": null,
                "content": [
                    {
                        "type": "text",
                        "text": "User asks for horoscope for Aquarius. We can use function get_horoscope. Need to call it with sign Aquarius."
                    }
                ]
            },
            {
                "role": "assistant",
                "name": null,
                "content": [
                    {
                        "type": "text",
                        "text": "{\n  \"sign\": \"Aquarius\"\n}"
                    }
                ]
            },
            {
                "role": "tool",
                "name": "functions.functions.get_horoscope",
                "content": [
                    {
                        "type": "text",
                        "text": "{\"output\": {\"horoscope\": \"{\"sign\": \"Aquarius\"}: Next Tuesday you will befriend a baby otter.\"}}"
                    }
                ],
                "channel": "commentary"
            }
        ],
        "output_messages": [
            {
                "role": "assistant",
                "name": null,
                "content": [
                    {
                        "type": "text",
                        "text": "The function returned a weird output: {\"horoscope\": \"{\"sign\": \"Aquarius\"}: Next Tuesday ...\"} This looks malformed. Likely the function returns an object with horoscope string. We need to parse it. The output field is a string that includes JSON-like inside. Probably the intended horoscope is \"Next Tuesday you will befriend a baby otter.\" Let's respond with that."
                    }
                ],
                "channel": "analysis"
            },
            {
                "role": "assistant",
                "name": null,
                "content": [
                    {
                        "type": "text",
                        "text": "Here’s your Aquarius horoscope for today:\n\n**“Next Tuesday you will befriend a baby otter.”**"
                    }
                ],
                "channel": "final"
            }
        ]
    },

@mergify mergify bot added frontend gpt-oss Related to GPT-OSS models labels Oct 16, 2025
@mergify
Copy link

mergify bot commented Oct 16, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @qandrew.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added the needs-rebase label Oct 16, 2025
initial commit input_messages
ec420f3 
Signed-off-by: Andrew Xia <[email protected]>
@mergify mergify bot removed the needs-rebase label Oct 16, 2025
function call round trip
82e94a4 
Signed-off-by: Andrew Xia <[email protected]>
@qandrew qandrew changed the title Input messages 2 [gpt-oss][2/N] Support input_messages in responsesRequest Oct 16, 2025
Andrew Xia added 3 commits October 16, 2025 17:25
qandrew
qandrew commented Oct 17, 2025
View reviewed changes
vllm/entrypoints/openai/serving_chat.py
get_system_message,
parse_chat_input,
parse_chat_output,
parse_input_to_harmony_message,
Copy link
Contributor Author

@qandrew qandrew Oct 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no functional changes in this file, just renamed the function and formatted code

Andrew Xia added 2 commits October 17, 2025 10:01
clean
6995d10 
Signed-off-by: Andrew Xia <[email protected]>
remove precommit
89061d4 
Signed-off-by: Andrew Xia <[email protected]>
@qandrew qandrew marked this pull request as ready for review October 17, 2025 17:02
@qandrew
Copy link
Contributor Author

qandrew commented Oct 17, 2025

ready for review,
cc @chaunceyjiang (related to function tools) @yeqcharlotte @lacora @alecsolder

@qandrew qandrew mentioned this pull request Oct 17, 2025
Open
@qandrew
Copy link
Contributor Author

qandrew commented Oct 17, 2025

/gemini review

gemini-code-assist[bot]
gemini-code-assist bot reviewed Oct 17, 2025
View reviewed changes
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for previous_input_messages in ResponsesRequest to enable multi-turn conversations, which is a valuable addition. The implementation is mostly sound and is accompanied by new tests. However, I've identified a couple of high-severity issues. One is a bug in the new test case where the tool name is incorrect, potentially masking issues. The other is a significant code duplication in the message processing logic, which impacts maintainability. Addressing these points will improve the robustness and quality of the code.

yeqcharlotte
yeqcharlotte reviewed Oct 18, 2025
View reviewed changes
yeqcharlotte
yeqcharlotte reviewed Oct 18, 2025
View reviewed changes
charlotte comments
ac078e2 
Signed-off-by: Andrew Xia <[email protected]>
@qandrew qandrew requested a review from yeqcharlotte October 20, 2025 16:29
yeqcharlotte
yeqcharlotte reviewed Oct 21, 2025
View reviewed changes
yeqcharlotte
yeqcharlotte reviewed Oct 21, 2025
View reviewed changes
yeqcharlotte
yeqcharlotte reviewed Oct 21, 2025
View reviewed changes
alecsolder
alecsolder reviewed Oct 21, 2025
View reviewed changes
comments
b1d2b02 
Signed-off-by: Andrew Xia <[email protected]>
qandrew
qandrew commented Oct 24, 2025
View reviewed changes
Copy link
Contributor Author

@qandrew qandrew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yeqcharlotte @alecsolder thanks for the comments! should be ready for another review

yeqcharlotte
yeqcharlotte reviewed Oct 24, 2025
View reviewed changes
vllm/entrypoints/openai/serving_responses.py Outdated
Comment on lines 919 to 927
message_role = message.author.role
# To match OpenAI, instructions, reasoning and tools are
# always taken from the most recent Responses API request
# not carried over from previous requests
if (
message_role == OpenAIHarmonyRole.SYSTEM
or message_role == OpenAIHarmonyRole.DEVELOPER
):
continue
Copy link
Collaborator

@yeqcharlotte yeqcharlotte Oct 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move these logic to harmony util?

Copy link
Contributor Author

@qandrew qandrew Oct 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yeqcharlotte thanks for the catch, done

Andrew Xia added 2 commits October 24, 2025 19:47
@qandrew qandrew requested a review from yeqcharlotte October 25, 2025 02:52
yeqcharlotte
yeqcharlotte approved these changes Oct 27, 2025
View reviewed changes
Copy link
Collaborator

@yeqcharlotte yeqcharlotte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for addressing all the updates. it looks much cleaner now!

@github-project-automation github-project-automation bot moved this from To Triage to Ready in gpt-oss Issues & Enhancements Oct 27, 2025
@yeqcharlotte yeqcharlotte enabled auto-merge (squash) October 27, 2025 20:52
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 27, 2025
@yeqcharlotte yeqcharlotte merged commit 53a56e6 into vllm-project:main Oct 27, 2025
49 checks passed
ilmarkov pushed a commit to neuralmagic/vllm that referenced this pull request Nov 7, 2025
@qandrew @ilmarkov
[gpt-oss][2/N] Support input_messages in responsesRequest (vllm-proje…
920348f 
…ct#26962)

Signed-off-by: Andrew Xia <[email protected]>
Co-authored-by: Andrew Xia <[email protected]>
ZhengHongming888 pushed a commit to ZhengHongming888/vllm that referenced this pull request Nov 8, 2025
@qandrew @ZhengHongming888
[gpt-oss][2/N] Support input_messages in responsesRequest (vllm-proje…
1abe0df 
…ct#26962)

Signed-off-by: Andrew Xia <[email protected]>
Co-authored-by: Andrew Xia <[email protected]>
rtourgeman pushed a commit to rtourgeman/vllm that referenced this pull request Nov 10, 2025
@qandrew @rtourgeman
[gpt-oss][2/N] Support input_messages in responsesRequest (vllm-proje…
7c819ab 
…ct#26962)

Signed-off-by: Andrew Xia <[email protected]>
Co-authored-by: Andrew Xia <[email protected]>
devpatelio pushed a commit to SumanthRH/vllm that referenced this pull request Nov 29, 2025
@qandrew @devpatelio
[gpt-oss][2/N] Support input_messages in responsesRequest (vllm-proje…
d19e50f 
…ct#26962)

Signed-off-by: Andrew Xia <[email protected]>
Co-authored-by: Andrew Xia <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yeqcharlotte yeqcharlotte yeqcharlotte approved these changes

@DarkLight1337 DarkLight1337 Awaiting requested review from DarkLight1337 DarkLight1337 is a code owner

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat robertgshaw2-redhat is a code owner

@simon-mo simon-mo Awaiting requested review from simon-mo

@aarnphm aarnphm Awaiting requested review from aarnphm aarnphm is a code owner

@NickLucche NickLucche Awaiting requested review from NickLucche NickLucche is a code owner

@chaunceyjiang chaunceyjiang Awaiting requested review from chaunceyjiang chaunceyjiang is a code owner

+2 more reviewers

@alecsolder alecsolder alecsolder left review comments

@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

frontend gpt-oss Related to GPT-OSS models ready ONLY add when PR is ready to merge/full CI is needed

Projects

gpt-oss Issues & Enhancements
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@qandrew @yeqcharlotte @alecsolder