Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
82d7df2
[None][fix] GLM tool parsers: stream exactly what the whole-text pars…
JunyiXu-nv Oct 7, 2026
c4c08d0
[None][feat] GLM reasoning parser
JunyiXu-nv Oct 7, 2026
f9f698a
[None][fix] Responses API: honour web_search when the client disables…
JunyiXu-nv Aug 28, 2026
a676def
[TRTLLM-15894][feat] Responses API: support disaggregated serving
JunyiXu-nv Aug 28, 2026
83bea9d
[None][fix] Say what happens to `store` on disagg, and accept dict-sh…
JunyiXu-nv Sep 4, 2026
a0987ab
[None][feat] Serve /v1/models on the disaggregated server
JunyiXu-nv Sep 4, 2026
c45513b
[None][fix] Accept the input shapes a Responses client replays
JunyiXu-nv Sep 5, 2026
f0f56b4
[None][fix] Honour the tools Codex declares as an input item
JunyiXu-nv Sep 6, 2026
0aba576
[None][fix] Translate a tool result that arrives as content parts
JunyiXu-nv Sep 6, 2026
c6842ec
[None][fix] Render `developer` messages instead of dropping them
JunyiXu-nv Sep 7, 2026
9422984
[None][fix] Forward a request under the field names it arrived with
JunyiXu-nv Sep 7, 2026
e18cec3
[None][fix] Write responses under their wire field names too
JunyiXu-nv Sep 7, 2026
d2bc0d2
[None][fix] Report the cached token count the context phase measured
JunyiXu-nv Sep 9, 2026
1edd91d
[None][fix] Emit the reasoning that shares a chunk with the answer
JunyiXu-nv Sep 12, 2026
efc3755
[None][fix] Give the streaming test fake the accumulated text it reads
JunyiXu-nv Sep 12, 2026
36276b4
[None][fix] Order the response snapshot reasoning-first, as the strea…
JunyiXu-nv Sep 12, 2026
2ba85de
[None][fix] Say so when a Responses stream stops before completing
JunyiXu-nv Sep 12, 2026
e114013
[None][fix] responses: tool calls and usage a client can actually use
JunyiXu-nv Sep 22, 2026
e5dec9b
[None][fix] responses: degrade images in tool outputs too, cap error …
JunyiXu-nv Sep 30, 2026
271a1ed
[None][fix] responses: the final response must describe the stream th…
JunyiXu-nv Sep 30, 2026
76ae88e
[None][fix] responses: honor what the request asked and what the clie…
JunyiXu-nv Sep 30, 2026
b73e569
[None][fix] responses: stored context that survives replay, terminals…
JunyiXu-nv Sep 30, 2026
1593a0d
[None][fix] Make Responses final snapshot agree with the streamed view
JunyiXu-nv Oct 6, 2026
c6f66ca
[None][test] Run the new Responses test files in the CPU stage
JunyiXu-nv Oct 6, 2026
7c9cdcb
[None][fix] Cover the routes the disaggregated server gained
JunyiXu-nv Sep 4, 2026
6829f2e
[None][fix] Stop a test from flipping prometheus into multiprocess mode
JunyiXu-nv Sep 4, 2026
67e6990
[None][fix] Responses: review fixes, and less code and tests to review
JunyiXu-nv Oct 7, 2026
c13a8f0
[None][feat] Route Responses requests on the disaggregated server
JunyiXu-nv Oct 7, 2026
a4c91dc
[None][fix] Responses: keep a processor's numbering when no frame cou…
JunyiXu-nv Oct 7, 2026
524b242
[None][chore] Merge main (ff94288d99) into dev-junyix-fix-responses-api
JunyiXu-nv Oct 9, 2026
95e275a
[None][test] Responses: fix the two tests CI run 76914 failed
JunyiXu-nv Oct 9, 2026
f487fa6
[None][fix] Responses: context-only prompt ids with postprocessing wo…
JunyiXu-nv Oct 10, 2026
363e4be
[None][test] Responses: a list-mode context-only reply carries no bas…
JunyiXu-nv Oct 10, 2026
4399d53
[None][fix] Responses: address review of the input, stream and disagg…
JunyiXu-nv Oct 10, 2026
0c38b38
[None][doc] Responses API on the disaggregated server; release notes
JunyiXu-nv Oct 10, 2026
6208a8e
[None][chore] Merge main (7179c5b01b) into dev-junyix-fix-responses-api
JunyiXu-nv Oct 10, 2026
1ed6fb2
[None][fix] Responses: folding a tool call does not change the reques…
JunyiXu-nv Oct 10, 2026
42c02b4
[None][chore] Merge main (16df7eaf25) into dev-junyix-fix-responses-api
JunyiXu-nv Oct 11, 2026
571a1e3
[None][fix] Responses: reject a malformed prompt_token_ids_b64 at val…
JunyiXu-nv Oct 11, 2026
f5408be
[None][chore] Merge main (efeda44574) into dev-junyix-fix-responses-api
JunyiXu-nv Oct 12, 2026
9fb0942
[None][doc] Responses: file the release notes under Release 1.4
JunyiXu-nv Oct 12, 2026
81999f9
[None][chore] Merge main (a8d7911bed) into dev-junyix-fix-responses-api
JunyiXu-nv Oct 12, 2026
2283d92
[None][fix] Responses: keep the disaggregated handoff off client requ…
JunyiXu-nv Oct 13, 2026
05e644f
[None][fix] Responses disagg: drop client prompt ids; complete a stre…
JunyiXu-nv Oct 13, 2026
3de4ca9
[None][chore] Merge main (c018848da6) into dev-junyix-fix-responses-api
JunyiXu-nv Oct 20, 2026
8851a43
[None][fix] Responses: leave image input to main's handling
JunyiXu-nv Oct 20, 2026
4ee109c
[None][fix] Responses disagg: time the first token from the first out…
JunyiXu-nv Oct 21, 2026
778dfc4
[None][chore] Merge main (afc1c168fd) into dev-junyix-fix-responses-api
JunyiXu-nv Oct 22, 2026
9fdcdba
[None][chore] Merge main (9bb7931d6e) into dev-junyix-fix-responses-api
JunyiXu-nv Oct 22, 2026
7b2d9c3
[None][chore] Merge main (9cc37929e8) into dev-junyix-fix-responses-api
JunyiXu-nv Oct 22, 2026
771bb8d
[None][fix] Responses: final snapshot keeps the status the stream rep…
JunyiXu-nv Oct 23, 2026
39f7307
[None][chore] Merge main (3024eb446e) into dev-junyix-fix-responses-api
JunyiXu-nv Oct 23, 2026
7381ced
Merge commit '39f7307584' into dev-junyix-fix-responses-snapshot-status
JunyiXu-nv Oct 23, 2026
0179f70
[None][chore] Merge main (9619d5b2ee) into dev-junyix-fix-responses-api
JunyiXu-nv Oct 23, 2026
54fea18
[None][chore] Merge main (687359e228) into dev-junyix-fix-responses-api
JunyiXu-nv Oct 26, 2026
7628d6f
[None][chore] Merge dev-junyix-fix-responses-api (54fea183bb) into de…
JunyiXu-nv Oct 26, 2026
d77d894
[None][chore] Merge main (dd75334918) into dev-junyix-fix-responses-api
JunyiXu-nv Oct 26, 2026
149dbfc
[None][chore] Merge dev-junyix-fix-responses-api (d77d894f59) into de…
JunyiXu-nv Oct 26, 2026
9e67daf
[None][chore] Merge main (9ffa6f4395) into dev-junyix-fix-responses-api
JunyiXu-nv Oct 26, 2026
e9fab98
[None][chore] Merge dev-junyix-fix-responses-api (9e67daf619) into de…
JunyiXu-nv Oct 26, 2026
e1f6458
[None][chore] Merge main (64c1ce103c) into dev-junyix-fix-responses-s…
JunyiXu-nv Oct 26, 2026
58319dc
Merge remote-tracking branch 'origin/main' into dev-junyix-fix-respon…
JunyiXu-nv Nov 2, 2026
4faabf1
[None][test] Check the status of Harmony reasoning and function-call …
JunyiXu-nv Nov 2, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 12 additions & 5 deletions tensorrt_llm/serve/responses_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -662,7 +662,7 @@ def _parse_output_message_harmony(message: Message) -> list[ResponseOutputItem]:
summary=[],
type="reasoning",
content=[Content(text=content.text, type="reasoning_text")],
status=None,
status="completed",
Comment thread
JunyiXu-nv marked this conversation as resolved.
Resolved
)
output_items.append(reasoning_item)
elif message.channel == "commentary":
Expand All @@ -677,6 +677,7 @@ def _parse_output_message_harmony(message: Message) -> list[ResponseOutputItem]:
type="function_call",
name=function_name,
id=f"fc_{_random_uuid()}",
status="completed",
)
output_items.append(response_item)
elif message.recipient.startswith(
Expand All @@ -687,7 +688,7 @@ def _parse_output_message_harmony(message: Message) -> list[ResponseOutputItem]:
summary=[],
type="reasoning",
content=[Content(text=content.text, type="reasoning_text")],
status=None,
status="completed",
)
output_items.append(reasoning_item)
else:
Expand Down Expand Up @@ -1895,7 +1896,9 @@ def _streamed_or_fresh_id(item_type: str) -> str:
content=[
Content(text=record.text, type="reasoning_text")
],
status=None,
# The stream closed this item as completed; the
# snapshot reports the same status.
status="completed",
))
else:
output_items.append(
Expand Down Expand Up @@ -1928,7 +1931,7 @@ def _streamed_or_fresh_id(item_type: str) -> str:
content=[
Content(text=reasoning_text, type="reasoning_text")
],
status=None,
status="completed",
))
stored_reasoning = reasoning_text

Expand All @@ -1955,8 +1958,12 @@ def _streamed_or_fresh_id(item_type: str) -> str:
tool_calls_item = list(streamed_tool_calls)
else:
tool_resolution = _tool_resolution(tools)
# As streamed: the stream delivers every call it announces as
# completed.
tool_calls_item = [
_tool_call_output_item(call, tool_resolution) for call in calls
_tool_call_output_item(call,
tool_resolution,
status="completed") for call in calls
]
output_items.extend(tool_calls_item)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
ResponsesStreamingProcessor,
StreamedItem,
_create_output_content,
_parse_output_message_harmony,
_whole_text_tool_calls,
create_response_non_store,
)
Expand Down Expand Up @@ -385,3 +386,67 @@ def test_the_stream_releases_what_the_whole_text_parse_reads_beyond_it(
SimpleNamespace(index=0, text=whole), request, helper, None, "qwen3", []
)
assert (released, calls, logger.warning.called) == (owed, [], warned)


# ---------------------------------------------------------------------------
# Item status: the snapshot reports what the stream closed each item with
# ---------------------------------------------------------------------------


def test_final_items_report_the_status_the_stream_closed_them_with():
"""Reasoning and function-call items stay `completed` in the snapshot.

Their done events say `completed`; a snapshot that built them without a
status read `null` for the same items, so a consumer that filters items by
status reconstructed a different turn from each view.
"""
processor = _processor(tools=[_tool()], reasoning_parser="glm47", tool_parser="glm47")
frames, result = _stream(processor, ["Plan it.", "</think>Calling: ", _EXEC_CALL])

streamed = {item["type"]: item.get("status") for item in _done_items(frames)}
final = {item["type"]: item.get("status") for item in _final_output(processor, result)}

assert streamed == {
"reasoning": "completed",
"message": "completed",
"function_call": "completed",
}
assert final == streamed


def test_non_streaming_items_are_completed_too():
"""No stream ran; the body is the only account, and it says completed."""
items, _messages, _reasoning = _create_output_content(
SimpleNamespace(outputs=[SimpleNamespace(index=0, text="Plan it.</think>" + _EXEC_CALL)]),
reasoning_parser="glm47",
tool_parser="glm47",
tools=[_tool()],
streamed_item_ids=None,
)
assert [(item.type, item.status) for item in items] == [
("reasoning", "completed"),
("function_call", "completed"),
]


@pytest.mark.parametrize(
"channel, recipient, item_type",
[
("analysis", None, "reasoning"),
("commentary", "functions.get_weather", "function_call"),
("commentary", "python", "reasoning"),
],
ids=["analysis", "function_call", "python_commentary"],
)
def test_harmony_reasoning_and_function_call_items_are_completed(channel, recipient, item_type):
openai_harmony = pytest.importorskip("openai_harmony")
message = openai_harmony.Message.from_role_and_content(
openai_harmony.Role.ASSISTANT, '{"city": "Rome"}'
).with_channel(channel)
if recipient is not None:
message = message.with_recipient(recipient)

(item,) = _parse_output_message_harmony(message)

assert item.type == item_type
assert item.status == "completed"
Loading