fix(responses): mark truncated output incomplete

[rmccorm4]

Summary

• Map FinishReason::Length to incomplete response and item states.
• Report max_output_tokens as the incomplete reason.
• Emit response.incomplete instead of response.completed for truncated streams.
• Add unary and streaming regression coverage.

Root cause

The Responses conversion path treated every terminal backend finish reason as successful completion, including length-limited generation.

Impact

Clients can distinguish complete output from output truncated by the configured maximum-token limit.

Validation

• cargo test -p dynamo-llm protocols::openai::responses (72 passed)
• cargo fmt --all --check
• cargo clippy -p dynamo-llm --tests -- -D warnings

Linear: DIS-2514

Summary by CodeRabbit

• Bug Fixes
  • Responses now accurately report when generation stops because the maximum output token limit is reached.
  • Incomplete responses include the appropriate status and reason, with affected message and function-call items marked as incomplete.
  • Streaming responses now emit a response.incomplete event instead of incorrectly indicating completion when output limits are reached.
• Tests
  • Added coverage for incomplete non-streaming and streaming responses caused by output limits.

[datadog-official]

⚠️ Warnings

🚦 2 Pipeline jobs failed

PR | backend-status-check     

PR | trtllm-runtime / Test cuda13.1, amd64     

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 9c3b48a | Docs | Datadog PR Page | Give us feedback!}

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 650a4176-29e2-4f05-8c4b-c305fc662a6c

📥 Commits

Reviewing files that changed from the base of the PR and between 769ee07 and 50a946f.

📒 Files selected for processing (3)

• lib/llm/src/protocols/openai/responses/mod.rs
• lib/llm/src/protocols/openai/responses/stream_converter.rs
• tests/serve/test_sglang.py

---

Walkthrough

Changes

The OpenAI Responses converters now represent FinishReason::Length as an incomplete response, including incomplete output-item statuses and max_output_tokens details. Streaming responses emit response.incomplete. SGLang aggregated tests disable reasoning in response payloads.

Responses output-limit handling

Layer / File(s)	Summary
Batch conversion status propagation lib/llm/src/protocols/openai/responses/mod.rs	Length finish reasons produce incomplete response status, unset completed_at, incomplete output items, and max_output_tokens details; unit coverage verifies the behavior.
Streaming termination propagation lib/llm/src/protocols/openai/responses/stream_converter.rs	Streaming conversion tracks length termination, derives output and terminal statuses, and emits response.incomplete with matching response details; unit coverage verifies partial-text handling.
SGLang response payload setup tests/serve/test_sglang.py	Aggregated streaming and non-streaming payloads set reasoning effort to none.

Estimated code review effort: 3 (Moderate) | ~25 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description covers the change and validation, but it doesn't follow the required template sections or include the required issue-link choice.	Reformat the PR description to match the template: add Overview, Details, Where should reviewer start?, and the required Related Issues section with the issue link or no-issue checkbox.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes the main change: treating truncated Responses output as incomplete.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

---

_{Comment @coderabbitai help to get the list of available commands.}

[michaelfeil]

Adding downstream validation context from a fork-side investigation.

We hit the same Responses adapter behavior when a backend terminal chat-completion chunk carries finish_reason: "length": the /v1/responses adapter currently reports the terminal Response as status: "completed" with incomplete_details: null, so clients that follow the Responses contract do not know they should continue or retry with a higher output-token budget.

Minimal sanitized repro shape:

{
  "request": {
    "model": "example-model",
    "input": [
      {"role": "user", "content": "Synthetic task that requires more output than the configured cap."}
    ],
    "max_output_tokens": 8
  },
  "backend_chat_completion_terminal_choice": {
    "finish_reason": "length",
    "message": {
      "role": "assistant",
      "content": "partial synthetic output"
    }
  }
}

Expected Responses result:

{
  "status": "incomplete",
  "completed_at": null,
  "incomplete_details": {"reason": "max_output_tokens"}
}

Expected streaming terminal event:

{
  "type": "response.incomplete",
  "response": {
    "status": "incomplete",
    "incomplete_details": {"reason": "max_output_tokens"}
  }
}

This aligns with the OpenAI Responses guidance for max-output-token exhaustion and is important for OpenAI Agents SDK / Codex-style clients, which inspect response.status == "incomplete" and incomplete_details.reason == "max_output_tokens" to decide whether continuation is needed. The repro above is intentionally synthetic and contains no customer prompts, model outputs, tenant/model IDs, request IDs, URLs, headers, or logs.

[michaelfeil]

cc borjan on this issue. Once merged, we can pull it in.

[michaelfeil]

@codex review with above context

[indrajit96]

LGTM!

Just for mu understand why are we doing this?
So as to give unblock clients/benchmarks who rely on finish_reason to fetch tool calls?