Skip to content

fix(dashboard): test Responses nodes on /v1/responses, not chat completions (#13070) - #13087

Merged
diegosouzapw merged 2 commits into
diegosouzapw:release/v3.8.51from
ntdatt812:fix/13070-responses-model-test
Sep 10, 2026
Merged

diegosouzapw merged 2 commits into
diegosouzapw:release/v3.8.51from
ntdatt812:fix/13070-responses-model-test

Conversation

@ntdatt812

Copy link
Copy Markdown
Contributor

Fixes #13070.

The defect

detectTestKind() reads the provider node's apiType, but only maps it to audio, rerank and embeddings. There is no Responses branch, so every text model on a node configured with apiType: "responses" falls through to:

return postChatCompletion(buildInternalChatRequest(testBody, signal, connectionId));

which is hardcoded to ${INTERNAL_ORIGIN}/v1/chat/completions.

A Responses-native upstream can answer 200 to that while carrying nothing a Chat Completions reader recognises, so the dashboard marks the model red with Provider returned HTTP 200 but no text content — while the same model answers normally through /v1/responses.

What changed

  • detectTestKind() now also reports isResponses. It is last in the chain, after audio / rerank / embeddings: a Responses-typed node can still host an embedding or rerank model, and those endpoints were already right for it. Losing that ordering would break working setups instead of fixing a broken one.
  • New buildInternalResponsesRequest(), carrying the same health-check bypass headers as the chat and rerank builders.
  • The probe body is { model, input, max_output_tokens }. Responses ignores max_tokens, which would let a reasoning model spend the whole default budget before emitting visible text.
  • The probe is non-streaming on purpose. extractComboTestStreamResult() understands Chat Completions deltas and the output_text / output[] shapes, but not Responses stream events (response.output_text.delta), so a streamed probe would read as empty — the very failure being fixed. The reader is passed !isResponses && streamChat so it takes the JSON path.

No reader change was needed: extractComboTestResponseText() already handles output_text and output[]. Only the request side was ever wrong.

Evidence

Six new tests in tests/unit/responses-node-model-test-13070.test.ts, plus the four existing detectTestKind deep-equal assertions extended with the new field.

The wiring is pinned, not just the classification. My first version of the last test asserted on the request that reached globalThis.fetch, and it was worthless: for a Responses node the router translates a Chat Completions body into Responses shape before it leaves, so the upstream body carries input either way. That test stayed green with the dispatch reverted. It now reads the call log, which records the internal endpoint path — the same field the report used as evidence.

Two-way mutation, run locally:

Mutation Result
none (the branch as submitted) 6 / 6 pass; full model-test-runner.test.ts 32 / 32
remove the if (isResponses) dispatch, keep detectTestKind 1 fail — the call-log test; the other five stay green
detectTestKind always returns isResponses: false 2 fail — classification and call-log

Not covered

The Responses streaming path. The probe avoids it rather than teaching extractComboTestStreamResult() the Responses event shape; that is a larger change and a separate one. If you would rather the probe stream, say so and I will send the reader work instead.

…etions

detectTestKind mapped a provider node's apiType to audio, rerank and
embeddings only, so a text model on a node configured with
apiType: "responses" fell through to the chat branch and was probed with a
Chat Completions body on /v1/chat/completions. A Responses-native upstream
can answer 200 to that while carrying nothing a Chat Completions reader
recognises, so the dashboard marked the model unhealthy with "Provider
returned HTTP 200 but no text content" even though the same model answered
normally through /v1/responses.

detectTestKind now reports isResponses, last in the chain so that embedding,
rerank and audio models hosted on such a node keep the endpoints that were
already right for them. The probe sends { input, max_output_tokens } to
/v1/responses and is deliberately non-streaming: the existing reader
understands the output_text/output[] shapes but not Responses stream events,
so a streamed probe would read as empty -- the very failure being fixed.

Fixes diegosouzapw#13070
…upstream

The first version of this test asserted on the request that reached
globalThis.fetch. That proved nothing: for a node with apiType "responses"
the router translates a Chat Completions body into Responses shape before it
leaves, so the upstream body carries `input` either way and the test stayed
green with the dispatch reverted.

It now reads the call log, which records the internal endpoint path -- the
same field the bug report used as evidence. Reverting the dispatch turns this
test red and leaves the other five green.
@diegosouzapw
diegosouzapw merged commit f2d5728 into diegosouzapw:release/v3.8.51 Sep 10, 2026
10 of 16 checks passed
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…etions (diegosouzapw#13070) (diegosouzapw#13087)

Boarded with 13 sibling PRs into one worktree off release/v3.8.51 and validated as a set: 132 focused tests pass across all 15 test files in the batch, typecheck:core is clean, check-changelog-integrity reports no lost base bullets, and check-file-size is green. Your PR merged without conflict against its siblings.

Thank you — the write-up made this reviewable: measuring the behaviour on the release tip and showing the before/after table meant the defect could be confirmed rather than taken on faith.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(api): Responses provider model tests use /chat/completions and report false HTTP 200/no-text failures

2 participants