Skip to content

fix(combo): ignore benign empty error fields in streaming quality validation (502 false-positive on opencode tool calls) - #9786

Merged
diegosouzapw merged 1 commit into
release/v3.8.50from
fix/escalated-quality-validation-benign-error
Aug 9, 2026
Merged

diegosouzapw merged 1 commit into
release/v3.8.50from
fix/escalated-quality-validation-benign-error

Conversation

@diegosouzapw

Copy link
Copy Markdown
Owner

Causa-raiz

A qualidade de streaming (validateResponseQuality → isStreamingUpstreamError em open-sse/services/combo/validateQuality.ts) usava uma checagem de presença de chave em vez de conteúdo:

if (parsed.error != null) return true;

parsed.error != null é true para {}, "", false, 0 — qualquer valor exceto null/undefined. Quando um cliente como o opencode faz uma chamada de tool call, o upstream SSE abre com frames de role-only (sem conteúdo reconhecido) e um chunk posterior que carrega tool_calls reais mais um campo error vazio benigno (que alguns backends emitem em todo chunk). O gate de erro roda ANTES dos reconhecedores de conteúdo, então esse único frame vira "error" → 502 "streaming upstream error".

O mesmo combo via kilocode funciona porque o wire format do kilocode não emite o campo error vazio → o gate nunca dispara. Por isso o bug parece tool-call-specific e client-specific.

Fix

Novo helper isSubstantiveError(): só trata error como falha real quando tem conteúdo substantivo:

  • String não-vazia → erro
  • Objeto com chaves (ex.: {message, code}) → erro
  • true explícito → erro
  • {}, "", false, 0 → benigno (não erro)

Aplicado ao gate de top-level (parsed.error). O caso nested (parsed.response.status === "failed") permanece intacto por já ser guardado pelo status.

TDD

  • Teste novo: tests/unit/quality-validation-benign-error.test.ts — 3 casos:
    1. tool_calls + error:{} → valid (antes: inválido/502)
    2. tool_calls + error:"" → valid (antes: inválido/502)
    3. erro real {message, code} → invalid (garante que erro verdadeiro ainda falha)
  • Suíte existente: 19/19 verdes (streaming-empty-content-block, combo-quality-validator-reasoning, combo-responses-sse-failure-fallback)
  • combo-routing-engine + typecheck em validação

Escopo

Toca apenas open-sse/services/combo/validateQuality.ts (1 helper novo + 1 linha) + 1 teste novo.

⚠️ base-red inherited: #9737

…idation

isStreamingUpstreamError used a key-presence check (parsed.error != null)
which false-positives on benign values some backends emit on every chunk
({}, '', false, 0). When opencode issues a tool-call turn, the upstream SSE
opens with role-only frames (no recognized content) and a later chunk that
carries real tool_calls content PLUS a benign empty error field. The error
gate runs BEFORE content recognizers, so that single frame short-circuits
to 'error' -> 502 'streaming upstream error'. Same combo via kilocode works
because its wire format never emits the empty error field.

Fix: isSubstantiveError() helper — only treat error as real when it carries
non-empty string, non-empty object, or explicit true. Empty object {}, empty
string '', false, and 0 are benign.

TDD: tests/unit/quality-validation-benign-error.test.ts proves tool_calls
chunk with error:{} or error:'' is valid (was 502), while a real error
{message, code} still correctly fails.
@diegosouzapw
diegosouzapw merged commit 13dcbfd into release/v3.8.50 Aug 9, 2026
22 checks passed
@diegosouzapw

Copy link
Copy Markdown
Owner Author

Thanks for the contribution. PR looks structurally viable from diff-level metadata. Please confirm gates with /merge-prs flow before merge.

2 similar comments
@diegosouzapw

Copy link
Copy Markdown
Owner Author

Thanks for the contribution. PR looks structurally viable from diff-level metadata. Please confirm gates with /merge-prs flow before merge.

@diegosouzapw

Copy link
Copy Markdown
Owner Author

Thanks for the contribution. PR looks structurally viable from diff-level metadata. Please confirm gates with /merge-prs flow before merge.

@diegosouzapw
diegosouzapw deleted the fix/escalated-quality-validation-benign-error branch August 10, 2026 09:59
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…quality-validation-benign-error

fix(combo): ignore benign empty error fields in streaming quality validation (502 false-positive on opencode tool calls)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant