Skip to content

fix(sse): prevent Anthropic 400s for Claude-native handoffs - #12668

Merged
diegosouzapw merged 3 commits into
cheaper-inference:release/v3.8.51from
insoln:fix/anthropic-native-handoff-system
Sep 17, 2026
Merged

diegosouzapw merged 3 commits into
cheaper-inference:release/v3.8.51from
insoln:fix/anthropic-native-handoff-system

Conversation

@insoln

@insoln insoln commented Sep 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Route Claude-native context handoffs through Anthropic's top-level system parameter instead of inserting a non-empty role: "system" message at messages[0].
  • Add a final Claude executor invariant that hoists leading prompt-bearing system/developer messages, preserves directive-only output_config envelopes, and relocates those directives away from the forbidden first position.
  • Preserve the existing OpenAI-format and Responses API handoff paths.

Anthropic rejects the previous Claude-native wire shape with HTTP 400 because initial system prompts belong in the top-level system parameter. The directive-only form (content: [] with output_config) remains supported and is handled separately.

Related Issues

Validation

Change type and focused loop from the Contribution Golden Path:

  • Change type: routing
  • Focused regression and adjacent compatibility tests:
    • node --import tsx/esm --test tests/unit/context-handoff.test.ts tests/unit/universal-handoff.test.ts tests/unit/claude-directive-final-dispatch.test.ts tests/unit/claude-directive-only-relocation.test.ts tests/unit/claude-directive-midconv-passthrough.test.ts tests/unit/claude-system-role-cache-boundary.test.ts tests/unit/claude-code-compatible-request.test.ts tests/unit/chatcore-claude-upstream-messages.test.ts — 113 passed
  • TDD evidence: the six new regression cases failed on the untouched release base, then passed after the implementation
  • npm run typecheck:core
  • Focused ESLint on touched surfaces
  • npm run test:vitest — 51 files, 465 tests passed
  • git diff --check
  • Reconciled with the current release/v3.8.51 tip (910f58c5c); focused checks rerun afterward
  • Production-code changes include new or updated automated tests in this PR
  • npm run lint — inherited failure at src/app/(dashboard)/dashboard/combos/page.tsx:774; the file is untouched by this PR
  • npm run test:unit / npm run test:coverage — long local runs were externally terminated after more than 20 minutes. Before termination, failures were limited to unrelated environment/performance tests and reproduced independently of the changed modules. CI remains authoritative for the complete runs.
  • Fast-path unit shards / API route typecheck — inherited from the moving release base. After this PR opened, release/v3.8.51 advanced from 910f58c5c to 2265ce761 through the public-error-boundary campaign. On clean base SHA 2265ce761 (without this PR), the exact failing files reproduce: dashboard request-failure redaction, generic stream public error boundaries, HuggingChat transport redaction, Kiro Responses stream errors, model-sync invalid-JSON handling, and tunnel-error sanitization. API Route Typecheck and No new ESLint warnings are likewise red on unrelated/base runs; none of their failing files are modified here.

⚠️ base-red inherited: #12581

Tests Added Or Updated

  • tests/unit/context-handoff.test.ts
  • tests/unit/universal-handoff.test.ts
  • tests/unit/claude-directive-final-dispatch.test.ts

Coverage Notes

  • Direct handoff tests cover Claude-native top-level system placement and the unchanged OpenAI message-based path.
  • Universal/combo handoff tests cover existing system arrays, absent system prompts, and source-format discrimination.
  • Final-dispatch tests capture the serialized Anthropic request and cover prompt hoisting, directive relocation, cache-control preservation, and top-level output_config precedence.

Reviewer Notes

The two repair layers are intentional:

  1. Source-level placement prevents universal and direct Claude-native handoffs from creating an invalid body.
  2. The final executor guard protects direct executor callers and payloads altered by later transformations before serialization/fingerprinting/signing.

The guard order matters: prompt-bearing leading system messages are hoisted first, then any directive-only envelope exposed at messages[0] is relocated, then existing thinking constraints are normalized.

No migrations, feature flags, provider weights, combo policy, credentials, or deployment configuration are changed.

@insoln
insoln marked this pull request as ready for review September 4, 2026 01:54
@insoln
insoln requested a review from diegosouzapw as a code owner September 4, 2026 01:54
diegosouzapw pushed a commit that referenced this pull request Sep 4, 2026
…g conversation (#12646)

Validado sobre o tip de `release/v3.8.51`, com duas coisas resolvidas antes do merge.

**A falha de CI era stale.** O job `No new ESLint warnings` deste PR apontava `react-hooks/set-state-in-effect` em `src/app/(dashboard)/dashboard/combos/page.tsx:774` — arquivo que este PR não toca, e o mesmo erro aparecia em #12668 e #12672, que também não o tocam. A linha do tempo: o #12355 introduziu a violação de manhã, os CIs rodaram nessa janela, e o #12607 acrescentou a entrada de supressão à tarde. Medido no tip atual com o comando exato do job: **0 ocorrências não suprimidas**. A supressão sobrevivente é "unpruned", e o script passa `--pass-on-unpruned-suppressions` justamente para isso não bloquear.

**Faltava o teste que a regra do projeto exige** para mudanças em `src/`. Acrescentei `tests/unit/ui/log-detail-conversation-link-12646.test.tsx`, verificado **RED-then-GREEN** em vez de escrito contra o código pronto: revertendo `RequestLoggerDetail.sections.tsx` para o tip, 2 dos 3 casos falham; com a mudança deste PR, 3/3 passam.

Detalhe que valeu a pena descobrir: a seção curto-circuita em `allTurns.length === 0`, então o fixture precisa de um `requestBody` que normalize em pelo menos um turno — sem isso o cabeçalho inteiro nunca monta e as asserções passariam pelo motivo errado. O teste fixa três coisas: o href para um `sessionTag` simples, o percent-encoding para um que não é URL-safe, e a ausência de link quando não há `sessionTag`.

Obrigado, @hartmark.
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Thanks for the detailed TDD writeup and live-repro evidence — this is a real Anthropic 400
and the top-level system parameter fix looks correct. I ran a sample of your new/changed
tests (context-handoff, universal-handoff, claude-directive-final-dispatch — 51/51 pass) in
an isolated worktree. Before merge we'd like to see the full focused suite + test:vitest
rerun against the current release tip (you flagged that the base moved mid-PR), and we'll
coordinate the merge order with #12735/#13355 since all three touch the handoff-flag reading
code in src/sse/handlers/chat.ts.

@insoln
insoln force-pushed the fix/anthropic-native-handoff-system branch from df4ed00 to 452fa6a Compare September 16, 2026 18:05
@insoln

insoln commented Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto the current release/v3.8.51 tip (86b8bbfd3e) and rerun the full validation you asked for:

  • npm run typecheck:core — clean
  • Focused suite (context-handoff, universal-handoff, claude-directive-final-dispatch, claude-directive-only-relocation, claude-directive-midconv-passthrough, claude-system-role-cache-boundary, claude-code-compatible-request, chatcore-claude-upstream-messages) — 113/113 pass
  • npm run test:vitest — all green

Rebase note: the universal-handoff injection call site moved out of combo.ts into combo/executeTargetAttempt.ts on the new base, so the sourceFormat parameter now flows through AttemptLoopDeps (attemptLoopTypes.ts) instead of a local closure. No behavioral change — the same sixth-argument call reaches injectUniversalHandoffBody.

Re #12735/#13355 merge-order coordination: after the rebase there is no file overlap with either PR (#12735 touches executors/dario.ts/ninerouter.ts, #13355 touches config/cliFingerprints.ts; this PR no longer modifies those or vice versa). The only shared surface is conceptual — all three sanitize the upstream request body — so any merge order works; if either of them lands first I'll rebase again to keep this PR conflict-free.

@insoln

insoln commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor Author

Rebased onto the current release/v3.8.51 tip (86b8bbf) and confirmed merge-order coordination with #12735 / #13355: after the rebase there is no file overlap with either PR, so merge order is free.

Gates this PR is responsible for are green on the latest run (77a3fed): Fast file-size (PR mode), complexity ratchets (complexity 56=base 56, cognitive 24=base 24), Vitest, ESLint, docs, semgrep, typecheck.

The remaining red checks reproduce from files this PR does not touch — they are pre-existing on the release tip:

  • check:mutation-test-coverage: 3 covering tests missing from stryker.conf.json (tests/unit/combo-identical-error-streak.test.ts, tests/unit/13601-header-drop-count-surfaced.test.ts, tests/unit/semantic-cache-no-truncated-writes.test.ts) — drift from the 2026-09-16 merge wave (chore: reconcile the 2026-09-16 merge wave with the release tip #13904), unrelated modules.
  • Unit shards: raw NUL byte in src/app/api/providers/[id]/test/route.ts; stale docs/screenshots/free-tier-budget-card.svg (regenerate via gen-budget-card-svg.mjs); chatCore xhigh-effort downgrade; authz pipeline 403; context-editing-relays F4.x; reasoning-effort clamp/retry hydration; sse-auth forced-pin fix(sse): keep unavailable forced connections scoped #12080.

None of these files appear in this PR's diff. Happy to split any of them into a separate fix PR if useful.

Keep the Claude wire-body guard while staying under the frozen per-file line baselines and the complexity ratchets measured against release/v3.8.51.

Extract the final constraint coordinator, split system-message normalization into focused helpers, and isolate handoff response parsing. Reflow the universal-handoff explanation to absorb the added source-format argument without growing the frozen file.
@insoln
insoln force-pushed the fix/anthropic-native-handoff-system branch from 77a3fed to b53f15f Compare September 16, 2026 20:41
@diegosouzapw
diegosouzapw merged commit c97f61b into cheaper-inference:release/v3.8.51 Sep 17, 2026
10 of 16 checks passed
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…g conversation (cheaper-inference#12646)

Validado sobre o tip de `release/v3.8.51`, com duas coisas resolvidas antes do merge.

**A falha de CI era stale.** O job `No new ESLint warnings` deste PR apontava `react-hooks/set-state-in-effect` em `src/app/(dashboard)/dashboard/combos/page.tsx:774` — arquivo que este PR não toca, e o mesmo erro aparecia em cheaper-inference#12668 e cheaper-inference#12672, que também não o tocam. A linha do tempo: o cheaper-inference#12355 introduziu a violação de manhã, os CIs rodaram nessa janela, e o cheaper-inference#12607 acrescentou a entrada de supressão à tarde. Medido no tip atual com o comando exato do job: **0 ocorrências não suprimidas**. A supressão sobrevivente é "unpruned", e o script passa `--pass-on-unpruned-suppressions` justamente para isso não bloquear.

**Faltava o teste que a regra do projeto exige** para mudanças em `src/`. Acrescentei `tests/unit/ui/log-detail-conversation-link-12646.test.tsx`, verificado **RED-then-GREEN** em vez de escrito contra o código pronto: revertendo `RequestLoggerDetail.sections.tsx` para o tip, 2 dos 3 casos falham; com a mudança deste PR, 3/3 passam.

Detalhe que valeu a pena descobrir: a seção curto-circuita em `allTurns.length === 0`, então o fixture precisa de um `requestBody` que normalize em pelo menos um turno — sem isso o cabeçalho inteiro nunca monta e as asserções passariam pelo motivo errado. O teste fixa três coisas: o href para um `sessionTag` simples, o percent-encoding para um que não é URL-safe, e a ausência de link quando não há `sessionTag`.

Obrigado, @hartmark.
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…inference#12668)

* fix(sse): prevent Anthropic 400s for Claude-native handoffs

* docs(changelog): add fragment for Claude-native handoff 400 fix

* refactor(sse): satisfy file-size and complexity ratchets

Keep the Claude wire-body guard while staying under the frozen per-file line baselines and the complexity ratchets measured against release/v3.8.51.

Extract the final constraint coordinator, split system-message normalization into focused helpers, and isolate handoff response parsing. Reflow the universal-handoff explanation to absorb the added source-format argument without growing the frozen file.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants