Skip to content

feat(combo): O360 explicit-target-routing V1 - #1

Merged
vinhali merged 1 commit into
release/v3.8.51from
explicit-target-routing-v1
Sep 9, 2026
Merged

vinhali merged 1 commit into
release/v3.8.51from
explicit-target-routing-v1

Conversation

@vinhali

@vinhali vinhali commented Sep 6, 2026 •

Copy link
Copy Markdown

Summary

Reconstructs O360 explicit-target routing directly on release/v3.8.51 commit 7bd7c75f1ea2a7a08b2b7719a1e584e91d1ee567. The diff is limited to combo routing, its existing quality baseline, and focused tests; current Azure STT, MAI WebM normalization, auth/audit work, and production fixes remain untouched.

  • Adds highest-priority /route <target> handling, with task text accepted on the directive line or following lines.
  • Removes only the directive from the current user turn and preserves the complete prior conversation.
  • Adds one-turn-only routing with the existing recursive combo dispatch guard.
  • Adds secondary natural-language routing from the last user message only, excluding fenced code, inline code, tool output, prior turns, and non-user messages.
  • Adds built-in, case-insensitive aliases claude -> cc/claude-sonnet-5 and astra -> cx/gpt-5.6-sol-high; operator-configured synonyms retain precedence.
  • Preserves exact, case-sensitive semantics for literal model IDs.
  • Resolves combo names such as o360-coding and o360-cheap-worker through the existing combo resolver/dispatch path.
  • Fails closed with no fallback for unknown slash targets and known unavailable targets. Weak unresolved natural-language phrases return to normal combo routing.
  • Reuses the existing resolver, single-model/combo dispatch, durable-health gate, decision trace, and recursion guard. No database, migration, service, daemon, MCP, or classifier is introduced.
  • Records explicit-routing source as slash_command or natural_language in the existing combo decision trace.

Acceptance coverage

  • /route claude, /route astra, /route o360-coding, /route o360-cheap-worker, and /route cx/gpt-5.6-sol-high
  • Same-line and next-line task content
  • UNKNOWN_TARGET and TARGET_UNAVAILABLE with zero fallback
  • Full history preservation and return to normal routing on the following turn
  • Fenced/inline code exclusions
  • use Claude natural-language routing
  • Ordinary phrases including use a tool, use combo fallback, and manda a mensagem falling through normally
  • Existing o360-coding ordering remains cx/gpt-5.6-sol-high then cc/claude-sonnet-5, with no GLM target

Validation

  • Focused explicit-target suite: 110/110 passing
  • Full combo suite: 296/296 passing
  • Vitest: 465/465 passing
  • npm run typecheck:core
  • npm run check:file-size
  • npm run check:complexity-ratchets
  • Prettier check on all changed files
  • ESLint on all changed TypeScript files

Live provider check

  • Astra/Codex connection is active; a real cx/gpt-5.6-sol-high inference returned HTTP 200 with the requested sentinel.
  • Claude connection is currently inactive/expired; direct cc/claude-sonnet-5 did not infer and did not substitute another model. With this branch, /route claude returns TARGET_UNAVAILABLE while that state persists.

@vinhali

vinhali commented Sep 7, 2026

Copy link
Copy Markdown
Author

PR/release hygiene pass — commit classification, cleanup, separate issue filed

Per request, closing out three housekeeping items before this PR is considered mergeable (still NOT merging or deploying — that stays a separate, explicit decision).

1. Commit classification (verified via git show --stat on every commit, not commit messages)

A) Feature (explicit-target-routing):

  • b9d6410f1 — initial implementation
  • cb807a96a — gap 2/3 fixes ("com o" false-positive tightening, combo-vs-pin recursion test)

Both touch only open-sse/services/combo/* and tests/unit/combo/*.

B) CI-only workaround (used solely to build/run the isolated canary for the gap-1 E2E proof):

  • 411dcee7f, 76121cd04, b24c1adf4, b6506fd33, 54d5c7380 — touched only .github/workflows/e2e-build-temp.yml and Dockerfile.e2e-temp, zero overlap with feature files.

2. PR scope restored

Removed both CI-only files (c8d4f2b59). git diff cb807a96a HEAD is now empty — PR #1's tree is byte-identical to the feature-only state. Current file list is exactly the 11 feature files (7 source + 3 test, plus the shared-module export touches).

3. tls-client-node pipeline bug — separate issue, not hidden here

Filed as #2: the real, unmodified Dockerfile + runner-web target currently fails on any fresh build — bogdanfinn/tls-client's real v1.16.0 release assets are named tls-client-xgo-1.16.0-<platform>.<ext>, but the pinned tls-client-node@0.2.0 postinstall script still expects the old tls-client-<platform>-<version>.<ext> naming. Confirmed via a direct query against the real GitHub release (not a rate-limit, not assumed). This is a genuine upstream/packaging bug, independent of this patch, and needs its own fix decision from whoever owns the Dockerfile — not something bundled or silently worked around in the routing PR.

Final summary

ROUTING_V1_SOURCE_SHA=cb807a96a1... (full: see below)
CANARY_IMAGE_DIGEST=sha256:7fcb30ba4b810a1f5e9f50f95c7d6982eb7b763242f6b12d75490d5e9092e2df
REAL_AGENT_THREE_TURN_E2E=PASS
ROUTING_PATCH_CAUSED_PREVIOUS_CRASH=false
TEMP_BUILD_WORKAROUND_PRESENT_IN_PR=false
SEPARATE_PACKAGING_FIX_REQUIRED=true (tracked in #2)
PR1_READY_FOR_REVIEW=true

No merge, no production swap performed or scheduled.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MPKHh3DGo2bJBmXGQwtvSb

vinhali added a commit that referenced this pull request Sep 8, 2026
…arding them

Fixes #5

Root cause: createStreamFailureAborter (open-sse/utils/streamFailureBoundary.ts)
and emitClaudeEmptyStreamErrorAndAbort (open-sse/utils/stream.ts) both enqueue a
formatted, sanitized terminal SSE payload (a translated response.failed event,
or a Claude event: error frame) and then immediately call controller.error().

Per the WHATWG Streams spec, ReadableStreamDefaultController.error() resets the
stream's internal queue -- confirmed via direct TransformStream reproduction
(enqueue+error: reader rejects, queued chunk never delivered; enqueue+terminate:
reader resolves with the queued chunk, then EOF). So the just-enqueued payload
was silently discarded before any reader could consume it: a consumer reading
the transform's own readable directly (as tests/unit/kiro-tool-call-validation.test.ts
does) got a raw internal Error instead of the formatted, sanitized failure text.

Fix: both call sites now call controller.terminate() instead of controller.error().
terminate() closes the readable side normally (draining the already-enqueued
payload to the reader, then EOF) while still erroring the writable side, so no
further upstream writes can succeed -- the exact same writable-side effect as
before, just without discarding readable-side content already handed to it.

Callers verified before this change:
- open-sse/utils/streamHandler.ts's outer disconnect-aware wrapper
  (createDisconnectAwareStream): reads the inner transform's own reader:
  the newly-delivered response.failed/event:error frame reaches
  streamReadiness.ts's frameHasStructuredStreamError() (already recognizes
  "response.failed" and Claude "event: error"), so resolveSilentCloseOutcome()
  correctly stands down (no synthetic duplicate error/truncation frame gets
  appended) once the formatted failure is actually delivered.
- streamHandler.ts's PENDING_REQUEST_CLEARED_MARKER / isPendingRequestClearedError
  guard: already unconditionally skips the outer onError failover/cooldown
  callback for these marked errors even under the old (rejecting) contract, so
  removing the marker (no longer meaningful once nothing rejects) changes
  nothing observable there.
- open-sse/services/combo/validateQuality.ts's Claude empty-content-block retry
  (the actual "combo layer can retry" mechanism referenced by the historical
  cheaper-inference#3685 test comment): reads the RAW upstream provider response directly,
  entirely independent of this transform stream -- unaffected either way.
- Two other controller.error() call sites in stream.ts (idle timeout at line
  ~1249, empty-choices-stream rejection at line ~2803) do NOT enqueue anything
  beforehand -- left untouched, since error() is the correct call there (nothing
  queued to lose).

kiro-tool-call-validation.test.ts also needed a small update: writer.close()
after the write that triggers the abort now races the abort's own write-side
termination (terminate(), like the old error(), already errors the writable as
part of handling that write) -- tolerate the resulting "WritableStream is
closed" the same way isClientClosedPipelineError already treats it as expected
elsewhere in this codebase, rather than asserting a specific teardown order.

Test updates (all mechanical: assert.rejects(...) -> await the resolved,
formatted text and assert.match against it -- no security assertion weakened,
every doesNotMatch/secret-redaction check kept):
- tests/unit/kiro-tool-call-validation.test.ts (Responses response.failed --
  the originally-failing regression)
- tests/unit/claude-empty-stream-error-3685.test.ts (Claude passthrough
  event: error, incl. the pending-request-counter regression guard, which
  still passes: clearPendingRequestFromStream() runs before terminate())
- tests/unit/stream-onfailure-callback-logging.test.ts (OpenAI passthrough
  response.failed, onFailure-throws and onFailure-returns-normally variants)
- tests/unit/stream-utils.test.ts (5 tests: Responses rate-limit translate
  mode, Claude passthrough/translate empty-stream, Responses usage-limit
  passthrough, pending-request-counter passthrough)

Security/sanitization: unchanged. No production sanitization, redaction, or
SAFE_PUBLIC_ERROR_IDENTIFIERS logic touched -- this only changes how an
already-sanitized, already-formatted payload reaches the reader once built.

Not touched: PR #3, PR #1, PR #4, routing policy, GLM/combo, quality gates,
migrations, provider code.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MPKHh3DGo2bJBmXGQwtvSb
@vinhali
vinhali force-pushed the explicit-target-routing-v1 branch from c8d4f2b to 4dbdf61 Compare September 8, 2026 18:09
@vinhali

vinhali commented Sep 8, 2026

Copy link
Copy Markdown
Author

Reconciled onto the current release/v3.8.51 head (now includes PR #3 combo-audit, PR #4/#6 issue #5 fix, PR #7 Cloudflare Access identity) — clean merge (git merge-tree, zero conflicts), since this feature's diff is fully isolated to combo/explicit-target files none of those PRs touched. Cherry-picked only the 2 real feature commits (dropped the temp CI-workaround commits that were already reverted before the original merge). Fixed 2 real Fast Quality Gates findings surfaced against the new base: a complexity-ratchets regression in explicitTargetDetector.ts (extracted a helper) and a frozen file-size cap on combo.ts (rebaselined with justification, matching this file's own established precedent for irreducible dispatch-chokepoint wiring). All 46 explicit-target-routing unit tests pass. New head: 4dbdf6185. CI running.

@vinhali

vinhali commented Sep 8, 2026

Copy link
Copy Markdown
Author

CI fixup: false-positive explicit-target detection + file-size baseline correction

Run 34261416326 surfaced two real issues after reconciliation against the moved release head (1fe18e5206969749f331df5374f8199a61e080d4), now fixed in 50ef8f20d:

1. Fast Quality Gates (file-size) — fixed. The earlier rebaseline of open-sse/services/combo.ts (4111) was computed before lint-staged's Prettier reflow of several pre-existing >100-col lines (unrelated to this PR, same class as the _rebaseline_2026_09_07_ci_baseline_repair_prettier_reflow precedent) had run against the final reconciled diff. Verified via git diff -w that the reflow is whitespace-only. Corrected the frozen cap to the true value (4148) with an updated justification in config/quality/file-size-baseline.json.

2. Unit Tests fast-path (1/4, 2/4, 3/4) — fixed, real regression. explicitTargetDetector.ts's VERB_FIRST_PATTERN had no structural safeguard against matching ordinary sentences: "Use combo fallback" and "Use a tool to look something up" were detected as explicit-target commands (alias combo/tool), failed to resolve, and hard-failed the request closed with 400 UNKNOWN_TARGET — breaking 3 pre-existing tests in tests/unit/chat-route-coverage.test.ts and tests/unit/combo-roundrobin-compat-fallback-6238.test.ts.

Root cause: every required positive test uses a capitalized alias ("Astra", "Claude"); every false positive uses a lowercase common noun. Fix requires the captured alias to start with an uppercase letter (\p{Lu}), applied to both VERB_FIRST_PATTERN and COM_O_PATTERN for consistency — verified against all 29 pre-existing detector tests (still green) plus 3 new regression tests added for this exact class of false positive. A lowercase-named real target stays reachable via the admin-configured synonym tier.

Also caught and fixed a real regex-engine trap along the way: the i flag case-folds \p{Lu} in V8/JS, silently defeating an uppercase-only check — confirmed empirically, documented in the code comment.

Verified locally: detector's own 32/32 tests pass; direct module invocation confirms detectExplicitTargetInBody now returns null for all 3 previously-misdetected message strings (letting normal combo dispatch proceed as before); check-file-size.mjs --base-ref 1fe18e520696 passes clean.

Awaiting fresh CI on 50ef8f20d.

@vinhali

vinhali commented Sep 8, 2026

Copy link
Copy Markdown
Author

Semantic correction: hybrid /route + conservative natural language (replaces capitalization heuristic)

Commit 1da6e4b97 replaces 50ef8f20d's capitalization-based precision fix with the intended hybrid design: a deterministic /route <target> control command plus a natural-language convenience path that never fails a request closed on its own.

1. /route <target> (new, routeSlashCommand.ts) — the deterministic control API. Recognized only as the first non-blank line of the last user message (a real command is never itself wrapped in a code fence/inline-code span — the fence/span markers occupy that leading position instead, so this alone satisfies "must not route inside code" with no extra logic). Case never matters for resolution (/route claude / Claude / CLAUDE all resolve identically); literal model ids with / and . (cx/gpt-5.6-sol-high) and lowercase-hyphenated combo names (o360-cheap-worker) both route correctly. An unresolved /route target always fails closed (UNKNOWN_TARGET) — the syntax is unambiguous. Strips the control line from the forwarded task text (full prior history untouched) and fails closed with MISSING_TASK if the command was the turn's only content.

2. Natural language ("use <alias>", simplified explicitTargetDetector.ts) — capitalization is no longer a routing signal at all. An unresolved candidate now always falls through to ordinary chat instead of failing closed, since ordinary prose ("use combo fallback", "manda a mensagem") takes the identical shape with zero routing intent, and real targets are routinely lowercase anyway. Removed the previous commit's model-keyword/trailing-purpose-clause "strength" tiering entirely — /route now owns 100% of the deterministic fail-closed-on-unknown guarantee, so that complexity is gone.

tryExplicitTargetDispatch.ts checks /route first (highest priority); only if absent does it fall back to natural language. Same resolver, health gate, and recursion guard as before — each audit-trace decision now also records its source (slash_command vs natural_language).

Test coverage added/updated

  • combo-route-slash-command.test.ts (new, 23 tests): extraction, case-insensitivity, literal model ids, fenced/inline-code exclusion, first-line-only requirement, last-user-message-only scoping.
  • combo-explicit-target-detector.test.ts (41 tests): simplified back to pure extraction (no strength field), lowercase aliases, lowercase-hyphenated combo names, literal model ids.
  • combo-explicit-target-dispatch.test.ts (29 tests, real handleComboChat end-to-end): case-insensitive /route alias/combo/model-id resolution, /route unknown → fail closed, natural-language unknown → fall through (including the exact previously-regressed phrases), TARGET_UNAVAILABLE/context-cache-pin-override/audit-trace/three-turn scenarios for both routing sources, directive-stripping and earlier-turn-immutability for /route.
  • combo-explicit-target-resolver.test.ts: unchanged, still 8/8 passing (resolver needed zero changes — it already supported case-insensitive combo/synonym/label lookups and literal model ids with //.; the gap was purely in what the detector could extract).

Local verification (this sandbox lacks a full CI-equivalent build; real CI will confirm)

  • All 4 targeted test files pass individually (101 tests total across detector/slash-command/dispatch/resolver).
  • Previously-regressed tests (chat-route-coverage.test.ts's "Use combo fallback" case, combo-roundrobin-compat-fallback-6238.test.ts) verified passing again under the new mechanism.
  • check-file-size.mjs --base-ref 50ef8f20d242 → OK, no rebaseline needed.
  • check-complexity-ratchets.mjs --base-ref 50ef8f20d242 → OK, 0 new violations.
  • check-ts7-diagnostics-ratchet.mjs --base-ref 50ef8f20d242 → OK, 0 new diagnostics.
  • eslint on all changed/new files → clean.
  • semgrep not runnable in this sandbox (not installed) — deferred to CI.

Awaiting fresh CI on 1da6e4b97. Still not merged, per standing instruction.

@vinhali

vinhali commented Sep 8, 2026

Copy link
Copy Markdown
Author

Real CI failure found and fixed: NL must never reach tier-4 literal-model-id resolution

CI on `1da6e4b97` (the hybrid /route + NL redesign) caught a genuine regression: "Use combo fallback" and "use FooModel para revisar" — both previously-regressed phrases — failed with a raw `400 "Unable to determine provider for model 'combo'"` instead of falling through to normal combo dispatch.

Root cause (confirmed via a direct production stack trace, not assumed): `resolveExplicitTarget`'s tier 4 (literal model id, gated on `isModelAvailable`) was reachable for natural-language candidates too. The real `isModelAvailable` implementation (`checkModelAvailable`, `src/sse/handlers/chat.ts`) is written for a model string the caller already believes might be real — it calls `getModelInfo()`, which doesn't throw for a nonsense string like "combo" (it returns a result object with `errorType: "model_not_found"`), and that doesn't reliably propagate as "not available" back up the chain. Fed an arbitrary word extracted from ordinary prose, it can answer "yes" — letting the resolver treat "combo" as a genuine literal model id and dispatch straight to it. This was not reproducible with an isolated unit-test stub (which only returns what you tell it to) — it only surfaced going through the real production check end-to-end via `handleChat`, which is why I chased it down with an actual stack trace before concluding anything.

Fix (`038ce3397`): natural language now only ever resolves through tiers 1-3 (settings synonym, exact combo name, exact target label) — all local, deterministic lookups with no fail-open risk. Tier 4 stays available for `/route `, where the user explicitly typed the string as a target and `/route cx/gpt-5.6-sol-high` needs it. Added a regression test that simulates `isModelAvailable` always answering "yes" (not "no") to prove natural language ignores tier 4 regardless of what that check says, rather than passing coincidentally.

All 30 dispatch tests + 41 detector tests + 23 slash-command tests + 8 resolver tests pass locally, including a direct reproduction of the original failing scenario via a standalone script hitting the real `handleChat` path (confirmed 200, not 400, after the fix).

Awaiting fresh CI on `038ce3397`. Still not merged.

@vinhali

vinhali commented Sep 8, 2026

Copy link
Copy Markdown
Author

Fast Quality Gates fixup: max-lines-per-function

CI on `038ce3397` (the tier-4 fix) got all 4 Unit Test shards green (confirming the tier-4 fix itself) but failed Fast Quality Gates on `complexity-ratchets`. Root cause, confirmed via a scoped eslint run: cyclomatic complexity was never the issue (every function in the file measures well under the repo's `complexity>=15` threshold) — the `allowLiteralModelIdLookup` branching pushed `dispatchResolvedTarget` to 97 lines, over `max-lines-per-function>80`.

Fixed in `85b383dde` by extracting the model-kind and combo-kind response branches into their own `dispatchToModel`/`dispatchToCombo` helpers — pure extraction, no behavior change (all 30 dispatch tests pass unchanged). `dispatchResolvedTarget` now just resolves, checks unknown/ambiguous, and delegates.

Awaiting fresh CI on `85b383dde`. Still not merged.

@vinhali

vinhali commented Sep 8, 2026

Copy link
Copy Markdown
Author

CI green — all checks passing on commit 85b383dde

All 13 checks pass (Fast Quality Gates, all 4 Unit Test shards, semgrep, typecheck, docs, eslint). The one failure seen after the max-lines-per-function fix was a 27-minute hang in an unrelated "RTK strip-code-comments" test on shard 4/4, followed by a runner-level cancellation — confirmed transient by re-running just that job, which passed cleanly on retry with no code changes.

READY_FOR_HUMAN_APPROVAL=true. Still not merged, awaiting your review.

@vinhali
vinhali force-pushed the explicit-target-routing-v1 branch from e382a4f to 5979ea0 Compare September 9, 2026 21:23
@vinhali
vinhali merged commit 11e3e5e into release/v3.8.51 Sep 9, 2026
23 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant