Skip to content

feat: add Claude Opus 4.7 support (adaptive-only thinking, sampling param constraints) - #1386

Closed
i1hwan wants to merge 65 commits into
cheaper-inference:mainfrom
i1hwan:feat/opus-4.7-support
Closed

i1hwan wants to merge 65 commits into
cheaper-inference:mainfrom
i1hwan:feat/opus-4.7-support

Conversation

@i1hwan

@i1hwan i1hwan commented Apr 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Add Claude Opus 4.7 compatibility to the OpenAI-compat → Claude OAuth pipeline, addressing all Anthropic breaking changes while preserving full Opus 4.6 backward compatibility.

Anthropic Breaking Changes Addressed

Breaking Change Solution
thinking: {type:"enabled"} → 400 error coerceThinkingForModel() converts to type:"adaptive" at both thinkingBudget and translator layers
temperature/top_p/top_k non-default → 400 error Stripped in translator for rejectsSamplingParams models
thinking.display defaults to "omitted" Auto-inject display:"summarized" so OpenCode sees thinking text
xhigh effort (4.7 only) downgradeEffort() converts to max on unsupported models

Scope

OpenAI-compatible → Claude OAuth path only. Other paths (Claude-native passthrough, Claude→Claude, CC-compatible) are out of scope.

Actual Payload Diff (4.6 vs 4.7)

4.6 Provider Request (unchanged):

{ "thinking": { "type": "adaptive" }, "output_config": { "effort": "max" }, "temperature": 0.7 }

4.7 Provider Request (new):

{ "thinking": { "type": "adaptive", "display": "summarized" }, "output_config": { "effort": "max" } }

Changes: display field added, sampling params stripped.

Tests

  • New: 26/26 (opus-4.7-compat.test.mjs)
  • Regression: thinking-budget 30/30, provider-service 9/9, chatcore 43/43, translator 15/15, stream 16/16

Verification

  • Oracle validation: main OpenAI→Claude path confirmed correct, no type:enabled leak
  • Runtime payload trace: verified actual before/after transformations
  • Baseline tag: apex-v1.0.0 for safe rollback

i1hwan and others added 30 commits April 11, 2026 05:47
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
feat: add configurable Claude OAuth forwarding rules
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
fix(docker): harden runtime image packaging
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
fix(proxy): rewrite native Claude OAuth lexical triggers
fix(settings): preserve default forwarding keyword rewrites
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
fix(proxy): rewrite Claude Code compatible requests
fix(proxy): cover remaining Claude lexical rewrite bypasses
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
i1hwan and others added 18 commits April 12, 2026 02:31
fix(auth): restore upstream Claude refresh contract
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
fix(auth): restore oauth client id bootstrap defaults
…ails

Cache token data (cache_read_input_tokens, cache_creation_input_tokens)
was silently dropped when translating Claude responses to OpenAI format.
Clients received only prompt_tokens/completion_tokens/total_tokens,
breaking session length estimation in ai-sdk/openai-compatible consumers.

- filterUsageForFormat: promote flat cache/reasoning tokens to nested
  prompt_tokens_details and completion_tokens_details for OpenAI target
- claude-to-openai streaming: capture cache tokens from message_start,
  merge (not overwrite) in message_delta, emit details in both
  message_delta and message_stop fallback paths
- responseTranslator: include cache tokens in non-streaming Claude path
  with prompt_tokens = input + cache_read + cache_creation
- Add 6 regression tests covering all translation paths
…d details

Copilot review feedback: flat keys (cache_read_input_tokens, cached_tokens,
reasoning_tokens) remained on the usage object after promotion to nested
prompt_tokens_details / completion_tokens_details. ai-sdk z.looseObject
drops unknown keys anyway, but the duplicates polluted the output for
other clients. Now delete flat keys after merge and preserve existing
nested details.
stream.ts extractUsage overwrote state.usage on every SSE event,
destroying cache tokens captured from message_start when message_delta
arrived with only output_tokens. claude-to-openai.ts translator also
clobbered state.usage with raw input_tokens (26) instead of the full
prompt total (input + cache_read + cache_creation = 92123).

- stream.ts: merge extracted usage into state.usage with > 0 guards
  so earlier cache/prompt values survive later delta-only updates
- claude-to-openai.ts message_start: compute prompt_tokens including
  cache tokens and merge into existing state instead of overwriting
- claude-to-openai.ts message_delta: recompute prompt_tokens with
  accumulated cache when input_tokens arrives
- Remove dead cached_tokens/reasoning_tokens from OpenAI allowlist
- Update stream-utils test expectations for correct prompt inclusion

Oracle-validated: end-to-end simulation confirms prompt_tokens: 94123
(92123 + buffer) with prompt_tokens_details.cached_tokens: 91945.
fix(auth,usage): restore OAuth bootstrap + translate Claude cache tokens
reasoning_effort was silently dropped for Claude Code-compatible
requests because the preserveCacheControl passthrough path skipped
all translation, including applyThinkingBudget(). This meant clients
sending reasoning_effort='max' via OpenCode/ai-sdk had the field
passed through verbatim to Anthropic, which ignores it.

Anthropic Claude 4.6 requires thinking: { type: 'adaptive' } with
output_config.effort = 'max' instead. applyThinkingBudget() handles
this conversion cleanly and preserves cache_control markers, so we
can safely call it in the passthrough path without losing caching
semantics.

Verified: reasoning_effort='max' → thinking.type='adaptive' +
output_config.effort='max', with cache_control preserved.
normalizeThinkingConfig stripped thinking/reasoning_effort whenever the
last message was not from user. This broke multi-turn tool-use flows
on Anthropic: after the first assistant+tool_use turn, subsequent
requests had thinking silently removed, and Anthropic's stateless API
fell back to no-thinking mode.

Anthropic Messages API requires thinking on every request — it is not
session-sticky. Dropping it between turns disables extended thinking
and also invalidates the prompt cache (parameter change causes cache
miss).

For Claude targets, skip the last-message-from-user heuristic entirely
and let thinking/reasoning_effort/output_config pass through. For
non-Claude targets, keep the original behavior but also strip
output_config for consistency.

Evidence: dashboard showed thinking/output_config on request 1 (ends
with user), missing on request 2+ (ends with tool_result). After fix,
Claude target preserves these on every request regardless of last
message role.
…ontrol path

Adds 3 integration tests addressing Copilot review feedback on PR #15:

1. Reasoning_effort + cache_control: verifies that applyThinkingBudget()
   converts reasoning_effort='max' to thinking={type:'adaptive'} +
   output_config.effort='max' while preserving all cache_control markers.

2. No reasoning_effort: verifies that the passthrough path leaves the
   body untouched (no thinking/output_config injection) when client did
   not request reasoning, and cache_control is still preserved.

3. Multi-turn assistant-last: verifies thinking/output_config persist
   through tool_use rounds where the last message is assistant text,
   covering the normalizeThinkingConfig fix in commit cdae57b.
fix(thinking): apply thinking budget on cache_control passthrough path
…nstraints

- Add claude-opus-4-7 model entry (adaptive-only, rejects sampling params)
- Add supportedThinkingModes, supportedEfforts, defaultThinkingDisplay fields
- Add helpers: isAdaptiveOnlyModel, rejectsSamplingParams, downgradeEffort
- Fix claude-opus-4-6 contextWindow metadata (200k → 1M)

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
…s for Opus 4.7

- Add coerceThinkingForModel() post-processor to all applyThinkingBudget exits
- Coerce type:enabled → type:adaptive for adaptive-only models (prevents 400)
- Strip temperature/top_p/top_k for models that reject sampling params
- Inject thinking.display:summarized when model defaults to omitted
- Update reasoning_effort fallback to emit adaptive for 4.7
- Add adaptive detection to hasThinkingConfig()
- Register claude-opus-4-7 in OAuth provider model catalog

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Cover modelSpecs helpers, thinkingBudget coercion, translator sampling
strip, thinking.display injection, and hasThinkingConfig adaptive support.
Verify 4.6 backward compatibility across all scenarios.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Copilot AI review requested due to automatic review settings April 18, 2026 08:15
@i1hwan
i1hwan requested a review from diegosouzapw as a code owner April 18, 2026 08:15
@i1hwan

i1hwan commented Apr 18, 2026

Copy link
Copy Markdown
Contributor Author

Accidentally opened against upstream. This is a private fork change.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for Claude Opus 4.7, featuring adaptive thinking and specific model constraints such as the removal of sampling parameters. It implements a lexical rewriting system for Claude OAuth requests to manage keyword and tag replacements across various content blocks. Furthermore, it enhances usage tracking by mapping cache and reasoning tokens to nested OpenAI-style details and adds a new settings tab for configuring forwarding keyword rules. Feedback identifies a redundant ternary operator in the thinking budget coercion logic and warns of potential double-rewriting issues in the iterative lexical replacement loop.

result.thinking.type !== "adaptive" &&
result.thinking.type !== "disabled"
) {
const effort = result.output_config?.effort || result.thinking?.budget_tokens ? "high" : "high";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The ternary operator here is logically redundant as it returns "high" regardless of whether the condition is truthy or falsy. This makes the code confusing and suggests a potential logic error or a leftover from a refactor. If the intention is to always default to "high" when output_config.effort is missing, a simple constant assignment would be clearer.

Suggested change
const effort = result.output_config?.effort || result.thinking?.budget_tokens ? "high" : "high";
const effort = "high";

Comment on lines +173 to +175
for (const rule of rules.text) {
rewrittenText = rewrittenText.replaceAll(rule.match, rule.replace);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Applying multiple replaceAll calls in a loop can lead to unexpected results if a replacement string contains a keyword that matches a subsequent rule (double-rewriting). While the current default rules seem safe, this pattern is fragile for general-purpose lexical rewriting. Consider using a single-pass regex-based replacement or ensuring that rules are mutually exclusive.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Claude Opus 4.7 compatibility to the OpenAI-compatible → Claude OAuth pipeline by enforcing Anthropic’s new request constraints (adaptive-only thinking, sampling param rejection) while keeping Opus 4.6 behavior intact, plus adds configurable lexical rewrite rules for the Claude forwarding lane.

Changes:

  • Extend model specs + request translation to support Opus 4.7 constraints (adaptive-only thinking, effort downgrades, sampling param stripping, thinking.display injection).
  • Introduce configurable “forwarding keyword rules” (tool/text/tag rewrites) with API + dashboard UI, and apply them in the proxy forwarding path.
  • Improve usage accounting by preserving Claude cache token usage across streaming/non-streaming translations and promoting cache/reasoning fields into OpenAI *_details.

Reviewed changes

Copilot reviewed 41 out of 41 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
tests/unit/translator-resp-claude-to-openai.test.mjs Adds coverage for cache token usage propagation in Claude→OpenAI translation (streaming + non-stream).
tests/unit/translator-openai-to-claude.test.mjs Expands coverage for tool prefixing + lexical rewrites + thinking/effort promotion behaviors.
tests/unit/token-refresh-service.test.mjs Updates Claude OAuth refresh contract assertions and adds rejection-path coverage.
tests/unit/token-health-check.test.mjs Adds coverage for proxy unwrap helper used by refresh checks.
tests/unit/thinking-budget.test.mjs Updates expectations for new default thinking budget behavior and Claude adaptive-effort promotion logic.
tests/unit/stream-utils.test.mjs Adjusts expected total token accounting after usage merge changes.
tests/unit/provider-service.test.mjs Adds target-format-aware thinking normalization coverage (Claude vs non-Claude).
tests/unit/opus-4.7-compat.test.mjs New end-to-end-ish unit suite validating Opus 4.7 constraints across specs/budget/translation.
tests/unit/oauth-providers-config.test.mjs Adds redirectUri assertion for Claude OAuth config.
tests/unit/claude-code-compatible-request.test.mjs Updates effort handling to allow xhigh and validates lexical rewrites in CC-compatible requests.
tests/unit/chatcore-translation-paths.test.mjs Adds coverage for lexical rewrites and thinking budget behavior in chatCore Claude paths.
tests/unit/cc-compatible-provider.test.mjs Updates expected CC-compatible effort value (xhigh).
tests/unit/bootstrap-env.test.mjs Adds coverage for loading OAuth client-id defaults from .env.example when .env is absent.
src/sse/handlers/chat.ts Applies forwarding keyword + thinking budget settings during request handling.
src/shared/validation/settingsSchemas.ts Introduces Zod schemas for forwarding keyword rules + thinking budget updates.
src/shared/constants/modelSpecs.ts Adds Opus 4.6/4.7 model specs and helper functions for adaptive-only, effort, display, and sampling constraints.
src/lib/tokenHealthCheck.ts Adds helper to unwrap proxy resolution result for refresh calls.
src/app/api/settings/thinking-budget/route.ts Loads thinking budget settings on GET and switches schema import to settingsSchemas.
src/app/api/settings/forwarding-keywords/route.ts New API to GET/PUT forwarding keyword rewrite configuration persisted in settings DB.
src/app/api/settings/tests/forwarding-keywords.test.ts New Vitest coverage for the forwarding-keywords API route.
src/app/(dashboard)/dashboard/settings/page.tsx Adds ForwardingKeywordsTab to the Advanced settings section.
src/app/(dashboard)/dashboard/settings/components/ThinkingBudgetTab.tsx Extends adaptive mode UI to include “max” effort and clarifies adaptive behavior copy.
src/app/(dashboard)/dashboard/settings/components/ForwardingKeywordsTab.tsx New UI tab for editing forwarding keyword rules as JSON with reload/reset/save flows.
scripts/bootstrap-env.mjs Adds parsing of OAuth client-id defaults from .env.example and adjusts env layer priority.
open-sse/utils/usageTracking.ts Promotes Claude flat cache tokens + reasoning_tokens into OpenAI nested *_details and prunes flat keys.
open-sse/utils/stream.ts Changes usage extraction to merge rather than overwrite across events, preserving message_start usage.
open-sse/translator/response/claude-to-openai.ts Captures cache usage on message_start and includes cache tokens in OpenAI prompt_tokens_details.
open-sse/translator/request/openai-to-claude.ts Adds lexical rewrites, adaptive-only thinking coercion, effort downgrades, thinking.display injection, and sampling param stripping.
open-sse/translator/index.ts Makes thinking normalization target-format-aware.
open-sse/translator/helpers/claudeHelper.ts Adds Claude OAuth lexical rewrite helper for Claude-native payloads and tracks tool name mapping.
open-sse/services/tokenRefresh.ts Adds Claude OAuth refresh request meta logging and aligns refresh request contract.
open-sse/services/thinkingBudget.ts Shifts defaults to adaptive/max and adds Claude-specific adaptive thinking/effort promotion + adaptive-only coercion.
open-sse/services/provider.ts Recognizes adaptive thinking in hasThinkingConfig and preserves thinking for Claude targets regardless of last role.
open-sse/services/claudeCodeCompatible.ts Applies lexical rewrites to CC-compatible requests and expands supported effort values.
open-sse/handlers/responseTranslator.ts Includes cache tokens when computing OpenAI prompt_tokens and emits prompt_tokens_details in non-stream responses.
open-sse/handlers/chatCore.ts Applies thinking budget in Claude cache_control passthrough and applies lexical rewrite + toolNameMap handling for Claude.
open-sse/config/providerRegistry.ts Adds claude-opus-4-7 to the Claude provider registry.
open-sse/config/forwardingKeywordRules.ts New data-driven rewrite rule engine with defaults + normalization + per-lane rewrite helpers.
Dockerfile Cleans standalone build output and ensures .env.example is included in runtime image.
.github/workflows/docker-publish.yml Changes published image names/tags and limits platforms to linux/amd64.
.dockerignore Expands ignored files for container build context (CI/config/docs metadata).

Comment on lines +252 to +256
log?.warn?.(
"TOKEN_REFRESH",
"Claude OAuth refresh request details",
buildClaudeRefreshRequestMeta(refreshToken, headers, params, proxyConfig)
);
Comment on lines +443 to +449
result.thinking.type !== "disabled"
) {
const effort = result.output_config?.effort || result.thinking?.budget_tokens ? "high" : "high";
result.thinking = { type: "adaptive" };
if (!result.output_config?.effort) {
result.output_config = { ...(result.output_config || {}), effort };
}
Comment thread src/sse/handlers/chat.ts
Comment on lines +452 to +456
try {
const settings = await getSettings();
applyForwardingKeywordSettings(settings);
applyThinkingBudgetSettings(settings);
} catch (error) {
Comment on lines 45 to 50
// Default config (passthrough = backward compatible)
export const DEFAULT_THINKING_CONFIG = {
mode: ThinkingMode.PASSTHROUGH,
mode: ThinkingMode.ADAPTIVE,
customBudget: 10240,
effortLevel: "medium",
effortLevel: "max",
};
Comment on lines 22 to +74
@@ -65,13 +65,13 @@ jobs:
with:
context: .
target: runner-base
platforms: linux/amd64,linux/arm64
platforms: linux/amd64
push: true
tags: |
${{ env.IMAGE_NAME }}:${{ steps.version.outputs.version }}
${{ env.IMAGE_NAME }}:latest
ghcr.io/diegosouzapw/omniroute:${{ steps.version.outputs.version }}
ghcr.io/diegosouzapw/omniroute:latest
ghcr.io/i1hwan/apexroute:${{ steps.version.outputs.version }}
ghcr.io/i1hwan/apexroute:latest
This was referenced Apr 30, 2026
i1hwan added a commit to i1hwan/ApexRoute that referenced this pull request May 9, 2026
…redact PII from CHANGELOG

R3-1 (defect): touchSession(sessionId, connectionId) compat shim was
updating lastActive BEFORE forwarding to bindSessionConnection. That
made the forwarded call always see 'within affinity window' (because
lastActive had just been refreshed to Date.now()), so a session idle
>5 minutes would mis-fire the within-window alarm on its first
compat-shim rebind. The shim now binds first (so the binding decision
sees the pre-touch lastActive). Only when an entry pre-existed is
requestCount incremented — first_bind already sets it to 1, so a fresh
session via the compat shim sees requestCount == 1 (preserves the
original 1-touch == 1-request semantics that session-manager.test.mjs
relies on).

R3-2 (defect): selectByEarliestResetFirst was binding fall-through
selections itself with a hard-coded source: 'fall_through'. Since the
strategy ran BEFORE chat.ts:548's post-credential bind, the actual
within-window rebind always carried the wrong source — the
emergency_fallback alarm suppression added in stage-2 audit could
never engage in the emergency retry path. The strategy no longer binds.
chat.ts:548 is now the single binding site (single source of truth) and
selects source: 'emergency_fallback' vs 'explicit_post_credential' from
runtimeOptions.emergencyFallbackTried, so the suppression works as
intended. The unused bindSessionConnection import was removed from
the strategy.

Test count: 2875/2875 PASS (was 2874; +1 new compat-shim first-bind
contract test).

PII REDACTION (urgent user demand): CHANGELOG.md previously contained
real account names (APEXATGNU, GNUMAX) in production observation
narratives across multiple [3.8.x] sections. All occurrences have been
replaced with generic placeholders (Account A / Account B). The same
substitution was applied to .sisyphus/plans/*.md (workspace-only,
not git-tracked, but kept consistent for future readers). PR #28 body
on GitHub has also been edited to use the generic names. Merged commit
messages on main (PR #25 / #26 / #27) cannot be redacted without a
force-push to main, which is forbidden by AGENTS.md (incident cheaper-inference#1386
on 2026-04-18); their messages remain as-is on origin/main.
i1hwan added a commit to i1hwan/ApexRoute that referenced this pull request May 9, 2026
…Oracle two-stage verified) (#28)

* fix(routing,session): real session-affinity fix \u2014 D1 + D2 + D3 (Oracle two-stage verified)

Production observation on 3.8.5/3.8.6: a Claude OAuth session bound to
APEXATGNU switched to GNUMAX only 4 seconds after the last successful
APEXA request, on the same opencode-supplied x-session-affinity header,
losing 147k tokens of cache_write. Oracle hard-debug (ses_1fa7165c0...)
identified three independent design defects, all fixed here.

D1: extractExternalSessionId now recognises x-session-affinity (between
the generic forms and the omniroute-specific alias). Each header source
gets a hyphenated namespace prefix (ext-xsa: / ext-omr:) so that a legacy
generic value of e.g. 'xsa:foo' cannot collide with a newer
x-session-affinity: 'foo' request. The opencode CLI's stable session
continuity signal now flows through to bound-conn lookup; fingerprint
fallback only kicks in when no header is present.

D2: touchSession is split. touchSession(sessionId) is now a pure
activity update (lastActive, requestCount). All connection binding
mutations go through bindSessionConnection(sessionId, connectionId,
{ source }) which returns a BindResult with reasons no_session_id /
first_bind / no_change / rebind_within_window / rebind_after_window.
A within-window rebind (the diagnostic anomaly) emits a structured
log.warn unless source is 'emergency_fallback' (legitimate retry path
in chat.ts). Critically, null oldConnectionId is treated as first_bind
(NOT rebind_within_window) so the production flow of touchSession()
followed by bindSessionConnection() does not raise the alarm on every
new session's normal first request. The deprecated 2-arg touchSession
remains as a one-shot-warned compat shim for tests/legacy callers.

D3: selectByEarliestResetFirst and isAffinityValid accept an optional
provider argument plumbed from auth.ts. For provider 'claude' (Anthropic
OAuth lane), the urgent-alt heuristic break (PR #26 v8) is suppressed
unless boundScore <= 0 (genuinely unusable via accumulated backoff).
Other break paths (terminal, rate_limited, quota_exhausted, low-quota
< 15%, session_expired, affinity_window_passed) are unaffected. Other
providers retain the original break behaviour.

Trace logs added at every routing decision: log.debug 'kept' on affinity
preserved, log.warn 'broken' with bound/altscore on heuristic break, and
log.debug 'urgent-alt break suppressed by cache-sensitive gate' with
gate='claude_urgent_alt_suppressed' when the Claude gate fires. A single
log line during production traffic now answers 'did affinity break, and
why?' deterministically.

Oracle two-stage verification:
- Stage 1 plan v1 (ses_1f85345c8...): NEEDS_REVISION on 7 items.
- Stage 1 plan v2 (ses_1f84ae893...): NEEDS_REVISION (narrow) on 5
  wording-mismatch items.
- Stage 2 implementation (ses_1f83ab6ed...): NEEDS_REVISION on 4 real
  defects + 1 nit. Defect A namespace collision, defect B critical
  null-first-bind misclassification, defect D emergency-fallback alarm
  noise, defect H lazy-require silent swallow, nit E gate trace log
  reason. ALL FIVE FIXED in this commit.

New tests: SA-12 / SA-13 / SA-14 (Claude block / score=0 allow / GitHub
allow), session-manager-affinity-headers (9 tests incl. collision-proof
namespace), session-manager-binding (10 tests incl. null-first-bind and
emergency_fallback). Test count: 2874/2874 PASS (was 2852; +22 net).

Version 3.8.6 -> 3.8.7.

* docs(session,test): address Copilot PR #28 review (3 doc/test nits)

R1-1: CHANGELOG line 31-32 said '8 tests' for both new session-manager
test files, but line 34 (and the actual files) say 9 + 10. Reconciled to
9 (affinity-headers) and 10 (binding); aligned with the +22 net total.

R1-2: the 'rebind_after_window' test was named as if it deterministically
asserted the after-window classification, but it only asserts the weaker
'one of the two rebind variants' contract because it does not advance
real time. Test renamed to 'rebind to a different conn always succeeds
and updates the bound conn (window classification is wall-clock
dependent)' so the assertion matches the name. The deterministic
within-window branch is exercised by the previous test in the same file.

R1-3: SA-2 comment claimed the quotaCache lookup keys on conn.provider,
but the cache is keyed by connectionId only — the provider string is
descriptive metadata, not part of the lookup key. Comment corrected to
state that the Claude gate decision is driven entirely by the
selectByEarliestResetFirst third argument, not by any cache-key
provider field.

No production code change. 2874/2874 unit tests still pass.

* fix(routing,docs): address Copilot PR #28 R2 review (1 defect + 3 nits)

R2-1 (defect): selectByEarliestResetFirst was logging at warn level for
EVERY isAffinityValid invalid reason — including expected non-heuristic
ones like affinity_window_passed / rate_limited / terminal /
quota_exhausted_unknown_reset / inactive / track-excluded. That polluted
production warn-level logs with normal routing events. Now only the two
HEURISTIC break reasons (affinity_break_low_quota /
affinity_break_p1_too_urgent) emit at warn level — those are the
diagnostic alarm cases where the smart-affinity logic actively chose to
break a healthy binding. All other invalid reasons drop to debug.

R2-2 (changelog): aligned the [3.8.7] wording with the actual emission
("ONLY when affinity is broken via the heuristic rules" + explicit list
of expected reasons that drop to debug).

R2-3 (test comment): the binding test note 'logger may be unavailable
in this test environment' was stale after the PR's switch to a static
logger import. Updated to the real reason: keeping the unit test
decoupled from logger side effects, with verification of the warning
emission deferred to an integration test.

R2-4 (test comment): the affinity-headers test rationale was slightly
off: collision-proofing comes from the prefixes themselves differing
('ext-xsa:' vs 'ext:'), not from any restriction on what legacy raw
values can begin with. Reworded to make the proof source explicit.

No production behavior change beyond R2-1's log-level gating. 2874/2874
unit tests still pass.

* fix(routing,session): address Copilot PR #28 R3 review (2 defects) + redact PII from CHANGELOG

R3-1 (defect): touchSession(sessionId, connectionId) compat shim was
updating lastActive BEFORE forwarding to bindSessionConnection. That
made the forwarded call always see 'within affinity window' (because
lastActive had just been refreshed to Date.now()), so a session idle
>5 minutes would mis-fire the within-window alarm on its first
compat-shim rebind. The shim now binds first (so the binding decision
sees the pre-touch lastActive). Only when an entry pre-existed is
requestCount incremented — first_bind already sets it to 1, so a fresh
session via the compat shim sees requestCount == 1 (preserves the
original 1-touch == 1-request semantics that session-manager.test.mjs
relies on).

R3-2 (defect): selectByEarliestResetFirst was binding fall-through
selections itself with a hard-coded source: 'fall_through'. Since the
strategy ran BEFORE chat.ts:548's post-credential bind, the actual
within-window rebind always carried the wrong source — the
emergency_fallback alarm suppression added in stage-2 audit could
never engage in the emergency retry path. The strategy no longer binds.
chat.ts:548 is now the single binding site (single source of truth) and
selects source: 'emergency_fallback' vs 'explicit_post_credential' from
runtimeOptions.emergencyFallbackTried, so the suppression works as
intended. The unused bindSessionConnection import was removed from
the strategy.

Test count: 2875/2875 PASS (was 2874; +1 new compat-shim first-bind
contract test).

PII REDACTION (urgent user demand): CHANGELOG.md previously contained
real account names (APEXATGNU, GNUMAX) in production observation
narratives across multiple [3.8.x] sections. All occurrences have been
replaced with generic placeholders (Account A / Account B). The same
substitution was applied to .sisyphus/plans/*.md (workspace-only,
not git-tracked, but kept consistent for future readers). PR #28 body
on GitHub has also been edited to use the generic names. Merged commit
messages on main (PR #25 / #26 / #27) cannot be redacted without a
force-push to main, which is forbidden by AGENTS.md (incident cheaper-inference#1386
on 2026-04-18); their messages remain as-is on origin/main.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants