Gate combo reasoning token buffer

[rdself]

Summary

• Add a combo defaults feature flag (reasoningTokenBufferEnabled) so users can fully disable the fix: add reasoning token buffer for combo routing (fixes #3587) #3588 reasoning token buffer from Settings.
• Make the buffer conservative: it only applies when the model is explicitly thinking-capable, has a non-default known output cap, and the full buffered value fits inside that cap.
• Preserve client max_tokens for unknown models, capless/default-cap models, and near-cap requests such as 64000 against a 65536 provider limit.

@herjarsa the original #3588 handling was too blunt: multiplying every reasoning-model request by 50% can turn a valid client limit into an invalid upstream maxOutputTokens value, which breaks strict providers. This PR keeps the #3587 mitigation only where the cap is known and safe, and gives operators a kill switch.

Tests

• DISABLE_SQLITE_AUTO_BACKUP=true node --import tsx --import ./open-sse/utils/setupPolyfill.ts --test tests/unit/combo-routing-engine.test.ts
• DISABLE_SQLITE_AUTO_BACKUP=true node --import tsx --import ./open-sse/utils/setupPolyfill.ts --test tests/unit/combo-config.test.ts
• DISABLE_SQLITE_AUTO_BACKUP=true node --import tsx --import ./open-sse/utils/setupPolyfill.ts --test tests/unit/combo-quality-validator-reasoning.test.ts
• npm run typecheck:core
• npx eslint open-sse/services/combo.ts open-sse/services/comboConfig.ts src/shared/validation/schemas.ts src/app/api/settings/combo-defaults/route.ts src/app/(dashboard)/dashboard/settings/components/ComboDefaultsTab.tsx src/types/settings.ts tests/unit/combo-routing-engine.test.ts tests/unit/combo-config.test.ts (warnings only from pre-existing test any usage)
• npm run check:fabricated-docs (returns 0; reports existing documentation drift)

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds a safer “reasoning token buffer” mechanism for known reasoning models and exposes it as a configurable feature flag across backend config, API defaults, UI, docs, and tests.

Changes:

• Implemented capability-aware max_tokens buffering for reasoning models with a known output cap.
• Added reasoningTokenBufferEnabled to combo defaults/config schema, API defaults, and dashboard settings UI.
• Expanded unit tests and updated user guide documentation for the new flag and buffering behavior.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/unit/combo-routing-engine.test.ts	Adds test coverage for capped buffering + feature-flag behavior.
tests/unit/combo-config.test.ts	Verifies default + cascade behavior for the new config flag.
src/types/settings.ts	Extends ComboDefaults with reasoningTokenBufferEnabled.
src/shared/validation/schemas.ts	Allows reasoningTokenBufferEnabled in validated combo runtime config.
src/app/api/settings/combo-defaults/route.ts	Exposes the new default value via settings API.
src/app/(dashboard)/dashboard/settings/components/ComboDefaultsTab.tsx	Adds a dashboard toggle to control buffering behavior.
open-sse/services/comboConfig.ts	Sets server-side default reasoningTokenBufferEnabled: true.
open-sse/services/combo.ts	Introduces resolveReasoningBufferedMaxTokens and applies it in routing.
docs/guides/USER_GUIDE.md	Documents when/why to disable the reasoning token buffer.

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

The changes implement a reasoning token buffer feature for combo routing with careful handling of edge cases:

1. open-sse/services/reasoningTokenBuffer.ts (new file) - Clean implementation with:

   • toPositiveInteger() handles both number and string inputs (important for JSON body parsing)
   • resolveReasoningBufferedMaxTokens() correctly:
     • Returns null when feature is disabled
     • Returns null for non-thinking models
     • Returns null for unknown caps (default values)
     • Clamps to cap when current > maxOutputTokens
     • Preserves original value when buffer would exceed cap
     • Applies 50% buffer + 1000 floor when safe

2. open-sse/services/combo.ts - Two code paths:

   • handleComboChat: Uses deep-cloned attemptBody, mutation is safe
   • handleRoundRobinCombo: Uses original body reference, creates new object only when buffering needed - prevents compounding across iterations

3. src/lib/modelCapabilities.ts: Added getSyncedCapabilityForResolved() for fallback lookups - supports alias/canonical resolution

4. Tests (tests/unit/combo-routing-engine.test.ts): Comprehensive coverage including:

   • Normal buffering case (4096 → 6144)
   • Near-cap preservation (64000 stays 64000 when buffer would exceed cap)
   • Over-cap clamping (70000 → 65536)
   • Feature flag disabled behavior
   • Non-reasoning models unaffected
   • Round-robin compounding prevention

The existing comments in the tracking table appear to be outdated or refer to pre-existing patterns outside the diff scope. The new implementation handles the edge cases correctly as verified by the test assertions.

Files Reviewed (7 files)

• open-sse/services/reasoningTokenBuffer.ts - new module, logic verified
• open-sse/services/combo.ts - reasoning buffer integration, mutation safety verified
• open-sse/services/comboConfig.ts - default config added
• src/lib/modelCapabilities.ts - capability resolution helper added
• src/types/settings.ts - interface extended
• src/shared/validation/schemas.ts - schema extended
• tests/unit/combo-routing-engine.test.ts - test coverage added

---

_{Reviewed by laguna-m.1-20260312:free · 5,793,831 tokens}

[diegosouzapw]

Thanks @rdself — merged into release/v3.8.23 🎉

You correctly diagnosed that the original #3588 buffer was too blunt: multiplying every reasoning request by 50% could push a valid client max_tokens past a strict provider's maxOutputTokens and trigger 400s. The new resolveReasoningBufferedMaxTokens only adds headroom where the cap is known, non-default, and the buffered value fits — and the reasoningTokenBufferEnabled kill switch (cascaded through config → schema → ComboDefaults UI) gives operators a clean opt-out.

Especially appreciated the thorough TDD: disabled-flag, near-cap (64000/65536), over-cap clamp (70000→65536), capless/default-cap non-inflation, and the round-robin non-compounding cases.

Validated locally: combo suite 118/118 (routing-engine 81 · config 25 · quality-validator-reasoning 12), phase1f 10/10, typecheck:core clean. (resilience-http-e2e is a pre-existing slow/hanging e2e harness, unrelated to this change — your edit there correctly adds the real providerCooldown key to the expected-keys list.) Ships in v3.8.23.