Skip to content

fix(codex): preserve GPT-5.6 reasoning contract - #7012

Merged
diegosouzapw merged 6 commits into
cheaper-inference:release/v3.8.49from
xz-dev:fix/codex-gpt56-core-contract
Jul 19, 2026
Merged

diegosouzapw merged 6 commits into
cheaper-inference:release/v3.8.49from
xz-dev:fix/codex-gpt56-core-contract

Conversation

@xz-dev

@xz-dev xz-dev commented Jul 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Preserve native max reasoning effort for GPT-5.6 Sol, Terra, and Luna through the Chat Completions → Responses → Codex request path while retaining the legacy max → xhigh compatibility mapping for older models.
  • Align the Codex GPT-5.6 catalog context window with the 372K OpenAI Codex model contract.
  • Publish exact native reasoning-effort metadata and defaults: Sol defaults to low and supports through ultra; Terra defaults to medium and supports through ultra; Luna defaults to medium and supports through max.
  • Keep Responses API text-generation models in the VS Code supplemental metadata routes and point their imports at /responses, while retaining the existing non-text and specialty-model exclusions.
  • Preserve the pinned 372K/128K GPT-5.6 token contract when merging live Codex discovery metadata: remote-only models, names, and capabilities remain live-driven, while legacy Codex models retain their existing live-limit precedence.
  • Keep pricing and the chat/image catalog-collision fix out of this PR.

First-party reference: OpenAI Codex rust-v0.144.1 model catalog.

Related Issues

Validation

  • npm run lint
  • npm run test:unit
  • npm run test:coverage
  • Coverage is still >= 60% for statements, lines, functions, and branches
  • SonarQube PR analysis is green or any remaining issues are explicitly documented below

Completed locally:

  • Original GPT-5.6 focused affected suites: 66 tests passed, 0 failed.
  • Responses metadata regression: 1 test passed, 0 failed; existing VS Code route suite: 36 tests passed, 0 failed.
  • Codex discovery merge regression was developed TDD: before the fix it failed at live 272000 versus pinned 372000; after the fix its focused suite passed 28/28.
  • Full affected catalog/discovery/GPT-5.6/Responses set after the discovery fix: 96 tests passed, 0 failed.
  • npm run typecheck:core: passed after the Responses metadata and discovery follow-ups.
  • Targeted ESLint with repository suppressions for changed source and focused test files: passed.
  • Final-head full npm run lint:json -- --max-warnings 0: passed locally with 0 errors and 0 warnings. The corresponding CI job exited 1 silently after its full-tree run with no reported diagnostic; a failed-job rerun was unavailable because the fork author lacks upstream repository admin permission.
  • npm run check:file-size: passed.
  • npm run check:dead-code: passed (225 vs baseline 227).
  • npm run check:cognitive-complexity: passed (890 vs baseline 890).
  • git diff --check: passed.
  • Full npm run test:unit: 22,799 passed, 6 failed, 14 skipped. The same six failures reproduce on the unmodified origin/release/v3.8.47 tree: three platform-specific doas -S DNS tests, two existing i18n integrity tests, and the existing Cloud Code Gemini request-default assertion.
  • Direct ESLint of tests/unit/vscode-token-routes.test.ts reports its existing 65 no-explicit-any errors on both this branch and the same unmodified release tree; changed source and other focused tests are clean.

Tests Added Or Updated

  • tests/unit/executor-codex-gpt56.test.ts
  • tests/unit/codex-gpt56-catalog.test.ts
  • tests/unit/vscode-shared-metadata.test.ts
  • tests/unit/vscode-token-routes-gpt56.test.ts
  • tests/unit/vscode-token-routes.test.ts
  • tests/unit/vscode-responses-models.test.ts
  • tests/unit/provider-models-discovery-split.test.ts

Coverage Notes

The cross-layer executor test starts with a Chat Completions request, passes it through openaiToOpenAIResponsesRequest, then through CodexExecutor.transformRequest. It catches the original max → xhigh loss while retaining a GPT-5.5 compatibility assertion. Existing catalog and VS Code route tests cover the context, supported-effort, and default-effort metadata exposed to clients.

The Responses metadata regression seeds a future Responses-only text model, asserts it survives both raw and grouped supplemental endpoints, and verifies the grouped import URL targets /responses.

Coverage was not run locally; CI should provide the authoritative coverage result.

Reviewer Notes

  • The translator change is intentionally model-aware. Unknown and legacy models keep the prior max → xhigh behavior.
  • The discovery merge exception is likewise narrow: only the pinned GPT-5.6 Sol/Terra/Luna contract (including their supported effort suffixes) keeps local token limits when live metadata is stale; model existence and non-limit live fields still come from discovery.
  • ultra remains a client/catalog preset for Sol and Terra; CodexExecutor intentionally sends upstream wire effort max. Luna advertises no ultra support.
  • Reasoning-summary catalog metadata remains absent; this PR does not add an implicit summary default.
  • No pricing files or pricing behavior are changed.
  • PR fix(models): preserve chat-capable image model rows #7004 independently fixes the same-ID chat/image catalog collision needed for the text base row to appear in /v1/models.

@xz-dev
xz-dev requested a review from diegosouzapw as a code owner July 13, 2026 02:29
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@diegosouzapw
diegosouzapw changed the base branch from release/v3.8.47 to release/v3.8.48 July 13, 2026 05:04
@steveepreston

Copy link
Copy Markdown

Thanks for merge request

@diegosouzapw
diegosouzapw changed the base branch from release/v3.8.48 to release/v3.8.49 July 13, 2026 21:57
@steveepreston

Copy link
Copy Markdown

sad to see this still not merged

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Really thorough validation write-up. This is orthogonal to the other Codex-reasoning PRs in this batch (#7176/#7095 touch response-side reasoning-summary synthesis; this is request-side effort mapping + catalog contract), so no reconciliation needed there — it merges independently. Confirmed locally: the executor/catalog regression tests reproduce the pre-fix max→xhigh collapse and pass after your fix, and the 372K context-window correction matches the cited upstream rust-v0.144.1 catalog. One thing we'll double check on our end before merge: your PR notes CI's full-tree lint exited 1 with no reported diagnostic and you couldn't rerun it (no admin perms on the fork) — we'll rerun that job ourselves.

xz-dev and others added 2 commits July 15, 2026 06:22
Splitting the supported_endpoints/output_modalities guard clauses into
excludesChatAndResponsesEndpoints() / excludesTextOutputModality() drops
isUsableChatModel's cyclomatic complexity from 16 to under the ratchet's
max of 15 (complexity-ratchets gate: 2057 -> 2056, back at baseline).
Behavior is unchanged; existing vscode/codex route tests cover it.

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Babysit summary

Drove the CI reds down from 4 to a single root cause. 10/12 checks now green; the 2 remaining reds are the same assertion and need a maintainer decision (see below).

Fixed

Red Cause Fix
Fast Quality Gates (file-size ratchet) Branch was 57 commits behind release/v3.8.49, carrying a stale config/quality/file-size-baseline.json (translator-openai-to-gemini.test.ts frozen at 1541 vs the tip's 1553). Not caused by this PR. Merged the base tip in (bbc131646, clean merge).
No new ESLint warnings Same staleness — ESLint crashed on tests/unit/ui/provider-plan-config.test.tsx, deleted upstream. Same merge. ✅ now passing.
Unit Tests fast-path (3/4) check-test-masking-selfref-6634 resolves git show origin/main, unavailable in the fork checkout. Environmental, not a defect. Same merge. ✅ now passing.
Fast Quality Gates (complexity ratchet) Real regression from this PR: isUsableChatModel reached cyclomatic complexity 16 (max 15), pushing the ratchet 2056 → 2057. a52a1b696 — extracted excludesChatAndResponsesEndpoints() / excludesTextOutputModality(). Behavior identical; ratchet back to exactly 2056.

No test was weakened, no CI/baseline edited. All 76 codex/vscode tests pass locally.


🛑 Remaining red — needs a maintainer decision (not auto-fixable)

Both Unit Tests fast-path (1/4) and Fast Quality Gates fail on one assertion:

tests/unit/provider-models-route-codex.test.ts:181
AssertionError: 372000 !== 999999

This is not a broken test — it is this PR's intent colliding with an existing contract.

Verified this is caused by the PR, not a base-red: 6/6 pass on the release/v3.8.49 tip, 5/6 on this head.

Updating that assertion means ratifying the policy change ("live is source of truth" → "pinned local contract wins for GPT-5.6"), so I've deliberately left it to a maintainer rather than editing the guard myself.

⚠️ Policy conflict with #7101

#7101 ("strip reasoning_effort when GPT-5.x models carry function tools") does the opposite of this PR for all gpt-5*. Same root cause, opposite strategies, disjoint files — git merges both cleanly and CI goes green on both, but the runtime outcome depends on merge order. Recommend deciding these two together. No reconciliation attempted here.

Not merging — handing off for human review.

…pinned wins)

Resolve the cheaper-inference#7012 catalog-merge policy collision: instead of the pinned
GPT-5.6 contract always winning for a fixed set of model ids, capacity
limits (inputTokenLimit/outputTokenLimit) now merge via
mergeCapacityLimitConservatively — Math.min(pinned, live) when both are
present, so OmniRoute never promises more context than the account can
actually serve. All other overlapping fields still take the live value
unconditionally.

Guard tests cover both directions (pinned smaller wins / pinned larger
loses) at the route level and via an isolated helper-level unit test.

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
@diegosouzapw
diegosouzapw merged commit e5b2404 into cheaper-inference:release/v3.8.49 Jul 19, 2026
15 checks passed
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Merged into release/v3.8.49 — thank you for the contribution, @xz-dev! 🎉 Validated in today's full-suite merge-train (33 PRs, 19k+ tests green) before landing.

@steveepreston

Copy link
Copy Markdown

Please merge this as soon as possible. Thanks

diegosouzapw added a commit that referenced this pull request Jul 26, 2026
…g routes (#7587) (#8564)

isUsableChatModel() was copy-pasted into 5 vscode listing routes. PR #7012
widened only models/route.ts to accept api_format "responses"/"openai-responses"
alongside "chat-completions"; the other 4 copies (token+raw api/tags and
api/show) still rejected anything that wasn't literally "chat-completions",
silently dropping Codex-discovery-synced GPT models (apiFormat "responses")
from the Ollama-compatible /api/tags endpoint VS Code's "Ollama" provider
import flow actually calls.

Extracted the predicate into a single shared module
(vscode/[token]/usableChatModel.ts) imported by all 5 routes so this
one-fixed-four-left-behind drift cannot recur.

Co-authored-by: ikelvingo <im.kelvinwong@gmail.com>
HouMinXi pushed a commit to HouMinXi/OmniRoute that referenced this pull request Aug 2, 2026
* fix(codex): preserve GPT-5.6 reasoning contract

* fix(vscode): expose Responses text models

* fix(codex): keep GPT-5.6 limits through discovery

* fix(ci): extract isUsableChatModel helpers to satisfy complexity ratchet

Splitting the supported_endpoints/output_modalities guard clauses into
excludesChatAndResponsesEndpoints() / excludesTextOutputModality() drops
isUsableChatModel's cyclomatic complexity from 16 to under the ratchet's
max of 15 (complexity-ratchets gate: 2057 -> 2056, back at baseline).
Behavior is unchanged; existing vscode/codex route tests cover it.

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>

* fix(codex): merge capacity limits conservatively (smaller of live vs pinned wins)

Resolve the cheaper-inference#7012 catalog-merge policy collision: instead of the pinned
GPT-5.6 contract always winning for a fixed set of model ids, capacity
limits (inputTokenLimit/outputTokenLimit) now merge via
mergeCapacityLimitConservatively — Math.min(pinned, live) when both are
present, so OmniRoute never promises more context than the account can
actually serve. All other overlapping fields still take the live value
unconditionally.

Guard tests cover both directions (pinned smaller wins / pinned larger
loses) at the route level and via an isolated helper-level unit test.

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>

---------

Co-authored-by: Xiangzhe <xz-dev@users.noreply.github.com>
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
HouMinXi pushed a commit to HouMinXi/OmniRoute that referenced this pull request Aug 2, 2026
…g routes (cheaper-inference#7587) (cheaper-inference#8564)

isUsableChatModel() was copy-pasted into 5 vscode listing routes. PR cheaper-inference#7012
widened only models/route.ts to accept api_format "responses"/"openai-responses"
alongside "chat-completions"; the other 4 copies (token+raw api/tags and
api/show) still rejected anything that wasn't literally "chat-completions",
silently dropping Codex-discovery-synced GPT models (apiFormat "responses")
from the Ollama-compatible /api/tags endpoint VS Code's "Ollama" provider
import flow actually calls.

Extracted the predicate into a single shared module
(vscode/[token]/usableChatModel.ts) imported by all 5 routes so this
one-fixed-four-left-behind drift cannot recur.

Co-authored-by: ikelvingo <im.kelvinwong@gmail.com>
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
* fix(codex): preserve GPT-5.6 reasoning contract

* fix(vscode): expose Responses text models

* fix(codex): keep GPT-5.6 limits through discovery

* fix(ci): extract isUsableChatModel helpers to satisfy complexity ratchet

Splitting the supported_endpoints/output_modalities guard clauses into
excludesChatAndResponsesEndpoints() / excludesTextOutputModality() drops
isUsableChatModel's cyclomatic complexity from 16 to under the ratchet's
max of 15 (complexity-ratchets gate: 2057 -> 2056, back at baseline).
Behavior is unchanged; existing vscode/codex route tests cover it.

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>

* fix(codex): merge capacity limits conservatively (smaller of live vs pinned wins)

Resolve the cheaper-inference#7012 catalog-merge policy collision: instead of the pinned
GPT-5.6 contract always winning for a fixed set of model ids, capacity
limits (inputTokenLimit/outputTokenLimit) now merge via
mergeCapacityLimitConservatively — Math.min(pinned, live) when both are
present, so OmniRoute never promises more context than the account can
actually serve. All other overlapping fields still take the live value
unconditionally.

Guard tests cover both directions (pinned smaller wins / pinned larger
loses) at the route level and via an isolated helper-level unit test.

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>

---------

Co-authored-by: Xiangzhe <xz-dev@users.noreply.github.com>
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…g routes (cheaper-inference#7587) (cheaper-inference#8564)

isUsableChatModel() was copy-pasted into 5 vscode listing routes. PR cheaper-inference#7012
widened only models/route.ts to accept api_format "responses"/"openai-responses"
alongside "chat-completions"; the other 4 copies (token+raw api/tags and
api/show) still rejected anything that wasn't literally "chat-completions",
silently dropping Codex-discovery-synced GPT models (apiFormat "responses")
from the Ollama-compatible /api/tags endpoint VS Code's "Ollama" provider
import flow actually calls.

Extracted the predicate into a single shared module
(vscode/[token]/usableChatModel.ts) imported by all 5 routes so this
one-fixed-four-left-behind drift cannot recur.

Co-authored-by: ikelvingo <im.kelvinwong@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants