Skip to content

Fix gpt-5.4 context length resolution for Codex - #5174

Open
zaycruz wants to merge 1 commit into
NousResearch:mainfrom
zaycruz:fix/gpt54-context-window
Open

Fix gpt-5.4 context length resolution for Codex#5174
zaycruz wants to merge 1 commit into
NousResearch:mainfrom
zaycruz:fix/gpt54-context-window

Conversation

@zaycruz

@zaycruz zaycruz commented Apr 5, 2026

Copy link
Copy Markdown

Summary

  • map openai-codex and inferred OpenAI endpoints to the models.dev openai registry
  • ignore suspicious small cached context values for OpenAI/Codex models when provider-aware metadata says the true context is much larger
  • add regression tests for the stale gpt-5.4@https://chatgpt.com/backend-api/codex = 32000 cache case

Problem

Hermes can show gpt-5.4 with a 32k context window in Codex-backed setups.

I reproduced a bad persisted cache entry locally:
gpt-5.4@https://chatgpt.com/backend-api/codex: 32000

OpenAI's current model docs list gpt-5.4 at 1,050,000 context and 128,000 max output tokens, so the cached 32000 value is incorrect.

Testing

  • pytest -q -o addopts='' tests/agent/test_models_dev.py tests/agent/test_model_metadata.py

@zaycruz
zaycruz marked this pull request as ready for review April 5, 2026 19:27
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/openai OpenAI / Codex Responses API labels May 1, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Fixes #5173 — stale gpt-5.4 context cache entry. Related to merged #15661 and #14935 (similar Codex context resolution bugs).

1 similar comment
@alt-glitch

Copy link
Copy Markdown
Collaborator

Fixes #5173 — stale gpt-5.4 context cache entry. Related to merged #15661 and #14935 (similar Codex context resolution bugs).

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for identifying the stale 32K cache path; that specific gap is still present on current main.

Problems

  • Current main already established that ChatGPT Codex OAuth must not use the direct OpenAI models.dev value: agent/model_metadata.py:1834-1852 sets gpt-5.4 to 272K, and agent/model_metadata.py:1912-1942 resolves from Codex /models before that fallback. The PR's new models.dev recovery would instead return 1.05M for this Codex route.
  • The actual stale-32K bug remains: agent/model_metadata.py:2110-2118 only invalidates Codex cache values >= 400_000; the non-matching 32K entry reaches return cached at agent/model_metadata.py:2173-2178.

Suggested changes

  • Adapt the stale-32K cache guard to current get_model_context_length() and fall through to the existing Codex resolver rather than models.dev.
  • Add a regression test for a persisted gpt-5.4 32K Codex cache entry resolving to the existing 272K Codex value.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 12, 2026

Copy link
Copy Markdown
Contributor

Heads up: this looks like it landed on main via #68554 (commit 64702f8) — both prevent a stale Codex OAuth context cache from short-circuiting provider-aware resolution; main’s agent/model_metadata.py::_skip_persistent_context_cache() bypasses all openai-codex disk entries and re-resolves from the authenticated /models catalogue. Flagging in case you'd like to close or rebase.

@alt-glitch alt-glitch removed the sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades label Jul 23, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related to merged #68554: current main now bypasses persistent openai-codex cache entries and resolves the authenticated Codex catalog, correcting the stale-cache failure. This PR still maps Codex to generic OpenAI models.dev metadata, so it is not the same mechanism and remains unsuitable without rebasing to the current resolver design.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists provider/openai OpenAI / Codex Responses API sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants