Skip to content

fix(codex): self-heal codex auth.json across all agents (API-key→OAuth recovery) - #222

Merged
KrasimirKralev merged 3 commits into
ID-Robots:betafrom
KrasimirKralev:fix/codex-auth-self-heal
Jun 24, 2026
Merged

KrasimirKralev merged 3 commits into
ID-Robots:betafrom
KrasimirKralev:fix/codex-auth-self-heal

Conversation

@KrasimirKralev

@KrasimirKralev KrasimirKralev commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Problem (found on a customer device)

A customer's box 401'd on every agent turn after switching OpenAI from API-key mode to the ChatGPT subscription:

Incorrect API key provided: sk-proj-…jtIA … url: https://api.openai.com/v1/responses … invalid_api_key

Root cause: the Codex app-server reads the agent-scoped …/agents/<agent>/agent/codex-home/auth.json, but gateway-pre-start.sh synthesized the OAuth session only to ~/.codex/auth.json and only write-if-missing. So a stale key-only auth.json (left in codex-home from the earlier API-key setup) was never corrected — the app-server kept sending that dead sk-proj-… key to api.openai.com and 401'd. It recurred on every restart because the boot script never touched codex-home. (Earlier the same device showed invalid ID token format — same stale-file root cause, different surfacing.)

The on-box agent confirmed the mechanism: writing the real OAuth session into codex-home/auth.json fixed it immediately; it only came back because nothing made it stick across reboots.

Fix

gateway-pre-start.sh now:

  1. Synthesizes the OAuth auth.json once from the codex:default profile (unchanged logic).
  2. Syncs it to ~/.codex AND every agent's codex-home (loops agents/*/agent/, so main, vtol-expert, etc. all heal).
  3. Overwrites stale-key-only / corrupt files instead of write-if-missing — so an API-key→OAuth switch self-heals on the next boot.
  4. Leaves a healthy OAuth file untouched (no key + has id_token) so the app-server still owns token refresh.

Verification

bash -n + node --check on the embedded block both pass. This is a Jetson boot script and can't be exercised on CI/PC (no OpenClaw/Codex runtime) — needs a final check on a real Codex device before release. The mechanism is already device-validated (the on-box fix worked); this makes it durable + fleet-wide.

Not in this PR (follow-up)

The same stale sk-proj key also lingers in the openai direct provider config + a stale fallback (openai/gpt-5.3-codex). A clean API-key→subscription migration in ai-models/configure (purge the stale openai key, migrate the primary openai/<m>codex/<m>, clear invalid fallbacks) is the companion fix — scoping separately.

Summary by CodeRabbit

  • Bug Fixes
    • Improved Codex sign-in session handling during startup.
    • Existing healthy OAuth sessions are now preserved, while stale or corrupted sessions are automatically refreshed.
    • Authentication state is now kept in sync across the Codex CLI and all agent environments for more reliable access.

…uth recovers

The Codex app-server reads the agent-scoped <home>/agents/<agent>/agent/
codex-home/auth.json, but gateway-pre-start synthesized the OAuth session only
to ~/.codex/auth.json and only write-if-missing. So a device that set up OpenAI
in API-key mode and later switched to the ChatGPT subscription kept a stale
key-only auth.json in codex-home — and the app-server sent that dead key to
api.openai.com/v1/responses, 401-ing every turn ("Incorrect API key sk-proj-…"
/ "invalid ID token format"). It recurred on every restart because the boot
script never corrected codex-home.

Now: synthesize once, sync to ~/.codex AND every agent's codex-home, and
OVERWRITE stale-key-only/corrupt files. A healthy OAuth file (no key, has
id_token) is left untouched so the app-server still owns refresh.
@KrasimirKralev
KrasimirKralev requested a review from a team as a code owner June 24, 2026 14:33
@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@KrasimirKralev, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 44 minutes and 59 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 09586d08-ad19-4ca0-827c-ea0f296936e5

📥 Commits

Reviewing files that changed from the base of the PR and between 2c62d6c and e0d70f4.

📒 Files selected for processing (1)
  • scripts/gateway-pre-start.sh
📝 Walkthrough

Walkthrough

The gateway-pre-start.sh script's Codex plugin branch is updated to replace single-file auth.json creation with a multi-target sync. An embedded Node.js script now decodes account_id from the OAuth access token JWT, builds an oauth payload, and writes or heals auth.json across the Codex CLI home directory and every agent's codex-home, applying strict filesystem permissions.

Changes

Codex OAuth Session Bootstrap

Layer / File(s) Summary
OAuth auth.json synthesis, heal, and multi-target sync
scripts/gateway-pre-start.sh
The Node.js heredoc now decodes account_id from the access token JWT, constructs an oauth object (OPENAI_API_KEY: null plus token fields), and iterates over the Codex CLI home and each agent's codex-home. For each target it either skips (healthy session: no OPENAI_API_KEY and non-empty tokens.id_token) or overwrites (stale/corrupt), creating missing directories and applying chmod 700/600 before writing.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • ID-Robots/clawbox#201: Modifies the Codex OAuth flow to persist the JWT as the profile id, which is the upstream source that this PR's synthesis and heal logic reads id_token from.

Poem

🐇 Hoppity-hop through each agent's home,
A little auth.json — no need to roam!
Stale tokens beware, the healer hath come,
chmod 600 set, the rabbit strikes dumb.
Healthy sessions spared, the rest overwritten,
By the fluffiest script that was ever written! 🗝️

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description explains the bug, fix, and verification, but it doesn't follow the required template sections or include the checklist/type-of-change fields. Reformat it to the repo template with Summary, Type of change, How was this tested, Checklist, and optional Screenshots/logs.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly matches the main change: self-healing Codex auth.json sync from API-key to OAuth across all agents.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/gateway-pre-start.sh`:
- Around line 526-532: The per-target auth sync in the target loop can abort the
entire gateway pre-start when one path fails. Update the `for (const dir of
targets)` block in `gateway-pre-start.sh` to wrap each directory’s `mkdirSync`,
`chmodSync`, and `writeFileSync` work in its own `try/catch`, and log a warning
that includes the failing target path so other targets still continue. Keep the
existing `isHealthyOAuth`, `fs.existsSync`, and `console.log` behavior for
successful syncs, but ensure failures are isolated and do not propagate out of
the loop.
- Around line 528-531: The existing-file branch in the OAuth token handling path
leaves a healthy auth.json untouched, so permissive permissions can persist.
Update the logic around the fs.existsSync/isHealthyOAuth check and the
fs.writeFileSync call to explicitly re-harden the file with fs.chmodSync(file,
0o600) before continuing when it already exists, and also after writing new
contents so auth.json is always owner-only regardless of prior state.
- Around line 500-523: The health check in isHealthyOAuth currently accepts any
non-empty tokens.id_token, which can preserve invalid Codex auth states. Update
the auth validation in scripts/gateway-pre-start.sh so it only treats the
session as healthy when id_token matches a JWT-shaped value before returning
true, and keep the sync logic from reusing non-JWT auth.json entries. Use the
isHealthyOAuth function and the tokens.id_token / OPENAI_API_KEY checks as the
main place to apply the stricter validation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: fe11b40c-118a-4446-a36b-8b48123f630c

📥 Commits

Reviewing files that changed from the base of the PR and between 2364e6b and 2c62d6c.

📒 Files selected for processing (1)
  • scripts/gateway-pre-start.sh

Comment thread scripts/gateway-pre-start.sh Outdated
Comment thread scripts/gateway-pre-start.sh Outdated
Comment thread scripts/gateway-pre-start.sh Outdated
On-device test on a real Jetson caught the healed file landing 0664, not 0600:
fs.writeFileSync's `mode` is only applied when CREATING a file, so overwriting
a stale auth.json left its old (group/world-readable) perms on a file holding
OAuth tokens. Add an explicit chmod 0600 after the write.
…abbit)

- isHealthyOAuth now requires a JWT-shaped id_token, so an OAuth-shaped but
  non-JWT auth.json (which the app-server rejects with 'invalid ID token
  format') is re-healed instead of preserved; also skip synthesis when the
  profile's own id_token isn't a JWT (re-auth needed).
- Wrap each target sync in try/catch so one broken codex-home (perms/path)
  logs a warning instead of throwing — under the shell's set -e an uncaught
  error would have failed the whole gateway pre-start.

Both re-validated on a real Jetson (georgi): non-JWT -> re-healed, locked
target -> non-fatal, plus the existing stale/missing/healthy/mode-0600 cases.
@KrasimirKralev
KrasimirKralev merged commit 65baf52 into ID-Robots:beta Jun 24, 2026
7 checks passed
KrasimirKralev added a commit that referenced this pull request Jun 25, 2026
…on is active (#224)

* fix(openai): self-heal stale openai/* primary -> codex when subscription active

The 'Mark Spivey' class: a device set up OpenAI in API-key mode, then switched
to the ChatGPT subscription, but the agent stayed pinned to openai/<model>. That
primary keeps calling api.openai.com with the now-invalid sk-proj key (401 every
turn) while the working subscription (codex) sits unused — and #222's codex-home
self-heal doesn't touch it because the openai DIRECT lane is keyed separately.

gateway-pre-start now, on every boot, migrates it: if a codex OAuth profile is
configured but the primary is openai/*, repoint to codex/<same-model> and drop
the stale openai direct provider, its auth profile, and any openai/* fallbacks.
Runs on the gateway restart that follows any provider save, so it covers both
the wizard switch and agent-driven switches (Mark's case).

Validated on a real Jetson (georgi, python3): migrates Mark's case
(openai/gpt-5.4 -> codex/gpt-5.4, purges the dead provider/profile/fallback),
leaves a no-subscription box untouched, and no-ops when already on codex.

* fix(codex): map migrated model through codex catalog + drop name (CodeRabbit)

- Critical: openai-direct and codex catalogs differ (gpt-5 is openai-only,
  gpt-5.4 codex-only). Keep the model id only when codex serves it, else
  fall back to codex's default gpt-5.4 — avoids setting an invalid codex/gpt-5
  that fails at startup.
- Privacy: replace the customer name in the shipped comment with a neutral
  description.

Re-validated on a real Jetson: gpt-5.4/gpt-5.5 preserved, gpt-5 -> codex/gpt-5.4,
no-subscription untouched.
KrasimirKralev added a commit that referenced this pull request Jun 25, 2026
…227)

Nine follow-ups from CodeRabbit's review of the v3.1.5 payload (already on beta):
- gateway-pre-start.sh: gate the openai->codex migration on a USABLE codex JWT
  read from auth-profiles.json (openclaw.json holds only metadata), so an
  unauthenticated codex profile can't strand the device by dropping openai;
  accept legacy openai-codex:default. (#224)
- gateway-pre-start.sh: require the full OAuth set (access+refresh+JWT id) before
  writing/preserving codex auth.json — partial files fail after token expiry. (#222)
- issue-triage.mjs: ensure the category label exists before applying (gh issue
  edit fails atomically on any missing label). (#221)
- issue-triage.yml: persist-credentials:false on checkout; pin @anthropic-ai/sdk. (#221)
- reset/route.ts: return 409 when the reset doesn't start. (#223)
- SystemUpdateApp.tsx: move autoFocus off the destructive reset button to Cancel. (#223)
- updater.ts: only persist the channel pin after startUpdate() accepts. (#223)
- updater.test.ts: add the ahead-only divergence case. (#223)

Validated on a real Jetson via synthetic harnesses (migration gate: JWT->migrate,
non-JWT/missing->skip, legacy key; full-OAuth-set health). /simplify-reviewed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant