Skip to content

landscape-sweep: treat Codex's 429 report as a limit; retain the 2026-09-29 run's cost and GPT-6 job outcomes - #513

Merged
seathatflowsinourveins merged 2 commits into
mainfrom
claude/sweep-gateway-429-limit-20260929
Sep 29, 2026
Merged

seathatflowsinourveins merged 2 commits into
mainfrom
claude/sweep-gateway-429-limit-20260929

Conversation

@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator

Scope

  • What this PR changes: the landscape-sweep GPT-6 job runner (codex_job.py) treats Codex's report of an HTTP 429 that carries no usage-limit body as a limit: the job ends with exit 3, <work-dir>/LIMIT holds a reason and no reset time, and later starts are refused. It adds tests for it, corrects the harness README bullet and the recipe's cost class, retains two sanitized records of the 2026-09-29 run (its spend scan and its GPT-6 job outcomes) and adds a decision record. It changes no selection, pin, winner, verdict or sealed artifact.
  • Base commit: be97e882
  • Lane: lane:foundation (tools/, recipes/, tests/, docs/, and new files under evidence/artifacts/landscape-sweep-20260929-attempts/; manifests/evidence.json changes only file registrations, which docs/lanes.md does not treat as shared).
  • Owned paths touched: tools/sota-convergence/landscape-sweep/codex_job.py, tools/sota-convergence/landscape-sweep/README.md, recipes/saturation-sweep.md, tests/test_landscape_sweep_harness.py, docs/decisions/2026-09-29-sweep-429-limit-marker.md (new), evidence/artifacts/landscape-sweep-20260929-attempts/spend-scan-wf_08a5b367-311.json (new), evidence/artifacts/landscape-sweep-20260929-attempts/gpt6-job-outcomes.json (new), manifests/evidence.json (registrations only).

SOTA sources

Evidence-class table

Claim Evidence class Command / receipt
Codex retries no 429 and reports the first one as exceeded retry limit, last status: 429 ..., whether the pool is exhausted or briefly rate limited; the report cannot tell the two apart source_review the upstream lines above at the tag. The independent reviewer read the same lines from an extracted copy; I re-read them at the tag with gh api
Ten GPT-6 jobs ended in that report, each within one or two seconds, with no completed turn and one distinct error text; nine belong to the follow-up round (nine of its twelve GPT-6 jobs) and one is a probe after the run (15:46Z) local_integration (one-off read of the run's private job directories; the full output stays there) evidence/artifacts/landscape-sweep-20260929-attempts/gpt6-job-outcomes.json
The runner now treats that report as a limit (exit 3, marker with reason and no reset time, later starts refused); a usage-limit report keeps its empty marker; a completed turn, item.* content, a stderr diagnostic and any other status are not limits local_integration tests.test_landscape_sweep_harness: 126 tests OK. The other status cases (500, 502, 4290, 404) are synthetic pattern-boundary checks, since Codex builds this report for a 429 only
The new behavior tests fail without the fix local_integration (mutation check, restored) with codex_job.py from origin/main and this head's tests: FAILED (failures=2), test_gateway_429_sets_the_marker_with_a_reason_and_blocks_later_starts and test_limit_detection_reads_only_codex_error_reports
The run cost $425.12 at Claude list price: Opus 5.5 $399.96, Sonnet 5.5 $25.16; advisor inference inside the workers is $101.14 (117 calls, 23.8%); the executor part, $323.98, equals the retained child-usage record priced at the same rates; first round $337.10, follow-up round $88.02, wrappers $27.06, nine superseded attempts $19.27; 7.09 h between the first and last call, with a 2.61 h idle gap native_proven (list-price proxy from provider usage in the native transcripts) evidence/artifacts/landscape-sweep-20260929-attempts/spend-scan-wf_08a5b367-311.json (the retained scan run once per group over the transcripts of the agents the child-usage record lists; ccusage shows $398.35 for the run, Opus only)
Whether a single transient 429 on a native login occurs often enough for the stop to cost more than it saves not_measured recorded in the decision record; no run has used the marker yet
GPT-6 capacity is still exhausted native_proven (reads at the time) a probe job at 15:46Z ended in the same 429 in one second; scripts/codex_quota.py --json reads 100% used on the 10080-minute window

Local commands run

$ python3 scripts/validate.py
exit 0  {"components": 69, "hashed_files": 8145, "profiles": 4, "receipts": 171, "status": "passed"}
$ TMPDIR=/dev/shm/wt429-tmp python3 -m unittest tests.test_landscape_sweep_harness tests.test_evidence_manifest tests.test_gitleaks_config tests.test_saturation_ledger tests.test_agentsview_qualification tests.test_landscape
Ran 335 tests, OK (skipped=5)   (before the rebase onto be97e882; after it: harness, manifest and gitleaks modules, 168 tests, OK)
$ gitleaks git --pre-commit --staged --redact --no-banner --config .gitleaks.toml .
no leaks found
$ python3 scripts/saturation_ledger.py --check --base origin/main
ok: 5 sweep record(s), chain intact, bindings verified
$ python3 scripts/component_matrix.py --write && python3 scripts/new_host_grand_list.py --write
no generated file changed

Decision record

docs/decisions/2026-09-29-sweep-429-limit-marker.md: the defect, the decision (one 429 stops the sweep), the upstream facts, alternatives (keep the old behavior; stop after two 429s; read the pool in the runner), the consequence, the residual and what would overturn it.

An independent read-only Opus review of the first head found three fix-level defects and four notes, all repaired here: (1) the code, the LIMIT reason, the README and the tests said "after Codex's retries" and "exhaustion", but Codex retries no 429, so the report appears at the first one and cannot tell exhaustion from a brief limit; (2) the recipe called the facts refuters Sonnet, but the retained record shows all 29 requested Opus (the Sonnet children are the GPT-6 wrappers); (3) the cost, duration and job-count figures had no receipt in the repository, and the README's "$82" had no basis (both records are retained now, and the figure is gone); notes on a wrong incident date in the README, the "remove LIMIT after the reset" line for a path with no reset time, the recipe's "probe before the follow-up round" (the follow-up starts by itself after the critic), and the precedence comment (a usage-limit report wins within one source). This repaired head was not re-reviewed (one review, one repair round), and the cross-family Codex review was not run because Codex has no capacity.

Flags for the owners:

  • flag (Codex capacity, unchanged): the six layers reopened by Landscape sweep 2026-09-29 record: 20 foundation layers, 188 candidates, 37 survivors, manifest-20260929 (six layers reopened by Codex 429s) #506 (document-retrieval, web-research, token-efficiency, ci-supply-chain, recovery-portability, observation-inference) need their GPT-6 follow-up votes; that needs a new Codex account or the reset at 2026-10-04T00:35Z.
  • flag (child-usage.mjs): it does not count advisor iterations, which were 23.8% of this run's cost; the change is queued for after the Gate A window (agent-lab first).
  • flag (convert.py): it keeps each GPT-6 job's status but not its error text, so returns.json cannot show why a job failed; the job-outcome record here was read from the private work directory.

Host evidence

Not applicable (no evidence/hosts/ files).

Checklist

  • New/changed GitHub Actions are pinned to a full commit SHA with a version comment (none changed).
  • New/changed workflows declare top-level permissions: contents: read (none changed).
  • No secrets are printed, logged or committed; no new required secret was added; the new records hold no path, user name, session id, e-mail or request id.
  • No new paid hosting, subscription or billing surface was introduced.
  • Peer-owned untracked files and worktrees were preserved.

🤖 Generated with Claude Code

…-09-29 run's cost and GPT-6 job outcomes

Codex retries no 429 (rust-v0.157.1: retry_429 is false for every provider, and api_bridge.rs builds RetryLimit in its
429 arm), so a pooled route's 429 ends a job at once with "exceeded retry limit, last status: 429 Too Many Requests".
codex_job.py now treats that report as a limit: exit 3, LIMIT holding a reason and no reset time, later starts refused.
The README, the recipe's cost class and a decision record say what the report can and cannot tell.

Retains the run's spend scan (executor + advisor, by round and role, wall clock) and its GPT-6 job outcomes as
sanitized records, and corrects the recipe's cost class (the facts refuters were Opus, not Sonnet).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@seathatflowsinourveins seathatflowsinourveins added the lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers label Sep 29, 2026
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

…ulation

The completed-job duration statistics in gpt6-job-outcomes.json cover the 44 jobs with a completed turn (43 sweep jobs
and one probe); say so in the record.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@seathatflowsinourveins
seathatflowsinourveins merged commit ed29398 into main Sep 29, 2026
25 checks passed
@seathatflowsinourveins
seathatflowsinourveins deleted the claude/sweep-gateway-429-limit-20260929 branch September 29, 2026 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant