Skip to content

Token stack inside native Codex sessions: 11/15 tools with passing checks, four real limits, and a correction to #296's QMD row - #343

Merged
seathatflowsinourveins merged 2 commits into
mainfrom
claude/token-e2e-codex-20260926
Sep 26, 2026
Merged

seathatflowsinourveins merged 2 commits into
mainfrom
claude/token-e2e-codex-20260926

Conversation

@seathatflowsinourveins

Copy link
Copy Markdown
Owner

Scope

SOTA sources

  • Codex CLI 0.155.1 (https://github.com/openai/codex): codex exec --json --output-schema -o (native help), and turn.completed.usage events for the returned usage.
  • Tools, at the versions installed on the host:
    • rtk-ai/rtk 0.50.0, instruction mode through rtk init --codex;
    • mksglu/context-mode 1.0.169, the plugin;
    • chopratejas/headroom 0.37.0 with HEADROOM_OFFLINE=1 (Headroom: run the MCP server with its usage beacon off (HEADROOM_OFFLINE=1, DO_NOT_TRACK=1) #304);
    • oraios/serena at the pinned commit;
    • giancarloerra/SocratiCode 1.14.0;
    • DeusData/codebase-memory-mcp v0.11.0;
    • tobi/qmd 2.8.3;
    • ast-grep 0.45.3, MarkItDown 0.1.8, TOON 4.1.1, Repomix 1.18.1, Context Hub 0.1.4, agentsview v0.43.0;
    • akitaonrails/ai-memory 2.4.1.
  • Exact counts: gpt-tokenizer 3.4.0, o200k_base, via tools/token-report/token_manifest.py compare.

Evidence-class table

Claim Evidence class Where
11 of 15 tools worked inside native Codex sessions, and each answer passed a deterministic check against the baseline native_proven on this host receipt.json tools[]
The four failures are tool or wiring limits: jCodeMunch is project-scoped (#240); QMD's index excludes adoption/; Repomix --compress dropped a declaration; Context Hub has a content gap native_proven (retained sessions and reruns) tools[].verdict_note
RTK +17,782 saved for this run's worktree only (41 commands) upstream estimate, attributable counters.rtk_gain_codex_worktree_only
9 exact comparisons, 28.2% to 99.8% removed local_integration (exact counts; not provider savings) tools[].exact_comparison
Usage: 7,085,015 input (6,253,312 cached), 164,906 output, 91,997 reasoning Codex-returned codex_usage_total
#296's QMD row retrieved the wrong document for its question source_review corrections_to_296

The Headroom and Context Mode (Codex) deltas are host-wide: the after snapshot came about 12 hours after the runs. They are recorded but not credited to this run.

Local commands run

$ uv run … scripts/validate.py; evidence_manifest.py --check; build_ecosystem.py --check   # passed (6584 hashed files)
$ uv run … -m unittest <27 modules that read token-efficiency-stack or evidence/artifacts>   # Ran 1589, OK (skipped=26)
$ uv run … -m unittest tests.test_landscape_sweep_harness   # Ran 71, OK alone, both on this branch and on clean main.
  # It fails only when run in one process with the other modules, which is a test-ordering interaction that also exists on main.
$ gitleaks-guarded dir evidence/artifacts/token-e2e-codex-20260926   # no leaks found

Checklist

  • No workflow changes. [x] No secrets. The receipt was scanned: 0 home paths, 0 scratch paths, 0 ids. [x] No paid surface. [x] Peer files preserved.
  • Independent review: a cross-family Codex review is requested next.

🤖 Generated with Claude Code

…s QMD row

Codex side of #296: one native `codex exec` session per tool (codex-cli
0.155.1, gpt-6-astra, effort max) in a dedicated worktree, using RTK's
standing instructions, the Context Mode plugin, the #289 user-scope MCP
servers and the CLIs. 11 of 15 tools pass their answer checks; RTK +17782
saved for the run-only worktree; nine exact o200k comparisons from 28.2% to
99.8%; Codex-returned usage 7,085,015 input (6,253,312 cached) and 164,906
output over 17 sessions. The four failures are real limits: jCodeMunch is
project-scoped (#240), QMD's catalog index excludes adoption/, Repomix
--compress dropped a declaration, Context Hub lacks the upstream facts.
#296's QMD PASS retrieved a wrong document for its question; its README now
says so.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@seathatflowsinourveins seathatflowsinourveins added the lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers label Sep 26, 2026
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Codex review (gpt-6-astra, read-only) confirmed every number and evidence
class and asked for three fixes: replace the live-clone path, the
session-derived codebase-memory project id and the SocratiCode collection id
with placeholders, and state the QMD document count per attempt (121, then
122 as the live index grew).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@seathatflowsinourveins

Copy link
Copy Markdown
Owner Author

Cross-family review, Codex gpt-6-astra, read-only, effort high. needs_changes → fixed. Every README and docs number matches receipt.json, the usage totals sum correctly, and the evidence classes, the four failure descriptions and the #296 QMD correction are all supported. It found three receipt defects:

  • a live-clone path;
  • a session-derived codebase-memory project id and the SocratiCode collection id;
  • a QMD document count stated as 121 while attempt 2 read 122.

All three are fixed in the latest commit. The receipt now scans clean (0 matches for those paths and ids), and validate and the evidence manifest pass.

@seathatflowsinourveins
seathatflowsinourveins merged commit ca8e37f into main Sep 26, 2026
24 checks passed
@seathatflowsinourveins
seathatflowsinourveins deleted the claude/token-e2e-codex-20260926 branch September 26, 2026 14:24
seathatflowsinourveins pushed a commit that referenced this pull request Sep 27, 2026
…asks,

metrics, gate, procedure)

Freezes the E2E preregistration half of full-save/PLAN.md step 9 (the
baseline-receipt half already merged via #369) before any organic run:
77 frozen task definitions (16 reused Claude + 15 reused Codex information
needs from #296/#343's receipts, 46 seeded/control tasks), every AA §8.3
metric/threshold/guardrail/outcome rule with its three "proposed" bounds now
stated as frozen, the AA §8.1b capability gate, the AA §8.4 procedure, the
AA §8.5 overturn conditions, the identity scheme, the Q1 blind-process
qualification, the Workflow script that will run arms B/A/A0 (syntax-checked
only, never executed), and the measurement runbook.

Corrected on independent Claude review (11 findings, 95 turns) and a GPT-6
cross-family review (2 findings, 1 high) plus one combined repair round (13/13
addressed with failing-first evidence): relocated from
blueprints/token-practice-e2e/ to evidence/artifacts/token-adoption-e2e-20260926/
per adoption-audit/PLAN.md line 402; moved the Workflow script out of
examples/claude-native/workflows/ to avoid that directory's shared
PACKET/ROUTING contract test, which this one-off frozen script correctly does
not match; corrected PR-A's status (its base branch merged as #369 -- baseline
only, tooling-fix scope still open); added the six AA §6 machine-readable
policy fields the JSON was missing; fixed a wrong Codex arm-A lifecycle
binding; blocked the one information need (an otel-tui receiver task) that no
current role may legally run, pending a named permitted role; froze the
coordinator's own main-dispatch task at xhigh instead of max; fixed a
denylist word-boundary gap that let underscore-qualified MCP names slip past;
and sealed the five normative files' SHA256 with an append-only amendment
rule. Also refreshes PR #376 and #364's status to merged (both landed while
this unit was in review), while recording that #364's field-preservation gap
(workflow.run_id/tool_use_id still missing from collector.yaml) persists after
its merge.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
seathatflowsinourveins added a commit that referenced this pull request Sep 27, 2026
…asks, (#381)

metrics, gate, procedure)

Freezes the E2E preregistration half of full-save/PLAN.md step 9 (the
baseline-receipt half already merged via #369) before any organic run:
77 frozen task definitions (16 reused Claude + 15 reused Codex information
needs from #296/#343's receipts, 46 seeded/control tasks), every AA §8.3
metric/threshold/guardrail/outcome rule with its three "proposed" bounds now
stated as frozen, the AA §8.1b capability gate, the AA §8.4 procedure, the
AA §8.5 overturn conditions, the identity scheme, the Q1 blind-process
qualification, the Workflow script that will run arms B/A/A0 (syntax-checked
only, never executed), and the measurement runbook.

Corrected on independent Claude review (11 findings, 95 turns) and a GPT-6
cross-family review (2 findings, 1 high) plus one combined repair round (13/13
addressed with failing-first evidence): relocated from
blueprints/token-practice-e2e/ to evidence/artifacts/token-adoption-e2e-20260926/
per adoption-audit/PLAN.md line 402; moved the Workflow script out of
examples/claude-native/workflows/ to avoid that directory's shared
PACKET/ROUTING contract test, which this one-off frozen script correctly does
not match; corrected PR-A's status (its base branch merged as #369 -- baseline
only, tooling-fix scope still open); added the six AA §6 machine-readable
policy fields the JSON was missing; fixed a wrong Codex arm-A lifecycle
binding; blocked the one information need (an otel-tui receiver task) that no
current role may legally run, pending a named permitted role; froze the
coordinator's own main-dispatch task at xhigh instead of max; fixed a
denylist word-boundary gap that let underscore-qualified MCP names slip past;
and sealed the five normative files' SHA256 with an append-only amendment
rule. Also refreshes PR #376 and #364's status to merged (both landed while
this unit was in review), while recording that #364's field-preservation gap
(workflow.run_id/tool_use_id still missing from collector.yaml) persists after
its merge.

Co-authored-by: Scout <scout@local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant