Skip to content

Enforce a retrieval-CLI-free PATH for blind Codex children; stop voiding lanes on search terms - #206

Merged
seathatflowsinourveins merged 4 commits into
mainfrom
claude/codex-audit-precision-20260924
Sep 24, 2026
Merged

seathatflowsinourveins merged 4 commits into
mainfrom
claude/codex-audit-precision-20260924

Conversation

@seathatflowsinourveins

Copy link
Copy Markdown
Owner

Why

The first 2026-09-24 two-family re-record (wave 20260924, catalog 149940af) could not be recorded.

  • The Codex lane's blind audit voided 15 of 32 layers, and every flag was a false positive; no command read outside the export:

    Reason Flags Example
    Retrieval-CLI name used as a search term or Python list item 15 rg -i 'qmd|…', for t in ['serena', …]
    Quoted '/' joining path parts 8 p+'/'+k
    Python path join or URL read as an absolute path 2 Path.cwd()/ref, 'https://'
    Backtick inside a quoted heredoc 1 kept as a known low (REG7-3)

    A rerun would repeat these, because the lanes must search for candidates such as qmd, serena and socraticode by name.

  • The headless Claude lane was killed 20 minutes in. claude -p ends background tasks 600 s after its turn ends: "Background tasks still running after 600s; terminating. Set CLAUDE_CODE_PRINT_BG_WAIT_CEILING_MS=0 to wait indefinitely."

What changes

  • Enforcement instead of a name heuristic.
    • A blind child's PATH is /usr/bin:/bin:/usr/sbin:/sbin.
    • codex is launched by the absolute path the caller's PATH resolves (run_attempt passes executable=; argv[0] stays codex).
    • blind_path_issue refuses a blind run (codex_lane and adjudicate codex, exit 2) when ai-memory, agentsview, mcporter, qmd, socraticode, jcodemunch, serena, codex or claude resolves on that PATH, or on what the login shell adds. The latter is measured each run: Codex runs commands with bash -lc, and /etc/profile.d adds /snap/bin on this host.
    • Real probe (a blind child using the same home, environment and launch path): every one of those CLIs, plus node, npx and uvx, was MISSING. python3, git and Codex's bundled rg resolved.
  • Audit rules.
    • In blind runs the audit no longer flags retrieval-CLI names; a non-blind run still flags them.
    • git, sqlite3, curl and wget keep the raw word rule.
    • The root rule skips a quoted '/' joined with +.
    • The path rule skips a / after ) or ], and a URL's // after :.
  • Recipe: CLAUDE_CODE_PRINT_BG_WAIT_CEILING_MS=0 for the headless lane and adjudication runs.
  • lane-provenance.json: new codex and adjudication entries appended (append-only against main).

Evidence

  • Replayed over the real run's 420 commands, 1 of 32 layers is flagged (the kept backtick case), down from 15.
  • New BlindPathAndPrecisionTests:
    • every voided form from the re-record is now clean;
    • real reaches still flag: ls /, os.listdir('/'), git log, curl, out-of-export paths inside Python, x:/etc/passwd, and a retrieval CLI run by its absolute path;
    • the PATH refusal, including a login shell that cannot be measured;
    • non-blind reporting.
  • 11 of 11 mutants killed. They cover: no child PATH, no absolute executable, the names put back, no root exemption, no lookbehind, no or overly broad URL skip, no login-shell measurement, an unmeasured shell accepted, no non-blind extension, and no refusal.
  • Checks: full suite 4343 OK on 3.12 and on 3.13 (run sequentially); envelope 214/214; validate.py passed; ruff clean on the changed files; gitleaks: no leaks.

Remaining limits

The audit is still a heuristic lower bound for paths a program computes, as the README states. A retrieval CLI that a host installs in a system directory makes the lane refuse to run, rather than run unaudited.

🤖 Generated with Claude Code

…ing lanes on search terms

The 2026-09-24 re-record's Codex lane voided 15 of 32 blind layers with no
read outside the export: retrieval-CLI names were search terms or Python list
items (rg -i 'qmd|...'), a quoted '/' joined path parts (p+'/'+k), and
Path.cwd()/ref and 'https://' read as absolute paths.

- A blind child's PATH is the system directories; codex runs by the absolute
  path the caller's PATH resolves. blind_path_issue refuses a blind run
  (codex_lane, adjudicate codex) when ai-memory, agentsview, mcporter, qmd,
  socraticode, jcodemunch, serena, codex or claude resolves there or in what
  the login shell adds (measured per run). A real blind child confirmed each
  is missing; python3, git and Codex's bundled rg resolve.
- The audit stops flagging those names in blind runs (a non-blind run still
  does); git, sqlite3, curl and wget keep the raw word rule.
- The root rule skips a quoted '/' joined with +; the path rule skips a '/'
  after ')' or ']' and a URL's '//' after ':'.
- Replayed over the run's 420 commands: 1 of 32 layers flagged (the kept
  REG7-3 backtick). 11/11 mutants killed.
- Recipe: CLAUDE_CODE_PRINT_BG_WAIT_CEILING_MS=0 for the headless Claude lane,
  which claude -p otherwise ends 600 s after its turn (the lane was killed).
- lane-provenance.json: new codex and adjudication entries appended.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

…olute CLIs, file URLs, no-PATH shells)

- P1/C4: an env-style launcher (npm's `#!/usr/bin/env node` codex) runs with
  the interpreter the caller's PATH resolves (blind_child_argv).
- P2: a CLI named by an absolute path outside the roots is "runs <cli> by
  path", so /usr/local/bin/qmd (an exempt system executable token) flags.
- P2/C1: only a network scheme's '//' is skipped, and never '///'
  (file:///etc/passwd, https:///etc/passwd flag).
- C2: the root exemption needs '+' on both sides ('/'+'etc/passwd' flags).
- C3: blind_path_issue also measures the PATH a shell sets when a command
  drops PATH (env={}: /usr/local/bin) and os.defpath.
- C5: each measured PATH is read after a sentinel, not all of stdout.
- Real blind child re-probed with the new launch: exit 0, retrieval CLIs,
  codex, claude and node missing. 21/21 mutants killed; replay of the
  aborted run still 1/32 flagged (the kept backtick).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…cord the sandbox's network block

- Codex review of 882ba4b (P2): `rg -n 'https://github.com/tobi/qmd'` read
  as "runs qmd by path"; network_authority is now shared by the path and
  by-path rules.
- Independent round 1 (a): the root exemption needs non-literal operands on
  both sides of the '+' ('' + '/' + 'etc/passwd' and ''+'/'+x flag).
- Round 1 (b): a real blind child's connection to a local listener the caller
  reached, and to 127.0.0.1:6333, failed with EPERM (read-only sandbox).
- Recipe: step 5's headless adjudication names the background-wait setting.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… refusals, shim launchers, URL schemes

- R2-3: test_codex_lane and test_adjudicate pin login_shell_path for the
  module; on a simulated host whose probe fails, 1 test fails (the real-probe
  one) instead of 65.
- R2-4: login_shell_path raises PathUnmeasured naming the probe, its exit code
  or error and the stderr tail; profile output is decoded leniently (a login
  shell printing non-UTF-8 bytes is measured).
- R2-2: a shell-script codex launcher (pnpm cmd-shim) is refused up front in
  codex_lane and adjudicate codex; a failed child's last stderr lines go to the
  console, never to the recorded reason.
- R2-6: only env's leading -S is dropped; an interpreter's -S stays.
- R2-1 part 2: any URL scheme's // authority is skipped except file
  (jar:file://, FILE://localhost and https:/// still flag; qmd://, s3:// and
  attachment:// search terms are clean).
- README: new-host note (/usr/local/bin CLIs, macOS zsh ~/.zshenv refusal).
- Mutants 11/11 killed for this delta; replay of the aborted run still 1/32.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@seathatflowsinourveins
seathatflowsinourveins merged commit b122312 into main Sep 24, 2026
22 checks passed
@seathatflowsinourveins
seathatflowsinourveins deleted the claude/codex-audit-precision-20260924 branch September 24, 2026 20:37
seathatflowsinourveins pushed a commit that referenced this pull request Sep 25, 2026
…resolve every allocated generation, and harden the warm-monitor preflight

Four low-severity items from the final independent review, plus the test
gaps two of them exposed:

1. An ambiguous-only response (a previously confirmed record whose row
   loses its governing date on a later fetch) wrote an empty list over
   the last good records in complete_attempt, turning a confirmed
   must_flatten back into mere block-and-flag and losing an already-live
   intraday exit signal. Fixed: an ambiguous symbol with zero valid
   records in the new response now degrades (preserving the last good
   records) instead of overwriting them with [].

2. Mutation G4 (drop the scheduler's refresh_timed_out call for a
   two-phase guard) survived because every existing short-timeout test
   drove a legacy single-arg guard, never the real CorporateActionMonitor's
   own generation-aware timeout path. Added a test driving the real
   monitor through a short-timeout scenario, asserting the symbol is
   flagged and the next attempt is allowed after retry_seconds (60s), not
   throttled for the ordinary refresh_seconds (900s).

3. A pre-allocated generation was left permanently unresolved if
   Thread.start() itself raised, or if an unexpected exception escaped the
   worker's own run_attempt call -- refreshes would then be silently
   throttled with nothing ever flagged for up to 900s. Both paths now call
   fail_attempt on the pre-allocated generation before propagating.

4. _preflight_corporate_action_refresh unconditionally awaited
   _schedule_corporate_action_refresh's return value, which is None (not a
   task) when the two-phase begin_attempt itself reports a throttled
   no-op -- a second preflight call on an already-warmed monitor raised
   TypeError from awaiting None. Now only awaits when a task was actually
   created.

The review's own mutation script also caught two gaps in the round-5
fixes themselves once the two-phase API was exercised more directly: G5
(a cancellation mutated to a no-op) survived because the existing test
only checked that a clean end state stayed clean, which held regardless
of whether invalidate_attempt ran; strengthened to assert the generation
counter is actually bumped past the cancelled attempt. A generation-
specific refresh_timed_out check reverted to a round-4-era coarse
heuristic (comparing only _generation to _committed_generation) also
survived; added a test where a stale timeout for an already-superseded
(but not yet committed) generation must be a complete no-op.

Merged origin/main (added #206 and #218 since the round-5 push);
source-hashes.json and manifests/evidence.json were recomputed from
current file content for every changed path, not resolved by picking a
side. No overlap between main's changes and this branch's files beyond
manifests/evidence.json, which merged cleanly.

Re-ran: the full adaptive suite (373 tests) with network-blocked
pagination tests, test_native_faults_min, test_order_throughput,
test_order_contract, test_trading_gates, all six validators (validate.py,
validate_catalogs.py, landscape.py, evidence_manifest.py,
verdict_review_gate.py, trading_gates.py -- all passed), and the
reviewer's own mutate_rev5.py retargeted at this worktree (65 mutations,
all killed except one PATTERN_COUNT=0 entry already superseded by an
equivalent round-5 mutation of my own). Ran gitleaks git history scan
(509 commits, 1.12 GB) with the pinned binary and repo config: all 76,054
findings are pre-existing sourcegraph-access-token/generic-api-key
pattern matches inside the generated docs/ecosystem/index.html artifact;
zero findings in any file this branch touches.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant