Skip to content

Audit follow-up: gap-ledger join and dated verdict fields in sweep inputs; durable-memory note; landed #575 citations and four trading acceptance classes - #585

Merged
seathatflowsinourveins merged 6 commits into
mainfrom
foundation/audit-followup-20261001
Oct 1, 2026
Merged

seathatflowsinourveins merged 6 commits into
mainfrom
foundation/audit-followup-20261001

Conversation

@seathatflowsinourveins

@seathatflowsinourveins seathatflowsinourveins commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

Scope

  • What this PR changes: the follow-up to the Codex catalog lane's audit of its 32 frozen sweep inputs (2026-10-01), which found 14 winner pins behind the stack and gap texts that committed receipts had already answered. Five parts, each its own commit, with the hot-file commit last.
    1. tools/sota-convergence/landscape-sweep/build_inputs.py: every repository-layer sweep input gains verdict_checked_at, verdict_note and open_gaps_followup, a join of the committed gap-wave owner ledgers onto the shown open gaps. Existing fields keep their shape; no template changes, so the pinned prompt hashes stay.
    2. catalogs/landscape/foundation.json: one dated 2026-10-01 limitations entry on the durable-memory row. No verdict field changes.
    3. docs/decisions/2026-10-01-definitive-sota-wsl-program.md, decision 5: how a comparison's result reaches the landscape catalog.
    4. catalogs/foundation/new-wsl-architecture-20261001.json and its record: the 39 citations of merged Review all replacement WSL architecture layers and retained gates #575 become landed sources, and four trading verdict winners get the acceptance class their committed receipts support.
    5. The trading lane owner's verbatim note on the identity-provenance row, from its acknowledgement comment on this pull request (one sentence; the manifest is unchanged by it).
  • Base commit: d6d3bf7d6e28e6ff1c557a07806f757d16b71ccd
  • Lane: lane:shared (the edition's three trading rows change; the trading lane owner asked for the correction and re-reads the rows before merge)
  • Owned paths touched: the seven files above and, in the last commit, manifests/evidence.json.

Why the stale inputs happened: winners, alternatives and open_gaps of every landscape row are the sealed 2026-09-22 verdict wave. record_verdicts.py --write refuses that run id, build_verdicts.py --check fails when such a row differs from the hash-registered wave document, and tests/test_layer_verdicts.py pins it. The sweep's input builder copied those fields without their date and did not join what the repository recorded afterwards. Each input already carried the current stack pin in components_vs_upstream; nothing said which pin was which.

What the join gives on main's catalogs (the Codex lane's scope and freshness manifest, a scratch work directory):

32 layers; 20 foundation; ...; gap follow-ups: 133 joined from 3 ledger(s), 4 dropped (no sealed open gap with that index and text), 190 for gaps beyond the 5 shown

323 of the 327 ledger entries match a sealed open gap by index and exact text; the 4 that do not are in ci-supply-chain and are counted, not joined. 110 of the 152 gap snippets the inputs show get a follow-up. What the ledgers do not record, such as the 2026-09-25 memory run, still reaches a sweep through --seeds.

SOTA sources

  • Ledger reading: this repository's reviewed implementation, tools/sota-convergence/lane_packets.py at d6d3bf7d (GAP_LEDGER_GLOB, gap_receipts_index: the same ledgers, the same rule that a receipt path counts only when it is a file in the checkout). The ledgers' own rule text (catalogs/landscape/gap-wave2-20260923--agent-lab-17.json, key rule) defines the statuses that are passed through unchanged.
  • Sealed-wave rule: tools/sota-convergence/README.md at d6d3bf7d, "Record verdicts" (a grandfathered wave is never re-recorded) and its practice on this row in Grand catalogs carry each winner's evidence basis; ai-memory is the retained control, not a measured best #269 (89cf253c) and Pin ai-memory 2.4.1 and mcporter 0.14.1 on Linux; workstation SOTA refresh record (headroom 0.39.0 qualified, not switched) #307 (ff20470b): dated limitations entries.
  • Adopted-only winners: tools/sota-convergence/lane_packets.py build_candidate (adopted is a disposition of selected or conditional) and scripts/landscape.py lane_winner_components (rejects a non-adopted winner key), both at d6d3bf7d.
  • Memory comparison facts: evidence/artifacts/memory-stack-20260925/convergence.json (arms C3 and D2, amendment A14) and blueprints/memory-stack/longmemeval/v4/PREREGISTRATION.md (arms D2h, K1, M1, M2). Upstream currency: https://github.com/akitaonrails/ai-memory/releases/tag/v2.5.2, published 2026-10-01T07:40:18Z (gh api repos/akitaonrails/ai-memory/releases/latest).
  • The four corrected acceptance classes: the committed receipts under evidence/artifacts/gap-wave2-20260923/ (us-equities__identity-provenance/0-dvc-repro-restore-contract.json and 4-four-store-comparison.json, us-equities__data-quality-orchestration/5-duckdb-gate-input.json, us-equities__evaluation-experiments/1-five-package-installs.json and 2-matched-promptfoo-inspect-comparison.json), classed by docs/acceptance-evidence-policy.md.
  • The merit rule: the user's message of 2026-10-01, quoted in decision 5 of the program record.

Evidence-class table

Claim Evidence class Command / receipt
The builder joins the ledgers and dates the sealed fields; the old fields keep their shape local_integration python3 -m unittest tests.test_landscape_sweep_harness tests.test_landscape_sweep_skills (three new tests, written failing first); a real build on main's catalogs, summary line above
The row note passes the landscape validator and leaves the sealed wave untouched local_integration python3 scripts/landscape.py --root . rc 0; python3 tools/sota-convergence/build_verdicts.py --check rc 0
A label-withheld packet keeps one neutral sentence of the note and none of its selection statements local_integration lane_packets.build_all_packets(..., withhold=True) in both builds of RealLedgerIncumbentBlindnessTests; tests.test_lane_packets
DVC, pandera, MLflow and Inspect AI each have a recorded run source_review of committed receipts the five receipt files named above, read on d6d3bf7d; no run was repeated
Nothing was run on a model, a memory system or a provider none n/a

Local commands run

$ python3 scripts/landscape.py --root .                              # rc 0
$ python3 tools/sota-convergence/build_verdicts.py --check          # rc 0
$ python3 scripts/build_ecosystem.py --check                        # rc 0, status passed
$ python3 scripts/validate.py                                       # rc 0: 69 components, 8901 hashed files, 183 receipts
$ python3 -m unittest -q tests.test_landscape_sweep_harness tests.test_landscape_sweep_skills tests.test_ecosystem_manifest tests.test_lane_packets tests.test_blind_checkout tests.test_landscape tests.test_layer_verdicts tests.test_record_verdicts tests.test_component_matrix tests.test_practice_references tests.test_handbook_summary tests.test_verdict_review_gate tests.test_saturation_ledger tests.test_platform_status tests.test_adjudicate tests.test_blind_layer_decisions tests.test_verdict_lane_vendoring tests.test_agentsview_qualification
Ran 1305 tests in 138.230s
OK (skipped=46)
$ git diff --check origin/main HEAD                                 # rc 0

Manifest: 4 of the 7 changed files are registered and re-registered; files rows 8,901 before and after, receipts 183, convergence records 26, none lost.

Decision record

docs/decisions/2026-10-01-definitive-sota-wsl-program.md, decision 5 (extended here), and docs/decisions/2026-10-01-new-wsl-architecture-edition.md (counts and the lesson that a none_recorded is a claim that needs the same receipt search as a stronger class). Alternatives considered for the stale inputs: a hand edit of the verdict fields (refused by the tooling); a Claude-only single-lane re-record (one model family, and it can only re-affirm an adopted candidate); a sentence in the worker templates (the fit template is shared with the skills modality and both prompt hashes are pinned, so the note travels inside each repository-layer input instead); a two-lane re-record of all rows, which is program unit U4 and stays queued. Overturn: a recorded head-to-head on the new distribution changes a candidate's disposition, and a new wave re-records the row.

Host evidence

Not relevant: no file under evidence/hosts/ changes.

Checklist

  • No workflow or action changes.
  • No secrets are printed, logged or committed.
  • No new paid hosting, subscription or billing surface was introduced.
  • Peer-owned untracked files and worktrees were preserved.

🤖 Generated with Claude Code

@seathatflowsinourveins seathatflowsinourveins added the lane:shared Touches files owned by both lanes; needs both lanes' acknowledgement label Oct 1, 2026
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator Author

Trading lane: acknowledged at 339d57c6 (base d6d3bf7d). I read the 12 us-equities rows in code against the base.

  • identity-provenance is now local_integration_check. DVC is cited to 0-dvc-repro-restore-contract.json as native operation, with its text and the four-store comparison reason.
  • data-quality-orchestration is now structural_validation. pandera is cited to 5-duckdb-gate-input.json as a local integration check.
  • evaluation-experiments is now structural_validation. MLflow and Inspect AI are cited to 1-five-package-installs.json as native operation, and the overturn-comparison reason is added.
  • Every other trading row changes only in citation form: pending_source becomes source_path (Review all replacement WSL architecture layers and retained gates #575 landed), with the same text.
  • No change to any verdict, gate or closure cell.

One note change, verbatim. In identity-provenance's notes, replace "no gate blocks it, and it needs a recorded DVC install first. The verdict's own winner, DVC 3.67.1, has no pin of record." with "no gate blocks it. DVC 3.67.1 already ran natively on synthetic fixtures (2026-09-23), so the comparison needs a fresh DVC install on the target host and the point-in-time fixture. DVC's pin is the verdict's baseline 3.67.1; it has no stack pin, and the new distribution installs the current upstream release."

Scout and others added 6 commits October 1, 2026 11:10
…, the arms without a result and upstream v2.5.2

The row's verdict fields (winners, alternatives, open_gaps) are the sealed 2026-09-22 wave and stay as recorded.
The note follows the practice of #269 and #307: a dated limitations entry. It records that ai-memory is the source
host's selection of record and the comparison's reference arm, that the merit winner is undetermined, which arms of
the frozen harness have no result (C4, D2h, K1, M1, M2), that the 2026-09-25 result is D2 against C3 with the
reranker off, that upstream published v2.5.2 on 2026-10-01, and that a verdict wave can name only an adopted
candidate as winner.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…dgers onto the shown gaps

An audit of the 32 frozen sweep inputs (the Codex catalog lane, 2026-10-01) found 14 winner pins behind the stack
and gap texts that committed receipts had already answered. Both come from one cause: build_inputs.py copies
winners, alternatives and the first five open_gaps from the row's sealed 2026-09-22 verdict, which only a new verdict
wave re-records, and says nothing about their date.

A repository-layer input now also carries verdict_checked_at (the row's checked_at), verdict_note (which fields are
the sealed verdict; components_vs_upstream[].pin is the pin installed today) and open_gaps_followup: the gap-wave
owner ledgers joined onto the shown gaps by index and exact text, with status, receipts present in the checkout,
waves and ledgers. Existing fields keep their shape and no template changes, so the pinned prompt hashes stay.
The summary line counts the ledger entries joined, dropped and beyond the five shown. Reference for the ledger
reading: tools/sota-convergence/lane_packets.py gap_receipts_index.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…dscape catalog

A verdict wave can name only an adopted candidate as a layer's winner (a candidates[] entry with a disposition of
selected or conditional; lane_packets.py build_candidate, landscape.py lane_winner_components), and a lane's return
has no undetermined form. A comparison's result therefore enters catalogs/landscape/ through a candidate entry with
an adopted disposition that cites the executed comparison, then a new verdict wave. The sealed 2026-09-22 verdict
fields are not edited by hand; a dated limitations entry and the sweep input's dated fields carry what changed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…; four trading acceptance classes corrected from committed receipts

The 39 pending sources that named pull request 575 (37 in rows, 2 in the header) are plain source_path entries now
that it merged as 75c83d8. DVC, pandera, MLflow and Inspect AI were rated none_recorded from the assessment's
source review only; committed gap-wave2 receipts of 2026-09-23 record a run of each (the Codex lane's audit named
them, the trading lane owner corrected its rating, and each receipt was read before its class changed). DVC, MLflow
and Inspect AI become upstream_example_or_native_operation and pandera local_integration_check; the rows'
classes follow (identity-provenance local_integration_check; data-quality-orchestration and evaluation-experiments
structural_validation). The identity-provenance row names the four-store run as the comparison so far, not a
closing one. Counts in the record: 130 winner entries, 14 none_recorded.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…nance (verbatim from its acknowledgement of #585)

The note said the comparison needs a recorded DVC install first. The committed 2026-09-23 receipt records a native
DVC run, so the owner replaced the sentence: the comparison needs a fresh DVC install on the target host and the
point-in-time fixture; DVC's pin is the verdict's baseline 3.67.1 with no stack pin.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ot-file protocol)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@seathatflowsinourveins
seathatflowsinourveins force-pushed the foundation/audit-followup-20261001 branch from 339d57c to e4e7282 Compare October 1, 2026 15:11
@seathatflowsinourveins
seathatflowsinourveins merged commit a5ee23a into main Oct 1, 2026
25 checks passed
@seathatflowsinourveins
seathatflowsinourveins deleted the foundation/audit-followup-20261001 branch October 1, 2026 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:shared Touches files owned by both lanes; needs both lanes' acknowledgement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant