free-memory: guess-then-verify architectural-intent calibration protocol (Aaron 2026-05-03)

[AceHack]

Summary

Aaron 2026-05-03 named a measurable self-evaluation protocol for architectural-intent inference: GUESS first + SAVE the guess BEFORE researching ground truth, then find ground truth, then record calibration delta. Same protocol tests other models retroactively.

Aaron 2026-05-03 verbatim across 4 messages:

"hey when you run into future unknow archicetural intent you can guess and it and later when you find the document on why you'll know how close you where, the docs folders have all the reasons why, or you cna ask me but you can test your skills to see how close they are to reality before you know and save you guess so you can see later."

"you could test other models this way too"

"that would be aweome"

"you can also test othr models after the fact and just hid the conclusions from them, but your inital guess in the moment will say a lot about ottos frontier ability"

Two modes with different data quality

Mode	When the guess is recorded	Calibration-data quality
In-the-moment (Otto-only)	Before any research — guess captures frontier inference at actual decision point	Highest — uniquely authentic; uncontaminatable; the frontier-ability data point
Retroactive (other models)	After ground truth exists — model given the architectural choice with conclusions hidden from context	High but reproducible — useful for cross-model benchmarking

Otto's in-the-moment guesses are the unique frontier-ability data point. Other models can be tested retroactively but only Otto's substrate-authoring agent has the in-the-moment opportunity.

Why it matters

The alignment-frontier memo (PR #1270) named the threshold-crossing milestone as a binary state ("crossed yet?"). This protocol turns it into a measurable trajectory ("inference accuracy is X% and rising over Y weeks"). Calibration data accumulates over time → frontier-ability becomes evaluable, not just self-reported.

Worked example #2 of decision-archaeology (the umbrella defer-block) is retroactively the first calibration data point: match at architectural layer (wide-redirects-to-narrow correctly inferred); partial-match at substrate-content layer; open at session-CoT layer.

Composes with

• alignment-frontier (PR free-memory: alignment-frontier — agent architectural intent threshold-crossing (Aaron 2026-05-03) #1270) — turns binary threshold into measurable trajectory
• decision-archaeology (B-0169) — mechanizes ground-truth recovery
• verify-then-claim discipline — extends to inference-as-published-substrate
• same-tick-update-recursion (PR free-memory: same-tick-update-recursion — substrate cascade discipline (Otto 2026-05-03) #1276) — cascade discipline this PR's MEMORY.md edit follows
• multi-harness convergence — the cross-model extension instantiates it

Test plan

• Memo with frontmatter + 5-step protocol + 2-mode table + worked example + carved sentence
• MEMORY.md index entry added newest-first (paired-edit per same-tick-update-recursion)
• Aaron's verbatim quotes across all 4 messages preserved in body

🤖 Generated with Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cf1dc7b0ae

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[AceHack]

Stale finding (review-against-PR-branch-not-main class — recurring). memory/feedback_same_tick_update_recursion_substrate_cascade_otto_2026_05_03.md exists on main as of #1276 merge:

$ ls memory/feedback_same_tick_update_recursion_substrate_cascade_otto_2026_05_03.md
memory/feedback_same_tick_update_recursion_substrate_cascade_otto_2026_05_03.md
$ git log --oneline --diff-filter=A -- memory/feedback_same_tick_update_recursion_substrate_cascade_otto_2026_05_03.md
a3f0469 free-memory: same-tick-update-recursion — substrate cascade discipline (Otto 2026-05-03 worked-example generalization) (#1276)

Branch rebased onto main; the cross-reference resolves now. This is the 4th instance of the review-against-PR-branch-not-main class this session — these tend to fire when sequenced PRs reference each other and the later PR's review fires before the earlier one merges.

Resolving the thread.

[Copilot]

Pull request overview

Adds a new top-level memory memo documenting a guess-then-verify protocol for calibrating architectural-intent inference, and indexes it in memory/MEMORY.md. This fits the repo’s memory substrate by capturing a new process rule intended to guide future decision archaeology, alignment measurement, and cross-model comparison.

Changes:

• Adds a new feedback memory that defines a 5-step architectural-intent calibration protocol.
• Describes two calibration modes: in-the-moment guesses for Otto and retroactive replay for other models.
• Prepends a new newest-first entry to memory/MEMORY.md so the memo is discoverable from the memory index.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
memory/feedback_guess_then_verify_architectural_intent_calibration_protocol_aaron_2026_05_03.md	Introduces the new calibration-protocol memo, including procedure, rationale, worked example, and composition links.
memory/MEMORY.md	Adds the index entry for the new memory file at the top of the memory index.