Skip to content

compiler: W4-EFFECTS-1 slice E1 — the recorded-product carrier and its verifier - #629

Merged
gstoner merged 2 commits into
mainfrom
agent/w4-e1-recorded-product
Aug 25, 2026
Merged

gstoner merged 2 commits into
mainfrom
agent/w4-e1-recorded-product

Conversation

@gstoner

@gstoner gstoner commented Aug 25, 2026

Copy link
Copy Markdown
Owner

First slice of the merged admissible-effects plan (#628): the carrier every later slice builds on. No pass is wired to it yet — that's E2.

The module encodes the criterion, it doesn't describe it

Each effect class declares the fields that make replay a function of the product, and both review corrections from the plan are required fields, not advice:

  • recorded_mutation without a content_digest is rejected. The existing state_buffer_lineage identity hashes metadata and not contents, so binding "version N" alone lets a replay bind different bytes and produce a different gradient — silently.
  • ordered_collective without a reduction_algorithm (+ topology) is rejected. Floating-point addition isn't associative, so the tree is part of the value; LANGUAGE_AND_IR_SPEC §11 already required fixed ordering and fixed trees.

io is not a reachable class — constructing one fails with the argument for why, so the closure lives in the code rather than only in prose.

Confinement is checked both directions: over-declaring is allowed (conservative, can't hide a stray write); writing something undeclared is rejected; classes that write nothing may not declare a write-set. Region-level totality keeps "admissible" from meaning "unchecked" — an effectful op with no product stays fail-closed, and a product naming an op that isn't on the path is equally rejected.

Verification beyond the unit assertions

Digest injectivity, combinatorially. 120 distinct objects across the three value-bearing classes → 120 distinct digests, zero collisions. This repo has now hit the shared-content-address failure twice (the MegaMoE schedule digest; the f32-only lineage), so I confirmed every field reaches the digest rather than trusting SHA-256 to be handed everything.

(R) demonstrated against the live S4 generator. Record a key, discard the values, rebuild the draw from the product alone, require bit identity — plus a negative showing a different product does not reproduce it, so the test can't pass vacuously. That's the acceptance bar the plan sets and one a distributional check cannot meet.

Gates

19/19 new tests · governance + audit-doc gates green · ruff clean · mypy 0 (476 files) · generated docs in sync · full unit suite 16,448 passed, environmental baseline unchanged — same four files as before (fp8 20, apple 17, scheduled_matmul 3, wmma 1), no new failure site. E1 is additive: one new module, one new test file.

Plan status updated: E1 landed, E2–E5 open.

🤖 Generated with Claude Code

…s verifier

The load-bearing piece of the admissible-effects plan: one content-addressed
carrier per admitted effect, plus a verifier that fails closed rather than
trusting the producer. No pass is wired to it yet — that is E2.

The module encodes the criterion rather than describing it. Each effect class
declares the fields that make replay a function of the product, and BOTH
review corrections from the plan are required fields, not advice:

* `recorded_mutation` without a `content_digest` is rejected. The existing
  `state_buffer_lineage` identity hashes metadata and NOT contents, so
  binding "version N" alone lets a replay bind different bytes and produce a
  different gradient silently.
* `ordered_collective` without a `reduction_algorithm` (and topology) is
  rejected. Floating-point addition is not associative, so the reduction tree
  is part of the value, and LANGUAGE_AND_IR_SPEC section 11 already requires
  fixed ordering AND fixed trees.

`io` is not a reachable class: constructing one fails with the argument for
why (an external read is not a function of any recorded value), so the
closure is visible in the code rather than only in prose.

Confinement is checked in both directions: over-declaring a write-set is
allowed because it is conservative and cannot hide a stray write; writing
something undeclared is rejected; and classes that write nothing may not
declare a write-set at all. Region-level totality keeps "admissible" from
meaning "unchecked" — an effectful op with no product stays fail-closed, and
a product naming an op that is not on the path is equally rejected.

Verification beyond the unit assertions:
* Digest injectivity checked combinatorially — 120 distinct objects across
  the three value-bearing classes produced 120 distinct digests, zero
  collisions. This repo has now hit the shared-content-address failure twice
  (the MegaMoE schedule digest, the f32-only lineage), so every field was
  confirmed to reach the digest rather than assumed to.
* (R) demonstrated against the LIVE S4 generator: record a key, discard the
  values, rebuild the draw from the product alone, require BIT identity —
  with a negative showing a different product does not reproduce it, so the
  test cannot pass vacuously.

Gates: 19/19 new tests; governance + audit-doc gates green; ruff clean;
mypy 0 over 476 files; generated docs in sync; full unit suite 16448 passed
with the recorded environmental baseline unchanged (same four files, no new
failure site).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2a3ccf95d1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread python/tessera/compiler/recorded_product.py Outdated
Comment thread python/tessera/compiler/recorded_product.py
Comment thread python/tessera/compiler/recorded_product.py
Comment thread docs/audit/compiler/W4_ADMISSIBLE_EFFECTS_PLAN.md
… accept the scalar draw

P1 occurrence keying. Keying a region's products by op NAME broke both ways
for a region containing two calls of one op: the second product was rejected
as a duplicate, and — worse — a set of names collapses the repetition, so a
SINGLE product satisfied both occurrences and one effect went unchecked, the
exact failure the totality check exists to prevent. Products now carry a
required `occurrence_id` (the region carrier's existing
`StructuredOperation.operation_id` shape, reused rather than invented) and
the verifier keys by it. A repeated occurrence id in the requested set is
itself rejected, since totality cannot be checked against an ambiguous key.

P2 immutability. A frozen dataclass only stops rebinding the field: _freeze
stored ordinary dicts and lists, so a caller could mutate a nested value
after construction and, because the digest was recomputed on access, move
the content address of a carrier that had already been indexed or
serialized. The product is now deeply immutable (MappingProxyType + tuples,
the pattern schedule_object.py already uses) and the digest is computed ONCE
at construction, so it cannot drift even if the freeze were defeated.
canonical_payload() thaws into fresh plain containers, so the copy a caller
receives cannot alias the carrier.

P2 scalar draw. `shape = ()` is the canonical scalar draw — rng.normal both
accepts and defaults to it — but the required-field check tested membership
in (None, "", ()), reading a legitimate value as an absent field, while the
equivalent JSON form [] passed. The check now tests ABSENCE (key missing, or
None, or ""), and fields whose emptiness really is meaningless (key,
lineage_id, content_digest, communicator, sequence_digest,
reduction_algorithm, topology) are listed per class and fail closed
separately. () and [] now produce the same digest.

P1 backend queues. All four said "plan only, no code", which went stale the
moment E1 landed. Each now records its E1 outcome: rocm and x86 still
follow-up-required with the reason (E1 is host-neutral Python; no surface on
either backend consumes it until E2, so there is no evidence to claim);
nvidia and apple not-applicable-at-this-slice with what each inherits on
adoption.

Gates: 23/23; ruff clean; mypy 0 over 476 files; governance and audit-doc
gates green; generated docs in sync.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@gstoner

gstoner commented Aug 25, 2026

Copy link
Copy Markdown
Owner Author

All four fixed in af909e9. The first is a real hole in the totality guarantee, and it fails in the more dangerous direction.

P1 (occurrence keying) — correct, and it breaks both ways. Keying by op name meant a region with two tessera.dropout calls rejected the second product as a duplicate; worse, set(effectful_ops) collapses the repetition, so one product satisfied both occurrences and an effect went unchecked — precisely what the totality check exists to prevent. Products now carry a required occurrence_id — reusing the region carrier's existing StructuredOperation.operation_id shape rather than inventing an identity — and the verifier keys by it. A repeated occurrence id in the requested set is itself rejected, since totality can't be checked against an ambiguous key. Pinned by a test with two dropout occurrences that asserts both failure directions.

P2 (deep immutability) — correct. A frozen dataclass only stops rebinding the field. product and its nested values are now MappingProxyType + tuples (the pattern schedule_object.py already uses), and — belt and braces — the digest is computed once at construction, so the content address cannot drift even if the freeze were somehow defeated. canonical_payload() thaws into fresh plain containers, so the copy a caller receives can't alias the carrier either. The test mutates at three depths and asserts the digest is unmoved.

P2 (scalar draw) — correct, including the inconsistency you spotted. () is the canonical scalar shape and rng.normal defaults to it, but the check tested membership in (None, "", ()) and so read a legitimate value as absent — while the JSON-equivalent [] passed. The check now tests absence (key missing / None / ""), and fields whose emptiness is meaningless (key, lineage_id, content_digest, communicator, sequence_digest, reduction_algorithm, topology) are listed per class and fail closed separately. () and [] now produce the same digest.

P1 (backend queues) — fair. All four said "plan only, no code", which went stale the moment E1 landed. Each now records its E1 outcome, and the honest one is no evidence: E1 is host-neutral Python that no backend surface consumes until E2, so rocm/x86 stay follow-up-required with that reason rather than claiming a parity they haven't earned; nvidia/apple are not-applicable-at-this-slice with what each inherits on adoption.

Gates: 23/23 · ruff clean · mypy 0 (476 files) · governance + audit-doc green · dashboards in sync.

🤖 Generated with Claude Code

@gstoner
gstoner merged commit 10877fa into main Aug 25, 2026
17 checks passed
@gstoner
gstoner deleted the agent/w4-e1-recorded-product branch August 25, 2026 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant