Skip to content

docs(plans): ADR-010 immutable result store, every entry materialized, one owning process (rejected) - #212

Closed
paddymul wants to merge 3 commits into
mainfrom
docs/adr-010-immutable-store-one-owner
Closed

paddymul wants to merge 3 commits into
mainfrom
docs/adr-010-immutable-store-one-owner

Conversation

@paddymul

@paddymul paddymul commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Rejected (2026-09-22). This PR keeps ADR-010 in the repo as a rejected proposal. Paddy rejected it after review: it does not solve the problem comprehensively, and it does not simplify the design as much as expected. The ADR's new "Why it was rejected" section has the findings. In short:

  • D7 can't work as written. D7 says a re-created result that differs makes its descendants stale. But a heal writes the different rows under an unchanged name, so the children still look fresh. Recalc replays them to the same hash and does nothing.
  • Leaving row order to the engine breaks determinism well beyond paging. That's what default partitioning does. It affects file order in grids, children of sorted parents, collect/first/unsorted limit/window ties, and float digests. It also breaks docs/system-contract.md I2 and I6.
  • The float tolerance fails near zero. Two runs disagreed on 901 of 1000 group sums that cancel.
  • Getting determinism back without single-partition execution brings much of ADR-008 back. It needs tie-breaks at every order-sensitive step, a private carry of the input order and ordered ingest. Explicit sorts also cost memory.

ADR-007, ADR-008 and ADR-009 stand as written. The issues ADR-010 listed as closed stay open.

Original description (the proposal as it stood)

Proposed ADR following the review of #189, which filed #193 to #211. None of those is an xorq bug. They come from row order carried through recipes, result files with a mutable lifecycle, two kinds of entry, and Buckaroo executing expressions.

Decisions, in short:

  • D1 Every entry is materialized. The cheap/worthy verdict is still recorded but not acted on.
  • D2/D3 Reproducible means the same multiset of rows (floats within tolerance). Order-insensitive result_digest. Single-partition execution is retired. Sort ties are broken by sortable columns, not __row_order.
  • D4 The writer stamps __row_order, and recipes never see it. Ordered copies are removed.
  • D5/D6 Immutable content-addressed store: written once, never overwritten. Reset moves alias pointers only (no bullpen).
  • D7 A missing object is re-created. A changed result makes descendants stale, with no pin.
  • D8 GC only by explicit tallyman gc.
  • D9 Buckaroo gets result files only.
  • D10 One owning process per TALLYMAN_HOME (a flock held for its lifetime). The MCP server becomes a client.

It amends ADR-007/008/009, which are on #184/#189 and not yet on main, so those links resolve once #189 merges. The ADR has a per-decision table of what stands, changes or is retired, and a list of the issues it closes.

🤖 Generated with Claude Code

paddymul and others added 3 commits September 21, 2026 13:09
…, one owning process (proposed)

Follows the review of #189 (#193-#211). Amends ADR-007, ADR-008 and ADR-009.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…re name and the GC depth

Adds scripts/spike_multiset_digest.py: an order-insensitive digest built on numpy uint64 arithmetic, 4.3s on 18M rows.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Paddy rejected ADR-010 on 2026-09-22: it does not solve the problem comprehensively or simplify the design as much as
expected. The status block says so, and a new "Why it was rejected" section records what the review found. The
proposal itself is unchanged below it, kept for the record. ADR-007, ADR-008 and ADR-009 stand as written.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@paddymul paddymul changed the title docs(plans): ADR-010 immutable result store, every entry materialized, one owning process (proposed) docs(plans): ADR-010 immutable result store, every entry materialized, one owning process (rejected) Sep 22, 2026
@paddymul
paddymul marked this pull request as ready for review September 22, 2026 16:07
@paddymul

Copy link
Copy Markdown
Contributor Author

Closing: ADR-010 (rejected) now lives on #189 as commits c594898, 7e6bdf8 and 34f7138, next to ADR-007/008/009. The branch is left in place.

@paddymul paddymul closed this Sep 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant