feat(spine): recompute the temporal spine from the log, and name the gap (#1283) - #1336
Conversation
There was a problem hiding this comment.
Sorry @robotrocketscience, you have reached your weekly rate limit of 500000 diff characters.
Please try again later or upgrade to continue using Sourcery
|
Warning Review limit reached
Next review available in: 40 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Reviewer's GuideImplements a diagnostic command to recompute the temporal spine from the ingest log, report divergence by cause, and codifies the recomputation rules and verification tests without changing the writer. Sequence diagram for aelf spine verify recomputation and divergence reportsequenceDiagram
actor Operator
participant CLI as _cmd_spine
participant Verify as _cmd_spine_verify
participant Store as MemoryStore
participant Recompute as spine_divergence
participant Edges as recompute_spine_edges
Operator->>CLI: aelf spine verify
CLI->>Verify: _cmd_spine_verify(out)
Verify->>Store: MemoryStore(db_path, read_only=True)
Verify->>Recompute: spine_divergence(Store)
Recompute->>Edges: recompute_spine_edges(Store)
Edges-->>Recompute: recomputed_edges, no_log_ids
Recompute-->>Verify: SpineDivergence report
Verify->>Store: close()
Verify-->>Operator: print shipped / recomputed / reproduced and buckets
File-Level Changes
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
PR-size soft capThis PR is over the advisory size threshold:
Bigger PRs collide with more open work, which under the parallel-session workflow tends to produce repeated This is advisory only — nothing is blocked. If the size is intentional (large refactor, module removal, generated code), apply the |
|
[claim:review:Garsecg:2026-08-04T22:51:25Z] |
|
[claim:review:Setr:2026-08-04T22:52:27Z] |
|
[release:review:Setr:2026-08-04T22:52:32Z] |
Review — the recompute is right and well-proven. One vacuous test fixed here; the branch needs a rebase.I ran the gate measurement this PR was funded by, so I reviewed it against what that measurement actually established rather than against the issue body. Pushed to this branch: Reproduced, not taken on trust
Consistency with the operator's constraints, checked one by one: you ship item (1) only and explicitly defer the writer change and the AC4 fan-in assertion — you do not build past what was cleared. You refuse to key on migration-synth ULIDs via Fixed here — nothing distinguished the ULID key from a plain belief-id sort
The production code is correct — I confirmed with a probe where ULID order is the reverse of id order. But nothing pinned it, and this is the one property the whole PR rests on. It matters concretely: 96.5% of this store shares a Added Also fixed here
Left for you
Housekeeping: running this suite currently rewrites the operator's real home (#1320, fix open at PR #1334). I ran everything behind a process-level |
|
[release:review:Garsecg:2026-08-04T23:13:09Z] |
|
[claim:review:Setr:2026-08-05T01:26:38Z] |
|
[claim:review:Garsecg:2026-08-05T01:27:26Z] |
|
[release:review:Garsecg:2026-08-05T01:27:32Z] |
|
[claim:review:Idnn:2026-08-05T01:27:37Z] |
|
[release:review:Idnn:2026-08-05T01:27:43Z] |
The ratified contract says edges are log-derived and the recompute key is `(created_at, ingest_log ULID)`. The shipped writer orders by `(created_at, rowid)`, and `rowid` is implicit — VACUUM may renumber it — which is why the ratified key is the log's ULID rather than anything read off the belief table. This is the recompute half only; the writer is unchanged, so measuring against a live store reports a gap, not drift. A zero here today would mean the recompute had been fitted to the defect. Three rules, none of them a heuristic. Synth log rows are excluded by `source_kind = 'legacy_unknown'`, a stated durable column that is exact in both directions on the development store. Their ULID prefix is migration wall-clock and their order is `beliefs.rowid` relabelled, so honouring them would launder rowid order into the key the contract calls durable. Two heuristic detectors were measured first and both failed: a prefix-vs-`ts` disagreement threshold catches legitimately delayed derivation and collapses reproduction to 34.75%, and excluding the largest ULID-prefix date cluster would drop 51.8% of the log — a real backfill carrying 20,095 session-scoped beliefs. A belief takes its earliest qualifying log row, because later rows are corroborations and only the first records insertion. Beliefs with no qualifying log row sort last within their `created_at` group, then by id. This is a forward convention and explicitly not a recovery: 2,426 such beliefs carry only 433 distinct timestamps, so 94% sit inside a tie where the only other durable column is a content-addressed id. Three placement rules were measured and produced one link of spread between them. The convention buys determinism, not correctness about the past. Divergence is reported in three buckets rather than one percentage, because only one of them is a defect anyone can fix and a single number would let the unreconstructible bucket mask a real key disagreement. On the development store: 41,929 shipped, 39,280 reproduced (93.68%), misses 2,100 no-log / 546 fan-in / 3 other, in 0.86s. Those figures reproduce the #1283 gate measurement and its correction exactly, from an independent implementation.
Reports the recompute divergence by bucket. Opened read-only (#1328), because a bare `MemoryStore` open runs migrations and the #1314 lock sweep, and a diagnostic must not mutate its subject. Exits 0 whatever the divergence. The writer has not moved onto the ratified key yet, so a non-zero gap is the predicted state; making it an error would turn every run red on a number the contract already anticipates. The note in the output says which of the two it is, since "93.68%" alone reads as drift.
Each rule has a test that goes red under a mutation of exactly that rule, verified: dropping the synth exclusion, flipping the no-log sentinel from last to first, keying on the ULID ahead of `created_at`, taking the latest log row instead of the earliest, and chaining across session boundaries. Two of these are controls rather than assertions about the fix. `test_divergence_is_zero_when_the_writer_agrees` keeps the bucket tests honest — a recompute that produced nothing would file every shipped edge under some bucket and look like correct attribution. And the empty-store case pins that zero edges is full reproduction rather than 0%, so a fresh store does not read as broken. Nothing here asserts that divergence is zero on a real store. The writer still keys on `rowid`, so a passing zero would mean the recompute had been fitted to the defect rather than to the contract.
States the two things a reader would otherwise get wrong: that the number is a gap against the ratified key rather than drift, because the writer has not moved yet; and that the no-log bucket is a stated forward convention rather than a recovery, because that ordering is unreconstructible.
…claiming durability
Three things, all found by review.
1. Nothing distinguished the ratified sort key from a plain
`(created_at, id)` sort. Replacing the log key with a constant left
the whole file green: every fixture picked ids whose alphabetical
order coincides with their assigned ULID order, so the two sorts
produce identical chains. The new test opposes them — `aaa` carries
the last ULID, `ccc` the first — so an id sort yields
{(bbb,aaa),(ccc,bbb)} and the log yields {(bbb,ccc),(aaa,bbb)}.
That mutation now fails, and only on the new test. This matters
because 96.5% of the store shares a created_at, so the belief id is
doing the ordering wherever the ULID is ignored.
2. The docstring said the sort key's components are all durable.
Operator constraint (3) on #1283 forbids exactly that wording: state
it as a guarantee with a deterministic intra-millisecond tiebreak, or
as an observed property with the cross-process exposure attached.
There is no tiebreak here, so it is now stated as observed, with the
measured 0.017% (10 groups / 20 rows, deciding zero links today) and
the pointer to ulid.py's own 'cross-process drift is possible but
tolerated'.
3. SYNTH_SOURCE_KIND redeclared the literal 'legacy_unknown' that
models.INGEST_SOURCE_LEGACY_UNKNOWN already owns and that store.py
and replay.py import. Rebound to the shared constant.
Suite: 7116 passed, 69 skipped, 71 xfailed.
|
Rebased onto current main (was behind and could not fast-forward) — full suite 7,183 passed / 69 skipped / 71 xfailed under a sandboxed The operator ruling on #1283 landed after this PR was opened, and it ratifies the framing this PR already ships — recording that here so a reviewer does not re-open it:
One thing the ruling adds that is worth carrying in the docstring if anyone touches this again: the ULID prefix is backfill processing time (p50 14.1 d, max 58.5 d behind its own content), so it is a sound ordering key — 98.94% of adjacent pairs agree with content I authored this, so I cannot review it — |
3257ef5 to
51b4558
Compare
The module docstring was corrected to state the ULID component as an observed property rather than a guarantee, per the #1283 constraint against wording that implies the stronger claim while resting on the weaker. The test file's own header still read "every component durable" — the same claim, in the file whose job is to pin it.
The docstring enumerated {backfill,clear} and predated the verify
action added alongside it, so the one action that opens read-only was
the one the docstring did not mention.
The command had no coverage at all: 62 lines including the `read_only=True` open that is the entire reason a diagnostic may be pointed at a live store (#1328). Dropping that keyword, or swapping the open for `_open_store()`, shipped green. Three arms, because one is not enough. A kwarg spy on the constructor names the mutation at the call site but cannot see a refactor that reaches past the patched name; a sha256 over the store file catches that but reports a hash mismatch rather than a line; and an expired user lock left locked pins the specific write a bare open performs — the #1314 sweep — with a control asserting a read-write open does flip it, so neither byte arm can pass vacuously. The report-shape fixture gives all six printed counts distinct values (10/7/4, buckets 3/2/1) so a bucket printed from the wrong field fails; with equal counts that mutation is green.
`read_only=True` opens through SQLite's `mode=ro` URI, which refuses to create the file. Every other `spine` action reaches the store through `_open_store()`, so `verify` is the only one that meets a repo with nothing built and it did so with an uncaught sqlite3.OperationalError: `main()` wraps the dispatch in nothing, so the user got a raw traceback indistinguishable from a corrupt store or a permissions fault. A store that was never built is the expected state of a fresh repo, not a diagnostic failure, so it reports one line and keeps the exit at 0 along with every other outcome of this command.
The entry read "exact in both directions here", which is true of the column but reads as a measured effect. The measured effect is zero: neutralising the exclusion leaves every bucket identical and the recomputed edge set unchanged. 20,852 beliefs take a different sort key and only 5 are session-scoped, each also carrying a non-synth row. The rule is still right — it is a forward safeguard against a store whose migration ran at a different time — but an inert lever that nobody labels becomes folklore about why the number is what it is.
Review — every published number re-derives. One real defect fixed here, the CLI is now guarded, and the prior review's three items are closed.I re-derived the load-bearing figures from a fresh Store census on my copy: 44,652 active beliefs / 46,004 total / 139,893 log rows / 41,983 Reproduced independently
One correction to the record, in your favour. I initially could not re-derive the CHANGELOG's "flags 30,738 rows"; a first pass over all log rows gives 86,858. That pass is measuring the wrong population. Applied where the rule actually operates — the earliest log row per belief, which is exactly what Fixed here —
|
|
merge-train: merged 94d0ddb → |
|
[release:review:Setr:2026-08-05T02:05:41Z] |
The memo still read 'Until that recompute exists', which went stale when #1336 merged. It also omitted the rowid-exclusion rationale, which is the load-bearing half of the 2026-08-01 ruling: the writer orders on (created_at, rowid), that rowid is implicit under 'id TEXT PRIMARY KEY', VACUUM may renumber it and it survives no rebuild — which is why the ratified key is the log's ULID. States what shipped rather than implying more: 'aelf spine verify' covers TEMPORAL_NEXT only, reproduces ~93.7% against a ~95.0% ceiling under the current writer, and reports the gap rather than closing it. Reaching the 98.70% structural ceiling needs the unfunded writer change, so points 3 and 4 stay edge-incomplete. The pre-v2.0 'historical memo' header now notes that the ratified edge contract it carries is current.
…edges #1283 restated AC2 in two halves. The recompute half shipped in #1336; this is the other one. Edges that are neither TEMPORAL_NEXT nor DERIVED_FROM are a function of the belief set AND of detector thresholds, so "edges are recomputable" holds for them only if those thresholds are pinned and versioned. They were bare module constants with no guard. detector_thresholds records 22 constants across 8 modules behind DETECTOR_THRESHOLDS_VERSION: the relationship_detector cutoffs, caps and vocabularies, the contradiction precedence ladder, the triple_extractor phrase-to-edge-type patterns, the value_compare slot gate, and the constants on the two paths that decide which phantoms reach the RELATES_TO writer. The module holds hand-written literals and imports nothing from aelfrice. That is deliberate: importing the constants it describes would make it tautological in exactly the way the tests it replaces were, and would drag store/bm25 into the import graph of any reader. Scalars pin as literals so a reviewer can check them by eye; collections pin as a digest of a canonical form that includes regex flags, since dropping re.IGNORECASE changes which triples match without changing any pattern text. Entries were checked for reachability rather than assumed. The wonder bake-off constants are NOT pinned -- the package docstring states the strategies are research-only and their sole importer builds against an in-memory store, so they decide no edge a user holds. The value_compare entries are pinned but labelled dormant: nothing shipped passes use_value_comparison=True. Three upstream suppliers the call-site sweep cannot see (bm25._TOKEN_PATTERN, models.ANCHOR_TEXT_MAX_LEN, wonder_consolidation._TOKENIZER_DROP) are pinned by hand, and the two suppliers left unpinned are named so their absence is a decision. Forward-only. The edges table has no version and no created_at, so this does not make a historical edge attributable to the thresholds that produced it; adding those columns is the migration that bricked stores in #1161, and historical reproduction stays out of scope.
…usion `gates` is the manifest's substance -- "a constant that cannot answer this does not belong here" -- and nothing tests these strings, so five wrong ones shipped. All five were reproduced against live source before being rewritten. - SUPERSEDES_WEIGHT claimed "propagation arithmetic". There is none: Edge.weight is read by a BFS sort key, a clustering floor and persistence. Propagation is EDGE_VALENCE, keyed on edge TYPE, which this constant does not touch. The real effect is sharper -- at 1.0 it clears DEFAULT_CLUSTER_EDGE_FLOOR (0.4), and any value below that silently drops every SUPERSEDES edge out of candidate clustering. - UNCERTAINTY_THRESHOLD claimed to filter the anchor tuple. It does not: `anchors` is built from the unfiltered `known_beliefs`. It selects high_uncertainty_beliefs, which decides whether an uncertainty_deep_dive axis is emitted -- still a real path to RELATES_TO, but not the stated one. - QUANT_AXIS said the score is half the axis distance. It is a quarter: q_term halves the distance and the score halves it again. `always` vs `sometimes` is 1.0 apart and scores 0.25, so it lands as POTENTIALLY_STALE rather than CONTRADICTS -- the wrong side of the very split DEFAULT_CONFIDENCE_MIN exists to record. Sizing an edit with the old prose picks the wrong value. - DEFAULT_JACCARD_MIN claimed lowering it "can only add edges". The candidate pool is monotonic; the written set is not, because DEFAULT_MAX_EDGES_PER_BELIEF is spent in sorted pair order, so a newly-admitted pair can evict a previously written one. - _PATTERNS omitted TEMPORAL_NEXT from edge_types while four of its 25 patterns mint exactly that type (`follows`, `comes after`, `is after`, `succeeds`). That last one has a consequence outside the manifest, so the exclusion entry now carries it: EXCLUDED_WRITERS said temporal_spine "writes TEMPORAL_NEXT only", which reads as the spine accounting for the whole TEMPORAL_NEXT population. It does not -- triple_extractor is a second, prose-driven producer the #1336 spine recompute does not cover. Also records two limits the manifest was silent on. `--axes-budget` (default 24) caps the anchor tuple and so how many RELATES_TO edges each persisted phantom writes -- a bigger lever than several pinned constants, but a signature default rather than a module constant, out of reach of the (module, name) scheme for the same reason the inline weights are. And belief ARRIVAL ORDER is a third input beside the belief set and these thresholds: the per-belief cap is spent on whichever pairs arrived first, so full-store and incremental runs can disagree on identical beliefs, and re-deriving edges from beliefs plus this manifest gives a false mismatch on any incrementally built store -- which is every real one.
…edges #1283 restated AC2 in two halves. The recompute half shipped in #1336; this is the other one. Edges that are neither TEMPORAL_NEXT nor DERIVED_FROM are a function of the belief set AND of detector thresholds, so "edges are recomputable" holds for them only if those thresholds are pinned and versioned. They were bare module constants with no guard. detector_thresholds records 22 constants across 8 modules behind DETECTOR_THRESHOLDS_VERSION: the relationship_detector cutoffs, caps and vocabularies, the contradiction precedence ladder, the triple_extractor phrase-to-edge-type patterns, the value_compare slot gate, and the constants on the two paths that decide which phantoms reach the RELATES_TO writer. The module holds hand-written literals and imports nothing from aelfrice. That is deliberate: importing the constants it describes would make it tautological in exactly the way the tests it replaces were, and would drag store/bm25 into the import graph of any reader. Scalars pin as literals so a reviewer can check them by eye; collections pin as a digest of a canonical form that includes regex flags, since dropping re.IGNORECASE changes which triples match without changing any pattern text. Entries were checked for reachability rather than assumed. The wonder bake-off constants are NOT pinned -- the package docstring states the strategies are research-only and their sole importer builds against an in-memory store, so they decide no edge a user holds. The value_compare entries are pinned but labelled dormant: nothing shipped passes use_value_comparison=True. Three upstream suppliers the call-site sweep cannot see (bm25._TOKEN_PATTERN, models.ANCHOR_TEXT_MAX_LEN, wonder_consolidation._TOKENIZER_DROP) are pinned by hand, and the two suppliers left unpinned are named so their absence is a decision. Forward-only. The edges table has no version and no created_at, so this does not make a historical edge attributable to the thresholds that produced it; adding those columns is the migration that bricked stores in #1161, and historical reproduction stays out of scope.
…usion `gates` is the manifest's substance -- "a constant that cannot answer this does not belong here" -- and nothing tests these strings, so five wrong ones shipped. All five were reproduced against live source before being rewritten. - SUPERSEDES_WEIGHT claimed "propagation arithmetic". There is none: Edge.weight is read by a BFS sort key, a clustering floor and persistence. Propagation is EDGE_VALENCE, keyed on edge TYPE, which this constant does not touch. The real effect is sharper -- at 1.0 it clears DEFAULT_CLUSTER_EDGE_FLOOR (0.4), and any value below that silently drops every SUPERSEDES edge out of candidate clustering. - UNCERTAINTY_THRESHOLD claimed to filter the anchor tuple. It does not: `anchors` is built from the unfiltered `known_beliefs`. It selects high_uncertainty_beliefs, which decides whether an uncertainty_deep_dive axis is emitted -- still a real path to RELATES_TO, but not the stated one. - QUANT_AXIS said the score is half the axis distance. It is a quarter: q_term halves the distance and the score halves it again. `always` vs `sometimes` is 1.0 apart and scores 0.25, so it lands as POTENTIALLY_STALE rather than CONTRADICTS -- the wrong side of the very split DEFAULT_CONFIDENCE_MIN exists to record. Sizing an edit with the old prose picks the wrong value. - DEFAULT_JACCARD_MIN claimed lowering it "can only add edges". The candidate pool is monotonic; the written set is not, because DEFAULT_MAX_EDGES_PER_BELIEF is spent in sorted pair order, so a newly-admitted pair can evict a previously written one. - _PATTERNS omitted TEMPORAL_NEXT from edge_types while four of its 25 patterns mint exactly that type (`follows`, `comes after`, `is after`, `succeeds`). That last one has a consequence outside the manifest, so the exclusion entry now carries it: EXCLUDED_WRITERS said temporal_spine "writes TEMPORAL_NEXT only", which reads as the spine accounting for the whole TEMPORAL_NEXT population. It does not -- triple_extractor is a second, prose-driven producer the #1336 spine recompute does not cover. Also records two limits the manifest was silent on. `--axes-budget` (default 24) caps the anchor tuple and so how many RELATES_TO edges each persisted phantom writes -- a bigger lever than several pinned constants, but a signature default rather than a module constant, out of reach of the (module, name) scheme for the same reason the inline weights are. And belief ARRIVAL ORDER is a third input beside the belief set and these thresholds: the per-belief cap is spent on whichever pairs arrived first, so full-store and incremental runs can disagree on identical beliefs, and re-deriving edges from beliefs plus this manifest gives a false mismatch on any incrementally built store -- which is every real one.
The entry read "published as `93.68%` (39,335 / 41,984) and is now `94.86%`", but 39,335 / 41,984 is 93.69%. The 93.68% belongs to #1336's measurement, 39,280 / 41,929 — a snapshot 55 shipped edges smaller — which the unchanged #1283 entry three lines below still reports as such. So the pair presented as a pure denominator correction spanned two store states, and about 0.01pp of the 1.18pp move was snapshot drift rather than the correction. The module's own docstring already said 93.69%, so the branch shipped both answers. Both figures now come from one `spine_divergence()` call on one store, and the #1336 figure is named separately rather than standing in as the before. The design memo's "98.70% structural ceiling" is corrected the same way: it is defined by the fan-in misses this change removes from the denominator, so it is not the ceiling for 94.86% and the memo now says so rather than inviting the subtraction it forbids two sentences earlier.
The entry read "published as `93.68%` (39,335 / 41,984) and is now `94.86%`", but 39,335 / 41,984 is 93.69%. The 93.68% belongs to #1336's measurement, 39,280 / 41,929 — a snapshot 55 shipped edges smaller — which the unchanged #1283 entry three lines below still reports as such. So the pair presented as a pure denominator correction spanned two store states, and about 0.01pp of the 1.18pp move was snapshot drift rather than the correction. The module's own docstring already said 93.69%, so the branch shipped both answers. Both figures now come from one `spine_divergence()` call on one store, and the #1336 figure is named separately rather than standing in as the before. The design memo's "98.70% structural ceiling" is corrected the same way: it is defined by the fan-in misses this change removes from the denominator, so it is not the ceiling for 94.86% and the memo now says so rather than inviting the subtraction it forbids two sentences earlier.
…edges #1283 restated AC2 in two halves. The recompute half shipped in #1336; this is the other one. Edges that are neither TEMPORAL_NEXT nor DERIVED_FROM are a function of the belief set AND of detector thresholds, so "edges are recomputable" holds for them only if those thresholds are pinned and versioned. They were bare module constants with no guard. detector_thresholds records 22 constants across 8 modules behind DETECTOR_THRESHOLDS_VERSION: the relationship_detector cutoffs, caps and vocabularies, the contradiction precedence ladder, the triple_extractor phrase-to-edge-type patterns, the value_compare slot gate, and the constants on the two paths that decide which phantoms reach the RELATES_TO writer. The module holds hand-written literals and imports nothing from aelfrice. That is deliberate: importing the constants it describes would make it tautological in exactly the way the tests it replaces were, and would drag store/bm25 into the import graph of any reader. Scalars pin as literals so a reviewer can check them by eye; collections pin as a digest of a canonical form that includes regex flags, since dropping re.IGNORECASE changes which triples match without changing any pattern text. Entries were checked for reachability rather than assumed. The wonder bake-off constants are NOT pinned -- the package docstring states the strategies are research-only and their sole importer builds against an in-memory store, so they decide no edge a user holds. The value_compare entries are pinned but labelled dormant: nothing shipped passes use_value_comparison=True. Three upstream suppliers the call-site sweep cannot see (bm25._TOKEN_PATTERN, models.ANCHOR_TEXT_MAX_LEN, wonder_consolidation._TOKENIZER_DROP) are pinned by hand, and the two suppliers left unpinned are named so their absence is a decision. Forward-only. The edges table has no version and no created_at, so this does not make a historical edge attributable to the thresholds that produced it; adding those columns is the migration that bricked stores in #1161, and historical reproduction stays out of scope.
…usion `gates` is the manifest's substance -- "a constant that cannot answer this does not belong here" -- and nothing tests these strings, so five wrong ones shipped. All five were reproduced against live source before being rewritten. - SUPERSEDES_WEIGHT claimed "propagation arithmetic". There is none: Edge.weight is read by a BFS sort key, a clustering floor and persistence. Propagation is EDGE_VALENCE, keyed on edge TYPE, which this constant does not touch. The real effect is sharper -- at 1.0 it clears DEFAULT_CLUSTER_EDGE_FLOOR (0.4), and any value below that silently drops every SUPERSEDES edge out of candidate clustering. - UNCERTAINTY_THRESHOLD claimed to filter the anchor tuple. It does not: `anchors` is built from the unfiltered `known_beliefs`. It selects high_uncertainty_beliefs, which decides whether an uncertainty_deep_dive axis is emitted -- still a real path to RELATES_TO, but not the stated one. - QUANT_AXIS said the score is half the axis distance. It is a quarter: q_term halves the distance and the score halves it again. `always` vs `sometimes` is 1.0 apart and scores 0.25, so it lands as POTENTIALLY_STALE rather than CONTRADICTS -- the wrong side of the very split DEFAULT_CONFIDENCE_MIN exists to record. Sizing an edit with the old prose picks the wrong value. - DEFAULT_JACCARD_MIN claimed lowering it "can only add edges". The candidate pool is monotonic; the written set is not, because DEFAULT_MAX_EDGES_PER_BELIEF is spent in sorted pair order, so a newly-admitted pair can evict a previously written one. - _PATTERNS omitted TEMPORAL_NEXT from edge_types while four of its 25 patterns mint exactly that type (`follows`, `comes after`, `is after`, `succeeds`). That last one has a consequence outside the manifest, so the exclusion entry now carries it: EXCLUDED_WRITERS said temporal_spine "writes TEMPORAL_NEXT only", which reads as the spine accounting for the whole TEMPORAL_NEXT population. It does not -- triple_extractor is a second, prose-driven producer the #1336 spine recompute does not cover. Also records two limits the manifest was silent on. `--axes-budget` (default 24) caps the anchor tuple and so how many RELATES_TO edges each persisted phantom writes -- a bigger lever than several pinned constants, but a signature default rather than a module constant, out of reach of the (module, name) scheme for the same reason the inline weights are. And belief ARRIVAL ORDER is a third input beside the belief set and these thresholds: the per-belief cap is spent on whichever pairs arrived first, so full-store and incremental runs can disagree on identical beliefs, and re-deriving edges from beliefs plus this manifest gives a false mismatch on any incrementally built store -- which is every real one.
…edges #1283 restated AC2 in two halves. The recompute half shipped in #1336; this is the other one. Edges that are neither TEMPORAL_NEXT nor DERIVED_FROM are a function of the belief set AND of detector thresholds, so "edges are recomputable" holds for them only if those thresholds are pinned and versioned. They were bare module constants with no guard. detector_thresholds records 22 constants across 8 modules behind DETECTOR_THRESHOLDS_VERSION: the relationship_detector cutoffs, caps and vocabularies, the contradiction precedence ladder, the triple_extractor phrase-to-edge-type patterns, the value_compare slot gate, and the constants on the two paths that decide which phantoms reach the RELATES_TO writer. The module holds hand-written literals and imports nothing from aelfrice. That is deliberate: importing the constants it describes would make it tautological in exactly the way the tests it replaces were, and would drag store/bm25 into the import graph of any reader. Scalars pin as literals so a reviewer can check them by eye; collections pin as a digest of a canonical form that includes regex flags, since dropping re.IGNORECASE changes which triples match without changing any pattern text. Entries were checked for reachability rather than assumed. The wonder bake-off constants are NOT pinned -- the package docstring states the strategies are research-only and their sole importer builds against an in-memory store, so they decide no edge a user holds. The value_compare entries are pinned but labelled dormant: nothing shipped passes use_value_comparison=True. Three upstream suppliers the call-site sweep cannot see (bm25._TOKEN_PATTERN, models.ANCHOR_TEXT_MAX_LEN, wonder_consolidation._TOKENIZER_DROP) are pinned by hand, and the two suppliers left unpinned are named so their absence is a decision. Forward-only. The edges table has no version and no created_at, so this does not make a historical edge attributable to the thresholds that produced it; adding those columns is the migration that bricked stores in #1161, and historical reproduction stays out of scope.
…edges #1283 restated AC2 in two halves. The recompute half shipped in #1336; this is the other one. Edges that are neither TEMPORAL_NEXT nor DERIVED_FROM are a function of the belief set AND of detector thresholds, so "edges are recomputable" holds for them only if those thresholds are pinned and versioned. They were bare module constants with no guard. detector_thresholds records 22 constants across 8 modules behind DETECTOR_THRESHOLDS_VERSION: the relationship_detector cutoffs, caps and vocabularies, the contradiction precedence ladder, the triple_extractor phrase-to-edge-type patterns, the value_compare slot gate, and the constants on the two paths that decide which phantoms reach the RELATES_TO writer. The module holds hand-written literals and imports nothing from aelfrice. That is deliberate: importing the constants it describes would make it tautological in exactly the way the tests it replaces were, and would drag store/bm25 into the import graph of any reader. Scalars pin as literals so a reviewer can check them by eye; collections pin as a digest of a canonical form that includes regex flags, since dropping re.IGNORECASE changes which triples match without changing any pattern text. Entries were checked for reachability rather than assumed. The wonder bake-off constants are NOT pinned -- the package docstring states the strategies are research-only and their sole importer builds against an in-memory store, so they decide no edge a user holds. The value_compare entries are pinned but labelled dormant: nothing shipped passes use_value_comparison=True. Three upstream suppliers the call-site sweep cannot see (bm25._TOKEN_PATTERN, models.ANCHOR_TEXT_MAX_LEN, wonder_consolidation._TOKENIZER_DROP) are pinned by hand, and the two suppliers left unpinned are named so their absence is a decision. Forward-only. The edges table has no version and no created_at, so this does not make a historical edge attributable to the thresholds that produced it; adding those columns is the migration that bricked stores in #1161, and historical reproduction stays out of scope.
…usion `gates` is the manifest's substance -- "a constant that cannot answer this does not belong here" -- and nothing tests these strings, so five wrong ones shipped. All five were reproduced against live source before being rewritten. - SUPERSEDES_WEIGHT claimed "propagation arithmetic". There is none: Edge.weight is read by a BFS sort key, a clustering floor and persistence. Propagation is EDGE_VALENCE, keyed on edge TYPE, which this constant does not touch. The real effect is sharper -- at 1.0 it clears DEFAULT_CLUSTER_EDGE_FLOOR (0.4), and any value below that silently drops every SUPERSEDES edge out of candidate clustering. - UNCERTAINTY_THRESHOLD claimed to filter the anchor tuple. It does not: `anchors` is built from the unfiltered `known_beliefs`. It selects high_uncertainty_beliefs, which decides whether an uncertainty_deep_dive axis is emitted -- still a real path to RELATES_TO, but not the stated one. - QUANT_AXIS said the score is half the axis distance. It is a quarter: q_term halves the distance and the score halves it again. `always` vs `sometimes` is 1.0 apart and scores 0.25, so it lands as POTENTIALLY_STALE rather than CONTRADICTS -- the wrong side of the very split DEFAULT_CONFIDENCE_MIN exists to record. Sizing an edit with the old prose picks the wrong value. - DEFAULT_JACCARD_MIN claimed lowering it "can only add edges". The candidate pool is monotonic; the written set is not, because DEFAULT_MAX_EDGES_PER_BELIEF is spent in sorted pair order, so a newly-admitted pair can evict a previously written one. - _PATTERNS omitted TEMPORAL_NEXT from edge_types while four of its 25 patterns mint exactly that type (`follows`, `comes after`, `is after`, `succeeds`). That last one has a consequence outside the manifest, so the exclusion entry now carries it: EXCLUDED_WRITERS said temporal_spine "writes TEMPORAL_NEXT only", which reads as the spine accounting for the whole TEMPORAL_NEXT population. It does not -- triple_extractor is a second, prose-driven producer the #1336 spine recompute does not cover. Also records two limits the manifest was silent on. `--axes-budget` (default 24) caps the anchor tuple and so how many RELATES_TO edges each persisted phantom writes -- a bigger lever than several pinned constants, but a signature default rather than a module constant, out of reach of the (module, name) scheme for the same reason the inline weights are. And belief ARRIVAL ORDER is a third input beside the belief set and these thresholds: the per-belief cap is spent on whichever pairs arrived first, so full-store and incremental runs can disagree on identical beliefs, and re-deriving edges from beliefs plus this manifest gives a false mismatch on any incrementally built store -- which is every real one.
The entry read "published as `93.68%` (39,335 / 41,984) and is now `94.86%`", but 39,335 / 41,984 is 93.69%. The 93.68% belongs to #1336's measurement, 39,280 / 41,929 — a snapshot 55 shipped edges smaller — which the unchanged So the pair presented as a pure denominator correction spanned two store states, and about 0.01pp of the 1.18pp move was snapshot drift rather than the correction. The module's own docstring already said 93.69%, so the branch shipped both answers. Both figures now come from one `spine_divergence()` call on one store, and the The design memo's "98.70% structural ceiling" is corrected the same way: it is defined by the fan-in misses this change removes from the denominator, so it is not the ceiling for 94.86% and the memo now says so rather than inviting the subtraction it forbids two sentences earlier.
Refs #1283 — AC2's recompute half, on the operator-funded scope. Does not close the issue: the writer change (item 3 of the corrected ruling) and AC4's non-increasing fan-in assertion are follow-ups.
Stacked on #1331, which lands
benchmarks/ingest_log_ulid_clusters.py— referenced by the CHANGELOG entry and by the module docstring as the evidence for the synth rule. No code dependency; merge order doesn't matter for correctness, only for the reference.This reports a gap, not drift, and the command says so
The ratified contract keys the recompute on
(created_at, ingest_log ULID). The shipped writer orders by(created_at, rowid)— androwidis exactly the problem: it is implicit andVACUUMmay renumber it, which is why the contract picked the log's ULID over anything read off the belief table.The writer has not moved yet, so a zero divergence here would mean the recompute had been fitted to the defect. That's why nothing in the tests asserts zero on a real store, and why
aelf spine verifyexits 0 regardless and prints the distinction — "93.68%" on its own reads as decay.Those reproduce the issue's gate measurement and its correction to the digit, from a clean-room implementation — 93.68%, 2,100 / 546 / 3. Three sessions have now landed on the same decomposition by different routes.
Three rules, none of them a heuristic
**Synth rows excluded by
source_kind = 'legacy_unknown'.'** A stated durable column, exact in both directions on this store. The #263 synthesis minted 20,852 rows inside a 201 ms window whose order isbeliefs.rowid` relabelled; honouring their ULIDs would launder rowid order into the key the contract calls durable.Two heuristic detectors were measured first and both failed loudly, which is why this one keys on a column:
tsby >1 daysource_kind = 'legacy_unknown'Earliest qualifying log row wins. Later rows are corroborations; only the first records insertion.
No-log beliefs sort last within their
created_atgroup, then by id — a forward convention, explicitly not a recovery. 2,426 such beliefs carry only 433 distinct timestamps, so 94% sit inside a tie where the only other durable column is a content-addressed id. Three placement rules were measured with one link of spread between them. That ordering is unreconstructible, and the module says so in those words rather than leaving it as a caveat. The convention buys determinism, not correctness about the past.Why three buckets and never one percentage
Only one of the three is a defect anyone can fix. A single number lets the unreconstructible bucket mask a real key disagreement — which is precisely how the first reading of this data concluded that AC2a was recoverable.
missing_otheris the bucket a wrong key moves, and it is 3.Verification
created_at, taking the latest log row instead of the earliest, chaining across session boundaries.test_divergence_is_zero_when_the_writer_agreeskeeps the bucket tests honest — a recompute producing nothing would file every shipped edge under some bucket and look like correct attribution. The empty-store case pins that zero edges is full reproduction, not 0%, so a fresh store doesn't read as broken.Follow-ups this deliberately does not do
session_predecessor_idoff(created_at, rowid)onto the ratified key. Until that lands, AC4's divergence test measures the gap rather than drift — the ruling is explicit that funding the recompute without it buys a test red by construction.Summary by Sourcery
Add a read-only command to recompute the temporal spine from the ingest log, report divergence by cause, and document the gap in the changelog, backed by tests for recomputation and bucketed divergence attribution.
New Features:
aelf spine verifyto recomputeTEMPORAL_NEXTfrom the ingest log and report divergence by bucket on a live store.Enhancements:
spine_recomputemodule.Documentation:
Tests: