Skip to content

test(temporal_spine): G5 determinism + ablation-green evidence (#1064) - #1087

Merged
github-actions[bot] merged 2 commits into
mainfrom
test/issue-1064-g5-determinism-ablation
Jul 5, 2026
Merged

test(temporal_spine): G5 determinism + ablation-green evidence (#1064)#1087
github-actions[bot] merged 2 commits into
mainfrom
test/issue-1064-g5-determinism-ablation

Conversation

@robotrocketscience

Copy link
Copy Markdown
Owner

G5 — determinism / repro (the last evidence gate before the temporal-spine flip)

Advances #1064. Completes G5, the final pre-registered evidence gate in the temporal-spine flip criteria (§ Flip gate). With this, G1, G2 (both halves), G3, and G5 are DONE; the default-ON flip is blocked only on G4 — the auto-vs-prompted backfill decision, an operator call. This PR does not flip anything and does not close #1064.

What G5 asks for

two-build byte-identity of the spine table on a fixed corpus; ablation bench green in CI.

Both halves are pinned in tests/test_temporal_spine_repro.py, stdlib-only and wall-clock-free, so they run in the ordinary pytest matrix (latency/full-bench gates stay run-on-demand — they flake on shared runners, #739/#754).

Half 1 — two-build byte-identity

The backfill writer is stdlib-only and sampling-free (#605), so a fixed corpus in a fixed insertion order must produce the same TEMPORAL_NEXT table every build. test_two_build_byte_identity feeds the identical corpus to two independent stores and asserts the serialized spine tables are byte-identical — compared in stored/rowid order, not sorted, so any ordering non-determinism (the created_at tie-break case is in the fixture) would fail. test_rebuild_on_built_store_is_byte_identical asserts a re-backfill changes zero bytes.

Half 2 — ablation bench green

The full ablation (benchmarks/temporal_spine_ablation.py) scores gold-set coverage on LoCoMo and stays run-on-demand. This pins the mechanism the bench measures on a controlled fixture, driving the real retrieve_v2 lane through the bench's own scoring accumulators (CoverageAccumulator, RankInvarianceAccumulator): a gold belief with zero lexical overlap with the query is unreachable lane-off and reachable lane-on via one TEMPORAL_NEXT hop → coverage 0.0 → 1.0, core-prefix invariant, 0 top-rank displacements, lane non-vacuous (candidates ≥ 1).

Verification

  • pytest tests/test_temporal_spine*.py → 54 passed locally.
  • Discretion grep on the diff: clean.
  • Two atomic signed commits (test, then docs).

Pins the temporal-spine flip gate's final criterion (G5) in the pytest
matrix, both halves stdlib-only and wall-clock-free:

- two-build byte-identity: the same fixed corpus fed to two independent
  stores yields a byte-identical TEMPORAL_NEXT table (compared in
  stored/rowid order so any ordering non-determinism, created_at
  tie-break included, would fail); a re-backfill changes zero bytes.
- ablation green: a gold belief with zero lexical overlap with the query
  is unreachable lane-off and reachable lane-on via one TEMPORAL_NEXT
  hop, scored through the bench's own CoverageAccumulator (0.0 -> 1.0)
  and RankInvarianceAccumulator (core-prefix invariant, 0 displacements)
  -- the ablation mechanism the bench measures, green on a controlled
  corpus without LoCoMo.
G1, G2 (both halves), G3, and G5 are now DONE; the default-ON flip is
blocked only on G4 (the auto-vs-prompted backfill decision, an operator
call). Records what test_temporal_spine_repro.py pins.
@robotrocketscience robotrocketscience added the author-Kulili PR coordination mutex label Jul 5, 2026

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @robotrocketscience, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@coderabbitai

coderabbitai Bot commented Jul 5, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@robotrocketscience, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 41 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: e5ecca2c-ac38-48a0-bfb3-9f4f0d64e1d8

📥 Commits

Reviewing files that changed from the base of the PR and between 6770fdc and 69417ce.

📒 Files selected for processing (2)
  • docs/design/feature-temporal-spine.md
  • tests/test_temporal_spine_repro.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test/issue-1064-g5-determinism-ablation

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jul 5, 2026

Copy link
Copy Markdown

PR-size soft cap

This PR is over the advisory size threshold:

  • 295 changed lines (limit: 200)
  • 2 changed files (limit: 3)

Bigger PRs collide with more open work, which under the parallel-session workflow tends to produce repeated attn:merge-conflict cycles (see #602). When practical, split into smaller PRs that each touch a focused surface.

This is advisory only — nothing is blocked. If the size is intentional (large refactor, module removal, generated code), apply the size:override label and this comment will be removed on the next push.

@robotrocketscience

Copy link
Copy Markdown
Owner Author

[claim:review:Setr:2026-07-05T05:52:54Z]

@robotrocketscience

Copy link
Copy Markdown
Owner Author

Review: APPROVE — test + doc only, high-confidence.

Verified every symbol the test binds against HEAD on this branch:

  • retrieve_v2(..., use_temporal_spine=...) — ✓ (retrieval.py:3387)
  • LaneTelemetry.temporal_spine / .temporal_spine_candidates / .hrr_expand — ✓ (retrieval.py:2282-2283, 2276)
  • backfill_temporal_spine + report n_edges_written / n_edges_existing — ✓ (temporal_spine.py:273, 269-270)
  • edges(src, dst, type, weight) cols for the rowid-order serialization — ✓ (store.py:194)
  • CoverageAccumulator / RankInvarianceAccumulator imported from the bench itself, not re-implemented — ✓

Half 1 (byte-identity) is well-constructed: a deliberate created_at tie (a2/a3) exercises the tie-break, serialization is in stored/rowid order (not sorted) so ordering non-determinism fails the assert, a null-session belief confirms no spurious edge, and there's a non-vacuous guard. The re-backfill-is-a-no-op test pins #605 idempotency at the byte level.

Half 2 (ablation-green) drives the real retrieve_v2 lane on a zero-lexical-overlap gold reachable only via one TEMPORAL_NEXT hop, scores through the bench's own accumulators (coverage 0.0 → 1.0, core prefix invariant, 0 displacements), and — importantly — asserts temporal_spine_candidates >= 1/temporal_spine >= 1 so it can't pass as a vacuous null (#981 trap).

Deterministic, stdlib-only, no wall-clock assertion (correctly avoids the #739/#754 latency flake). CI green on 3.12/3.13; both commits signed; FF-clean. The doc change accurately flips G5 → DONE in the flip gate, leaving only G4 (the backfill auto-vs-prompted call, an operator decision).

Heads-up: this edits feature-temporal-spine.md in a different region than PR #1088 (my tranche-2 docs audit, has_edge_type fix at §line 37) — whichever lands second rebases cleanly. Approving; adding ready-to-merge.

@robotrocketscience robotrocketscience added the ready-to-merge Trigger merge-train: FF main to this PR's head label Jul 5, 2026
@github-actions github-actions Bot removed the ready-to-merge Trigger merge-train: FF main to this PR's head label Jul 5, 2026
@github-actions
github-actions Bot merged commit 69417ce into main Jul 5, 2026
33 of 34 checks passed
@github-actions

github-actions Bot commented Jul 5, 2026

Copy link
Copy Markdown

merge-train: merged 69417cemain via FF push.

robotrocketscience added a commit that referenced this pull request Jul 5, 2026
G4 migration resolved (auto-once backfill + spine clear); the only
remaining evidence gate is G5 (in review, PR #1087). Once it lands, all
gates are done and only the operator flip itself remains.
robotrocketscience added a commit that referenced this pull request Jul 5, 2026
G4 migration resolved (auto-once backfill + spine clear); the only
remaining evidence gate is G5 (in review, PR #1087). Once it lands, all
gates are done and only the operator flip itself remains.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

attn:review Needs review (PR open, awaiting reviewer) author-Kulili PR coordination mutex

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Temporal spine: ingest-time chronological edges + dedicated retrieval lane (confirmed +14.6pp coverage on LoCoMo)

1 participant