v0.4.11 — latent drift fix (range-diff sweep + distinct counters)

[jinhongkuan]

Summary

Fixes a class of "invisible drift" where decisions silently went stale because `link_commit` only swept files in HEAD's own diff. After a gap of N commits without a bicameral invocation, drift introduced by intermediate commits stayed hidden until someone happened to re-edit the same files.

Two changes:

1. Range-diff sweep. `ingest_commit` now diffs `last_synced..HEAD` via `git diff --name-only` and sweeps every file in the range. New `sweep_scope` field on `LinkCommitResponse` reports `head_only` (first sync, or fallback) vs `range_diff` (default after first sync) vs `range_truncated` (range exceeded 200-file cap). Falls back to head-only when the base SHA is unreachable. New `range_size` field reports how many files were swept.

2. Distinct intent counters. `decisions_drifted` and `decisions_reflected` now count distinct intent_ids that flipped, not (region, intent) pairs. Witnessed on the Accountable demo where one Google Calendar decision flipped 4 regions and the old counter reported `decisions_drifted=4` while only 1 distinct intent was actually drifted.

Both changes were teed up in conversation after the Phase 2 drift demo on Accountable surfaced the per-region count discrepancy and the latent-drift mental model.

Test plan

• 11 cases in `tests/test_v0411_latent_drift.py` covering:
  • `get_changed_files_in_range` helper (in-range, empty range, unreachable base)
  • `sweep_scope` semantics (first sync = head_only, second sync after gap = range_diff, fast-path = head_only with size 0, unreachable cursor = head_only fallback)
  • Distinct intent counter dedup (one decision with N drifted regions = 1 in counter)
• Full v0.4.11 regression: 163 passed in 14s
• Manual: run on Accountable to verify latent drift surfaces correctly

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Enhanced commit synchronization now scans the full range from last sync to current HEAD, surfacing drift previously undetected during gaps
  • Added response metadata reporting scan scope and file count swept

• Bug Fixes

  • Fixed overcounting of drift and reflection metrics by deduplicating based on intent rather than individual region changes

• Tests

  • Added comprehensive regression test suite validating latent drift detection and range-diff functionality

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 3129ad88-9be3-4cac-a7eb-e79f875d8274

📥 Commits

Reviewing files that changed from the base of the PR and between 96abd45 and fa1804d.

📒 Files selected for processing (7)

• CHANGELOG.md
• contracts.py
• handlers/link_commit.py
• ledger/adapter.py
• ledger/status.py
• pyproject.toml
• tests/test_v0411_latent_drift.py

---

📝 Walkthrough

Walkthrough

The pull request extends the git synchronization system to sweep files across the full last_synced..HEAD commit range instead of only the current HEAD commit, surfacing latent drift during sync-cursor lag periods. It introduces new metadata fields (sweep_scope, range_size) to report sweep provenance, adds a range-diff utility function, and deduplicates decision counters by distinct intent identifiers rather than per-region flips.

Changes

Cohort / File(s)	Summary
Configuration & Package Metadata pyproject.toml	Version bump from 0.4.10 to 0.4.11.
Type Definitions contracts.py	Updated LinkCommitResponse with new fields sweep_scope (literal enum) and range_size (int); redefined decisions_reflected and decisions_drifted semantics from per-flip counts to distinct intent-ID flips.
Git Utilities ledger/status.py	Added get_changed_files_in_range(base_sha, head_sha, repo_path) function that runs git diff --name-only base..head and returns None sentinel on failure to enable safe fallback to HEAD-only scanning.
Sync Implementation ledger/adapter.py	Refactored ingest_commit() to use cursor-based range sweeps by default; added range capping (200 file limit), fallback to HEAD-only on range failure, decision dedup via intent-ID sets, and sweep-scope reporting (head_only, range_diff, range_truncated).
API Handler handlers/link_commit.py	Populated sweep_scope and range_size fields on returned LinkCommitResponse from adapter result payload.
Documentation & Tests CHANGELOG.md, tests/test_v0411_latent_drift.py	Added changelog entry and comprehensive regression test suite validating range-diff behavior, latent-drift discovery, fallback handling, and distinct-intent deduplication logic across multiple scenarios.

Sequence Diagram(s)

sequenceDiagram
    actor Client
    participant Handler as handle_link_commit
    participant Adapter as SurrealDBLedgerAdapter
    participant GitStatus as ledger/status
    participant Git as git (subprocess)
    
    Client->>Handler: link_commit(commit_hash)
    Handler->>Adapter: ingest_commit(commit_hash)
    
    Adapter->>Adapter: read last_synced_commit from sync state
    
    alt Range cursor exists & not empty
        Adapter->>GitStatus: get_changed_files_in_range(last_synced, commit_hash)
        GitStatus->>Git: git diff --name-only last_synced..commit_hash
        
        alt git diff succeeds
            Git-->>GitStatus: file list
            GitStatus->>GitStatus: check file count vs 200 limit
            
            alt file count <= 200
                GitStatus-->>Adapter: file list (sweep_scope=range_diff)
            else file count > 200
                GitStatus-->>Adapter: truncated list (sweep_scope=range_truncated)
            end
        else git diff fails/times out
            Git-->>GitStatus: error
            GitStatus-->>Adapter: None (sentinel)
            Adapter->>GitStatus: get_changed_files(commit_hash) [fallback]
            GitStatus->>Git: git show --name-only commit_hash
            Git-->>GitStatus: file list
            GitStatus-->>Adapter: file list (sweep_scope=head_only)
        end
    else First sync or empty cursor
        Adapter->>GitStatus: get_changed_files(commit_hash)
        GitStatus->>Git: git show --name-only commit_hash
        Git-->>GitStatus: file list
        GitStatus-->>Adapter: file list (sweep_scope=head_only)
    end
    
    Adapter->>Adapter: process files & compute decision flips (by intent_id dedup)
    Adapter->>Adapter: construct LinkCommitResponse with sweep_scope & range_size
    Adapter-->>Handler: response payload
    
    Handler->>Handler: populate sweep_scope & range_size on LinkCommitResponse
    Handler-->>Client: LinkCommitResponse

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Poem

🐰 A sweep across the git-time span,
No drift shall hide—we have a plan!
From last_synced to HEAD we run,
Intent dedup when flips are done,
Latent shadows now bring light,
Version bumped to point-four-one-one—all right! 🌙✨

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch chore/bump-v0.4.11

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}