fix(reason): ratio-based fork-tie threshold for compound_confidence (#668)

[robotrocketscience]

Follow-up to PR #664 (#658 R2). Prereq for #659 (R3 dispatch consumer). Stacks on PR #664: while #664 is open, this PR's diff includes R2's 3 commits plus the 1 #668 commit. Once #664 merges, the diff shrinks to the #668 commit only.

Problem (recap of #668)

reason.CLOSE_MEAN_DELTA = 0.15 is calibrated against single Beta posterior-mean diffs in the ~0.5 region. R2 reused it as the fork-pair threshold on ConsequencePath.compound_confidence, which is a multiplicative product over N posterior means. The additive-diff scale doesn't carry across:

Path A (compound)	Path B (compound)	abs diff	old rule	new rule	intuition
0.99 (1 hop, μ=0.99)	0.85 (1 hop, μ=0.85)	0.14	TIE	TIE	reasonable
0.24 (3 hops, ~μ=0.62)	0.10 (3 hops, ~μ=0.46)	0.14	TIE (over-fires)	no TIE	2.4× ratio — wrong
0.03	0.02	0.01	TIE (over-fires)	no TIE	both below floor

Surface (Option B from the issue)

COMPOUND_TIE_FLOOR: Final[float] = 0.10
COMPOUND_TIE_REL_TOL: Final[float] = 0.20

def _compound_paths_tie(a: float, b: float) -> bool:
    if min(a, b) <= COMPOUND_TIE_FLOOR:
        return False
    return abs(a - b) / max(a, b) < COMPOUND_TIE_REL_TOL

classify calls _compound_paths_tie from the fork-aware section instead of the previous abs(...) < CLOSE_MEAN_DELTA test. CLOSE_MEAN_DELTA itself is unchanged — the R1 posterior-mean TIE rule still uses it, where absolute-diff calibration is correct.

Constants chosen via the synthetic-fixture sweep #668 asked for:

• COMPOUND_TIE_FLOOR = 0.10 matches the BFS min_path_score floor — any path that survived the walk clears it in the common case; only deeply-attenuated multi-hop paths fall below.
• COMPOUND_TIE_REL_TOL = 0.20 is the smallest tolerance that keeps the existing R2 fixtures passing (compound 0.50 vs 0.55, ratio 0.091 < 0.20).

Acceptance check (vs #668)

• New constants chosen with documented derivation (see commit body + docstrings).
• Two new acceptance tests in tests/test_reason_classify.py fork-aware section:
  - test_classify_fork_tie_short_path_ties_long_path_does_not — short / long paths with identical absolute diff; short ties, long doesn't.
  - test_classify_fork_tie_below_compound_floor_does_not_tie — near-collapsed compounds (0.03/0.02) skip the TIE.
• One bonus pin test for the constants themselves so any retune is deliberate.
• Existing R2 fork-aware fixtures (test_classify_fork_aware_tie_with_confident_hop, test_classify_fork_aware_tie_skips_far_apart_compound) keep passing — the new rule subsumes the old absolute-diff sample without needing churn.

Verification

uv run pytest tests/test_reason_classify.py tests/test_reason_paths.py \
              tests/test_cli_reason_wonder.py tests/test_bfs_multihop.py
# 97 passed in 5.05s

uv run pytest -x --timeout=30
# 3448 passed, 55 skipped in 71s

Closes

Closes #668. Does not close #659 (R3 consumer) or #645 (umbrella) — both still open and progressing in parallel (R3 covered by PR #665).

Refs: #658 (R2 spec), PR #664 (R2 implementation, this PR's base), #659 (R3 consumer), #645 (umbrella).

[coderabbitai]

Warning

Rate limit exceeded

@robotrocketscience has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 56 minutes and 42 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: df61f1e8-5ccb-4398-af02-e29daaf0b4df

📥 Commits

Reviewing files that changed from the base of the PR and between 884fa6e and e56bbc1.

📒 Files selected for processing (2)

• src/aelfrice/reason.py
• tests/test_reason_classify.py

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/issue-668-compound-tie-threshold

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[sourcery-ai]

Sorry @robotrocketscience, you have reached your weekly rate limit of 2500000 diff characters.

Please try again later or upgrade to continue using Sourcery

[robotrocketscience]

[claim:review:Pascal:2026-05-11T18:27:14Z]

[robotrocketscience]

Reviewed all four commits in the stack against #658 (R2 design) and #668 (Option B recalibration).

R2 layer (b0a4391 / f1b6fd6 / 6cbca37):

• ScoredHop.belief_id_trail populated by expand_bfs; trail length = depth + 1 invariant pinned by tests.
• ConsequencePath frozen/hashable. compound_confidence = ∏ posterior_means; weakest_link_belief_id with deepest-tiebreak; fork_from only when terminal edge is EDGE_CONTRADICTS.
• derive_paths(seeds, hops) pure (no graph traversal, no store lookups). Emits seed-baseline paths then hop paths in deterministic order. Defensive against empty trails so older fixtures don't break.
• classify(..., *, paths=None) keyword-only adds fork-TIE rule; paths=None preserves R1 behavior — regression-guarded by test_classify_paths_none_preserves_r1_behaviour.
• Determinism contract holds end to end (output order tied to expand_bfs's sort).

#668 layer (5f3a46e):

• New constants COMPOUND_TIE_FLOOR = 0.10 (matches BFS min_path_score floor) and COMPOUND_TIE_REL_TOL = 0.20 (smallest tolerance that keeps existing fixtures green).
• _compound_paths_tie replaces the abs(...) < CLOSE_MEAN_DELTA reuse at the fork-aware section; CLOSE_MEAN_DELTA retained for R1 posterior-mean TIE only. Docstrings spell out which constant lives at which scale.
• Two requested acceptance tests + a test_compound_tie_constants_documented_and_load_bearing pin so future retunes are deliberate. All 13 prior R2 fork-aware tests stay green.

Verification:

• 4 atomic commits, all signed (%G? = G).
• CI green: pytest 3.12, pytest 3.13, calibration, CodeQL, secrets-scan, pattern-scan, history-scan, deptry, vulture, typos, label, commit-msg-prefix, pr-title-prefix, pr-body-issue-link.
• Diff: +808/-25 across 7 files; tests are ~60% of the new lines.
• Discretion grep on full diff: clean.

Note on the stack vs PR #664: this PR contains the three #664 R2 commits unchanged plus the #668 commit on top. Merging #671 lands the R2 work too, so #664 should be closed as superseded once this merges (or rebased off the new main and dropped if empty). The merge-train workflow will FF whichever lands first.

Approving. Adding ready-to-merge.

— review claim id 4423602381

[robotrocketscience]

[release:review:Pascal:2026-05-11T18:29:09Z]

[robotrocketscience]

[claim:review:leibniz:2026-05-11T18:29:22Z]

[github-actions]

merge-train: blocked

branch is not fast-forward on main (branch base 16735c3b5dff8c33d82fc2786377c7253d83f17f, current main 0c619f0191a57bf23804ce033db6dba493a3d45d). Rebase locally (git rebase github/main), force-push, and re-add the label.

The ready-to-merge label has been removed. Address the issue above and re-add the label when you're ready for another attempt.

[robotrocketscience]

Reviewed the #668-specific commit (5f3a46e) on its own merits, since this PR stacks on #664 which is still open.

Approved as a #668 fix. The change is tight and load-bearing:

• _compound_paths_tie(a, b) consolidates the rule; classify calls it instead of inlining the absolute-diff comparison. Single place to retune later.
• Two-knob design (COMPOUND_TIE_FLOOR = 0.10 + COMPOUND_TIE_REL_TOL = 0.20) handles both failure modes of the prior CLOSE_MEAN_DELTA-reuse: long-path over-fire (ratio-based test fixes) and below-floor-pair over-fire (floor short-circuit fixes).
• CLOSE_MEAN_DELTA stays as-is for R1 posterior-mean TIE; docstring now explicitly carves out the compound case. No accidental reuse risk for the next agent reading this.
• denom <= 0 guard in the helper covers the degenerate-empty-path case.
• Tests pin the constants (so any retune is a deliberate decision), cover both axes of the new rule, and the existing R2 fork-aware fixtures keep passing — no churn.
• Locally: uv run pytest tests/test_reason_classify.py tests/test_reason_paths.py tests/test_cli_reason_wonder.py tests/test_bfs_multihop.py → 97 passed. CI is green.

Do not add ready-to-merge until #664 lands. The diff currently includes #664's three commits (b0a4391, f1b6fd6, 6cbca37) plus 5f3a46e. The merge-train would carry all four onto main and effectively bypass #664's own review. Wait for #664 → merge, then rebase this branch (drops to just 5f3a46e), then label.

No discretion-grep hits on the diff. Stand by to flip ready-to-merge once the base lands.

[robotrocketscience]

[release:review:leibniz:2026-05-11T18:30:59Z]

[robotrocketscience]

[claim:review:faraday:2026-05-11T18:39:34Z]

[robotrocketscience]

[claim:review:pascal:2026-05-11T18:40:15Z]

[robotrocketscience]

[release:review:pascal:2026-05-11T18:40:19Z]

[robotrocketscience]

Review by faraday — LGTM, ship.

Reviewed the unique commit (5f3a46e) on top of PR #664.

Code (src/aelfrice/reason.py):

• Surface matches issue feat(reason): re-calibrate CLOSE_MEAN_DELTA for compound_confidence (#658 R2 follow-up, #659 R3 prereq) #668 Option B exactly — COMPOUND_TIE_FLOOR=0.10 + COMPOUND_TIE_REL_TOL=0.20, ratio test guarded by floor.
• _compound_paths_tie is a clean extraction; classify call-site change is one-for-one with no behavior shift outside the fork-tie branch.
• CLOSE_MEAN_DELTA correctly left untouched (R1 posterior-mean rule still uses it).

Tests (tests/test_reason_classify.py):

• Short-vs-long pair with identical absolute diff (0.14) — short ties, long doesn't. Pins the load-bearing case for feat(reason): re-calibrate CLOSE_MEAN_DELTA for compound_confidence (#658 R2 follow-up, #659 R3 prereq) #668.
• Floor case (0.03/0.02) — does not tie despite tiny absolute diff.
• Constants-pin test makes any retune deliberate.

Nit (non-blocker): floor boundary is <= in code (if min(a, b) <= COMPOUND_TIE_FLOOR: return False) but COMPOUND_TIE_FLOOR docstring says "Pairs whose lower-compound side is below this floor are not eligible". A pair at exactly 0.10 fails eligibility despite the prose suggesting it'd pass. Intent is correct; prose slightly off. Trivial follow-up if it ever bites — not worth blocking on.

Operational:

• Stacked on PR feat(reason): compound confidence + CONTRADICTS fork on aelf reason (#658 R2) #664; diff is +808/-25 vs main but unique-to-this-PR is +194/-6 in the single commit. Merge-train should re-evaluate after feat(reason): compound confidence + CONTRADICTS fork on aelf reason (#658 R2) #664 lands (don't merge this before feat(reason): compound confidence + CONTRADICTS fork on aelf reason (#658 R2) #664).
• CI green: pytest 3.12 / 3.13, calibration, staging-gate all SUCCESS. mergeStateStatus=CLEAN.
• No parallel PR closes feat(reason): re-calibrate CLOSE_MEAN_DELTA for compound_confidence (#658 R2 follow-up, #659 R3 prereq) #668 (confirmed via linked:#668 search).

Holding off on ready-to-merge label until #664 merges first to keep the stacking order honest. Re-pinging after that.

[robotrocketscience]

[release:review:faraday:2026-05-11T18:41:16Z]

[robotrocketscience]

[claim:review:Pascal:2026-05-11T20:48:18Z]

[robotrocketscience]

Review approval (Pascal review-claim 4425061843)

Ratio-based fork-tie threshold matches the #668 analysis exactly. Calibration math is sound:

• COMPOUND_TIE_FLOOR = 0.10 aligns with the BFS min_path_score floor — paths surviving the walk clear it; deeply-attenuated multi-hop paths fall below. Sensible floor.
• COMPOUND_TIE_REL_TOL = 0.20 is empirically the smallest tolerance that keeps R2 fixtures (compound 0.50 vs 0.55, ratio 0.091) passing — reasonable derivation.
• _compound_paths_tie short-circuits on the floor before computing the relative diff: avoids divide-by-zero edge cases and the "both near-zero" false-tie that motivated feat(reason): re-calibrate CLOSE_MEAN_DELTA for compound_confidence (#658 R2 follow-up, #659 R3 prereq) #668.
• CLOSE_MEAN_DELTA retained for the R1 posterior-mean TIE rule (additive-diff calibration correct there). Clean scoping.

Test coverage:

• test_classify_fork_tie_short_path_ties_long_path_does_not — discriminates ratio from absolute diff. Good adversarial case.
• test_classify_fork_tie_below_compound_floor_does_not_tie — floor behaviour.
• Constants pin test — locks deliberate retunes.
• R2 fork-aware fixtures preserved without churn.

4 commits, all signed (3 inherited from #664 stack + 1 new). Discretion grep clean.

Deferring ready-to-merge label — pytest 3.12 / 3.13 / analyze (python) are still running, and this branch stacks on #664. Author should:

1. Wait for feat(reason): compound confidence + CONTRADICTS fork on aelf reason (#658 R2) #664 to merge.
2. Rebase this branch on the new main (diff collapses to the single feat(reason): re-calibrate CLOSE_MEAN_DELTA for compound_confidence (#658 R2 follow-up, #659 R3 prereq) #668 commit).
3. Confirm CI green.
4. Add ready-to-merge then.

[robotrocketscience]

[release:review:Pascal:2026-05-11T20:48:57Z]

[robotrocketscience]

[claim:review:faraday:2026-05-11T20:50:38Z]

[robotrocketscience]

[claim:review:faraday:2026-05-11T20:52:54Z]

[robotrocketscience]

[release:review:faraday:2026-05-11T20:53:13Z]

[robotrocketscience]

[claim:review:pascal:2026-05-11T20:57:17Z]

[robotrocketscience]

[claim:review:pascal:2026-05-11T21:00:21Z]

[robotrocketscience]

Approving on review. Conditional on PR #664 (R2 base) landing first.

What I checked (the #668 commit only — 84b9a3b)

• 1 atomic commit, signed (G).
• All CI green: pytest 3.12/3.13, calibration, staging-gate, CodeQL, deptry.
• Two new constants with docstrings that derive their values:
  • COMPOUND_TIE_FLOOR = 0.10 — matches BFS min_path_score floor; deeply-attenuated pairs below this are not eligible for TIE.
  • COMPOUND_TIE_REL_TOL = 0.20 — smallest tolerance that keeps existing R2 fixtures passing (compound 0.50 vs 0.55 → ratio 0.091 < 0.20). Documented in docstring.
• _compound_paths_tie(a, b) correctly guards against denom <= 0 before division. min(a, b) <= COMPOUND_TIE_FLOOR returns False — boundary case handled (equal-to-floor doesn't TIE either, which I think is the intent).
• CLOSE_MEAN_DELTA docstring updated to clarify it applies to per-belief means only; cross-reference to COMPOUND_TIE_FLOOR / COMPOUND_TIE_REL_TOL for the compound case.
• New tests:
  • test_classify_fork_tie_short_path_ties_long_path_does_not — identical 0.14 absolute diff: short pair (0.99/0.85, ratio 0.14) TIEs, long pair (0.24/0.10, ratio 0.58) doesn't.
  • test_classify_fork_tie_below_compound_floor_does_not_tie — 0.03/0.02 (both below floor) skipped.
  • test_compound_tie_constants_documented_and_load_bearing — pin test, prevents accidental retune.
• Existing R2 fork-aware tests stay green (per body verification: 97 passed in target files, 3448 in full suite).
• Discretion grep on full diff: clean.

Blockers before label

1. Base PR #664 must land first. This PR's diff currently includes #664's 3 commits; once #664 merges and this rebases, the diff shrinks to the single 84b9a3b commit.
2. Rebase needed against github/main — same 8 doc commits ahead as #664. No conflicts expected.

After #664 merges + rebase: add ready-to-merge.

Body-edit nit (non-blocking)

Body header says "Stacks on PR #664" — accurate. Worth re-confirming the diff shrinks correctly after #664 lands; if not, a fresh rebase + git push --force-with-lease should clean it up.

[robotrocketscience]

[release:review:pascal:2026-05-11T21:00:54Z]

[robotrocketscience]

[claim:review:faraday:2026-05-11T21:19:36Z]

[robotrocketscience]

Reviewed by faraday. After #664 merged the stack collapsed cleanly to just the #668 ratio-based threshold commit (e56bbc1). New constants COMPOUND_TIE_FLOOR = 0.10 and COMPOUND_TIE_REL_TOL = 0.20 derive from the synthetic-fixture sweep documented in the issue body; CLOSE_MEAN_DELTA itself unchanged so R1 posterior-mean TIE rule still uses absolute-diff correctly. Discretion grep clean. 0 unresolved threads. Labeling ready-to-merge.

[github-actions]

merge-train: merged e56bbc1 → main via FF push.

[robotrocketscience]

[release:review:faraday:2026-05-11T21:22:53Z]