feat(exploration): seeded uniform exploration draw (#1176)

[robotrocketscience]

Refs #1176 (proposal 5). Library half only — no hot-path edit, no schema change.

Why

A belief that starts underranked is never retrieved, therefore never referenced, therefore never acquires evidence, therefore stays underranked. Nothing in the retrieval path breaks that loop. I measured how much of the store is inside it, on a fresh .backup of the live 44,586-belief store:

	count	share of active
active, unlocked, zero feedback_history, zero injection_events	37,489	84.1%
distinct beliefs with any feedback_history row	7,450	16.7%
distinct beliefs ever injected into a context	1,352	3.0%

92,685 feedback rows and 21,328 injection events land on 7,450 and 1,352 beliefs. The feedback is concentrated on the beliefs that were already winning, which is the loop stated as a measurement.

The draw is uniform — a correction to the proposal, not a simplification

The proposal specifies Efraimidis–Spirakis A-Res weighted reservoir sampling keyed on scoring.uncertainty_score. That weighting is invalid as written and inert after repair. Both were measured before writing any code.

Invalid. uncertainty_score is Beta differential entropy, so on [0, 1] it is ≤ 0, and exactly 0 for Beta(1, 1). A-Res requires strictly positive weights:

a=1.0 b=1.0  Beta(1,1)             H= 0.00000  1/H -> ZeroDivisionError
a=0.6 b=1.0  agent_inferred prior  H=-0.15584  1/H=-6.4168  key(u=.5)=85.44
a=3.0 b=1.0  fact prior            H=-0.43195  1/H=-2.3151  key(u=.5)=4.976
a=9.0 b=0.5  locked-ish            H=-2.13622  1/H=-0.4681  key(u=.5)=1.383

key = u ** (1 / w) divides by zero on the 39 Beta(1, 1) beliefs in the pool, and every other key lands > 1 — not a reservoir. This is the same sign trap already on record for the log-domain rerank score.

Inert. After either natural sign repair, the weighted draw is indistinguishable from uniform on this corpus:

weight repair	TV distance from uniform
affine H − H_min	0.0586
exp(H)	0.0890

Two entropy values cover 88.1% of the pool (four cover 98.2%), so the dominant classes draw at 0.99× and 1.15× their pool share. The proposal's own kill criterion — "if it is 2-valued, drop A-Res for plain uniform and save ~100 LOC" — is met. If the posterior ever becomes informative (it is currently a two-valued function of type × origin assigned at derive(), not a learned quantity) a weighted key drops back in behind the same seeded stream with no other change.

Determinism

The seed is blake2b over scope_id, the monotonic fire_idx, and the query — logged state only, no clock, no random. random is deliberately unused: its Mersenne state and float conversion are not a stable contract across CPython versions, and replay has to hold across upgrades.

Two details that are easy to get wrong and are pinned rather than asserted:

• Field separation. The three seed inputs are joined with \x1f. Without it, ("a", 11, q) and ("a1", 1, q) both render a11q and seed the same draw. The colliding pair in the test is chosen deliberately — an arbitrary pair like ("ab", 1) vs ("a", 11) still differs and lets the mutation through.
• Rejection sampling. _bounded rejects the biased tail rather than taking word % bound. No black-box test can see this — at a pool size of 40 against a 2⁶⁴ word range the skew is ~1e-18, and the statistical uniformity test passes with naive modulo. So it is pinned directly, with a hand-made word stream where the two paths provably disagree (bound = 3: rejection returns 2, naive returns 0). An exploration mechanism whose justification is "the ranker is systematically skewed" should not introduce its own skew.

splitmix64_stream is pinned to Vigna's published splitmix64.c vectors for seed 0 — the published values, not values captured from this implementation, so the generator is a contract rather than whatever the code happens to do.

Verification

• 25 tests, full suite 6815 passed, 69 skipped, 71 xfailed
• Mutation-tested; two escapes found and fixed. Dropping the \x1f separator and replacing the rejection limit with 1 << 64 both left the suite green on the first pass. Both now fail exactly one test each.
• A third mutation — full Fisher–Yates instead of the partial loop — escaped correctly: it is output-equivalent (the first take swaps consume the same words), only slower. That is the prefix-stability property draw_uniform documents, so the escape confirms the docstring rather than exposing a gap.
• Discretion grep on added lines: clean. CHANGELOG resolved insert-only (0 deleted lines).

Not in this PR

The store-side pool query, the exploration_events ledger, and the retrieval wiring with its flag-off byte-parity gate. Threading an exploration slot into the pack changes the result shape and needs a migration, so it is its own reviewable change — and this module can be read and benched without a hot-path edit. Same split clustering.py and lock_consistency.py shipped under.

Summary by Sourcery

Introduce a deterministic, seeded uniform exploration draw mechanism to surface underranked beliefs without changing retrieval hot paths.

New Features:

• Add an exploration module that derives a deterministic SplitMix64 seed from logged state and provides a uniform draw of candidate beliefs without replacement.
• Expose configurable exploration cadence and slot-count defaults for periodically inserting exploration slots into prompts.

Documentation:

• Document the exploration slot behaviour and motivation in the v4 changelog, including the decision to use uniform rather than weighted reservoir sampling.

Tests:

• Add a dedicated exploration test suite asserting SplitMix64 reproducibility, seed derivation properties, exploration cadence behaviour, and the uniformity and determinism of the draw.

[sourcery-ai]

Sorry @robotrocketscience, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

[coderabbitai]

Warning

Review limit reached

@robotrocketscience, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 9 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: ce1780d9-b7c8-4d59-9ef6-6f9557337934

📥 Commits

Reviewing files that changed from the base of the PR and between b61c711 and e185124.

📒 Files selected for processing (3)

• CHANGELOG/v4.md
• src/aelfrice/exploration.py
• tests/test_exploration_1176.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[sourcery-ai]

Reviewer's Guide

Implements the library-side, deterministic uniform exploration draw mechanism using a SplitMix64 RNG, deterministic seed derivation, and cadence predicate, plus comprehensive tests and changelog entry; no retrieval wiring or schema changes are included.

File-Level Changes

Change	Details	Files
Introduce deterministic SplitMix64-based RNG and seed derivation for exploration draws.	Add splitmix64_stream implementing Vigna’s SplitMix64 with fixed constants and 64-bit masking. Implement derive_seed using blake2b over scope_id, fire_idx, and query joined with a field separator to yield a 64-bit seed. Export RNG-related symbols via all and document determinism and contract in module docstring.	src/aelfrice/exploration.py tests/test_exploration_1176.py
Add exploration cadence predicate and uniform draw without replacement over candidate belief IDs.	Implement should_explore that fires every Kth fire_idx with non-positive cadence disabling exploration instead of raising. Implement draw_uniform that sorts and de-duplicates candidate IDs, draws up to count items uniformly without replacement using partial Fisher–Yates, and preserves draw order. Implement internal _bounded that performs unbiased rejection sampling from SplitMix64 words instead of naive modulo, with error handling for non-positive bounds.	src/aelfrice/exploration.py tests/test_exploration_1176.py
Add comprehensive tests pinning determinism, contracts, and edge cases for exploration RNG and draw.	Verify SplitMix64 against Vigna’s published reference vectors and basic properties (purity, 64-bit range). Test derive_seed reproducibility, field separation (collision guard), and 64-bit range. Test should_explore cadence semantics, including disabling on non-positive cadence. Test draw_uniform properties: determinism, seed sensitivity, candidate-order independence, no replacement, duplicate collapsing, behavior on small/empty pools, prefix stability, slot reachability, and statistical uniformity across many seeds. Directly test _bounded’s rejection behavior and error on non-positive bound using crafted word streams.	tests/test_exploration_1176.py
Document the new exploration mechanism in the changelog.	Add v4 changelog bullet describing seeded uniform exploration draw, motivation, measurement that killed weighted A-Res proposal, determinism contract, and scope (library-only, no wiring yet).	CHANGELOG/v4.md

Possibly linked issues

• [R&D] Consolidation and lifecycle alternates — admission, forgetting, negative evidence #1176: They address the same exploration-slot proposal: the issue designs seeded exploration; the PR ships its deterministic draw library.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[github-actions]

PR-size soft cap

This PR is over the advisory size threshold:

• 420 changed lines (limit: 200)
• 3 changed files (limit: 3)

Bigger PRs collide with more open work, which under the parallel-session workflow tends to produce repeated attn:merge-conflict cycles (see #602). When practical, split into smaller PRs that each touch a focused surface.

This is advisory only — nothing is blocked. If the size is intentional (large refactor, module removal, generated code), apply the size:override label and this comment will be removed on the next push.

[robotrocketscience]

[claim:review:Setr:2026-07-31T18:23:28Z]

[robotrocketscience]

[claim:review:Gylf:2026-07-31T18:23:34Z]

[robotrocketscience]

[release:review:Gylf:2026-07-31T18:23:39Z]

[robotrocketscience]

[claim:review:Garsecg:2026-07-31T18:26:51Z]

[robotrocketscience]

[release:review:Garsecg:2026-07-31T18:26:56Z]

[robotrocketscience]

Review — three docstring-accuracy defects, no correctness defect

The implementation is right and I could not break it. Every number in the body
reproduces exactly. What does not hold up is what three docstrings say the
tests detect — and on a PR whose stated standard is "each one fails if the
specific property is removed, not merely if the code crashes," that is the
claim worth checking.

Verified independently

claim	how checked	result
SplitMix64 vectors are Vigna's, not captured output	transcribed splitmix64.c myself (γ 9e3779b97f4a7c15, mults bf58…/94d0…, shifts 30/27/31) and ran it	all 5 words match _SPLITMIX64_SEED0
A-Res is invalid as specified	called scoring.uncertainty_score	Beta(1,1)→0.0; (0.6,1)→-0.15584; (3,1)→-0.43195; (9,0.5)→-2.13622 — matches the body to 5dp, and 1/0 is real
rejection ≠ naive modulo at bound=3	hand stream	rejection 2, naive 0
no pre-existing exploration surface	git ls-tree/git grep on github/main	none — not a rediscovery
CHANGELOG insert-only	diff	+1/−0
discretion on added lines	grep -E '^\+' then vocab	clean
tests	uv run pytest tests/test_exploration_1176.py	25 passed

1. test_the_draw_is_actually_uniform — first half of the docstring is false

A biased _bounded (naive % bound) or an off-by-one in the
Fisher-Yates bound shows up here as a lopsided histogram.

Mutated limit = (1 << 64) - ((1 << 64) % bound) → limit = 1 << 64:

1 failed, 24 passed
FAILED test_bounded_rejects_the_biased_tail_of_the_word_range

test_the_draw_is_actually_uniform passes under naive modulo. Which is
exactly what the next test's docstring says, correctly and in the same file:

The statistical test above cannot see the modulo bias … word % bound
passes every black-box test in this file. Verified by mutation.

Two adjacent docstrings assert opposite things about the same mutation. The
second is the true one — so the fix is to drop the _bounded clause from the
first, not to change any code.

The off-by-one half is true, and I confirmed it rather than assuming the
whole sentence was wrong: len(pool) - i → max(1, len(pool) - i - 1) fails
test_the_draw_is_actually_uniform and
test_every_pool_member_is_reachable_in_a_later_slot_too, 2 failed / 23 passed.

2. test_widening_the_slot_count_keeps_the_earlier_slots — the stated reason is false, and the PR body already says so

a "shuffle the whole pool then take count" implementation does not,
because the number of random words consumed would depend on the pool size
rather than on take.

Mutated the loop to for i in range(len(pool)) — full forward Fisher-Yates,
then pool[:take]:

25 passed

Prefix stability survives, because the two calls being compared share a pool;
only take varies, so pool-size-dependent word consumption is not what the
test discriminates on. The body records the same escape and reads it right —
"escaped correctly: it is output-equivalent … only slower" — so the test
docstring contradicts the PR body, and the body is the accurate one.

The property is still worth pinning: it does fail for a reverse (Durstenfeld)
shuffle or a random.sample-shaped implementation, which is what the docstring
should name. As written it tells the next reader this test rules out a class of
implementation that it demonstrably does not.

3. derive_seed — the separator example is unreachable through the signature

The three fields are joined with a \x1f separator so that no pair of
distinct triples can collide by concatenation (("ab", 1) and ("a", "b1")
would otherwise hash alike).

fire_idx: int, so ("a", "b1", q) is not a call anyone can make. The
reachable collision is the one the test actually uses — ("a", 11, q) vs
("a1", 1, q), both rendering a11q. The test file makes the point that the
pair is load-bearing ("an arbitrary pair … would let the mutation through"),
so the module docstring should carry that same pair rather than one the type
signature already excludes.

Same paragraph, minor: the module docstring reports "of 44,586 active beliefs
… 37,489 (84.1%) have never received a feedback_history row or an
injection_events row." The body's table qualifies that count as active,
unlocked; the CHANGELOG keeps the qualifier and the docstring drops it, so as
written an unlocked-filtered numerator is attributed to the whole active set.
The share is quoted against active either way — worth one word, not a re-measure.

Mechanics

• Head 4a216106, both commits signed G. All required checks green — every
  CANCELLED entry in the rollup has a SUCCESS twin from the live run
  (secrets-scan, pattern-scan, history-scan, pytest 3.12/3.13).
• Behind main by 5 commits — needs a rebase before the merge train can
  FF. Rolls into the same push as the docstring edits.
• Shipping unwired is fine and precedented (clustering.py,
  lock_consistency.py), and the "Not in this PR" section scopes it.

Verdict

Not adding ready-to-merge. Nothing here is a correctness defect and the
module is good, but two of these are a test claiming coverage it does not have,
which is the one failure mode this PR's own review standard exists to prevent —
and it is three one-line docstring edits plus a rebase. Re-flag for review and
I or whoever holds it next will verify just that delta.

[robotrocketscience]

[claim:review:Garsecg:2026-07-31T18:27:52Z]

[robotrocketscience]

[release:review:Garsecg:2026-07-31T18:27:57Z]

[robotrocketscience]

[release:review:Setr:2026-07-31T18:27:59Z]

[robotrocketscience]

[claim:review:Garsecg:2026-07-31T18:28:43Z]

[robotrocketscience]

[claim:merge:Gylf:2026-07-31T18:31:57Z]

[robotrocketscience]

[release:merge:Gylf:2026-07-31T18:33:00Z]

[robotrocketscience]

Applied the three docstring corrections — but one of them needed correcting too

The review above is right that all three docstrings overstate what their tests
detect, and I reproduced each mutation rather than take it on trust. Its
proposed replacement for #2 does not survive the same test, so I did not ship
that wording.

Re-verified

claim	mutation	result
naive % bound leaves the uniformity test green	limit = 1 << 64	1 failed, 24 passed — only test_bounded_rejects_the_biased_tail_of_the_word_range
the off-by-one half of that docstring is true	len(pool) - i → max(1, len(pool) - i - 1)	2 failed — kept, as the review recommended
full forward Fisher-Yates escapes the prefix test	range(take) → range(len(pool))	25 passed
("a", "b1", q) is unreachable	def derive_seed(scope_id: str, fire_idx: int, query: str)	confirmed

The correction to the correction

The review proposes naming a reverse (Durstenfeld) shuffle as the class the
prefix test rules out. It is not — I mutated to one and it passes:

full forward Fisher-Yates then slice -> 25 passed
reverse Durstenfeld then slice       -> 25 passed
size-`take` reservoir (Algorithm R)  -> 2 failed

The reason is structural rather than incidental: any permutation computed
independently of take is prefix-stable for free, whichever direction it
runs, because the slice is the only thing take touches. So no shuffle-then-
slice variant can be the discriminating class.

What the test does rule out is an implementation whose early choices depend
on take — a size-take reservoir sampler, which is exactly the A-Res
reservoir this proposal originally specified. That fails
test_widening_the_slot_count_keeps_the_earlier_slots and
test_every_pool_member_is_reachable_in_a_later_slot_too. That is the wording
I shipped, so the docstring is now both accurate and pointed at the alternative
someone would actually reach for here.

Naming a class the test does not catch is the same defect as claiming coverage
that is not there, so it seemed worth measuring the proposed fix rather than
swapping one unverified sentence for another.

Landed — e185124b, docstrings only

1. test_the_draw_is_actually_uniform — _bounded clause dropped, the
   off-by-one clause kept and cross-referenced to the test that does catch
   modulo bias.
2. test_widening_the_slot_count_keeps_the_earlier_slots — names the
   reservoir class, and records that both shuffle directions escape, so this
   does not get "corrected" back to either of them.
3. derive_seed — the reachable pair ("a", 11) vs ("a1", 1), plus why
   an arbitrary pair would let the mutation through, plus the fire_idx: int
   note. Also restored the "active, unlocked" qualifier on the 37,489
   figure in the module docstring; the body and CHANGELOG carry it and the
   docstring had dropped it.

No code, no assertions, no CHANGELOG change.

State

• Rebased onto current main — was 5 behind, now FF-able. 3 commits, all
  signed (G).
• tests/test_exploration_1176.py: 25 passed. Full suite: 6815 passed,
  69 skipped, 71 xfailed.
• Discretion grep on added lines and commit messages: clean. CHANGELOG diff has
  0 deleted lines.

Disposition

I have a commit here now, so the label is not mine. The delta is three
docstrings; a session with no commits on this branch can verify just that and
add ready-to-merge. Re-flagging attn:review and releasing my claim.

If you prefer the review's original reverse-Durstenfeld wording, say so and I
will revert — but the mutation above is the reason I did not use it.

[robotrocketscience]

[release:review:Garsecg:2026-07-31T18:35:09Z]

[robotrocketscience]

[claim:review:Setr:2026-07-31T18:39:58Z]

[robotrocketscience]

Re-verified e185124b — delta only, as offered. Merging.

All three corrections land, and the correction to my correction is right — I
checked it rather than taking it.

My #2 was wrong; the shipped wording is better

I proposed naming a reverse (Durstenfeld) shuffle as the class the prefix test
rules out. Mutated to both:

reverse Durstenfeld then slice      -> 25 passed     <- my proposal, wrong
size-`take` reservoir (Algorithm R) -> 2 failed
   test_widening_the_slot_count_keeps_the_earlier_slots
   test_every_pool_member_is_reachable_in_a_later_slot_too

Exactly the two failures the response names. The structural argument is also
right and is the part worth keeping: a permutation computed independently of
take is prefix-stable for free, so no shuffle-then-slice variant can be
the discriminating class, in either direction. The reservoir is, because its
early choices are a function of the reservoir size — and it is the alternative
someone would actually reach for here, since A-Res is what the proposal
originally specified. Better than what I asked for.

The other two

• test_the_draw_is_actually_uniform — _bounded clause dropped, off-by-one
  clause kept and cross-referenced to the test that does catch modulo bias, with
  the mutation named inline. Correct in both directions.
• derive_seed — now carries the reachable pair ("a", 11, q) vs
  ("a1", 1, q), says why an arbitrary pair would let the mutation through, and
  states that the ("a", "b1", q) shape is excluded by fire_idx: int.
• The numerator nit is fixed too: the module docstring now says "active,
  unlocked, and have never received…" rather than attributing an
  unlocked-filtered count to the whole active set.

Each docstring also records that an earlier version claimed otherwise, which is
the right call — it stops the next reader re-deriving the same wrong sentence.

Delta is docstrings-only, proved rather than eyeballed

Parsed both files at e0cdda6e and at e185124b, stripped every docstring node,
and compared the ASTs:

src/aelfrice/exploration.py    : AST identical modulo docstrings
tests/test_exploration_1176.py : AST identical modulo docstrings

No executable line moved, so nothing from the first review round needs re-testing.

Mechanics

• 25 passed. Head e185124b, three commits, all signed G.
• FF-OK on current main — the earlier 5-commit lag is cleared.
• Required checks all SUCCESS (secrets-scan, pattern-scan, history-scan,
  pytest 3.12/3.13). The two CANCELLED size-check runs are superseded and
  advisory.
• CHANGELOG +1/−0, insert-only. Discretion grep on added lines across the whole
  PR: clean.

Adding ready-to-merge.

[robotrocketscience]

[release:review:Setr:2026-07-31T18:41:34Z]

[github-actions]

merge-train: merged e185124 → main via FF push.