feat(recall): per-strategy retrieval boost via env config - #1974
Merged
Conversation
nicoloboschi
force-pushed
the
feat/recall-strategy-boost
branch
2 times, most recently
from
June 4, 2026 09:44
cbfa7b0 to
06fbd9c
Compare
Add HINDSIGHT_API_RECALL_STRATEGY_BOOSTS, a single env knob that lets a
deployment prioritise one or more retrieval arms (semantic/bm25/graph/temporal)
over the others using a human priority level — e.g. "graph:high" to strongly
favour graph hits, or "graph:high,semantic:low". Valid levels: low | medium |
high. A strategy listed without a level ("graph") defaults to medium; arms you
don't list keep their normal weight; empty disables the feature.
A named level (not a raw number) is the knob because the boost is applied in
two structurally different places on different score scales:
1. Before the reranker cap, as a weighted-RRF sort key, so boosted-arm
candidates survive the global candidate budget instead of being trimmed by
raw RRF score (rank-aware).
2. After the reranker, as a flat additive bump to the final ranking weight.
Level -> per-stage magnitudes (in engine/search/recall_boost.py) are tuned
against real recall traces (LoCoMo bank, 336 merged candidates -> 300-cap,
local ms-marco cross-encoder): the observed cap boundary RRF was ~0.0055, so
the stage-1 multipliers 1/3/6 map to rescue/promote/dominate; the cross-encoder
weight scale is [0,1] and bimodal, so the stage-2 additives 0.05/0.2/0.5 map to
nudge/compete/win-over-most-matches. A guard test keeps the level names in sync
with config. Global, read via get_config(), mirroring
recall_max_candidates_per_source.
nicoloboschi
force-pushed
the
feat/recall-strategy-boost
branch
from
June 4, 2026 11:38
06fbd9c to
03d5df7
Compare
nicoloboschi
pushed a commit
that referenced
this pull request
Jun 5, 2026
…STS (#1974) (#1991) #1974 added HINDSIGHT_API_RECALL_STRATEGY_BOOSTS (named low/medium/high per-source boosts), making retrieval.md's absolute claim 'There are no per-strategy weight multipliers' factually wrong. Scope the equal-weight statement to RRF fusion itself, point readers to the boost knob, and note at the pre-filter cap stage that boosted sources are more likely to survive. Regenerated the skills mirror.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
HINDSIGHT_API_RECALL_STRATEGY_BOOSTS— a single env knob to prioritise one retrieval arm (semantic/bm25/graph/temporal) over the others on recall. Requested by a user who wants to favour graph over semantic.Empty/unset → complete no-op (current behaviour). Valid strategies:
semantic,bm25,graph,temporal. Negative weights deprioritise. Malformed/unknown entries warn-and-skip rather than break recall.How — two insertion points
The boost is applied in the two places it needs to land:
Before the reranker cap (
memory_engine.py, the 300-candidate pre-filter) — used as a weighted-RRF sort key so boosted-arm candidates survive the global candidate budget instead of being trimmed out by raw RRF score. Rank-aware: a candidate ranked Fix dependencies and docker #1 in the boosted arm gets more protection than ci: add upgrade tests #200. Only the sort key changes;rrf_scoreis left untouched for trace fidelity.After the reranker (right after
apply_combined_scoring) — a flat additive bump to the final rankingweight, then the existing sort reorders.combined_scoreis left untouched; onlyweight(the ranking key) changes.Design notes
get_config()— mirrors the existingrecall_max_candidates_per_source/reranker_max_candidatesretrieval knobs. Not per-bank/hierarchical (keeps scope minimal; easy to promote later).engine/search/recall_boost.py(unit-testable, out of the giantmemory_engine). Parsing lives inconfig.pyto avoid early-importing the heavyengine.searchpackage.MergedCandidate.source_ranks({"graph_rank": 3, ...}) produced by RRF — no new plumbing.Tests
tests/test_recall_boost.py— 16 deterministic unit tests: env parsing (empty/valid/case/zero-drop/unknown/malformed/negative) and both boost functions (no-op, rank-awareness, flat additive, summing, non-matching arm, negative weights).test_fusion_cap,test_config_validation,test_recall_config,test_reranker_score_normalization(75 passed). No LLM-behaviour change → no judge test needed.Docs
configuration.md(+ skills mirror) documents the new env var.