fix: lower default reranker_max_candidates from 300 to 50

[handnewb]

Summary

DEFAULT_RERANKER_MAX_CANDIDATES was 300, causing 30-45s recall latency with local reranker (e.g. jina-reranker-v3). Changed to 50, reducing latency to 13-19s with no measurable quality degradation.

Impact

Setting	Recall Latency	Quality
300 (old default)	30-45s	Excellent
50 (new default)	13-19s	No degradation
20	8-12s	Slight degradation

Users can still override via HINDSIGHT_API_RERANKER_MAX_CANDIDATES.

Change

1 line in config.py:

-DEFAULT_RERANKER_MAX_CANDIDATES = 300
+DEFAULT_RERANKER_MAX_CANDIDATES = 50

Closes #2554

[benfrank241]

thanks @handnewb — the latency point is real and worth addressing, but i want to hold this one for @nicoloboschi to call rather than merge it directly. it's a change to a global recall-quality default, not a bug fix, so it's a product tradeoff rather than a correctness question.

two things worth surfacing for that decision:

• the quality claim needs evidence. reranker_max_candidates is exactly how many fused (RRF) candidates the cross-encoder gets to re-score, so 300→50 means any relevant memory that fusion ranks 51–300 can no longer be rescued by the reranker. "no measurable degradation" is very plausible, but on an accuracy-focused product this is the kind of default we'd want backed by a longmemeval/locomo run at 300 vs 50 rather than a subjective read. happy to run that.

• it's tuned for one profile. the 30–45s is a local jina-reranker-v3; TEI/GPU and hosted rerankers (cohere/voyage) chew through 300 fast and would just lose headroom. so a global flip optimizes the local-cpu case at some cost to everyone else — which is partly why a middle value (say 100) or the startup-warning-over-100 alternative from [Bug] reranker_max_candidates default=300 causes 30s+ recall latency — should default to 50 #2554 might land better.

also, if we do change it, the documented default (300 in configuration.md + the generated skill mirror) needs the same bump or the docs drift.

net: not blocking the direction — 300 is arguably high — just don't want to move a core-quality default on anecdote. nicolo, your call on the value (50 vs 100 vs warn-only), and i can get benchmark numbers first if useful.

[nicoloboschi]

just change your server value. 300 is for production environments and it's fine as default, thanks.