feat: add Weibull decay and rerank pipeline for search results

[zackchiutw]

Summary

Introduces a config-driven post-retrieval rerank pipeline that adjusts search ranking beyond raw cosine distance. Four stages, independently toggleable via ~/.mempalace/config.json:

• Weibull time-decay — newer memories surface higher; old ones decay toward a configurable floor
• Keyword overlap boost — lexical overlap between query and hit text nudges distance down
• Importance boost — hits with higher emotional_weight metadata get a small boost
• LLM rerank (optional) — final top-K pass via Anthropic API

All stages are off by default. No config = identical behavior to before. When any stage is enabled, search_memories over-fetches 3× candidates, runs the pipeline after the existing closet-boost + BM25 hybrid rank (both preserved), re-applies max_distance against the post-rerank fused_distance, and flags the response with reranked: True for transparency.

Why

Raw cosine distance treats every memory as if it were written yesterday. For a memory system where identity and recency matter, that's a regression trap — old notes crowd out fresh ones. The pipeline is modular so users can tune the trade-off per palace (e.g., λ=90 days for a work palace, λ=365 for a personal history wing).

What changes

File	Change
mempalace/reranker.py (new)	4-stage pipeline, unified fused_distance field
mempalace/searcher.py	rerank=True param on search_memories; pipeline runs after _hybrid_rank so closet/BM25 signals are preserved; _load_rerank_config + _apply_rerank helpers keep complexity under the C901 threshold
mempalace/config.py	rerank_config property reading config.json["rerank"]
mempalace/mcp_server.py	rerank param + schema entry on mempalace_search
mempalace/layers.py	Layer1 optional read-time decay (k=1.2, λ=365, floor=0.6); Layer3 refactored to consume search_memories() dict
mempalace/knowledge_graph.py	query_entity(apply_decay=...) applies the same Weibull curve to stored confidence at read time
tests/test_reranker.py (new)	77 unit tests covering every stage + edge cases
tests/test_layers.py	Updated for new Layer3 return shape

Example config

{
  "rerank": {
    "weibull_decay":   {"enabled": true,  "k": 1.5, "lambda": 90,  "floor": 0.3},
    "keyword_boost":   {"enabled": true,  "weight": 0.30},
    "importance_boost":{"enabled": false, "weight": 0.15},
    "llm_rerank":      {"enabled": false, "model": "claude-haiku-4-5-20251001", "top_k": 10}
  }
}

Test plan

• pytest tests/test_reranker.py tests/test_searcher.py tests/test_layers.py tests/test_config.py tests/test_mcp_server.py — 186 passed
• ruff check . — all checks passed
• ruff format --check on modified files — clean
• Backward compatibility verified: with no rerank key in config, search_memories returns identical output (no reranked flag, no new fields in response)
• mempalace_search MCP tool accepts rerank: false to bypass the pipeline when desired

Notes for reviewers

• The pipeline composes on top of the existing closet-boost + BM25 hybrid rank, rather than replacing either — earlier drafts that bypassed them regressed on closet-rich corpora
• When rerank is enabled but no stages are configured, the pipeline is an identity function and the reranked flag stays false
• No new dependencies required; llm_rerank stage is gated behind a local import of anthropic and skipped if unavailable

🤖 Generated with Claude Code

[ATKabli]

Went through this in detail because we run a very similar patch in production (listwise LLM rerank on top of a vector+BM25 candidate pool) and wanted to compare notes. A few observations, offered as independent data since our implementation differs from stage 4 here.

On the idea itself: we have measurement that LLM rerank at the retrieval layer is worth doing. On a private 200-question eval it took us from 129/200 to 161/200 correct (sign test p=0.00004), and answer-in-top-3 from 40% to 63%. On the public LongMemEval held-out 450, our hybrid retrieval + local-model rerank hits R@5 = 1.000 against the published 0.984 no-rerank baseline. That's a different implementation than stage_llm_rerank here, but it's evidence the general direction (post-retrieval LLM pass over a widened candidate pool) is a real win, not noise.

Three things from our production experience that might be worth folding in before this lands:

1. Endpoint is hardcoded to api.anthropic.com. stage_llm_rerank builds the request against the Anthropic Messages API directly with no base_url override. We needed to point rerank at a self-hosted OpenAI-compatible endpoint (local model, no per-query API cost, ~2-5s/query), so we made the endpoint, model, and timeout all config keys under rerank.llm_rerank. Might be worth genericizing the transport the same way here, since it's a small change and opens this up to anyone not on Claude API budget for every search.

2. It's a single-pick promotion, not a full reorder. Right now the LLM is asked for one best excerpt number, and that one hit gets spliced into rank 1 (fused_distance = min_dist * 0.5); ranks 2..N stay wherever the earlier stages left them. We send the same top-N candidates but ask for a full ranked JSON array in one call and reorder the whole slice (with backfill from the prior order if the model returns a short/malformed list). It's roughly the same request cost and gives the LLM's judgment on the whole top-N instead of just the winner, which is presumably why we're seeing the numbers above.

3. The fail-open guarantee has a gap. stage_llm_rerank's own network call is safely wrapped, but stage_weibull_decay, stage_keyword_boost, and stage_importance_boost all do unguarded float(cfg.get(...)) casts on config values. Neither rerank() nor _apply_rerank() in searcher.py catches exceptions from those stages, and _load_rerank_config's try/except only covers building the MempalaceConfig() object, not running the pipeline. So a bad value in config.json (wrong type, typo) raises uncaught inside search_memories() and takes down the whole search call, not just rerank. Given the PR description explicitly frames "no config = identical behavior" as the safety net, it's probably worth wrapping each stage call (or the whole rerank() loop) in try/except so a malformed config degrades to "rerank off" instead of a hard failure.

One smaller note: the diff wires rerank into search_memories() (and from there into the mempalace_search MCP tool), but the CLI search() printer in the same file isn't touched, so mempalace search from the command line won't rerank even with it configured on. Probably intentional for this PR's scope, but worth a follow-up or a mention in the docs so it's not a surprise.

Happy to share more detail on our listwise implementation/eval setup if useful.