fix(audit): Hermes audit batch 2 — 6 issues fixed - #138
Conversation
…ntradiction vocab, opt-in LLM rerank, archive checksums Closes #94 Closes #93 Closes #103 Closes #105 Closes #108 Closes #111 #94: temporal_rerank now writes the boost to rrf_score (not similarity) when hybrid fusion results are present, and re-sorts by the same field. Previous behaviour silently boosted a field the ranker didn't read. #93: _adapt_kg incoming edges now use object_id as source_id (the queried anchor), not subject_id (the far node). The incoming branch previously produced identical output to the outgoing branch — direction information was lost. #103: VectorMemory now maintains a per-mode BM25 index cache. The index is built once per corpus state and reused on subsequent queries; _bm25_dirty is set on every add() / supersede() call so the next query rebuilds when the active corpus has changed. Avoids a full re-index on every query. #105: SINGULAR_PREDICATES moved from a hardcoded set inside TemporalKnowledgeGraph to predicate_vocab.SINGULAR_PREDICATES, extending coverage to include works_at, moved_to, and birth_place which were missing from the old list. TemporalKnowledgeGraph.SINGULAR_PREDICATES is now an alias for the canonical vocab set. #108: retrieve() gains an opt-in llm_reranker dict parameter (default None). When supplied with client/ollama_url/model keys, llm_listwise_rerank() is called as a second-pass stage after the cross-encoder in thorough and custom strategies. Default behaviour is completely unchanged. Full-scale LoCoMo-1540 validation still pending before enabling by default. #111: ArchiveStore.record() now writes a per-entry sha256 field (SHA-256 over the serialised event line, without the checksum key itself). New verify_entry() static method and verify_day() async method allow callers to audit archive integrity. Existing archives without checksums are treated as valid (backwards-compatible, legacy entries pass through).
|
Warning Review limit reached
More reviews will be available in 1 minute and 32 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (7)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Design for the held #138 Phase 2, unblocked by the registry identity layer. Covers target state (from derived from token sub, never the body), handle normalisation as the first hazard, fail-closed pubkey rules, and a three-stage transition where each stage advances on an exit test rather than a date. Also records measured defects in the authenticated read path we are about to recommend to every new agent: channel=all returns 200 with zero messages (identical to an unknown channel), and every cursor param is silently ignored.
1 participant
Summary
Addresses 6 of the 7 remaining open Hermes Agent Audit issues. Issue #102 (transactional consistency) received a design-options analysis comment instead of code changes, as directed.
temporal_boostfield mismatch: boost now written torrf_scorewhen present (hybrid results), falls back tosimilarityfor pure-semantic results. Re-sort key matches the field used._adapt_kgdirection: incoming edges now recordobject_id(the queried anchor) assource_id, notsubject_id(the far node). Both arms previously produced identical output.VectorMemorynow caches the built index per fusion mode. Invalidated onadd()/supersede(). No behaviour change, no result change.SINGULAR_PREDICATESmoved topredicate_vocab(the closed-vocab source of truth) and extended withworks_at,moved_to,birth_place.TemporalKnowledgeGraph.SINGULAR_PREDICATESis now an alias.retrieve()acceptsllm_reranker=None(default off). When supplied with{client, ollama_url, model},llm_listwise_rerank()runs after the cross-encoder in thorough/custom strategies. Full-scale LoCoMo-1540 validation pending before enabling by default.record()appends asha256field per entry. Newverify_entry()+verify_day()helpers. Backwards-compatible — legacy entries without checksums pass through.Test plan
python3 -m pytest tests/ -q— 455 passed (434 baseline + 21 new)tests/test_hermes_audit_batch2.pycovering each fixed issueretrieve()behavior unchanged (nollm_rerankerset by default)