feat(integrations): Hermes memory provider (revives #3, ABC-current + dim-fix) - #1684
Closed
raman325 wants to merge 24 commits into
Closed
feat(integrations): Hermes memory provider (revives #3, ABC-current + dim-fix)#1684raman325 wants to merge 24 commits into
raman325 wants to merge 24 commits into
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces a native integration between MemPalace and the Hermes agent, allowing MemPalace to act as a local, semantic memory provider. It adds the Hermes memory provider implementation, a backfill script to import historical sessions, CLI commands for installation, and comprehensive unit tests. Feedback on the changes highlights critical issues, including a potential embedding dimension mismatch and performance bottleneck in the backfill script due to raw client instantiation on every turn, a memory scaling issue when reading the diary file, and potential data loss on shutdown because the background worker does not drain its queue before exiting.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
…read safety, diary safety)
The Hermes MemoryProvider ABC (`agent/memory_provider.py`) has evolved
since the original integration was written. This commit brings the
plugin current with `initialize(session_id, **kwargs)`, the str-returning
`on_pre_compress`, the separation of `get_tool_schemas` from
`handle_tool_call`, and the optional `on_session_switch`, `on_delegation`,
`on_memory_write`, `on_turn_start`, `queue_prefetch`, `save_config`, and
`post_setup` hooks.
Also fixes the embedding-dimension mismatch that silently broke the three
in-tree Hermes-side PRs (NousResearch/hermes-agent #5671, #12203, #9761):
ChromaDB access now goes through `mempalace.backends.chroma.ChromaBackend`,
which binds the canonical embedding function returned by
`mempalace.embedding.get_embedding_function()`. Existing palaces import
without rebuild.
Other notable changes:
* Cron-context guard: provider becomes inactive under
`agent_context in {"cron", "flush"}` or `platform == "cron"` so
system-generated turns do not corrupt the user representation
* Bounded background queue (maxsize=500) for non-blocking writes;
`shutdown()` drains it and logs if drain times out
* `cmd_hermes_install` now targets `$HERMES_HOME/plugins/mempalace/` —
the canonical user-installed plugin directory scanned by
`discover_memory_providers()` — instead of the closed-to-new-additions
bundled `plugins/memory/<name>/` tree
* `cmd_hermes_install` writes a `plugin.yaml` manifest, loads `backfill.py`
by path rather than relying on an in-Hermes-tree import, and uses
`subprocess.run` instead of the deprecated `os.system`
* SHA-256 dedup over full content (not a prefix) — prior audits flagged
the prefix hash as a silent-collision risk on long repeated turns
19 focused tests covering the provider class shape, cron-context guard, session-switch bookkeeping, `on_pre_compress` hint behavior, wing classification, and shutdown safety. Tests load the integration module by file path (it lives in `integrations/hermes/`, not on the import path) and install a stubbed `agent.memory_provider` so the module imports cleanly without hermes-agent in the venv.
15 new tests reuse the project's own fixtures (palace_path, seeded_collection, seeded_kg, tmp_dir) so end-to-end behavior runs against the real ChromaBackend + KnowledgeGraph rather than mocks. Specifically: * test_initialize_opens_chroma_via_backend — asserts the dim-mismatch fix path: provider._backend is mempalace.backends.chroma.ChromaBackend, not a raw chromadb.PersistentClient * test_sync_turn_persists_through_worker — files a turn, drains the queue via _worker_queue.join(), verifies the drawer landed with source=hermes * test_status_tool_counts_seeded_drawers + test_list_wings_tool_returns_seeded_wings + test_list_rooms_tool_filters_by_wing — provider opens the same palace that seeded_collection prepared via raw chromadb.PersistentClient. The fact that these read back cleanly is the regression check that the two construction paths bind compatible embedding functions * test_kg_add_persists_to_palace_sibling_sqlite — verifies handle_tool_call writes the triple to <palace>/../knowledge_graph.sqlite3 where an independent KnowledgeGraph instance can read it * test_diary_write_read_roundtrip — round-trip via handle_tool_call * test_initialize_reads_mempalace_json + test_env_vars_override_config_file — config precedence (env > $HERMES_HOME/mempalace.json > defaults) Total: 34 integration tests, 22s runtime (dominated by ChromaDB embedding model load on first use).
Per gemini-code-assist on PR MemPalace#1684: ``file_exchange`` was creating a fresh ``chromadb.PersistentClient`` per exchange and calling ``get_or_create_collection`` without ``embedding_function=``. Two problems: 1. Reintroduced the embedding-dimension mismatch the provider itself was updated to fix. Users with palaces on bge-m3 (1024-dim) or embeddinggemma (Matryoshka-truncated 384-dim) would hit a hard dimension error mid-backfill instead of silently bypassing it. 2. Per-exchange ``PersistentClient`` instantiation is expensive — for a user with thousands of historical Hermes sessions it would dominate backfill time. ``backfill()`` now opens the collection once via ``mempalace.backends.chroma.ChromaBackend.get_or_create_collection`` (the same path the runtime provider uses, so the canonical embedding function is bound) and threads the collection into ``file_exchange``.
Two issues raised by gemini-code-assist on PR MemPalace#1684: * ``_background_worker``: the loop exited as soon as ``_worker_stop`` was set, losing any pending items in the bounded queue. The compound condition now waits for both ``stop set`` AND ``queue empty`` before exiting, so turns enqueued just before ``shutdown()`` still get filed. * ``_tool_diary_read``: ``Path.read_text().splitlines()`` was loading the entire ``diary.jsonl`` into memory just to keep the last N lines. ``collections.deque(f, maxlen=n)`` streams the file and retains only the tail.
mempalace's library modules (searcher.py, knowledge_graph.py, backends/chroma.py) put all imports at module top; only mempalace/cli.py defers imports inside command functions, for CLI startup speed. This plugin is library code, not a CLI command, so it should follow the library convention. Hoist ``ChromaBackend``, ``KnowledgeGraph``, ``MemoryStack``, ``search_memories``, and ``collections.deque`` to the module-top import block. The ``agent.memory_provider`` try/except stays where it is — it has to gate on a runtime dependency (Hermes) that the mempalace package cannot import unconditionally. ``is_available`` simplifies: if mempalace were missing, this module would have failed to import before Hermes' plugin loader could call the method. The check is kept as a stable hook for future config-based disabling.
raman325
force-pushed
the
feat/hermes-integration
branch
from
June 3, 2026 17:27
be66733 to
40e6689
Compare
raman325
marked this pull request as ready for review
June 3, 2026 17:28
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds a Hermes memory-provider integration for MemPalace, including an install command, optional backfill tooling, and accompanying docs/tests to validate the Hermes plugin behavior.
Changes:
- Introduce a Hermes
MemoryProviderplugin (integrations/hermes/) with tool dispatch, background filing worker, and cron-context guard. - Add
mempalace hermes installCLI flow to copy plugin artifacts into$HERMES_HOMEand optionally backfill sessions. - Add pytest integration coverage + documentation for usage and rationale (embedding dim-mismatch fix).
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 11 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_hermes_integration.py | New integration tests that load the Hermes plugin by path and validate lifecycle/tool behavior. |
| mempalace/cli.py | Adds mempalace hermes install subcommand, plugin.yaml generation, config.yaml update, and optional backfill invocation. |
| integrations/hermes/backfill.py | New standalone backfill script to mine Hermes session exports into a MemPalace Chroma collection. |
| integrations/hermes/init.py | New Hermes memory provider implementation + tool schemas and background worker. |
| integrations/hermes/README.md | New documentation explaining install, hooks, tools, and why the Chroma backend path avoids dim mismatches. |
| README.md | Adds a top-level section pointing users to the Hermes integration. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
raman325
marked this pull request as draft
June 3, 2026 17:33
Addresses the highest-severity findings from the PR MemPalace#1684 review pass. Every fix here is about the provider not silently breaking the "every turn is filed verbatim" promise mempalace sells. * ``_initialized`` now reflects backend readiness, not just "the initialize() function returned". If ``ChromaBackend`` open fails (locked SQLite, slow disk, missing palace) the flag stays False so ``get_tool_schemas``, ``handle_tool_call``, ``prefetch``, ``sync_turn`` and ``on_pre_compress`` short-circuit uniformly instead of advertising tools the handlers can't service. * ``on_pre_compress`` only returns the "search will find these later" hint when the worker can actually persist the payload. If the backend is down OR the queue is full, return an empty hint — the summarizer falls back to its own conservative discarding instead of acting on a false promise. * Worker's ``pre_compress`` branch now pairs adjacent (user, assistant) messages into turns rather than filing only ``role == 'user'``. The hint said "every message" — silently dropping assistant content was the largest data-loss path in the review. * New ``_normalize_content`` flattens Anthropic-style ``content`` lists (``[{type:'text', text:...}, {type:'tool_use', ...}]``) into plain text before persistence. Without it, ``f"User: {content}"`` was storing the literal ``repr`` of the list and corrupting search recall. Applied in ``sync_turn`` and the worker's ``pre_compress`` branch. * ``sync_turn`` queue-full path now logs at WARNING, not DEBUG, and mentions the likely cause. Operators need a visible signal when the verbatim invariant is at risk; DEBUG is invisible under the default root logger. * ``_cron_skipped`` is now cleared at the start of every non-cron ``initialize()``. Previously a cron-context init permanently poisoned the provider instance. * ``initialize()`` is serialised on a new ``_init_lock`` so a re-entrant call from ``on_session_switch`` can't spawn duplicate worker threads sharing one queue. * ``collection_name`` removed from the config schema entirely. Writes went through ``self._collection_name``; reads through ``search_memories`` use mempalace's own configured collection name from ``~/.mempalace/config.json``. Two paths, one knob → silent mismatch when customised. Now the provider always uses the default; users wanting a different collection name set it once in mempalace's own config and both sides read it. * ``_load_config`` ignores empty env-var values. ``export MEMPALACE_WING=`` is intent to unset, not to set wing to the empty string and clobber the value from ``mempalace.json``. Tests updated to reflect the new contract: the pre-compress hint is now gated on ``_initialized``, and a new test asserts ``collection_name`` is absent from the schema with a comment recording why.
Per Copilot review on PR MemPalace#1684: ``_mirror_mem_write``, ``_tool_kg_query``, and ``_tool_kg_add`` each opened a ``KnowledgeGraph`` (which holds a ``sqlite3.Connection`` to ``knowledge_graph.sqlite3``) and never closed it. In a long-running Hermes session calling ``mempalace_kg_query`` repeatedly, the open SQLite handles accumulate and eventually hit the process FD limit or fail subsequent writes with "database is locked". Wrap each in ``try/finally: kg.close()``. Treat a failure inside ``close()`` as best-effort — we already returned (or are about to) the query result.
… + stable room Three related fixes flagged by the Copilot review on PR MemPalace#1684: * ``_classify_wing`` previously matched keywords via unbounded substring containment (``kw.lower() in text_lower``). Short keywords routed turns to wrong wings: ``ai`` matched inside ``said``/``rain``/``main``; ``go`` inside ``good``/``google``; ``rb`` inside ``orbit``. Now use a ``\b...\b`` regex pattern (with ``re.escape``) so keywords match only whole-word occurrences. * The ``wing`` config field was documented as a "default wing for filing", but ``_file_turn`` always ran keyword classification on top. Users setting ``wing: wing_dev`` would still see turns routed to ``wing_general`` when no keyword matched. Now ``_classify_wing`` honors ``self._config['wing']`` first and short-circuits. * ``room`` was set to ``payload.get("session_id") or "conversations"``. Using the session id created one room per session — pollutes ``mempalace_list_rooms`` with high-cardinality entries and split- brained against ``backfill.py``, which files everything under ``room: "conversations"``. Now ``_file_turn`` always writes ``room: "conversations"`` and stashes the session id in a dedicated ``session_id`` metadata field so room aggregation stays useful and session filtering remains possible.
Per Copilot review on PR MemPalace#1684: * ``doc_id`` was computed from a 120-char text prefix and truncated to 16 hex chars. Both choices independently cause collisions; together they made it likely. A session file with 12 exchanges that share the opening ``"User: hi can you help me with…"`` (under 120 chars including the prefix) all hashed to the same id; ``col.upsert`` silently overwrote each, persisting only the last one. The mempalace verbatim invariant requires every exchange survive. Hash now: full text + timestamp + source_file path, 32 hex chars (matches the live provider's id width). * ``datetime.utcnow().isoformat()`` produced a naive timestamp without the ``+00:00`` offset. The live provider writes ``datetime.now(timezone.utc).isoformat()`` (offset-aware). Mixed naive/aware timestamps in the same collection break downstream time-window filtering. Aligned backfill to the timezone-aware form.
Addresses six Copilot/code-review findings on PR MemPalace#1684 around ``mempalace hermes install``: 1. Hardcoded palace path: backfill always wrote to ``~/.mempalace/palace`` even when the user had ``MEMPALACE_PALACE_PATH`` set. The runtime provider honors the env var, so backfill drawers were orphaned in a second palace the agent never read. ``_resolve_install_palace_path`` now mirrors the provider's precedence (env > default). 2. YAML edit was a fragile line-based heuristic that produced invalid YAML for ``memory: ~``, dropped inline comments via ``line.replace(stripped, ...)``, and assumed 2-space indentation. Replaced with ``yaml.safe_load`` + ``yaml.safe_dump`` round-trip plus atomic write. Trade-off: comments are not preserved; documented as a known limitation in the helper docstring. 3. Atomic writes: ``config.yaml`` and ``plugin.yaml`` now go through ``_atomic_write_text`` (tmp + ``os.replace``). A crash mid-write no longer leaves the user with a truncated Hermes config. 4. Windows venv path: ``_resolve_install_python`` now checks ``venv/bin/python3``, ``venv/bin/python``, and ``venv/Scripts/python.exe`` in turn before falling back to ``sys.executable``. 5. ``--upgrade`` added to the ``pip install`` invocation. Without it pip silently leaves a stale mempalace in place on an existing install. 6. ``--hermes-home .`` silently installed the plugin into CWD; ``_resolve_hermes_home`` now refuses CWD or ``/`` with a clear message and ``sys.exit(2)``. Also: exit code 1 on pip-install failure, missing source files, or backfill exception — so CI / scripted installs can detect failure. Soft warning (not abort) when the target ``hermes_home`` doesn't look like an existing Hermes install — preserves the fresh-Hermes-setup path while surfacing the situation. 22 new unit tests in ``tests/test_hermes_install_cli.py`` cover the helpers (path resolution, env-var precedence, atomic writes, YAML edit for missing/scalar/malformed/already-set/non-mapping inputs).
The first review fixed each issue at its most visible call site but
missed the parallel paths that share the same logic. Copilot caught four
of these on the post-fix HEAD:
* ``_mine_session`` (the worker's ``session_end`` handler) called
``_file_turn`` with the raw ``msg.get("content")`` instead of going
through ``_normalize_content``. Anthropic-format list content would
end up persisted as its Python ``repr`` — the very corruption the
normalizer was added to prevent for ``sync_turn`` and the
``pre_compress`` handler.
* ``on_session_end`` had no readiness gate. If ``initialize()`` set
``_initialized=False`` after a backend init failure, the worker
thread never started — but ``on_session_end`` would still
``put_nowait`` into the bounded queue, eventually saturating it with
tasks that can never drain.
* ``on_memory_write`` had the same gap.
* ``backfill.py:classify_wing`` still used bare substring matching
(``kw.lower() in text_lower``) while the live provider had been
updated to word-boundary regex. Backfilled drawers and live writes
would route to different wings for the same content.
Also factor the keyword-matching logic into a module-level
``_match_wing_by_keywords`` helper so the provider's ``_classify_wing``
uses one definition. ``backfill.py``'s ``classify_wing`` keeps a
verbatim copy (it cannot import across the boundary — it gets
``importlib.util.spec_from_file_location``-loaded by the install
command) with a docstring marker noting both copies must change
together.
Queue-full paths in ``on_session_end`` / ``on_memory_write`` upgraded
from silent ``pass`` to ``logger.warning``, matching the discipline
already applied to ``sync_turn`` and ``on_pre_compress``.
New tests pin the regressions:
* ``test_on_session_end_no_op_when_not_initialized``
* ``test_on_memory_write_no_op_when_not_initialized``
* ``test_normalize_content_flattens_anthropic_list``
* ``test_match_wing_by_keywords_word_boundary``
* ``test_backfill_classify_wing_matches_live_provider``
Plus a test-internal fix from finding MemPalace#10: an assertion comment said
"one occurrence" but the condition allowed up to 2; replaced the
substring count with an actual top-level ``memory:`` key count.
…nv var After the verbatim-contract fix removed ``collection_name`` from the supported config and the env-var loader was changed to ignore empty strings, both the module docstring and ``integrations/hermes/README.md`` were left stale on two points: 1. Said "first match wins" — implies file > env > defaults. Actually env vars override the file, so it's the other way around. Tests already assert the override direction. 2. Listed ``MEMPALACE_COLLECTION_NAME`` as supported. The provider now pins the collection name to ``mempalace_drawers`` so writes and reads can't silently diverge. Aligned the docstring and README to describe the actual precedence (file < env vars, with non-empty values), drop the ``MEMPALACE_COLLECTION_NAME`` reference, and add a short rationale for why ``collection_name`` is intentionally absent from the schema. README now uses a table for the supported keys.
Per Copilot review on PR MemPalace#1684: the install command resolved ``integrations/hermes/`` via ``Path(__file__).parent.parent`` and a ``find_spec("mempalace")`` fallback that assumed ``integrations/`` lived next to ``<site-packages>/mempalace/``. Neither held for a normal wheel install (``packages = ["mempalace"]`` in ``pyproject.toml``), so ``mempalace hermes install`` would fail with "could not locate integrations/hermes/" the moment the user installed via ``pip install mempalace`` instead of editable mode. Moved the integration into the package as ``mempalace/integrations/hermes/`` and added ``mempalace/integrations/__init__.py`` so it's a proper subpackage. The hatchling ``packages = ["mempalace"]`` config now ships it automatically — verified with ``uv build``: mempalace/integrations/__init__.py mempalace/integrations/hermes/README.md mempalace/integrations/hermes/__init__.py mempalace/integrations/hermes/backfill.py are all inside the resulting wheel. ``cmd_hermes_install`` now finds the source via ``importlib.util.find_spec("mempalace.integrations.hermes")``. Works identically in source-tree, editable install, and wheel install. Tests and READMEs updated to reference the new path. The top-level ``integrations/openclaw/`` directory is untouched — that integration is a Claude Code skill (markdown only), so it doesn't need packaging.
Per Copilot review on PR MemPalace#1684: ``_update_hermes_config_yaml`` returned ``(updated: bool, message: str)`` where ``updated=False`` collapsed two very different cases: * the file already had the desired ``memory.provider`` (a clean noop, install should succeed) * the file couldn't be parsed, wasn't a YAML mapping, PyYAML wasn't installed, or the write failed (real error, install should exit 1) ``cmd_hermes_install`` had no way to tell them apart, so a corrupted ``config.yaml`` would silently produce a successful install summary. Return shape is now ``(status: str, message: str)`` where ``status`` is one of ``"updated"`` / ``"noop"`` / ``"error"``. The install command exits non-zero only on ``"error"``. Tests updated to assert each path returns the correct status.
Per Copilot review on PR MemPalace#1684: ``_tool_status``, ``_tool_list_wings``, and ``_tool_list_rooms`` all materialized every drawer's metadata into Python memory via ``col.get(include=["metadatas"])``. On a long-running mempalace install (200k+ drawers in a single wing) the model's ``mempalace_status`` tool call would block for seconds and risk an OOM on small hosts. The verbatim invariant cuts the other way too — the palace grows monotonically, so what's small today is large later. Introduces ``STATUS_SCAN_LIMIT = 5000`` and a ``_scan_metadatas`` helper that passes ``limit=`` through to ``col.get`` (falling back to the full scan if the pinned chroma version doesn't accept ``limit``). Total count still comes from ``col.count()``, which is O(1). When the scan is truncated, the response carries an explicit ``"truncated":`` field so the model knows the wing breakdown is sampled rather than authoritative.
raman325
marked this pull request as ready for review
June 3, 2026 18:42
|
This is impressively thorough integration work — the dimension-mismatch fix alone is worth merging. Three observations from our production memory architecture: 1. The cron-context guard is exactly right. We learned this the hard way: system-generated turns (maintenance, backfills, health checks) will absolutely poison your user representation if you don't gate them. Your 2. The ChromaDB embedding-function routing is the right architecture. Going through 3. The The pkg structure debate (mempalace/integrations/hermes/ vs top-level): Your reasoning is sound — the integration has to be importable Python for One follow-up question: The PR mentions Excellent work overall — the test coverage (62 new tests) and the round-trip review discipline are production-grade. This comment reflects patterns from SwarmAI's memory architecture. Discussion: T-MEM |
…rooms Previously, when ``_tool_status`` / ``_tool_list_wings`` / ``_tool_list_rooms`` hit ``STATUS_SCAN_LIMIT`` they returned ``"truncated": "Wing breakdown sampled from first 5000 of N drawers."`` — a human-readable sentence the model had to parse to know how partial the view was. Returning structured fields lets the model compute coverage directly without prose parsing: * ``truncated: True`` (bool) — explicit flag, easy to gate on. * ``scanned: N`` (int) — exact number of drawers the breakdown is computed from. * ``total_drawers: N`` (int) — palace total, the 100% reference. Already present in ``_tool_status`` unconditionally; now also surfaced in ``_tool_list_wings`` when truncated so the model can compute ``scanned / total_drawers`` without a second tool call. ``_tool_list_rooms`` intentionally omits ``total_drawers`` when truncated: ChromaDB's ``count()`` doesn't support ``where=`` filtering in the pinned version, so we can't cheaply give an exact wing total. The model still has ``truncated`` + ``scanned`` to know the view is partial; documented why ``total_drawers`` is absent at the call site. Four new tests cover the structured shape: ``truncated`` absent under the cap, ``truncated/scanned`` present and exact when forced, and ``total_drawers`` present for status/list_wings but absent for list_rooms.
raman325
added a commit
to raman325/hermes-mempalace-mcporter
that referenced
this pull request
Jun 6, 2026
…atch Hermes' ``agent.memory_manager._register_provider`` at line 285 of hermes-agent's source builds the ``tool_name → provider`` routing table by snapshotting ``provider.get_tool_schemas()`` once at registration time, then never re-queries. If we return ``[]`` because ``_initialized`` hasn't been flipped to True yet, the dispatcher learns zero tool names and every later call to ``mempalace_status`` / ``mempalace_search`` / etc. returns the dispatcher's ``"Unknown tool: <name>"`` error without ever reaching our ``handle_tool_call``. This was the original symptom openclaw reported: tools visible in its schema (because ``get_all_tool_schemas()`` at line 388 IS queried dynamically per turn for the system prompt) but uncallable. Fix: schemas describe the *interface*, not runtime readiness. Always return the 8 schemas (except under cron, which is a hard "no tools at all" gate). ``handle_tool_call`` already checks ``_initialized`` and returns a structured ``"MemPalace MCP not initialized."`` error when the backend is down, so the model still gets a clear signal — it just flows through our handler now instead of being silently lost by the dispatcher. Verified live: ``before init: 8 schemas`` on the deployed plugin. This same fix is needed in Phase 1 (PR MemPalace/mempalace#1684) which has the identical ``if self._cron_skipped or not self._initialized`` gate.
…mes dispatch
Hermes' ``agent.memory_manager._register_provider`` snapshots
``provider.get_tool_schemas()`` once at registration time to build the
``tool_name → provider`` routing table; it never re-queries. If we
return ``[]`` because ``_initialized`` is still False (initialize runs
*after* registration), the dispatcher learns zero tool names and every
later call returns the dispatcher's ``"Unknown tool: <name>"`` error
without reaching our ``handle_tool_call``.
Live reproduction from the Phase 2 plugin (same gate, same code path)
showed exactly this: tools visible in the agent's prompt-time schema
list (because ``get_all_tool_schemas()`` IS re-queried per turn for the
system prompt) but every call uncallable.
Fix: schemas describe the *interface*, not runtime readiness. Always
return the 8 schemas (except under cron, which is a hard "no tools at
all" gate). ``handle_tool_call`` already checks ``_initialized`` and
returns ``{"error": "MemPalace not initialized."}`` when the backend
is down, so the model still gets a clear signal — it just flows
through our handler now.
This matches the discipline in the in-tree honcho provider, which only
gates schemas on cron + recall_mode, never on init state.
Live reproduction in the companion Phase 2 plugin (which shares this tool surface): an agent asked to update an existing drawer correctly identified ``mempalace_add_drawer`` / ``mempalace_update_drawer`` / ``mempalace_delete_drawer`` as the operations it needed, found none of them exposed, and fell back to shelling out via ``npx mcporter call``. Suboptimal — the user shouldn't have to leave the tool layer for ordinary memory management. Mirrors mempalace's own reference openclaw skill exactly (``mempalace/integrations/openclaw/SKILL.md``) — the maintainer's vetted shape for an agent surface. 11 new tools: * Drawer ops: ``add_drawer`` (auto-tags ``added_by="hermes"``), ``delete_drawer``, ``check_duplicate`` * Knowledge graph: ``kg_invalidate`` (palace-protocol step 5), ``kg_timeline``, ``kg_stats`` * Structure / spec: ``get_taxonomy``, ``get_aaak_spec`` * Room graph: ``traverse``, ``graph_stats``, ``find_tunnels`` Intentionally NOT exposed (matches openclaw's omissions): * ``update_drawer`` / ``list_drawers`` / ``get_drawer`` — append-first design; navigate via ``search`` and supersede with new adds rather than editing. * ``create_tunnel`` / ``list_tunnels`` / ``delete_tunnel`` / ``follow_tunnels`` — tunnels are created by mining; agents only discover them via ``find_tunnels`` / ``traverse``. * ``sync`` / ``hook_settings`` / ``reconnect`` — admin operations. * ``memories_filed_away`` — internal. Dispatch for the 11 new tools delegates to ``mempalace.mcp_server.tool_*`` — the same entry points the MCP server exposes. These share mempalace's own ``MempalaceConfig`` for ``palace_path`` resolution rather than this plugin's ``self._palace_path``. Documented as a known asymmetry; the original eight tools still honor the plugin's path, but the new eleven follow mempalace's tool-server boundary. In the common case (where the user hasn't customised ``palace_path`` away from ``~/.mempalace/palace``) both resolve to the same place. README updated to show all 19 tools and explain the omitted ones; tests pin the exact 19-name set so any future addition / removal is a deliberate edit.
…not curated Following up on the prior commit's "mirror openclaw" rationale: investigated MemPalace#491 (the PR that introduced ``integrations/openclaw/SKILL.md``) and discovered the rationale was wrong. * At MemPalace#491 merge time (Apr 2026): mempalace had 19 ``tool_*`` functions; the skill covered all 19. * As of today: mempalace has 32 ``tool_*`` functions; the skill still covers only the original 19. The "intentionally omitted" tools I justified as deliberate design (drawer CRUD beyond add, tunnel management) **didn't exist yet** when openclaw was written. The skill is a stale snapshot; carrying its omissions forward handicaps Hermes without principle. Restored the 8 agent-facing tools openclaw missed by accident of timing — drawer CRUD (``update_drawer`` / ``list_drawers`` / ``get_drawer``), tunnel management (``create_tunnel`` / ``list_tunnels`` / ``delete_tunnel`` / ``follow_tunnels``), and session-level (``memories_filed_away``). Still omitted (genuine admin, not stale coverage): * ``sync`` — mines a project directory into the palace; writes to disk; reserve for user-initiated terminal commands. * ``hook_settings`` / ``reconnect`` — admin operations. Dispatch refactored — the 19 new-style tools (those delegating to ``mempalace.mcp_server.tool_*``) now route through a small helper ``_dispatch_mcp_passthrough`` that derives the function name from the tool name (``mempalace_X`` → ``tool_X``) with one explicit remap (``mempalace_traverse`` → ``tool_traverse_graph`` per mempalace's naming). Keeps ``handle_tool_call`` under the McCabe complexity ceiling and makes adding tools a one-line change in the allowlist.
4 tasks
Member
|
This is one of the more substantial agent-support contributions, and the Hermes direction is worth preserving. I reviewed it during the community merge pass, but did not include it in the clean integration branch because it still needs a rebase and some architectural tightening. Main blockers I see:
A mergeable revision would probably be smaller in two PRs: first the Hermes MCP/provider core with tests, then backfill/install docs. The core idea is good; I just want it to land through the same local-first, verbatim, backend-safe paths the rest of MemPalace now uses. |
igorls
pushed a commit
that referenced
this pull request
Jun 30, 2026
The openclaw skill was last updated when mempalace exposed 19 MCP tools. Since then 13 more agent-facing tools have landed; this PR documents the 8 that openclaw should expose so agents can call them natively instead of falling back to `npx mcporter call ...`: Search & Browse: - mempalace_list_drawers (paginated drawer listing) - mempalace_get_drawer (fetch a single drawer by id) Palace Graph: - mempalace_create_tunnel (explicit cross-wing link) - mempalace_list_tunnels (enumerate explicit tunnels) - mempalace_delete_tunnel (remove an explicit tunnel) - mempalace_follow_tunnels (walk explicit tunnels from a room) Write / Session: - mempalace_update_drawer (mutate content or relocate a drawer) - mempalace_memories_filed_away (ack the silent auto-save hook) The 3 admin-only tools (mempalace_sync, mempalace_hook_settings, mempalace_reconnect) are intentionally left out — they're host/admin operations, not agent-facing memory operations. The Hermes MemoryProvider plugin landing in #1684 makes the same call. Version bumped 3.3.0 -> 3.4.0 (additive tool surface, no breaking changes to existing tool docs).
jphein
added a commit
to techempower-org/mempalace
that referenced
this pull request
Jul 2, 2026
…s) (#369) * feat(miner): add C# and .NET file extensions to READABLE_EXTENSIONS Adds .cs, .csproj, .sln, .razor, and .cshtml so C#/.NET projects are indexed by the project miner. .razor/.cshtml are analogous to the already-supported .jsx/.tsx. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(miner): add support for Swift and Kotlin file extensions - Updated READABLE_EXTENSIONS in miner.py to include ".swift", ".kt", and ".kts". - Added tests in test_miner.py to ensure scanning includes Swift and Kotlin files. * feat: add Pi agent JSONL session normalizer Add _try_pi_jsonl parser for Pi agent session files stored at ~/.config/pi/agent/sessions/{encoded-cwd}/{timestamp}_{uuid}.jsonl. Uses type "message" entries with role "user"/"assistant". Skips toolResult messages, model_change, thinking_level_change, and other operational events. Requires session header (type "session" with "version" key) to avoid false positives. Format documented at github.com/badlogic/pi-mono session.md and verified via Context7. Sample data provided by tunnckoCore in #59. Refs: #59 * feat: add Gemini CLI / AI Studio JSON session import support Adds _try_gemini_json parser to normalize.py for three layouts: 1. Gemini API contents format (~/.gemini/sessions/*.json): {"contents": [{"role": "user", "parts": [{"text": "..."}]}, ...]} 2. Messages-wrapper variant: {"messages": [{"role": "user", ...}, {"role": "model", ...}]} 3. Flat top-level list with role="model". This complements the existing _try_gemini_jsonl parser (which handles ~/.gemini/tmp/<hash>/chats/session-*.jsonl with session_metadata sentinel) — JSONL covers Gemini CLI runtime sessions, JSON covers exported / Studio-saved transcripts. ## Review feedback addressed (PR #204) bgauryy review: - #1 Parser-precedence bug: _try_gemini_json runs *before* _try_claude_ai_json so the {"messages":[..., role=model, ...]} layout is no longer silently claimed by the Claude parser. The Gemini parser's has_model_role guard prevents false-positives against Claude / ChatGPT data. - #2 Layout 2a coverage: TestGeminiJson.test_messages_wrapper_format + test_messages_wrapper_does_not_get_claimed_by_claude pin the fix in place. - #3 Test conflicts with current main: rebased onto develop; tests restructured into TestGeminiJson class. - #4 tempfile/os.unlink → pytest tmp_path everywhere. - #5 elif not text → else (the elif branch was dead). - #6 Module docstring updated to mention Google AI Studio. Tests: 9 new cases in TestGeminiJson covering all three layouts, multi-part text joining, non-text part skipping, has_model_role disambiguation, dispatch-chain regression for review #1. * feat(normalize): add Continue.dev session parser Add _try_continue_json() normalizer for Continue.dev AI assistant sessions (~/.continue/sessions/*.json). Parses history array with role/content pairs, handles tool calls, system messages, and metadata. Closes #59 (partial — adds Continue.dev format support) Includes comprehensive test coverage for valid sessions, edge cases, malformed input, and unicode content. * fix: preserve collection name on MCP search retry * feat: add Cursor IDE support (hooks, plugin, skill, docs, tests) Adds first-class Cursor IDE integration alongside the existing Claude Code and Codex hook flows, so Cursor users get the same automatic diary saves, pre-compaction transcript capture, and session-start memory recall — without changing any default behaviour for existing users. What's included --------------- Cursor hook scripts (hooks/cursor/): - mempal_save_hook_cursor.sh — Stop event, counter + loop_count guard, pending-save marker consumption, background mempalace mine, followup_message emission. - mempal_precompact_hook_cursor.sh — synchronous mine before compaction, drops a pending_save marker, returns user_message. - mempal_wake_hook_cursor.sh — sessionStart event, wing-scoped recall guidance via additional_context. - lib/common.sh — shared parsing + state helpers (bash 3.2 safe, no heredoc-in-subshell traps). - install.sh — idempotent installer with --scope, --variant, --dry-run, --uninstall. Recognises existing entries by basename so re-installs across paths work. - STDIN_SHAPE.md, README.md — payload schemas + quick reference. Cursor plugin (.cursor-plugin/ + repo-root components): - plugin.json, marketplace.json, README.md. - skills/mempalace/SKILL.md — model-invocable skill mirroring the Claude plugin's skill surface. - commands/mempalace-{help,init,mine,search,status}.md — slash commands for marketplace-published installs (filename = slug). - mcp.json — auto-registers the mempalace MCP server, wrapped under the documented mcpServers key. Examples + docs: - examples/cursor/hooks.json, hooks.minimal.json + README. - website/guide/cursor-hooks.md + sidebar entry. - README.md and CHANGELOG.md updates. Tests (129 new, all green): - tests/test_cursor_hooks_shell.py — 75 behavioural tests for the three hook scripts: kill switches, input parsing, counter logic, loop prevention, pending markers, wing inference, logging. - tests/test_cursor_hooks_install.py — 19 contract tests for the installer: dry-run, idempotent merge, basename-matched uninstall, refusal to overwrite malformed JSON. - tests/test_cursor_plugin_manifest.py — 35 contract tests for the plugin: manifest validity, version sync with mempalace.version, mcp.json shape, skill/command frontmatter, default-discovery layout invariants. Design notes ------------ - Local-first and zero-API by default; hooks never call external services. Same privacy model as the existing Claude Code hooks. - Fail-open: hook scripts deliberately do not use set -e so a broken hook can never block the user's conversation. - Cursor preCompact cannot block + return a followup, so we synchronously mine the transcript and drop a pending_save marker that the next stop hook consumes — guarantees verbatim capture before context window compression. - Cursor's default plugin discovery requires real commands/, skills/, and mcp.json at the plugin root (verified against the cached cloudflare plugin); .cursor-plugin/{commands,skills} are convenience symlinks back to those canonical locations. - bash 3.2 compatibility throughout: avoids heredoc-in-command- substitution parser bugs; uses python -c for JSON parsing; basename-matched entry recognition in install.sh. - All changes are additive. No existing files are removed, no existing hooks change behaviour, and no new runtime dependencies are introduced. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cursor): address gemini-code-assist review on PR #1632 Five fixes from the Gemini Code Assist review on https://github.com/MemPalace/mempalace/pull/1632 — three real bugs, two cleanups, all consistent with the bash-3.2-compatibility contract documented in the original commit. Bug fixes (high) ---------------- 1. hooks/cursor/lib/common.sh — config.json kill-switch check used a `python3 - <<'PYEOF' ... PYEOF` heredoc inside a `$(...)` command substitution. The heredoc body contains parens which trips the macOS bash 3.2.57 parser bug. Replaced with a `python -c '...'` call passing the config path as argv[1]. Matches the pattern already used in mempal_parse_stdin in the same file. 2. hooks/cursor/install.sh — a relative `--install-dir` was written verbatim into hooks.json. Cursor invokes hook commands from its own working directory (typically the project root), so a relative command path would silently fail to launch the hook. Now resolved to an absolute path against `$PWD` before being baked in. 3. hooks/cursor/mempal_save_hook_cursor.sh — `MEMPAL_SAVE_INTERVAL=0` would crash bash on `$((NEXT % 0))` (division by zero). Extended the existing sanitiser case to coerce 0 to the default interval alongside empty / non-numeric values. Cleanups (medium) ----------------- 4. hooks/cursor/install.sh — the EMPTY_CHECK_PY temp file is now inlined as `python -c '...'`. Removes a small leak window (tmpfile would linger if the script were interrupted between mktemp and rm -f) and shortens the script. 5. hooks/cursor/install.sh — `mktemp -t prefix` has subtly different semantics on BSD (macOS) vs GNU mktemp. Switched to the portable absolute-template form `mktemp "${TMPDIR:-/tmp}/...XXXXXX"` which behaves identically on both. Regression tests ---------------- - tests/test_cursor_hooks_shell.py test_save_interval_zero_is_coerced_to_default — guards fix #3. - tests/test_cursor_hooks_install.py — new TestInstallDirAbsolutePath class: test_relative_install_dir_is_absolutized_in_hooks_json — guards fix #2 against regression. test_absolute_install_dir_is_preserved_verbatim — guards that the relative-to-absolute resolution does not mangle paths that were already absolute. Verification ------------ - bash -n on all three edited scripts: clean. - uv run pytest tests/test_cursor_hooks_*.py tests/test_cursor_plugin_manifest.py: 132 passed (was 129; +3 regression tests). - uv run pytest tests/ --ignore=tests/benchmarks: 2399 passed, 3 skipped (pre-existing). - uv run ruff check . / ruff format --check .: clean. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cursor): address igorls review on PR #1632 Resolves the maintainer review on the Cursor IDE support PR. Cursor-only scope; cross-IDE items (wing-naming convention, shared-file merge order) are coordinated on the separate Antigravity branch. followup_message default (the one "decide before merge" item): - Keep the stop-hook followup ON by default. Cursor's transcript format is undocumented and mempalace/normalize.py has no Cursor parser, so the background `mempalace mine --mode convos` is best-effort only and does not yet yield clean verbatim drawers. The followup is therefore the load-bearing verbatim-capture path; defaulting it off would leave a default Cursor install capturing nothing. - Add an opt-out (MEMPAL_CURSOR_SILENT=1, or MEMPAL_VERBOSE=false) for users who want the Claude-style "zero tokens in chat" behaviour. The hook still mines and keeps its counters/markers when silenced. - Correct the misleading "background mine captures it" comments in the save and precompact hooks; update hooks/cursor/README.md and the guide. Hygiene fixes: - Drop the hardcoded "version" field from .cursor-plugin/plugin.json and marketplace.json (mempalace/version.py is the single source of truth); tests now assert the field stays absent. - Remove the committed .cursor-plugin/{commands,skills} symlinks (they break on Windows clones with core.symlinks=false and were redundant with the real repo-root components that `source: "."` already serves); add a guard test that no symlinks exist under .cursor-plugin/. - Document the preCompact synchronous-mine timeout tradeoff and that an incremental/append-only mine is recoverable if killed (no corruption). - Add a Cursor-namespaced, daily-throttled TTL sweep (MEMPAL_STATE_TTL_DAYS, default 30) to lib/common.sh that GCs stale cursor_*.count/.pending only, after the kill-switch check; shared logs and antigravity_* are untouched. Verification: full suite green (2424 passed, 3 skipped), ruff check + format clean, bash -n clean on all cursor scripts. +30 Cursor tests (followup opt-out, state GC, TTL validation, no-symlink/version guards). Co-authored-by: Cursor <cursoragent@cursor.com> * style(tests): apply ruff 0.4.x format to test_normalize Fixes lint CI: ruff format --check flagged blank-line and long-dict wrapping in the Continue.dev parser tests. * fix(searcher): scope neighbor expansion by parent_drawer_id (#1580) * fix(mcp): drop top-level anyOf from diary_write schema The mempalace_diary_write tool declared a top-level anyOf in its input schema to require either entry or content. Anthropic's Messages API rejects any tool schema with a top-level anyOf/oneOf/allOf and returns a 400 for the entire tools array, so every MCP session failed to start. The entry/content constraint is already enforced at dispatch: content is remapped to entry before the handler runs, and a missing value returns -32602. Removing the combinator restores compatibility without weakening validation. Closes #1711 * docs(openclaw): catch up SKILL.md with 8 newer MCP tools The openclaw skill was last updated when mempalace exposed 19 MCP tools. Since then 13 more agent-facing tools have landed; this PR documents the 8 that openclaw should expose so agents can call them natively instead of falling back to `npx mcporter call ...`: Search & Browse: - mempalace_list_drawers (paginated drawer listing) - mempalace_get_drawer (fetch a single drawer by id) Palace Graph: - mempalace_create_tunnel (explicit cross-wing link) - mempalace_list_tunnels (enumerate explicit tunnels) - mempalace_delete_tunnel (remove an explicit tunnel) - mempalace_follow_tunnels (walk explicit tunnels from a room) Write / Session: - mempalace_update_drawer (mutate content or relocate a drawer) - mempalace_memories_filed_away (ack the silent auto-save hook) The 3 admin-only tools (mempalace_sync, mempalace_hook_settings, mempalace_reconnect) are intentionally left out — they're host/admin operations, not agent-facing memory operations. The Hermes MemoryProvider plugin landing in MemPalace/mempalace#1684 makes the same call. Version bumped 3.3.0 -> 3.4.0 (additive tool surface, no breaking changes to existing tool docs). * docs(openclaw): address review round 1 - Fix mempalace_find_tunnels params: (required) -> optional. The MCP handler defaults both wing_a and wing_b to None (mempalace/mcp_server.py:1277), so the prior docs were factually wrong. Caught by gemini-code-assist on PR #1719. - Clarify implicit-vs-explicit tunnel distinction with consistent casing and a brief in-line definition (implicit = discovered from drawer content overlap; explicit = user/agent-declared link). Suggested by copilot-pull-request-reviewer. - Split the mempalace_memories_filed_away one-liner into a short description plus 'Returns' and 'When to call' sub-bullets for readability. Suggested by copilot-pull-request-reviewer. * fix: detect Java project manifests * fix: handle rootless Java subprojects * fix(mcp): fail closed when add_drawer idempotency pre-check fails * feat: add mempalace-recall skill and optional Cursor recall rule Ports the OpenClaw "search before answering" protocol to the Cursor and Claude plugin surfaces so the agent reads the palace before answering about past work, people, projects, or prior decisions instead of guessing from model memory. - integrations/shared/recall-protocol.md: single source of truth for the recall protocol, referenced by the skill and the rule so they cannot drift. - skills/mempalace-recall/SKILL.md: recall-only skill (the mempalace skill keeps setup/mine/status); cross-linked from the ops skill. - rules/mempalace-recall.mdc: plugin recall rule, alwaysApply: false so it only fires on recall-relevant turns and never adds MCP latency to greenfield work. - examples/cursor/rules/: opt-in copies for non-plugin users, including an aggressive alwaysApply: true variant documented with its latency tradeoff. - .claude-plugin/skills/mempalace-recall/SKILL.md: Claude plugin parity. - tests: assert the recall skill and rules/ discovery layout; the shipped rule must be alwaysApply: false. - docs: .cursor-plugin/README.md and the cursor-hooks guide now describe the three layers of recall (hook + skill + rule). The Antigravity plugin mirror lands as a follow-up on the antigravity branch, where .antigravity-plugin/ exists. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(repair): run post-rebuild FTS5 cleanup on legacy cmd_repair path (#1747) A clean `mempalace repair --yes` (legacy path) finished without _vacuum_and_rebuild_fts5: the bulk delete_collection + re-upsert cycle leaves the FTS5 inverted index inconsistent, so the next repair aborts at the sqlite integrity preflight. rebuild_index() got this cleanup when #1517 was fixed; cmd_repair never did. Extract the shared epilogue _post_rebuild_cleanup() (close chroma handles, then VACUUM + rebuild FTS5) and call it from both full-rebuild paths so they cannot drift apart again. Cleanup runs on the legacy success path only; failure/restore paths are unchanged. Closes #1747 Co-Authored-By: nord- <3777600+nord-@users.noreply.github.com> * test(backends): live-substrate conformance module for pgvector Mirrors the portable fake-client arms of test_pgvector_backend.py against a real PostgreSQL+pgvector server and adds live-only arms the in-memory fake cannot exercise: real <=> operator ground truth, JSONB pushdown vs local-fallback equivalence, cross-namespace isolation on real tables, 8-connection concurrent writers, and the advisory-lock serialization of run_maintenance('reindex') under a 2-connection race. Gated on MEMPALACE_PGVECTOR_LIVE_DSN (same pattern as the qdrant live gate); skips cleanly when unset. First run: 15/15 pass on PostgreSQL 16.10 + pgvector 0.8.2 (+AGE 1.6.0 in the same server), psycopg 3.3.4. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * review(gemini): marker-race stub, created-list lock, exact-order + exactly-one-ran asserts - Stub _write_marker on the 8 concurrent writer backends: upsert() rewrites the marker on every call with a plain open('w'), so backends sharing one local_path race on the same file (sharing violations on Windows) — a test-design artifact, not the contract under test - Guard the fixture's created list with a lock for the threaded tests - Assert exact distance-ordered ids in the query/filter arms - Reindex race: exactly one 'ran' (index absent beforehand, so the advisory-lock winner must build) Re-run live after changes: 15/15 pass (PG 16.10, pgvector 0.8.2). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(embedding): chunk EmbeddinggemmaONNX batches to bound ONNX memory (#1770) One session.run over a repair-scale batch (5000 docs) allocates attention buffers far beyond available RAM and the kernel kills the process. Mirror chromadb's ONNXMiniLM_L6_V2 and embed in sub-batches of 32; per-chunk padding also stops one long doc inflating the whole batch. Co-Authored-By: mojie5 <262519016+mojie5@users.noreply.github.com> * fix(embedding): lock EF cache and lazy load, guard inputs (#1770) Two threads sharing a cold EmbeddinggemmaONNX via _EF_CACHE could each build a full model session, and two factory callers could each keep a private instance. The load is now double-check locked with the session published last, and the factory cache has an atomic check-then-construct behind a lock-free fast path. __call__ wraps a bare string, returns [] for None and empty input before the lazy download, and its annotation matches the accepted types. * fix(hallways): scope hallway-file path to MempalaceConfig.palace_path (#1778) Pre-3.4 the hallway store was hardcoded at ~/.mempalace/hallways.json regardless of the configured palace_path, so two palaces on one host silently shared one file. Mining into palace-A leaked records into palace-B's hallway code paths. Apply the 3.3.6 tunnel-file migration pattern: * MempalaceConfig.hallway_file resolves to <dirname(palace_path)>/hallways.json * hallways._get_hallway_file(config) reads through MempalaceConfig * hallways._legacy_hallway_file() exposes the pre-migration hardcoded path for one-time orphan detection; _load_hallways logs a one-line warning when the legacy file exists but the configured one doesn't, matching palace_graph._load_tunnels behavior. No auto-migration — silent merging risks clobbering newer data. Atomic-write + 0600 semantics unchanged. Module-level _HALLWAY_FILE constant kept and honored when monkey-patched directly, so the three existing test sites that patch it (test_hallways.py, test_hallways_pagination.py, test_mcp_server.py) keep working without modification. New coverage in tests/test_hallways_palace_scoped.py mirrors the analogous tunnel tests: resolver default + custom palace_path + env-var redirect, orphaned-legacy warning + no-warning when paths match, and an end-to-end multi-palace isolation regression guard. Closes #1778 * fixup(hallways): drop _HALLWAY_FILE back-compat shim, migrate existing tests to resolver Replaces the back-compat shim in _load_hallways/_save_hallways (which honored direct monkey-patches of the _HALLWAY_FILE module constant) with a clean single-source-of-truth resolver, matching the palace_graph tunnel-file migration in 3.3.6. The three existing test sites (tests/test_hallways.py, tests/test_hallways_pagination.py, tests/test_mcp_server.py) now monkey-patch _get_hallway_file and _legacy_hallway_file directly, exactly mirroring the helper in tests/test_palace_graph_tunnels.py. Production code now has one branch through the path resolution instead of two. No behavior change. 269/269 hallway + tunnel + mcp-server tests pass on Python 3.11 and 3.12, ruff clean. * fixup(hallways): address gemini-code-assist review on PR #1780 Two catches on tests/test_hallways_palace_scoped.py TestMultiPalaceIsolation.test_save_then_load_under_different_palace_returns_empty: 1. Stale comment referencing the removed _HALLWAY_FILE back-compat shim (deleted in the prior fixup commit). Removed. 2. _legacy_hallway_file was not monkey-patched, so the test isolation gap let _load_hallways check the host's real ~/.mempalace/hallways.json when evaluating the legacy-warning branch. Now patched to a tmp_path sibling, matching the helper pattern used in test_palace_graph_tunnels. * fix(ids): use length-prefixed recipe v3 * fix(mcp): treat chunked drawers as logical drawers * fix(mcp): avoid mutating drawer metadata * fix(ids): simplify v3 length-prefixed hashing * chore(deps): bump docker/metadata-action from 5 to 6 Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 5 to 6. - [Release notes](https://github.com/docker/metadata-action/releases) - [Commits](https://github.com/docker/metadata-action/compare/v5...v6) --- updated-dependencies: - dependency-name: docker/metadata-action dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump docker/build-push-action from 6 to 7 Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6 to 7. - [Release notes](https://github.com/docker/build-push-action/releases) - [Commits](https://github.com/docker/build-push-action/compare/v6...v7) --- updated-dependencies: - dependency-name: docker/build-push-action dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump docker/login-action from 3 to 4 Bumps [docker/login-action](https://github.com/docker/login-action) from 3 to 4. - [Release notes](https://github.com/docker/login-action/releases) - [Commits](https://github.com/docker/login-action/compare/v3...v4) --- updated-dependencies: - dependency-name: docker/login-action dependency-version: '4' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * fix(hooks): normalize Windows transcript paths in shell hooks * fix(backends): serialize first connect in sqlite_exact and pgvector (#1774, #1775) Co-Authored-By: jphein <19301265+jphein@users.noreply.github.com> * fix(hooks): preserve fail-loud parse diagnostics * fix(hooks): typo regression addressed * fix(tests): run fact_checker __main__ via subprocess to clear runpy warning `tests/test_fact_checker.py` imports symbols from `mempalace.fact_checker` at module top (putting it in sys.modules), then `TestCLI.test_exits_nonzero_when_ issues_found` re-executed the same module as __main__ via `runpy.run_module("mempalace.fact_checker", run_name="__main__")`. runpy warns because it re-runs an already-imported module against a half-initialized state: RuntimeWarning: 'mempalace.fact_checker' found in sys.modules after import of package 'mempalace', but prior to execution of 'mempalace.fact_checker' Run the CLI in a fresh process via `subprocess.run([sys.executable, "-m", "mempalace.fact_checker", ...])` instead — no sys.modules collision, and it exercises the real `python -m` entry point. Assertions are preserved (SystemExit code 1 → returncode 1; captured stdout substring → result.stdout). The child's entity registry (`~/.mempalace/known_entities.json`, resolved via expanduser at import) is redirected by overriding both HOME and USERPROFILE in the subprocess env so it works on POSIX and Windows. Verified: `pytest tests/test_fact_checker.py -W error::RuntimeWarning` passes (26) with the warning promoted to error — proving it no longer fires. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(mcp): avoid Chroma open when cached DB disappears * test: stabilize release validation on develop * fix(palace): clean source mine locks safely * fix: close blob seq sqlite migration connection * fix: address mine lock review feedback * test: stabilize closet boost fixture on Windows * chore(release): 3.4.1 Bump version across all sources (version.py, pyproject.toml, both Claude plugin manifests, Codex plugin manifest, README badge, uv.lock) and promote the Unreleased changelog to 3.4.1. Shipping: Cursor IDE plugin + hooks, first-class Antigravity IDE support (with zero-config interpreter resolution), embeddinggemma bulk re-embed OOM fix, and backup-retention pruning. Also rebuilds the CHANGELOG compare-link block, which had been left at v3.2.0: adds the full 3.3.0-3.4.1 chain plus the previously undocumented 3.4.0, and points Unreleased at v3.4.1...HEAD. Every version header now resolves to a compare link. * fix(hooks): portable mtime in macOS hook throttles; doc cleanup Address review feedback surfaced on the 3.4.1 release promotion (#1810). Bug fix — `date -r FILE` is GNU-only. On BSD/macOS `date -r` expects epoch seconds, not a path, so the staleness/throttle checks in the new Cursor and Antigravity hooks silently failed on macOS: the state GC swept on every fire and the pending-save guard was skipped. Replace with a portable `os.path.getmtime` one-liner via the already-resolved $MEMPAL_PYTHON_BIN (cursor/lib, antigravity/lib, antigravity save hook). This restores the "bash 3.2.57 / macOS default" compatibility the Antigravity changelog claims. Docs: - Correct the MCP tool count to 33 (was 19/29/31 in 21 places across plugin manifests, READMEs, and website docs — all drifted from the TOOLS dict / mcp-tools.md reference, which both have 33). - Fix broken CHANGELOG link to the Cursor skill (skills/, not .cursor-plugin/skills/). - Fix one-too-many `../` in skills/mempalace/SKILL.md's cursor-hooks link (resolved above the repo root). - Add the required `mcpServers` wrapper to the mcp.json example in .cursor-plugin/README.md so copy-paste yields a valid Cursor config. Left intentionally unchanged: the os.dup2 fd-1 redirect in mcp_server.py is deliberate (#225 keeps JSON-RPC off fd 1). * style(hooks): single-quote the static python -c mtime snippet The snippet has no shell interpolation — the path arrives via argv, not string interpolation — so single quotes are correct and make it unambiguous that nothing is shell-expanded. Behavior is identical: `sys.argv[1]` contains no `$`, so it was never expanded (verified empirically). Matches the single-quoted `python -c` blocks already in hooks/cursor/lib/common.sh. No functional change. * test(migrate): cover swap-failure rollback Adds end-to-end regression coverage for the migration swap path where os.replace hits EXDEV, the shutil.move fallback fails, and the original palace must be restored from the rename-aside copy. * feat(mcp): add mempalace_delete_by_source bulk-cleanup tool (#1722) Adds an MCP tool to remove every drawer mined from a given source_file exact match, for cleaning up benchmark/test data accidentally mined into a user wing (ShareGPT dumps, results_mempal_*.jsonl, language config JSON) that drowns out real memories in semantic search. Matching is pushed to the backend via delete(where={"source_file": ...}) the same idiom the miner and diary-ingest paths already use so it is not subject to the SQLite variable limit regardless of how many drawers share the source. Defaults to a dry run reporting match count and a sample; dry_run=false commits. Absent source is an idempotent no-op, not an error. * fix(mcp): harden delete_by_source per review — strip surrogates + type guard Address Gemini review on #1729: - normalize source_file with strip_lone_surrogates so exact matching hits rows mined from non-ASCII paths via cp1252 stdin (#1488), mirroring tool_add_drawer's ingestion-side normalization - isinstance(str) guard so a non-string source_file returns a clean error instead of AttributeError - default missing wing/room to "" in the dry-run sample, consistent with the rest of the file - add tests: non-string rejection + surrogate-normalization match * feat(miner): add PHP ecosystem file extensions * fix(claude-plugin): run final mine on SessionEnd * fix reviewer feedback: To prevent a KeyError and provide a clear, actionable assertion failure message if PreCompact is ever missing * fix(chroma): route stale hnsw divergence to sqlite fallback * fix reviewer feedback for chroma and tests * fix(mcp): refuse second writer for same palace * fix(mcp): cache writer lock setup failures * feat: add opt-in local daemon for queued MemPalace writes - New mempalace/daemon.py: long-lived localhost HTTP server (127.0.0.1) with a SQLite WAL job queue, single worker thread, bearer-token auth, and owner-only file perms (0600/0700) on queue DB, token, endpoint, and log. - New mempalace/service.py: transport-neutral job execution surface shared by the daemon, with per-job env isolation so one job's backend/palace switch cannot leak into the next. mcp_tool is allowlisted to write-classified tools only. - Crash recovery re-queues jobs left 'running' by a killed daemon; jobs that already exhausted MAX_ATTEMPTS are dead-lettered to 'failed' instead of being retried (non-idempotent diary_write would otherwise duplicate verbatim content on every restart). - Bounded retention prunes terminal jobs older than 7 days (MEMPALACE_DAEMON_RETENTION_DAYS); queued/running jobs are never touched so a crash mid-prune cannot drop in-flight work. - CLI: --daemon/--background on mine/sync submit to the queue; new `mempalace daemon {start,stop,status,jobs,wait}` subcommand. Strictly opt-in: no flag, env, or config means no daemon and no behavior change. - Hooks opt in via MEMPALACE_HOOKS_DAEMON or config hooks.daemon; when the daemon is not already running, hooks fall back to the existing direct/spawn path so the 500ms hook budget is preserved (hooks never auto-start the daemon). - service.run_sync renders the same operator-facing report shape as the direct CLI sync path (no_source, out_of_scope, by_source, Re-run/Removed hints) and drops the old KeyError-prone 'deleted' read. * fix(mcp): gate startup on sqlite integrity failures * chore(deps-dev): bump ruff from 0.15.15 to 0.15.18 Bumps [ruff](https://github.com/astral-sh/ruff) from 0.15.15 to 0.15.18. - [Release notes](https://github.com/astral-sh/ruff/releases) - [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md) - [Commits](https://github.com/astral-sh/ruff/compare/0.15.15...0.15.18) --- updated-dependencies: - dependency-name: ruff dependency-version: 0.15.18 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * fix(mcp): applied 3 of the 4 reviewer suggestions * fix(mcp): guard remaining None palace_path in _mcp_sqlite_integrity_refusal. Added one regression test calling the function directly with palace_path=None. * fix: unblock daemon PR CI + address review comments CI was red on all three platforms for the daemon-mode draft PR. Root causes and fixes: - Linux 3.9 collection error: `_submit_daemon_job`'s `dedupe_key: str | None` parameter annotation is evaluated at def time, and hooks_cli.py has no `from __future__ import annotations` — `str | None` raises TypeError on 3.9. Reverted to `dedupe_key: str = None` (the original, 3.9-safe). The other `int | None` in the file is a function-local annotation, which is never evaluated, so it was never the problem. - macOS/Windows daemon lifecycle flakes: the 3 HTTP-lifecycle tests failed at the 10s readiness deadline on contended CI runners (localhost bind is sub-second locally but took ~5s when it passed on the macOS fleet, >10s when it didn't), and because the server thread never shuts down on timeout, run_server's `os.environ["MEMPALACE_PALACE_PATH"]` + `os.umask(0o077)` mutations leaked into the rest of the suite — poisoning every later test that reads MempalaceConfig().palace_path (the 60+ test_mcp_server cascade on macOS; the at-exit socket hang → SIGINT on Windows). Bumped the readiness deadline to 30s and added a module-scoped snapshot + autouse fixture in test_daemon.py that force-restores the env + umask to the pre-suite baseline after every daemon test, so a leaked server thread can't poison other test files. Gemini review comments (fixed in code, no thread replies per convention): - daemon.py `_connect()` was a bare `sqlite3.connect` whose `with`-block only managed the transaction, not the connection — an unbounded FD leak in a long-lived daemon running thousands of jobs (also the source of the Windows "unclosed database" ResourceWarning noise). Converted to a closing @contextlib.contextmanager. - `QueueStore.finish()` gained `only_if_running`; `_safe_finish` passes it so a late worker finish can't overwrite a shutdown-cancelled job back to succeeded/failed — removes the reliance on process-exit timing. - `DaemonClient.request` wraps the final `json.loads` in try/except JSONDecodeError → DaemonError, so a non-JSON 2xx response surfaces as a structured error instead of a bare JSONDecodeError. - test_sync.py: removed the module-level `import mempalace.mcp_server` and moved the stdout-rebinding side effect into an autouse fixture scoped to TestServiceRunSyncReport, so the embedder/Chroma import chain is no longer forced at collection time for the existing sync tests. Coverage: added focused happy-path tests for service.run_sync early-returns, run_mine backend application + invalid mode, execute_job kind dispatch, run_diary_write arg forwarding, run_mcp_tool write-tool dispatch, and print_job_result — lifts service.py from 57% to 85% so the new files (service 85%, daemon 80%) don't drag the total below the 80% CI gate now that the daemon tests complete and the gate is actually evaluated. * fix: daemon client bypasses proxy discovery; tests force-shutdown server thread DaemonClient.request now uses a no-proxy opener (build_opener(ProxyHandler({}))) instead of urllib.urlopen. The daemon is always on 127.0.0.1, so a request must never go through an HTTP proxy — this is the correct production choice. It also bypasses urllib's proxy discovery (macOS _scproxy via SystemConfiguration), which runs on the first request to any host and is NOT bounded by the per-request timeout: on a CI runner with no network it hangs for tens of seconds, which looked exactly like the daemon never came up (test_daemon_http_lifecycle_executes_job timed out at 30.18s). With the no-proxy opener the lifecycle runs in 0.78s and no server thread is leaked — which also removes the timing skew that made the sqlite_exact concurrent-connection test flake on macOS CI. The leaked server thread was also the Windows exit-hang root cause: a slow/failed client.shutdown() POST left serve_forever running, and the interpreter blocked on the open listening socket at process exit. Tests now capture the httpd run_server creates (by subclassing daemon.ThreadingHTTPServer) and force httpd.shutdown() + server_close() from the test thread if the normal shutdown path leaves the thread alive, asserting the thread died so a leak becomes a visible failure instead of a silent exit hang. * test: win32-only diagnostic for daemon process-exit hang The Windows CI run passes all 666 tests then hangs at interpreter shutdown (KeyboardInterrupt at socket.py:723) until the runner kills it. All daemon lifecycle tests assert their server threads died, so the hang is a different non-daemon thread blocked on a socket — not the daemon server thread. CI round-trips can't show which thread it is. Add a win32-only session fixture that: - arms faulthandler.dump_traceback_later(130s) to print every thread's stack to stderr once the hang has run a while, and - prints every live thread (name + daemon flag) at session teardown — a non-daemon thread present there is the shutdown blocker. Gated to sys.platform == 'win32' so Linux/macOS CI see no extra output. Remove once the Windows hang is fixed. * fix(daemon): Windows-safe pid liveness probe; finalize cross-platform daemon tests _pid_alive used os.kill(pid, 0) as an existence check. On Windows signal 0 is signal.CTRL_C_EVENT, so Python routes it to GenerateConsoleCtrlEvent and sends a console Ctrl-C to the target's process group rather than probing the pid. DaemonClient polls a same-process endpoint during startup, so on a CI runner with an attached console that Ctrl-C was delivered back to the interpreter as a spurious KeyboardInterrupt — the Windows CI hang that interrupted the suite at the first daemon HTTP-lifecycle test (socket.py recv). Probe via the Win32 OpenProcess/WaitForSingleObject handle API instead, which has no signalling side effects. This is also a real Windows production bug, not just a test artifact. Tests: - Skip the two owner-only (0600) permission tests on Windows: os.chmod cannot represent POSIX mode bits there (files report 0o666); the daemon relies on user-profile ACLs on Windows. - _start_server now captures and re-surfaces a run_server thread crash instead of spinning for 30s and failing with a bare assert (diagnoses the macOS startup flake). - Add a regression test asserting _pid_alive is correct and emits no console control event when hammered like the poll loop. - Remove the temporary win32 exit-hang diagnostic fixture from conftest now that the root cause is fixed. * fix(daemon): skip reverse-DNS in server_bind so startup can't block ~30s HTTPServer.server_bind() resolves server_name via socket.getfqdn(host). For the daemon's 127.0.0.1 bind that lookup is pointless, and on a host with slow or absent reverse DNS it blocks startup until the resolver times out (~30s) — which looks exactly like the daemon never coming up. This is why the first daemon HTTP-lifecycle test timed out on the macOS CI runner (httpd_bound=False after 30s) while every later one bound in seconds once the OS had cached the negative lookup. Bind via TCPServer directly and set server_name from the literal host. * feat(hooks): add a budget-safe SessionEnd save hook for clean exits (#1341) Short sessions that exit cleanly below SAVE_INTERVAL and without a PreCompact were never saved. Add a SessionEnd hook that takes one final flush. Claude Code budgets SessionEnd hooks at 1.5s and a plugin-provided timeout cannot raise it, and a cold mempalace start exceeds that, so the wrapper backgrounds the work and returns immediately; the detached child completes the transcript ingest, project mine, and diary checkpoint after the session exits. The handler validates transcript_path through _validate_transcript_path before any ingest or diary write, so a traversal or wrong-suffix path is rejected while the independent project mine still runs. Adds hook_session_end, both shell wrappers, the plugin hooks.json entry, the session-end CLI choice, and focused tests. (cherry picked from commit 10e1450e04fc7cec72984ab7442d3b4fca1490e8) * fix(claude-plugin): resolve SessionEnd merge semantics * fix(claude-plugin):reviewer feedback for _validate_transcript_path function calls Path.resolve(), which can raise an OSError * fix(daemon): address post-merge review feedback on #1826 Five fixes from the Copilot review of the merged daemon PR: 1. Privacy: the queue DB's SQLite WAL/SHM sidecars hold un-checkpointed verbatim payloads but were created with the caller's umask. Set the owner-only umask in run_server BEFORE DaemonRuntime builds the QueueStore (not only once the HTTP server starts), and harden any existing sidecars in QueueStore._init_db as defense-in-depth. 2. DoS guard: reject a negative Content-Length in the request reader. rfile.read(-1) would block until the client disconnects and bypass the MAX_BODY_BYTES cap. 3. Side effects: extract _wal_log (+ _ensure_wal, _WAL_FILE, _WAL_REDACT_KEYS) into a new side-effect-free mempalace/wal.py. The CLI sync path and the daemon service layer obtained _wal_log via `from .mcp_server import _wal_log`, which runs mcp_server's import-time stdio protection (os.dup2(2, 1); sys.stdout = sys.stderr) in a non-MCP process and misroutes operator output. mcp_server/cli/service now import from mempalace.wal. 4. Correctness: run_mcp_tool treated any dict as success. Write tools that return a bare {"error": ...} (e.g. tool_create_tunnel/tool_delete_tunnel validation) were recorded as succeeded; now the "error" key infers failure. 5. Hook budget: get_client_if_running()/health() take an explicit timeout, and the hook "is the daemon up?" precheck uses a short HOOK_PROBE_TIMEOUT (0.5s) so a wedged daemon can't stall the hook for the default 5s. Adds tests/test_wal.py (import isolation + redaction) and daemon tests for the umask ordering, negative Content-Length, run_mcp_tool error inference, and the short probe timeout. * fix(backends): single-scroll bulk metadata fetch for Qdrant; bump scroll page size (#1796) * fix: apply suggested reviewer suggestions * updated tests/test_qdrant_bulk_metadata_scroll.py because of CI failure after push the 2nd commit * perf(embedding): cap ORT intra-op threads so a background mine doesn't pin every core (#1068) ChromaDB's ONNX embedder builds its InferenceSession without a thread cap, so ORT's intra-op pool defaults to the physical core count. OMP_NUM_THREADS is inert against it (ORT owns its own pool), so a background `mempalace mine` pins 4-5 cores and stacked Stop-hook fires turn the machine into a thermal event. Add an `embedding_threads` config knob (env MEMPALACE_EMBEDDING_THREADS or config.json). Unset/"auto" caps the intra-op pool at half the logical CPUs so a fresh install stays usable out of the box; a positive integer sets an exact count; 0/negative leaves ORT uncapped for users who want max throughput. The cap is applied via SessionOptions at session construction: - `_MempalaceONNX` (default minilm) overrides the `model` cached_property to rebuild the session the same way upstream does plus the cap, falling back to upstream's uncapped build if chromadb internals shift. - `EmbeddinggemmaONNX` builds its session through the shared `_intra_op_session_options()` helper. * perf(mcp): answer overview tools from the sqlite aggregate to fix large-palace timeouts (#1748, #1379) tool_status / list_wings / list_rooms / get_taxonomy paged the entire collection metadata through the chroma client (`_fetch_all_metadata`, a 1000-row offset loop), which cold-loads the HNSW index and materializes hundreds of MB of dicts. On six-figure palaces these exceed the MCP host tool-call limit (180k drawers ~3-4 min; 349k times out at 120-240s). The 5s metadata cache only dedups repeat calls — it does not stop the cold-call timeout. A correct single-query SQL cross-tab already exists (`backends.chroma._sqlite_wing_room_counts`) and is already the CLI default (`miner.status`), but the MCP tools never used it — and the MCP-side sqlite reader only ran behind the `vector_disabled` recovery path. Add `_sqlite_taxonomy()` (guards on `_is_chroma_backend()`, returns None to fall back) and wire it as the default path into all four overview tools. They now answer from one GROUP BY without touching HNSW. Non-chroma backends (qdrant, sqlite_exact) and unbootstrapped/legacy layouts fall back to the existing client path unchanged. graph_stats (also named in #1379) builds an in-memory graph via build_graph() and needs its own treatment — tracked separately. * fix(pgvector): strip NUL bytes so a transcript NUL no longer aborts the mine (#1829) PostgreSQL cannot store NUL (0x00) in text or jsonb. On the pgvector write path a NUL in `document` is rejected by psycopg ("PostgreSQL text fields cannot contain NUL (0x00) bytes") and a NUL in `metadata` becomes a JSON unicode escape the jsonb cast rejects ("unsupported Unicode escape sequence"). `_execute` re-wraps either as BackendError and `_mine_impl` re-raises, so the whole mine exits non-zero and every file after the offending one is left unmined. ChromaDB, SQLite, and Qdrant store the byte verbatim, so only pgvector hard-fails. Add a recursive `_strip_nul` helper and apply it to id, document, and metadata in `_PgVectorClient.upsert_rows`, mirroring the backend-layer sanitization `_sanitize_documents_for_chromadb` already does for lone surrogates on the same bulk-ingest paths. ids are SHA-256 hashes and metadata keys are fixed field names, so the id and key passes are no-ops in practice; only transcript-derived values change. Co-authored-by: hrabbach <181709360+hrabbach@users.noreply.github.com> * fix: address PR review feedback (preserve "unknown" label; use super().model) #1748: normalize the sqlite fast path's "?" COALESCE placeholder (and None) back to "unknown" inside _sqlite_taxonomy, so drawers missing wing/room metadata keep the client path's output contract — no observable API change for MCP clients on legacy/partial drawers. #1068: invoke the parent embedder build via super().model instead of reaching into cached_property's .func attribute, so the uncapped/fallback path survives chromadb changing `model` to a plain @property or other descriptor. * perf(mcp): sqlite fast path for graph_stats to fix large-palace timeouts (#1379) tool_graph_stats built the whole palace graph via build_graph(), which pages every metadata row (col.get limit/offset) and cold-loads the HNSW index — the remaining overview-tool timeout from #1379 (#1836 fixed status / list_wings / list_rooms / get_taxonomy but deliberately left graph_stats out, as it builds an in-memory graph rather than a flat tally). Add _sqlite_graph_stats(): one GROUP BY room, wing, hall over chroma.sqlite3, reconstructing build_graph's room_data and the same stats (total_rooms, tunnel_rooms, total_edges, rooms_per_wing, top_tunnels) with the same per-drawer filter (room present, != "general", wing present) and edge semantics (C(wings, 2) * halls per multi-wing room). Same _is_chroma_backend() guard + client-path fallback as the #1748 overview tools. Test seeds a real chroma palace mirroring the build_graph parity case in test_palace_graph, with a tripwire on graph_stats proving the fast path runs and that "general"/wing-less drawers are excluded. Idea adapted from #1381's _sqlite_graph_stats. * fix: address PR review feedback on graph_stats sqlite fast path (#1379) - Soft-fallback on any exception, not just sqlite3.Error, so an unexpected schema shape tripping the reconstruction degrades to build_graph() instead of raising — matching the sibling sqlite fast paths (Copilot). - Guard an empty/None _config.palace_path before building db_path (Gemini). - Test: tripwire _get_collection in addition to graph_stats, directly asserting the fast path never opens the chroma client / cold-loads HNSW (Copilot). * fix: percent-encode sqlite read-only URIs so spaced/special-char paths open sqlite3.connect(f"file:{db_path}?mode=ro", uri=True) mis-parses paths containing spaces or other URI-reserved characters — common in real home directories (a Windows "First Last" user folder, many macOS paths), and made worse by Windows backslashes. The database silently fails to open and the read-only fast paths fall back (or error) on those machines. Add config.sqlite_read_uri(), which percent-encodes the path via urllib.request.pathname2url (lazy-imported to keep config import light), and route every read-only sqlite reader through it: - mcp_server._tool_status_via_sqlite - searcher BM25 sqlite fallback - repair (status / scan / max-seq read paths) - backends/chroma (5 readers: counts, wing/room tally, id maps, etc.) All previously used the same naive f-string construction. Surfaced as a gemini-code-assist review note on #1837. * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * fix(mcp): route _sqlite_graph_stats through sqlite_read_uri The graph_stats sqlite reader (#1837) and the sqlite_read_uri encoding fix (#1838) landed in parallel, so _sqlite_graph_stats was the one reader left on the naive f"file:{db_path}?mode=ro" construction that mis-parses paths with spaces/special chars. Convert it too — now every read-only sqlite reader percent-encodes its path. sqlite_read_uri is already imported in mcp_server from #1838, so this is a call-site-only change. * fix(pgvector): push get(limit, offset) pagination into SQL (#1830) PgVectorCollection.get(limit=, offset=) ignored pagination at the SQL layer: scroll_rows ran SELECT ... WHERE <where> with no LIMIT/OFFSET, so get() fetched the whole table and sliced in Python. prefetch_mined_set pages the whole palace on every mine, so mining was O(N^2) in rows transferred and Python objects built as the palace grows; every other paginating caller (exporter, migrate, repair, hallways, closet_llm, miner, palace_graph) paid the same cost. Push LIMIT/OFFSET into scroll_rows/_scroll with ORDER BY id (the primary key) for stable offset pagination. get() uses the pushed path only for an unfiltered page (no ids, no where/where_document, non-negative bounds); a filtered get keeps the full-scan path because the metadata @> ... pushdown is broader than the exact _matches_where re-filter for array/object values, so that re-filter must run before pagination. Full-scroll callers pass no bound, so their SQL is unchanged. Co-authored-by: hrabbach <181709360+hrabbach@users.noreply.github.com> * fix(pgvector): replace lone surrogates so a transcript surrogate no longer aborts the mine (#1833) A lone UTF-16 surrogate (U+D800-U+DFFF) in transcript content has no UTF-8 encoding, so pgvector's bulk upsert_rows makes psycopg raise UnicodeEncodeError and the whole mine aborts, leaving later files unmined. Apply config.strip_lone_surrogates (-> U+FFFD) to id, document, and the serialized metadata JSON in upsert_rows. json.dumps(ensure_ascii=False) leaves a metadata surrogate raw in the string, so one pass over the serialized JSON covers it; NUL, by contrast, json-escapes and must be stripped before serialization (see #1829). Replace rather than drop, matching ChromaDB's document handling. Verified end to end against live Postgres + pgvector: before, a surrogate in document or metadata aborts the mine; after, it ingests and round-trips as U+FFFD. Fixes #1833 * fix(backends): push sqlite_exact get(limit, offset) pagination into SQL SQLiteExactCollection.get(limit, offset) fetched the whole collection via _rows() (SELECT ... FROM documents ORDER BY rowid, no LIMIT/OFFSET) and sliced in Python, so every paginating caller (prefetch_mined_set, status, exporter, migrate, dedup, sync, ...) re-scanned the entire table per page, making the sweep O(N^2) in rows materialized. Push LIMIT/OFFSET into the scan on the unfiltered page (no ids/where/ where_document and non-negative bounds); filtered, id, and negative pages keep the full-scan plus Python-slice path so the post-filter still runs first. SQLite requires a LIMIT before OFFSET, so an offset-only page uses LIMIT -1. ORDER BY rowid keeps pages stable. * ci: re-trigger checks (unrelated Windows closet flake) * refactor(qdrant): reuse _rows() in get_all_metadata(); fix sys.modules test pollution Addresses maintainer review on #1832 (the two non-blocking 🟡 items plus two 🟢 nits) * test(mcp_server): add missing _fetch_all_metadata delegation/fallback tests. * feat(search): add an optional source_file filter to mempalace_search (#1815) Expose source_file alongside wing/room on mempalace_search. build_where_filter generalizes to 0/1/2+ clauses and the filter threads through the main vector path, the index-mismatch fallback, the vector-disabled BM25/SQLite path, and the union lexical path so it never silently no-ops. Matching is on the exact full stored value; results now expose source_path (the full path) for round tripping, since the displayed source_file is a basename. The MCP schema gains the source_file property and a path-tolerant sanitizer rejects null bytes, lone surrogates, and overlong values. Fixes #1815 Co-Authored-By: rendigua2025-gif <253093224+rendigua2025-gif@users.noreply.github.com> * fix(mcp): reject non-string source_file with a clean error (#1815) A JSON number or boolean passed for source_file is not coerced by the string schema type, so it reached _sanitize_optional_source_file and raised AttributeError from .strip() rather than a clean validation error. Add an isinstance guard that raises ValueError, which tool_search returns as a structured error. Regression test added. * ci: re-trigger Windows (flaky closet-boost test) * fix(repair): point index-read failures to repair --mode from-sqlite (#1843) When the chromadb compactor cannot apply the WAL into the drawers HNSW segment (InternalError: Failed to apply logs to the hnsw segment writer), the legacy repair paths fail on their first Collection.count() read and advise re-mining from source files. The drawer rows are intact in chroma.sqlite3, so repair --mode from-sqlite rebuilds them; re-mining silently drops drawers added via the MCP server and diary entries that have no source file. Both legacy read-failure sites (cmd_repair and rebuild_index) now emit shared guidance pointing at the from-sqlite recovery, worded conditionally so it also covers a live server or mine still holding the palace open. Co-Authored-By: undeadindustries <9536461+undeadindustries@users.noreply.github.com> * fix: use CREATE_NO_WINDOW so Windows hook miner spawns don't flash a console (#1783) Fixes #1783 * fix: point diverged-index recovery at from-sqlite, not re-mine (#1843) A diverged HNSW index (for example after a failed chromadb compaction) leaves the drawer rows intact in chroma.sqlite3 but the vector index out of sync. Re-mining to recover silently drops drawers added through the MCP server and diary entries, which have no source file. - repair-status now recommends `mempalace repair --mode from-sqlite --archive-existing` when DIVERGED, instead of the generic `mempalace repair`, and explains why re-mining loses data. - The shared recall protocol and the recall skills (Cursor + Claude plugin) document the compactor / "Not connected" recovery path: stop the server, rebuild from SQLite, verify, restart — never repair in-process from the agent. Complements #1847 (legacy repair error messages); does not duplicate it. Does not close #1843 — MCP reconnect resilience and honest add_drawer write signalling remain open. Co-authored-by: Cursor <cursoragent@cursor.com> * docs: add Windows backup alternative to corrupt-index recovery (#1843) Gemini review on PR #1849: the optional palace backup step used the Unix-only `cp -a`, which fails on Windows. MemPalace ships on win32, so add a PowerShell `Copy-Item` alternative alongside the macOS/Linux form and note that `--archive-existing` already moves the old palace aside. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: add mempalace_checkpoint batch save tool Collapse the Cursor auto-save sequence (check_duplicate Nx + add_drawer Nx + diary_write 1x) into a single mempalace_checkpoint MCP call so the host UI renders one tool-call card and keeps its spinner up for the whole save. The new tool reuses the existing single-item handlers, so semantic dedup, idempotency, and verbatim guarantees are unchanged. - mcp_server.py: add tool_checkpoint + register mempalace_checkpoint - service.py: classify mempalace_checkpoint as a write tool - cursor save hook: followup now drives one mempalace_checkpoint call - docs: new mcp-tools.md section, help.md entry, 33 -> 34 tool count sweep - tests: checkpoint add/dedup/malformed/registry + classify_tool Co-authored-by: Cursor <cursoragent@cursor.com> * fix: harden tool_checkpoint input validation Address PR review: guard untrusted MCP client payloads in mempalace_checkpoint so a single malformed item cannot raise deep in sanitization and abort the whole batch. - coerce dedup_threshold to float - require wing/room/content to be non-empty strings (skip + record error) - validate the diary object and entry type, recording errors instead of silently ignoring a malformed diary On a genuine dedup-check error we still file the drawer rather than skip: verbatim recall is the priority and add_drawer's idempotency blocks exact duplicates. Adds tests for the non-string, dedup-error, and malformed-diary paths. Co-authored-by: Cursor <cursoragent@cursor.com> * test: update Cursor followup assertion for checkpoint tool The save-hook followup now drives a single mempalace_checkpoint call, so test_threshold_emits_followup_message must assert that tool name instead of the old add_drawer/check_duplicate/diary_write trio. Fixes the test-macos / test-linux CI failures on this branch. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): purge matching closets in delete_by_source (#1722) delete_by_source removed only the drawers, leaving the matching closets (the AAAK index layer, keyed independently by source_file) behind as stale pointers at the now-deleted source. Mirror the closet-purge step used by sync_palace / purge_file_closets: after the drawer delete, best-effort purge the closets via push-down delete(where=...) so it survives large palaces and can never abort an already-committed drawer delete. Dry run now also reports closet_match_count so the caller sees the full blast radius; commit reports closets_deleted. Adds tests that seed the closet collection directly (tool_add_drawer doesn't build closets) and assert the matching closets are purged on commit and counted on dry run. * feat(mcp): add opt-in HTTP transport * fix suggestion of reviewer to avoid a critical race condition and other fixes * test(mcp): keep HTTP transport tests Python 3.9 compatible * test(mcp): avoid subprocess flakiness in HTTP transport tests * test(mcp): bypass proxies in HTTP transport loopback tests * test(mcp): make HTTP transport loopback tests proxy-free * test(mcp): make HTTP transport tests network-free * fix(mcp): move _HTTP_REQUEST_LOCK and _HTTP_MAX_REQUEST_BYTES * fix(mcp): move _HTTP_REQUEST_LOCK * fix reviewer: handling JSON-RPC * fix(tests): rewrite test_mcp_http_transport for Python 3.9-3.13 + Windows * fix(lint): resolve 7 ruff errors in test_mcp_http_transport * fix(mcp): harden HTTP transport — DNS-rebinding guard, optional token, real tests The opt-in HTTP transport reuses the stdio dispatcher and binds loopback by default, but /mcp was unauthenticated with no protection against a malicious web page reaching a DNS-rebound localhost server, and its tests reached for Starlette/uvicorn (not project deps) so they were silently skipped in CI — the production _serve_http handler had zero coverage. Hardening: - Pin the Host header to loopback literals + the bound host on a loopback bind (DNS-rebinding defense); relaxed for a deliberately non-loopback bind, which is the operator's call and may sit behind a Host-rewriting proxy. - Reject any browser Origin that isn't a loopback origin (rebinding/SSRF guard); non-browser MCP clients omit Origin and are unaffected. - Optional bearer token via MEMPALACE_MCP_HTTP_TOKEN (constant-time compare); required on /mcp, never on /healthz so liveness probes work credential-free. - Warn loudly when bound to a non-loopback host (palace reachable from network). Testability: - Split _build_http_server() out of _serve_http() so tests bind 127.0.0.1:0 and drive the real handler over a loopback socket via stdlib http.client. - Replace the skipped Starlette reimplementation with 12 tests covering dispatch, initialize, /healthz, 404, parse-error, the 16 MiB cap, notification 202, and the Host/Origin/token rejections — no third-party deps. * ci(test-windows): retry the transient ChromaDB HNSW compaction flake ChromaDB's rust HNSW core intermittently fails compaction on Windows with "Failed to apply logs to the hnsw segment writer" during add/update — a long-standing, non-reproducible-on-Linux/macOS flake that hits different tests (test_migrate_wings, test_closets) across unrelated commits and has been turning otherwise-green release/CI runs red at random. Add pytest-rerunfailures and wire `--reruns 2 --only-rerun "Failed to apply logs to the hnsw segment writer"` into the test-windows job only. The --only-rerun scope means a real, deterministic failure still fails on the first run; only this specific transient native-dependency error is retried. The Linux and macOS jobs deliberately keep zero reruns so genuine regressions surface there loudly. * chore(release): 3.5.0 Bump version to 3.5.0 across version.py, pyproject.toml, the Claude/Codex plugin manifests, the README badge, and uv.lock. Refresh the "N MCP tools" prose from 34 to 35 (delete_by_source #1729 and checkpoint #1851 each added a tool). Add the 3.5.0 CHANGELOG entry. * fix: tighten local guards and file handling * fix: restore convo miner scan indentation * fix: green up CI for hardened file handling - ruff format llm_client.py and miner.py (lint job) - _copy_file_no_follow: close src fd if the dst open fails (no leak), and route the rebuild restore through it so backup + restore share one no-follow/regular-file path - update repair tests to assert the unified hardened copy instead of the removed shutil.copy2 calls; backup paths are now timestamped - update normalize large-file test to stub fstat (size is checked on the open fd, not via a pre-open os.path.getsize) * test(wal): cover crash-safety, idempotent setup, and redaction edge paths The write-ahead log gained its own module in v3.5.0 but sat at 82% coverage; the uncovered lines were exactly the failure/guard branches that uphold its contracts: the cache-hit early return, the restricted-FS chmod/mkdir swallow paths, and the promise that a WAL write failure is logged and never crashes the calling tool. Add five tests covering those branches plus the non-string redaction marker, bringing mempalace/wal.py to 100% and locking the crash-safety guarantees against regression. Test-only; no production change. * fix: spawn daemon with CREATE_NO_WINDOW to match hook miner (#1783) (#1857) daemon.py:_detached_kwargs was the last production spawn site still using DETACHED_PROCESS. Swap it to CREATE_NO_WINDOW, matching the hook miner's _detached_popen_kwargs fixed in #1848 — the dedicated follow-up the review bot asked for. `grep -rn DETACHED_PROCESS mempalace/` now returns zero production hits. Survivability is unchanged: CREATE_BREAKAWAY_FROM_JOB (escapes the parent Job Object's kill-on-close) plus the daemon never being attached to the launching console carry survive-terminal-close; CREATE_NEW_PROCESS_GROUP (also kept) isolates Ctrl-C/Break. CREATE_NO_WINDOW is ignored when OR'd with DETACHED_PROCESS, so this replaces the flag rather than adding it. The daemon already redirects stdout/stderr to daemon.log and reads no stdin, so it needs no console. Adds the first tests for _detached_kwargs (posix + windows, cross-platform monkeypatch of the Windows-only flag constants, mirroring the #1848 hooks_cli tests). * fix(cli): add repair rebuild-index alias (#1670) * Fix/wing slug special chars (#1852) * fix: sanitize wing slug for project dirs with special characters Project folders containing characters outside sanitize_name's set (e.g. a leading '+') leaked into the derived wing name, producing names like 'wing_+project' that config.sanitize_name rejects, silently breaking diary auto-save for that project. Add _safe_wing_slug(): collapse non-word runs to '_', trim, and fall back to 'sessions' when a name reduces to nothing. Route the three wing-derivation sites through it. Tests: unit cases for the helper plus a hypothesis property test asserting wing_<slug> always passes sanitize_name for any inp…
This was referenced Jul 2, 2026
Contributor
Author
|
closing this in favor of the two linked PRs! |
raman325
added a commit
to raman325/mempalace
that referenced
this pull request
Jul 2, 2026
Second half of the MemPalace#1684 split — stacks on the provider core branch. Review-driven changes vs the original branch: - backfill.file_exchange routes through convo_miner.file_conversation_exchange, the same canonical write path live _file_turn uses, so historical and live drawers carry identical routing, normalization, and metadata (and authored_at now reflects the session file's mtime instead of the backfill run time). - backfill.classify_wing delegates to the provider's _match_wing_by_keywords instead of maintaining a synced copy; the parity test now guards the delegation surviving path-based loading. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
raman325
added a commit
to raman325/mempalace
that referenced
this pull request
Jul 6, 2026
Second half of the MemPalace#1684 split — stacks on the provider core branch. Review-driven changes vs the original branch: - backfill.file_exchange routes through convo_miner.file_conversation_exchange, the same canonical write path live _file_turn uses, so historical and live drawers carry identical routing, normalization, and metadata (and authored_at now reflects the session file's mtime instead of the backfill run time). - backfill.classify_wing delegates to the provider's _match_wing_by_keywords instead of maintaining a synced copy; the parity test now guards the delegation surviving path-based loading. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
raman325
added a commit
to raman325/mempalace
that referenced
this pull request
Jul 6, 2026
Second half of the MemPalace#1684 split — stacks on the provider core branch. Review-driven changes vs the original branch: - backfill.file_exchange routes through convo_miner.file_conversation_exchange, the same canonical write path live _file_turn uses, so historical and live drawers carry identical routing, normalization, and metadata (and authored_at now reflects the session file's mtime instead of the backfill run time). - backfill.classify_wing delegates to the provider's _match_wing_by_keywords instead of maintaining a synced copy; the parity test now guards the delegation surviving path-based loading. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
raman325
added a commit
to raman325/mempalace
that referenced
this pull request
Jul 8, 2026
Second half of the MemPalace#1684 split — stacks on the provider core branch. Review-driven changes vs the original branch: - backfill.file_exchange routes through convo_miner.file_conversation_exchange, the same canonical write path live _file_turn uses, so historical and live drawers carry identical routing, normalization, and metadata (and authored_at now reflects the session file's mtime instead of the backfill run time). - backfill.classify_wing delegates to the provider's _match_wing_by_keywords instead of maintaining a synced copy; the parity test now guards the delegation surviving path-based loading. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
raman325
added a commit
to raman325/mempalace
that referenced
this pull request
Jul 8, 2026
…port jsonl Two verbatim violations found in review (thanks @GoXLd — both pre-existing from MemPalace#1684): _messages_to_exchanges paired a user message with the immediately following assistant message, which on tool-using turns is the tool_use stub — the real final answer was dropped and an empty assistant side landed in the palace. Tool-using turns are the common case for a coding agent. It now delegates to the live provider's _segment_turns (one segmentation implementation, same rule as classify_wing), which folds tool traffic and the final answer into the turn via _normalize_content. The .jsonl parser treated each line as a message, but usage: hermes sessions [-h] {list,export,delete,prune,optimize,repair,stats,rename,browse} ... View and manage the SQLite session store positional arguments: {list,export,delete,prune,optimize,repair,stats,rename,browse} list List recent sessions export Export sessions to a JSONL file delete Delete a specific session prune Delete old sessions optimize Reclaim disk space: merge FTS5 segments + VACUUM (no data change) repair Repair a malformed state.db schema so hidden sessions reappear stats Show session store statistics rename Set or change a session's title browse Interactive session picker — browse, search, and resume sessions options: -h, --help show this help message and exit declare -x AI_AGENT="claude-code_2-1-193_agent" declare -x BASH_COMPLETION_COMPAT_DIR="/opt/homebrew/etc/bash_completion.d" declare -x BASH_SILENCE_DEPRECATION_WARNING="1" declare -x CLAUDECODE="1" declare -x CLAUDE_CODE_CHILD_SESSION="1" declare -x CLAUDE_CODE_ENTRYPOINT="cli" declare -x CLAUDE_CODE_EXECPATH="/Users/raman/.local/share/claude/versions/2.1.193" declare -x CLAUDE_CODE_SESSION_ID="765fcf96-9a9b-4567-b6d8-91a981813efa" declare -x CLAUDE_EFFORT="high" declare -x COLORFGBG="7;0" declare -x COLORTERM="truecolor" declare -x COMMAND_MODE="unix2003" declare -x COREPACK_ENABLE_AUTO_PIN="0" declare -x DISPLAY="/var/run/com.apple.launchd.hpP35rW8cb/org.xquartz:0" declare -x EDITOR="vim" declare -x GITHUB_PERSONAL_ACCESS_TOKEN="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" declare -x GIT_EDITOR="true" declare -x GOPATH="/Users/raman/golang" declare -x GOROOT="/opt/homebrew/opt/go/libexec" declare -x GPG_TTY="/dev/ttys002" declare -x HISTCONTROL="ignoreboth" declare -x HISTFILESIZE="32768" declare -x HISTSIZE="32768" declare -x HOME="/Users/raman" declare -x HOMEBREW_CELLAR="/opt/homebrew/Cellar" declare -x HOMEBREW_PREFIX="/opt/homebrew" declare -x HOMEBREW_REPOSITORY="/opt/homebrew" declare -x INFOPATH="/opt/homebrew/share/info:" declare -x ITERM_PROFILE="Default" declare -x ITERM_SESSION_ID="w0t2p0:FC783A53-299F-4B57-97F2-CAC98B10E339" declare -x LANG="en_US.UTF-8" declare -x LC_ALL="en_US.UTF-8" declare -x LC_TERMINAL="iTerm2" declare -x LC_TERMINAL_VERSION="3.6.10" declare -x LESS_TERMCAP_md=$'\E[38;5;136m' declare -x LOGNAME="raman" declare -x LS_COLORS="no=00:fi=00:di=01;31:ln=01;36:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.gz=01;31:*.bz2=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.avi=01;35:*.fli=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.ogg=01;35:*.mp3=01;35:*.wav=01;35:" declare -x LaunchInstanceID="9ABB072B-4AD3-4E38-82F5-EA4DEDFAF2C2" declare -x MANPAGER="less -X" declare -x NODE_REPL_HISTORY="/Users/raman/.node_history" declare -x NODE_REPL_HISTORY_SIZE="32768" declare -x NODE_REPL_MODE="sloppy" declare -x NVM_BIN="/Users/raman/.nvm/versions/node/v22.22.1/bin" declare -x NVM_CD_FLAGS="" declare -x NVM_DIR="/Users/raman/.nvm" declare -x NoDefaultCurrentDirectoryInExePath="1" declare -x OLDPWD="/Users/raman/projects/home-assistant" declare -x OSLogRateLimit="64" declare -x PATH="/Users/raman/projects/home-assistant/.venv/bin:/Users/raman/.local/bin:/Users/raman/.antigravity/antigravity/bin:/usr/local/var/rbenv/shims:/Users/raman/.local/bin:/Users/raman/.nvm/versions/node/v22.22.1/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/Users/raman/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/pkg/env/global/bin:/opt/X11/bin:/Library/Apple/usr/bin:/usr/local/MacGPG2/bin:/Applications/Little Snitch.app/Contents/Components:/Applications/VMware Fusion.app/Contents/Public:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Applications/iTerm.app/Contents/Resources/utilities:/Users/raman/golang/bin:/opt/homebrew/opt/go/libexec/bin:/Users/raman/.local/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/github/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/code-review/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/feature-dev/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/commit-commands/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/security-guidance/2.0.6/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/pr-review-toolkit/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/explanatory-output-style/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/pyright-lsp/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/typescript-lsp/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/php-lsp/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/csharp-lsp/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/jdtls-lsp/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/learning-output-style/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/code-simplifier/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/frontend-design/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/claude-md-management/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/claude-code-setup/1.0.0/bin:/Users/raman/.claude/plugins/cache/Mixedbread-Grep/mgrep/0.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/ralph-loop/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/playground/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/gopls-lsp/1.0.0/bin:/Users/raman/.claude/plugins/cache/anthropic-agent-skills/document-skills/9d2f1ae18723/bin:/Users/raman/.claude/plugins/cache/superpowers-marketplace/superpowers/5.0.7/bin:/Users/raman/.claude/plugins/cache/superpowers-marketplace/superpowers-chrome/1.6.1/bin:/Users/raman/.claude/plugins/cache/superpowers-marketplace/superpowers-lab/0.4.0/bin:/Users/raman/.claude/plugins/cache/superpowers-marketplace/episodic-memory/1.0.15/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/clangd-lsp/1.0.0/bin" declare -x PWD="/Users/raman/projects/mempalace" declare -x PYTHONIOENCODING="UTF-8" declare -x RBENV_ROOT="/usr/local/var/rbenv" declare -x RBENV_SHELL="bash" declare -x SECURITYSESSIONID="186b7" declare -x SHELL="/opt/homebrew/bin/bash" declare -x SHLVL="2" declare -x SSH_AUTH_SOCK="/var/run/com.apple.launchd.alsvaxBUwM/Listeners" declare -x TERM="xterm-256color" declare -x TERMINFO_DIRS="/Applications/iTerm.app/Contents/Resources/terminfo:/usr/share/terminfo" declare -x TERM_FEATURES="T3CwLrMSc7UUw9Ts3BFGsSyHNoSxFP" declare -x TERM_PROGRAM="iTerm.app" declare -x TERM_PROGRAM_VERSION="3.6.10" declare -x TERM_SESSION_ID="w0t2p0:FC783A53-299F-4B57-97F2-CAC98B10E339" declare -x TMPDIR="/var/folders/lr/8ht1xr690696dlplq8xpz_p40000gn/T/" declare -x USER="raman" declare -x VIRTUAL_ENV="/Users/raman/projects/home-assistant/.venv" declare -x VIRTUAL_ENV_PROMPT=".venv" declare -x XPC_FLAGS="0x0" declare -x XPC_SERVICE_NAME="0" declare -x __CFBundleIdentifier="com.googlecode.iterm2" declare -x __CF_USER_TEXT_ENCODING="0x1F5:0x0:0x0" writes one SESSION object per line — real exports silently parsed to zero exchanges. The .jsonl branch now unwraps messages/turns keys exactly like the .json dict branch, while still accepting message-per-line transcripts. Also: file_exchange composes drawer text via _compose_exchange_text (byte-identical to live filing — what the dedup safety net's exact-text matching compares against), keeps assistant-only preamble segments instead of dropping them, and the verbose summary no longer reads a conditionally-unbound wing variable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
raman325
added a commit
to raman325/mempalace
that referenced
this pull request
Jul 16, 2026
Second half of the MemPalace#1684 split — stacks on the provider core branch. Review-driven changes vs the original branch: - backfill.file_exchange routes through convo_miner.file_conversation_exchange, the same canonical write path live _file_turn uses, so historical and live drawers carry identical routing, normalization, and metadata (and authored_at now reflects the session file's mtime instead of the backfill run time). - backfill.classify_wing delegates to the provider's _match_wing_by_keywords instead of maintaining a synced copy; the parity test now guards the delegation surviving path-based loading. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
raman325
added a commit
to raman325/mempalace
that referenced
this pull request
Jul 16, 2026
…port jsonl Two verbatim violations found in review (thanks @GoXLd — both pre-existing from MemPalace#1684): _messages_to_exchanges paired a user message with the immediately following assistant message, which on tool-using turns is the tool_use stub — the real final answer was dropped and an empty assistant side landed in the palace. Tool-using turns are the common case for a coding agent. It now delegates to the live provider's _segment_turns (one segmentation implementation, same rule as classify_wing), which folds tool traffic and the final answer into the turn via _normalize_content. The .jsonl parser treated each line as a message, but usage: hermes sessions [-h] {list,export,delete,prune,optimize,repair,stats,rename,browse} ... View and manage the SQLite session store positional arguments: {list,export,delete,prune,optimize,repair,stats,rename,browse} list List recent sessions export Export sessions to a JSONL file delete Delete a specific session prune Delete old sessions optimize Reclaim disk space: merge FTS5 segments + VACUUM (no data change) repair Repair a malformed state.db schema so hidden sessions reappear stats Show session store statistics rename Set or change a session's title browse Interactive session picker — browse, search, and resume sessions options: -h, --help show this help message and exit declare -x AI_AGENT="claude-code_2-1-193_agent" declare -x BASH_COMPLETION_COMPAT_DIR="/opt/homebrew/etc/bash_completion.d" declare -x BASH_SILENCE_DEPRECATION_WARNING="1" declare -x CLAUDECODE="1" declare -x CLAUDE_CODE_CHILD_SESSION="1" declare -x CLAUDE_CODE_ENTRYPOINT="cli" declare -x CLAUDE_CODE_EXECPATH="/Users/raman/.local/share/claude/versions/2.1.193" declare -x CLAUDE_CODE_SESSION_ID="765fcf96-9a9b-4567-b6d8-91a981813efa" declare -x CLAUDE_EFFORT="high" declare -x COLORFGBG="7;0" declare -x COLORTERM="truecolor" declare -x COMMAND_MODE="unix2003" declare -x COREPACK_ENABLE_AUTO_PIN="0" declare -x DISPLAY="/var/run/com.apple.launchd.hpP35rW8cb/org.xquartz:0" declare -x EDITOR="vim" declare -x GITHUB_PERSONAL_ACCESS_TOKEN="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" declare -x GIT_EDITOR="true" declare -x GOPATH="/Users/raman/golang" declare -x GOROOT="/opt/homebrew/opt/go/libexec" declare -x GPG_TTY="/dev/ttys002" declare -x HISTCONTROL="ignoreboth" declare -x HISTFILESIZE="32768" declare -x HISTSIZE="32768" declare -x HOME="/Users/raman" declare -x HOMEBREW_CELLAR="/opt/homebrew/Cellar" declare -x HOMEBREW_PREFIX="/opt/homebrew" declare -x HOMEBREW_REPOSITORY="/opt/homebrew" declare -x INFOPATH="/opt/homebrew/share/info:" declare -x ITERM_PROFILE="Default" declare -x ITERM_SESSION_ID="w0t2p0:FC783A53-299F-4B57-97F2-CAC98B10E339" declare -x LANG="en_US.UTF-8" declare -x LC_ALL="en_US.UTF-8" declare -x LC_TERMINAL="iTerm2" declare -x LC_TERMINAL_VERSION="3.6.10" declare -x LESS_TERMCAP_md=$'\E[38;5;136m' declare -x LOGNAME="raman" declare -x LS_COLORS="no=00:fi=00:di=01;31:ln=01;36:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.gz=01;31:*.bz2=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.avi=01;35:*.fli=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.ogg=01;35:*.mp3=01;35:*.wav=01;35:" declare -x LaunchInstanceID="9ABB072B-4AD3-4E38-82F5-EA4DEDFAF2C2" declare -x MANPAGER="less -X" declare -x NODE_REPL_HISTORY="/Users/raman/.node_history" declare -x NODE_REPL_HISTORY_SIZE="32768" declare -x NODE_REPL_MODE="sloppy" declare -x NVM_BIN="/Users/raman/.nvm/versions/node/v22.22.1/bin" declare -x NVM_CD_FLAGS="" declare -x NVM_DIR="/Users/raman/.nvm" declare -x NoDefaultCurrentDirectoryInExePath="1" declare -x OLDPWD="/Users/raman/projects/home-assistant" declare -x OSLogRateLimit="64" declare -x PATH="/Users/raman/projects/home-assistant/.venv/bin:/Users/raman/.local/bin:/Users/raman/.antigravity/antigravity/bin:/usr/local/var/rbenv/shims:/Users/raman/.local/bin:/Users/raman/.nvm/versions/node/v22.22.1/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/Users/raman/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/pkg/env/global/bin:/opt/X11/bin:/Library/Apple/usr/bin:/usr/local/MacGPG2/bin:/Applications/Little Snitch.app/Contents/Components:/Applications/VMware Fusion.app/Contents/Public:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Applications/iTerm.app/Contents/Resources/utilities:/Users/raman/golang/bin:/opt/homebrew/opt/go/libexec/bin:/Users/raman/.local/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/github/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/code-review/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/feature-dev/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/commit-commands/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/security-guidance/2.0.6/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/pr-review-toolkit/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/explanatory-output-style/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/pyright-lsp/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/typescript-lsp/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/php-lsp/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/csharp-lsp/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/jdtls-lsp/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/learning-output-style/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/code-simplifier/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/frontend-design/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/claude-md-management/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/claude-code-setup/1.0.0/bin:/Users/raman/.claude/plugins/cache/Mixedbread-Grep/mgrep/0.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/ralph-loop/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/playground/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/gopls-lsp/1.0.0/bin:/Users/raman/.claude/plugins/cache/anthropic-agent-skills/document-skills/9d2f1ae18723/bin:/Users/raman/.claude/plugins/cache/superpowers-marketplace/superpowers/5.0.7/bin:/Users/raman/.claude/plugins/cache/superpowers-marketplace/superpowers-chrome/1.6.1/bin:/Users/raman/.claude/plugins/cache/superpowers-marketplace/superpowers-lab/0.4.0/bin:/Users/raman/.claude/plugins/cache/superpowers-marketplace/episodic-memory/1.0.15/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/clangd-lsp/1.0.0/bin" declare -x PWD="/Users/raman/projects/mempalace" declare -x PYTHONIOENCODING="UTF-8" declare -x RBENV_ROOT="/usr/local/var/rbenv" declare -x RBENV_SHELL="bash" declare -x SECURITYSESSIONID="186b7" declare -x SHELL="/opt/homebrew/bin/bash" declare -x SHLVL="2" declare -x SSH_AUTH_SOCK="/var/run/com.apple.launchd.alsvaxBUwM/Listeners" declare -x TERM="xterm-256color" declare -x TERMINFO_DIRS="/Applications/iTerm.app/Contents/Resources/terminfo:/usr/share/terminfo" declare -x TERM_FEATURES="T3CwLrMSc7UUw9Ts3BFGsSyHNoSxFP" declare -x TERM_PROGRAM="iTerm.app" declare -x TERM_PROGRAM_VERSION="3.6.10" declare -x TERM_SESSION_ID="w0t2p0:FC783A53-299F-4B57-97F2-CAC98B10E339" declare -x TMPDIR="/var/folders/lr/8ht1xr690696dlplq8xpz_p40000gn/T/" declare -x USER="raman" declare -x VIRTUAL_ENV="/Users/raman/projects/home-assistant/.venv" declare -x VIRTUAL_ENV_PROMPT=".venv" declare -x XPC_FLAGS="0x0" declare -x XPC_SERVICE_NAME="0" declare -x __CFBundleIdentifier="com.googlecode.iterm2" declare -x __CF_USER_TEXT_ENCODING="0x1F5:0x0:0x0" writes one SESSION object per line — real exports silently parsed to zero exchanges. The .jsonl branch now unwraps messages/turns keys exactly like the .json dict branch, while still accepting message-per-line transcripts. Also: file_exchange composes drawer text via _compose_exchange_text (byte-identical to live filing — what the dedup safety net's exact-text matching compares against), keeps assistant-only preamble segments instead of dropping them, and the verbose summary no longer reads a conditionally-unbound wing variable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
igorls
pushed a commit
that referenced
this pull request
Aug 11, 2026
The Hermes provider from feat/hermes-integration, split out per review on #1684 — provider + tests only; backfill, the hermes install CLI, and docs follow in a stacked PR. Changes vs the original branch: - _file_turn routes through convo_miner.file_conversation_exchange() instead of a hand-rolled col.upsert, so live turns carry canonical drawer metadata and the ids.py ID recipe. - The backfill/live wing-routing parity test moves to the backfill PR. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
raman325
added a commit
to raman325/mempalace
that referenced
this pull request
Aug 11, 2026
Second half of the MemPalace#1684 split — stacks on the provider core branch. Review-driven changes vs the original branch: - backfill.file_exchange routes through convo_miner.file_conversation_exchange, the same canonical write path live _file_turn uses, so historical and live drawers carry identical routing, normalization, and metadata (and authored_at now reflects the session file's mtime instead of the backfill run time). - backfill.classify_wing delegates to the provider's _match_wing_by_keywords instead of maintaining a synced copy; the parity test now guards the delegation surviving path-based loading. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
raman325
added a commit
to raman325/mempalace
that referenced
this pull request
Aug 11, 2026
…port jsonl Two verbatim violations found in review (thanks @GoXLd — both pre-existing from MemPalace#1684): _messages_to_exchanges paired a user message with the immediately following assistant message, which on tool-using turns is the tool_use stub — the real final answer was dropped and an empty assistant side landed in the palace. Tool-using turns are the common case for a coding agent. It now delegates to the live provider's _segment_turns (one segmentation implementation, same rule as classify_wing), which folds tool traffic and the final answer into the turn via _normalize_content. The .jsonl parser treated each line as a message, but usage: hermes sessions [-h] {list,export,delete,prune,optimize,repair,stats,rename,browse} ... View and manage the SQLite session store positional arguments: {list,export,delete,prune,optimize,repair,stats,rename,browse} list List recent sessions export Export sessions to a JSONL file delete Delete a specific session prune Delete old sessions optimize Reclaim disk space: merge FTS5 segments + VACUUM (no data change) repair Repair a malformed state.db schema so hidden sessions reappear stats Show session store statistics rename Set or change a session's title browse Interactive session picker — browse, search, and resume sessions options: -h, --help show this help message and exit declare -x AI_AGENT="claude-code_2-1-193_agent" declare -x BASH_COMPLETION_COMPAT_DIR="/opt/homebrew/etc/bash_completion.d" declare -x BASH_SILENCE_DEPRECATION_WARNING="1" declare -x CLAUDECODE="1" declare -x CLAUDE_CODE_CHILD_SESSION="1" declare -x CLAUDE_CODE_ENTRYPOINT="cli" declare -x CLAUDE_CODE_EXECPATH="/Users/raman/.local/share/claude/versions/2.1.193" declare -x CLAUDE_CODE_SESSION_ID="765fcf96-9a9b-4567-b6d8-91a981813efa" declare -x CLAUDE_EFFORT="high" declare -x COLORFGBG="7;0" declare -x COLORTERM="truecolor" declare -x COMMAND_MODE="unix2003" declare -x COREPACK_ENABLE_AUTO_PIN="0" declare -x DISPLAY="/var/run/com.apple.launchd.hpP35rW8cb/org.xquartz:0" declare -x EDITOR="vim" declare -x GITHUB_PERSONAL_ACCESS_TOKEN="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" declare -x GIT_EDITOR="true" declare -x GOPATH="/Users/raman/golang" declare -x GOROOT="/opt/homebrew/opt/go/libexec" declare -x GPG_TTY="/dev/ttys002" declare -x HISTCONTROL="ignoreboth" declare -x HISTFILESIZE="32768" declare -x HISTSIZE="32768" declare -x HOME="/Users/raman" declare -x HOMEBREW_CELLAR="/opt/homebrew/Cellar" declare -x HOMEBREW_PREFIX="/opt/homebrew" declare -x HOMEBREW_REPOSITORY="/opt/homebrew" declare -x INFOPATH="/opt/homebrew/share/info:" declare -x ITERM_PROFILE="Default" declare -x ITERM_SESSION_ID="w0t2p0:FC783A53-299F-4B57-97F2-CAC98B10E339" declare -x LANG="en_US.UTF-8" declare -x LC_ALL="en_US.UTF-8" declare -x LC_TERMINAL="iTerm2" declare -x LC_TERMINAL_VERSION="3.6.10" declare -x LESS_TERMCAP_md=$'\E[38;5;136m' declare -x LOGNAME="raman" declare -x LS_COLORS="no=00:fi=00:di=01;31:ln=01;36:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.gz=01;31:*.bz2=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.avi=01;35:*.fli=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.ogg=01;35:*.mp3=01;35:*.wav=01;35:" declare -x LaunchInstanceID="9ABB072B-4AD3-4E38-82F5-EA4DEDFAF2C2" declare -x MANPAGER="less -X" declare -x NODE_REPL_HISTORY="/Users/raman/.node_history" declare -x NODE_REPL_HISTORY_SIZE="32768" declare -x NODE_REPL_MODE="sloppy" declare -x NVM_BIN="/Users/raman/.nvm/versions/node/v22.22.1/bin" declare -x NVM_CD_FLAGS="" declare -x NVM_DIR="/Users/raman/.nvm" declare -x NoDefaultCurrentDirectoryInExePath="1" declare -x OLDPWD="/Users/raman/projects/home-assistant" declare -x OSLogRateLimit="64" declare -x PATH="/Users/raman/projects/home-assistant/.venv/bin:/Users/raman/.local/bin:/Users/raman/.antigravity/antigravity/bin:/usr/local/var/rbenv/shims:/Users/raman/.local/bin:/Users/raman/.nvm/versions/node/v22.22.1/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/Users/raman/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/pkg/env/global/bin:/opt/X11/bin:/Library/Apple/usr/bin:/usr/local/MacGPG2/bin:/Applications/Little Snitch.app/Contents/Components:/Applications/VMware Fusion.app/Contents/Public:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Applications/iTerm.app/Contents/Resources/utilities:/Users/raman/golang/bin:/opt/homebrew/opt/go/libexec/bin:/Users/raman/.local/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/github/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/code-review/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/feature-dev/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/commit-commands/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/security-guidance/2.0.6/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/pr-review-toolkit/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/explanatory-output-style/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/pyright-lsp/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/typescript-lsp/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/php-lsp/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/csharp-lsp/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/jdtls-lsp/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/learning-output-style/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/code-simplifier/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/frontend-design/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/claude-md-management/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/claude-code-setup/1.0.0/bin:/Users/raman/.claude/plugins/cache/Mixedbread-Grep/mgrep/0.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/ralph-loop/1.0.0/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/playground/unknown/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/gopls-lsp/1.0.0/bin:/Users/raman/.claude/plugins/cache/anthropic-agent-skills/document-skills/9d2f1ae18723/bin:/Users/raman/.claude/plugins/cache/superpowers-marketplace/superpowers/5.0.7/bin:/Users/raman/.claude/plugins/cache/superpowers-marketplace/superpowers-chrome/1.6.1/bin:/Users/raman/.claude/plugins/cache/superpowers-marketplace/superpowers-lab/0.4.0/bin:/Users/raman/.claude/plugins/cache/superpowers-marketplace/episodic-memory/1.0.15/bin:/Users/raman/.claude/plugins/cache/claude-plugins-official/clangd-lsp/1.0.0/bin" declare -x PWD="/Users/raman/projects/mempalace" declare -x PYTHONIOENCODING="UTF-8" declare -x RBENV_ROOT="/usr/local/var/rbenv" declare -x RBENV_SHELL="bash" declare -x SECURITYSESSIONID="186b7" declare -x SHELL="/opt/homebrew/bin/bash" declare -x SHLVL="2" declare -x SSH_AUTH_SOCK="/var/run/com.apple.launchd.alsvaxBUwM/Listeners" declare -x TERM="xterm-256color" declare -x TERMINFO_DIRS="/Applications/iTerm.app/Contents/Resources/terminfo:/usr/share/terminfo" declare -x TERM_FEATURES="T3CwLrMSc7UUw9Ts3BFGsSyHNoSxFP" declare -x TERM_PROGRAM="iTerm.app" declare -x TERM_PROGRAM_VERSION="3.6.10" declare -x TERM_SESSION_ID="w0t2p0:FC783A53-299F-4B57-97F2-CAC98B10E339" declare -x TMPDIR="/var/folders/lr/8ht1xr690696dlplq8xpz_p40000gn/T/" declare -x USER="raman" declare -x VIRTUAL_ENV="/Users/raman/projects/home-assistant/.venv" declare -x VIRTUAL_ENV_PROMPT=".venv" declare -x XPC_FLAGS="0x0" declare -x XPC_SERVICE_NAME="0" declare -x __CFBundleIdentifier="com.googlecode.iterm2" declare -x __CF_USER_TEXT_ENCODING="0x1F5:0x0:0x0" writes one SESSION object per line — real exports silently parsed to zero exchanges. The .jsonl branch now unwraps messages/turns keys exactly like the .json dict branch, while still accepting message-per-line transcripts. Also: file_exchange composes drawer text via _compose_exchange_text (byte-identical to live filing — what the dedup safety net's exact-text matching compares against), keeps assistant-only preamble segments instead of dropping them, and the verbose summary no longer reads a conditionally-unbound wing variable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Revives #3 (
feat: Hermes memory provider integration), which was closed for going stale per @bensig:This PR is that rebase + update. Targets
developper CONTRIBUTING.md. Original 4 commits from @ZK-Snarky are preserved verbatim at the base; everything beyond is additive across 16 focused commits.What this adds
A drop-in MemPalace memory provider for Hermes (NousResearch/hermes-agent #6323). One command after
pip install mempalace:Files land in
~/.hermes/plugins/mempalace/(the canonical user-installed plugin directory Hermes'discover_memory_providers()scans) with aplugin.yamlmanifest.~/.hermes/config.yamlgetsmemory.provider: mempalace. Optional interactive backfill of existing Hermes sessions.Why this lives at
mempalace/integrations/hermes/and not top-levelintegrations/hermes/Top-level
integrations/openclaw/exists and the natural read is "put it next to that one." The asymmetry comes from how each integration is consumed at runtime:SKILL.mdthat Claude Code reads from the source repo. It doesn't need to be importable Python and it doesn't need to ship in the wheel.MemoryProviderABC — Hermes' plugin loader imports the module and calls methods on it.mempalace hermes installhas to find the source files at runtime after a wheel install in order to copy them into~/.hermes/plugins/mempalace/.pyproject.tomlhas[tool.hatch.build.targets.wheel] packages = ["mempalace"]. Anything at the top level (including a hypotheticalintegrations/hermes/) is not in the wheel. Verified by building locally: with the integration outside the package,pip install mempalace && mempalace hermes installfails to locate the source for every user without a source checkout — silently breaking the install flow this PR sells.Placing the integration inside the package (
mempalace/integrations/hermes/) makes the standard subpackage layout work. The install command resolves it viaimportlib.util.find_spec("mempalace.integrations.hermes"); the same code path holds in source-tree, editable, and wheel installs; no custom hatchling config needed. Verified withuv build— the wheel contains:The alternative would be
[tool.hatch.build.targets.wheel.force-include]mapping top-levelintegrations/hermes/into the wheel under a different name. That preserves visual symmetry with openclaw but adds custom hatch config and a name-mangling step. Happy to switch if the symmetry matters more than the simpler standard layout.What's new vs. PR #3 (the ABC has evolved)
initialize(config: dict)(session_id, **kwargs)with cron-context guardsync_turn(turn: dict)(user_content, assistant_content, *, session_id, messages)get_tools()with embeddedhandlerattributeget_tool_schemas()+handle_tool_call(name, args) -> stron_pre_compresslist[dict]str(hint into the compression summary prompt)MempalaceProviderMemoryProvider(ABC stub fallback for non-Hermes envs)Opt-in hooks added:
on_session_switch,on_delegation,on_memory_write,on_turn_start,queue_prefetch,save_config,post_setup.The provider is inactive under
agent_context in {"cron", "flush"}orplatform == "cron"so system-generated turns don't corrupt the user representation (matches the discipline in built-in providers like honcho). The_cron_skippedflag is cleared at the start of every non-croninitialize()so a cron init doesn't permanently poison the instance.Tool surface — 27 of 30 documentable MCP tools
The provider exposes the full agent-facing mempalace MCP surface: all 27 tools across Search & Browse, Knowledge Graph, Palace Graph (incl. explicit tunnels), Write/Diary, and the silent-checkpoint ack. The remaining 3 tools —
mempalace_sync,mempalace_hook_settings,mempalace_reconnect— are intentionally omitted. They're host/admin operations that belong to the MCP host owner, not the conversation loop, and exposing them to the agent would be confusing at best and destructive at worst.Two changes were required to make this surface actually reachable:
_initializedgate fromget_tool_schemas(). Hermes' memory manager snapshotsget_tool_schemas()at provider-registration time, beforeinitialize()runs (agent.memory_manager._register_provider, registration-time dispatch-table build). The gate caused 0 tools to be registered with the dispatch router even thoughget_all_tool_schemas()(queried each turn for system-prompt assembly) returned the full set — the model saw the tools but dispatch couldn't route to them, surfacing asUnknown tool: mempalace_statusin real openclaw use. Schemas describe interface; readiness belongs inhandle_tool_call._dispatch_mcp_passthroughhelper derivestool_<name>frommempalace_<name>(with one explicit remap formempalace_traverse→tool_traverse_graph) and forwards tomempalace.mcp_server. The refactor also pulledhandle_tool_call's cyclomatic complexity below the C901 limit (was 31 with an if-chain).The OpenClaw memory skill (
integrations/openclaw/SKILL.md) was also stale relative to this surface (only 19 tools documented). Parity PR: #1719.The embedding-dimension fix
Three earlier Hermes-side PRs (NousResearch/hermes-agent #5671, #12203, #9761) all called
chromadb.PersistentClient.get_or_create_collection(...)directly without passingembedding_function=. ChromaDB bound its default 384-dim function to the collection; users with palaces built onbge-m3(1024-dim) orembeddinggemma-300mwould hit a hard dimension mismatch on the next write —@Motokiyoflagged this in the hermes-agent issue thread.This PR routes ChromaDB access (both live writes and
backfill.py) throughmempalace.backends.chroma.ChromaBackend.get_or_create_collection, which binds the canonical embedding function frommempalace.embedding.get_embedding_function(). Existing palaces import without rebuild. The end-to-end testtest_initialize_opens_chroma_via_backendasserts this construction path;test_status_tool_counts_seeded_drawersconfirms the provider reads back data written via rawchromadb.PersistentClient(the regression check).Review-feedback rounds
The PR went through two automated review rounds since opening. All findings are addressed in the commits below.
Round 1 — Gemini Code Assist (4 findings):
backfill.pyinstantiatedPersistentClientper exchange withoutembedding_function=→ reintroduced dim-mismatch + per-turn perf hit. Now goes throughChromaBackendonce perbackfill()call._tool_diary_readloaded the wholediary.jsonlto keep the tail →collections.deque(f, maxlen=n).Round 2 — Copilot (11 findings):
_initializedset True even when backend init failed → now reflects actual readiness.KnowledgeGraphinstantiated and never closed in 3 sites → wrapped intry/finally: kg.close().wingconfig field documented as "default wing" but always overridden by keyword classification → honored when set.session_idas theroomvalue (high-cardinality, inconsistent with backfill) → stableroom: "conversations", session_id moved to dedicated metadata field._classify_wingused substring matching (kw in text) → word-boundary regex.os.system, missed--upgrade, assumed POSIX venv layout, hard-coded~/.mempalace/palacefor backfill ignoringMEMPALACE_PALACE_PATH, line-based YAML edit broke onmemory: ~/ inline comments / non-2-space indent → rebuilt with cross-platform venv detection,subprocess.run+--upgrade, palace-path resolver honoring env vars,yaml.safe_load/safe_dumpround-trip with atomic writes via tmp + rename, and--hermes-home '.' '/'rejected.datetime.utcnow()in backfill →datetime.now(timezone.utc). 16-char doc_id from text-prefix hash → 32-char hash over full content + timestamp.Round 3 — Copilot (10 findings):
_mine_sessionskipped_normalize_content;on_session_endandon_memory_writeskipped the readiness gate;backfill.classify_wingskipped the word-boundary fix. All closed; new tests pin the regressions.integrations/hermes/README.mdsaid "first match wins" and listedMEMPALACE_COLLECTION_NAME— both stale after the env-precedence and collection-name-removal fixes. Aligned.[tool.hatch.build.targets.wheel] packages = ["mempalace"]meant top-levelintegrations/didn't ship in the wheel → moved tomempalace/integrations/hermes/(rationale above)._update_hermes_config_yamlreturned(bool, str)whereFalsecollapsed "already set" and "couldn't parse" → now(status, message)with"updated"/"noop"/"error"; install exits non-zero only on"error"._tool_status/_list_wings/_list_roomsmaterialized every drawer's metadata for counts →STATUS_SCAN_LIMIT = 5000cap, with a"truncated"field on the response so the model knows the breakdown is sampled.Tests
62 new tests across two files; full mempalace suite 2,340 passed, 3 skipped, 0 failed in ~68s locally (3 skips are pre-existing).
tests/test_hermes_integration.py(40 tests, reusespalace_path/seeded_collection/seeded_kg/ autouse_isolate_homefromtests/conftest.py):name,is_available, config schema, tool-schema visibility before/after init)agent_context=cron,agent_context=flush,platform=cron)_initialized=False(worker queue isn't fed byon_session_end/on_memory_write— Round 3 regression coverage)_normalize_contentflattens Anthropic list-shapedcontentwithout persisting therepr_match_wing_by_keywordsrespects word boundaries (substring would route"said"towing_ai)backfill.classify_wingmatches the live provider's classification (no live/backfill divergence)sync_turnend-to-end: enqueues, worker drains, drawer lands withsource=hermesand the configured wingmempalace_status,_list_wings,_list_rooms,_search, KG add+query, diary roundtrip$HERMES_HOME/mempalace.json> defaults; empty env vars treated as unsetcollection_nameintentionally absent from the schema (regression test)reset=Trueclears turn counter;reset=Falsepreserves it)tests/test_hermes_install_cli.py(22 tests, addresses Copilot's "no install coverage" finding):--hermes-home/$HERMES_HOME/ default precedence--hermes-home .and--hermes-home /(rather than silently installing into CWD)venv/bin/python3,python) and Windows (venv\Scripts\python.exe), falls back tosys.executableMEMPALACE_PALACE_PATHenv-var precedence + empty-value handling_atomic_write_textcreates parents and leaves no.tmpafter success_update_hermes_config_yaml: missing file, malformed YAML, scalarmemory: ~, already-set, replace existing — each returns the correct"updated"/"noop"/"error"statusCommits
20 commits since
upstream/develop— preserved ZK-Snarky's 4 plus 16 mine:Known follow-ups (not blocking)
ChromaBackend.close()on providershutdown(): would release the SQLite file lock so the same Python process can reopen the palace cleanly (matters for tests and gateway-reload scenarios). Skipped here because there's no public close method on the backend yet; needs an upstream API.prefetch()is synchronous on the hot path.queue_prefetch()is not yet overridden; a background-prewarm rewrite would match the discipline inhoncho. Significant enough to warrant its own PR.STATUS_SCAN_LIMIT, the wing breakdown is sampled. A SQLite-sidecar counter would give exact figures cheaply; mempalace doesn't expose one yet.Cross-references
integrations/openclaw/SKILL.mdto the same 27-tool surfacecc @ZK-Snarky for awareness — happy to fold in further changes you want.