Skip to content

fix(procedures): retire soft-deletes on read and stop updates duplicating - #17

Merged
gigabyte22 merged 1 commit into
mainfrom
fix/procedure-expiry-and-update-dedup
Jul 23, 2026
Merged

fix(procedures): retire soft-deletes on read and stop updates duplicating#17
gigabyte22 merged 1 commit into
mainfrom
fix/procedure-expiry-and-update-dedup

Conversation

@gigabyte22

Copy link
Copy Markdown
Owner

Summary

Fixes two procedure-lifecycle defects that were previously pinned in the integration suite behind KNOWN SERVER DEFECT comments rather than fixed. Both are unfixed on main and both are real:

  1. Soft-deleted / expired procedures kept resurfacing. Procedures soft-delete (and naturally expire) by stamping expiresAt, but ProcedureRepository's read paths had no expiry predicate, so retired procedures came straight back on the next read. ResearchRepository.list already carries this exact predicate (added 2026-07-20 for the same bug); it was simply never applied to procedures.

  2. A body-changing update duplicated the procedure. ProcedureService.update wrote the new body onto the original node and created a superseding clone, retiring neither — leaving two identical live nodes behind every edit. The duplicate count grows with each edit, so a live graph accumulates them over time.

What changed

1. Filter expired procedures on read (ProcedureRepository)

  • Added expiresAt IS NULL OR expiresAt > datetime() to list, mirroring ResearchRepository.list.
  • Applied the same predicate to listSimilar and fullTextSearch so the leak can't simply move from list to /recall.

2. Two-node supersession on update (ProcedureService.update + ProcedureRepository.supersede)

Rewrote the body-change path to the model already used by Facts and Preferences:

  • The old node keeps its old body and is retired (expiresAt = now) — it is no longer overwritten.
  • A new node is created at v+1 carrying the new body; only it stays live.
  • The version chain remains walkable via :SUPERSEDES (and audit).
  • ProcedureRepository.supersede now retires the old node, mirroring FactRepository.supersede setting validTo.
  • Metadata-only updates (telemetry / expiresAt) still revise in place on the same id.

3. Backfill for existing duplicates (scripts/backfill-procedure-dedup.ts)

One-shot, idempotent cleanup for procedures duplicated by the old behavior:

  • Groups every procedure touched by a :SUPERSEDES edge into lineages (union-find, no APOC dependency).
  • Keeps the highest-version node per lineage (tie-broken by newest UUIDv7 id) and retires the rest.
  • Robust to the fan-out case (an orchestrator that re-passed the original id, spawning multiple clones).
  • Run once, after deploying this fix. Wired as pnpm backfill:procedure-dedup.

4. Tests

Flipped the pinned KNOWN SERVER DEFECT assertions in mcp-live and openclaw-live to assert the corrected behavior: retired procedures leave the listing, and a body edit yields exactly one new live node.

Behavior change for API consumers

A body-changing PUT /procedures/:id still returns a new id (the superseding node) — unchanged from before. What changes is that the old id is now retired: it keeps its previous body and version, remains resolvable by id, but no longer appears in list or /recall. This matches how Facts already behave.

Testing

  • pnpm typecheck ✅ · pnpm lint (changed files clean) ✅
  • pnpm test127/127 unit ✅
  • Integration (Neo4j testcontainer, via vitest.integration.config.ts):
    • mcp-live 43/43
    • v1.2-thorough, openclaw-live, client-live, research98/98 ✅ (supersede-chain + audit-snapshot tests pass unchanged)
  • Backfill validated against a throwaway Neo4j (chain → keeps v3; fan-out → keeps tie-broken node; standalone untouched; second run is a clean no-op).

…ting

Two procedure-lifecycle defects, both previously pinned in the integration
suite behind KNOWN SERVER DEFECT comments rather than fixed.

1. Soft-deleted/expired procedures kept resurfacing. Procedures soft-delete
   (and naturally expire) by stamping `expiresAt`, but `ProcedureRepository`'s
   read paths had no expiry predicate, so retired procedures came straight back
   on the next read. `ResearchRepository.list` already carries this exact
   predicate (added 2026-07-20); it was never applied to procedures. Add it to
   `list`, and — so the leak can't just move to `/recall` — to `listSimilar`
   and `fullTextSearch` as well.

2. A body-changing update duplicated the procedure. `ProcedureService.update`
   wrote the new body onto the original node AND created a superseding clone,
   retiring neither, leaving two identical live nodes behind every edit (the
   count grew with each edit). Rewrite the body-change path to the two-node
   supersession model already used by Facts/Preferences: keep the old node's
   body intact and retire it, create a new node at v+1 carrying the new body,
   so only the new node stays live and the version chain remains walkable via
   :SUPERSEDES and audit. `ProcedureRepository.supersede` now retires the old
   node (sets `expiresAt`), mirroring `FactRepository.supersede` setting
   `validTo`. Metadata-only updates still revise in place on the same id.

Add `scripts/backfill-procedure-dedup.ts` to clean up procedures duplicated by
the old behavior: it collapses each :SUPERSEDES-connected lineage to a single
live node (highest version, tie-broken by newest id) and retires the rest. It
is idempotent and safe to re-run. Run it once, after deploying this fix.

Flip the pinned assertions in the mcp/openclaw integration specs to assert the
corrected behavior (retired procedures leave the listing; a body edit yields
exactly one new live node).
@gigabyte22
gigabyte22 merged commit 5cd4c3a into main Jul 23, 2026
@gigabyte22
gigabyte22 deleted the fix/procedure-expiry-and-update-dedup branch July 23, 2026 00:28
gigabyte22 added a commit that referenced this pull request Aug 6, 2026
…lugin (#32)

* feat(hermes): package the adapter as an installable hermes-elephant plugin

The adapter was a directory you were told to copy into a hermes checkout.
This makes it a real distribution, and fixes two things that stopped the
documented setup from working at all.

Packaging
- Distribution `hermes-elephant`, import package `hermes_elephant`. The
  import package is deliberately not `elephant`: that name belongs to the
  Electrophysiology Analysis Toolkit on PyPI and a top-level collision would
  break anyone with both installed. The provider is still `elephant`
  everywhere a user sees it — `memory.provider`, `hermes elephant`, and the
  installed plugin directory.
- `hermes-elephant install` copies the provider into
  `$HERMES_HOME/plugins/elephant/`. hermes discovers memory providers by
  directory scan only, never pip entry points (upstream #40101), so pip alone
  leaves it invisible. Same two-step shape the Memori provider ships.
- Declares the `hermes_agent.memory_providers` entry point so the install step
  becomes redundant once upstream discovery lands. Deliberately does NOT join
  `hermes_agent.plugins`: the general PluginManager eagerly imports that group
  in every hermes process and then calls register(ctx) on a context with no
  register_memory_provider.
- Stdlib-only, asserted in the tests — installing must never add a package to
  the hermes runtime.

Fixes
- `hermes elephant ...` was dead for every directory install. hermes registers
  a user-installed provider's package as a synthetic shell with no __file__,
  so cli.py's `from . import DEFAULT_URL` raised ImportError and
  discover_plugin_cli_commands() returned nothing — while plugin.yaml and the
  README advertised the subcommands. Shared names moved to `_shared.py`;
  submodule imports resolve through the shell's __path__. Regression test
  drives the real loader shape and fails against the old import.
- Episodes now carry `origin`. Without it the dreamer treats every transcript
  as human conversation, so a subagent's machine-written task text was mined
  for facts *about the operator* — elephant's extraction prompt keys off this
  field to suppress exactly that.

Integration
- `on_delegation` records a subagent's task and result against the parent
  session, the last unimplemented write hook.
- `config_schema.py` gives the provider a native hermes dashboard panel;
  validated against hermes's real ProviderField/ProviderConfigSchema.
- `plugin.yaml` states `kind: exclusive` rather than relying on hermes's
  source-text auto-coercion.
- README documents wiring elephant's pull-only `:Intention` nodes to hermes
  cron — elephant never fires them, and hermes is the scheduler.

* test(hermes): fix the procedure live test against supersession semantics

test_procedure_round_trip re-read the ORIGINAL id after a body update and
expected the new content. That stopped being true when #17 replaced in-place
procedure updates with supersession: a body change now creates a NEW node at
v+1 and retires the old one, leaving its body intact and walkable via
:SUPERSEDES. The adapter already reports the new id back to the model, so
nothing was broken except the test's expectation.

Now asserts the actual invariant — the edit is at the new id, the old id still
answers with v1, and lookup by name follows the live version so an agent
holding only the name is not stranded.

Full live suite against a real service: 127 passed, 1 skipped.

* refactor(hermes): make the shipped provider a directory, not an allowlist

Three review findings, all about the installer promising more than it verified.

Shipped set is now a directory boundary. The provider moved to
hermes_elephant/provider/ and install copies that tree wholesale, so copy mode,
--link mode and the built wheel are the same files by construction. The
hand-maintained PLUGIN_FILES allowlist and the test holding it in agreement are
both gone — the test could only ever compare the allowlist against the source
checkout, never against the wheel it was meant to guard. --link also stops
exposing install.py to hermes's submodule exec, which was an untested asymmetry
between dev and ship.

Install verifies the discovery contract. hermes classifies a provider directory
by scanning the first 8KB of its __init__.py for MemoryProvider /
register_memory_provider. A packaging gap used to print a warning and then
report success, leaving the user to reconcile "Installed" here with
"Plugin: NOT installed" there. Now it fails, cleans up, and says why — and pins
an upstream constraint that is otherwise invisible.

status detects a stale copy. Copying rather than symlinking means
`pip install -U hermes-elephant` leaves the profile running the old version with
nothing to notice. The marker now records the version and status reports STALE.

Also:
- hermes_home() defers to hermes_constants.get_hermes_home() when importable,
  and otherwise honours %LOCALAPPDATA%\hermes on Windows. It previously
  hardcoded ~/.hermes, which on Windows installs where nothing scans and then
  reports success.
- _is_ours() only claims a symlink that resolves to our own package, so
  uninstall cannot unlink one the user made pointing at their own tree.
- __pycache__ and dotfiles are never copied into the plugin directory.

Verified against a real elephant on a throwaway Neo4j testcontainer:
129 passed, 1 skipped. Entry-point discovery re-checked end to end against a
hermes checkout — pip-only install still resolves its directory, config panel,
CLI, and 34 tools.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant