feat: align LCM with Codex continuity and whitepaper control flow

[100yenadmin]

Important

This executable LCM-X PR was recreated by the migration operator from the exact upstream commit head. GitHub did not transfer the original PR actor, dates, review objects, or approval state.

Source and attribution

• Original pull request: feat: align LCM with Codex continuity and whitepaper control flow stephenschoettler/hermes-lcm#527
• Originally authored by @codejeet
• Upstream author profile: codejeet
• Preservation record: [Upstream PR #527] feat: align LCM with Codex continuity and whitepaper control flow #152
• Exact upstream head: 02428dde78f70a9bc101b9d3e70b2d7c5e6c4e02
• Original head repository/ref: codejeet/hermes-lcm:feat/codex-whitepaper-reliability
• Original base branch: main
• Destination branch: upstream/pr-527
• Original upstream draft state: False
• Migration containment state: draft (a maintainer can mark it ready when current-head review is wanted)

@coderabbitai ignore

Original commit authorship and history remain in the commits. Historical discussion and review text are imported below as attributed ordinary comments; they are not new approvals or change requests.

Original upstream PR description

Summary

• add lossless opaque Codex reasoning and native-compaction continuity without exposing capsules through FTS, summaries, inspection, or expansion
• add explicit soft/hard compaction, deterministic convergence, file lineage, and persistent llm_map / agentic_map operators
• make concurrent SQLite ownership and lock recovery durable, and bound restart reconciliation over large externalized-output archives

Why

Long-running concurrent gateway sessions exposed two production failures: restart reconciliation repeatedly scanned a 12,799-file, roughly 540 MB archive, and cloned engines could contend over independently owned SQLite helpers. The first failure pushed a 590K-token resume beyond the 30-second host timeout. The bounded lookup and per-pass identity cache reduce that reconciliation to roughly 2 seconds while preserving every source row and externalized payload.

Validation

• Focused validation: 483 passed
• Default validation:
  • pytest tests/test_lcm_core.py tests/test_lcm_engine.py tests/test_packaging_install.py -q
  • pytest -q: 2,846 passed, 1 skipped
  • low-FD pytest at ulimit -n 1024: 2,846 passed, 1 skipped
  • scripts/validate_release.sh --full --keep-going: all gates passed
  • benchmark smoke, stress smoke, and release stress tier: zero failures
  • git diff checks for origin/main...HEAD, working tree, and staged tree
  • Python compileall, script py_compile, and shell syntax checks
• Workflow validation not applicable: workflow files are unchanged

Notes

• Scope is broad because the storage, compaction, provider-state, and operator invariants share one schema and lifecycle boundary; the commit keeps the existing plugin name, engine name, and database path compatible.
• Background compaction remains opt-in.
• The separate Hermes gateway-core teardown patch is intentionally excluded from this plugin PR.
• Validation started and ended with a clean git status.

[100yenadmin]

Triage: this is one of three open PRs implementing the same architectural change — flagging so it is not reviewed in isolation.

#197, #202 and #215 each introduce a shared, reference-counted SQLite storage bundle across LCM engine clones. All three necessarily relax the same documented guarantee — that a clone owns isolated storage — by removing the six clone._store is not prototype._store / _dag / _lifecycle assertions at tests/test_lcm_core.py:7495 and :7538.

To be clear, that is not test-hiding, and I checked before saying so: all three add replacement assertions for the new invariant (9, 8 and 11 respectively — clone-local session state, one idempotent lease per engine, owner-first shutdown). This is a legitimate contract change with coverage.

The motivation is real and measured, from #197: ten retained clones cost 60 file descriptors and roughly 100 ms of duplicate SQLite helper construction, which risks descriptor exhaustion under parallel child-agent workloads. That is the defect tracked as #9.

So the problem is not any one of these PRs — it is that there are three. Reviewing them independently would mean deciding the same architecture question three times, and merging any one of them makes the other two conflict against a moved contract. The maintainer action is to pick one design, land it, and close the other two with evidence.

This needs an owner/architecture call, because it changes a documented guarantee for every embedder of the engine. It is not a call I will make unilaterally as maintainer. Recorded on #9 as the tracking issue; holding all three until it is decided.

[100yenadmin]

Closed superseded by the owner-ratified storage-trio decision: #197 is the pick (memo + ratification on #9, 2026-08-20). Reopenable — this records a product decision, not a merit refutation. The Codex-continuity/whitepaper control-flow ideas herein remain referenced from the memo for future consideration.