fix(cli): honor --resume/--continue in oneshot (-z) — rebase + real-SQLite coverage - #80799
fix(cli): honor --resume/--continue in oneshot (-z) — rebase + real-SQLite coverage#80799blairjordan wants to merge 5 commits into
Conversation
hermes -z parsed --resume but silently dropped it: both dispatch sites called run_oneshot() without the flag, and the agent was built with no session id and no history — every one-shot turn was stateless. Now: - run_oneshot(resume=...) loads the session transcript as conversation_history (walking compression chains via resolve_resume_session_id, dropping session_meta rows) and pins the agent to the SAME session id, mirroring the interactive resume path. - Create-on-first-use: an id Hermes hasn't seen is used as-is with no history, so scripted callers (gateways, cron workers) can mint stable session keys up front and pass them on every turn — no output parsing. - --continue resolves by name / most-recent exactly like interactive chat (_resolve_oneshot_resume). - Message flush dedup is inherited: history dicts seeded via conversation_history are skipped by identity, so resumed transcripts are never re-written to the store. - Best-effort: a broken session store degrades to a stateless turn. Verified live: ZEBRA planted and recalled across two -z invocations on a caller-minted id. tests/hermes_cli/test_oneshot_resume.py (7 new); tests/hermes_cli 7804 passed — failure set byte-identical to the pre-change baseline (259 pre-existing, attributed by clean-tree diff). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rebases nathansmithopenclaw-alt's NousResearch#57859 onto current main (which had since landed the MCP-discovery wait and requested_provider/usage_file params in _run_agent — this carries both forward alongside the resume hydration) and addresses the two points from @teknium1's review: - The existing test_oneshot_resume.py coverage is entirely mock-based (SessionDB, AIAgent, and _load_resume_history's own dependencies are all MagicMocks) — real create-on-first-use behavior, a second-process reload, and duplicate-row risk on the resume-and-reflush cycle were never actually exercised against disk. - Adds TestOneshotResumeIntegration: two _run_agent() calls against a real, temp-path-backed SessionDB (only AIAgent is replaced, with a fake that performs the SAME real create_session/append_messages_batch calls a genuine turn's flush does), then a THIRD read through a brand-new SessionDB instance — the same shape a second `hermes -z` process invocation would see. Asserts the full two-turn transcript loads in order with exactly 4 rows, not 8: turn 1's messages are seeded as conversation_history and never re-appended by the flush's identity dedup. tests/hermes_cli/test_oneshot_resume.py: 9 passed (7 existing + 2 new). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three gaps in the oneshot resume path, found reviewing NousResearch#80799 against CONTRIBUTING.md and the review findings on sibling PR NousResearch#70136. 1. `--resume` bypassed the ID-or-title resolution contract. `_parser.py` documents `--resume SESSION` as "by ID or title" and `cmd_chat` resolves titles via `_resolve_session_by_name_or_id`. `_resolve_oneshot_resume` passed the value through verbatim, so `hermes --resume "my project" -z "..."` silently minted an empty session literally named "my project" instead of resuming — the same silently-drops-a-flag class as the bug being fixed, and the exact finding teknium1 gave NousResearch#70136. Now resolves first; an unresolved value still falls through to create-on-first-use, so the design intent (scripted callers minting stable ids) is preserved. The asymmetry was visible inside the PR itself: `--continue "name"` already resolved. 2. Create-on-first-use was an unguarded entry boundary for caller-supplied session ids. A session id becomes a filename downstream — `SessionDB._remove_session_files` builds `sessions_dir / f"{id}.json"` and globs `request_dump_{id}_*.json` without sanitizing — so `hermes --resume ../../x -z "hi"` minted a row that a later `sessions delete`/`prune` unlinks outside the sessions dir (verified: the victim file is removed). `gateway/session.py` already rejects exactly this at its own entry boundary (`_is_path_unsafe`, CWE-22); the new CLI boundary now does too. Path-unsafe values that *resolve* to a real session are still fine — the guard only applies to values about to become new ids, so titles containing `/` keep working. 3. The integration test asserted a dedup property it never exercised. `_RecordingFakeAgent` appended only its own two new messages, so "4 rows, not 8" held by construction regardless of whether the flush's identity-based seeded-history dedup worked. It now drives the real `AIAgent._flush_messages_to_session_db` with the real `messages = conversation_history + this turn` shape; break the identity contract and the assertion fails (verified by mutation). Also adds the coverage teknium1's review would still have flagged: `_resolve_oneshot_resume` had zero tests (title, exact id, unknown id, path-unsafe, `--continue` by name / bare / unmatched), a third-turn accumulation case, and the compression-chain redirect — a caller passing the pre-compression parent id must land on the continuation child, with this turn's writes going there and not into the dead parent. Docs: `-z` help and the `hermes_cli/oneshot` module docstring now state ID-or-title, create-on-first-use, and that an unmatched `--continue` errors while an unmatched `--resume` does not. tests/hermes_cli/test_oneshot_resume.py: 26 passed (was 9).
Reconciles two independent reviews of this branch (review-branch / review-completeness) into one commit: From the correctness review: - _resolve_oneshot_resume's three failure paths (unresolvable --continue by name, no prior session for bare --continue, and the new path-unsafe --resume id) called sys.exit(2) directly. By that point _prepare_agent_startup() has already discovered plugins and can have started MCP server subprocesses, so unwinding through normal interpreter finalization both orphans them (only _cleanup_oneshot_runtime() reaps them) and re-enters the native finalizer teardown _exit_after_oneshot exists to skip (NousResearch#30387, NousResearch#43055). Added _fail_and_exit_oneshot() and routed all three through it. - _load_resume_history() was called before the try that owns session_db, contradicting that try's own stated invariant ("always closed"). _load_resume_history's own internals are individually guarded so this wasn't reachable today, but moved it inside for real. Test fallout from the above: _exit_after_oneshot does a genuine os._exit, not a raise, so the three tests that previously asserted `pytest.raises(SystemExit)` around these paths would now hard-kill the test process. Rewrote them (_assert_hardened_exit helper) to patch _cleanup_oneshot_runtime/_exit_after_oneshot and assert on the calls, matching the pattern the correctness review's own new test used. Verified: tests/hermes_cli/test_oneshot_resume.py 26/26 passed. tests/hermes_cli/ -k "resume or continue or oneshot or session": 203 passed, same 5 pre-existing dashboard-auth 401s as both individual review branches (unrelated to this change). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reconciles two more independent reviews of this branch into one commit, on top of the prior reconciliation (983b55f): Security review: - hermes_state.py: SessionDB._remove_session_files interpolated session_id RAW into a glob pattern (request_dump_{id}_*.json). An id of literally '*' (legal under the existing path-traversal guard - it's not a traversal character) expands to request_dump_*_*.json and unlinks every OTHER session's request dumps. Fixed with glob.escape. Pre-PR this was gateway-only reachable; --resume makes it CLI-reachable. - hermes_cli/main.py: _is_unsafe_new_session_id only ported one of the three checks the gateway's equivalent entry boundary (gateway.session._is_path_unsafe) actually applies together: added control-character rejection (CR/LF log-line forgery, ESC terminal escape injection) and the same 256-char length cap (past NAME_MAX the session snapshot file becomes permanently uncreatable). Rejection messages now echo the offending value through a safe-echo helper so the rejection itself can't replay the injection. Concurrency/production-robustness review: - hermes_cli/oneshot.py: the two --resume reads (resolve_resume_session_id, get_messages_as_conversation) had no lock-contention patience, unlike every write path in this codebase (20-60s jittered retry). Off WAL -- the default on any pre-3.51.3 SQLite (stock CPython) or NFS/SMB/ZFS -- a sibling connection holding the write lock for a few seconds (a large FTS append, incremental merge, checkpoint, VACUUM) made the read raise "database is locked", which _load_resume_history already swallowed to a stateless degrade. Result: a resumed turn silently runs with NO prior context, exits 0, and appends its context-less answer into the middle of the chain -- full token cost, wrong answer, poisoned transcript for every downstream step. This is the exact failure mode a Hatchet-style concurrent multi-workflow caller (the motivating use case for this feature) will hit under real contention. Added _read_with_lock_patience: waits out transient lock contention on the same budget the write path already uses; raises ResumeStoreContendedError (routed through run_oneshot's existing non-zero-exit error path, so an orchestrator can retry) only when the lock never clears. A genuinely broken store still degrades to stateless as before -- only the contended-but-healthy case now waits instead of silently losing context. Verified together: tests/hermes_cli/test_oneshot_resume.py 40/40 passed (26 prior + 6 lock-patience + 5 control-char/glob-escape + others). tests/hermes_cli/ -k "resume or continue or oneshot or session": 217 passed, same 5 pre-existing dashboard-auth 401s as every prior review branch (unrelated to this change). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
This has been running in production for us since 2026-08-07, so I can speak to the two risk labels with real operational evidence rather than just code review. Context: we hand-patched On On Happy to point at specific commits/logs if it helps move this past |
|
Separately — CI hasn't run on this PR yet (no checks show up), which I think is the first-time-contributor workflow-approval gate. If a maintainer can approve the workflow run, that'd help get this off the "no CI signal" pile — happy to push a rebase if that's easier to trigger a fresh run. |
What
Carries forward #57859 (
fix/oneshot-resume-hydration, @nathansmithopenclaw-alt):hermes --resume <id> -z "<prompt>"(and--continue [name]) chains — the session's prior transcript loads as conversation history and the turn appends to the SAME session id. Ids that don't exist yet are created on first use, so scripted callers can mint a stable session key per conversation up front and pass it on every turn.This closes #49195, and addresses everything from @teknium1's review of #57859:
ensure_mcp_discovery_before_agent_build) andrequested_provider/usage_fileadditions to_run_agent/run_oneshot— this carries the resume hydration forward alongside all three._run_agent()calls against a real, temp-path-backedSessionDB(only the LLM call is faked — the stand-in drives the genuineAIAgent._flush_messages_to_session_dbagainst real disk), then a reload through a brand-newSessionDBinstance — the shape a secondhermes -zprocess invocation would see. The row-count/dedup assertion is verified non-vacuous by mutation (breaking the real identity-dedup makes it fail).Update: this PR went through two independent adversarial reviews after opening (see below) that found and fixed three additional real issues beyond what's described above. Net diff vs. the originally-opened version: a hardened exit path for
-z --resume/--continuefailures, a path-traversal guard on caller-minted ids, and ID-or-title resolution for--resumematching its own documented contract.Why this needs to exist
hermes -z(oneshot) is the interface a piped/scripted caller uses precisely to avoidchat's banner/box/session chrome on stdout. Every such caller that wants multi-turn continuity — a gateway running a chat seat over-zone-shots, a cron worker, a CI agent pipeline driving Hatchet-style multi-step workflows — currently gets an amnesiac agent:--resume/--continueare accepted (exit 0, no warning) and silently dropped, so every turn starts from zero. This is the third independent fix attempt at this exact gap (#40333 closed in error unrelated to merit, #49204 takes the inverse "reject the flag" minimal path) — the bug report, the review signal, and the number of independent contributors who've hit this all say it's a real, wanted capability.Relationship to existing PRs/issues
salvageability=high) — rebased + tests added per that review.--no-restore-cwd. This PR fixes (a) — see below — and does not touch (b); flagging that gap below rather than silently leaving it unaddressed.Change
hermes_cli/oneshot.py:run_oneshot(resume=...)→_run_agent(resume=...)→_load_resume_history(session_db, resume): resolves the compression chain viaresolve_resume_session_id, loads messages viaget_messages_as_conversation(droppingsession_metarows), best-effortreopen_session, and pinsAIAgent(session_id=...). A broken session store degrades to a stateless turn —-znever gets less reliable than before.session_db, matching that block's own stated invariant that the connection is always closed (not reachable as a live bug today —_load_resume_history's own internals are individually guarded — but it contradicted the comment directly above it).hermes_cli/main.py:resume=_resolve_oneshot_resume(args)and forward it through_run_and_exit_oneshot.--resumeresolves by ID or title, exactly likecmd_chatand the flag's own_parser.pyhelp text ("Resume a previous session by ID or title"). Without this,--resume "my project"would silently mint an empty, wrongly-named session instead of resuming — the identical "documented behavior silently doesn't happen in-z" bug class as hermes -z (oneshot) silently ignores --resume / --continue (session never hydrated) #49195 itself, and the exact gap sibling PR fix: preserve one-shot resume context #70136 was reviewed down for. Resolution still falls through to create-on-first-use when nothing matches._is_unsafe_new_session_id): a create-on-first-use id becomes a filename downstream (SessionDB's session-file removal path is unsanitized), sohermes --resume ../../x -z ...would mint a row that a latersessions delete/pruneunlinks outside the sessions directory (CWE-22). Mirrors the gateway's own_is_path_unsafeguard at its equivalent entry boundary. Only applied to values about to become new ids — a title containing/that resolves to a real session is untouched._fail_and_exit_oneshot): all three_resolve_oneshot_resumefailure branches (unmatched--continueby name, no prior session for bare--continue, path-unsafe--resumeid) now exit via_cleanup_oneshot_runtime()+_exit_after_oneshot()instead of a baresys.exit(2). By the time oneshot dispatch runs,_prepare_agent_startup()has already discovered plugins and can have started MCP server subprocesses — unwinding through normal interpreter finalization both orphans them and re-enters the native finalizer teardown_exit_after_oneshotexists to skip (Bug: hermes -z prints successful response then aborts during shutdown with exit 134 #30387/Bug: hermes -z aborts with SIGABRT during teardown on AL2023 #43055).--continueresolves by name/most-recent exactly like interactive chat; an unresolvable--continueerrors (nothing sensible to chain onto).hermes_cli/_parser.py—-zhelp documents ID-or-title + create-on-first-use + the--continueerror semantics.--resume/--continue, behavior is byte-for-byte unchanged.One open question for a maintainer, not resolved here
Whether
--resume <title>should resolve at all in oneshot was reviewed both ways during this PR's own review cycle: one review argued for matching the documented ID-or-title contract (implemented, above); a second flagged a theoretical hijack risk — a caller-minted key that happens to collide with an existing session's human-readable title would land on the wrong session. I kept the resolution because it's the same resolution function and ordering (exact id first, title fallback) that interactive chat already uses today, so this isn't a new risk class introduced by this PR, and it matches what--parser.pypromises and what #70136's review already asked for. Flagging explicitly in case a maintainer weighs this differently.Tests
tests/hermes_cli/test_oneshot_resume.py: 26 passed (7 original mock-based + 2 real-SQLite integration + 3 new integration cases — third-turn accumulation, compression-chain redirect, unknown-id create-on-first-use — + 14 in a newTestResolveOneshotResumecovering title resolution, path-traversal rejection (5 shapes), the hardened-exit contract, and--continueresolution).tests/hermes_cli/ -k "resume or continue or oneshot or session": 203 passed, 5 failed — all 5 are pre-existing dashboard-auth 401s unrelated to this change (confirmed against a clean-main baseline in the same sandbox).tests/hermes_cli/suite: no regressions vs. a clean-mainbaseline in the same sandbox (identical pre-existing failure count; the only pass-count delta matches this PR's own new tests).nousresearch/hermes-agent:latestcontainer: planted a codeword on turn 1 under a caller-minted--resumeid, recalled it correctly on turn 2 under the same id, confirmed a different--resumeid sees no cross-session leakage.Review history
This branch was reviewed twice after opening, independently and adversarially (one focused on correctness of the session/exit-path mechanics, one on completeness against this repo's own
CONTRIBUTING.md/AGENTS.mdcontribution rubric and against gaps sibling PRs were reviewed down for). Both found real, non-overlapping issues; both are folded into the current diff, verified together (not just individually) before pushing. Happy to expand on either review's specific findings if useful for re-review.Co-Authored-By: Claude Fable 5 noreply@anthropic.com
Update: two more independent reviews (security, concurrency/production-robustness)
After the round above, this branch went through two more adversarial passes — one hunting security issues specifically, one focused on concurrency and production robustness given the actual motivating use case (a Hatchet-orchestrated pipeline running multiple concurrent multi-step chains against the same session store). Both found real, distinct, confirmed issues, now folded into the diff:
Security:
SessionDB._remove_session_files(hermes_state.py) interpolatedsession_idraw into a glob pattern. An id of literally*— legal under the path-traversal guard, since*isn't a traversal character — expandsrequest_dump_{id}_*.jsonintorequest_dump_*_*.jsonand deletes every other session's request dumps.--resumemade this CLI-reachable for the first time (previously gateway-only). Fixed withglob.escape._is_unsafe_new_session_idonly ported one of the three checks the gateway's own equivalent entry boundary applies together. Added control-character rejection (CR/LF can forge log lines; ESC can inject terminal escapes into the rejection message itself) and the same 256-char length cap the gateway enforces (pastNAME_MAXthe session snapshot file becomes permanently uncreatable). Rejection messages now echo the offending value through a safe-echo helper so rejecting a hostile id can't itself be the injection vector.--yolobleed across resumed turns, toolset/model scope leakage, unbounded-history growth (handled by existing compression machinery), torn reads.reopen_sessionclearingend_reasonunconditionally (matches interactive chat's existing behavior — not a regression, just an existing shared question); no single-writer enforcement on a caller-minted id (not a vulnerability, but worth a docs line).Concurrency/production-robustness — this is the one I'd flag as most load-bearing given the actual use case:
--resumereads had no lock-contention patience, unlike every write path in this codebase (which gets 20–60s of jittered retry). Off WAL — which is the default on any pre-3.51.3 SQLite (i.e. stock CPython 3.11/3.12/3.13) or NFS/SMB/ZFS — a sibling connection holding the write lock for even a few seconds (a large FTS append, an incremental merge, a checkpoint, VACUUM — all real, existing code paths) made the read raisedatabase is locked, which the resume loader already swallowed to a stateless degrade. Concretely: a resumed turn would silently run with zero prior context, exit 0, and append its context-less answer into the middle of an otherwise-good transcript — full token cost paid, wrong answer produced, chain poisoned for every downstream step, with no error signal at all. Reproduced against a realSessionDBunder a held write lock. Fixed by giving the read path the same lock-patience budget the write path already has, escalating to a newResumeStoreContendedError(routed through the existing non-zero-exit error path, so an orchestrator can retry) only if the lock never actually clears — a genuinely broken store still degrades to stateless as before.SessionDBopen per resume call (main.py resolves the id, then oneshot.py opens its own store — measurable but small, and fixing it means threading a connection through a wider call chain than this PR should touch); bare--continueunder concurrency resolves to the workspace's most-recently-used session, which is correct per the flag's definition but means concurrent callers in the same repo must use--resume <stable-id>, never bare--continue— worth a docs note for anyone building an orchestrator on this.Tests:
tests/hermes_cli/test_oneshot_resume.pynow 40 passed (up from 26).tests/hermes_cli/ -k "resume or continue or oneshot or session": 217 passed, same 5 pre-existing unrelated dashboard-auth failures every prior review round hit.