Skip to content

fix(oneshot): run memory-provider teardown on -z to stop SIGABRT (exit 134) at shutdown - #48664

Closed
hermes-tmw wants to merge 1 commit into
NousResearch:mainfrom
hermes-tmw:fix/oneshot-memory-teardown-sigabrt
Closed

fix(oneshot): run memory-provider teardown on -z to stop SIGABRT (exit 134) at shutdown#48664
hermes-tmw wants to merge 1 commit into
NousResearch:mainfrom
hermes-tmw:fix/oneshot-memory-teardown-sigabrt

Conversation

@hermes-tmw

Copy link
Copy Markdown

Problem

Running a one-shot prompt with hermes -z exits with 134 (SIGABRT, "Fatal Python error: Aborted") during interpreter shutdown when a memory provider that spawns background daemon threads is active (reproduced with the Honcho provider). The model response and the memory write both complete successfully first — the abort happens only as the process tears down, so it's cosmetic but produces a nonzero exit code and an alarming traceback.

Root cause

The -z oneshot path (hermes_cli/oneshot.py_run_agent) bypasses cli.py entirely and returns right after agent.chat(), so it never runs the session-end teardown that the interactive / -q paths run via _finalize_single_query / shutdown_memory_provider. The Honcho provider spawns several fire-and-forget daemon threads (prefetch / sync / dialectic / mem-write) that make blocking httpx calls. With no teardown, one can still be mid-request when Py_FinalizeEx runs; CPython routes the live daemon thread through PyThread_exit_threadpthread_exit, which glibc turns into the fatal abort.

Native backtrace (from coredumpctl), identical across crashes:

abort (libc)
pthread_exit (libc)
PyThread_exit_thread (python3.11)
PyEval_RestoreThread        # daemon thread re-acquiring the GIL after a blocking call
... pydantic_core / httpx ...  # mid-serialize of a Honcho request

Confirmed by toggling the provider: enabled: false → exit 0; re-enabling → exit 134. Switching Honcho's writeFrequency async→session was not sufficient (other untracked daemon threads remain), so this needs a code fix, not config.

Fix

hermes_cli/oneshot.py:

  1. _run_agent now calls agent.shutdown_memory_provider(...) after agent.chat() — the session-end teardown the oneshot path was missing — so providers flush their session and join their tracked threads (matching the interactive path). Best-effort: a failure here never masks the response already produced.
  2. A new _hard_exit(code) helper replaces the three post-agent returns in run_oneshot: it flushes logging + stdio (guarded by a SIGALRM deadman) then calls os._exit(code), bypassing Py_FinalizeEx so any untracked straggler daemon thread can't trigger the abort. This mirrors the existing os._exit worker-exit pattern already in cli.py. The early --provider/--toolsets validation returns (which run before any agent/threads exist) are left as plain returns.

Testing

  • hermes -p <profile> -z "Reply with exactly: ok" → prints ok, exit 0, no core dump. Reproduced cleanly across repeated runs, including the worst case with writeFrequency: async (async-writer daemon thread present).
  • Memory writes still land — the provider's session flush runs before exit (verified message count increasing in the Honcho store across runs).

Happy to adjust the approach — e.g. if you'd prefer to quiesce/join the provider's untracked daemon threads inside shutdown_memory_provider rather than hard-exit. The os._exit approach is the minimal, low-risk fix given the response + memory write are already complete at that point.

🤖 Generated with Claude Code

…t 134)

The hermes -z oneshot path bypassed cli.py's session-end finalizer, so it
never ran shutdown_memory_provider(). Honcho's fire-and-forget daemon threads
(prefetch/sync/dialectic/mem-write) could still be mid-httpx when Py_FinalizeEx
ran at interpreter exit, which CPython turns into 'Fatal Python error: Aborted'
(SIGABRT / exit 134) — after the response and memory write had already
completed.

Fix: (1) _run_agent runs shutdown_memory_provider() after agent.chat() to flush
the session and join tracked threads; (2) _hard_exit() flushes stdio/logging
and calls os._exit(), bypassing Py_FinalizeEx so untracked straggler daemon
threads can't trigger the abort (mirrors cli.py's worker-exit pattern, guarded
by a SIGALRM deadman).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@alt-glitch alt-glitch added type/bug Something isn't working comp/cli CLI entry point, hermes_cli/, setup wizard tool/memory Memory tool and memory providers P3 Low — cosmetic, nice to have duplicate This issue or pull request already exists labels Jun 18, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Duplicate of #37635 — both add the same fix in hermes_cli/oneshot.py: call agent.shutdown_memory_provider() after agent.chat() in the -z oneshot path to quiesce Honcho's untracked daemon threads before Py_FinalizeEx, eliminating the SIGABRT (exit 134) at shutdown. #37635 is the earlier still-open PR with this mechanism.

This PR additionally hardens exit via an os._exit-based _hard_exit() (bypassing interpreter finalization), but that's a superset at the same site rather than a distinct fix.

Related: #33543 (broader interpreter-finalization fix in hermes_cli/main.py — different mechanism), #43055 (the -z SIGABRT issue).

@hermes-tmw hermes-tmw closed this Jun 18, 2026
@hermes-tmw

Copy link
Copy Markdown
Author

Duplicate of #37635 — closing in favor of the earlier PR.

#37635 already adds agent.shutdown_memory_provider() after agent.chat() in the oneshot path, which is the core fix for the SIGABRT (exit 134). The _hard_exit() / os._exit hardening in this PR is a superset at the same site rather than a distinct mechanism, and can be proposed as a follow-up to #37635 if the maintainers want the extra belt-and-suspenders layer.

Thanks @alt-glitch for catching this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard duplicate This issue or pull request already exists P3 Low — cosmetic, nice to have tool/memory Memory tool and memory providers type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants