Skip to content

fix(security): handle tirith SIGSEGV crash and disable broken binary (#41400) - #41446

Closed
kyssta-exe wants to merge 1 commit into
NousResearch:mainfrom
kyssta-exe:auto-fix/issue-41400
Closed

fix(security): handle tirith SIGSEGV crash and disable broken binary (#41400)#41446
kyssta-exe wants to merge 1 commit into
NousResearch:mainfrom
kyssta-exe:auto-fix/issue-41400

Conversation

@kyssta-exe

Copy link
Copy Markdown
Contributor

Fixes #41400. When the tirith binary crashes with a signal (e.g. SIGSEGV), the return code is negative (-11). The old code treated this as an unknown exit code and allowed it through, but kept retrying the broken binary on every tool invocation. Now signal-killed processes are detected via negative return codes, tirith is disabled for the process lifetime after the first crash, and the failure is logged with the signal name.

@liuhao1024

Copy link
Copy Markdown
Contributor

One edge-case issue: _signal.Signals(-exit_code) will raise ValueError if -exit_code == 32, because signal 32 is not defined in Python's signal.Signals enum on most platforms (macOS has signals 1–31, Linux NPTL reserves 32–33 but they're not in the CPython enum).

The guard -exit_code <= 32 should be -exit_code < 32 to stay within the standard POSIX signal range (1–31), or wrap the lookup in a try/except:

try:
    sig_name = _signal.Signals(-exit_code).name
except (ValueError, OverflowError):
    sig_name = f"signal {-exit_code}"

This prevents an unhandled ValueError crash if tirith happens to be killed by signal 32.

@alt-glitch alt-glitch added type/security Security vulnerability or hardening P1 High — major feature broken, no workaround comp/tools Tool registry, model_tools, toolsets labels Jun 7, 2026
@kyssta-exe
kyssta-exe force-pushed the auto-fix/issue-41400 branch from e87d8d0 to dacef36 Compare June 7, 2026 20:03

@egilewski egilewski left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommendation: needs rework before merge.

Checked current GitHub main b5f8996ccc2163ef06b4265d0882019fc24b0682 against PR head dacef36f41aef4bb0039433ad6bfcbc0bb29f385.

What validates: a mocked Tirith SIGSEGV probe (subprocess.run returning -11) shows current main keeps invoking Tirith on every call, while the PR stops invoking it after three failures and returns tirith disabled (circuit breaker). The focused Tirith test file passes on both sides:

  • PYTHONDONTWRITEBYTECODE=1 HERMES_HOME=<run-root>/tmp/hermes-home-base python -m pytest -o addopts='' -p no:cacheprovider tests/tools/test_tirith_security.py -q -> 92 passed
  • PYTHONDONTWRITEBYTECODE=1 HERMES_HOME=<run-root>/tmp/hermes-home-pr python -m pytest -o addopts='' -p no:cacheprovider tests/tools/test_tirith_security.py -q -> 92 passed

Blocker: the open-circuit path ignores tirith_fail_open: false. In the PR replay, the same mocked -11 sequence with tirith_fail_open=False returns block for the first three calls, then returns allow with tirith disabled (circuit breaker) for all later calls. Current main blocks every mocked -11 under fail-closed mode. The early _circuit_open return in tools/tirith_security.py runs before fail-open policy is applied, so this changes a configured fail-closed scanner failure into fail-open after the circuit opens.

CodeRabbit also ran successfully and reported one critical finding: the new global _crash_count / _circuit_open updates are unsynchronized across concurrent check_command_security calls, so increments and the circuit-open transition can race.

Other validation:

  • git diff --check HEAD fails on tests/tools/test_tirith_security.py:1249 due added trailing whitespace.
  • GitHub reports mergeable=MERGEABLE, mergeStateStatus=BLOCKED; test (3) is failing in tests/run_agent/test_run_agent.py::TestConcurrentToolExecution::test_concurrent_handles_tool_error, separate from the focused Tirith test pass.

Signed: GPT-5.5-xhigh in Codex

@teknium1 teknium1 added sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jun 21, 2026
@kshitijk4poor

Copy link
Copy Markdown
Collaborator

Merged via #52997. Your circuit breaker was chosen as the base — clean, focused, matching the existing code patterns. Cherry-picked with authorship preserved (rebase-merge). A follow-up extracted _record_tirith_crash() helper to dedup the 3 copy-pasted blocks and documented thread safety. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/tools Tool registry, model_tools, toolsets P1 High — major feature broken, no workaround sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data type/security Security vulnerability or hardening

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Security scanner (Tirith) SIGSEGV crash causes 20-minute agent hang with no response delivery

6 participants