feat(gateway): resume interrupted sessions after restart

[malaiwah]

Summary

• persist an opt-in restart ledger for in-flight gateway turns during graceful shutdown
• restore interrupted sessions on startup and enqueue a hidden continuation turn from persisted context
• preserve queued follow-up user input when resuming and surface approval-loss context safely

Validation

• source venv/bin/activate && python -m pytest tests/gateway/test_restart_resume.py tests/gateway/test_gateway_shutdown.py tests/gateway/test_session_race_guard.py tests/gateway/test_reasoning_command.py tests/gateway/test_config.py tests/gateway/test_update_command.py tests/gateway/test_resume_command.py tests/gateway/test_queue_consumption.py tests/gateway/test_platform_reconnect.py tests/test_anthropic_error_handling.py -q
• full suite in this checkout remains at the current rebased baseline: 19 failed, 8228 passed, 27 skipped, 1 xfailed

Closes #5225

[trevorgordon981]

Big surface change but isolated to gateway shutdown/startup paths with a dedicated test file (+294). Opt-in ledger is the right call for a new persistence path. Proceed.

Note the rebased baseline still has 19 failing tests unrelated to this PR — worth confirming those are tracked separately before merge.

[malaiwah]

I checked the rebased baseline on the exact upstream branch/worktree for this PR. The original 19 failing tests note overstates the number of distinct issues.

What I could reproduce on the current branch:

• tests/gateway/test_matrix_voice.py is skipped here, not failing
• tests/tools/test_delegate.py and tests/tools/test_delegate_toolset_scope.py pass here
• the active failures collapse to 5 root causes:

1. tests/tools/test_skill_manager_tool.py (10 failures)

   • one shared regression in tools/skill_manager_tool.py::_find_skill()
   • it now resolves skill dirs via agent.skill_utils.get_all_skills_dirs(), which ignores the monkeypatched module-local SKILLS_DIR used by these tests

2. tests/test_api_key_providers.py (2 failures)

   • likely test-isolation drift rather than a product regression
   • _has_any_provider_configured() now checks all provider env vars from the registry, but these tests only clear the older common vars

3. tests/hermes_cli/test_update_gateway_restart.py (1 failure)

   • small output/assertion drift: restart still runs, but the test expects the old exact text

4. tests/tools/test_browser_camofox_state.py (1 failure)

   • stale _config_version expectation (11 vs current 12)

5. tests/test_codex_execution_paths.py (1 failure)

   • looks like a stale test seam around gateway model resolution; the gateway path now resolves model from config, while the test only sets HERMES_MODEL

So the short version is: not all of the old baseline reds are tracked already, but most of what remains is either test drift or one shared regression cluster unrelated to this PR’s gateway restart-resume changes.

[malaiwah]

Follow-up on the unrelated baseline reds:

• skill manager regression split out to #5284
• gateway message-shadow / Codex path fix split out to #5287

Those are the two concrete upstreamable follow-ups I peeled off from the baseline triage, separate from this restart-resume PR.

[malaiwah]

Follow-up cleanup PR for the trivial drift / baseline test issues discussed here: #5321.

It covers the stale test expectations and the execute_code/gateway_local sandbox guard alignment, and it is packaged as a clean branch on top of upstream/main.

[malaiwah]

Follow-up baseline cleanup is now split out in #5329. It isolates process-global terminal/code-execution test state so cached sandboxes and interrupt flags do not leak across the full suite. The internal-only Matrix fake-nio fallback was kept out of that PR because upstream already handles missing matrix-nio differently.

[teknium1]

Thanks for this PR, @malaiwah — the design here is solid and the opt-in ledger approach was exactly the right call.

This is an automated hermes-sweeper review. The feature this PR proposes has since been implemented on main and shipped in v2026.4.23:

• Commit: cb4addaca — fix(gateway): auto-resume sessions after drain-timeout restart (spec: automatic session resume after gateway restart #11852) (fix(gateway): auto-resume sessions after drain-timeout restart (#11852) #12301)
• SessionEntry.resume_pending / resume_reason fields added in gateway/session.py (line 462)
• SessionStore.mark_resume_pending() / clear_resume_pending() at lines 952–995
• Drain-timeout branch in _stop_impl() marks active sessions resume_pending before interrupting agents (gateway/run.py:2771)
• Reason-aware system note injection in _run_agent() (gateway/run.py:10180), subsuming the tool-tail auto-continue note
• suspend_recently_active() skips resume_pending entries; suspended=True still wins
• 610-line test suite in tests/gateway/test_restart_resume_pending.py (29 tests, superset of the 294-line file proposed here)

The follow-up PRs you split out (#5284, #5287, #5321, #5329) addressed the baseline test drift separately — those are tracked independently. Closing this PR as implemented.