feat(delegation): add persistent ACP background subagents

[malaiwah]

Summary

Add persistent ACP background subagents inside Hermes-managed Docker sandboxes.

This keeps delegate_task unchanged and adds an explicit background workflow for subagents that should stay alive across turns.

Closes #4949.

What changed

• added a persistent ACP background-subagent runtime and manager
• added background tools:
  • spawn_background_subagent
  • list_background_subagents
  • send_background_subagent
  • poll_background_subagent
  • get_background_subagent_status
  • stop_background_subagent
• extended the Docker backend with long-lived interactive exec sessions
• injected an ephemeral per-turn roster of open background subagents
• added one-shot hidden nudges for completion / timeout / stop / channel loss
• added config defaults for delegation.background_subagents.*
• kept the runtime ACP-first and left an inbound dispatch path open for future child-to-parent requests

Validation

Focused slices:

• python -m pytest tests/agent/test_background_subagents.py tests/tools/test_docker_environment.py tests/test_model_tools.py tests/test_run_agent.py -q
• 251 passed
• python -m pytest tests/tools/test_delegate.py tests/tools/test_delegate_toolset_scope.py tests/hermes_cli/test_set_config_value.py tests/hermes_cli/test_tools_config.py -q
• 109 passed
• python -m pytest tests/acp/test_tools.py tests/gateway/test_api_server_toolset.py -q
• 41 passed
• python -m pytest tests/test_model_tools_async_bridge.py tests/test_agent_guardrails.py -q
• 39 passed

Full suite in this checkout:

• 7894 passed, 172 skipped, 1 xfailed, 7 failed
• the 7 failures are outside this PR's touched files and match the current local baseline in this repo:
  • tests/test_api_key_providers.py::TestHasAnyProviderConfigured::test_claude_code_creds_ignored_on_fresh_install
  • tests/test_api_key_providers.py::TestHasAnyProviderConfigured::test_config_dict_no_provider_no_creds_still_false
  • tests/hermes_cli/test_update_gateway_restart.py::TestCmdUpdateLaunchdRestart::test_update_with_systemd_still_restarts_via_systemd
  • tests/test_codex_execution_paths.py::test_gateway_run_agent_codex_path_handles_internal_401_refresh
  • tests/tools/test_file_read_guards.py::TestCharacterCountGuard::test_content_under_limit_passes
  • tests/tools/test_file_read_guards.py::TestCharacterCountGuard::test_oversized_read_rejected
  • tests/tools/test_file_read_guards.py::TestConfigOverride::test_custom_config_raises_limit

Notes

This first slice targets opencode acp and the Docker terminal backend.

[teknium1]

Thanks for the persistent ACP prototype. I found blockers that need redesign before this can be salvaged onto current main.

Problems

• run_agent.py:6376 and run_agent.py:6891 append mutable subagent roster/nudge data to effective_system on every API call. That breaks the repository's stable system-prompt/prompt-cache invariant. Current async delegation deliberately re-enters completions as fresh turns (tools/async_delegation.py:9-22; gateway/run.py:15374-15419).
• tools/background_subagent_tool.py:11-12 only checks an enabled-by-default config flag, while toolsets.py:56 adds six ACP/Docker-only tools to the global core list. Docker or the configured ACP executable is not verified before these schemas reach every model request.
• agent/background_subagents.py:467-474 responds to every ACP session/request_permission with allow_once; permission requests need the normal Hermes approval boundary rather than unconditional approval.

Suggested changes

• Route mutable status/completion through the existing async completion/new-turn rail, not the system prompt.
• Gate any ACP runtime on verified Docker and configured executable availability, and minimize its model-schema footprint.
• Bridge ACP permission requests to Hermes approval handling or fail closed.

This is an automated hermes-sweeper review.

[teknium1]

This makes effective_system depend on live subagent state for every API call, invalidating the conversation's cached system prefix. Please deliver roster/completion updates through the existing async-delegation completion/new-turn path instead of appending them to the system prompt.

[teknium1]

This gate is true by default and does not verify Docker or the configured ACP executable, yet these schemas are added to the core tool list. Gate discovery on the actual runtime prerequisites so unsupported ACP tools are not exposed to every model request.

[teknium1]

Returning allow_once here auto-approves every ACP permission request from the spawned agent. Route this through Hermes's approval policy/callback, or deny it when no approval bridge is available.

[malaiwah]

Closing this PR. After re-reviewing the full comment history and the current state of main, the implementation is architecturally incompatible with the direction the codebase has taken. A salvage would amount to a ground-up rewrite, not a rework.

Why close

1. Prompt-cache invariant violation (teknium1's first blocker stands).
run_agent.py:6376 and :6891 append mutable subagent roster/nudge data to effective_system on every API call. Main's tools/async_delegation.py:16-19 now codifies the hard invariant: completions must surface as a NEW turn, "never spliced between a tool result and an assistant message... prompt cache intact (never mutate past context)." This PR's delivery mechanism is fundamentally wrong.

2. Redundant with main.
Main now has a 1,516-line tools/async_delegation.py that does what this PR's agent/async_delegate_tasks.py (461 lines) was trying to do — but correctly: completions push onto process_registry.completion_queue as type="async_delegation" events, CLI and gateway drain that queue and forge fresh turns when idle, with durable persistence, crash-recovery, and de-dup. delegate_task(background=True) already works for single tasks and batches. The PR is a parallel, inferior implementation.

3. Ungated tool schemas (second blocker stands).
toolsets.py:56 adds 6 ACP/Docker-only tools to the global core list, gated only by an enabled-by-default config flag. No verification that Docker or the ACP executable exists. Every model request gets these schemas even when the runtime can't use them.

4. Auto-approve security hole (third blocker stands).
agent/background_subagents.py:471 returns allow_once for every ACP session/request_permission request, unconditionally approving whatever the spawned agent asks for. Needs to route through Hermes's approval boundary.

5. Scope creep.
Bundles subagent capability profiles (issue #4928, separate PR #4929), broad tools/delegate_tool.py changes (+324/−18), and infrastructure changes to tools/terminal_tool.py (+99/−57), tools/environments/docker.py (+122/−0), tools/file_tools.py (+11/−49). GottZ's triage noted the patch broadens beyond #4949's core cause.

What a correct implementation would look like

A new PR for issue #4949 should:

1. Route completions through async_delegation.py's queue, not the system prompt.
2. Gate tools on Docker/ACP availability (probe at startup, not a config flag).
3. Bridge ACP permissions to Hermes's existing approval boundary.
4. Drop the profile/infrastructure scope creep — profiles belong in feat(delegation): add named subagent capability profiles #4929.
5. Consider @easyvibecoding's warm-pool reuse keyed on {profile, workspace, model} and @lee-b's point that sandboxing is orthogonal to persistent agency.

Issue #4949 stays open as a valid feature request. Happy to re-open this work as a focused PR built on top of main's async delegation rail when time allows.

Thanks to @teknium1 for the detailed review and to @GottZ, @easyvibecoding, and @lee-b for the design discussion.