Skip to content

feat(logging): opt-in HTTP/WS body capture to an isolated, share-excluded gui_bodies.log - #49044

Merged
teknium1 merged 1 commit into
feat/dashboard-ws-http-observabilityfrom
feat/gui-body-capture-optin
Jun 28, 2026
Merged

feat(logging): opt-in HTTP/WS body capture to an isolated, share-excluded gui_bodies.log#49044
teknium1 merged 1 commit into
feat/dashboard-ws-http-observabilityfrom
feat/gui-body-capture-optin

Conversation

@kshitijk4poor

Copy link
Copy Markdown
Collaborator

Summary

Stacked on #49003 (base = feat/dashboard-ws-http-observability; will auto-retarget to main once that merges). Part of #49035.

#49003 added always-on metadata to the gui surface (method/path/status/latency + WS lifecycle). This adds the heavy diagnostic tier — actual HTTP request bodies and PTY/WebSocket frames — for the hard dashboard/TUI bugs where metadata alone isn't enough.

Body content can carry conversation data, so it is opt-in and built to be structurally incapable of leaking into a shared debug report (see #22016).

Design

  • New config logging.capture_bodies (default false), surfaced in the dashboard / hermes tools config UI via _SCHEMA_OVERRIDES with a warning description.
  • When enabled, bodies go to a separate gui_bodies.log written by a dedicated logger (hermes_body_capture, propagate=False) that is deliberately not a member of any COMPONENT_PREFIXES.

Four structural guarantees (all tested)

  1. Not under any component prefix → bodies never land in gui.log / agent.log.
  2. Not in hermes_cli/logs.py::LOG_FILES → not tailable via hermes logs.
  3. Not in debug.py::_capture_default_log_snapshots()never uploaded by hermes debug share.
  4. Still redacted via RedactingFormatter as defence-in-depth.

Performance / safety

  • Disabled state attaches a NullHandler and sets level above CRITICAL, so _capture_body() is a cheap no-op (single isEnabledFor check) on the hot path.
  • Captured bodies truncated to 4096 bytes.
  • request.body() is Starlette-cached, so reading it in the access middleware does not consume the stream for downstream handlers.
  • Capture sites: HTTP request body (access middleware), PTY in/out frames.

Test plan

  • pytest tests/test_hermes_logging.py tests/hermes_cli/test_debug.py tests/hermes_cli/test_web_server.py447 passed (3 pre-existing TestPtyWebSocket round-trip flakes, unrelated, deselected — they fail identically on the base branch in a sandboxed runner without a real TTY).
  • New TestBodyCaptureOptIn (5 cases):
    • disabled-by-default → no gui_bodies.log, nothing captured
    • enabled → payload written to gui_bodies.log and absent from gui.log
    • large body truncates
    • body logger isolated from every COMPONENT_PREFIXES
    • body file excluded from both LOG_FILES and the debug-share snapshot set
  • Live E2E against a temp HERMES_HOME: disabled → nothing; enabled → captured but isolated.

Usage

hermes config set logging.capture_bodies true   # or the dashboard toggle
# reproduce the issue, then read ~/.hermes/logs/gui_bodies.log directly
hermes config set logging.capture_bodies false  # turn it back off

gui_bodies.log is intentionally read-directly-only — never tailed by hermes logs, never shipped by hermes debug share.

…uded gui_bodies.log

Stacked on #49003. That PR added always-on metadata (method/path/status/
latency + WS lifecycle) to the gui surface. This adds the heavy diagnostic
tier — actual HTTP request bodies and PTY/WebSocket frames — for the hard
dashboard/TUI bugs where metadata alone isn't enough.

Body content can carry conversation data, so this is opt-in and built to be
structurally incapable of leaking into a shared debug report (see #22016):

- New config logging.capture_bodies (default false), surfaced in the dashboard
  / hermes tools config UI via _SCHEMA_OVERRIDES with a warning description.
- When enabled, bodies go to a SEPARATE gui_bodies.log written by a dedicated
  logger (hermes_body_capture, propagate=False) that is deliberately NOT a
  member of any COMPONENT_PREFIXES. Four structural guarantees, all tested:
    1. not under any component prefix  -> never lands in gui.log / agent.log
    2. not in hermes_cli/logs.py LOG_FILES -> not tailable via --- ~/.hermes/logs/agent.log (last 50) ---
2026-06-19 18:48:30,316 INFO [20260619_173001_f45949] agent.conversation_loop: API call #4: model=anthropic/claude-opus-4.8 provider=openrouter in=288583 out=529 total=289112 latency=10.6s cache=284912/288583 (99%)
2026-06-19 18:48:30,318 INFO [20260619_173001_f45949] agent.conversation_loop: Turn ended: reason=text_response(finish_reason=stop) model=anthropic/claude-opus-4.8 api_calls=4/16 budget=4/16 tool_turns=110 last_msg_role=assistant response_len=1474 session=20260619_173001_f45949
2026-06-19 18:48:30,325 INFO [20260619_173001_f45949] run_agent: OpenAI client closed (agent_close, shared=True, tcp_force_closed=0) thread=bg-review:6349795328 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:48:30,652 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-747 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:48:30,653 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #139: model=anthropic/claude-opus-4.8 provider=openrouter in=243960 out=991 total=244951 latency=11.5s cache=242196/243960 (99%)
2026-06-19 18:48:31,348 INFO [20260619_153431_51fd01] agent.tool_executor: tool terminal completed (0.69s, 161 chars)
2026-06-19 18:48:31,384 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-749 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:48:51,510 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-749 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:48:51,511 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #140: model=anthropic/claude-opus-4.8 provider=openrouter in=245038 out=1783 total=246821 latency=20.1s cache=243477/245038 (99%)
2026-06-19 18:48:52,215 INFO [20260619_153431_51fd01] agent.tool_executor: tool terminal completed (0.70s, 153 chars)
2026-06-19 18:48:52,245 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-751 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:48:59,489 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-751 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:48:59,490 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #141: model=anthropic/claude-opus-4.8 provider=openrouter in=246873 out=493 total=247366 latency=7.3s cache=244127/246873 (99%)
2026-06-19 18:49:13,666 INFO [20260619_153431_51fd01] agent.tool_executor: tool terminal completed (14.17s, 979 chars)
2026-06-19 18:49:13,692 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-753 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:49:22,930 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-753 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:49:22,932 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #142: model=anthropic/claude-opus-4.8 provider=openrouter in=247686 out=548 total=248234 latency=9.3s cache=245109/247686 (99%)
2026-06-19 18:49:23,254 INFO [20260619_153431_51fd01] agent.tool_executor: tool patch completed (0.10s, 1394 chars)
2026-06-19 18:49:23,287 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-762 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:49:26,661 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-762 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:49:26,662 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #143: model=anthropic/claude-opus-4.8 provider=openrouter in=248814 out=104 total=248918 latency=3.4s cache=246934/248814 (99%)
2026-06-19 18:49:27,958 INFO [20260619_153431_51fd01] agent.tool_executor: tool terminal completed (1.29s, 14487 chars)
2026-06-19 18:49:27,984 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-764 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:49:43,991 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-764 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:49:43,992 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #144: model=anthropic/claude-opus-4.8 provider=openrouter in=255375 out=938 total=256313 latency=16.0s cache=247771/255375 (97%)
2026-06-19 18:49:44,087 INFO [20260619_153431_51fd01] agent.conversation_loop: Turn ended: reason=text_response(finish_reason=stop) model=anthropic/claude-opus-4.8 api_calls=36/90 budget=31/90 tool_turns=129 last_msg_role=assistant response_len=2300 session=20260619_153431_51fd01
2026-06-19 18:49:44,112 INFO run_agent: OpenAI client created (agent_init, shared=True) thread=bg-review:6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:49:44,454 INFO [20260619_153431_51fd01] agent.turn_context: conversation turn: session=20260619_153431_51fd01 model=anthropic/claude-opus-4.8 provider=openrouter platform=cli history=310 msg='Review the conversation above and update the skill library. Be ACTIVE — most ses...'
2026-06-19 18:49:44,573 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-765 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:49:54,258 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-765 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:49:54,259 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #1: model=anthropic/claude-opus-4.8 provider=openrouter in=258322 out=423 total=258745 latency=9.8s cache=248822/258322 (96%)
2026-06-19 18:49:54,360 INFO [20260619_153431_51fd01] agent.tool_executor: tool skills_list completed (0.10s, 21152 chars)
2026-06-19 18:49:54,383 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-766 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:50:02,705 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-766 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:50:02,706 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #2: model=anthropic/claude-opus-4.8 provider=openrouter in=266258 out=313 total=266571 latency=8.3s cache=258320/266258 (97%)
2026-06-19 18:50:02,814 INFO [20260619_153431_51fd01] agent.tool_executor: tool skill_view completed (0.11s, 111769 chars)
2026-06-19 18:50:02,836 INFO [20260619_153431_51fd01] tools.tool_result_storage: Persisted large tool result: skill_view (toolu_01G7Zvw8ttjsUkomENppFu5T, 111769 chars -> /var/folders/p5/nqn3gs293rv3wtvf01pl9_vr0000gn/T/hermes-results/toolu_01G7Zvw8ttjsUkomENppFu5T.txt)
2026-06-19 18:50:02,861 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-769 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:50:12,687 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-769 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:50:12,688 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #3: model=anthropic/claude-opus-4.8 provider=openrouter in=267367 out=386 total=267753 latency=9.8s cache=258694/267367 (97%)
2026-06-19 18:50:12,749 INFO [20260619_153431_51fd01] agent.tool_executor: tool skill_view completed (0.06s, 22856 chars)
2026-06-19 18:50:12,776 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-770 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:50:18,989 INFO [20260619_153431_51fd01] agent.turn_context: conversation turn: session=20260619_153431_51fd01 model=anthropic/claude-opus-4.8 provider=openrouter platform=cli history=310 msg='yes'
2026-06-19 18:50:19,032 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-772 (_call):12901707776 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:50:21,530 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-770 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:50:21,531 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #4: model=anthropic/claude-opus-4.8 provider=openrouter in=276591 out=415 total=277006 latency=8.8s cache=266515/276591 (96%)
2026-06-19 18:50:21,585 WARNING [20260619_153431_51fd01] agent.tool_executor: Tool skill_view returned error (0.05s): {"success": false, "error": "File 'references/stacked-feature-prs.md' not found in skill 'incremental-architecture-refactor'.", "available_files": {}, "hint": "Use one of the available file paths list
2026-06-19 18:50:21,613 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-773 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:50:36,019 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-772 (_call):12901707776 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:50:36,020 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #145: model=anthropic/claude-opus-4.8 provider=openrouter in=256317 out=997 total=257314 latency=17.0s cache=256311/256317 (100%)
    3. not in debug.py _capture_default_log_snapshots() -> NEVER uploaded by
       ⚠️  This will upload the following to a public paste service:
  • System info (OS, Python version, Hermes version, provider, which API keys
    are configured — NOT the actual keys)
  • Recent log lines (agent.log, errors.log, gateway.log, desktop.log — may
    contain conversation fragments and file paths)
  • Full agent.log, gateway.log, and desktop.log (up to 512 KB each — likely
    contains conversation content, tool outputs, and file paths)

Pastes auto-delete after 6 hours.

Collecting debug report...
Uploading...

Debug report uploaded:
  Report  https://paste.rs/nnfZj

  (failed to upload: agent.log: Failed to upload to any paste service:
  paste.rs: HTTP Error 500: Internal Server Error
  dpaste.com: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1016)>, gateway.log: Failed to upload to any paste service:
  paste.rs: HTTP Error 500: Internal Server Error
  dpaste.com: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1016)>, desktop.log: Failed to upload to any paste service:
  paste.rs: HTTP Error 500: Internal Server Error
  dpaste.com: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1016)>)

⏱  Pastes will auto-delete in 6 hours.
To delete now:  hermes debug delete <url>

Share these links with the Hermes team for support.
    4. still redacted via RedactingFormatter as defence-in-depth
- Disabled state attaches a NullHandler and sets the level above CRITICAL, so
  _capture_body() is a cheap no-op (single isEnabledFor check) on the hot path.
  Captured bodies are truncated to 4096 bytes. request.body() is Starlette-
  cached, so reading it in the access middleware does not consume the stream
  for downstream handlers.

Capture sites: HTTP request body (access middleware), PTY in/out frames.

Tests (tests/test_hermes_logging.py::TestBodyCaptureOptIn): disabled-by-default
creates no file and captures nothing; enabled writes to gui_bodies.log and the
payload is ABSENT from gui.log; large bodies truncate; the body logger is
isolated from every component; and the body file is excluded from both
LOG_FILES and the debug-share snapshot set.
@alt-glitch alt-glitch added type/feature New feature or request comp/cli CLI entry point, hermes_cli/, setup wizard area/config Config system, migrations, profiles P3 Low — cosmetic, nice to have labels Jun 19, 2026
@alt-glitch alt-glitch added the comp/dashboard Web dashboard / control panel UI (dashboard/, landing) label Jun 26, 2026
@teknium1
teknium1 merged commit ca52dd8 into feat/dashboard-ws-http-observability Jun 28, 2026
7 checks passed
@teknium1
teknium1 deleted the feat/gui-body-capture-optin branch June 28, 2026 11:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/cli CLI entry point, hermes_cli/, setup wizard comp/dashboard Web dashboard / control panel UI (dashboard/, landing) P3 Low — cosmetic, nice to have type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants