feat(logging): opt-in HTTP/WS body capture to an isolated, share-excluded gui_bodies.log - #49044
Merged
teknium1 merged 1 commit intoJun 28, 2026
Conversation
…uded gui_bodies.log Stacked on #49003. That PR added always-on metadata (method/path/status/ latency + WS lifecycle) to the gui surface. This adds the heavy diagnostic tier — actual HTTP request bodies and PTY/WebSocket frames — for the hard dashboard/TUI bugs where metadata alone isn't enough. Body content can carry conversation data, so this is opt-in and built to be structurally incapable of leaking into a shared debug report (see #22016): - New config logging.capture_bodies (default false), surfaced in the dashboard / hermes tools config UI via _SCHEMA_OVERRIDES with a warning description. - When enabled, bodies go to a SEPARATE gui_bodies.log written by a dedicated logger (hermes_body_capture, propagate=False) that is deliberately NOT a member of any COMPONENT_PREFIXES. Four structural guarantees, all tested: 1. not under any component prefix -> never lands in gui.log / agent.log 2. not in hermes_cli/logs.py LOG_FILES -> not tailable via --- ~/.hermes/logs/agent.log (last 50) --- 2026-06-19 18:48:30,316 INFO [20260619_173001_f45949] agent.conversation_loop: API call #4: model=anthropic/claude-opus-4.8 provider=openrouter in=288583 out=529 total=289112 latency=10.6s cache=284912/288583 (99%) 2026-06-19 18:48:30,318 INFO [20260619_173001_f45949] agent.conversation_loop: Turn ended: reason=text_response(finish_reason=stop) model=anthropic/claude-opus-4.8 api_calls=4/16 budget=4/16 tool_turns=110 last_msg_role=assistant response_len=1474 session=20260619_173001_f45949 2026-06-19 18:48:30,325 INFO [20260619_173001_f45949] run_agent: OpenAI client closed (agent_close, shared=True, tcp_force_closed=0) thread=bg-review:6349795328 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:48:30,652 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-747 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:48:30,653 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #139: model=anthropic/claude-opus-4.8 provider=openrouter in=243960 out=991 total=244951 latency=11.5s cache=242196/243960 (99%) 2026-06-19 18:48:31,348 INFO [20260619_153431_51fd01] agent.tool_executor: tool terminal completed (0.69s, 161 chars) 2026-06-19 18:48:31,384 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-749 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:48:51,510 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-749 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:48:51,511 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #140: model=anthropic/claude-opus-4.8 provider=openrouter in=245038 out=1783 total=246821 latency=20.1s cache=243477/245038 (99%) 2026-06-19 18:48:52,215 INFO [20260619_153431_51fd01] agent.tool_executor: tool terminal completed (0.70s, 153 chars) 2026-06-19 18:48:52,245 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-751 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:48:59,489 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-751 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:48:59,490 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #141: model=anthropic/claude-opus-4.8 provider=openrouter in=246873 out=493 total=247366 latency=7.3s cache=244127/246873 (99%) 2026-06-19 18:49:13,666 INFO [20260619_153431_51fd01] agent.tool_executor: tool terminal completed (14.17s, 979 chars) 2026-06-19 18:49:13,692 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-753 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:49:22,930 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-753 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:49:22,932 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #142: model=anthropic/claude-opus-4.8 provider=openrouter in=247686 out=548 total=248234 latency=9.3s cache=245109/247686 (99%) 2026-06-19 18:49:23,254 INFO [20260619_153431_51fd01] agent.tool_executor: tool patch completed (0.10s, 1394 chars) 2026-06-19 18:49:23,287 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-762 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:49:26,661 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-762 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:49:26,662 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #143: model=anthropic/claude-opus-4.8 provider=openrouter in=248814 out=104 total=248918 latency=3.4s cache=246934/248814 (99%) 2026-06-19 18:49:27,958 INFO [20260619_153431_51fd01] agent.tool_executor: tool terminal completed (1.29s, 14487 chars) 2026-06-19 18:49:27,984 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-764 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:49:43,991 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-764 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:49:43,992 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #144: model=anthropic/claude-opus-4.8 provider=openrouter in=255375 out=938 total=256313 latency=16.0s cache=247771/255375 (97%) 2026-06-19 18:49:44,087 INFO [20260619_153431_51fd01] agent.conversation_loop: Turn ended: reason=text_response(finish_reason=stop) model=anthropic/claude-opus-4.8 api_calls=36/90 budget=31/90 tool_turns=129 last_msg_role=assistant response_len=2300 session=20260619_153431_51fd01 2026-06-19 18:49:44,112 INFO run_agent: OpenAI client created (agent_init, shared=True) thread=bg-review:6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:49:44,454 INFO [20260619_153431_51fd01] agent.turn_context: conversation turn: session=20260619_153431_51fd01 model=anthropic/claude-opus-4.8 provider=openrouter platform=cli history=310 msg='Review the conversation above and update the skill library. Be ACTIVE — most ses...' 2026-06-19 18:49:44,573 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-765 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:49:54,258 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-765 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:49:54,259 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #1: model=anthropic/claude-opus-4.8 provider=openrouter in=258322 out=423 total=258745 latency=9.8s cache=248822/258322 (96%) 2026-06-19 18:49:54,360 INFO [20260619_153431_51fd01] agent.tool_executor: tool skills_list completed (0.10s, 21152 chars) 2026-06-19 18:49:54,383 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-766 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:50:02,705 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-766 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:50:02,706 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #2: model=anthropic/claude-opus-4.8 provider=openrouter in=266258 out=313 total=266571 latency=8.3s cache=258320/266258 (97%) 2026-06-19 18:50:02,814 INFO [20260619_153431_51fd01] agent.tool_executor: tool skill_view completed (0.11s, 111769 chars) 2026-06-19 18:50:02,836 INFO [20260619_153431_51fd01] tools.tool_result_storage: Persisted large tool result: skill_view (toolu_01G7Zvw8ttjsUkomENppFu5T, 111769 chars -> /var/folders/p5/nqn3gs293rv3wtvf01pl9_vr0000gn/T/hermes-results/toolu_01G7Zvw8ttjsUkomENppFu5T.txt) 2026-06-19 18:50:02,861 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-769 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:50:12,687 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-769 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:50:12,688 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #3: model=anthropic/claude-opus-4.8 provider=openrouter in=267367 out=386 total=267753 latency=9.8s cache=258694/267367 (97%) 2026-06-19 18:50:12,749 INFO [20260619_153431_51fd01] agent.tool_executor: tool skill_view completed (0.06s, 22856 chars) 2026-06-19 18:50:12,776 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-770 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:50:18,989 INFO [20260619_153431_51fd01] agent.turn_context: conversation turn: session=20260619_153431_51fd01 model=anthropic/claude-opus-4.8 provider=openrouter platform=cli history=310 msg='yes' 2026-06-19 18:50:19,032 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-772 (_call):12901707776 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:50:21,530 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-770 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:50:21,531 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #4: model=anthropic/claude-opus-4.8 provider=openrouter in=276591 out=415 total=277006 latency=8.8s cache=266515/276591 (96%) 2026-06-19 18:50:21,585 WARNING [20260619_153431_51fd01] agent.tool_executor: Tool skill_view returned error (0.05s): {"success": false, "error": "File 'references/stacked-feature-prs.md' not found in skill 'incremental-architecture-refactor'.", "available_files": {}, "hint": "Use one of the available file paths list 2026-06-19 18:50:21,613 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-773 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:50:36,019 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-772 (_call):12901707776 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8 2026-06-19 18:50:36,020 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #145: model=anthropic/claude-opus-4.8 provider=openrouter in=256317 out=997 total=257314 latency=17.0s cache=256311/256317 (100%) 3. not in debug.py _capture_default_log_snapshots() -> NEVER uploaded by⚠️ This will upload the following to a public paste service: • System info (OS, Python version, Hermes version, provider, which API keys are configured — NOT the actual keys) • Recent log lines (agent.log, errors.log, gateway.log, desktop.log — may contain conversation fragments and file paths) • Full agent.log, gateway.log, and desktop.log (up to 512 KB each — likely contains conversation content, tool outputs, and file paths) Pastes auto-delete after 6 hours. Collecting debug report... Uploading... Debug report uploaded: Report https://paste.rs/nnfZj (failed to upload: agent.log: Failed to upload to any paste service: paste.rs: HTTP Error 500: Internal Server Error dpaste.com: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1016)>, gateway.log: Failed to upload to any paste service: paste.rs: HTTP Error 500: Internal Server Error dpaste.com: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1016)>, desktop.log: Failed to upload to any paste service: paste.rs: HTTP Error 500: Internal Server Error dpaste.com: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1016)>) ⏱ Pastes will auto-delete in 6 hours. To delete now: hermes debug delete <url> Share these links with the Hermes team for support. 4. still redacted via RedactingFormatter as defence-in-depth - Disabled state attaches a NullHandler and sets the level above CRITICAL, so _capture_body() is a cheap no-op (single isEnabledFor check) on the hot path. Captured bodies are truncated to 4096 bytes. request.body() is Starlette- cached, so reading it in the access middleware does not consume the stream for downstream handlers. Capture sites: HTTP request body (access middleware), PTY in/out frames. Tests (tests/test_hermes_logging.py::TestBodyCaptureOptIn): disabled-by-default creates no file and captures nothing; enabled writes to gui_bodies.log and the payload is ABSENT from gui.log; large bodies truncate; the body logger is isolated from every component; and the body file is excluded from both LOG_FILES and the debug-share snapshot set.
3 tasks
teknium1
merged commit Jun 28, 2026
ca52dd8
into
feat/dashboard-ws-http-observability
7 checks passed
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked on #49003 (base =
feat/dashboard-ws-http-observability; will auto-retarget tomainonce that merges). Part of #49035.#49003 added always-on metadata to the gui surface (method/path/status/latency + WS lifecycle). This adds the heavy diagnostic tier — actual HTTP request bodies and PTY/WebSocket frames — for the hard dashboard/TUI bugs where metadata alone isn't enough.
Body content can carry conversation data, so it is opt-in and built to be structurally incapable of leaking into a shared debug report (see #22016).
Design
logging.capture_bodies(defaultfalse), surfaced in the dashboard /hermes toolsconfig UI via_SCHEMA_OVERRIDESwith a warning description.gui_bodies.logwritten by a dedicated logger (hermes_body_capture,propagate=False) that is deliberately not a member of anyCOMPONENT_PREFIXES.Four structural guarantees (all tested)
gui.log/agent.log.hermes_cli/logs.py::LOG_FILES→ not tailable viahermes logs.debug.py::_capture_default_log_snapshots()→ never uploaded byhermes debug share.RedactingFormatteras defence-in-depth.Performance / safety
NullHandlerand sets level aboveCRITICAL, so_capture_body()is a cheap no-op (singleisEnabledForcheck) on the hot path.request.body()is Starlette-cached, so reading it in the access middleware does not consume the stream for downstream handlers.Test plan
pytest tests/test_hermes_logging.py tests/hermes_cli/test_debug.py tests/hermes_cli/test_web_server.py— 447 passed (3 pre-existingTestPtyWebSocketround-trip flakes, unrelated, deselected — they fail identically on the base branch in a sandboxed runner without a real TTY).TestBodyCaptureOptIn(5 cases):gui_bodies.log, nothing capturedgui_bodies.logand absent fromgui.logCOMPONENT_PREFIXESLOG_FILESand the debug-share snapshot setHERMES_HOME: disabled → nothing; enabled → captured but isolated.Usage
gui_bodies.logis intentionally read-directly-only — never tailed byhermes logs, never shipped byhermes debug share.