feat: add opt-in execution receipts - #49371
Draft
mistakeknot wants to merge 1 commit into
Draft
Conversation
mistakeknot
force-pushed
the
feat/execution-receipts
branch
from
June 20, 2026 18:12
3fc012b to
b4d196c
Compare
teknium1
reviewed
Jul 14, 2026
teknium1
left a comment
Contributor
There was a problem hiding this comment.
Thanks for the privacy-focused local observability proposal. Current main still exposes raw post_tool_call observations (model_tools.py:974-1022) but has no redacted durable receipt stream, so the feature remains distinct.
Problems
agent/execution_receipts.py:157persists arbitrary mapping keys asfield_names. Mapping keys are caller-controlled data, so this can store sensitive argument content despite the documented no-raw-arguments guarantee.- The executor has moved substantially since the PR base. Current
agent/tool_executor.py:360-375models malformed arguments as a non-executed outcome, and current tests enforce aligned concurrent/sequential post-hook ownership attests/run_agent/test_run_agent.py:3389-3510. Receipt wiring needs to cover those current paths explicitly.
Suggested changes
- Use schema-derived safe field names, or omit field names; add a test with a secret-bearing key.
- Integrate against the current executor and test malformed-argument receipt status plus concurrent/sequential parity.
This is an automated hermes-sweeper review.
| "size_bytes": _payload_size_bytes(payload), | ||
| } | ||
| if isinstance(payload, Mapping): | ||
| metadata["field_names"] = sorted(str(key) for key in payload.keys())[:50] |
Contributor
There was a problem hiding this comment.
field_names is persisted metadata, but mapping keys are caller-controlled data. Please derive names from a trusted tool schema or omit them, and add a test where a key itself contains a secret; otherwise this violates the stated no-raw-arguments privacy contract.
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why this is useful
Hermes can already run long, multi-tool, delegated sessions across CLI, desktop, and gateways, but today there is no small built-in evidence stream that answers basic post-run questions without storing raw tool payloads:
This PR adds that minimal substrate as an opt-in, local-only observability primitive. It is intentionally not an analytics/exporter feature and not a signing/provenance system yet. The value is giving Hermes plugins, local dashboards, and future audit/provenance work a stable redacted event shape to build on without asking them to persist raw
post_tool_callargs/results.Refs #6642 and #5041.
Summary
Adds an opt-in local execution-receipts substrate for agent-loop tool outcomes:
execution_receiptplugin hook with a cheap no-op path when no listener is registeredok,error,blocked,cancelled)observability/execution_receiptsplugin that writes owner-only local JSONL under$HERMES_HOME/execution-receipts/receipts.jsonl/receipts status,/receipts tail [N], and/receipts gapshelper commandsAlternatives / existing overlaps
post_tool_callhooks are the closest integration seam today. They already power observability plugins, but intentionally pass rawargsandresultto observers. This PR adds a narrower redacted envelope so durable local evidence sinks do not each need to invent their own redaction contract.observability/langfuseandobservability/nemo_relayplugins provide richer tracing/export paths. They are great for tracing, but are external/dependency-oriented and not a minimal local JSONL receipt stream.Safety / privacy defaults
This is intentionally a small P1:
args/resultmetadata shape before writing, so a buggy caller cannot smuggle raw payload fields by settingredacted: trueHERMES_*env vars or.envconfigurationNon-goals in this PR
Design question for maintainers
I used a dedicated
execution_receipthook instead of extendingpost_tool_callbecause receipts are redacted durable evidence records, whilepost_tool_callremains useful for observers that intentionally receive raw tool args/results. Happy to collapse this into the existing hook surface if maintainers prefer fewer hook names.Test plan
Final local validation after rebasing onto current
origin/main: