feat(cron): skip background review on cron + usage audit logger (salvage of #18255) - #81254
Merged
Merged
Conversation
Phase 8 of the Hermes Agent token leak mitigation plan
(ralplan-hermes-token-leaks.md §3.9). Adds a boolean kwarg
`skip_background_review` (default False) to AIAgent.__init__ that
suppresses the end-of-turn _spawn_background_review fork.
Each background review fork instantiates a new AIAgent with its own
~15K input tokens + up to 8 LLM iterations, accumulating ~30K tokens
per event in the worst case. On cron sessions there is no
human-in-the-loop benefit from the review (no skill-creation pressure,
nobody curating MEMORY.md), so the cost is pure waste.
The end-of-turn guard now reads:
if (final_response and not interrupted
and not getattr(self, "skip_background_review", False)
and (_should_review_memory or _should_review_skills)):
skip_memory=True already disables the memory-review trigger; this
flag is the explicit single-switch off for both review paths.
Defaults to False, so behavior is unchanged for gateway/CLI callers
that omit the kwarg.
Tests: 5 new unit tests in tests/agent/test_skip_background_review.py
covering the default value, flag persistence, the gate short-circuit,
the gate fall-through, and a source-text assertion that the cron
scheduler sets the flag to True (separate commit).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…-presence
Phase 8 wire-in + Vector 8 doc comment from
ralplan-hermes-token-leaks.md.
(1) Phase 8 wire-in: cron AIAgent construction now passes
skip_background_review=True. This suppresses the end-of-turn
skill/memory review fork (~30K tokens/event, ≤30K typical and
≤150K worst-case daily on bluenode) which has no human-in-the-loop
value for cron sessions.
(2) Vector 8 doc comment: a one-line comment immediately above the
AIAgent(...) construction documenting the verified-negative
finding that title generation does not run on the cron path
(maybe_auto_title is gateway/CLI-side only). Future contributors
won't accidentally introduce title-gen here without realizing it
would add ~600-1000 tokens/fire on a path that explicitly opts out
of memory/review/title overhead.
No new tests required for the doc comment (no behavior change). The
skip_background_review wiring is covered by the existing source-text
assertion in tests/agent/test_skip_background_review.py.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ntation Phase 0.5 of the Hermes Agent token leak mitigation plan: append a single JSONL line to ~/.hermes/cron/usage_audit.jsonl after every cron LLM invocation, capturing prompt/completion/total tokens, model, duration_ms, deliver target, and error (when raised). Read from agent.session_*_tokens which run_conversation already returns in its result dict. Without this, we have no measured baseline to attribute token deltas to subsequent mitigation phases. The plan's hard gate: observability lands before any mitigation phase. Writer NEVER raises — wrapped in a single try/except that logs a warning on any json.dumps / mkdir / open failure so an audit-log bug cannot break a cron job. Failure-path audit guard via locals() check covers exceptions that fire before the fire_id is assigned. No new dependencies, no new env vars (the plan rejected one in v2). Tests: 7 new unit tests in tests/cron/test_usage_audit_logger.py covering the success path, missing token info, swallowed writer exception, parent dir creation, multiple appends, and unicode preservation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
kshitijk4poor
force-pushed
the
salvage/pr-18255
branch
from
August 7, 2026 18:16
265d7fe to
83ddb88
Compare
alt-glitch
added
type/perf
- Fix _usage_audit_path() to use _get_hermes_home() instead of hardcoded Path.home() / '.hermes' (profile-safe resolution, sweeper finding) - Rewrite skip_background_review tests to exercise finalize_turn() directly instead of duplicating the guard expression (sweeper finding) - Fix response_silent audit field to use _is_cron_silence_response() instead of the buggy SILENT_MARKER substring check it was meant to replace (simplify-code review finding) - Remove dead 'model' in locals() guard — model is always in scope before the try block (simplify-code review finding) - Extract _stub_agent_for_finalize() helper to eliminate ~40 lines of copy-pasted agent stubbing in tests (simplify-code review finding) - Clean up 'Phase 0.5' instrumentation comments
kshitijk4poor
force-pushed
the
salvage/pr-18255
branch
from
August 7, 2026 18:33
83ddb88 to
2d9c4f6
Compare
kshitijk4poor
enabled auto-merge (rebase)
August 7, 2026 18:33
Closed
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Salvages the worthwhile portions of PR #18255 (cron token-leak mitigation): adds a
skip_background_reviewflag onAIAgentto skip the_spawn_background_reviewfork on cron sessions, and ausage_audit.jsonllogger for per-fire cron token spend instrumentation.The
HERMES_CRON_MAX_TOKENSenv var portion was dropped (conflicts with the no-user-facing-max-tokens policy). Thedocs/wave-2-deferred.mdwas also dropped (not appropriate for the repo).What was salvaged
Three commits from @0xarkstar, cherry-picked onto current main:
feat(agent): add skip_background_review flag to AIAgent constructor— new boolean flag threaded throughinit_agent→AIAgent.__init__. DefaultFalse(unchanged behavior for CLI/gateway). Cron setsTrue.feat(cron): set skip_background_review=True— cron scheduler constructs AIAgent with the flag.feat(cron): add usage_audit.jsonl logger— append-only JSONL audit log per cron fire (tokens, duration, model, deliver target, error). Fail-safe (exceptions swallowed).Follow-up fixes (our commit)
_usage_audit_path()to use_get_hermes_home()instead of hardcodedPath.home() / ".hermes"— profile-safe resolution (sweeper finding).skip_background_reviewtests to exercisefinalize_turn()directly instead of duplicating the guard expression — catches divergence between production guard and tests (sweeper finding)._usage_audit_pathdoes NOT callPath.home().What was dropped
HERMES_CRON_MAX_TOKENSenv var — adds a user-facingmax_tokensknob, conflicting with the standing policy that.envis for secrets only and behavioral config belongs inconfig.yaml.docs/wave-2-deferred.md— internal planning doc, not appropriate for the repo.Validation
Closes #18255