feat(cron): token leak mitigation — instrumentation, max_tokens cap, background-review skip by 0xarkstar · Pull Request #18255 · NousResearch/hermes-agent

0xarkstar · 2026-05-01T05:41:39Z

Summary (TL;DR)

Three independently-revertable cron-path token-discipline fixes:

HERMES_CRON_MAX_TOKENS env var caps LLM response on cron jobs. Mitigates degenerate-loop hallucinations on small/open-source models (single repetition trap producing 64KB+ output flooding chat with 33+ message bursts). Default None = no cap (back-compat).
usage_audit.jsonl writer logs prompt/completion/total tokens, fire_id, deliver_target, response-silent flag, and duration to ~/.hermes/cron/usage_audit.jsonl per cron fire. Append-only, fail-safe (writer exceptions swallowed). Provides ground-truth measurement for any future leak claim and a substrate for cost dashboards.
skip_background_review AIAgent flag gates the _spawn_background_review fork (~15K input + up to 8 LLM iterations / event) on a single boolean. Cron sets True since cron sessions don't need skill-curation reviews; CLI / interactive paths default to False and behave unchanged.

Plus a one-line code comment documenting that maybe_auto_title is gateway/CLI-only and never invoked on the cron path (verified via grep across cron/scheduler.py, run_agent.py).

docs/wave-2-deferred.md registers items intentionally deferred from this wave: tool-description compaction (gated on a regression suite), provider-cache integration for SKILL.md blocks, a precheck_command schema RFC, and streaming-aware history compression.

Background

While operating a Discord bot on this gateway (cron */5 recommend-sweep + */15 escalation-sweep + daily reminders), I measured cron path daily token spend at ~11M tokens/day input. Audit traced the dominant share to: tool-schema bloat per call (~12K tokens), no provider-side cache on non-Anthropic backends, and a degenerate-loop incident producing 64KB output. This PR addresses the directly fixable surface; the larger architectural items (cron LLM-mandatory, precheck_command schema) are filed in docs/wave-2-deferred.md for follow-up RFC.

Production deployment

This branch is currently the production runtime for BlueNode Hermes profile (OCI ARM64 Graviton) — operations Discord bot for Inha University's blockchain club. Token-discipline fixes here were diagnosed and validated against that workload (cron */5 recommend-sweep, daily reminders, escalation handling).

Test plan

tests/cron/test_usage_audit_logger.py — 9 tests covering JSONL schema, missing-token fallback, writer exception swallow, missing-parent-dir creation
tests/agent/test_skip_background_review.py — 5 tests covering default-False, flag-persists, gate-short-circuits, gate-fires-when-unset, cron-source-asserts-True
pytest tests/cron/ tests/agent/ green locally (13/13 new tests pass; 342/342 cron suite pass; no regressions)
ruff check — no new warnings introduced (pre-existing E402 in cron/scheduler.py:113 is unchanged)
Rebased onto upstream/main HEAD 44cdf555a (2026-05-10) — clean, MERGEABLE
Maintainer smoke-test against an existing deployment with a representative cron job

Complementary upstream change

Note: complementary to upstream e5af1dd63 fix(review): tell background reviewer not to capture transient env failures as skills (#23004) — that PR keeps the reviewer smarter on non-cron paths; this PR skips the reviewer entirely on the cron path. Both can land independently in either order.

Batch consideration

Logical mini-batch with #15508 (configurable background review routing) and #21511 (skip external memory providers in background review). All three touch run_agent.py background-review path:

This PR (feat(cron): token leak mitigation — instrumentation, max_tokens cap, background-review skip #18255): instrumentation + a hard skip-gate for cron context
feat(background_review): add configurable background review routing #15508: softer routing knob (which model/provider for background review)
fix(agent): skip external memory providers in background review agent #21511: scope-limited skip for external memory providers

These are compatible and can be reviewed/merged in any order; bundling them into a single "background review hardening" cycle could be efficient if the maintainer prefers.

Salvage-friendly

This PR is structured for salvage: each commit is independently revertable, no abstractions added, all touchpoints isolated to cron / agent. Happy to revise scope, rebase, or split commits on demand. Feedback welcome.

🤖 Generated with Claude Code

Cron agents on small/open-source models can fall into degenerate-loop hallucinations (e.g. "output is not output is not..." repetition trap) that produce 64KB+ of repetitive text before the API max-tokens limit fires. When combined with cron deliver mechanism, this gets split into 30+ Discord messages via Hermes scheduler — channel spam. Add HERMES_CRON_MAX_TOKENS env var that, when set, caps the AIAgent max_tokens for cron jobs. Default None preserves existing behavior. With HERMES_CRON_MAX_TOKENS=2048, worst-case hallucination is bounded to ~6000 chars (3 Discord messages) instead of 33+ message bursts. Real-world incident: BlueNode bot (Gemma 4 31B) produced 33+ chunk burst at 22:08 KST 2026-04-29 from a single repetition-trapped fire. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…ntation Phase 0.5 of the Hermes Agent token leak mitigation plan: append a single JSONL line to ~/.hermes/cron/usage_audit.jsonl after every cron LLM invocation, capturing prompt/completion/total tokens, model, duration_ms, deliver target, and error (when raised). Read from agent.session_*_tokens which run_conversation already returns in its result dict. Without this, we have no measured baseline to attribute token deltas to subsequent mitigation phases. The plan's hard gate: observability lands before any mitigation phase. Writer NEVER raises — wrapped in a single try/except that logs a warning on any json.dumps / mkdir / open failure so an audit-log bug cannot break a cron job. Failure-path audit guard via locals() check covers exceptions that fire before the fire_id is assigned. No new dependencies, no new env vars (the plan rejected one in v2). Tests: 7 new unit tests in tests/cron/test_usage_audit_logger.py covering the success path, missing token info, swallowed writer exception, parent dir creation, multiple appends, and unicode preservation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Phase 8 of the Hermes Agent token leak mitigation plan (ralplan-hermes-token-leaks.md §3.9). Adds a boolean kwarg `skip_background_review` (default False) to AIAgent.__init__ that suppresses the end-of-turn _spawn_background_review fork. Each background review fork instantiates a new AIAgent with its own ~15K input tokens + up to 8 LLM iterations, accumulating ~30K tokens per event in the worst case. On cron sessions there is no human-in-the-loop benefit from the review (no skill-creation pressure, nobody curating MEMORY.md), so the cost is pure waste. The end-of-turn guard now reads: if (final_response and not interrupted and not getattr(self, "skip_background_review", False) and (_should_review_memory or _should_review_skills)): skip_memory=True already disables the memory-review trigger; this flag is the explicit single-switch off for both review paths. Defaults to False, so behavior is unchanged for gateway/CLI callers that omit the kwarg. Tests: 5 new unit tests in tests/agent/test_skip_background_review.py covering the default value, flag persistence, the gate short-circuit, the gate fall-through, and a source-text assertion that the cron scheduler sets the flag to True (separate commit). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…-presence Phase 8 wire-in + Vector 8 doc comment from ralplan-hermes-token-leaks.md. (1) Phase 8 wire-in: cron AIAgent construction now passes skip_background_review=True. This suppresses the end-of-turn skill/memory review fork (~30K tokens/event, ≤30K typical and ≤150K worst-case daily on bluenode) which has no human-in-the-loop value for cron sessions. (2) Vector 8 doc comment: a one-line comment immediately above the AIAgent(...) construction documenting the verified-negative finding that title generation does not run on the cron path (maybe_auto_title is gateway/CLI-side only). Future contributors won't accidentally introduce title-gen here without realizing it would add ~600-1000 tokens/fire on a path that explicitly opts out of memory/review/title overhead. No new tests required for the doc comment (no behavior change). The skip_background_review wiring is covered by the existing source-text assertion in tests/agent/test_skip_background_review.py. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Documents the four items intentionally deferred from ralplan-hermes-token-leaks v3 (2026-05-01) Wave 1 to give Wave 2 contributors a starting point with explicit gates and rationale: - Phase 4.3 — tool description compaction (gated on a 50-prompt tool-selection regression suite reporting ≤2% accuracy delta). - Phase 5 — SKILL.md content cache via provider cache_control breakpoints (blocked on bluenode running gemma-on-Nous-Portal which doesn't support breakpoints; revisit on Claude migration or Nous-Portal feature parity). - Phase 6 — precheck_command schema for cron (eliminates the cron-LLM-mandatory architecture; requires upstream RFC + cross-team approval, target 2026-05-15 RFC filing). - Vector 11 — conversation history compression on cron (Wave 3 target). Each entry records the gate, owner / target where known, and the specific blocker preventing immediate landing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

0xarkstar · 2026-06-11T11:04:43Z

Rebased onto current main (2026-06-11). Notes on conflict resolution:

cron/scheduler.py: additive conflicts with the new persistent thread-pool section — both kept.
The skip_background_review guard moved with the god-file decomposition: the spawn site now lives in agent/turn_finalizer.finalize_turn, so the guard was ported there (conversation_loop.py keeps upstream's shape). Constructor wiring in run_agent.py merged cleanly.

tests/agent/test_skip_background_review.py + tests/cron/: 403 passed on the rebased branch.

Related: #16530 is now closed — upstream's plugin replatform + resolve_channel_skills covers it natively. This PR is the remaining standalone piece (cron token-leak mitigation has no upstream equivalent yet: cron/scheduler.py on main has no response-token cap and review forks still fire for cron sessions).

teknium1

Thanks for isolating the cron review-suppression and audit work. The background-review premise remains present on current main: cron constructs AIAgent without a skip flag (cron/scheduler.py:3044-3075), while finalize_turn can still call _spawn_background_review (agent/turn_finalizer.py:454-480).

Problems

HERMES_CRON_MAX_TOKENS is a user-facing max_tokens knob (cron/scheduler.py:1761-1768 in this PR), which conflicts with the standing no-user-facing-max-tokens policy.
The audit path hardcodes Path.home() / ".hermes" (cron/scheduler.py:224-225), bypassing the profile-aware _get_hermes_home() resolver used by the scheduler on current main (cron/scheduler.py:542-551).
The review-skip tests repeat the desired condition rather than exercise finalize_turn (tests/agent/test_skip_background_review.py:64-76), so they would still pass if the production guard diverged.

Suggested changes

Salvage the review-suppression and profile-safe auditing portions separately; remove the max-token-cap portion.
Resolve the audit path through _get_hermes_home() and cover a non-default profile.
Test the actual finalizer call path with _spawn_background_review patched.

Automated hermes-sweeper review.

teknium1 · 2026-07-12T20:48:04Z

+        # hallucination is bounded to e.g. ~6000 chars (3 Discord messages)
+        # instead of 33+ message bursts via cron deliver mechanism.
+        # Set HERMES_CRON_MAX_TOKENS=2048 (or similar) to enable.
+        _cron_max_tokens_env = os.getenv("HERMES_CRON_MAX_TOKENS", "").strip()


Blocking: this introduces a user-facing max_tokens knob through HERMES_CRON_MAX_TOKENS. The standing policy disallows user-facing max-token configuration, so this portion needs to be removed rather than moved to another configuration surface.

teknium1 · 2026-07-12T20:48:04Z

+# Phase 0.5 token-leak instrumentation: per-fire usage audit log.
+# Resolved lazily via Path.home() so test envs that override HOME work.
+def _usage_audit_path() -> Path:
+    return Path.home() / ".hermes" / "cron" / "usage_audit.jsonl"


Use the scheduler's existing _get_hermes_home() resolver here. Hardcoding Path.home() / '.hermes' writes audit records outside the active non-default profile; current main documents _get_hermes_home() as the profile-scoped cron resolver at cron/scheduler.py:542-551.

teknium1 · 2026-07-12T20:48:04Z

+
+    with patch.object(agent, "_spawn_background_review") as mock_spawn:
+        # This is the exact guard from run_agent.py end-of-turn block.
+        if (


This duplicates the intended predicate instead of exercising agent.turn_finalizer.finalize_turn, so it cannot detect a production-site regression. Patch _spawn_background_review and invoke the actual finalizer with eligible review conditions.

- Fix _usage_audit_path() to use _get_hermes_home() instead of hardcoded Path.home() / '.hermes' (profile-safe resolution, sweeper finding) - Rewrite skip_background_review tests to exercise finalize_turn() directly instead of duplicating the guard expression (sweeper finding) - Fix response_silent audit field to use _is_cron_silence_response() instead of the buggy SILENT_MARKER substring check it was meant to replace (simplify-code review finding) - Remove dead 'model' in locals() guard — model is always in scope before the try block (simplify-code review finding) - Extract _stub_agent_for_finalize() helper to eliminate ~40 lines of copy-pasted agent stubbing in tests (simplify-code review finding) - Clean up 'Phase 0.5' instrumentation comments

kshitijk4poor · 2026-08-07T18:34:04Z

Merged via #81254 (salvage). Your three commits cherry-picked with authorship preserved:

feat(agent): add skip_background_review flag to AIAgent constructor
feat(cron): set skip_background_review=True
feat(cron): add usage_audit.jsonl logger for cron token leak instrumentation

The HERMES_CRON_MAX_TOKENS env var was dropped (conflicts with the no-non-secret-env-vars policy — behavioral config belongs in config.yaml, not .env). docs/wave-2-deferred.md was also dropped.

Follow-up fixes applied on top:

Fixed _usage_audit_path() to use _get_hermes_home() instead of hardcoded Path.home() / ".hermes" (profile-safe)
Fixed response_silent audit field to use _is_cron_silence_response() instead of the buggy SILENT_MARKER substring check
Rewrote tests to exercise finalize_turn() directly instead of duplicating the guard expression
Removed dead "model" in locals() guard

Thanks for the contribution!

- Fix _usage_audit_path() to use _get_hermes_home() instead of hardcoded Path.home() / '.hermes' (profile-safe resolution, sweeper finding) - Rewrite skip_background_review tests to exercise finalize_turn() directly instead of duplicating the guard expression (sweeper finding) - Fix response_silent audit field to use _is_cron_silence_response() instead of the buggy SILENT_MARKER substring check it was meant to replace (simplify-code review finding) - Remove dead 'model' in locals() guard — model is always in scope before the try block (simplify-code review finding) - Extract _stub_agent_for_finalize() helper to eliminate ~40 lines of copy-pasted agent stubbing in tests (simplify-code review finding) - Clean up 'Phase 0.5' instrumentation comments

alt-glitch added type/perf Performance improvement or optimization P2 Medium — degraded but workaround exists comp/cron Cron scheduler and job management comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels May 1, 2026

0xarkstar mentioned this pull request May 1, 2026

feat(gateway,skills): unified skill trigger framework with Discord components #16530

Closed

0xarkstar force-pushed the feat/cron-token-discipline-pr-a branch 3 times, most recently from 382b6dd to 52766ea Compare May 11, 2026 19:21

0xarkstar force-pushed the feat/cron-token-discipline-pr-a branch from 52766ea to 0cf03ac Compare May 18, 2026 16:05

0xarkstar and others added 5 commits June 11, 2026 19:59

0xarkstar force-pushed the feat/cron-token-discipline-pr-a branch from 0cf03ac to bdb9fed Compare June 11, 2026 11:04

teknium1 reviewed Jul 12, 2026

View reviewed changes

teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 12, 2026

kshitijk4poor mentioned this pull request Aug 7, 2026

feat(cron): skip background review on cron + usage audit logger (salvage of #18255) #81254

Merged

kshitijk4poor closed this Aug 7, 2026

Navigation Menu

Uh oh!

feat(cron): token leak mitigation — instrumentation, max_tokens cap, background-review skip - #18255

feat(cron): token leak mitigation — instrumentation, max_tokens cap, background-review skip#18255
0xarkstar wants to merge 5 commits into
NousResearch:mainfrom
0xarkstar:feat/cron-token-discipline-pr-a

0xarkstar commented May 1, 2026 •

edited

Loading

Uh oh!

0xarkstar commented Jun 11, 2026

Uh oh!

teknium1 left a comment

Uh oh!

teknium1 Jul 12, 2026

Uh oh!

teknium1 Jul 12, 2026

Uh oh!

teknium1 Jul 12, 2026

Uh oh!

kshitijk4poor commented Aug 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

0xarkstar commented May 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary (TL;DR)

Background

Production deployment

Test plan

Complementary upstream change

Batch consideration

Salvage-friendly

Uh oh!

0xarkstar commented Jun 11, 2026

Uh oh!

teknium1 left a comment

Choose a reason for hiding this comment

Problems

Suggested changes

Uh oh!

teknium1 Jul 12, 2026

Choose a reason for hiding this comment

Uh oh!

teknium1 Jul 12, 2026

Choose a reason for hiding this comment

Uh oh!

teknium1 Jul 12, 2026

Choose a reason for hiding this comment

Uh oh!

kshitijk4poor commented Aug 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

0xarkstar commented May 1, 2026 •

edited

Loading