feat: TOON-lite compact context + memory warn threshold by haibaoliu · Pull Request #31847 · NousResearch/hermes-agent

haibaoliu · 2026-05-25T03:53:25Z

Summary

Inspired by SochDB's TOON format — fit more useful information in the same token budget by removing structural overhead from context blocks injected into the system prompt.

This is NOT about saving tokens/money. It's about information density: every byte freed from format overhead goes to actual content the model can use.

Changes

New: `agent/toon_lite.py`

TOON encoder with three modes: toon_table(), toon_records(), toon_kv()

Skills index (`agent/prompt_builder.py`)

When context.compact_format: true, uses compact category-keyed format instead of <available_skills> XML wrapper. ~7% reduction.

Memory/profile blocks (`tools/memory_tool.py`, `agent/agent_init.py`)

When enabled, drops heavy decorative separators. ~20% reduction → room for ~1 extra memory entry per session.

Config (`hermes_cli/config.py`)

New context.compact_format flag, defaults to false.

Backward Compatibility

Fully backward compatible. No behavior change without explicit opt-in.

`memory_warn_pct` for proactive trimming

When memory usage exceeds memory.memory_warn_pct (default 80%):

⚠️ prefix added to system prompt header
warn field in memory tool responses prompts agent to consolidate
Works in both standard and TOON-lite compact mode
Set to 0 to disable

Fixes the gap where memory fills silently — agent gets proactive trim signals.

Files changed

File	Change
`agent/toon_lite.py`	New: compact TOON encoder
`hermes_cli/config.py`	`context.compact_format` + `memory.memory_warn_pct`
`tools/memory_tool.py`	`warn_pct` in `MemoryStore`, `_render_block`, `_success_response`
`agent/agent_init.py`	Pass `warn_pct` to `MemoryStore`

Testing

33/33 ✅ All default-disabled, no breaking changes

Copilot

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds an opt-in “TOON-lite” compact encoding to reduce formatting overhead in system prompt context blocks (memory/user profile and skills index).

Changes:

Introduces agent/toon_lite.py with compact formatting helpers.
Adds context.compact_format to default config and wires it into prompt-building and memory rendering.
Updates MemoryStore to optionally render compact memory/user blocks.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
tools/memory_tool.py	Adds `compact_format` option and TOON-lite rendering path for memory/user blocks.
hermes_cli/config.py	Adds `context.compact_format` default config flag.
agent/toon_lite.py	New module implementing TOON-lite compact encodings and formatting helpers.
agent/prompt_builder.py	Uses config flag to emit compact skills index format.
agent/agent_init.py	Passes config-driven `compact_format` into `MemoryStore`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+        if bool(cfg_get("context.compact_format", False)):
+            result += _toon_skills_index(skills_by_category, category_descriptions)


+                _compact = bool(cfg_get("context.compact_format", False))
                agent._memory_store = MemoryStore(


+import re
+from typing import Dict, List, Optional, Sequence, Tuple


+Design principles:
+  - LLM-readable first, compression second (this IS for LLMs to read)
+  - Header declares schema once; rows carry only data
+  - Tokenizer-friendly separators (comma, newline, colon)
+  - No structural noise: no XML tags, no JSON braces, no markdown bullets


+
+        for name, desc in sorted(set(skills), key=lambda x: x[0]):
+            clean_desc = _truncate_desc(desc.strip(), 100) if desc else ""
+            cat_lines.append(f"  - {name}: {clean_desc}")


haibaoliu · 2026-06-02T09:38:05Z

feat: TOON-lite compact context + memory warn threshold

Inspired by SochDB's TOON format — fit more useful information in the same token budget by removing structural overhead from context blocks injected into the system prompt. Changes: - agent/toon_lite.py: TOON encoder with table/records/kv modes - skills index: replace <available_skills> XML wrapper with compact category-keyed format (opt-in via config) - memory/profile: remove decorative ═══ separators, keep compact header with usage stats (opt-in via config) - config: add context.compact_format flag (default: false) Token savings: memory blocks ~20%, skills index ~7%. The real win is fitting MORE content in the same budget — every byte freed from format overhead goes to actual data. Backward compatible: disabled by default, no behavior change without explicit opt-in.

- Fix cfg_get() call signatures: pass config dict + separate keys instead of dot-path string (was always returning default=False, making compact format never activate) * agent/prompt_builder.py: use load_config() * agent/agent_init.py: use _agent_cfg already in scope - Remove unused imports (re, Optional) from agent/toon_lite.py - format_skills_index_toon(): replace markdown bullets (-) with plain indentation, consistent with the 'no structural noise' design principle - Update module docstring: remove 'no markdown bullets' claim

When memory usage exceeds the configured threshold (default 80%), show a ⚠️ warning in the system prompt header and include a 'warn' field in memory tool responses prompting the agent to trim entries. Changes: - config: add memory.memory_warn_pct (default 80, 0=disabled) - memory_tool: MemoryStore accepts warn_pct, _render_block and _success_response include warnings above threshold - toon_lite: format_memory_block_toon accepts warn_pct for compact mode - agent_init: pass warn_pct from config to MemoryStore Closes the gap where memory fills silently — agent now gets proactive signals to consolidate entries before hitting the limit.

- Add api_key_command support to ProviderConfig and resolution paths (hermes_cli/auth.py, config.py, runtime_provider.py) Allows retrieving API keys at runtime via external commands (e.g. 'op read op://vault/provider/credential'). Keys never appear in env/argv/history. - Add skill routing table (agent/skill_router.py, prompt_builder.py) Replaces full <available_skills> listing with compact intent→skill mapping when skills.routing_table=true. ~71% token savings vs full XML format (~897 vs ~3112 tokens for typical 60-skill install). - Update custom_provider normalizer to preserve api_key_command field in both _VALID_CUSTOM_PROVIDER_FIELDS and _KNOWN_KEYS.

teknium1

Thanks for the compact-context and proactive-warning proposal. The feature is not already present on current main, but this branch needs a focused salvage before it is safe to integrate.

Problems

tools/memory_tool.py:170-174 on the PR head injects raw disk entries into the frozen prompt snapshot. Current main sanitizes those entries first at tools/memory_tool.py:195-205; retain that shared strict threat-scanner path.
The PR removes current main’s external-drift guard (tools/memory_tool.py:704+) before mutation. Its _reload_target() at PR tools/memory_tool.py:220-227 can therefore lead to overwriting non-round-trippable manual edits.
Compact skill output drops <available_skills>, but hermes_cli/prompt_size.py:20-21,78-81 and agent/context_breakdown.py:15,102-108 parse only that block. Opt-in compact mode would report skills as zero and misattribute their context.
api_key_command execution in hermes_cli/auth.py:590-606 and hermes_cli/runtime_provider.py:716-744 is unrelated scope and should be reviewed separately.

Suggested changes

Reapply the compact rendering and threshold work onto current main without reverting its memory safety paths, update the two diagnostics, and add mode/threshold/safety regression tests.

Automated hermes-sweeper review.

teknium1 · 2026-07-13T12:56:52Z

        self._system_prompt_snapshot = {
-            "memory": self._render_block("memory", sanitized_memory),
-            "user": self._render_block("user", sanitized_user),
+            "memory": self._render_block("memory", self.memory_entries),


This now injects raw on-disk entries into the frozen system-prompt snapshot. Current main sanitizes every snapshot entry through tools.threat_patterns before rendering; preserve that sanitization and pass the sanitized lists to the compact renderer.

teknium1 · 2026-07-13T12:56:52Z

-        path = self._path_for(target)
-        bak = self._detect_external_drift(target)
-        fresh = self._read_file(path)
+        fresh = self._read_file(self._path_for(target))


Please retain current main's _detect_external_drift() check before accepting this reload. Without it, a following atomic write can overwrite non-round-trippable manual/external memory-file edits instead of refusing the mutation and preserving a backup.

teknium1 · 2026-07-13T12:56:52Z

+            _router_block = _rtr(skills_by_category, category_descriptions)
+            if _router_block:
+                result += _router_block
+        elif bool(cfg_get(load_config(), "context", "compact_format", default=False)):


The compact branch removes <available_skills>, but hermes_cli/prompt_size.py and agent/context_breakdown.py identify the skills section only with that tag pair. Update those consumers and add compact-mode accounting tests so diagnostics do not report zero skill usage.

teknium1 · 2026-07-13T12:56:52Z

+        import subprocess as _subprocess
+        try:
+            cmd_parts = shlex.split(pconfig.api_key_command)
+            result = _subprocess.run(


This arbitrary command-execution feature is unrelated to compact context and memory warnings. Please split api_key_command into a separately scoped change with its own security/configuration review and tests.

Copilot AI review requested due to automatic review settings May 25, 2026 03:53

Copilot AI reviewed May 25, 2026

View reviewed changes

alt-glitch mentioned this pull request May 28, 2026

feat(skills): configurable compact mode for skill index in system prompt #33934

Closed

haibaoliu changed the title ~~feat: TOON-lite compact context encoding for system prompt~~ feat: TOON-lite compact context + memory warn threshold Jun 2, 2026

MacBook added 5 commits June 3, 2026 18:20

chore: ruff format fix

e9cb397

haibaoliu force-pushed the feat/toon-lite-compact-context branch from 2eb2909 to bb450c9 Compare June 3, 2026 10:27

teknium1 reviewed Jul 13, 2026

View reviewed changes

Navigation Menu

Uh oh!

feat: TOON-lite compact context + memory warn threshold - #31847

feat: TOON-lite compact context + memory warn threshold#31847
haibaoliu wants to merge 5 commits into
NousResearch:mainfrom
haibaoliu:feat/toon-lite-compact-context

haibaoliu commented May 25, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

haibaoliu commented Jun 2, 2026

Uh oh!

teknium1 left a comment

Uh oh!

teknium1 Jul 13, 2026

Uh oh!

teknium1 Jul 13, 2026

Uh oh!

teknium1 Jul 13, 2026

Uh oh!

teknium1 Jul 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

		if bool(cfg_get("context.compact_format", False)):
		result += _toon_skills_index(skills_by_category, category_descriptions)

		_compact = bool(cfg_get("context.compact_format", False))
		agent._memory_store = MemoryStore(

		import re
		from typing import Dict, List, Optional, Sequence, Tuple

Uh oh!

Conversation

haibaoliu commented May 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

New: agent/toon_lite.py

Skills index (agent/prompt_builder.py)

Memory/profile blocks (tools/memory_tool.py, agent/agent_init.py)

Config (hermes_cli/config.py)

Backward Compatibility

memory_warn_pct for proactive trimming

Files changed

Testing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

haibaoliu commented Jun 2, 2026

Uh oh!

teknium1 left a comment

Choose a reason for hiding this comment

Problems

Suggested changes

Uh oh!

teknium1 Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

teknium1 Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

teknium1 Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

teknium1 Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

haibaoliu commented May 25, 2026 •

edited

Loading

New: `agent/toon_lite.py`

Skills index (`agent/prompt_builder.py`)

Memory/profile blocks (`tools/memory_tool.py`, `agent/agent_init.py`)

Config (`hermes_cli/config.py`)

`memory_warn_pct` for proactive trimming