Skip to content

fix(context): warn when a context file is truncated instead of dropping it silently - #72449

Open
BKStock wants to merge 2 commits into
NousResearch:mainfrom
BKStock:fix/context-truncation-warning
Open

BKStock wants to merge 2 commits into
NousResearch:mainfrom
BKStock:fix/context-truncation-warning

Conversation

@BKStock

@BKStock BKStock commented Jul 27, 2026

Copy link
Copy Markdown

Stacked on #72448 (shares tests/agent/test_subdirectory_hints.py). Review that one first; this diff is the two logger.warning calls plus their tests.

Problem

Both context-loading paths shed content without telling anyone who can act on it.

_truncate_content() inserts an in-band marker:

[...truncated AGENTS.md: kept 14000+4000 of 52488 chars. Use file tools to read the full file.]

That string is addressed to the model. The person who wrote the file gets nothing — no log line, no startup notice, no non-zero exit. A repo whose AGENTS.md outgrows CONTEXT_FILE_MAX_CHARS keeps booting cleanly while the middle of the file stops reaching the model.

This repo's own AGENTS.md is the case in point. At 52,488 chars against a 20,000 cap with a 70/20 head-tail split, 66% was dropped on every single session. The discarded middle was not filler — Plugins, Skills, Toolsets, Delegation, Curator, Cron, Kanban, Important Policies, Profiles, and Known Pitfalls all sat inside it. The prompt-caching rules and the entire pitfalls list were never actually loaded, and nothing anywhere said so.

The subdirectory-hint loader has the same shape at its own 8,000-char limit.

Change

Both truncation points log a WARNING naming the file, its size, the limit, how much was dropped, and what to do about it:

WARNING agent.prompt_builder: Context file AGENTS.md is 52488 chars (limit 20000)
— dropping 34488 chars (66%) from the middle. Split the overflow into separate
docs and reference them, or the omitted sections will never reach the model.

Files under the limit are returned untouched and stay silent, so a warning firing means content was genuinely lost — no ambient noise to tune out.

Tests

4 added. Each path: warns when oversized, silent when within budget. 159 passed, 1 skipped.

Run directly with python -m pytest; the wrapper's venv probe is fixed separately in #72447.

BK Bot added 2 commits July 27, 2026 11:42
…ckup/vendor dirs

SubdirectoryHintTracker re-injected identical context files whenever the same
AGENTS.md was reachable through more than one path. Symlinked shared
workspaces, hardlinks, and timestamped backup copies all alias a single file,
so a normal session could ship the same 8KB of instructions two or three
times. Nothing deduped it and nothing excluded directories that only ever
hold copies.

Two changes:

* Track a sha256 of every injected hint body. Repeat content is skipped, and
  the working directory's own context file is seeded at construction so the
  copy prompt_builder already loaded at startup is never sent again.
* Skip directories that hold copies rather than authoritative context
  (backups, node_modules, venv, site-packages, .git, .Trash, vendor, caches).
  Screening is relative to working_dir, so a project that legitimately lives
  under vendor/ keeps discovering its own subdirectory hints.

Measured on a real session that touched a symlinked shared workspace:
3 injections / ~24,000 chars before, 1 injection / 8,112 chars after.

14 new tests cover symlink aliasing, byte-identical copies, working-dir
seeding, distinct content still being injected, each excluded directory name,
excluded ancestors, and the working-dir-inside-excluded-name case.
…ng it silently

Both context-loading paths shed content without telling anyone. The in-band
'[...truncated]' marker is addressed to the model, not the author, so a repo
whose AGENTS.md outgrows CONTEXT_FILE_MAX_CHARS keeps booting cleanly while
the middle of the file stops reaching the model. This repo's own AGENTS.md sat
at 52,488 chars and lost 66% of itself on every session — the Plugins, Skills,
Delegation, Policies, and Known Pitfalls sections were never loaded, and
nothing anywhere said so.

_truncate_content() and the subdirectory-hint loader now log a WARNING naming
the file, its size, the limit, how much was dropped, and what to do about it.
Files under the limit are returned untouched and stay silent, so the warning
only fires when content is genuinely lost.

4 tests: each path warns when oversized and stays quiet when within budget.
@teknium1

Copy link
Copy Markdown
Collaborator

Thanks for identifying the silent subdirectory-context truncation path. The remaining subdirectory warning is still useful on current main.

Problems

  • The startup-context half is already superseded: agent/prompt_builder.py:1965-1971 logs and records truncation warnings, and agent/system_prompt.py:568-571 emits them through the user-visible status channel. The PR has diverged substantially, so that hunk should not be reapplied over the current dynamic-cap implementation.

Suggested changes

  • Salvage only the missing warning around agent/subdirectory_hints.py:235-239.
  • Make the new subdirectory test assert the warning’s file/path and limit fields as well as its presence.

Automated hermes-sweeper review.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels Jul 30, 2026
@teknium1 teknium1 added the sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit label Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants