Skip to content

fix(agent): reserve output tokens in compression threshold - #45351

Closed
lEWFkRAD wants to merge 1 commit into
NousResearch:mainfrom
lEWFkRAD:fix/output-cap-aware-compression-threshold
Closed

fix(agent): reserve output tokens in compression threshold#45351
lEWFkRAD wants to merge 1 commit into
NousResearch:mainfrom
lEWFkRAD:fix/output-cap-aware-compression-threshold

Conversation

@lEWFkRAD

Copy link
Copy Markdown
Contributor

What does this PR do?

Makes proactive context compression reserve the response-token budget that the outgoing request will send. When a provider/profile default like provider: custom reserves max_tokens=65536, the compressor now compares prompt estimates against (context_length - output_reservation) * compression.threshold instead of the raw context window.

Fixes #43547

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Test-only change
  • Refactor

Changes Made

  • agent/context_compressor.py derives threshold/tail budgets from an output-reservation-aware input budget when a response cap is known.
  • run_agent.py resolves the same output cap precedence used by request construction: request overrides, ephemeral retry caps, user model.max_tokens, Anthropic defaults, provider profile defaults, and Claude aggregator defaults.
  • agent/agent_init.py, agent/turn_context.py, agent/agent_runtime_helpers.py, and agent/chat_completion_helpers.py sync the reservation at init, preflight, model switch, and fallback activation.
  • tests/test_ctx_halving_fix.py adds regression coverage for the custom provider 65536 default against a 131072 context window.

How to Test

  1. scripts/run_tests.sh tests/test_ctx_halving_fix.py
  2. scripts/run_tests.sh tests/agent/test_context_compressor.py
  3. scripts/run_tests.sh tests/run_agent/test_switch_model_context.py tests/run_agent/test_compressor_fallback_update.py
  4. ruff check agent/context_compressor.py run_agent.py agent/agent_init.py agent/turn_context.py agent/agent_runtime_helpers.py agent/chat_completion_helpers.py tests/test_ctx_halving_fix.py
  5. git diff --check

Checklist

  • I have read the Contributing Guide
  • My changes follow the project's conventional commit format
  • I have searched existing issues/PRs for duplicates
  • This PR is focused and minimal
  • I have run tests with scripts/run_tests.sh
  • I have added or updated tests for behavior changes
  • I have tested on the relevant platform (WSL/Linux runner on Windows host)

Documentation / Housekeeping

  • N/A - no documentation changes needed

For New Skills

  • N/A - no skill changes

Screenshots / Logs

N/A

@liuhao1024

Copy link
Copy Markdown
Contributor

Verified — clean.

Reviewed the full diff across context_compressor.py, run_agent.py, agent_init.py, agent_runtime_helpers.py, chat_completion_helpers.py, turn_context.py, and tests.

What was checked:

  • Output reservation correctly subtracted from context_length before threshold percentage: input_budget = max(context_length - output_reservation, 1)threshold = max(input_budget * threshold_percent, 1)
  • When output_reservation_tokens = 0, behavior reverts to original context_length * threshold_percent path (no minimum floor applied — this is intentional for the reservation-aware path)
  • _resolved_output_reservation_tokens() cascades correctly: override_cap → ephemeral_cap → user_cap → anthropic/provider profile → fallback 0
  • Anthropic cap guard: if cap >= context_length: cap = max(context_length - 1, 1) prevents reserving the entire window
  • All 4 call sites (agent_init, switch_model, try_activate_fallback, turn_context) call _sync_context_compressor_output_reservation after compressor creation/update
  • _coerce_output_reservation_tokens handles bool, overflow, None, negative values
  • Tests validate: reservation reduces threshold correctly, custom provider default caps propagate to compressor

Why this matters: Without output reservation, proactive compression triggers at 50% of context_length (e.g. 64K of 128K). If the model reserves 32K for output, the effective input budget is only 96K — compression should trigger at 48K, not 64K. This fix prevents context overflow on models with large output caps.

Good change. No issues found.

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Approved

Clean, well-scoped fix/feature with comprehensive tests. No issues found.

  • Logic is correct and focused
  • Tests cover the new behavior
  • No security concerns
  • Good error handling

Reviewed by Hermes Agent

@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists duplicate This issue or pull request already exists labels Jun 13, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Duplicate of #43651 — same fix (reserve the output-token budget in the proactive compaction threshold, comparing against (context_length - output_reservation) * threshold) for #43547. #43651 is the earlier-open canonical.

@lEWFkRAD

Copy link
Copy Markdown
Contributor Author

Flagged as duplicate of #43651 by @alt-glitch. Closing this one in favor of the earlier PR.

@lEWFkRAD

Copy link
Copy Markdown
Contributor Author

Closing in favor of #43651 which contains the same fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Context compaction trigger ignores the output-token reservation (custom-provider default max_tokens=65536 halves the usable input budget)

5 participants