fix(agent): stop old user images from bloating every model request - #87555
Open
fangliquanflq wants to merge 1 commit into
Open
fix(agent): stop old user images from bloating every model request#87555fangliquanflq wants to merge 1 commit into
fangliquanflq wants to merge 1 commit into
Conversation
Contributor
fix(agent): stop old user images from bloating every model request
No blocking issues; (1) is the one to verify against the actual |
11 tasks
19 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Historical user-uploaded images can remain in every provider request indefinitely when a session stays below the compression threshold. A reported three-image turn added about 2.9 MB to every request, producing about 58 MB of redundant daily upload at 20 calls per day. This PR bounds older user-image payloads at request time while preserving the current user turn and the original persisted session history.
Symptom
After a user uploads images and continues the conversation, the original base64 image data is sent again with every later model request. Large-context models may never trigger compression, so the repeated payload has no practical eviction point.
Impact
Users with image-bearing sessions can incur repeated multi-megabyte request bodies, additional upload latency, and unnecessary provider traffic on every later turn. The reported production example retained 4,047,677 base64 characters from one three-image message for 19 days.
Bug Cause
Trigger:
agent/conversation_loop.py:2281/ outbound request assembly with historical user image partsCausal chain:
Why it is wrong: Historical user images had no bounded request-time lifecycle independent of compression. The compressor's historical-media pass only runs when compression is triggered and deliberately preserves the newest image-bearing user turn as its anchor.
Working sibling / contrast: Non-vision models strip images for compatibility, but vision-capable models retain them. Tool-result screenshot pruning is a separate role-specific path and does not cap user uploads.
Ruled out: Token-threshold tuning does not solve the request payload lifecycle because large-window sessions can remain below the compression threshold indefinitely, and the newest image-bearing anchor remains preserved when compression does run.
Fix
Add a request-only newest-first cap for image parts in older user turns. The newest user turn is always preserved in full, evicted parts become stable text placeholders, and copy-on-write handling keeps persisted history unchanged. Apply the same policy to the main conversation loop and the max-iterations summary request before prompt-cache planning. Expose
agent.max_history_user_imageswith a default of 3 and allow0to preserve the previous behavior.Related Issue
Closes #87513
Type of Change
Changes Made
agent/context_compressor.py- add copy-on-write historical user-image eviction with provider-compatible placeholders.agent/conversation_loop.py- apply the cap before prompt-cache planning and provider dispatch.agent/chat_completion_helpers.py- apply the same cap to max-iterations summary requests.agent/agent_init.pyandhermes_cli/config_defaults.py- load and default the new agent setting.cli-config.yaml.exampleandwebsite/docs/user-guide/configuration.md- document behavior and disable semantics.tests/agent/test_history_user_image_eviction.pyandtests/run_agent/test_history_user_image_eviction.py- cover retention order, current-turn preservation, provider shapes, request-path parity, and history immutability.How to Test
agent.max_history_user_images: 1.Result: 8 tests passed. The broader related suite also passed 117 tests across 7 files on the reviewed commit.
Checklist
Code
fix(scope):,feat(scope):, etc.)Documentation & Housekeeping
docs/, docstrings)cli-config.yaml.examplebecause this PR adds a config keyScreenshots / Logs