fix(vision): guard image pixel dimensions, not just bytes #37677 - #37727
Closed
kyssta-exe wants to merge 1 commit into
Closed
fix(vision): guard image pixel dimensions, not just bytes #37677#37727kyssta-exe wants to merge 1 commit into
kyssta-exe wants to merge 1 commit into
Conversation
…ch#37677) Anthropic enforces two independent ceilings per image: 1. 5 MB encoded byte size 2. 8000 px longest side Hermes only guarded #1. A tall screenshot (e.g. 1200x12000 at 0.06 MB) passes every byte check but fails the pixel check, returning a non-retryable HTTP 400 that permanently bricks the conversation thread. Fixes: - error_classifier: add 'image dimensions exceed' pattern to _IMAGE_TOO_LARGE_PATTERNS so the 400 is classified as image_too_large and triggers the shrink/retry path instead of falling through to non-retryable error. - conversation_compression: check pixel dimensions (via Pillow) even when byte size is under the 4 MB target. If max(dims) > 8000, force shrink. - vision_tools._resize_image_for_vision: add optional max_dimension param. When set, images exceeding the pixel cap are downscaled even if they're under the byte budget. The resize loop now checks both byte AND pixel limits before accepting a candidate. Closes NousResearch#37677
alt-glitch
added
type/bug
Collaborator
|
This overlaps heavily with existing open PRs solving the same oversized-image-bricks-session bug (#37677 / #25837): #37701 (caps pixel dimensions at 7900px on embed + reactive recover) and #25838 (clamps dimensions to 7999px before inline base64 encode). All three add dimension patterns to |
teknium1
added a commit
that referenced
this pull request
Jun 4, 2026
… Pillow Follow-up to the salvaged #37727. That PR fixed the reactive recovery path (classifier + post-failure shrinker) but left the PROACTIVE embed-time guard in vision_tools byte-only — a tall small-byte screenshot (e.g. 1200x12000 at 0.06 MB) still baked into immutable history un-resized, relying on a failed round-trip to trigger reactive shrink. - vision_tools: add _image_exceeds_dimension() + _EMBED_MAX_DIMENSION (7900px); the embed-time cap now fires on bytes OR pixels and passes max_dimension to the resizer, so tall small-byte images are shrunk before they're embedded. - vision_tools: best-effort lazy-install of Pillow (tool.vision) in the resize ImportError fallback so the soft dep self-heals (respects allow_lazy_installs). - error_classifier: add two more Anthropic dimension-cap wording variants. - pyproject + lazy_deps: declare Pillow as the [vision] extra / tool.vision lazy dep (it was undeclared everywhere; without it ALL resize recovery no-ops). - tests: cover _image_exceeds_dimension (tall/small/edge/no-Pillow/corrupt). Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
teknium1
added a commit
that referenced
this pull request
Jun 4, 2026
… Pillow Follow-up to the salvaged #37727. That PR fixed the reactive recovery path (classifier + post-failure shrinker) but left the PROACTIVE embed-time guard in vision_tools byte-only — a tall small-byte screenshot (e.g. 1200x12000 at 0.06 MB) still baked into immutable history un-resized, relying on a failed round-trip to trigger reactive shrink. - vision_tools: add _image_exceeds_dimension() + _EMBED_MAX_DIMENSION (7900px); the embed-time cap now fires on bytes OR pixels and passes max_dimension to the resizer, so tall small-byte images are shrunk before they're embedded. - vision_tools: best-effort lazy-install of Pillow (tool.vision) in the resize ImportError fallback so the soft dep self-heals (respects allow_lazy_installs). - error_classifier: add two more Anthropic dimension-cap wording variants. - pyproject + lazy_deps: declare Pillow as the [vision] extra / tool.vision lazy dep (it was undeclared everywhere; without it ALL resize recovery no-ops). - tests: cover _image_exceeds_dimension (tall/small/edge/no-Pillow/corrupt). Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
Contributor
|
Merged via #39033. Your commit was cherry-picked onto current main with your authorship preserved in git log (6bdbe30). Thanks for the clean diagnosis and fix. The salvage adds the one piece this PR didn't cover: the proactive embed-time guard in |
waym0reom3ga
pushed a commit
to waym0reom3ga/autolycus-agent
that referenced
this pull request
Jun 4, 2026
… Pillow Follow-up to the salvaged NousResearch#37727. That PR fixed the reactive recovery path (classifier + post-failure shrinker) but left the PROACTIVE embed-time guard in vision_tools byte-only — a tall small-byte screenshot (e.g. 1200x12000 at 0.06 MB) still baked into immutable history un-resized, relying on a failed round-trip to trigger reactive shrink. - vision_tools: add _image_exceeds_dimension() + _EMBED_MAX_DIMENSION (7900px); the embed-time cap now fires on bytes OR pixels and passes max_dimension to the resizer, so tall small-byte images are shrunk before they're embedded. - vision_tools: best-effort lazy-install of Pillow (tool.vision) in the resize ImportError fallback so the soft dep self-heals (respects allow_lazy_installs). - error_classifier: add two more Anthropic dimension-cap wording variants. - pyproject + lazy_deps: declare Pillow as the [vision] extra / tool.vision lazy dep (it was undeclared everywhere; without it ALL resize recovery no-ops). - tests: cover _image_exceeds_dimension (tall/small/edge/no-Pillow/corrupt). Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
Yuki-14544869
pushed a commit
to Yuki-14544869/hermes-agent
that referenced
this pull request
Jun 4, 2026
… Pillow Follow-up to the salvaged NousResearch#37727. That PR fixed the reactive recovery path (classifier + post-failure shrinker) but left the PROACTIVE embed-time guard in vision_tools byte-only — a tall small-byte screenshot (e.g. 1200x12000 at 0.06 MB) still baked into immutable history un-resized, relying on a failed round-trip to trigger reactive shrink. - vision_tools: add _image_exceeds_dimension() + _EMBED_MAX_DIMENSION (7900px); the embed-time cap now fires on bytes OR pixels and passes max_dimension to the resizer, so tall small-byte images are shrunk before they're embedded. - vision_tools: best-effort lazy-install of Pillow (tool.vision) in the resize ImportError fallback so the soft dep self-heals (respects allow_lazy_installs). - error_classifier: add two more Anthropic dimension-cap wording variants. - pyproject + lazy_deps: declare Pillow as the [vision] extra / tool.vision lazy dep (it was undeclared everywhere; without it ALL resize recovery no-ops). - tests: cover _image_exceeds_dimension (tall/small/edge/no-Pillow/corrupt). Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
davidgut1982
pushed a commit
to davidgut1982/hermes-agent
that referenced
this pull request
Jun 5, 2026
… Pillow Follow-up to the salvaged NousResearch#37727. That PR fixed the reactive recovery path (classifier + post-failure shrinker) but left the PROACTIVE embed-time guard in vision_tools byte-only — a tall small-byte screenshot (e.g. 1200x12000 at 0.06 MB) still baked into immutable history un-resized, relying on a failed round-trip to trigger reactive shrink. - vision_tools: add _image_exceeds_dimension() + _EMBED_MAX_DIMENSION (7900px); the embed-time cap now fires on bytes OR pixels and passes max_dimension to the resizer, so tall small-byte images are shrunk before they're embedded. - vision_tools: best-effort lazy-install of Pillow (tool.vision) in the resize ImportError fallback so the soft dep self-heals (respects allow_lazy_installs). - error_classifier: add two more Anthropic dimension-cap wording variants. - pyproject + lazy_deps: declare Pillow as the [vision] extra / tool.vision lazy dep (it was undeclared everywhere; without it ALL resize recovery no-ops). - tests: cover _image_exceeds_dimension (tall/small/edge/no-Pillow/corrupt). Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
changman
pushed a commit
to changman/hermes-agent
that referenced
this pull request
Jun 10, 2026
… Pillow Follow-up to the salvaged NousResearch#37727. That PR fixed the reactive recovery path (classifier + post-failure shrinker) but left the PROACTIVE embed-time guard in vision_tools byte-only — a tall small-byte screenshot (e.g. 1200x12000 at 0.06 MB) still baked into immutable history un-resized, relying on a failed round-trip to trigger reactive shrink. - vision_tools: add _image_exceeds_dimension() + _EMBED_MAX_DIMENSION (7900px); the embed-time cap now fires on bytes OR pixels and passes max_dimension to the resizer, so tall small-byte images are shrunk before they're embedded. - vision_tools: best-effort lazy-install of Pillow (tool.vision) in the resize ImportError fallback so the soft dep self-heals (respects allow_lazy_installs). - error_classifier: add two more Anthropic dimension-cap wording variants. - pyproject + lazy_deps: declare Pillow as the [vision] extra / tool.vision lazy dep (it was undeclared everywhere; without it ALL resize recovery no-ops). - tests: cover _image_exceeds_dimension (tall/small/edge/no-Pillow/corrupt). Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
alt-glitch
pushed a commit
that referenced
this pull request
Jun 14, 2026
… Pillow Follow-up to the salvaged #37727. That PR fixed the reactive recovery path (classifier + post-failure shrinker) but left the PROACTIVE embed-time guard in vision_tools byte-only — a tall small-byte screenshot (e.g. 1200x12000 at 0.06 MB) still baked into immutable history un-resized, relying on a failed round-trip to trigger reactive shrink. - vision_tools: add _image_exceeds_dimension() + _EMBED_MAX_DIMENSION (7900px); the embed-time cap now fires on bytes OR pixels and passes max_dimension to the resizer, so tall small-byte images are shrunk before they're embedded. - vision_tools: best-effort lazy-install of Pillow (tool.vision) in the resize ImportError fallback so the soft dep self-heals (respects allow_lazy_installs). - error_classifier: add two more Anthropic dimension-cap wording variants. - pyproject + lazy_deps: declare Pillow as the [vision] extra / tool.vision lazy dep (it was undeclared everywhere; without it ALL resize recovery no-ops). - tests: cover _image_exceeds_dimension (tall/small/edge/no-Pillow/corrupt). Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
kossteg
pushed a commit
to kossteg/hermes-agent
that referenced
this pull request
Jun 16, 2026
… Pillow Follow-up to the salvaged NousResearch#37727. That PR fixed the reactive recovery path (classifier + post-failure shrinker) but left the PROACTIVE embed-time guard in vision_tools byte-only — a tall small-byte screenshot (e.g. 1200x12000 at 0.06 MB) still baked into immutable history un-resized, relying on a failed round-trip to trigger reactive shrink. - vision_tools: add _image_exceeds_dimension() + _EMBED_MAX_DIMENSION (7900px); the embed-time cap now fires on bytes OR pixels and passes max_dimension to the resizer, so tall small-byte images are shrunk before they're embedded. - vision_tools: best-effort lazy-install of Pillow (tool.vision) in the resize ImportError fallback so the soft dep self-heals (respects allow_lazy_installs). - error_classifier: add two more Anthropic dimension-cap wording variants. - pyproject + lazy_deps: declare Pillow as the [vision] extra / tool.vision lazy dep (it was undeclared everywhere; without it ALL resize recovery no-ops). - tests: cover _image_exceeds_dimension (tall/small/edge/no-Pillow/corrupt). Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
T02200059
pushed a commit
to T02200059/hermes-agent
that referenced
this pull request
Jun 18, 2026
… Pillow Follow-up to the salvaged NousResearch#37727. That PR fixed the reactive recovery path (classifier + post-failure shrinker) but left the PROACTIVE embed-time guard in vision_tools byte-only — a tall small-byte screenshot (e.g. 1200x12000 at 0.06 MB) still baked into immutable history un-resized, relying on a failed round-trip to trigger reactive shrink. - vision_tools: add _image_exceeds_dimension() + _EMBED_MAX_DIMENSION (7900px); the embed-time cap now fires on bytes OR pixels and passes max_dimension to the resizer, so tall small-byte images are shrunk before they're embedded. - vision_tools: best-effort lazy-install of Pillow (tool.vision) in the resize ImportError fallback so the soft dep self-heals (respects allow_lazy_installs). - error_classifier: add two more Anthropic dimension-cap wording variants. - pyproject + lazy_deps: declare Pillow as the [vision] extra / tool.vision lazy dep (it was undeclared everywhere; without it ALL resize recovery no-ops). - tests: cover _image_exceeds_dimension (tall/small/edge/no-Pillow/corrupt). Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
waefrebeorn
pushed a commit
to waefrebeorn/slermes
that referenced
this pull request
Jul 2, 2026
… Pillow Follow-up to the salvaged NousResearch#37727. That PR fixed the reactive recovery path (classifier + post-failure shrinker) but left the PROACTIVE embed-time guard in vision_tools byte-only — a tall small-byte screenshot (e.g. 1200x12000 at 0.06 MB) still baked into immutable history un-resized, relying on a failed round-trip to trigger reactive shrink. - vision_tools: add _image_exceeds_dimension() + _EMBED_MAX_DIMENSION (7900px); the embed-time cap now fires on bytes OR pixels and passes max_dimension to the resizer, so tall small-byte images are shrunk before they're embedded. - vision_tools: best-effort lazy-install of Pillow (tool.vision) in the resize ImportError fallback so the soft dep self-heals (respects allow_lazy_installs). - error_classifier: add two more Anthropic dimension-cap wording variants. - pyproject + lazy_deps: declare Pillow as the [vision] extra / tool.vision lazy dep (it was undeclared everywhere; without it ALL resize recovery no-ops). - tests: cover _image_exceeds_dimension (tall/small/edge/no-Pillow/corrupt). Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
santhreal
pushed a commit
to santhreal/hermes-agent
that referenced
this pull request
Jul 13, 2026
… Pillow Follow-up to the salvaged NousResearch#37727. That PR fixed the reactive recovery path (classifier + post-failure shrinker) but left the PROACTIVE embed-time guard in vision_tools byte-only — a tall small-byte screenshot (e.g. 1200x12000 at 0.06 MB) still baked into immutable history un-resized, relying on a failed round-trip to trigger reactive shrink. - vision_tools: add _image_exceeds_dimension() + _EMBED_MAX_DIMENSION (7900px); the embed-time cap now fires on bytes OR pixels and passes max_dimension to the resizer, so tall small-byte images are shrunk before they're embedded. - vision_tools: best-effort lazy-install of Pillow (tool.vision) in the resize ImportError fallback so the soft dep self-heals (respects allow_lazy_installs). - error_classifier: add two more Anthropic dimension-cap wording variants. - pyproject + lazy_deps: declare Pillow as the [vision] extra / tool.vision lazy dep (it was undeclared everywhere; without it ALL resize recovery no-ops). - tests: cover _image_exceeds_dimension (tall/small/edge/no-Pillow/corrupt). Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
donbowman
pushed a commit
to donbowman/hermes-agent
that referenced
this pull request
Jul 13, 2026
… Pillow Follow-up to the salvaged NousResearch#37727. That PR fixed the reactive recovery path (classifier + post-failure shrinker) but left the PROACTIVE embed-time guard in vision_tools byte-only — a tall small-byte screenshot (e.g. 1200x12000 at 0.06 MB) still baked into immutable history un-resized, relying on a failed round-trip to trigger reactive shrink. - vision_tools: add _image_exceeds_dimension() + _EMBED_MAX_DIMENSION (7900px); the embed-time cap now fires on bytes OR pixels and passes max_dimension to the resizer, so tall small-byte images are shrunk before they're embedded. - vision_tools: best-effort lazy-install of Pillow (tool.vision) in the resize ImportError fallback so the soft dep self-heals (respects allow_lazy_installs). - error_classifier: add two more Anthropic dimension-cap wording variants. - pyproject + lazy_deps: declare Pillow as the [vision] extra / tool.vision lazy dep (it was undeclared everywhere; without it ALL resize recovery no-ops). - tests: cover _image_exceeds_dimension (tall/small/edge/no-Pillow/corrupt). Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
Gravezzz
pushed a commit
to Gravezzz/hermes-agent
that referenced
this pull request
Jul 21, 2026
… Pillow Follow-up to the salvaged NousResearch#37727. That PR fixed the reactive recovery path (classifier + post-failure shrinker) but left the PROACTIVE embed-time guard in vision_tools byte-only — a tall small-byte screenshot (e.g. 1200x12000 at 0.06 MB) still baked into immutable history un-resized, relying on a failed round-trip to trigger reactive shrink. - vision_tools: add _image_exceeds_dimension() + _EMBED_MAX_DIMENSION (7900px); the embed-time cap now fires on bytes OR pixels and passes max_dimension to the resizer, so tall small-byte images are shrunk before they're embedded. - vision_tools: best-effort lazy-install of Pillow (tool.vision) in the resize ImportError fallback so the soft dep self-heals (respects allow_lazy_installs). - error_classifier: add two more Anthropic dimension-cap wording variants. - pyproject + lazy_deps: declare Pillow as the [vision] extra / tool.vision lazy dep (it was undeclared everywhere; without it ALL resize recovery no-ops). - tests: cover _image_exceeds_dimension (tall/small/edge/no-Pillow/corrupt). Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
waym0reom3ga
pushed a commit
to waym0reom3ga/autolycus-agent
that referenced
this pull request
Jul 23, 2026
… Pillow Follow-up to the salvaged NousResearch#37727. That PR fixed the reactive recovery path (classifier + post-failure shrinker) but left the PROACTIVE embed-time guard in vision_tools byte-only — a tall small-byte screenshot (e.g. 1200x12000 at 0.06 MB) still baked into immutable history un-resized, relying on a failed round-trip to trigger reactive shrink. - vision_tools: add _image_exceeds_dimension() + _EMBED_MAX_DIMENSION (7900px); the embed-time cap now fires on bytes OR pixels and passes max_dimension to the resizer, so tall small-byte images are shrunk before they're embedded. - vision_tools: best-effort lazy-install of Pillow (tool.vision) in the resize ImportError fallback so the soft dep self-heals (respects allow_lazy_installs). - error_classifier: add two more Anthropic dimension-cap wording variants. - pyproject + lazy_deps: declare Pillow as the [vision] extra / tool.vision lazy dep (it was undeclared everywhere; without it ALL resize recovery no-ops). - tests: cover _image_exceeds_dimension (tall/small/edge/no-Pillow/corrupt). Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
leewenjie
pushed a commit
to leewenjie/hermes-agent
that referenced
this pull request
Aug 7, 2026
… Pillow Follow-up to the salvaged NousResearch#37727. That PR fixed the reactive recovery path (classifier + post-failure shrinker) but left the PROACTIVE embed-time guard in vision_tools byte-only — a tall small-byte screenshot (e.g. 1200x12000 at 0.06 MB) still baked into immutable history un-resized, relying on a failed round-trip to trigger reactive shrink. - vision_tools: add _image_exceeds_dimension() + _EMBED_MAX_DIMENSION (7900px); the embed-time cap now fires on bytes OR pixels and passes max_dimension to the resizer, so tall small-byte images are shrunk before they're embedded. - vision_tools: best-effort lazy-install of Pillow (tool.vision) in the resize ImportError fallback so the soft dep self-heals (respects allow_lazy_installs). - error_classifier: add two more Anthropic dimension-cap wording variants. - pyproject + lazy_deps: declare Pillow as the [vision] extra / tool.vision lazy dep (it was undeclared everywhere; without it ALL resize recovery no-ops). - tests: cover _image_exceeds_dimension (tall/small/edge/no-Pillow/corrupt). Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Anthropic enforces two independent ceilings per image:
"image exceeds 5 MB maximum""image dimensions exceed max allowed size: 8000 pixels"Hermes only guarded #1. A tall screenshot (e.g. 1200×12000 at 0.06 MB) passes every byte check but fails the pixel check, returning a non-retryable HTTP 400 that permanently bricks the conversation thread — every subsequent turn replays the oversized image and gets rejected.
Changes
1.
agent/error_classifier.pyAdded
"image dimensions exceed"to_IMAGE_TOO_LARGE_PATTERNSso the 400 is classified asimage_too_largeand triggers the shrink/retry path instead of falling through to a non-retryable error.2.
agent/conversation_compression.py—try_shrink_image_parts_in_messagesPreviously skipped shrink if
len(url) <= target_bytes(4 MB). Now also checks pixel dimensions via Pillow: ifmax(dims) > 8000, forces shrink even when bytes are under the budget. Passesmax_dimension=8000to the resize function.3.
tools/vision_tools.py—_resize_image_for_visionAdded optional
max_dimensionparameter. When set, images exceeding the pixel cap are downscaled even if they're under the byte budget. The resize loop now checks both byte AND pixel limits before accepting a candidate.Verification
Closes #37677