Skip to content

✨ feat(gateway): auto-downscale oversized outbound images before native attachment - #65

Merged
cwest merged 2 commits into
cwest/integrationfrom
topic/gateway-outbound-image-downscale
Jul 12, 2026
Merged

✨ feat(gateway): auto-downscale oversized outbound images before native attachment#65
cwest merged 2 commits into
cwest/integrationfrom
topic/gateway-outbound-image-downscale

Conversation

@cwest

@cwest cwest commented Jul 12, 2026

Copy link
Copy Markdown
Owner

Problem

Size-capped platforms silently drop an oversized native attachment: the message sends, the platform reports "Couldn't deliver the image attachment," and the file never arrives. A 4K image is routinely 15–20 MB (Nano Banana Pro at 5504×3072 ≈ 18 MB PNG); Discord's non-Nitro attachment cap is ~10 MB. Now that nano-banana defaults to 4K, every 4K render sent to Discord (and other size-capped platforms) is lost — a routine failure, not an edge case.

Mechanism

An outbound-delivery companion to the existing inbound media-size infrastructure in gateway/platforms/base.py. Before an image is sent as a native attachment, if it exceeds the target platform's outbound cap, the gateway sends a downscaled preview instead — leaving the full-resolution original untouched on disk. Generic across platforms (not Discord- or nano-banana-specific).

get_outbound_image_max_bytes(platform) — config-driven cap resolver, mirroring get_inbound_media_max_bytes:

  • gateway.max_outbound_image_bytes — global default.
  • gateway.max_outbound_image_bytes_by_platform — per-platform override map; a platform absent from the map falls back to the global default.
  • 0 disables the cap (never downscale).

prepare_outbound_image(path, *, platform, max_bytes=None) — the downscale helper:

  • Not an image / already ≤ cap / cap disabled → returns the original path unchanged.
  • Over cap → writes a downscaled JPEG preview into the image cache and returns its path. Ladder (long-edge px / JPEG quality): 2048/88 → 1568/82 → 1024/76 → 768/72, iterating until it fits or returning the best-effort smallest preview (never hard-fails delivery).
  • Preserves aspect ratio, never upscales, never mutates the source file.
  • Fail-open: on any error (missing file, PIL failure, unwritable cache) returns the original path — a downscale bug can never block a delivery that would otherwise succeed.

BasePlatformAdapter.prepare_outbound_image_paths(paths, platform) — the single shared prep call every local-path image dispatch site funnels through, avoiding per-site drift.

Wiring

Applied at the local-path image dispatch sites, keyed by source platform, funneled through the shared prep:

  • gateway/run.py — primary post-stream image batch (before building the file:// batch for send_multiple_images).
  • gateway/run.py — background-task path (send_image_file per-file send).
  • gateway/platforms/base.py — the base adapter's streaming batch path.

Remote image URLs (send_image(image_url=...)) pass through untouched — only local file paths are downscaled. The agent's message text still cites the original path; only the bytes sent to the platform change when over-cap. The full-res original stays on disk in the image cache.

Related

This is the delivery-side companion to the b64 format-sniff fix (PR #64): #64 ensures the generated image bytes are cached with the right container; this change ensures the cached image actually fits the platform's outbound attachment cap so it isn't silently dropped on send. It also complements the discord-message-formatting skill's Attachment SIZE section, which documents the ~10 MB Discord cap this mechanism enforces automatically.

Config (config.yaml, not env)

Behavioral config lives in config.yaml per the repo rubric (.env is for secrets only):

gateway:
  max_outbound_image_bytes: 10485760          # 10 MiB default (Discord non-Nitro)
  max_outbound_image_bytes_by_platform:       # optional per-platform overrides
    discord: 10485760
    telegram: 10485760
    slack: 0                                  # 0 → never downscale

Default is 10 MiB — the Discord non-Nitro limit, a safe conservative floor for any platform whose exact cap is unknown. Adding these keys to DEFAULT_CONFIG is deep-merge-safe (no _config_version bump).

Tests (TDD)

24 new tests in tests/gateway/test_outbound_image_downscale.py (real PIL on tiny synthetic images):

  • Cap resolver — default when config absent; global override; per-platform override beats global; 0 disables; unparseable/read-failure → default.
  • Downscale helper — under-cap passthrough; cap-0 passthrough; non-image passthrough; missing-file passthrough; over-cap → different preview path, bytes ≤ cap, dims ≤ original, aspect preserved; original file untouched; never-upscale; PIL-failure fail-open; resolves cap from platform when max_bytes=None.
  • Shared prep helper — maps over-cap → preview, passes under-cap through; empty/None → [].
  • Dispatch contract (the check that would have caught today's silent drop) — the path handed to the adapter's native image send is the preview, not the oversized original.
  • run.py wiring guards — every local-path dispatch site actually calls the shared prep (guards against "helper built but never wired").
  • Config defaultsDEFAULT_CONFIG.gateway carries both keys; resolver reads the shipped default end-to-end.

Test run summary

tests/gateway/test_outbound_image_downscale.py ........................  24 passed
tests/gateway/test_platform_base.py + test_outbound_image_downscale.py   199 passed, 2 skipped
image/media dispatch scope
  (test_send_multiple_images, test_send_image_file,
   test_tts_media_routing, test_signal)                                   185 passed
full tests/gateway/                                                       8673 passed, 12 failed, 35 skipped

The 12 failures in the full tests/gateway/ run are pre-existing — verified failing identically on a clean checkout of the base branch (24a741ce8) with none of this change present, or passing in isolation (order/global-state flakes). They are macOS environment artifacts (/private/var vs /var tmpdir symlink; subprocess-spawn PID timing), a missing optional XML dep (wecom callback), and Telegram markdown-escape / memory-monitor-timer flakes. None touch the outbound-image code paths, and every image/media dispatch test that exercises the wired paths passes.


Review rework — complete the wiring (all local-path image sends)

The initial change wired the run.py and base.py streaming dispatch paths, but
review found sibling local-path image send sites still un-wired — an
over-cap image shipped through those still silently dropped on size-capped
platforms. This rework funnels every remaining local-path image send through
prepare_outbound_image, so the whole bug class is fixed, not just the sites
first reported.

Sites wired in this rework

  • gateway/kanban_watchers.py_deliver_kanban_artifacts (BLOCKING):
    the completion-artifact delivery built a raw file:// batch and called
    send_multiple_images with no prep, so a 4K image shipped via
    kanban_complete(artifacts=[...]) still dropped on Discord. Now preps the
    batch through prepare_outbound_image_paths(...), keyed by the delivery
    target's platform, before building the file:// batch.
  • gateway/platforms/weixin.py — three local-path image sends (same bug
    class): the adapter send() media path (_deliver_media), and both
    send_message-tool direct-send branches (live reused adapter + fresh
    per-call adapter). All keyed by the weixin platform cap.
  • gateway/platforms/yuanbao.py — the send_message-tool direct-send
    media path, keyed by the yuanbao platform cap.

Where the weixin/yuanbao platform caps are unknown, the conservative 10 MiB
default applies (correct — better a downscaled preview than a silent drop).

Final grep sweep (send_multiple_images / send_image_file / send_image across gateway/ + gateway/platforms/)

All 16 call sites classified; no local-path image send is left un-wired:

  • Local-path sends → all funnel through prepare_outbound_image:
    run.py (batch 12626, bg 12852, telegram-topic 12971), base.py streaming
    batch (5369), kanban_watchers.py (1194 — new), weixin.py (1870, 2327,
    2375 — new), yuanbao.py (4638 — new).
  • Remote http(s) URL sends — pass-through by design (no local path):
    run.py 12818 (send_image(image_url=...) from extract_images, https-only),
    base.py 3472 / 3517 (remote URL / animation), base.py 5320
    (send_multiple_images over extract_images = https-only), bluebubbles.py
    626 / 681 (send_image(image_url) remote API).
  • Dispatcher downstream of prep: base.py 3458 — the default
    send_multiple_images implementation unquotes an already-prepped
    file:// batch; callers prep before building the batch, so no drift.

Remote sends and the dispatcher are intentionally untouched. Fail-open
preserved everywhere: prep returns the original path on any error, so a
downscale bug can never block a delivery that would otherwise succeed.

Tests added

tests/gateway/test_outbound_image_downscale.py (+5, now 29 in the file):

  • Kanban artifact contract (the exact gap review caught): over-cap
    artifact → the file:// batch handed to send_multiple_images carries the
    downscaled preview, not the oversized original; under-cap artifact →
    original passes through unchanged. Exercises the real
    _deliver_kanban_artifacts method end-to-end against a stub adapter.
  • Direct-send source guards: weixin.py, yuanbao.py, and
    kanban_watchers.py each reference the shared outbound prep (guards against
    a future un-wiring of the WS/session paths that can't be exercised without a
    live connection).

Rework test run summary

tests/gateway/test_outbound_image_downscale.py                            29 passed
tests/gateway/test_outbound_image_downscale.py + test_platform_base.py    204 passed, 2 skipped
kanban + weixin + yuanbao scope
  (test_kanban_notifier, test_kanban_watchers_mixin, test_weixin,
   test_weixin_typing, test_yuanbao_media_ssrf)                           106 passed
tests/hermes_cli/test_config.py                                           132 passed
consolidated touched scope (all of the above)                            442 passed, 2 skipped, 0 failed

ruff check clean on all changed files; commit signed (ED25519), verified G.

…ve attachment

Size-capped platforms silently drop an oversized native attachment: the
message sends, the platform reports "Couldn't deliver the image attachment,"
and the file never arrives. A 4K render (Nano Banana Pro at 5504x3072) is
routinely 15-20 MB, well past Discord's ~10 MB non-Nitro cap, so now that
nano-banana defaults to 4K every such image sent to Discord is lost.

Add an outbound image size cap + downscale-preview mechanism, mirroring the
existing inbound media-size infrastructure in gateway/platforms/base.py:

- get_outbound_image_max_bytes(platform): config-driven cap resolver reading
  gateway.max_outbound_image_bytes (global) and
  gateway.max_outbound_image_bytes_by_platform (per-platform override map).
  0 disables; default 10 MiB (Discord non-Nitro), a safe conservative floor
  for platforms whose exact cap is unknown.
- prepare_outbound_image(path, *, platform, max_bytes=None): when a local
  image exceeds the cap, write a downscaled JPEG preview into the image cache
  and return its path (long-edge/quality ladder: 2048/88 → 1568/82 →
  1024/76 → 768/72 until it fits, else best-effort smallest). Preserves
  aspect ratio, never upscales, never mutates the source. Fail-open: any
  error returns the original path so a downscale bug can't block a delivery
  that would otherwise succeed.
- BasePlatformAdapter.prepare_outbound_image_paths(): the single shared prep
  call every local-path image dispatch site funnels through, avoiding
  per-site drift.

Wire it into the local-path image dispatch sites in gateway/run.py (primary
post-stream batch + background-task send_image_file) and the base adapter's
streaming batch path. Remote image URLs pass through untouched. The
full-resolution original stays on disk; only the bytes sent to the platform
change when over-cap. Generic across platforms, not Discord- or
nano-banana-specific.

Behavioral config lives in config.yaml (not a HERMES_* env var). Delivery-side
companion to the inbound media cap and the outbound b64 format-sniff fix.

Tests: 24 new tests covering the resolver (default/global/per-platform/
disabled/fail-open), the downscale helper (under-cap passthrough, over-cap
preview under cap with aspect preserved, never-upscale, non-image/missing
passthrough, PIL-failure fail-open), the shared prep helper, the dispatch
contract (preview reaches the adapter, not the original), run.py wiring
guards, and config defaults. Full gateway/platforms/config scope green (two
pre-existing macOS env failures ruled out against the base).

@cwest cwest left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mechanism is right and the tests back it up. The resolver, the downscale helper, config-driven caps with 0-disables, fail-open on any PIL/read error, aspect-preserved never-upscale, preview into the image cache with the source left untouched, and the remote-URL passthrough all check out. I ran the outbound-image suite (24 passed), platform_base + outbound (199 passed, 2 skipped), and the image/media dispatch scope — send_multiple_images, send_image_file, tts routing, signal, stream_consumer (305 passed). The only failures in the touched scope are the two Mattermost cases that need aiohttp, and they fail identically on a clean base checkout, so nothing here is a new regression. Config defaults land without a version bump, which is correct for a deep-merge-safe key add.

The blocker is wiring coverage, not the mechanism. The claim is that every local-path image send funnels through the prep helper, but one mainstream path does not: kanban artifact delivery. gateway/kanban_watchers.py:1185 builds a file:// batch straight from image_paths and hands it to send_multiple_images with no prep call. A completed task that ships an image artifact via kanban_complete(artifacts=[...]) — a 4K render is exactly the case this change exists for — still gets silently dropped on Discord through this path. The fix mirrors the wired sites: run image_paths through BasePlatformAdapter.prepare_outbound_image_paths(image_paths, platform=adapter.platform) before building the batch at kanban_watchers.py:1180-1185.

Two secondary send paths have the same gap and are worth closing while you're here, or splitting into a follow-up: the direct-send tool paths at gateway/platforms/weixin.py:2322 and gateway/platforms/yuanbao.py:4637 dispatch media_files to send_image_file with the raw local path, no prep. Lower traffic and different platform limits, but the same class of silent drop under the global default cap.

The four sites you did wire (run.py post-stream batch, run.py background send_image_file, the telegram-topic setup image, and the base adapter streaming batch) are correct — prep sits upstream of the polymorphic send_multiple_images, so subclass overrides that rebuild the batch still carry the previews. Wire the kanban path and this is good to go.

…d downscale

The outbound downscale-preview helper only guarded the run.py and base.py
streaming dispatch paths, leaving sibling local-path image send sites
un-wired — an over-cap image shipped through those still silently dropped
on size-capped platforms (Discord's ~10MB non-Nitro cap).

Funnel every remaining local-path image send through prepare_outbound_image
so the whole bug class is fixed, not just the reported sites:

- kanban_watchers.py _deliver_kanban_artifacts: prep the file:// batch
  before send_multiple_images, so images shipped via completion artifacts
  are downscaled when over-cap (keyed by the delivery target's platform).
- weixin.py: the adapter send() media path and both send_message-tool
  direct-send branches (live + fresh adapter).
- yuanbao.py: the send_message-tool direct-send media path.

Remote http(s) URL sends and the default send_multiple_images dispatcher
(which unquotes an already-prepped file:// batch) are pass-through by
design and left untouched. Fail-open preserved: prep returns the original
path on any error, so a downscale bug can never block a delivery.

Add a behavioral regression for the kanban artifact path (asserts the
preview, not the oversized original, reaches the send when over-cap; the
under-cap original passes through unchanged) plus source guards that the
weixin/yuanbao/kanban paths reference the shared prep.

@cwest cwest left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rework closes the gap from the previous round. The kanban artifact path (_deliver_kanban_artifacts in gateway/kanban_watchers.py) now runs its file:// batch through the shared outbound prep before send_multiple_images, and the weixin and yuanbao direct-send paths are wired the same way.

I ran an independent sweep of send_multiple_images / send_image_file / send_image across gateway/ and gateway/platforms/. Every local-path image send now funnels through prepare_outbound_image: the three file:// batch builders (run.py, base.py streaming, kanban_watchers.py) and every send_image_file caller (run.py x2, weixin x3, yuanbao). The remaining hits are correctly left alone — extract_images only matches http(s) URLs, so those sends are remote pass-through, and base.py's default send_multiple_images implementation consumes an already-prepped file:// batch. Nothing local is left un-wired.

The kanban regression test carries its weight: reverting the wiring flips it red on the delivered != original assertion, so it reproduces the original silent-drop. The modules that touch the changed files are green (443 in the platform-base / kanban / weixin / yuanbao / media / config suites; 29 in the new downscale suite). The failures in the wider gateway run are the pre-existing telegram markdown-escape tests and the macOS subprocess/symlink artifacts — the failure set is identical on the current integration head, so none of them come from this change.

No changes needed.

@cwest
cwest marked this pull request as ready for review July 12, 2026 18:45
@cwest
cwest merged commit 99bd7fd into cwest/integration Jul 12, 2026
31 checks passed
@cwest
cwest deleted the topic/gateway-outbound-image-downscale branch July 12, 2026 18:46
cwest added a commit that referenced this pull request Jul 26, 2026
…ve attachment (#65)

* ✨ feat(gateway): auto-downscale oversized outbound images before native attachment

Size-capped platforms silently drop an oversized native attachment: the
message sends, the platform reports "Couldn't deliver the image attachment,"
and the file never arrives. A 4K render (Nano Banana Pro at 5504x3072) is
routinely 15-20 MB, well past Discord's ~10 MB non-Nitro cap, so now that
nano-banana defaults to 4K every such image sent to Discord is lost.

Add an outbound image size cap + downscale-preview mechanism, mirroring the
existing inbound media-size infrastructure in gateway/platforms/base.py:

- get_outbound_image_max_bytes(platform): config-driven cap resolver reading
  gateway.max_outbound_image_bytes (global) and
  gateway.max_outbound_image_bytes_by_platform (per-platform override map).
  0 disables; default 10 MiB (Discord non-Nitro), a safe conservative floor
  for platforms whose exact cap is unknown.
- prepare_outbound_image(path, *, platform, max_bytes=None): when a local
  image exceeds the cap, write a downscaled JPEG preview into the image cache
  and return its path (long-edge/quality ladder: 2048/88 → 1568/82 →
  1024/76 → 768/72 until it fits, else best-effort smallest). Preserves
  aspect ratio, never upscales, never mutates the source. Fail-open: any
  error returns the original path so a downscale bug can't block a delivery
  that would otherwise succeed.
- BasePlatformAdapter.prepare_outbound_image_paths(): the single shared prep
  call every local-path image dispatch site funnels through, avoiding
  per-site drift.

Wire it into the local-path image dispatch sites in gateway/run.py (primary
post-stream batch + background-task send_image_file) and the base adapter's
streaming batch path. Remote image URLs pass through untouched. The
full-resolution original stays on disk; only the bytes sent to the platform
change when over-cap. Generic across platforms, not Discord- or
nano-banana-specific.

Behavioral config lives in config.yaml (not a HERMES_* env var). Delivery-side
companion to the inbound media cap and the outbound b64 format-sniff fix.

Tests: 24 new tests covering the resolver (default/global/per-platform/
disabled/fail-open), the downscale helper (under-cap passthrough, over-cap
preview under cap with aspect preserved, never-upscale, non-image/missing
passthrough, PIL-failure fail-open), the shared prep helper, the dispatch
contract (preview reaches the adapter, not the original), run.py wiring
guards, and config defaults. Full gateway/platforms/config scope green (two
pre-existing macOS env failures ruled out against the base).

* 🐛 fix(gateway): wire remaining local-path image sends through outbound downscale

The outbound downscale-preview helper only guarded the run.py and base.py
streaming dispatch paths, leaving sibling local-path image send sites
un-wired — an over-cap image shipped through those still silently dropped
on size-capped platforms (Discord's ~10MB non-Nitro cap).

Funnel every remaining local-path image send through prepare_outbound_image
so the whole bug class is fixed, not just the reported sites:

- kanban_watchers.py _deliver_kanban_artifacts: prep the file:// batch
  before send_multiple_images, so images shipped via completion artifacts
  are downscaled when over-cap (keyed by the delivery target's platform).
- weixin.py: the adapter send() media path and both send_message-tool
  direct-send branches (live + fresh adapter).
- yuanbao.py: the send_message-tool direct-send media path.

Remote http(s) URL sends and the default send_multiple_images dispatcher
(which unquotes an already-prepped file:// batch) are pass-through by
design and left untouched. Fail-open preserved: prep returns the original
path on any error, so a downscale bug can never block a delivery.

Add a behavioral regression for the kanban artifact path (asserts the
preview, not the oversized original, reaches the send when over-cap; the
under-cap original passes through unchanged) plus source guards that the
weixin/yuanbao/kanban paths reference the shared prep.

(cherry picked from commit 99bd7fd)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant