feat(api-server): inline MEDIA: image tags as data URLs for remote frontends (#2696 salvage) - #56959
Merged
Merged
Conversation
alt-glitch
added
type/feature
…mote frontends Salvage of the surviving piece of #2696 by @tarunravi. The PR's other two changes (tool progress streaming, SSE None-sentinel fix) were independently superseded on main by the structured hermes.tool.progress SSE events and the rewritten queue-drain loop. Remote OpenAI-compatible frontends can't read server-local file paths, so MEDIA:<path> tags (browser screenshots, generated images) were dead text. _resolve_media_to_data_urls() now inlines small (<=5MB) local images as markdown data URLs across all four response surfaces: chat completions (non-streaming), session chat, session chat stream final event, and the Responses API. Non-image, missing, or oversized paths pass through untouched.
teknium1
force-pushed
the
salvage/2696-api-server-media
branch
from
July 2, 2026 10:14
8d3d218 to
484bc6b
Compare
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
API-server responses now inline
MEDIA:<path>image tags as base64 data URLs, so remote OpenAI-compatible frontends can actually render server-local screenshots and generated images instead of seeing dead file paths.Salvage of the surviving piece of #2696 by @tarunravi — the PR's other two changes (tool progress streaming, SSE None-sentinel fix) were independently superseded on main by the structured
hermes.tool.progressSSE events and the rewritten queue-drain loop.Changes
gateway/platforms/api_server.py:_resolve_media_to_data_urls()— inlines small (≤5MB) local images (png/jpg/jpeg/gif/webp/bmp) as markdown data URLs; non-image, missing, or oversized paths pass through untouched. Wired into all four response surfaces: chat completions (non-streaming), session chat, session chat stream final event, Responses APItests/gateway/test_api_server_media_data_urls.py: 7 tests (inline, backtick-wrapped, missing file, non-image, oversized, multi-tag, passthrough)scripts/release.py: AUTHOR_MAP entry for @tarunraviValidation
Infographic
Nous Research