fix(vision,gateway): wall-clock timeout for vision calls + 502/503/504 error hint - #37440
Open
daniel-rudaev wants to merge 3 commits into
Open
fix(vision,gateway): wall-clock timeout for vision calls + 502/503/504 error hint#37440daniel-rudaev wants to merge 3 commits into
daniel-rudaev wants to merge 3 commits into
Conversation
vision_analyze_tool awaited async_call_llm with only the per-attempt SDK/transport timeout. When a provider degrades (a 503 storm, or a stalled socket the transport read-timeout never trips) the await can hang far past the configured timeout. Vision runs inside the agent turn, so a hung call blocks the entire conversation and the user gets no response at all. Wrap the call in asyncio.wait_for with a wall-clock bound (auxiliary.vision.hard_timeout, default = per-attempt timeout + 60s) and return a clear 'provider taking too long, try again' message on timeout, instead of hanging the turn silently.
The per-status hint ladder covered 401/402/429/529/400/500 but not the common 502/503/504 'provider temporarily unavailable' class, so a transient provider 5xx surfaced as a bare 'Sorry, I encountered an error' with no actionable guidance. Add a 5xx branch telling the user it's transient and to retry.
teknium1
reviewed
Jul 13, 2026
teknium1
left a comment
Contributor
There was a problem hiding this comment.
Thanks for addressing two live resilience gaps. Current main still directly awaits async_call_llm in tools/vision_tools.py:1253, and the gateway hint ladder in gateway/run.py:12258-12296 still omits 502/503/504.
Problems
- The new
auxiliary.vision.hard_timeoutsurface is only read by the PR. It is absent from the defaults inhermes_cli/config.py:1577-1585and the documented auxiliary reference atwebsite/docs/user-guide/configuration.md:998-1009. - The diff adds no regression coverage. Existing vision config coverage in
tests/tools/test_vision_tools.py:424-481coverstimeoutandtemperature, not a stalled coroutine; no gateway test covers this status-hint branch.
Suggested changes
- Register and document
hard_timeoutalongside the existing vision timeout settings. - Add hermetic tests for a never-completing vision call and for 502/503/504 gateway hints.
This is an automated hermes-sweeper review.
| # Local vision models (llama.cpp, ollama) can take well over 30s. | ||
| vision_timeout = 120.0 | ||
| vision_temperature = 0.1 | ||
| vision_hard_timeout = None |
Contributor
There was a problem hiding this comment.
hard_timeout is a new user-facing config key, but this diff does not register it in DEFAULT_CONFIG or document it. Please add the default and user-facing reference alongside the existing auxiliary.vision.timeout settings so it is discoverable and has stable config semantics.
Surface the vision wall-clock cap (auxiliary.vision.hard_timeout) in DEFAULT_CONFIG and the config docs. Default 0 = auto (timeout + 60s), so current behavior is unchanged; operators can now set an explicit cap. Add hermetic tests: a stalled vision call is bounded by the hard timeout and returns the friendly "taking too long" message instead of hanging the turn, and the gateway 502/503/504 error hint maps to the transient-retry message.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two small resilience/UX fixes for provider degradation:
Bound vision calls with a wall-clock timeout (
tools/vision_tools.py).vision_analyze_toolawaitedasync_call_llmwith only the per-attempt SDK/transport timeout. When a provider degrades — a 503 storm, or a stalled socket the transport read-timeout never trips — theawaitcan hang far past the configuredtimeout. Vision runs inside the agent turn, so a hung call blocks the entire conversation and the user gets no response at all. Fix: wrap the call inasyncio.wait_forwith a wall-clock bound (auxiliary.vision.hard_timeout, default = per-attempttimeout+ 60s) and return a clear "provider taking too long, try again" message on timeout, instead of hanging the turn silently.Add a 502/503/504 hint to the agent error handler (
gateway/run.py).The per-status hint ladder covered 401/402/429/529/400/500 but not the common 502/503/504 "provider temporarily unavailable" class, so a transient provider 5xx surfaced as a bare "Sorry, I encountered an error" with no actionable guidance. Adds a 5xx branch telling the user it's transient and to retry.
Why
A turn that exhausts providers/retries — or stalls on a slow vision provider — should never leave the user in silence, and a single stalled image-analysis turn should not block the whole conversation.
Notes
hard_timeoutdefaults totimeout + 60swhen unset, so existing configs are unchanged.vision_analyze_toolpath; both files compile.