Skip to content

fix(vision,gateway): wall-clock timeout for vision calls + 502/503/504 error hint - #37440

Open
daniel-rudaev wants to merge 3 commits into
NousResearch:mainfrom
daniel-rudaev:fix/vision-wallclock-timeout-5xx-hint
Open

fix(vision,gateway): wall-clock timeout for vision calls + 502/503/504 error hint#37440
daniel-rudaev wants to merge 3 commits into
NousResearch:mainfrom
daniel-rudaev:fix/vision-wallclock-timeout-5xx-hint

Conversation

@daniel-rudaev

Copy link
Copy Markdown

What

Two small resilience/UX fixes for provider degradation:

  1. Bound vision calls with a wall-clock timeout (tools/vision_tools.py).
    vision_analyze_tool awaited async_call_llm with only the per-attempt SDK/transport timeout. When a provider degrades — a 503 storm, or a stalled socket the transport read-timeout never trips — the await can hang far past the configured timeout. Vision runs inside the agent turn, so a hung call blocks the entire conversation and the user gets no response at all. Fix: wrap the call in asyncio.wait_for with a wall-clock bound (auxiliary.vision.hard_timeout, default = per-attempt timeout + 60s) and return a clear "provider taking too long, try again" message on timeout, instead of hanging the turn silently.

  2. Add a 502/503/504 hint to the agent error handler (gateway/run.py).
    The per-status hint ladder covered 401/402/429/529/400/500 but not the common 502/503/504 "provider temporarily unavailable" class, so a transient provider 5xx surfaced as a bare "Sorry, I encountered an error" with no actionable guidance. Adds a 5xx branch telling the user it's transient and to retry.

Why

A turn that exhausts providers/retries — or stalls on a slow vision provider — should never leave the user in silence, and a single stalled image-analysis turn should not block the whole conversation.

Notes

  • Minimal + backward-compatible: hard_timeout defaults to timeout + 60s when unset, so existing configs are unchanged.
  • Validated: the wall-clock guard fires and returns the timeout message through the real vision_analyze_tool path; both files compile.

vision_analyze_tool awaited async_call_llm with only the per-attempt
SDK/transport timeout. When a provider degrades (a 503 storm, or a stalled
socket the transport read-timeout never trips) the await can hang far past
the configured timeout. Vision runs inside the agent turn, so a hung call
blocks the entire conversation and the user gets no response at all.

Wrap the call in asyncio.wait_for with a wall-clock bound
(auxiliary.vision.hard_timeout, default = per-attempt timeout + 60s) and
return a clear 'provider taking too long, try again' message on timeout,
instead of hanging the turn silently.
The per-status hint ladder covered 401/402/429/529/400/500 but not the
common 502/503/504 'provider temporarily unavailable' class, so a transient
provider 5xx surfaced as a bare 'Sorry, I encountered an error' with no
actionable guidance. Add a 5xx branch telling the user it's transient and
to retry.
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery tool/vision Vision analysis and image generation labels Jun 2, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing two live resilience gaps. Current main still directly awaits async_call_llm in tools/vision_tools.py:1253, and the gateway hint ladder in gateway/run.py:12258-12296 still omits 502/503/504.

Problems

  • The new auxiliary.vision.hard_timeout surface is only read by the PR. It is absent from the defaults in hermes_cli/config.py:1577-1585 and the documented auxiliary reference at website/docs/user-guide/configuration.md:998-1009.
  • The diff adds no regression coverage. Existing vision config coverage in tests/tools/test_vision_tools.py:424-481 covers timeout and temperature, not a stalled coroutine; no gateway test covers this status-hint branch.

Suggested changes

  • Register and document hard_timeout alongside the existing vision timeout settings.
  • Add hermetic tests for a never-completing vision call and for 502/503/504 gateway hints.

This is an automated hermes-sweeper review.

Comment thread tools/vision_tools.py
# Local vision models (llama.cpp, ollama) can take well over 30s.
vision_timeout = 120.0
vision_temperature = 0.1
vision_hard_timeout = None

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hard_timeout is a new user-facing config key, but this diff does not register it in DEFAULT_CONFIG or document it. Please add the default and user-facing reference alongside the existing auxiliary.vision.timeout settings so it is discoverable and has stable config semantics.

@teknium1 teknium1 added sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 13, 2026
Surface the vision wall-clock cap (auxiliary.vision.hard_timeout) in
DEFAULT_CONFIG and the config docs. Default 0 = auto (timeout + 60s), so
current behavior is unchanged; operators can now set an explicit cap.

Add hermetic tests: a stalled vision call is bounded by the hard timeout and
returns the friendly "taking too long" message instead of hanging the turn,
and the gateway 502/503/504 error hint maps to the transient-retry message.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages tool/vision Vision analysis and image generation type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants