fix(vision): pass auxiliary.vision provider/model/base_url/api_key to async_call_llm

[HalfRui]

Summary

vision_analyze silently ignores the user's auxiliary.vision provider/model/base_url/api_key configuration and falls back to the chat default model — so vision breaks for any user with a non-default vision backend (e.g. a domestic Chinese provider, a dedicated vision endpoint, or any model the chat default backend doesn't recognize).

The visible symptom is a 400 error from the vision backend like:

"invalid params, unknown model 'deepseek-v4-flash' (2013)"

…even though auxiliary.vision.model is set to something else in config.yaml.

Root cause

In tools/vision_tools.py, the call_kwargs dict passed to async_call_llm(task="vision", ...) only sets model when an explicit model parameter is supplied by the caller. It never reads provider, base_url, or api_key from auxiliary.vision config at all. So async_call_llm falls through to its internal default resolution, which uses the main chat model on the main chat endpoint, instead of the user's configured vision backend.

Fix

Read the full auxiliary.vision config block and pass provider, model, base_url, and api_key into call_kwargs explicitly. The explicit model parameter (when called programmatically) still wins, matching existing behavior.

Reproduction

# ~/.hermes/config.yaml
auxiliary:
  vision:
    provider: minimax-m3
    model: MiniMax-M3
    base_url: https://api.minimax.chat/v1
    api_key: ${MINIMAX_API_KEY}

Then call vision_analyze with any image. Before the fix: 400 unknown model 'deepseek-v4-flash' (the chat default). After the fix: correct vision result.

Test plan

• Configure auxiliary.vision to a non-chat provider
• Call vision_analyze with a public image
• Confirm success: true and accurate description
• Confirm the explicit model arg to _run_vision (when called programmatically) still overrides config

Notes

• Only the image path is patched. The video_url path (line 1614 in v0.18.0's installed copy) has the same shape but a different fallback chain — I left it alone to keep this PR minimal. Happy to extend if the maintainers want symmetry.
• Backward-compatible: if auxiliary.vision is unset, behavior is unchanged.

[teknium1]

Closing — the premise doesn't reproduce on current main. async_call_llm(task="vision") resolves the full auxiliary.vision config through _resolve_task_provider_model(): verified E2E with a temp HERMES_HOME configuring provider: custom + model + base_url + api_key, and the task-only resolution returns all four values, which the vision branch then passes into resolve_vision_provider_client(...). Recent fixes in this area (a8841e2 provider-identity preservation, #59140 explicit-provider config adoption) may have resolved what you were seeing.

Separately, duplicating the auxiliary.vision config read inside tools/vision_tools.py would be the wrong layer — endpoint resolution is owned by _resolve_task_provider_model in agent/auxiliary_client.py, and per-callsite config reads drift out of sync with it.

If you still hit the wrong-model failure on current main, please open a fresh issue with the exact config and the resolved endpoint logged — happy to dig in. Thanks!