[Bug]: _query_local_context_length unconditionally probes admin-gated /v1/models/{model} on LiteLLM proxies

[j4s0n]

Bug Description

When using a custom provider pointing at a LiteLLM proxy, Hermes unconditionally sends GET /v1/models/{model} during context-length resolution in _query_local_context_length(). LiteLLM gates this endpoint behind proxy_admin access (because it shares the same HTTP path as PUT/DELETE model mutation operations). The result is an ERROR-level log on the LiteLLM side on every agent turn, even though Hermes handles the 401 gracefully by falling through to the working /v1/models list endpoint.

This is a separate root cause from #8465 (LiteLLM context window detection) and #12153 (/v1/models endpoint unavailable), though it manifests similarly.

Steps to Reproduce

1. Configure a custom provider pointing at a LiteLLM proxy:

custom_providers:
name: litellm-local
base_url: http://:4000/v1
api_key: sk-...
models:
some-model:
context_length: 0 # let Hermes auto-probe

2. Set Hermes to use this provider: model.provider: custom:litellm-local

3. Send any message to the agent

4. Observe LiteLLM server logs

Expected Behavior

When detect_local_server_type() returns None (unrecognized server), Hermes should skip the admin-gated /v1/models/{model} probe and use only the universally-supported /v1/models list endpoint for context-length discovery.

Actual Behavior

Every agent turn produces this in LiteLLM's logs:

LiteLLM Proxy:ERROR: auth_exception_handler.py:95 -
  Only proxy admin can be used to generate, delete, update info
  for new keys/users/teams. Route=/v1/models/free-router.
  Your role=unknown. Your user_id=unknown
INFO: 172.16.15.123 - "GET /v1/models/free-router HTTP/1.1" 401 Unauthorized

Hermes then falls through to GET /v1/models (list), which returns 200 — so it's cosmetic log spam, not a functional break.

Affected Component

Agent Core (conversation loop, context compression, memory)

Messaging Platform (if gateway-related)

N/A (CLI only)

Debug Report

This issue was diagnosed on a headless agent host where hermes debug share is not applicable. The following evidence is provided in lieu:

- LiteLLM server log excerpt: shows the 15-second cadence of GET `/v1/models/{model}` → 401 errors with full traceback
- Hermes code trace: `agent/model_metadata.py` lines 1096–1187, function `_query_local_context_length()`
- LiteLLM config analysis: `/v1/models/{model}` is classified admin-only in LiteLLM's `route_checks.py::non_proxy_admin_allowed_routes_check`

Full reproduction environment: Hermes agent host at 172.16.15.123, LiteLLM proxy at 172.16.15.104:4000, Docker Compose stack litellm.

Operating System

Linux (Hermes host: Ubuntu; LiteLLM host: Debian via Docker)

Python Version

3.13.5

Hermes Version

Hermes Agent v0.13.0 (2026.5.7)

Additional Logs / Traceback (optional)

The LiteLLM-side full traceback (per-occurrence):


Exception: Only proxy admin can be used to generate, delete, update
info for new keys/users/teams. Route=/v1/models/free-router.
Your role=unknown. Your user_id=unknown

File "/app/.venv/lib/python3.13/site-packages/litellm/proxy/auth/
     user_api_key_auth.py", line 2130, in user_api_key_auth
    await _run_centralized_common_checks(...)
File ".../litellm/proxy/auth/auth_checks.py", line 683, in common_checks
    _is_route_allowed = _is_api_route_allowed(...)
File ".../litellm/proxy/auth/route_checks.py", line 300,
    in non_proxy_admin_allowed_routes_check
    RouteChecks._raise_admin_only_route_exception(...)


The Hermes-side flow is silent — httpx.Client.get() returns a 401 response, line 1166's if resp.status_code == 200: check fails, and execution falls through to the working /v1/models list probe at line 1173.

Root Cause Analysis (optional)

In agent/model_metadata.py, _query_local_context_length() at line ~1096:

1. Line 1112: Calls detect_local_server_type() — probes Ollama/LM Studio/vLLM/llamacpp-specific endpoints. On LiteLLM all return 404, so server_type = None.
2. Lines 1119–1161: Handle ollama and lm-studio server types (skipped for LiteLLM).
3. Lines 1164–1165 (BUG): Unconditionally probes GET /v1/models/{model}. LiteLLM requires proxy_admin for this endpoint because it shares the same HTTP method+path as model mutation operations (PUT/DELETE). Regular API keys get 401.
4. Line 1166: if resp.status_code == 200: fails → falls through.
5. Lines 1173–1175: Calls GET /v1/models (list) → 200 OK, returns correct metadata. Works universally.

The fix: gate the single-model probe on server_type is not None, since only vLLM/llamacpp are known to support this endpoint with non-admin keys.

Proposed Fix (optional)

--- a/agent/model_metadata.py
+++ b/agent/model_metadata.py
@@ -1164,12 +1164,14 @@ def _query_local_context_length(model, base_url, api_key=""):

•        # LM Studio / vLLM / llama.cpp: try /v1/models/{model}
  

•        resp = client.get(f"{server_url}/v1/models/{model}")
  

•        if resp.status_code == 200:
  

•            data = resp.json()
  

•            ctx = data.get("max_model_len") or data.get("context_length") or data.get("max_tokens")
  

•            if ctx and isinstance(ctx, (int, float)):
  

•                return int(ctx)
  

•        # vLLM / llama.cpp: try /v1/models/{model} for context window.
  

•        # Skip for unknown server types (e.g. LiteLLM proxy) that gate
  

•        # this endpoint behind admin auth — the /v1/models list call
  

•        # below works on all OpenAI-compatible servers.
  

Are you willing to submit a PR for this?

• I'd like to fix this myself and submit a PR

[alt-glitch]

Related: #22719 (first-class LiteLLM proxy provider that skips backend probes), #8509 (LiteLLM-aware context window detection), #18468 (honor LITELLM_KEY for model metadata probes), #8465 (umbrella "Proper LiteLLM support" issue). The proposed fix in #22719 would eliminate this probe entirely for LiteLLM targets.