[Bug]: Hermes probes non‑OpenAI endpoints (/api/tags, /v1/props, /version) on api.openai.com, causing noisy 404s

[lanwen]

Bug Description

When using the openai-api provider with the default base URL https://api.openai.com/v1, Hermes sends additional HTTP GET requests to non‑OpenAI paths on api.openai.com:

• /api/v1/models
• /api/tags
• /v1/props
• /props
• /version

All of these return 404 Not Found (as expected, since they’re not part of the OpenAI API), but they create a lot of noise in egress logs and monitoring.

Why this matters

• In environments with strict monitoring and logging of outbound traffic, these 404s:
  • pollute logs,
  • can trigger false positives in alerting/IDS rules,
  • make it harder to distinguish real issues from harmless probes.
• They also look confusing in service‑mesh dashboards when you see “Hermes hitting /version on api.openai.com with 404s” for no apparent reason.

Steps to Reproduce

Steps to reproduce

1. Configure Hermes with:
   model:
   provider: openai-api
   default: gpt-5.1 # or any OpenAI Chat model
   base_url: "https://api.openai.com/v1"

2. Use Hermes via the gateway (Telegram) or CLI to send a few prompts to an OpenAI model (including prompts that attach images, so image routing kicks in).

3. Inspect egress / proxy / mesh logs for api.openai.com.

4. You will see 404s to the following paths, in addition to normal /v1/... traffic:

   • GET /api/v1/models
   • GET /api/tags
   • GET /v1/props
   • GET /props
   • GET /version

Expected Behavior

• For known public endpoints like https://api.openai.com/v1, Hermes should not attempt Ollama/LM Studio/vLLM/llama.cpp detection via private endpoints like /api/tags, /v1/props, /version.
• Only local or obviously custom endpoints (e.g. localhost, 10.x.x.x, host.docker.internal, user‑provided local URLs) should be probed that way.

Actual Behavior

• Hermes probes those detection endpoints even when base_url is the public OpenAI endpoint.
• This causes regular 404s to non‑existent paths on api.openai.com.

Affected Component

Configuration (config.yaml, .env, hermes setup)

Messaging Platform (if gateway-related)

No response

Debug Report

--- hermes dump ---
version:          0.18.2 [7ecc822e] (2026-07-08)
os:               Linux 6.17.0-1011-oracle aarch64
python:           3.11.15
openai_sdk:       2.24.0
profile:          default
hermes_home:      ~/.hermes
model:            gpt-5.1
provider:         openai-api
terminal:         local

api_keys:
  openrouter           not set
  openai               set
  anthropic            not set
  anthropic_token      not set
  nous                 not set
  google/gemini        not set
  gemini               not set
  glm/zai              not set
  zai                  not set
  kimi                 not set
  minimax              not set
  deepseek             not set
  dashscope            not set
  huggingface          not set
  nvidia               not set
  opencode_zen         not set
  opencode_go          not set
  kilocode             not set
  firecrawl            not set
  tavily               not set
  browserbase          not set
  fal                  not set
  elevenlabs           not set
  github               set

features:
  toolsets:           hermes-cli
  mcp_servers:        0
  memory_provider:    built-in
  gateway:            running (systemd (system), pid 62619)
  platforms:          telegram
  cron_jobs:          1 active / 1 total
  skills:             76

config_overrides:
  agent.max_turns: 150
  display.streaming: True
  display.show_reasoning: False
--- end dump ---

Operating System

ubuntu 24.04

Python Version

No response

Hermes Version

No response

Additional Logs / Traceback (optional)

Example log entries (sanitized):
Method: GET
Host: api.openai.com:443
Path: /version
Status: 404
Matched service: openai
Credential keys: OPENAI_API_KEY
Actor: hermes(agent:...)
Latency: ~130 ms
Similar entries appear for /api/tags, /v1/props, /props, /api/v1/models.

Root Cause Analysis (optional)

Root cause (code walkthrough)

The behavior matches the logic in agent/model_metadata.py::detect_local_server_type

def detect_local_server_type(base_url: str, api_key: str = "") -> Optional[str]:
    """Detect which local server is running at base_url by probing known endpoints.

    Returns one of: "ollama", "lm-studio", "vllm", "llamacpp", or None.
    """
    import httpx

    normalized = _normalize_base_url(base_url)
    server_url = normalized
    if server_url.endswith("/v1"):
        server_url = server_url[:-3]
    lmstudio_url = _lmstudio_server_root(base_url)

    headers = _auth_headers(api_key)

    try:
        with httpx.Client(timeout=2.0, headers=headers) as client:
            # LM Studio exposes /api/v1/models — check first (most specific)
            try:
                r = client.get(f"{lmstudio_url}/api/v1/models")
                if r.status_code == 200:
                    return "lm-studio"
            except Exception:
                pass
            # Ollama exposes /api/tags ...
            try:
                r = client.get(f"{server_url}/api/tags")
                ...
            except Exception:
                pass
            # llama.cpp exposes /v1/props (older builds used /props)
            try:
                r = client.get(f"{server_url}/v1/props")
                if r.status_code != 200:
                    r = client.get(f"{server_url}/props")
                ...
            except Exception:
                pass
            # vLLM: /version
            try:
                r = client.get(f"{server_url}/version")
                ...
            except Exception:
                pass
    except Exception:
        pass

    return None

And this helper is used by:

• agent/image_routing.py::_should_probe_ollama_vision (for vision routing):

  def _should_probe_ollama_vision(provider: str, base_url: str) -> bool:
      p = (provider or "").strip().lower()
      if p == "ollama":
          return True
      if not base_url:
          return False
      try:
          from agent.model_metadata import detect_local_server_type

          return detect_local_server_type(base_url) == "ollama"
      except Exception:
          return False

With provider = "openai-api" and base_url = "https://api.openai.com/v1", detect_local_server_type strips /v1 and probes:

• https://api.openai.com/api/v1/models
• https://api.openai.com/api/tags
• https://api.openai.com/v1/props
• https://api.openai.com/props
• https://api.openai.com/version

which all 404, as seen in logs.

Functionally it’s harmless, but it produces a lot of noisy 404s against a public API.

Proposed Fix (optional)

One minimal approach:

1. Add a hostname check to detect_local_server_type:

   • Normalize the URL.
   • Parse hostname via urlparse.
   • For known public provider hosts (at least api.openai.com), short‑circuit and return None without doing any probes.

2. Optionally, constrain callers like _should_probe_ollama_vision to only invoke detect_local_server_type for “local‑looking” endpoints (loopback, RFC‑1918, container DNS, etc.).

Example patch (conceptual, based on current code):

from urllib.parse import urlparse

def detect_local_server_type(base_url: str, api_key: str = "") -> Optional[str]:
    """Detect which *local-style* server is running at base_url by probing known endpoints.

    Returns one of: "ollama", "lm-studio", "vllm", "llamacpp", or None.

    To avoid spamming public SaaS APIs (e.g. api.openai.com) with private
    discovery endpoints like `/api/tags` and `/v1/props`, this helper
    short-circuits for known public hosts.
    """
    import httpx

    normalized = _normalize_base_url(base_url)
    if not normalized:
        return None

    try:
        url = normalized if "://" in normalized else f"https://{normalized}"
        host = urlparse(url).hostname or ""
    except Exception:
        host = ""

    # Do not probe standard public OpenAI endpoints
    if host in {"api.openai.com"}:
        return None

    server_url = normalized
    if server_url.endswith("/v1"):
        server_url = server_url[:-3]
    lmstudio_url = _lmstudio_server_root(base_url)

    headers = _auth_headers(api_key)

    try:
        with httpx.Client(timeout=2.0, headers=headers) as client:
            # (existing /api/v1/models, /api/tags, /v1/props, /props, /version probes)
            ...
    except Exception:
        pass

    return None

This keeps the local‑server detection behavior intact for actual local/custom endpoints, while avoiding pointless 404s against api.openai.com.

Are you willing to submit a PR for this?

• I'd like to fix this myself and submit a PR

[GottZ]

This was generated by AI during triage.

Summary

Two PRs address #61421 by preventing local-server discovery probes from reaching public OpenAI-compatible endpoints. #61428 short-circuits three known provider hosts while retaining custom-endpoint probing; #61692 instead restricts probing to a newly defined set of local/private hosts, but omits supported local endpoint classes.

Related pull requests

• fix(agent): short-circuit local server detection for known public provider hosts #61428 best fix — (+96/-1) — fixes: Short-circuits discovery before HTTP requests for api.openai.com and two other known public provider hosts, directly eliminating the reported noisy 404s while preserving probes for local and custom endpoints. The keep_open review identifies one required correction: use the existing base_url_hostname() normalization and add a trailing-dot regression test so api.openai.com. cannot bypass the guard.
• fix(agent): skip local server probes on known public endpoints #61692 [closed] partial — (+48/-0) — partial fix: The closed PR also prevents probes to api.openai.com, but its new local-network classifier duplicates and narrows is_local_endpoint(), disabling detection for supported Tailscale CGNAT addresses and unqualified local hostnames, with no regression tests. Despite the keep_open review on fix(agent): skip local server probes on known public endpoints #61692, it should remain closed rather than be reopened: the diff's broader regressions remain unresolved and fix(agent): short-circuit local server detection for known public provider hosts #61428 provides the safer consolidation path.

Duplicates

#61428 and #61692 target the same public-endpoint probe traffic, but use competing denylist and local-only allowlist mechanisms; #61692 is functionally superseded by the safer, tested direction in #61428.

Suggested consolidation

Merge #61428 after addressing its contributor review by using base_url_hostname() and covering the trailing-dot OpenAI hostname; keep #61692 closed as the superseded duplicate because its narrower duplicated classifier drops supported local endpoint classes.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I61421(["issue #61421 (open)"])
    subgraph Dup61428 ["PRs duplicating each other"]
        P61428["PR #61428 (open)"]
        P61692["PR #61692 (closed)"]
    end
    P61428 -->|best fix| I61421
    P61692 -.->|partial| I61421
    class I61421 open
    class I61421 target
    class P61428 open
    class P61692 closed
    class P61428 best
    click I61421 "https://github.com/NousResearch/hermes-agent/issues/61421"
    click P61428 "https://github.com/NousResearch/hermes-agent/pull/61428"
    click P61692 "https://github.com/NousResearch/hermes-agent/pull/61692"

Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed or no verify verdict yet (state tag in the node label).

Cross-PR triage: Reviewed 2 pull requests and 1 issue in this complex. Each diff was read against this issue; Assessment working set: 8 kB of PR diffs, 13 kB of issue/PR text, 2 kB of discussion (4 comments), 4 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.