Skip to content

fix: support Ollama Cloud and remote Ollama endpoints - #2931

Closed
kshitijk4poor wants to merge 3 commits into
NousResearch:mainfrom
kshitijk4poor:fix/ollama_cloud
Closed

kshitijk4poor wants to merge 3 commits into
NousResearch:mainfrom
kshitijk4poor:fix/ollama_cloud

Conversation

@kshitijk4poor

Copy link
Copy Markdown
Contributor

Summary

Hermes assumed Ollama servers are always local (localhost / RFC-1918 IPs). Users pointing at Ollama Cloud or self-hosted remote Ollama instances found that:

  • Context length detection fell back to 128K default instead of querying /api/show
  • Server type detection (/api/tags) was skipped entirely
  • Model auto-detection only worked for localhost / 127.0.0.1

The workaround was to run Ollama in a local Docker container, pull cloud models there, and point hermes at the Docker IP — which worked only because private IPs pass the is_local_endpoint() check.

This PR removes that locality gate so any custom endpoint (not just local ones) gets full Ollama-aware probing.

Changes

agent/model_metadata.py

  • Rename detect_local_server_type()detect_server_type() — now accepts _is_local param to avoid redundant is_local_endpoint() calls
  • Rename _query_local_context_length()_query_server_context_length() — threads locality check through to avoid double computation
  • Remove is_local_endpoint() gate from context length probing at step 3 — any custom (non-known-provider) endpoint is now probed
  • Remove redundant step 9 duplicate probe (step 3 already covers the same URL)
  • Use longer httpx timeouts for remote endpoints (2s→5s for detection, 3s→8s for context queries)

cli.py

  • Broaden model auto-detect from localhost/127.0.0.1 check to _is_known_provider_base_url() — excludes all known providers (OpenRouter, OpenAI, Anthropic, etc.), not just OpenRouter

hermes_cli/runtime_provider.py

  • Same broadening in _get_model_config() using _is_known_provider_base_url()

Tests

  • Updated all references to renamed functions
  • Added test_remote_custom_endpoint_queries_server — verifies cloud Ollama URLs ARE probed
  • Added test_known_provider_endpoint_does_not_query_server — verifies known providers are NOT probed
  • Updated existing test to mock the new _query_server_context_length call path

Test plan

  • python -m pytest tests/agent/test_model_metadata.py tests/test_model_metadata_local_ctx.py -q — 95 tests pass
  • Full suite python -m pytest tests/ -q — 6136 passed, 0 new failures (4 pre-existing unrelated)
  • Point base_url at a remote Ollama instance (cloud or VPS) and verify context length detection works
  • Point base_url at local Ollama (localhost:11434) and verify existing behavior is unchanged
  • Verify known providers (OpenRouter, OpenAI, Anthropic) are NOT probed with /api/tags

The codebase assumed Ollama servers are always local (localhost/RFC-1918).
Cloud Ollama instances and self-hosted remote Ollama failed because:

- is_local_endpoint() rejected non-local URLs
- Ollama-specific metadata queries (/api/show, /api/tags) were skipped
- Context length detection fell back to generic 128K default
- Model auto-detection only checked localhost/127.0.0.1

Changes:
- Remove is_local_endpoint() gate from context length queries — any
  custom (non-known-provider) endpoint is now probed for server type
- Broaden model auto-detect in CLI and runtime_provider to work with
  any custom base_url (not just localhost)
- Increase httpx timeouts for remote endpoints (2s→5s for server type
  detection, 3s→8s for context length queries)
- Update tests to reflect new behavior
- Rename detect_local_server_type → detect_server_type (no longer local-only)
- Rename _query_local_context_length → _query_server_context_length
- Thread _is_local param through detect_server_type to avoid redundant
  is_local_endpoint() calls
- Use _is_known_provider_base_url() in cli.py and runtime_provider.py
  instead of raw "openrouter.ai" string check (safer normalization,
  excludes all known providers not just OpenRouter)
- Remove redundant _is_known_provider_base_url guard at step 3 (already
  inside step 2 block that checks the same condition)
- Remove duplicate step 9 server probe (step 3 already probes the same
  URL, so step 9 was wasted work for unreachable endpoints)
- Update all test references to match renamed functions
@krisiliev-dev

Copy link
Copy Markdown

Any updates on this PR? I am running into the issue mentioned above and I am not sure how to fix it.

@kshitijk4poor

Copy link
Copy Markdown
Contributor Author

Closing — this is our old PR from before Ollama Cloud was implemented. Ollama Cloud is now fully supported as a first-class provider on main (added in PR #10782). Provider registration, model fetching with caching, and credential resolution are all in place.

Note: there's a known bug where the /model TUI picker shows 0 models for Ollama Cloud — tracked in #10977 with a fix in PR #10964.

@dorusone

Copy link
Copy Markdown

Hermes agent 0.7. does not connect to een started and working local Ollama tried everything

@kshitijk4poor

Copy link
Copy Markdown
Contributor Author

Hermes agent 0.7. does not connect to een started and working local Ollama tried everything

hey please update the latest hermes, this is fixed!

@kshitijk4poor
kshitijk4poor deleted the fix/ollama_cloud branch August 5, 2026 07:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants