fix: support Ollama Cloud and remote Ollama endpoints - #2931
Closed
kshitijk4poor wants to merge 3 commits into
Closed
kshitijk4poor wants to merge 3 commits into
kshitijk4poor wants to merge 3 commits into
Conversation
The codebase assumed Ollama servers are always local (localhost/RFC-1918). Cloud Ollama instances and self-hosted remote Ollama failed because: - is_local_endpoint() rejected non-local URLs - Ollama-specific metadata queries (/api/show, /api/tags) were skipped - Context length detection fell back to generic 128K default - Model auto-detection only checked localhost/127.0.0.1 Changes: - Remove is_local_endpoint() gate from context length queries — any custom (non-known-provider) endpoint is now probed for server type - Broaden model auto-detect in CLI and runtime_provider to work with any custom base_url (not just localhost) - Increase httpx timeouts for remote endpoints (2s→5s for server type detection, 3s→8s for context length queries) - Update tests to reflect new behavior
- Rename detect_local_server_type → detect_server_type (no longer local-only) - Rename _query_local_context_length → _query_server_context_length - Thread _is_local param through detect_server_type to avoid redundant is_local_endpoint() calls - Use _is_known_provider_base_url() in cli.py and runtime_provider.py instead of raw "openrouter.ai" string check (safer normalization, excludes all known providers not just OpenRouter) - Remove redundant _is_known_provider_base_url guard at step 3 (already inside step 2 block that checks the same condition) - Remove duplicate step 9 server probe (step 3 already probes the same URL, so step 9 was wasted work for unreachable endpoints) - Update all test references to match renamed functions
1 task
|
Any updates on this PR? I am running into the issue mentioned above and I am not sure how to fix it. |
Contributor
Author
|
Closing — this is our old PR from before Ollama Cloud was implemented. Ollama Cloud is now fully supported as a first-class provider on Note: there's a known bug where the |
|
Hermes agent 0.7. does not connect to een started and working local Ollama tried everything |
Contributor
Author
hey please update the latest hermes, this is fixed! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Hermes assumed Ollama servers are always local (localhost / RFC-1918 IPs). Users pointing at Ollama Cloud or self-hosted remote Ollama instances found that:
/api/show/api/tags) was skipped entirelylocalhost/127.0.0.1The workaround was to run Ollama in a local Docker container, pull cloud models there, and point hermes at the Docker IP — which worked only because private IPs pass the
is_local_endpoint()check.This PR removes that locality gate so any custom endpoint (not just local ones) gets full Ollama-aware probing.
Changes
agent/model_metadata.pydetect_local_server_type()→detect_server_type()— now accepts_is_localparam to avoid redundantis_local_endpoint()calls_query_local_context_length()→_query_server_context_length()— threads locality check through to avoid double computationis_local_endpoint()gate from context length probing at step 3 — any custom (non-known-provider) endpoint is now probedcli.pylocalhost/127.0.0.1check to_is_known_provider_base_url()— excludes all known providers (OpenRouter, OpenAI, Anthropic, etc.), not just OpenRouterhermes_cli/runtime_provider.py_get_model_config()using_is_known_provider_base_url()Tests
test_remote_custom_endpoint_queries_server— verifies cloud Ollama URLs ARE probedtest_known_provider_endpoint_does_not_query_server— verifies known providers are NOT probed_query_server_context_lengthcall pathTest plan
python -m pytest tests/agent/test_model_metadata.py tests/test_model_metadata_local_ctx.py -q— 95 tests passpython -m pytest tests/ -q— 6136 passed, 0 new failures (4 pre-existing unrelated)base_urlat a remote Ollama instance (cloud or VPS) and verify context length detection worksbase_urlat local Ollama (localhost:11434) and verify existing behavior is unchanged/api/tags