Skip to content

fix(desktop): onboarding can configure a local/custom endpoint without an API key - #38265

Closed
xxxigm wants to merge 2 commits into
NousResearch:mainfrom
xxxigm:fix/desktop-onboarding-local-endpoint
Closed

fix(desktop): onboarding can configure a local/custom endpoint without an API key#38265
xxxigm wants to merge 2 commits into
NousResearch:mainfrom
xxxigm:fix/desktop-onboarding-local-endpoint

Conversation

@xxxigm

@xxxigm xxxigm commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Makes Hermes Desktop onboarding's Local / custom endpoint option actually work — including the common case where a self-hosted endpoint needs no API keywithout any onboarding UI/form changes (the existing single URL field is enough).

The bug (from Discord support thread)

Selecting "Local / custom endpoint" failed with:

No usable credentials found for custom. setup status reports configured credentials, but runtime resolution still failed.

…and would not let the user continue, even though local LLM servers (vLLM, llama.cpp, Ollama, …) typically require no key.

Root cause

The onboarding "local" option only wrote the OPENAI_BASE_URL env var (saveOnboardingApiKeysetEnvVar). But runtime resolution deliberately ignores OPENAI_BASE_URLhermes_cli/runtime_provider.py is explicit:

OPENAI_BASE_URL env var is no longer consulted — config.yaml is the single source of truth for endpoint URLs.

So setting it did nothing: resolution fell through to the default provider (no key) and setup.runtime_check reported "No usable credentials found for custom". There was also no API path to persist model.base_url from the GUI (/api/model/set actively cleared base_url).

Changes Made

Backend (hermes_cli/web_server.py)

  • POST /api/model/set accepts an optional base_url. For provider=custom it persists model.base_url (and echoes it back); for hosted providers it still clears any stale base_url. This is the missing primitive that wires a local endpoint into resolution.
  • POST /api/providers/validate now returns the model ids a custom endpoint advertises at /v1/models (tolerant of OpenAI/vLLM/llama.cpp shapes via _parse_model_ids), so the GUI can auto-pick a default.

Frontend (apps/desktop/src/…) — no onboarding form/UI changes

  • The local option routes through saveOnboardingLocalEndpoint: probe the endpoint, auto-discover the model from /v1/models, persist provider=custom + base_url + model via /api/model/set, then verify the runtime directly. It does not route through completeWithModelConfirm, which would re-assign the model without a base_url and wipe it.
  • The existing single URL field is reused; the user only pastes a URL. If the endpoint advertises no models, a clear error is shown.
  • Types updated (ModelAssignmentRequest.base_url?, validate response models?).

How to Test

# Backend
pytest tests/hermes_cli/test_web_server.py -k "set_model or parse_model_ids" -q   # 6 passed

# Frontend
cd apps/desktop && npx tsc -b                                 # type-checks clean
npx vitest run --environment jsdom src/store/onboarding.test.ts

Manual: onboarding → "Local / custom endpoint" → paste http://127.0.0.1:8000/v1 → Connect → model auto-discovered, no "No usable credentials" error, chatting works with no API key.

Tests added

  • Backend: _parse_model_ids tolerates OpenAI/bare/list shapes and never raises; /api/model/set persists base_url for custom and clears stale base_url for hosted providers.
  • Frontend: saveOnboardingLocalEndpoint errors when no models are advertised, auto-discovers + persists provider=custom+base_url+model and completes, and surfaces the runtime reason on failure.

Note: the frontend vitest suite wasn't runnable in my sandbox (the shared node_modules jsdom toolchain hit an unrelated @exodus/bytes ESM resolution error); tsc -b passes and the test is included for CI.

Scope note

A Nous maintainer mentioned in the support thread they'd improve this onboarding experience. This PR is a focused, self-contained fix for the local/custom no-key path that leaves the onboarding UI untouched; happy to rebase/trim if it overlaps with in-flight work.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/cli CLI entry point, hermes_cli/, setup wizard area/config Config system, migrations, profiles labels Jun 3, 2026
@xxxigm
xxxigm force-pushed the fix/desktop-onboarding-local-endpoint branch from b905b82 to 86179fe Compare June 3, 2026 14:41
xxxigm added 2 commits June 4, 2026 06:57
The runtime resolver reads model.base_url from config and ignores the
OPENAI_BASE_URL env var, so a self-hosted endpoint could not be configured
from the GUI. Two changes enable it:

- POST /api/model/set accepts an optional base_url and persists it as
  model.base_url when provider=custom (still clearing stale base_url for
  hosted providers).
- POST /api/providers/validate now returns the model ids a custom endpoint
  advertises at /v1/models, so the GUI can auto-pick a default without
  asking the user to type a model name.

Refs desktop onboarding "Local / custom endpoint" bug.
…I changes

Onboarding's "Local / custom endpoint" only wrote the OPENAI_BASE_URL env
var, which runtime resolution ignores — so a self-hosted endpoint was never
wired in and setup failed with "No usable credentials found for custom" even
though local servers need no key.

Route the local option through saveOnboardingLocalEndpoint: probe the
endpoint, auto-discover a model from /v1/models, persist provider=custom +
base_url + model via /api/model/set, then verify the runtime directly
(not via completeWithModelConfirm, which would re-assign the model without
base_url and wipe it). No onboarding form/UI changes — the existing single
URL field is enough.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/cli CLI entry point, hermes_cli/, setup wizard P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants