Skip to content

fix(runtime): honour providers-dict api_key/api_mode when resolving - #10910

Closed
luigileap wants to merge 1 commit into
NousResearch:mainfrom
luigileap:fix/runtime-provider-prefer-providers-api-key
Closed

fix(runtime): honour providers-dict api_key/api_mode when resolving#10910
luigileap wants to merge 1 commit into
NousResearch:mainfrom
luigileap:fix/runtime-provider-prefer-providers-api-key

Conversation

@luigileap

Copy link
Copy Markdown

Problem

$ python -m pytest tests/hermes_cli/test_runtime_provider_resolution.py::test_named_custom_provider_uses_providers_dict_when_list_missing
...
>       assert resolved["api_key"] == "dir-key"
E       AssertionError: assert 'no-key-required' == 'dir-key'

The test sets up a v12 providers dict with an inline API key and
explicit transport:

"providers": {
    "openai-direct-primary": {
        "api": "https://api.openai.com/v1",
        "api_key": "dir-key",
        "default_model": "gpt-5-mini",
        "transport": "codex_responses",
    }
}

and expects resolve_runtime_provider(requested="openai-direct-primary")
to come back with api_key="dir-key" and api_mode="codex_responses".
Instead it gets api_key="no-key-required" and the default
chat_completions mode.

Root cause

_get_named_custom_provider (in hermes_cli/runtime_provider.py) only
resolved the API key by reading the env var whose name is stored in the
providers entry's key_env field:

key_env = str(entry.get("key_env", "") or "").strip()
resolved_api_key = os.getenv(key_env, "").strip() if key_env else ""

It never looked at the literal api_key field on the entry. When
users wrote api_key directly (the natural thing given the field
name), the returned dict had api_key="". Downstream,
_resolve_named_custom_runtime walked its fallback chain, found no
candidate, and stamped the final result with "no-key-required"
which the HTTP client treats as "skip auth" and the remote API then
rejects.

Similarly, api_mode/transport was read out of the entry but never
placed in the returned dict, so downstream always fell back to
_detect_api_mode_for_url(base_url) / chat_completions, ignoring
the user's explicit choice.

Fix

In _get_named_custom_provider:

  • Prefer the literal api_key field on the providers entry; fall back
    to the key_env lookup when api_key is missing.
  • Read api_mode (or its transport alias) and include it in the
    returned dict when present.

Both custom_providers:-list-style and providers:-dict-style user
endpoints now behave the same way with respect to API keys and
transport selection.

Verification

$ python -m pytest --override-ini="addopts=" -q \
    tests/hermes_cli/test_runtime_provider_resolution.py
...................................................................      [100%]
67 passed in 0.44s

All 67 runtime-provider resolution tests pass, including the previously
failing test_named_custom_provider_uses_providers_dict_when_list_missing.

When a user defines a custom provider via the v12 providers dict:

    providers:
      openai-direct-primary:
        api: https://api.openai.com/v1
        api_key: dir-key
        default_model: gpt-5-mini
        transport: codex_responses

_get_named_custom_provider() only looked at ``key_env`` for the API
key and ignored the literal ``api_key`` field, so the provider came
back with an empty api_key.  _resolve_named_custom_runtime() then
fell through all remaining candidates and stamped the final result
with ``api_key: "no-key-required"`` — which the downstream client
treats as "no auth required" and fails on the first request.

api_mode/transport were similarly dropped: the entry field existed
but never made it into the returned dict, so the default
``chat_completions`` mode was used for providers that had explicitly
declared ``transport: codex_responses``.

Prefer the literal ``api_key`` on the providers entry and fall back
to the ``key_env`` lookup when absent; propagate ``api_mode`` /
``transport`` into the returned dict so named custom providers using
the v12 format behave like the legacy ``custom_providers`` list
format did.

Regression test: test_named_custom_provider_uses_providers_dict_when_list_missing
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/cli CLI entry point, hermes_cli/, setup wizard area/config Config system, migrations, profiles labels Apr 25, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Related to #13087 (api_mode fix) and #14065 (api_key tracking issue). This PR fixes both api_key and api_mode in the same code path (_get_named_custom_provider).

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the thorough write-up, @luigileap! Both fixes described here are already on main — this is an automated hermes-sweeper review.

api_key inline fallback landed in commit 2558d28a9 (PR #9483, Apr 14 2026) — _get_named_custom_provider now tries key_env first and falls back to entry.get('api_key') at hermes_cli/runtime_provider.py lines 347–351, two days before this PR was opened.

api_mode propagation landed in commit b29287258 (PR #15059, Apr 24 2026) — both match paths in the providers: dict branch now call _parse_api_mode(entry.get('api_mode')) and include it in the returned dict (lines 363–365, 381–383). The member comment linking to #13087 and #14065 is consistent with this: those issues were resolved as part of that broader fix.

The target test (test_named_custom_provider_uses_providers_dict_when_list_missing) is already present in the test suite at line 641 of tests/hermes_cli/test_runtime_provider_resolution.py and passes on current main — the codex_responses assertion holds via URL-based auto-detection for api.openai.com even without an explicit transport alias read.

Closing as implemented on main.

@teknium1 teknium1 closed this Apr 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/cli CLI entry point, hermes_cli/, setup wizard P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants