Allow custom providers to configure default headers

[sunnysabor]

Summary

Allow custom OpenAI-compatible providers to configure client-level default_headers.

Some custom gateways require provider-specific HTTP headers or block certain SDK request fingerprints. This change lets users configure those headers in custom_providers without patching Hermes
provider code.

Example:

custom_providers:
- name: my-gateway
  base_url: https://example.com/v1
  api_mode: chat_completions
  default_headers:
    User-Agent: curl/8.7.1

## Changes

- Preserve default_headers during custom provider config normalization.
- Include default_headers in custom provider runtime request_overrides.
- Apply configured custom provider headers to main-agent OpenAI client construction.
- Apply configured custom provider headers to named custom provider auxiliary clients.
- Add regression tests for runtime resolution and auxiliary client construction.

## Motivation

This is a compatibility improvement for OpenAI-compatible custom providers. It avoids hardcoding gateway-specific headers globally while giving users a supported way to adapt to provider-specific
HTTP requirements.

## Test plan

HERMES_HOME=/private/tmp/hermes-headers-test-home \
venv/bin/python -m pytest \
  tests/hermes_cli/test_runtime_provider_resolution.py::test_named_custom_runtime_propagates_default_headers \
  tests/hermes_cli/test_runtime_provider_resolution.py::test_custom_provider_config_normalizes_default_headers \
  tests/agent/test_auxiliary_client.py::TestResolveProviderClientUniversalModelFallback::test_named_custom_provider_passes_default_headers \
  -q

Result:

3 passed

I also ran:

HERMES_HOME=/private/tmp/hermes-headers-test-home \
venv/bin/python -m pytest \
  tests/agent/test_auxiliary_client.py \
  tests/hermes_cli/test_runtime_provider_resolution.py \
  -q

Most tests passed, but two existing Qwen OAuth runtime-provider tests failed due to local credential state being picked up during the test run, unrelated to this change.

[iamfoz]

+1 on landing a canonical per-provider header map. One use case worth designing for while the shape is still open: header-routed proxies that fan one endpoint out to many upstreams.

Some gateways (mine is a local OpenAI-compatible proxy that picks the upstream from an X-Upstream: style header) expose several logical providers behind a single base_url, distinguished only by header value. The same key, the same URL, different header per upstream. That works fine for client construction with this PR as written, but it has knock-on implications elsewhere that are easier to get right now than to retrofit:

• The /model picker currently groups custom providers by (base_url, api_key), so entries that differ only by header collapse into one row and all but the first are dropped. If headers are part of the provider identity, the grouping key needs to include them so each upstream shows as its own selectable row.
• /models probing against the gateway needs the entry's headers, or every row gets the same upstream's catalog.
• On switch and on restart, the per-entry headers have to ride along so the selected upstream is the one actually used.

None of that needs to be in this PR. I just want to flag that "headers are part of what distinguishes one custom provider from another," not only an auth detail on a single gateway, so the config shape you settle on here leaves room for it. Happy to contribute the picker and switching side as a follow-up once the header key lands. I have it working against my fork already.

[kshitijk4poor]

Closing as superseded by #41096, which adds model.default_headers support for custom OpenAI-compatible providers across both the main and auxiliary client paths (overriding the OpenAI SDK's User-Agent/X-Stainless-* headers that some gateways/WAFs reject — #40033). Thanks for the contribution! 🙏