fix(auxiliary): prepend zai coding-plan vision endpoint before metered paas/v4 (fixes #55112) - #55116
fix(auxiliary): prepend zai coding-plan vision endpoint before metered paas/v4 (fixes #55112)#55116Morad37 wants to merge 2 commits into
Conversation
|
Thanks for the quick fix — the ordering is exactly right, prepending One small nit: the new zai_openai_urls = [
"https://api.z.ai/api/coding/paas/v4", # coding-plan quota
"https://open.bigmodel.cn/api/paas/v4", # metered fallthrough <-- 8 spaces
"https://api.z.ai/api/paas/v4",
]It's valid Python (indentation is free inside the list literal), but zai_openai_urls = [
"https://api.z.ai/api/coding/paas/v4", # coding-plan quota
"https://open.bigmodel.cn/api/paas/v4", # metered fallthrough
"https://api.z.ai/api/paas/v4",
] |
🟢 Superseded — please close in favor of the new PR@Morad37 thanks for the original PR. The rebased and consolidated fix is now #62080. The vision helper The new PR also includes:
If maintainers agree, please close this PR in favor of #62080. Credit remains in git history. |
End-to-end fix for the Z.AI credential pool cascade bug discovered
during a live audit of 8 production keys. Five coordinated layers
ensure the right endpoint is selected, the wrong error is not treated
as a payment error, and the user-configured overrides work end-to-end.
== Problem statement ==
Z.AI Coding Plan subscriptions (id.secret.* format) authenticate on
/api/coding/paas/v4 but the metered /api/paas/v4 endpoint also returns
HTTP 200 for glm-5. Without explicit routing, the metered endpoint
gets cached in auth.json and Coding Plan traffic silently draws from
the metered billing pool instead of the subscription.
A second symptom: Z.AI Coding Plan error codes 1113 (Insufficient
balance or no resource package) and 1308 (Usage limit reached for 5
hour) are per-key rolling quotas. They were misclassified as payment
errors via substring traps intended for Vertex AI (resource exhausted)
and Nous Portal (reached your session usage limit). A single exhausted
key cascade-marked every other key in the pool, taking the whole
provider offline.
A third gap: keys on the Anthropic Messages wire (/api/anthropic) were
not in ZAI_ENDPOINTS at all, so detect_zai_endpoint() returned the
wrong URL for any Anthropic-wire Coding Plan key.
== Solution overview ==
Five layers, all coordinated, all tested end-to-end:
Layer 1 - _is_payment_error() exemption for Z.AI Coding Plan
Detects api.z.ai / z.ai/api/coding / zhipuai+coding context plus
codes 1113, 1308 and message patterns. Returns False so the
credential pool rotation layer handles per-key 5h rolling quotas
correctly. Real Z.AI payment errors (e.g. code 1311 plan-block)
continue to flow through the normal payment-fallback path.
Layer 2 - Vision auto-detect zai_openai_urls ordering
The vision helper list now probes Coding Plan endpoints (global +
China) before the metered paas/v4 fallbacks. Coding Plan keys
authenticate on first vision call. Corrects an indentation bug in
the original PR NousResearch#55116.
Layer 3 - Runtime pool base_url re-resolution
_resolve_api_key_provider() now re-invokes _resolve_zai_base_url()
for provider_id == zai so manual-pool entries added via
her mes auth add honor the cached detected_endpoint state in
auth.json and the GLM_BASE_URL env override. Brings manual-pool
parity with the env-seeded path fixed in commit 9e84416.
Probe failures are caught and logged so they never break pool
selection.
Layer 4 - config.yaml model.base_url precedence
New _configured_zai_base_url() helper reads model.base_url from
config.yaml when model.provider is a Z.AI alias (zai / glm / z-ai /
z.ai / zhipu). The precedence chain in _resolve_zai_base_url() is:
1. GLM_BASE_URL env var (highest)
2. model.base_url from config.yaml (when provider is Z.AI)
3. cached detected_endpoint in auth.json
4. live probe of all candidate endpoints
5. registry default
The provider alias guard prevents leakage from non-Z.AI configs.
Layer 5 - Anthropic-wire endpoints in ZAI_ENDPOINTS
Extended from 4 to 6 candidates with anthropic-global and
anthropic-cn. coding-global is probed FIRST (99% of keys that
accept coding endpoint also accept metered, so probing coding
first caches the right URL on first try). anthropic-global is
position 2 (fallthrough for pure Anthropic-wire keys).
New _zai_probe_path() and _zai_probe_body() helpers dispatch the
right HTTP body shape: Anthropic Messages for anthropic-* ids
(/v1/messages, no stream field, anthropic-version header) and
OpenAI chat completions for everything else.
== Audit results ==
Validated against 8 real production keys (see
tests/agent/test_zai_8keys_audit.py):
- 6/8 keys: pure Anthropic-wire subscribers (Anthropic Messages)
- 1/8 keys: pure OpenAI-wire (standard paas/v4)
- 1/8 keys: full-stack (works on all 3 endpoints)
Live test audit (tests/agent/test_zai_live_audit.py) covers every
Z.AI error category against the real api.z.ai API: 200, 1305, 1308,
1113, 401, 402 - all classified correctly.
Before fix: detect_zai_endpoint() returned the wrong endpoint for
6/8 keys. After fix: 9/9 keys correctly routed and verified working.
== Test coverage ==
249 unit + integration tests, 21 live tests (opt-in via env var):
tests/hermes_cli/test_zai_5gateway_extension.py (29 tests)
ZAI_ENDPOINTS structure, dispatch helpers, signature stability,
backward compat, probe order pinning
tests/hermes_cli/test_zai_config_yaml_precedence.py (8 tests)
GLM_BASE_URL > model.base_url > cached > probe > default
tests/agent/test_zai_manual_pool_routing.py (7 tests)
Runtime re-resolution, GLM_BASE_URL forwarding, probe failure
tolerance, non-zai provider leak guard
tests/agent/test_auxiliary_client_zai_payment_classification.py
(14 tests)
_is_payment_error Z.AI Coding Plan exemption, verbatim Z.AI
response bodies, negative cases for Vertex/Bedrock/OpenRouter
tests/agent/test_zai_e2e_pool.py + test_zai_e2e_pool_rotation.py
(8 tests)
Real HTTP round-trip via local mock Z.AI server, 5-key
round_robin, revoked key, probe failure recovery
tests/agent/test_zai_live.py + test_zai_live_audit.py +
test_zai_8keys_audit.py (21 tests)
Live against api.z.ai, opt-in via HERMES_RUN_LIVE=1 or
GLM_AUDIT_KEYS env var. Keys masked to 8-char prefix in all
output. File is safe to commit.
== Related work ==
PR NousResearch#61492 - _is_payment_error() Z.AI exemption (cloned in Layer 1)
PR NousResearch#55116 - vision helper coding endpoint (Layer 2)
PR NousResearch#58088 - config.yaml base_url precedence (Layer 4)
PR NousResearch#24915 - 4-variant provider split (intentionally NOT adopted:
orthogonal refactor, deferred)
PR NousResearch#54643 - /api/anthropic to /api/coding/paas/v4 rewrite
(orthogonal; this PR probes the right wire, NousResearch#54643 rewrites at
runtime if a user forces the OpenAI wire via config)
PR NousResearch#55007 - stream probes without reading bodies (orthogonal;
this PR can refactor _zai_probe_path to use httpx.stream() once
NousResearch#55007 lands)
PR NousResearch#61333 - skip /anthropic to /v1 rewrite (complementary)
PR NousResearch#60753 - preserve /anthropic for custom vision (complementary)
PR NousResearch#32174 - add zai-coding provider (different design choice)
PR NousResearch#60034 - Z.AI Coding overload adaptive backoff (already merged)
== Related issues ==
NousResearch#61487 - cascade _is_payment_error (Layer 1 closes)
NousResearch#61563 - manual-pool routing audit (this PR implements)
NousResearch#47970 - GLM-5.2 context_length fallback (Layer 5 helps)
NousResearch#55112 - auxiliary vision hardcoded zai (Layer 5 helps)
NousResearch#47685 - Hermes Agent prompt block on Z.ai (orthogonal)
== Test commands ==
Unit + integration (no network):
pytest tests/hermes_cli/test_api_key_providers.py \
tests/hermes_cli/test_zai_config_yaml_precedence.py \
tests/hermes_cli/test_zai_5gateway_extension.py \
tests/agent/test_auxiliary_client.py::TestIsPaymentError \
tests/agent/test_auxiliary_client_zai_payment_classification.py \
tests/agent/test_zai_manual_pool_routing.py \
tests/agent/test_zai_e2e_pool.py \
tests/agent/test_zai_e2e_pool_rotation.py -v
Live (consumes Z.AI quota, requires HERMES_RUN_LIVE=1):
pytest tests/agent/test_zai_live.py \
tests/agent/test_zai_live_audit.py \
tests/agent/test_zai_8keys_audit.py -v --runlive
Cross-platform check:
scripts/check-windows-footguns.py --diff upstream/main
== Security ==
- All live test keys read from env var (GLM_TEST_KEYS, GLM_AUDIT_KEYS,
GLM_WORKING_KEY) NEVER from files. Files are safe to commit
publicly.
- All keys masked to 8-char prefix in any printed output.
- No hardcoded credentials, no path leaks, no LAN IPs.
- Privacy scan: 0 secrets, 0 private paths in diff.
Co-authored-by: Hermes triage bot <bot@nousresearch.com>
Refs: NousResearch#61487, NousResearch#61563, PR NousResearch#61492, PR NousResearch#55116, PR NousResearch#58088, PR NousResearch#24915,
PR NousResearch#54643, PR NousResearch#55007, PR NousResearch#61333, PR NousResearch#60753, PR NousResearch#32174,
PR NousResearch#60034, NousResearch#47970, NousResearch#55112, NousResearch#47685
|
Superseded by #62467. Your vision coding-endpoint ordering fix is now handled by the unified resolver Step 4 (provider-specific resolver) which probes coding endpoints before metered for all Z.AI calls, including vision. Credit for identifying the ordering issue. |
|
Thanks for isolating the Z.AI auxiliary-vision routing issue. The current fallback list still begins with metered endpoints at Problems
Suggested changes
Automated hermes-sweeper review. |
Gentle bump (2026-08-12)Issue #55112 is still reproducible on current This PR is still the smallest fix for that root cause. Earlier feedback that is still outstanding (from review / nit):
@Morad37 — if you are busy, I am happy to open a tiny follow-up (or take over the branch with credit) to land (1)+(2) so this can merge cleanly. Maintainers: if you prefer a different vehicle (#62080 bundle / #74071), a quick signal would help avoid further duplicate effort. Thanks! |
Closes #55112
Prepend the coding-plan quota endpoint as the first URL tried for ZAI vision requests, so subscription users hit the quota-billed endpoint before falling through to the metered pay-as-you-go endpoints.