feat: add Volcengine Ark (火山引擎) as built-in provider

[david-bowiegxw]

Summary

Adds Volcengine Ark (火山引擎) — ByteDance's unified AI model platform — as a first-class built-in provider. Users can discover and configure it directly from Settings → Providers.

Settings → Providers → "Volcengine Ark (火山引擎)" → paste ARK_API_KEY → done

What's included

• plugins/model-providers/volcengine-ark/__init__.py — ProviderProfile subclass + plugin.yaml manifest
  • Anthropic-compatible /api/coding endpoint via api_mode="anthropic_messages"
  • Overrides fetch_models() to return the maintained text-model list (11 models)
  • Aliases: ark, volcengine, volcano, bytedance
  • Display name includes Chinese (火山引擎) for discoverability
• agent/agent_runtime_helpers.py — prompt-caching enablement so adopting Ark gives complete support (routing + caching) in one change
  • Ark serves non-Claude model families, so the is_claude gate in anthropic_prompt_cache_policy() excluded it → 0% cache hits, re-billing the full prefix every turn
  • Adds an explicit Ark allowlist branch (provider id / host) returning native Anthropic layout — same class and fix as MiniMax's [Bug]: Prompt caching silently disabled for MiniMax's own models (MiniMax-M2.7 etc.) on anthropic_messages transport #17332
  • Covered by TestVolcengineArkAnthropicWire (4 cases) in tests/run_agent/test_anthropic_prompt_cache_policy.py

Validated end-to-end (real measurements, not just code)

I ran this against a live Ark subscription through the actual Hermes CLI, not only as a unit. Sharing the results because they're useful feedback on Ark's behavior regardless of whether this exact PR lands.

Discovery + registration: loaded under Hermes' runtime, the provider is discovered as volcengine-ark, VolcengineArkProfile instantiates with the right fields, and fetch_models() returns the 11 hardcoded models. ✅

Prompt caching — measured per model. Ark's /api/coding endpoint speaks the Anthropic Messages protocol, so I sent an identical cache_control-marked prefix twice per model and read usage.cache_read_input_tokens, sweeping prefix sizes 4k→64k tokens (2026-06-26):

Model	Caching on /api/coding
deepseek-v4-pro	✅ caches ≥ ~4k
glm-5.2	✅ caches ≥ ~4k
doubao-seed-2.0-lite	✅ caches ≥ ~4k (flaky hit rate)
kimi-k2.7-code	✅ caches ≥ ~4k
minimax-m3	✅ caches ≥ ~4k
doubao-seed-2.0-mini	✅ caches ≥ ~8k
doubao-seed-2.0-code	✅ caches ≥ ~8k
doubao-seed-2.0-pro	✅ caches ≥ ~48k (high threshold)
deepseek-v4-flash	❌ no cache (0 across 4k–64k)
kimi-k2.6	❌ no cache (0 across 4k–64k)
minimax-m2.7	❌ no cache (0 across 4k–64k)

Two findings worth knowing:

• Caching is per-model and prefix-size-dependent (per-model minimum 4k–48k tokens) — 8/11 cache, 3 never did across 12 samples each.
• Hits are probabilistic, not deterministic. A supported model can miss on any given call; doubao-seed-2.0-lite returned 0 at one size and cached at others. This matches Ark's own docs (隐式缓存「不保证命中,分布式路由影响命中概率」) and diverges from Anthropic's deterministic cache_control guarantee — worth flagging for anyone relying on cache behavior here. Note this is Ark's /api/coding surface; Ark also has separate /api/v3 implicit and /api/v3/context explicit caching with different model support.

Aux model: default_aux_model was deepseek-v4-flash, which never caches — so high-frequency background tasks (titles, compression) re-bill the full prefix every call. Switched to doubao-seed-2.0-lite, the cheapest model that caches and at the lowest prefix threshold.

This PR wires the emission, not just measures it: the anthropic_prompt_cache_policy() Ark branch (see "What's included") makes Hermes actually send the cache_control markers. Without it Ark's non-Claude models hit the is_claude gate and get 0% caching even when routed correctly.

Prerequisite: resolver fix in #53055 (separate PR)

In the interest of honest feedback: as a pure 2-file plugin this provider 404s at runtime without a one-time framework fix, which is now in its own PR — #53055 (issue #53054). Root cause is a real gap in the runtime resolver, not in this plugin:

• _resolve_runtime_from_pool_entry (hermes_cli/runtime_provider.py) derives api_mode for a selected provider from config or from _detect_api_mode_for_url(base_url). The ProviderProfile.api_mode declared by a plugin is dropped at the bridge — ProviderConfig (hermes_cli/auth.py) had no api_mode field.
• _detect_api_mode_for_url recognizes anthropic_messages only for /anthropic paths and api.kimi.com + /coding. Ark's ark.cn-beijing.volces.com + /api/coding matches neither, so it silently falls back to chat_completions → posts OpenAI-style /chat/completions to Ark's Anthropic-only endpoint → nginx 404.

Note this is not "the first anthropic_messages plugin" — MiniMax is one too. MiniMax only avoids the bug by coincidence: its base_url ends in /anthropic, so the URL heuristic re-derives the right mode and its declared api_mode field is never actually read. Ark's /api/coding endpoint isn't URL-self-describing, so it's the first to expose that the declaration is dropped. #53055 makes the declaration authoritative for any such plugin (additive, non-regressing, with tests). This provider PR depends on #53055.

Design decisions

1. anthropic_messages protocol — Ark's /api/coding endpoint speaks the Anthropic Messages API and supports prompt caching (validated above); the OpenAI-protocol path on Ark exposes the Responses API, which Hermes doesn't use. A ProviderProfile declares a single api_mode, so we default to the one that gives Hermes caching.
2. One unified subscription — Ark's Agent Plan and Coding Plan share the same base_url, API key, protocol, and model IDs; they differ only in Volcengine-side billing/quota, which the client can't and shouldn't detect. So this is a single provider entry rather than two near-identical profiles (unlike alibaba vs alibaba-coding-plan, which hit different endpoints).
3. Hardcoded model list — Ark's /models endpoint returns a large stale catalog rather than the user's active models, so fetch_models() is overridden to return the current maintained list (update when Ark's lineup changes).
4. Minimal core footprint — the provider itself is pure plugin (no tips.py or unrelated edits). The required api_mode resolver wiring lives in its own PR (fix(providers): gate plugin api_mode by endpoint (#53054) #53055), not here, so this PR stays a self-contained plugin.

Relation to prior attempts

Volcengine Ark has been requested repeatedly and attempted several times, but no prior PR has landed. This one differs by being the only single-focus implementation built on the current pluggable plugins/model-providers/ architecture (which postdates those PRs):

• feat(providers): add Volcengine and BytePlus support #8952 (Apr 2026) — add Volcengine and BytePlus support: ~600 LOC editing core (hermes_cli/auth.py, main.py, gateway/run.py, model_metadata.py); predates the pluggable-provider refactor; bundles BytePlus.
• feat(providers): add Volcengine and BytePlus support #14070 — add Volcengine and BytePlus support: same bundled, pre-plugin approach.
• Add Volcengine and BytePlus providers #32549 (May 2026) — Add Volcengine and BytePlus providers: ~800 LOC editing hermes_cli/ core (ark_providers.py, model_setup_flows.py, providers.py); does not use the plugins/model-providers/ mechanism; bundles BytePlus.

By contrast this PR is a self-contained ProviderProfile plugin (2 files) scoped to Volcengine Ark only, plus the small resolver wiring noted above. Happy to consolidate the older PRs into this one if maintainers prefer.

Related issues

• Closes feat: add Volcengine (火山引擎) as built-in provider #29331 — feat: add Volcengine (火山引擎) as built-in provider
• Closes Feature Request: Add official ByteDance / BytePlus ModelArk provider #40195 — Feature Request: Add official ByteDance / BytePlus ModelArk provider
• Closes Feature Request: 新增豆包 (Doubao/Volcano Engine) 作为原生 Provider #51319 — Feature Request: 新增豆包 (Doubao/Volcano Engine) 作为原生 Provider

[david-bowiegxw]

@teknium1 — Ark (火山引擎 / Doubao) is still one of the few major providers not built-in, and I've kept this current. Two notes after 0.18.0:

1. Still needed. ProviderConfig in hermes_cli/auth.py still has no api_mode field, so a plugin's declared ProviderProfile.api_mode is dropped at the bridge. Any anthropic_messages plugin whose endpoint isn't URL-self-describing — Ark's is …/api/coding, not /anthropic — still degrades to chat_completions and 404s on every request. MiniMax only escapes by coincidence (its base_url ends in /anthropic). Issue #53054, fix #53055.

2. Happy to make this trivial to take. I can fold the provider (#52836) + the small resolver fix (#53055) into a single PR rebased onto current main — same shape as the Vertex salvage in #56363 — so there's one thing to review. It closes the three standing requests (#29331, #40195, #51319) and consolidates the earlier bundled attempts (#8952, #14070, #32549) into one focused, plugin-native provider.

Ark's /api/coding speaks the Anthropic Messages protocol and supports prompt caching (measured per-model — 8/11 models cache), so with the resolver fix it's complete routing + caching in one. Want me to open that consolidated PR?

[fuleinist]

+1 on this

[teknium1]

Thanks for the focused provider implementation and the detailed endpoint/cache investigation.

This automated hermes-sweeper review is closing this as not planned under the in-tree-provider-integration policy.

• The PR adds a Volcengine-specific provider directory at plugins/model-providers/volcengine-ark/__init__.py.
• AGENTS.md:797-813 requires new third-party product integrations to ship as standalone plugins rather than in the repository tree; this is a coupling and maintenance decision, not a judgment on the implementation quality.
• Hermes already discovers standalone provider plugins from $HERMES_HOME/plugins/model-providers/ (providers/__init__.py:140-171), and the picker auto-extends from registered profiles (hermes_cli/models.py:1092-1107).
• The related requests feat: add Volcengine (火山引擎) as built-in provider #29331 and Feature Request: Add official ByteDance / BytePlus ModelArk provider #40195 were closed under the same standing policy.

Please publish Ark as a standalone model-provider plugin repository for installation under ~/.hermes/plugins/ (or via a pip entry point), and it can be promoted in #plugins-skills-and-skins.

---

Closed as not planned per standing maintainer policy (in-tree-provider-integration). This is an automated hermes-sweeper review.

---

Closed as not-planned per standing maintainer policy (in-tree-provider-integration). This is a design-direction decision, not a code-quality judgment — see the Contribution Rubric in AGENTS.md for what the project is looking for. If you believe this policy was misapplied to your change, comment here and a maintainer will take a look.

[david-bowiegxw]

@teknium1 — replying under the escalation path in the close notice ("If you believe this policy was misapplied to your change, comment here and a maintainer will take a look"). I believe in-tree-provider-integration was misapplied here:

1. The policy's own text doesn't cover model providers. The June 2026 policy (added in #54001, 2026-06-27) enumerates "observability/metrics backends, vendor SaaS connectors, analytics dashboards, paid-service tie-ins" under plugins/. The very next section of AGENTS.md documents plugins/model-providers/<name>/ as the normal home for inference backends: "Every inference backend (openrouter, anthropic, gmi, deepseek, nvidia, …) ships as a plugin here."

2. Post-policy merge practice confirms that reading. After the policy landed, two new third-party provider directories were merged in-tree: Vertex AI (#56363, 2026-07-01) and Fireworks AI (c97d9a4, 2026-07-08). Fireworks is a third-party commercial inference vendor — exactly this PR's category.

3. The cited precedent is the same sweeper batch, not standing practice. #29331 (closed 07-13) and #40195 (closed 07-14) were closed by the same automated review in the same three-day window as this PR (07-15). The third standing request, #51319, is still open.

4. Ark isn't a niche backend. Volcengine Ark holds 49.5% of China's public-cloud MaaS market (IDC, 2025 full-year — vs. Alibaba Cloud 27%, Baidu 17%), serving 180T+ tokens/day as of June 2026 (IDC via 36kr). Like OpenRouter, it fronts third-party models (DeepSeek, GLM, Kimi, MiniMax) behind one API. The tree already ships its direct peers: alibaba, alibaba-coding-plan, deepseek, kimi-coding, minimax, stepfun, xiaomi, zai, qwen-oauth.

5. The policy's stated rationale — maintenance load — actually cuts the other way here. Users already run Ark through custom_providers and generate a steady bug stream against that path: #63792 (opened two days before this close), #51773, #17199, #12988 all open, plus #25354 / #33007 — six issues since April, on top of the four standing requests (#29331, #40195, #51319, #44167). That load exists either way; a built-in provider declaring the correct api_mode and a curated model list is what removes it.

If the considered call going forward is that new model providers should also ship standalone, I'll happily publish it that way — but that would be a new policy beyond the current AGENTS.md text and recent merge practice, so I'd ask for a human look before this close stands.

Meanwhile I'm addressing the review on #53055 (declared-profile fallback across all three API-key resolver paths + a full discovery→registration→resolve regression test), which this provider depends on for correct anthropic_messages routing.

Could this be reopened pending that look?

[Icather]

I'm in strong agreement with @david-bowiegxw's escalation — the in-tree-provider-integration policy was misapplied to a model provider here, and I can add a first-hand data point.

On the policy itself: AGENTS.md's plugins/ section lists observability/metrics backends, vendor SaaS connectors, analytics dashboards, and paid-service tie-ins — it does not name plugins/model-providers/<name>/, which the very next section explicitly documents as the normal home for inference backends ("Every inference backend … ships as a plugin here"). The post-policy in-tree merges of Vertex AI (#56363) and Fireworks AI only reinforce that the rule was never meant to cover inference backends. Auto-closing a model provider under a policy that doesn't mention model providers is the exact contradiction the escalation flags.

For what it's worth, I've hit the same wall maintaining a provider in this tree. I work on the Z.AI / GLM (智谱) provider, and two of those PRs were closed without merging:

• fix(models): merge Z.AI curated models with live API results #46623 — merge Z.AI curated models with live API results — closed by a maintainer on a "split surfaces" rationale (glm-5.2 is probed as a GLM Coding Plan model under /api/coding/paas/v4).
• feat(models): add GLM vision Flash models to Z.AI curated list #46562 — add GLM vision Flash models to Z.AI curated list — I closed it myself as superseded by fix(zai): resolve model list for China API users (endpoint-aware) #46724 (fix(zai): resolve model list for China API users, endpoint-aware), which is still open today.

So even an already-peered vendor's models stall on the in-tree path regardless of code quality. Reopening this (or at minimum carving model-providers out of the sweeper's in-tree-provider-integration rule — given alibaba, deepseek, kimi-coding, minimax, stepfun, xiaomi, zai, qwen-oauth already ship in-tree) would be the consistent fix.

[Icather]

Adding a documentation follow-up to my comment above, since I went and read the actual guidance.

I pulled the current AGENTS.md and CONTRIBUTING.md. They back up the escalation with primary-source text — and reveal an internal contradiction the sweeper ignored.

The June 2026 "third-party-product" policy (AGENTS.md 797–813, mirrored at CONTRIBUTING.md 88–101) lists exactly:

observability/metrics backends, vendor SaaS connectors, analytics dashboards, paid-service tie-ins

It does not name model providers. Immediately below it, AGENTS.md 815+ "### Model-provider plugins (plugins/model-providers/<name>/)" states:

Every inference backend (openrouter, anthropic, gmi, deepseek, nvidia, …) ships as a plugin here.

and its scan order puts Bundled: <repo>/plugins/model-providers/<name>/ as item 1 — i.e. in-tree is an explicitly documented, supported home for inference backends, not a forbidden one.

So the doc simultaneously says (a) "third-party products don't land in-tree" and (b) "every inference backend — including third-party ones like deepseek, minimax, kimi — ships as a plugin here, bundled in-tree." Volcengine Ark is both a "third-party product" and an "inference backend," and the sweeper applied only (a). That's the misapplication in one screenshot: the policy paragraph never covered model providers, while the model-provider paragraph three lines down explicitly blesses the in-tree bundled form.

This isn't a judgment on code quality (the close notice says as much) — it's that the automation read the June policy paragraph and skipped the model-provider paragraph right beneath it. Carving model-providers out of the sweeper rule (or just letting the model-provider paragraph govern inference backends) resolves it without any new policy wording.

[david-bowiegxw]

Small maintenance update on this branch (3ef44cb1e): Volcengine sent an account notice today (2026-07-22) that the legacy bare Doubao-Seed-Code model ID retires on 2026-08-05 in favor of the Seed 2.0 lineup.

Re-verified all 11 hardcoded model IDs live against /api/coding — all still return 200. This provider already used the Seed 2.0 names (doubao-seed-2.0-pro/lite/mini/code) from the original pass, so the migration needed no code change, just a docstring note recording the re-verification.

Flagging mainly as a liveness signal: the vendor's model lineup keeps shifting (this is the second such churn since the PR opened), which is exactly the kind of upkeep an in-tree provider absorbs on behalf of users rather than leaving to a fork nobody watches. Still hoping for a look at the reopen request above whenever you get a chance.