feat(delegation): opt-in per-task heterogeneous model routing - #54581
feat(delegation): opt-in per-task heterogeneous model routing#54581TimberFell wants to merge 1 commit into
Conversation
Add delegation.per_task_routing (default false, fail-closed). When enabled, each task in a delegate_task batch may carry its own provider/model/base_url/ api_mode and is resolved independently via the existing _resolve_delegation_credentials() path, letting one batch fan out across heterogeneous backends (e.g. local Gemma worker + OpenRouter Claude worker) chosen per task by model strength / hardware / token budget. When the flag is off, per-task routing keys are ignored and batch-wide creds apply unchanged (identical to prior behavior). API keys are never accepted from the model-supplied task dict; they resolve from the runtime provider system or are inherited from the parent. Unresolvable per-task providers fail loud with a task-identifying tool_error. Tool schema gains four per-task fields (provider/model/base_url/api_mode), advertised in the tasks description only when the flag is enabled. Adds TestPerTaskRouting (12 cases) covering the flag gate, isolated resolution, the no-key-injection invariant, and fail-loud resolution.
Related: this is part of a saturated per-task delegation model-routing cluster — #16163 (earliest open per-task model/provider overrides) and #34752 (Phase 7 per-call overrides) implement the same capability via different mechanisms. This PR adds an opt-in |
|
Thank you for the careful flag-gating, credential-isolation work, and focused tests. This is an automated hermes-sweeper review. The requested behavior matches the standing
A focused contribution improving the existing config-level delegation routing path would remain welcome, but per-task heterogeneous routing cannot be merged. Closed as not-planned per standing maintainer policy ( |
Summary
Adds an opt-in config flag
delegation.per_task_routing(defaultfalse)that lets a single
delegate_taskbatch fan out across heterogeneousmodels/providers per task. Each task in
tasks=[...]may carry its ownprovider/model/base_url/api_modeand is resolved independentlythrough the existing
_resolve_delegation_credentials()path, so one batch canmix e.g. a cheap local model for scanning with a strong model for synthesis,
chosen per task by model strength, hardware, or token budget.
When the flag is off, behaviour is identical to today: per-task routing keys
are ignored and the batch-wide
delegation.*credentials apply to every child.Motivation
_build_child_agent()already accepts a full per-child credential bundle(
override_provider/override_base_url/override_api_key/override_api_mode/model) and re-derives the API transport when a child'sprovider differs from the parent's. However, the batch dispatcher resolved
credentials once before the loop and passed the same bundle to every child —
so the engine supported heterogeneous routing but the dispatcher never exercised
it. This PR closes that gap additively and behind a flag.
What changed
tools/delegate_tool.py_per_task_routing_enabled()(config > env > default false),_task_routing_overrides()(extracts only{provider, model, base_url, api_mode}, string + non-blank),_resolve_task_credentials()(resolves atask's routing keys in isolation when enabled; otherwise returns batch-wide
creds unchanged).
provider →
tool_errornaming the task index (fail-loud).tasks[].itemsgainsprovider / model / base_url / api_mode, eachdocumented as honoured only when the flag is on.
tests/tools/test_delegate.pyTestPerTaskRouting(12 cases): flag gate, override extraction, fail-closedidentity passthrough, isolated resolution, no-key-injection invariant,
fail-loud resolution.
Security / design invariants
behaviour.
from the runtime provider system (env /
hermes auth) or are inherited fromthe parent. The model selects which backend, never injects a credential.
endpoint/key cannot bleed into a differently-routed task.
error rather than silently using the wrong model.
Testing
tests/tools/test_delegate.py: 157 passed (145 existing + 12 new).test_delegate_toolset_scope,test_delegate_composite_toolsets,test_async_delegation): 29 passed.and flag-on exposes the four per-task fields and the routing hint.
Example
{"tasks": [ {"goal": "scan logs for the error signature", "model": "gemma-4-12B-it-Q4_K_M.gguf", "base_url": "http://10.90.20.17:8761/v1"}, {"goal": "synthesize root cause", "provider": "anthropic", "model": "claude-opus-4-8"}, {"goal": "draft remediation notes", "provider": "openrouter", "model": "anthropic/claude-sonnet-4"} ]}Backwards compatibility
Fully backwards compatible. No behaviour change unless
delegation.per_task_routingis explicitly enabled.