Skip to content

feat(delegation): opt-in per-task heterogeneous model routing - #54581

Closed
TimberFell wants to merge 1 commit into
NousResearch:mainfrom
TimberFell:feat/delegation-per-task-routing
Closed

feat(delegation): opt-in per-task heterogeneous model routing#54581
TimberFell wants to merge 1 commit into
NousResearch:mainfrom
TimberFell:feat/delegation-per-task-routing

Conversation

@TimberFell

Copy link
Copy Markdown

Summary

Adds an opt-in config flag delegation.per_task_routing (default false)
that lets a single delegate_task batch fan out across heterogeneous
models/providers per task
. Each task in tasks=[...] may carry its own
provider / model / base_url / api_mode and is resolved independently
through the existing _resolve_delegation_credentials() path, so one batch can
mix e.g. a cheap local model for scanning with a strong model for synthesis,
chosen per task by model strength, hardware, or token budget.

When the flag is off, behaviour is identical to today: per-task routing keys
are ignored and the batch-wide delegation.* credentials apply to every child.

Motivation

_build_child_agent() already accepts a full per-child credential bundle
(override_provider / override_base_url / override_api_key /
override_api_mode / model) and re-derives the API transport when a child's
provider differs from the parent's. However, the batch dispatcher resolved
credentials once before the loop and passed the same bundle to every child —
so the engine supported heterogeneous routing but the dispatcher never exercised
it. This PR closes that gap additively and behind a flag.

What changed

tools/delegate_tool.py

  • New helpers: _per_task_routing_enabled() (config > env > default false),
    _task_routing_overrides() (extracts only {provider, model, base_url, api_mode}, string + non-blank), _resolve_task_credentials() (resolves a
    task's routing keys in isolation when enabled; otherwise returns batch-wide
    creds unchanged).
  • Dispatcher loop resolves per-task creds inside the loop. Unresolvable per-task
    provider → tool_error naming the task index (fail-loud).
  • Schema: tasks[].items gains provider / model / base_url / api_mode, each
    documented as honoured only when the flag is on.
  • Tasks description advertises heterogeneous routing only when enabled.

tests/tools/test_delegate.py

  • TestPerTaskRouting (12 cases): flag gate, override extraction, fail-closed
    identity passthrough, isolated resolution, no-key-injection invariant,
    fail-loud resolution.

Security / design invariants

  • Fail-closed: default off; with it off, byte-for-byte identical to prior
    behaviour.
  • API keys never accepted from the model-supplied task dict. Keys resolve
    from the runtime provider system (env / hermes auth) or are inherited from
    the parent. The model selects which backend, never injects a credential.
  • Isolated resolution: per-task keys resolved on their own, so a batch-wide
    endpoint/key cannot bleed into a differently-routed task.
  • Fail-loud: unresolvable per-task provider aborts with a task-identifying
    error rather than silently using the wrong model.

Testing

  • tests/tools/test_delegate.py: 157 passed (145 existing + 12 new).
  • Adjacent suites (test_delegate_toolset_scope, test_delegate_composite_toolsets,
    test_async_delegation): 29 passed.
  • Runtime checks confirm flag-off = identity passthrough + no schema leakage,
    and flag-on exposes the four per-task fields and the routing hint.

Example

# config.yaml
delegation:
  per_task_routing: true
{"tasks": [
  {"goal": "scan logs for the error signature",
   "model": "gemma-4-12B-it-Q4_K_M.gguf", "base_url": "http://10.90.20.17:8761/v1"},
  {"goal": "synthesize root cause", "provider": "anthropic", "model": "claude-opus-4-8"},
  {"goal": "draft remediation notes", "provider": "openrouter", "model": "anthropic/claude-sonnet-4"}
]}

Backwards compatibility

Fully backwards compatible. No behaviour change unless delegation.per_task_routing
is explicitly enabled.

Add delegation.per_task_routing (default false, fail-closed). When enabled,
each task in a delegate_task batch may carry its own provider/model/base_url/
api_mode and is resolved independently via the existing
_resolve_delegation_credentials() path, letting one batch fan out across
heterogeneous backends (e.g. local Gemma worker + OpenRouter Claude worker)
chosen per task by model strength / hardware / token budget.

When the flag is off, per-task routing keys are ignored and batch-wide creds
apply unchanged (identical to prior behavior). API keys are never accepted
from the model-supplied task dict; they resolve from the runtime provider
system or are inherited from the parent. Unresolvable per-task providers
fail loud with a task-identifying tool_error.

Tool schema gains four per-task fields (provider/model/base_url/api_mode),
advertised in the tasks description only when the flag is enabled. Adds
TestPerTaskRouting (12 cases) covering the flag gate, isolated resolution,
the no-key-injection invariant, and fail-loud resolution.
@alt-glitch alt-glitch added type/feature New feature or request tool/delegate Subagent delegation P3 Low — cosmetic, nice to have labels Jun 29, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related: this is part of a saturated per-task delegation model-routing cluster — #16163 (earliest open per-task model/provider overrides) and #34752 (Phase 7 per-call overrides) implement the same capability via different mechanisms. This PR adds an opt-in delegation.per_task_routing flag with independent per-task credential resolution. Not a duplicate of either (distinct flag-gated approach); flagging the cluster so a maintainer can pick the canonical implementation.

@teknium1

Copy link
Copy Markdown
Contributor

Thank you for the careful flag-gating, credential-isolation work, and focused tests.

This is an automated hermes-sweeper review. The requested behavior matches the standing delegation-model-routing policy: Hermes does not accept per-call or per-task provider/model routing through delegate_task. The supported configuration is one provider:model choice for all delegated children via delegation.provider and delegation.model in config.yaml.

  • Current delegate_task intentionally resolves one delegation credential bundle before building batch children (tools/delegate_tool.py:2453, tools/delegate_tool.py:2511-2534).
  • Its live tool instructions explicitly say the subagent model is not selectable per call and point to delegation.provider / delegation.model as the sanctioned configuration (tools/delegate_tool.py:3296).
  • The existing docs describe that same installation-wide configuration path (website/docs/user-guide/configuration.md:1936-1944).

A focused contribution improving the existing config-level delegation routing path would remain welcome, but per-task heterogeneous routing cannot be merged.


Closed as not-planned per standing maintainer policy (delegation-model-routing). This is a design-direction decision, not a code-quality judgment — see the Contribution Rubric in AGENTS.md for what the project is looking for. If you believe this policy was misapplied to your change, comment here and a maintainer will take a look.

@teknium1 teknium1 closed this Jul 15, 2026
@teknium1 teknium1 added the sweeper:not-planned Sweeper: closed per standing maintainer policy (design direction) label Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P3 Low — cosmetic, nice to have sweeper:not-planned Sweeper: closed per standing maintainer policy (design direction) tool/delegate Subagent delegation type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants