fix(stream): the context-scaled W2 budget was dead code on the runtime path

[OmarB97]

Why

A live 79.8k-token turn on deepseek-v4-flash-w2 logged this:

Local dflash has produced no first chunk for 240s (budget 240s)
but the backend reports an ACTIVE generation — extending.

The extension is #278 working. But budget 240s is wrong — the cost model added in #278 says ~679s at that context. Reading why exposed two gaps.

1. The runtime resolver never calls the cost model

resolve_dflash_local_first_chunk_timeout — the function the hot path actually uses — falls straight through to the stale ladder:

# without a config value or a legacy env override:
return resolved_stream_stale_timeout   # >50k -> 240s, >100k -> 300s

_dflash_context_timeout_default (cold-start allowance + per-1k prefill) is only reached by the legacy _dflash_local_first_chunk_timeout helper, which nothing on the hot path calls. So the budget shipped in #278 was dead code, and the step function — calibrated against no measurement of anything — kept winning.

Only the progress-probe half of #278 was ever live. That is why W2 still works: the watchdog extends on the gate's liveness signal, not on a correct budget. It was being carried by the right mechanism while the wrong number sat underneath it.

2. ai-router was not a recognised managed W2 route

provider in {"taro", "meshboard-qualified-local"}

The desktop routes through ai-router — the ko-nas fleet router (:9081), which proxies to taro's ai-gate and on to the same llama-swap/vLLM. Same physical lane, same slow cold prefill. It simply was not listed, so the path the desktop actually uses fell through to the generic DFlash ladder and never got even the 360s W2 floor.

What changed

• ai-router added to _MANAGED_W2_PROVIDERS.
• The runtime resolver now takes max(existing floors, context-scaled budget) on the managed W2 lane only.
• Budget rounded to whole seconds (a deadline of 360.036s is noise).

Managed W2 lane, no config/env override:

tokens	old	new
50,000	180	560
79,800	240	679
90,700	240	723
131,072	300	884

Generic LAN DFlash routes: unchanged (180 / 240 / 300).

How to review

1. resolve_dflash_local_first_chunk_timeout — note the early return for non-managed routes. That is the blast-radius control, and it is the thing to argue with.
2. _MANAGED_W2_PROVIDERS — and the comment explaining why this stays an allowlist.

The tempting fix for (2) is "any local endpoint serving a W2-named model". I tried that and it was wrong: it broke test_managed_local_w2_timeout_floor_is_route_and_model_specific, which deliberately asserts that an arbitrary LAN provider serving a W2-named model does not get the exception. That test is right and the broadening was wrong, so the allowlist stays.

The same reasoning bounds the budget: it is calibrated on one setup (W2 behind llama-swap on taro). Applying its cold-start allowance to an arbitrary local DFlash provider would inflate that provider's deadline on the strength of a measurement that says nothing about it. So generic routes keep the existing policy byte-for-byte.

Evidence

$ python3 -m pytest tests/agent/test_local_stream_timeout.py -q
102 passed

$ python3 -m pytest tests/agent/ -q -k "stream or timeout or interrupt or chat_completion or watchdog"
516 passed, 5079 deselected

Mutation checks — neither fix is vacuous:

# drop "ai-router" from the allowlist:
FAILED TestRuntimeFirstChunkBudgetIsReachable::test_runtime_budget_scales_with_context_instead_of_the_ladder

# make the resolver fall back to the stale ladder again:
FAILED TestRuntimeFirstChunkBudgetIsReachable::test_runtime_budget_scales_with_context_instead_of_the_ladder

Backwards compatibility

• Explicit overrides still win. A config.yaml stale_timeout_seconds or HERMES_DFLASH_FIRST_CHUNK_TIMEOUT returns earlier in the function and is untouched — an operator who pinned a number still gets exactly that number. Pinned by test_explicit_env_override_still_wins.
• The budget can only widen, never narrow. max(), not min(). Pinned by test_runtime_budget_never_shrinks_below_the_old_floors.
• Non-managed routes are byte-for-byte unchanged, including a disabled watchdog (float("inf")).

Risks / gaps

• The provider allowlist will drift again as the fleet is re-wired; the next new front-end for W2 will need adding. That is the deliberate cost of route-specificity — the failure mode is a too-tight deadline on a new route, which the liveness probe already compensates for, not a wrong one. Accepted tradeoff — no work item needed.
• The per_1k constant is still calibrated from a noisy measurement (prefix caching makes TTFT vary by an order of magnitude, as documented on the constant). It carries generous headroom and is only a fallback for backends with no progress signal. Accepted risk — no work item needed.