Skip to content

feat(agent): apply per-reasoning-model stale-timeout floor in stream + non-stream detectors (#52217) - #52845

Merged
teknium1 merged 1 commit into
mainfrom
salvage/52238-reasoning-floor-wiring
Jun 26, 2026
Merged

feat(agent): apply per-reasoning-model stale-timeout floor in stream + non-stream detectors (#52217)#52845
teknium1 merged 1 commit into
mainfrom
salvage/52238-reasoning-floor-wiring

Conversation

@teknium1

Copy link
Copy Markdown
Contributor

Summary

Known reasoning models now tolerate their multi-minute thinking phase instead of having the upstream cloud gateway idle-kill the socket (BrokenPipeError / RemoteProtocolError) before the first token arrives.

The reasoning-model allowlist (agent/reasoning_timeouts.py) already landed on main via #52795. This PR wires its floor into the two stale detectors that previously used the chat-model defaults (180s stream / 90s non-stream), which are too short for reasoning models.

Salvage of #52238 (fixes #52217), cherry-picked onto current main with @DavidMetcalfe's authorship preserved.

Changes

  • agent/chat_completion_helpers.py — stream stale detector consults get_reasoning_stale_timeout_floor() and applies it as max(stream_timeout, floor).
  • run_agent.py — non-stream detector (_resolved_api_call_stale_timeout_base) returns the reasoning floor with uses_implicit_default=False, so the local-endpoint short-circuit doesn't disable stale detection for reasoning models on a local NIM endpoint.
  • tests/agent/test_reasoning_stale_timeout_floor.py — 51 tests covering the floor table, anchoring, and wiring.

Dropped from the original PR branch (now redundant / stale): the duplicate agent/reasoning_timeouts.py (already on main), and the stale-base reverts of the MoA client path / _extract_landed_file_mutation_paths / moa_config that the original branch carried because it predated those landing on main.

Contract (verified)

  • Never overrides explicit user config — providers.<id>.models.<model>.stale_timeout_seconds, HERMES_API_CALL_STALE_TIMEOUT, etc. resolve first.
  • Never lowers an existing threshold — applied as max(default, floor).
  • Zero effect on chat models — they return None from the allowlist.

Validation

Live E2E through the real AIAgent._resolved_api_call_stale_timeout_base() resolution chain (real imports, temp HERMES_HOME):

Model Resolved stale timeout
nvidia/nemotron-3-ultra-550b-a55b 600s (floor), not implicit default
openai/o3-mini 300s (floor)
openai/gpt-4o 90s implicit default (unchanged)
nemotron + HERMES_API_CALL_STALE_TIMEOUT=45 45s (user config wins over floor)

Stream-detector path verified: max(180 base, 600 floor) == 600; chat model floor None → base unchanged; floor never lowers (max(300, 300) == 300).

Tests: tests/agent/test_reasoning_stale_timeout_floor.py51/51 passing.

Test plan

scripts/run_tests.sh tests/agent/test_reasoning_stale_timeout_floor.py

Infographic

reasoning-model-stale-timeout-floor

@github-actions

github-actions Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

🔎 Lint report: salvage/52238-reasoning-floor-wiring vs origin/main

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 11455 on HEAD, 11399 on base (🆕 +56)

🆕 New issues (12):

Rule Count
invalid-argument-type 11
unresolved-import 1
First entries
tests/agent/test_reasoning_stale_timeout_floor.py:154: [invalid-argument-type] invalid-argument-type: Argument to `AIAgent.__init__` is incorrect: Expected `bool`, found `str | bool`
tests/agent/test_reasoning_stale_timeout_floor.py:154: [invalid-argument-type] invalid-argument-type: Argument to `AIAgent.__init__` is incorrect: Expected `list[str] | None`, found `str | bool`
tests/agent/test_reasoning_stale_timeout_floor.py:154: [invalid-argument-type] invalid-argument-type: Argument to `AIAgent.__init__` is incorrect: Expected `str`, found `str | bool`
tests/agent/test_reasoning_stale_timeout_floor.py:154: [invalid-argument-type] invalid-argument-type: Argument to `AIAgent.__init__` is incorrect: Expected `list[str]`, found `str | bool`
tests/agent/test_reasoning_stale_timeout_floor.py:154: [invalid-argument-type] invalid-argument-type: Argument to `AIAgent.__init__` is incorrect: Expected `int | float`, found `str | bool`
tests/agent/test_reasoning_stale_timeout_floor.py:154: [invalid-argument-type] invalid-argument-type: Argument to `AIAgent.__init__` is incorrect: Expected `list[dict[str, Any]]`, found `str | bool`
tests/agent/test_reasoning_stale_timeout_floor.py:154: [invalid-argument-type] invalid-argument-type: Argument to `AIAgent.__init__` is incorrect: Expected `dict[str, Any]`, found `str | bool`
tests/agent/test_reasoning_stale_timeout_floor.py:154: [invalid-argument-type] invalid-argument-type: Argument to `AIAgent.__init__` is incorrect: Expected `((str, dict[Unknown, Unknown], /) -> None) | None`, found `str | bool`
tests/agent/test_reasoning_stale_timeout_floor.py:38: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`
tests/agent/test_reasoning_stale_timeout_floor.py:154: [invalid-argument-type] invalid-argument-type: Argument to `AIAgent.__init__` is incorrect: Expected `int | float | None`, found `str | bool`
tests/agent/test_reasoning_stale_timeout_floor.py:154: [invalid-argument-type] invalid-argument-type: Argument to `AIAgent.__init__` is incorrect: Expected `int`, found `str | bool`
tests/agent/test_reasoning_stale_timeout_floor.py:154: [invalid-argument-type] invalid-argument-type: Argument to `AIAgent.__init__` is incorrect: Expected `IterationBudget`, found `str | bool`

✅ Fixed issues: none

Unchanged: 6003 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

@alt-glitch alt-glitch added type/feature New feature or request comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists labels Jun 26, 2026
…+ non-stream detectors

Wire get_reasoning_stale_timeout_floor() into both stale detectors so known
reasoning models (Nemotron 3 Ultra, OpenAI o1/o3, Opus 4.x thinking, DeepSeek
R1, Qwen QwQ, Grok reasoning) tolerate multi-minute thinking phases instead of
the upstream gateway idle-killing the socket (BrokenPipeError) before first
token. Applied as max(default, floor) — never overrides explicit user config,
never lowers an existing threshold.

The reasoning_timeouts.py allowlist module already landed on main via #52795,
so this salvage carries only the wiring + tests (the duplicate module and the
stale-base MoA reverts from the original PR branch are dropped).

Salvaged from #52238. Fixes #52217.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists type/feature New feature or request

Projects

None yet

3 participants