Skip to content

feat(plugins): adaptive-reasoning — context-aware effort routing with closed-loop feedback - #88522

Open
zons-zhaozhy wants to merge 2 commits into
NousResearch:mainfrom
zons-zhaozhy:plugin/adaptive-reasoning
Open

feat(plugins): adaptive-reasoning — context-aware effort routing with closed-loop feedback#88522
zons-zhaozhy wants to merge 2 commits into
NousResearch:mainfrom
zons-zhaozhy:plugin/adaptive-reasoning

Conversation

@zons-zhaozhy

@zons-zhaozhy zons-zhaozhy commented Aug 17, 2026

Copy link
Copy Markdown

What does this PR do?

Adaptive reasoning-effort routing as a standalone plugin — zero core changes. Addresses #40306 / #74725 as the edge-capability alternative: no new env vars, no core tool, no schema changes.

Routes per-turn reasoning effort through the existing llm_request middleware:

  • Context-aware classifier (v3): text prior + conversation context. A terse "ok" mid-task inherits session difficulty instead of collapsing to minimal; cold acknowledgements stay cheap. Bilingual (EN/CJK) keyword signals with correct word-boundary handling (no \b on CJK).
  • Closed-loop feedback (v4): an llm_execution observer records reasoning_tokens + finish_reason per turn; monotone feedback rules adjust the next turn — length truncation → medium (never down), rt=0 + clean stop → low, rt>=600 + clean stop → high. A staleness guard (turn_id) rejects cross-turn leakage.
  • Provider-native wire scale: the middleware fires after the transport, so top-level reasoning_effort is rewritten ONLY with the value the provider profile itself emits (_translate_effort re-runs the same profile mapping). Hermes-only levels never leak onto the wire.
  • Off-switch: plugins.entries.adaptive-reasoning.enabled: false in config.yaml.

Related Issue

Addresses #40306 and its duplicate #74725 (also #13663). Not using Closes — those issues also cover per-turn none and an integrated reasoning display, which are outside this PR's scope.

Related prior art: open #82578 takes a core approach (19 files: restoration, delegation semantics, Desktop UI); open #61410 also uses deterministic local heuristics; closed #58305 explored a classifier-model call. This PR differs on placement: the same per-turn routing delivered entirely at the edge via llm_request middleware, so the core stays narrow and the feature is opt-in per install.

Type of Change

  • ✨ New feature (non-breaking change that adds functionality)

Changes Made

  • plugins/adaptive-reasoning/plugin.yaml — manifest (middleware: llm_request; hooks: post_tool_call, llm_execution)
  • plugins/adaptive-reasoning/__init__.py — classifier, feedback controller, profile-scale translation, middleware
  • tests/plugins/test_adaptive_reasoning.py — 42 tests

How to Test

  1. Enable the plugin: plugins: {entries: {adaptive-reasoning: {enabled: true}}} in config.yaml
  2. Run scripts/run_tests.sh tests/plugins/test_adaptive_reasoning.py — 42 pass
  3. Live: send a trivial "ok" mid-task → effort drops next turn without losing session context; send a complex debugging request → effort rises; truncated responses raise effort monotonically

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits
  • I searched for existing PRs to make sure this isn't a duplicate (prior art referenced above — different placement: plugin vs core)
  • My PR contains only changes related to this feature
  • I've run the plugin test suite (tests/plugins/: 1587 passed, 1 pre-existing a2a order-sensitive failure reproducible on pure upstream main, unrelated to this PR) — full-suite run pending CI
  • I've added tests for my changes
  • I've tested on my platform: macOS 26.2 (arm64)

Documentation & Housekeeping

  • Documentation update — N/A (plugin is self-documenting via plugin.yaml description; will add docs if maintainers prefer)
  • cli-config.yaml.example — N/A (no new config keys outside plugins.entries)
  • CONTRIBUTING.md/AGENTS.md — N/A (no architecture change)
  • Cross-platform impact — N/A (pure Python, stdlib only)
  • Tool descriptions/schemas — N/A (no tool changes)

Evidence

  • Test expectations derived from the design contract in the issue discussion, not from the implementation.
  • A/B measured on glm-5.3 (coding endpoint) with a 20-item / 8-category benchmark: adaptive routing cut reasoning-token spend on trivial turns while keeping (or raising) effort on complex ones. Methodology: a message's sufficiency level is the LOWEST effort tier that still answers it correctly.
  • Feedback rules refuted an earlier dual-threshold design: measured data showed hard+high collides with rt=1999/length truncation (same region as starvation), causing self-oscillation on 3-tier models. The monotone rules above are the measured-stable replacement.

… closed-loop feedback

Addresses NousResearch#40306 / NousResearch#74725 as a zero-core-change plugin (issues NousResearch#74725, NousResearch#13663).

Routes per-turn reasoning effort through the existing llm_request middleware:
- v3 classifier: text prior + conversation context (request.messages work
  depth) — brevity mid-task inherits session difficulty, cold acks stay minimal
- v4 closed loop: llm_execution observer records the previous call's real
  reasoning_tokens/finish_reason; mid-loop feedback corrects the prior
  (starvation-first, monotone — measured glm-5.3 distribution showed
  hard+high always hits rt~2000/finish=length, so escalate-on-rt rules
  oscillate on 3-tier native scales)
- provider-native scales: reuses get_provider_profile().build_api_kwargs_extras()
  so wire values always match what the transport itself would send
- floor/ceiling clamps via user config; tool-error rescue path unchanged

Measured evidence (glm-5.3, coding endpoint): in-context brevity burns
1022 rt vs isolated 151; hard-task rt grid refuted the two-threshold
feedback design; observer handles all three response shapes (pydantic,
relay dict, SimpleNamespace). 42 unit tests.
… profiles

The three provider-scale tests were written against a fork-local zai
profile that clamps glm-5.3 reasoning_effort to {low, high, max}.
Upstream zai has no glm-5.3 handling (glm-5.2 only: {high, max}), so the
tests failed on the PR target repo.

Re-anchor to upstream-real behavior:
- kimi-coding _K3_EFFORT_MAP (low/high/max) for the three-tier cases
- zai glm-5.2 two-tier collapse for the no-op case (already-high request
  yields None — assert the invariant, not the wrapper shape)

The invariant under test is unchanged: top-level reasoning_effort must
only ever hold values the provider profile itself would emit.
@alt-glitch alt-glitch added type/feature New feature or request comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have labels Aug 17, 2026
@Enough1122

Copy link
Copy Markdown
Contributor

AI code review — automated review for reference; please use your judgment.

Review of "feat(plugins): adaptive-reasoning" (sampled: classifier/config/middleware wiring in the 777-line module plus the test file; the closed-loop llm_execution observer was reviewed at its contract seams). Standout plugin engineering: the docstring reports MEASURED A/B data instead of vibes, states its own blind spot (short-and-hard prompts route low; real accuracy loss observed) alongside built-in mitigations (tool-error escalation, floor=medium escape hatch), keeps the prompt-cache prefix untouched by rewriting only the request-scoped effort field, handles the CJK \b-boundary pitfall explicitly, adds a closed-loop observer with an anti-oscillation guard for 3-tier scales, and forgets stale turn counters. Config caching on (mtime_ns, size) matches core conventions; floor>ceiling self-corrects. Suggestions:

  1. tools/delegate_tool-adjacent concern — _session_work_depth reads full message history per call on every API call of every turn when enabled; for long sessions consider bounding the scan to the tail window that actually feeds classification.

  2. nit — session tool-error counters are keyed (session_id, turn_id) in a plain dict; _forget_stale_turns bounds staleness, but a hard cap on map size would future-proof against id-churn.

  3. nit — EFFORT_SCALE is duplicated from hermes_constants.VALID_REASONING_EFFORTS "mirrored"; import it so scale additions can't fork the two lists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants