fix(agent): preserve Qwen3.6 / vLLM preserve_thinking history (#56004) - #56019
Open
Tranquil-Flow wants to merge 1 commit into
Open
Tranquil-Flow wants to merge 1 commit into
Tranquil-Flow wants to merge 1 commit into
Conversation
Contributor
Related: fixes #56004; same reasoning-replay family as #20577 and its open fix PR #20607. This PR and #20607 are competing fixes with different mechanisms: #20607 re-embeds |
19 tasks
Collaborator
|
Thanks for isolating the normal replay stripping path; current main still removes Problems
Suggested changes
Automated hermes-sweeper review. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was fixed
Qwen3.6 (and similar self-hosted models served via vLLM with the
preserve_thinkingchat-template flag) loses its prior-turn chain-of-thought on every replay. The agent loop strips bothreasoning_content(strict-provider gate incopy_reasoning_content_for_api) andreasoning(unconditional pop inconversation_loop.py), so vLLM ≥0.20 has nothing to render under thepreserve_thinkingtemplate.Root cause
Two client-side strip paths in
agent/agent_runtime_helpers.pyandagent/conversation_loop.pyunconditionally remove the reasoning fields that vLLM/Qwen3.6 need to re-render prior thinking. vLLM ≥0.20 also dropsreasoning_contenton incoming assistant messages (vllm-project/vllm#38488), so the existing DeepSeek/Kimi/MiMoreasoning_contentecho-back path is incompatible regardless.Fix
Added a
_preserves_thinking_history()predicate onAIAgentthat detects Qwen3.6-style endpoints by model name (qwen3.6/qwen-3.6, conservative word-boundary regex — does NOT match plainqwen3per the model card). Result cached on the AIAgent instance keyed by(provider, model, base_url)— same invalidation contract as the adjacent_thinking_pad_cache, so allswitch_model/_try_activate_fallbackpaths automatically invalidate it.Three touch points now gated on the predicate:
copy_reasoning_content_for_api— skip the strict-providerreasoning_contentstrip on preserves-thinking-history providers.conversation_loop.py— skip the unconditionalreasoningpop (keep the field on the outgoing message for vLLM to render).reapply_reasoning_echo_for_provider— stripreasoningon the way out ONLY for strict providers (handles the strict-fallback case where Qwen3.6 → Mistral would 422 on the unknown field).Verification
AttributeError: 'AIAgent' object has no attribute '_preserves_thinking_history'(real production-path coverage)._needs_thinking_reasoning_pad).copy_reasoning_content_for_apiandreapply_reasoning_echo_for_providerin both directions (preserve-history provider AND strict-fallback).Files changed (1 focused commit, 4 files, +289/-19)
agent/agent_runtime_helpers.py— gatedreasoning_contentstrip + newreasoningstrip branch inreapply_reasoning_echo_for_provider(handles strict-fallback).agent/conversation_loop.py— gated the unconditionalreasoningpop.run_agent.py— new_preserves_thinking_history()predicate on AIAgent.tests/run_agent/test_qwen_preserve_thinking_56004.py— 11 new regression tests.Notes
custom_providers[*].preserve_thinking: true) is documented in the docstring as a more robust follow-up — today this covers the reported Qwen3.6 / vLLM case correctly.extra_body: chat_template_kwargs: {preserve_thinking: true}already works server-side today; this PR completes the client-side loop.Auto-published by Moonsong via Path B automated pipeline.