fix(combo): trust finish_reason stop over the reasoning-ratio heuristic (carries #14980) - #15020
Merged
Merged
Conversation
The 90% reasoning-ratio check rejects responses where reasoning fills the budget and visible content is empty. That is correct for models that optionally think and run out of budget mid-reasoning, but it misfires on models that cannot disable thinking: Claude Opus 4.7+ and GLM-5.3 both spend most of their output on reasoning and still finish normally with finish_reason "stop". OpenAI, Anthropic, and Z.ai all document "stop" as a natural end of turn and "length"/"max_tokens" as truncation. Trust the stop signal and keep the ratio check only as the fallback when the provider reports no finish_reason. Signed-off-by: Minxi Hou <houminxi@gmail.com> (cherry picked from commit 4a03936)
2 tasks done
diegosouzapw
added a commit
that referenced
this pull request
Sep 29, 2026
…d release date - Every one of the 1972 cycle commits is covered by a bullet (reconcile-changelog vs origin/release/v3.8.50); fragments folded under [3.8.51] with the PR link and the author of the commit that added them. - Re-land credit: #15019/#15020/#15119/#15120/#15121 credit @HouMinXi; #12383 credits its author @kareem-jalal alongside the original credit. - Contributors hall regenerated (301 external contributors). - [3.8.51] dated 2026-09-29 in the root CHANGELOG and the 66 i18n mirrors.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Carries the one commit of #14980 by @HouMinXi that matches its title, cherry-picked unchanged (authorship preserved) onto the current
release/v3.8.51. #14980 also carried ~20 unrelated deploy-branch commits (grok-bot tool bridge executor, Opus 5.5 defaults, rebaselines), so only4a03936lands here.What changes
The combo quality gate now trusts an explicit
finish_reason: "stop"over the reasoning-ratio heuristic: a response that stopped normally with reasoning but no visible text is accepted instead of failing the combo over. This intentionally changes the contract pinned by the 2026-09-01 client audit (finish_reason:stop + empty content + reasoning → ratio heuristic still applies); the maintainer approved the policy change for this wave.Validation
tests/unit/combo-quality-validator-reasoning.test.ts+tests/unit/validate-response-quality.test.ts: 33/33 on the current tip.Supersedes #14980. Thank you @HouMinXi!
Co-authored-by: Minxi Hou houminxi@gmail.com