fix: Missing final structured output should provide guidance to model #4654
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Scenario
When using structured output in a recipe and the agent completes its reply without calling the final_output tool. The agent will immediately be triggered to try again until it does call the final_output tool.
Problem
When this scenario occurs we create a user message telling the agent to call the final_output tool. This is yielded from the agent loop so users see it. However the problem was that this message was not being added to the messages vector, therefore the agent never saw this correctional guidance.
This can result in the agent repeating its mistake multiple times or taking entirely different trajectories rather than immediately fixing itself and calling the final_output tool on the next turn.
Fix
Append the correction message to messages vector so agent loop sees it on next turn.
I've also updated the tool prompt a little but cannot tell if has helped.
Steps to reproduce
Non deterministic.
I've found o3 model with a recipe that tells it to output markdown manages to make it not do the tool call maybe 25% of the time which is enough to demonstrate the issue. Attached the recipe, you should notice it fixing itself on the next turn.
output_fail.txt