.Net: ADR for Automatic Tool Calling Controls #4686
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gitri-ms
requested review from
stephentoub,
dmytrostruk,
alliscode and
markwallace-microsoft
stephentoub
reviewed
Jan 23, 2024
stephentoub
reviewed
Jan 23, 2024
gitri-ms
added
PR: ready for review
crickman
added
.NET
github-actions
bot
changed the title
4 tasks
|
@dmytrostruk FYI - Historical |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
ADR proposing additional controls for automatic tool calling
Task 4300 is underway to use automatic tool calling in the
FunctionCallingStepwisePlanner. This will allow for more robust and consistent handling of tool calls and minimize duplication of code.Currently, as part of the planner configuration, the caller can set the
MaxIterationsfield to limit the number of steps (and therefore, calls to the model) utilized by the planner. This enables the caller to control things such as cost, time, etc., spent by the planner.The current implementation of automatic tool calling allows for multiple consecutive tool calls to be requested/invoked by the model for a single request. However, the limit on how many consecutive tool calls can be performed per request is not exposed to the caller. Therefore, the caller cannot control or even measure after the fact how many model roundtrips occur for a single request. This control is necessary for the planner to accurately enforce the
MaxIterationslimit.Code snippets are provided in the ADR, but you can see the full changes in #4687 (options 1 & 2) and #4688 (option 3)