feat: conversation api toolcalling support documentation #4769
Conversation
Signed-off-by: Samantha Coyle <[email protected]>
| | `temperature` | A float value to control the temperature of the model. Used to optimize for consistency and creativity. Optional | | ||
| | `parameters` | Parameters for all custom fields. Optional | | ||
| | `metadata` | [Metadata](#metadata) passed to conversation components. Optional | | ||
| | `scrub_pii` | A boolean value to enable obfuscation of sensitive information returning from the LLM. Optional | |
There was a problem hiding this comment.
This has changed and anyway should not have an underscore.
There was a problem hiding this comment.
prompt caching is done on a component perspective and is set through the component metadata fields, not from an api perspective in the incoming request to the conversation api so the cacheTTL field should be removed from what i can tell. It was never a setting from an api POV, only from a component POV. I addressed the camelCase correction here for scrubPii.
...ocs/content/en/developing-applications/building-blocks/conversation/conversation-overview.md
Outdated
Show resolved
Hide resolved
|
|
||
| The `tool_choice` parameter controls how the model uses available tools: | ||
|
|
||
| - **`""`**: The model will not call any tool and instead generates a message (default when no tools are present) |
There was a problem hiding this comment.
This part is confusing: (default when no tools are present)
When no tools are present, would it behave like "auto" , or it will behave like ""
There was a problem hiding this comment.
it is unset if no tools are present, so hopefully my latest change makes this clearer.
|
@sicoyle - PTAL at the comments |
Signed-off-by: Samantha Coyle <[email protected]>
Ready for ya @msfussell 🙏 |
Signed-off-by: Marc Duiker <[email protected]>
See here more context on this: #4769 (comment) The current reference as this being a parameter to pass into the API should be removed, as it is not an API field to pass in. Prompt caching is done on a component perspective and is set through the component metadata fields, not from an api perspective in the incoming request to the conversation api. It was never a setting from an api POV, only from a component POV. |
5 participants
Thank you for helping make the Dapr documentation better!
Please follow this checklist before submitting:
In addition, please fill out the following to help reviewers understand this pull request:
Description
Add tool calling support in teh conversation api docs. I did not update the "How to converse" page as I need the rest of the SDKs updated to reference to bring in python & js code snippets and the other languages sdks updated with tool call support. So the corresponding updates here will be in a separate PR after SDKs are updated: https://docs.dapr.io/developing-applications/building-blocks/conversation/howto-conversation-layer/
Issue reference
#4687