Merged
Conversation
hugoShaka
force-pushed
the
hugo/assist-stream-audit-query
branch
from
efeae34 to
2f65b9f
Compare
avatus
approved these changes
Aug 29, 2023
jakule
reviewed
Aug 30, 2023
| // StreamToDeltas converts an openai.CompletionStream into a channel of strings. | ||
| // This channel can then be consumed manually to search for specific markers, | ||
| // or directly converted into a StreamingMessage with NewStreamingMessage. | ||
| func StreamToDeltas(stream *openai.ChatCompletionStream) chan string { |
Contributor
There was a problem hiding this comment.
Does this function need to be public? For me, it sounds like some internal helper that should not be called outside of the module.
Contributor
Author
There was a problem hiding this comment.
lib/ai/model and lib/ai/model/tools need to do this and are in separate packages. I don't think we need to expose this outside of lib/ai/model/... though. lib/ai/model manually processes the deltas, while lib/ai/model/tools can create the streaming message directly, so we can't make a single StreamtoStramingMessage() function
Contributor
Author
There was a problem hiding this comment.
I can create a new lib/ai/model/utils package, or a lib/ai/model/internal one
| // (like a marker to identify the kind of response the model is providing), | ||
| // the prefix can be passed through the prefix parameter. It will be stripped | ||
| // but will still be reflected in the token count. | ||
| func NewStreamingMessage(deltas <-chan string, alreadyStreamed, prefix string) (*StreamingMessage, *tokens.AsynchronousTokenCounter, error) { |
Contributor
There was a problem hiding this comment.
The same here, it sounds more like an internal helper rather than something that can be used outside of this module.
jakule
reviewed
Aug 30, 2023
jakule
approved these changes
Aug 30, 2023
public-teleport-github-review-bot
Bot
removed the request for review
from xacrimon
jakule
pushed a commit
that referenced
this pull request
Oct 18, 2023
* stream audit query responses * fixup! stream audit query responses * fixup! fixup! stream audit query responses
github-merge-queue Bot
pushed a commit
that referenced
this pull request
Oct 19, 2023
* Add the ability to run a specific tool to Assist. (#31112) This commit contains: - a refactoring of the model action logic to be exposed through the `DoAction` function - Expose `RunTool()` capability through ai.Client and assist.Assist - Add the "audit-query" action that invokes directly the audit generation tool * stream audit query responses (#31092) * stream audit query responses * fixup! stream audit query responses * fixup! fixup! stream audit query responses --------- Co-authored-by: Hugo Shaka <hugo.hervieux@goteleport.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR makes the audit query generation tool return streaming messages.
It extracts the existing streaming logic in
lib/ai/model/output/streaming.goso it can be used both by the main agent and by the tools. It also removes the stale tokenizer field from theai.Chatstruct.