#785 allows agents to utilise streaming even when they have a functio… #786
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…a function definition. Streaming is only diabled when a function call is to be made
ragyabraham
had a problem deploying
to
openai1
GitHub Actions
Failure
ragyabraham
had a problem deploying
to
openai1
GitHub Actions
Failure
ragyabraham
had a problem deploying
to
openai1
GitHub Actions
Failure
ragyabraham
had a problem deploying
to
openai1
GitHub Actions
Failure
ragyabraham
had a problem deploying
to
openai1
GitHub Actions
Failure
ragyabraham
had a problem deploying
to
openai1
GitHub Actions
Failure
ragyabraham
had a problem deploying
to
openai1
GitHub Actions
Failure
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #786 +/- ##
==========================================
+ Coverage 27.61% 37.17% +9.56%
==========================================
Files 27 27
Lines 3480 3502 +22
Branches 789 794 +5
==========================================
+ Hits 961 1302 +341
+ Misses 2447 2080 -367
- Partials 72 120 +48
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
sonichi
reviewed
Nov 28, 2023
Alvaromah
reviewed
Nov 29, 2023
| @@ -269,7 +269,7 @@ def _completions_create(self, client, params): | |||
| completions = client.chat.completions if "messages" in params else client.completions | |||
| # If streaming is enabled, has messages, and does not have functions, then | |||
| # iterate over the chunks of the response | |||
| if params.get("stream", False) and "messages" in params and "functions" not in params: | |||
| if params.get("stream", False) and "messages" in params: | |||
There was a problem hiding this comment.
This change is dangerous.
If we want to support streaming with functions, we need to handle function chunks.
There was a problem hiding this comment.
The response when using streaming with functions is something like this:
delta=ChoiceDelta(
content=None,
function_call=ChoiceDeltaFunctionCall(
arguments='',
name='get_current_weather'),
role='assistant', tool_calls=None),
finish_reason=None, index=0
)
As you can see, content is None. We have to handle the 'function_call' field instead.
If we remove the "...and functions not in param", we will be looking for content when we should be looking for 'function_call'.
There was a problem hiding this comment.
hey @Alvaromah fair point but I believe line 284 checks the ChoiceDelta to ensure choice.delta.content is None and that function_call is present, which, if I understand correctly, would then handle that use case
ragyabraham
had a problem deploying
to
openai1
GitHub Actions
Failure
ragyabraham
had a problem deploying
to
openai1
GitHub Actions
Failure
ragyabraham
had a problem deploying
to
openai1
GitHub Actions
Failure
ragyabraham
had a problem deploying
to
openai1
GitHub Actions
Failure
ragyabraham
had a problem deploying
to
openai1
GitHub Actions
Failure
ragyabraham
had a problem deploying
to
openai1
GitHub Actions
Failure
ragyabraham
had a problem deploying
to
openai1
GitHub Actions
Failure
Alvaromah
reviewed
Dec 1, 2023
| params = params.copy() | ||
| params["stream"] = False | ||
| response = completions.create(**params) | ||
| return response | ||
| content = choice.delta.content |
There was a problem hiding this comment.
In these lines, "completions.create()" is invoked twice, resulting in two requests to OpenAI. The solution involves managing the scenario when "choice.delta.function_call" is not None and extracting the data to stream, which is notably more intricate.
There was a problem hiding this comment.
I think I have resolved these issues in #1118
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 12853598 | Triggered | Generic High Entropy Secret | 79dbb7b | test/oai/test_utils.py | View secret |
| 10404693 | Triggered | Generic High Entropy Secret | e43a86c | test/oai/test_utils.py | View secret |
| 10404693 | Triggered | Generic High Entropy Secret | bdb40d7 | test/oai/test_utils.py | View secret |
| 10404693 | Triggered | Generic High Entropy Secret | 954ca45 | test/oai/test_utils.py | View secret |
| 10404693 | Triggered | Generic High Entropy Secret | 79dbb7b | test/oai/test_utils.py | View secret |
| 10404662 | Triggered | Generic CLI Secret | eff19ac | .github/workflows/dotnet-release.yml | View secret |
| 10404662 | Triggered | Generic CLI Secret | 06a0a5d | .github/workflows/dotnet-release.yml | View secret |
| 10404662 | Triggered | Generic CLI Secret | 0524c77 | .github/workflows/dotnet-release.yml | View secret |
| 10404662 | Triggered | Generic CLI Secret | d7ea410 | .github/workflows/dotnet-release.yml | View secret |
| 10404662 | Triggered | Generic CLI Secret | e43a86c | .github/workflows/dotnet-build.yml | View secret |
| 10404662 | Triggered | Generic CLI Secret | 841ed31 | .github/workflows/dotnet-release.yml | View secret |
| 10404662 | Triggered | Generic CLI Secret | 802f099 | .github/workflows/dotnet-release.yml | View secret |
| 10404662 | Triggered | Generic CLI Secret | 9a484d8 | .github/workflows/dotnet-build.yml | View secret |
| 10404662 | Triggered | Generic CLI Secret | e973ac3 | .github/workflows/dotnet-release.yml | View secret |
| 10404662 | Triggered | Generic CLI Secret | 89650e7 | .github/workflows/dotnet-release.yml | View secret |
| 10404662 | Triggered | Generic CLI Secret | e07b06b | .github/workflows/dotnet-release.yml | View secret |
| 10404662 | Triggered | Generic CLI Secret | abe4c41 | .github/workflows/dotnet-build.yml | View secret |
| 10404662 | Triggered | Generic CLI Secret | 7362fb9 | .github/workflows/dotnet-release.yml | View secret |
| 12853599 | Triggered | Generic High Entropy Secret | 79dbb7b | test/oai/test_utils.py | View secret |
| 10404694 | Triggered | Generic High Entropy Secret | e43a86c | test/oai/test_utils.py | View secret |
| 10404694 | Triggered | Generic High Entropy Secret | 954ca45 | test/oai/test_utils.py | View secret |
| 10404694 | Triggered | Generic High Entropy Secret | bdb40d7 | test/oai/test_utils.py | View secret |
| 10404695 | Triggered | Generic High Entropy Secret | abad9ff | test/oai/test_utils.py | View secret |
| 10404695 | Triggered | Generic High Entropy Secret | 954ca45 | test/oai/test_utils.py | View secret |
| 10404695 | Triggered | Generic High Entropy Secret | c7bb588 | test/oai/test_utils.py | View secret |
| 10404695 | Triggered | Generic High Entropy Secret | b97b99d | test/oai/test_utils.py | View secret |
| 10404695 | Triggered | Generic High Entropy Secret | e43a86c | test/oai/test_utils.py | View secret |
| 12853600 | Triggered | Generic High Entropy Secret | 79dbb7b | test/oai/test_utils.py | View secret |
| 12853601 | Triggered | Generic High Entropy Secret | 79dbb7b | test/oai/test_utils.py | View secret |
| 10493810 | Triggered | Generic Password | 49e8053 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
| 10493810 | Triggered | Generic Password | 501610b | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
| 10493810 | Triggered | Generic Password | 49e8053 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
| 10493810 | Triggered | Generic Password | 501610b | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
| 10493810 | Triggered | Generic Password | d422c63 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
| 10493810 | Triggered | Generic Password | 97fa339 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
| 10493810 | Triggered | Generic Password | 49e8053 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
| 10493810 | Triggered | Generic Password | d422c63 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
| 10493810 | Triggered | Generic Password | 97fa339 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
| 10493810 | Triggered | Generic Password | d422c63 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
| 10493810 | Triggered | Generic Password | 97fa339 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
| 10493810 | Triggered | Generic Password | 501610b | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
| 10404696 | Triggered | Generic High Entropy Secret | 954ca45 | test/oai/test_utils.py | View secret |
| 10404696 | Triggered | Generic High Entropy Secret | bdb40d7 | test/oai/test_utils.py | View secret |
| 10404696 | Triggered | Generic High Entropy Secret | 79dbb7b | test/oai/test_utils.py | View secret |
| 10404696 | Triggered | Generic High Entropy Secret | e43a86c | test/oai/test_utils.py | View secret |
| 10422482 | Triggered | Generic High Entropy Secret | 79dbb7b | test/oai/test_utils.py | View secret |
| 10422482 | Triggered | Generic High Entropy Secret | bdb40d7 | test/oai/test_utils.py | View secret |
| 12853602 | Triggered | Generic High Entropy Secret | 79dbb7b | test/oai/test_utils.py | View secret |
| 11616921 | Triggered | Generic High Entropy Secret | a86d0fd | notebook/agentchat_agentops.ipynb | View secret |
| 11616921 | Triggered | Generic High Entropy Secret | 394561b | notebook/agentchat_agentops.ipynb | View secret |
| 11616921 | Triggered | Generic High Entropy Secret | 3eac646 | notebook/agentchat_agentops.ipynb | View secret |
| 11616921 | Triggered | Generic High Entropy Secret | f45b553 | notebook/agentchat_agentops.ipynb | View secret |
| 11616921 | Triggered | Generic High Entropy Secret | 6563248 | notebook/agentchat_agentops.ipynb | View secret |
| 12853598 | Triggered | Generic High Entropy Secret | 2b3a9ae | test/oai/test_utils.py | View secret |
| 12853598 | Triggered | Generic High Entropy Secret | c03558f | test/oai/test_utils.py | View secret |
| 10404693 | Triggered | Generic High Entropy Secret | c03558f | test/oai/test_utils.py | View secret |
| 10404693 | Triggered | Generic High Entropy Secret | 2b3a9ae | test/oai/test_utils.py | View secret |
| 10404693 | Triggered | Generic High Entropy Secret | 0a3c6c4 | test/oai/test_utils.py | View secret |
| 10404693 | Triggered | Generic High Entropy Secret | 76f5f5a | test/oai/test_utils.py | View secret |
| 10404662 | Triggered | Generic CLI Secret | 954ca45 | .github/workflows/dotnet-build.yml | View secret |
| 12853599 | Triggered | Generic High Entropy Secret | 2b3a9ae | test/oai/test_utils.py | View secret |
| 12853599 | Triggered | Generic High Entropy Secret | c03558f | test/oai/test_utils.py | View secret |
| 10404694 | Triggered | Generic High Entropy Secret | 76f5f5a | test/oai/test_utils.py | View secret |
| 10404694 | Triggered | Generic High Entropy Secret | 0a3c6c4 | test/oai/test_utils.py | View secret |
| 10404695 | Triggered | Generic High Entropy Secret | 3b79cc6 | test/oai/test_utils.py | View secret |
| 10404695 | Triggered | Generic High Entropy Secret | 11baa52 | test/oai/test_utils.py | View secret |
| 12853600 | Triggered | Generic High Entropy Secret | c03558f | test/oai/test_utils.py | View secret |
| 12853600 | Triggered | Generic High Entropy Secret | 2b3a9ae | test/oai/test_utils.py | View secret |
| 12853601 | Triggered | Generic High Entropy Secret | c03558f | test/oai/test_utils.py | View secret |
| 12853601 | Triggered | Generic High Entropy Secret | 2b3a9ae | test/oai/test_utils.py | View secret |
| 10493810 | Triggered | Generic Password | 3b79cc6 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
| 10493810 | Triggered | Generic Password | 11baa52 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
| 10493810 | Triggered | Generic Password | 11baa52 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
| 10493810 | Triggered | Generic Password | 3b79cc6 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
| 10404696 | Triggered | Generic High Entropy Secret | 0a3c6c4 | test/oai/test_utils.py | View secret |
| 10404696 | Triggered | Generic High Entropy Secret | 76f5f5a | test/oai/test_utils.py | View secret |
| 10404696 | Triggered | Generic High Entropy Secret | c03558f | test/oai/test_utils.py | View secret |
| 10404696 | Triggered | Generic High Entropy Secret | 2b3a9ae | test/oai/test_utils.py | View secret |
| 10422482 | Triggered | Generic High Entropy Secret | 2b3a9ae | test/oai/test_utils.py | View secret |
| 10422482 | Triggered | Generic High Entropy Secret | c03558f | test/oai/test_utils.py | View secret |
and 16 others.
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secrets safely. Learn here the best practices.
- Revoke and rotate these secrets.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
Currently, streaming is disabled when a function definition is associated with an agent for all messages. This means that whether an agent is about to execute a function or send a normal text-based message, they will not be able to utilise streaming.
Changes made
A small change has been made to the
oai.client.pymodule.These changes allow agents to utilise streaming for text-based messages, i.e. non-function calls, even when they have functions in their config.
Related issue number
#785
Checks