Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cost summary to client.py #812

Merged
merged 11 commits into from
Dec 3, 2023
Merged

Add cost summary to client.py #812

merged 11 commits into from
Dec 3, 2023

Conversation

yiranwu0
Copy link
Collaborator

@yiranwu0 yiranwu0 commented Nov 30, 2023

Why are these changes needed?

Add cost summary to client.py.
Including calculations for two types of cost:

  • actual cost incurred excluding compeletions retrieved from caching.
  • cost of all completions using the instance of client, including caching.

Note: cost from completions that doesn't pass a filter is also included.

Minor fix: store cost before caching.

Related issue number

Checks

@codecov-commenter
Copy link

codecov-commenter commented Nov 30, 2023

Codecov Report

Attention: 45 lines in your changes are missing coverage. Please review.

Comparison is base (5c92fb3) 28.00% compared to head (d852377) 41.47%.

Files Patch % Lines
autogen/oai/client.py 31.81% 41 Missing and 4 partials ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main     #812       +/-   ##
===========================================
+ Coverage   28.00%   41.47%   +13.46%     
===========================================
  Files          27       27               
  Lines        3524     3578       +54     
  Branches      803      860       +57     
===========================================
+ Hits          987     1484      +497     
+ Misses       2465     1954      -511     
- Partials       72      140       +68     
Flag Coverage Δ
unittests 41.36% <31.81%> (+13.41%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@qingyun-wu qingyun-wu mentioned this pull request Dec 3, 2023
3 tasks
@qingyun-wu qingyun-wu linked an issue Dec 3, 2023 that may be closed by this pull request
@qingyun-wu
Copy link
Contributor

Could you fix the test error?

@qingyun-wu
Copy link
Contributor

The location of the added notebook link is not right. I plan to fix that in another PR #851

@yiranwu0
Copy link
Collaborator Author

yiranwu0 commented Dec 3, 2023

The location of the added notebook link is not right. I plan to fix that in another PR #851

Got it. Updated for test error.

@qingyun-wu qingyun-wu added this pull request to the merge queue Dec 3, 2023
Merged via the queue into main with commit 7a4ba1a Dec 3, 2023
78 of 83 checks passed
@sonichi sonichi deleted the addcostsummary branch December 3, 2023 22:25
rlam3 pushed a commit to rlam3/autogen that referenced this pull request Dec 19, 2023
* init commit

* add doc, notebook and test

* fix test

* update

* update

* update

* update
whiskyboy pushed a commit to whiskyboy/autogen that referenced this pull request Apr 17, 2024
* init commit

* add doc, notebook and test

* fix test

* update

* update

* update

* update
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cost, time estimate / task , pls
4 participants