fix(litellm): parse nested usage payloads#8081
Conversation
Signed-off-by: James Totah <135163520+jamestotah@users.noreply.github.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a7de7abc9a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Signed-off-by: James Totah <135163520+jamestotah@users.noreply.github.com>
jamestotah
changed the title
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7e39bd6ccd
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Signed-off-by: James Totah <135163520+jamestotah@users.noreply.github.com>
Signed-off-by: James Totah <135163520+jamestotah@users.noreply.github.com>
The cache-field fallback for input_tokens when prompt_tokens is absent was dead code — all OpenAI-compatible APIs include prompt_tokens when cache fields are present. Remove the branch and inline the variable. Signed-off-by: Douwe Osinga <douwe@squareup.com>
DOsinga
left a comment
DOsinga
left a comment
There was a problem hiding this comment.
The core fix is solid: unwrapping the nested "usage" key in get_usage() correctly handles the litellm provider passing the full response body, and adding cache_read_input_tokens/cache_write_input_tokens to Usage is a clean forward-looking addition. Both Codex P1 comments were properly addressed by the author.
I merged main in and simplified get_usage slightly: the cache-field fallback for input_tokens when prompt_tokens is absent was dead code (all OpenAI-compatible APIs include prompt_tokens when cache fields are present), so I removed it and inlined the variable.
Signed-off-by: James Totah <135163520+jamestotah@users.noreply.github.com> Signed-off-by: Douwe Osinga <douwe@squareup.com> Co-authored-by: Douwe Osinga <douwe@squareup.com> Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2 participants
Summary
usageobjectTesting