Skip to content

(fix) GCS bucket logger - apply truncate_standard_logging_payload_content to standard_logging_payload and ensure GCS flushes queue on fails - #7519

Merged
ishaan-jaff merged 3 commits into
mainfrom
litellm_gcs_large_payload
Jan 3, 2025
Merged

ishaan-jaff merged 3 commits into
mainfrom
litellm_gcs_large_payload

Conversation

@ishaan-jaff

Copy link
Copy Markdown
Contributor

(fix) GCS bucket logger - apply truncate_standard_logging_payload_content to standard_logging_payload and ensure GCS flushes queue on fails

sending async requests containing images.
Very quickly, the memory usage spikes up and the proxy shuts down

Key fixes:

  • use truncate_standard_logging_payload_content for all logging integrations. Ensures no field in standard_logging_payload is more than length 10_000
  • for gcs_bucket logging, clear the in-memory queue even if the current batch has a failure

Relevant issues

Type

🐛 Bug Fix
✅ Test

Changes

[REQUIRED] Testing - Attach a screenshot of any new tests passing locally

If UI changes, send a screenshot/GIF of working UI fixes

@vercel

vercel Bot commented Jan 3, 2025 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
litellm ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jan 3, 2025 2:51pm

@codecov

codecov Bot commented Jan 3, 2025

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 89.65517% with 3 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
litellm/integrations/gcs_bucket/gcs_bucket.py 80.00% 3 Missing ⚠️

📢 Thoughts on this report? Let us know!

@ishaan-jaff
ishaan-jaff merged commit 9fef0a6 into main Jan 3, 2025
@ishaan-berri
ishaan-berri deleted the litellm_gcs_large_payload branch March 26, 2026 21:54
fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026
…tent to standard_logging_payload and ensure GCS flushes queue on fails (BerriAI#7519)

* fix async_send_batch for gcs

* fix truncate GCS logger

* test_truncate_standard_logging_payload

This branch was successfully deployed

1 active deployment
Preview — b8fe9053 Deployed Jan 3, 2025 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant