Add semgrep & Fix OOMs

[AlexsanderHamir]

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Type

🧹 Refactoring

Changes

• Added semgrep for extensive custom rules.
• Fixed OOMs across integrations.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 11, 2026 1:50am

[greptile-apps]

Greptile Overview

Greptile Summary

Adds Semgrep to CI as a release gate and bounds all asyncio.Queue() instances with a configurable maxsize (default 1000 via LITELLM_ASYNCIO_QUEUE_MAXSIZE) to prevent unbounded memory growth.

• Semgrep CI integration: New job runs custom rules from .semgrep/rules/ on main and litellm_* branches, gating publish_to_pypi.
• Bounded queues: GCS bucket logger, spend update queues, and daily spend update queues now use bounded asyncio.Queue(maxsize=...).
• Dead aggregation checks: The spend queue aggregation safety net (qsize >= MAX_SIZE_IN_MEMORY_QUEUE=2000) is now unreachable because put() blocks at maxsize=1000. This means the in-memory compaction logic will never trigger, and under high load the blocking put() will stall the request path until the periodic consumer drains the queue.
• GCS race condition: The full() → flush_queue() → put() pattern is not atomic, and the flush_queue() override skips the flush_lock. Concurrent producers can cause put() to still block despite the flush attempt.
• No tests added: The PR checklist indicates tests were added, but the diff contains no test files.

Confidence Score: 2/5

• The queue bounding introduces blocking backpressure on the request path and makes the existing aggregation safety net unreachable — needs adjustment before merging.
• The core goal (preventing OOM via bounded queues) is sound, but the implementation creates new issues: blocking put() stalls request processing, the aggregation checks become dead code due to maxsize < threshold mismatch, and the GCS flush has a TOCTOU race. No tests were added despite the checklist claim.
• litellm/proxy/db/db_transaction_queue/base_update_queue.py, litellm/proxy/db/db_transaction_queue/spend_update_queue.py, litellm/proxy/db/db_transaction_queue/daily_spend_update_queue.py, litellm/integrations/gcs_bucket/gcs_bucket.py

Important Files Changed

Filename	Overview
.circleci/config.yml	Adds semgrep CI job with custom rules only, gates publish_to_pypi on semgrep passing. Clean addition.
cookbook/nova_sonic_realtime.py	Bounds audio queue with configurable maxsize (default 10,000). Uses shared env var name LITELLM_ASYNCIO_QUEUE_MAXSIZE but different default (10,000 vs 1,000).
litellm/constants.py	Adds LITELLM_ASYNCIO_QUEUE_MAXSIZE constant (default 1000). Clean addition.
litellm/integrations/gcs_bucket/gcs_bucket.py	Bounds GCS log queue and adds pre-put flush. Race condition: full() + flush_queue() + put() is not atomic; concurrent producers can cause put() to still block. Also flush_queue() override does not use the flush_lock from the parent class.
litellm/proxy/db/db_transaction_queue/base_update_queue.py	Bounds base queue to LITELLM_ASYNCIO_QUEUE_MAXSIZE (1000). add_update() uses blocking put() which will block callers on the request path when queue is full.
litellm/proxy/db/db_transaction_queue/daily_spend_update_queue.py	Overrides queue with same bounded maxsize. Aggregation check (qsize >= 2000) is unreachable since queue maxsize is 1000.
litellm/proxy/db/db_transaction_queue/spend_update_queue.py	Same issue as daily_spend_update_queue: aggregation check at 2000 is unreachable with maxsize 1000.

Sequence Diagram

sequenceDiagram
    participant Req as Request Handler
    participant SQ as SpendUpdateQueue
    participant AQ as asyncio.Queue(maxsize=1000)
    participant Sched as Scheduler (10-15s)
    participant DB as Database

    Req->>SQ: add_update(spend_item)
    SQ->>AQ: await put(item)
    alt Queue not full
        AQ-->>SQ: item added
        SQ->>SQ: check qsize >= 2000 (unreachable)
    else Queue full (1000 items)
        AQ-->>SQ: BLOCKS until space available
        Note over Req,SQ: Request stalls here
    end

    loop Every 10-15 seconds
        Sched->>SQ: flush_all_updates_from_in_memory_queue()
        SQ->>AQ: get() up to 1000 items
        AQ-->>SQ: drained items
        SQ->>DB: write aggregated spend
        Note over AQ: Space freed, blocked put() resumes
    end

Loading

[greptile-apps]

_{10 files reviewed, 5 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

TOCTOU race: between full() returning True and put() executing, concurrent coroutines can re-fill the queue after the flush. The put() on line 78 can still block. Also, the overridden flush_queue() (line 381) doesn't acquire self.flush_lock, so concurrent flushes from periodic_flush() and this manual flush can race.

Consider using put_nowait() in a try/except QueueFull block to avoid blocking the logging path.

[greptile-apps]

This uses the same env var LITELLM_ASYNCIO_QUEUE_MAXSIZE as the proxy queues but with a different default (10,000 vs 1,000). This could confuse users who set the env var expecting uniform behavior — proxy queues get 1000 from constants.py while this cookbook gets 10,000 from the hardcoded fallback here.

[greptile-apps]

Additional Comments (3)

litellm/proxy/db/db_transaction_queue/base_update_queue.py
await self.update_queue.put(update) will block the caller when the queue is full (maxsize=1000). Since add_update is called on the request path (spend tracking), this introduces backpressure that stalls request processing until the periodic consumer (every ~10-15s) drains the queue. Consider using put_nowait() with a try/except asyncio.QueueFull to either drop, log, or aggregate instead of blocking the request.

---

litellm/proxy/db/db_transaction_queue/spend_update_queue.py
self.MAX_SIZE_IN_MEMORY_QUEUE defaults to 2000, but the queue maxsize is now 1000. put() on line 40 blocks at 1000 items, so qsize() can never reach 2000 — this aggregation check is now dead code.

Either lower MAX_SIZE_IN_MEMORY_QUEUE to be below LITELLM_ASYNCIO_QUEUE_MAXSIZE, or trigger aggregation before/at the maxsize boundary.

---

litellm/proxy/db/db_transaction_queue/daily_spend_update_queue.py
Same issue as SpendUpdateQueue: qsize() >= 2000 is unreachable since the queue blocks put() at maxsize=1000. The aggregation safety net is dead code.