[Fix] Spend Update Queue Aggregation Never Triggers with Default Presets

[yuneng-jiang]

Relevant issues

Summary

MAX_SIZE_IN_MEMORY_QUEUE defaulted to 2000 while LITELLM_ASYNCIO_QUEUE_MAXSIZE (the hard cap on the underlying asyncio.Queue) defaulted to 1000. Because asyncio.Queue blocks on .put() once it reaches its maxsize, qsize() can never reach 2000, so the aggregation branch in SpendUpdateQueue.add_update() was dead code.

Fix

• Moved LITELLM_ASYNCIO_QUEUE_MAXSIZE before MAX_SIZE_IN_MEMORY_QUEUE in constants.py so the former can be referenced in the latter's default.
• MAX_SIZE_IN_MEMORY_QUEUE now defaults to int(LITELLM_ASYNCIO_QUEUE_MAXSIZE * 0.8) (800), ensuring the aggregation threshold is always reachable.
• Added a startup warning in BaseUpdateQueue.__init__ that fires whenever MAX_SIZE_IN_MEMORY_QUEUE >= LITELLM_ASYNCIO_QUEUE_MAXSIZE, catching any misconfiguration set via environment variables.

Testing

Added test_misconfigured_queue_thresholds_warns in tests/test_litellm/proxy/db/db_transaction_queue/test_base_update_queue.py — patches both constants to simulate the broken configuration and asserts the warning is emitted.

Type

🐛 Bug Fix
✅ Test

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 24, 2026 0:21am

[greptile-apps]

Greptile Summary

This PR fixes a bug where MAX_SIZE_IN_MEMORY_QUEUE (defaulting to 2000) could never be reached because the underlying asyncio.Queue was capped at LITELLM_ASYNCIO_QUEUE_MAXSIZE (1000). Since asyncio.Queue.put() blocks when the queue is full, the spend aggregation check in SpendUpdateQueue.add_update() and DailySpendUpdateQueue.add_update() was dead code under default settings.

• Changed MAX_SIZE_IN_MEMORY_QUEUE default from 2000 to int(LITELLM_ASYNCIO_QUEUE_MAXSIZE * 0.8) (800), ensuring the aggregation threshold is always below the queue capacity
• Added a startup warning in BaseUpdateQueue.__init__ to catch user misconfiguration via environment variables where the threshold >= queue maxsize
• Added a mock-only test verifying the warning is emitted for the misconfigured case

Confidence Score: 5/5

• This PR is safe to merge — it fixes a clear default misconfiguration bug with a minimal, well-targeted change.
• The fix is straightforward and correct: the default value relationship between two constants was inverted, making the aggregation logic unreachable. The new default (80% of queue maxsize) ensures the threshold is always reachable. The startup warning is a good defensive measure. The test is properly mocked with no network calls. No changes to critical request paths or database queries.
• No files require special attention

Important Files Changed

Filename	Overview
litellm/constants.py	Reordered LITELLM_ASYNCIO_QUEUE_MAXSIZE before MAX_SIZE_IN_MEMORY_QUEUE and changed default from 2000 to 80% of LITELLM_ASYNCIO_QUEUE_MAXSIZE (800). Correctly fixes the dead code bug.
litellm/proxy/db/db_transaction_queue/base_update_queue.py	Added a startup warning when MAX_SIZE_IN_MEMORY_QUEUE >= LITELLM_ASYNCIO_QUEUE_MAXSIZE, catching user misconfiguration via environment variables. Clean defensive check.
tests/test_litellm/proxy/db/db_transaction_queue/test_base_update_queue.py	Added mock-only test verifying the misconfiguration warning is emitted. No network calls; patches module-level constants and logger correctly.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["add_update() called"] --> B["await queue.put(update)"]
    B --> C{"queue full?\n(qsize == LITELLM_ASYNCIO_QUEUE_MAXSIZE)"}
    C -- Yes --> D["BLOCKS (await)"]
    D --> B
    C -- No --> E{"qsize >= MAX_SIZE_IN_MEMORY_QUEUE?"}
    E -- Yes --> F["aggregate_queue_updates()\nFlush + re-enqueue aggregated items"]
    E -- No --> G["Return"]
    F --> G

    style C fill:#ffcc00,stroke:#333
    style E fill:#66ccff,stroke:#333
    style D fill:#ff6666,stroke:#333
    style F fill:#66ff66,stroke:#333

Loading

_{Last reviewed commit: a9c44d8}

[greptile-apps]

_{3 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}