Skip to content

ci(circleci): test Redis behavior against local Redis and print short tracebacks - #44062

Merged
yuneng-berri merged 2 commits into
mainfrom
litellm_cci_redis_tests_and_short_tracebacks
Oct 1, 2026
Merged

yuneng-berri merged 2 commits into
mainfrom
litellm_cci_redis_tests_and_short_tracebacks

Conversation

@yuneng-berri

@yuneng-berri yuneng-berri commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • One CircleCI job ran three legacy test files against a shared remote Redis
  • Most of those tests never needed a real Redis server
  • CircleCI failure output used pytest's long traceback format

How it solves it:

  • DualCache and batch-read logic now runs as unit tests with a mocked RedisCache
  • Behavior that needs a real server moved to tests/integration, which starts a local Redis
  • redis_caching_unit_tests job removed; its coverage now comes from the unit and integration jobs
  • Every CircleCI pytest call, including the integration runner, now passes --tb=short
  • The integration README now allows Redis in the sdk shard, which CircleCI already provides

Pre-Submission checklist

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

This PR only changes tests and CI config, so there is no proxy behavior to curl. The proof is that each new test fails when the code it covers is broken. All runs below are at the PR tip, with no secrets in the environment and a local redis-server for the integration files

Before (bba85f0)

  1. tests/local_testing/test_dual_cache.py, test_redis_batch_optimizations.py and test_redis_increment_with_floor.py needed REDIS_HOST, REDIS_PORT and REDIS_PASSWORD pointing at the shared Redis, so they only ran in the redis_caching_unit_tests job
  2. test_returned_settings in tests/local_testing/test_router_utils.py read REDIS_PORT but never touched Redis

After (61cc20f)

  1. tests/unit/caching/test_dual_cache.py and tests/unit/test_router_get_settings.py: all pass with no Redis and no network
  2. tests/integration/routing/test_redis_increment_with_floor.py and tests/integration/sdk/test_dual_cache_redis.py: 9 passed against a local Redis
  3. Mutations to litellm/caching/dual_cache.py and litellm/caching/redis_cache.py, each run and reverted, all turned at least one new test red:
    • always read Redis
    • increment ignores local_only
    • no batch-read throttle
    • sadd skips memory
    • no floor clamp in the Lua increment
    • TTL always reset
    • sync delete skips Redis
    • no backfill of memory after a Redis hit
  4. circleci config validate .circleci/config.yml reports the config as valid, and every pytest call in it passes --tb=short

Type

🚄 Infrastructure
✅ Test

Caveats (if any)

Low

  • test_basic_functionality_not_broken dropped, since the existing unit get/set test already covers it
  • The other CircleCI jobs still use the shared Redis; this PR only covers the Redis caching job

… tracebacks

The redis_caching_unit_tests job ran three legacy files against the shared remote Redis. The DualCache and
batch-read logic that never needed a server now lives in tests/unit/caching/test_dual_cache.py with a mocked
RedisCache, and the behavior that does need one (the increment-with-floor Lua script, read-through, deletes,
batch reads) moved to tests/integration, which starts a local Redis. test_returned_settings only read
REDIS_PORT and is replaced by a unit test of Router.get_settings

CircleCI pytest runs now use --tb=short so failure output stays readable in the test results tab
@yuneng-berri
yuneng-berri requested a review from a team October 1, 2026 19:35
@greptile-apps

greptile-apps Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Low risk] Test infrastructure and CI configuration updates.

The PR appears safe to merge based on the reviewed changes and resolved threads

Summary

The PR moves Redis-dependent cache checks into integration tests, replaces other checks with Redis-free unit tests, removes the dedicated Redis caching job, and requests short pytest tracebacks throughout CircleCI. The latest changes clarify SDK-shard Redis usage and add the traceback option to the main integration runner

Reviews (2) · Last reviewed commit: "ci(integration): print short tracebacks ..."

Comment thread .circleci/config.yml
Comment thread tests/integration/sdk/test_dual_cache_redis.py
Comment thread .circleci/scripts/run_integration.sh
@yuneng-berri

Copy link
Copy Markdown
Contributor Author

@greptileai

@codecov

codecov Bot commented Oct 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@yuneng-berri
yuneng-berri enabled auto-merge (squash) October 1, 2026 19:59
@yuneng-berri
yuneng-berri merged commit 65a316f into main Oct 1, 2026
98 of 99 checks passed
@yuneng-berri
yuneng-berri deleted the litellm_cci_redis_tests_and_short_tracebacks branch October 1, 2026 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants