fix(test): clear tokenizer LRU cache for test isolation

[jquinter]

Summary

Fixes test isolation issue in TestTokenizerSelection that was exposed by --dist=loadscope in PR #21277.

Problem

_select_tokenizer_helper is decorated with @lru_cache, causing:

• When tests run sequentially with --dist=loadscope, cached results persist between tests
• Mock for Tokenizer.from_pretrained is never called (cache hit instead)
• Test assertion fails: "Expected 'from_pretrained' to be called once. Called 0 times."

Root Cause

Solution

Implemented triple-layer cache clearing strategy:

1. Module-level: Clear cache on import
2. Class-level: Clear in setUpClass()
3. Function-level: Clear in both setUp() and @pytest.fixture(autouse=True)

This ensures cache is cleared before each test regardless of how pytest schedules them.

Why This Matters

Data from PR #21277 testing shows:

• WITH --dist=loadscope: 70% test pass rate (7/10)
• WITHOUT --dist=loadscope: 40% test pass rate (4/10)

This fix allows us to keep --dist=loadscope (better overall stability) while fixing the specific cache issue it exposed.

Testing

Related

• PR improve(ci): enhance test stability with better isolation and distribution #21277 - CI improvements that exposed this issue

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 15, 2026 11:38pm

[greptile-apps]

Greptile Summary

This PR fixes a test isolation issue in TestTokenizerSelection caused by _select_tokenizer_helper's @lru_cache decorator. When tests run with --dist=loadscope, cached results persist between test methods, preventing mocks from being invoked. The fix adds cache clearing at three levels: module import, class setup, and per-test setup (via both setUp() and a pytest autouse fixture).

• Core fix is correct: Clearing the LRU cache before each test method ensures mocks are properly exercised instead of hitting cached values from prior test runs.
• Redundancy in approach: The "triple-layer" strategy is over-engineered — the setUp() method alone handles per-test clearing for unittest.TestCase, and setUpClass is redundant since setUp runs before every test. The autouse fixture also applies file-wide to ~30 unrelated standalone tests that don't need cache clearing.
• Minor formatting: Missing blank line between setUp() and the @patch decorator on the next method.

Confidence Score: 4/5

• This PR is safe to merge — it only modifies test infrastructure with no impact on production code.
• Score of 4 reflects that the fix correctly addresses the cache isolation problem and only touches test code. Deducted one point for the redundant triple-layer approach which adds unnecessary complexity and a minor formatting issue.
• No files require special attention — changes are confined to test infrastructure only.

Important Files Changed

Filename	Overview
tests/test_litellm/litellm_core_utils/test_token_counter.py	Adds triple-layer LRU cache clearing for _select_tokenizer_helper to fix test isolation under --dist=loadscope. The approach is correct but over-engineered — setUp() plus module-level clear would suffice. Minor formatting issue with missing blank line.

Sequence Diagram

sequenceDiagram
    participant TF as Test Framework
    participant ML as Module Load
    participant SC as setUpClass
    participant SU as setUp / autouse fixture
    participant T as Test Method
    participant LRU as _select_tokenizer_helper LRU Cache

    ML->>LRU: cache_clear() (module import)
    Note over LRU: Cache empty

    TF->>SC: Run setUpClass()
    SC->>LRU: cache_clear()
    Note over LRU: Cache empty (redundant)

    loop For each test method
        TF->>SU: Run setUp() + autouse fixture
        SU->>LRU: cache_clear()
        Note over LRU: Cache empty before test
        TF->>T: Run test with @patch mock active
        T->>LRU: _select_tokenizer_helper("model")
        Note over LRU: Cache MISS → calls mocked function
        LRU-->>T: Returns mocked result
        T->>T: assert mock called ✓
    end

Loading

_{Last reviewed commit: 5f79bf4}

[greptile-apps]

_{1 file reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

[jquinter]

@greptile-apps re-review please

[greptile-apps]

Greptile Summary

This PR fixes a test isolation issue in TestTokenizerSelection caused by the @lru_cache decorator on _select_tokenizer_helper. When tests run on the same worker under --dist=loadscope, cached results from earlier tests prevent mocked methods from being called, causing assertion failures. The fix adds a module-level cache_clear() call (to handle cross-module cache pollution) and a setUp() method that clears the cache before each test method.

• Adds _select_tokenizer_helper.cache_clear() at module level after import
• Adds setUp() method to TestTokenizerSelection that clears the LRU cache before each test
• The final implementation is clean and minimal after simplifying from an earlier "triple-layer" approach
• Note: The PR description references a "triple-layer cache clearing strategy" which was from an earlier commit — the final version correctly uses only two layers (module-level + setUp())
• All tests in the class use @patch decorators (no real network calls)

Confidence Score: 5/5

• This PR is safe to merge — it only adds cache clearing to test setup with no production code changes.
• The change is test-only, minimal (adds ~13 lines), and addresses a well-understood caching issue. It modifies no production code and only clears an LRU cache in test setup to ensure isolation. The approach is standard and correct.
• No files require special attention.

Important Files Changed

Filename	Overview
tests/test_litellm/litellm_core_utils/test_token_counter.py	Adds module-level cache_clear() and setUp() method to TestTokenizerSelection to clear the _select_tokenizer_helper LRU cache before each test, fixing test isolation issues under --dist=loadscope. Clean, minimal change.

Flowchart

flowchart TD
    A[Module Load] -->|cache_clear| B[LRU Cache Empty]
    B --> C[TestTokenizerSelection]
    C --> D[setUp runs]
    D -->|cache_clear| E[LRU Cache Empty]
    E --> F["@patch mocks Tokenizer"]
    F --> G[_select_tokenizer_helper called]
    G --> H{Cache miss}
    H --> I[Mock is invoked]
    I --> J[assert_called_once passes]
    J --> K[Next test method]
    K --> D

Loading

_{Last reviewed commit: 706792b}

[greptile-apps]

_{1 file reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}