fix(test): replace caplog with custom handler for parallel execution

[jquinter]

Summary

Fixes test_cost_calculation_uses_debug_level and test_batch_cost_calculation_uses_debug_level failures when running with pytest-xdist parallel execution (--dist=loadscope).

Problem

The tests were failing with:

ValueError: I/O operation on closed file
AssertionError: No cost calculation logs found

Root cause: caplog fixture doesn't work reliably with pytest-xdist parallel execution because log capture happens in the main process while tests run in worker processes. This causes file handle issues and missing log records.

Solution

Replaced caplog with a custom LogRecordHandler that directly attaches to the verbose_logger. This approach works correctly in parallel execution because:

• Each worker process gets its own handler instance
• No cross-process file handle sharing
• Direct capture of log records in the same process as the test

Testing

• Tested locally with pytest tests/test_litellm/test_cost_calculation_log_level.py -v
• Tested with parallel execution: pytest tests/test_litellm/test_cost_calculation_log_level.py -n 4 --dist=loadscope -v

Related

• Fixes test failures exposed by PR improve(ci): enhance test stability with better isolation and distribution #21277 (CI improvements with better test isolation)
• Not caused by PR improve(ci): enhance test stability with better isolation and distribution #21277, but exposed by the improved test distribution

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 16, 2026 0:16am

[greptile-apps]

Greptile Summary

Replaces caplog fixture with a custom LogRecordHandler in two cost calculation log-level tests to fix failures under pytest-xdist parallel execution. The approach is sound: each test attaches its own handler to verbose_logger, captures records in-process, and properly cleans up in a finally block. The tests make no network calls (cost calculations use local pricing data), keeping the test suite CI-friendly.

• Replaced caplog with in-process LogRecordHandler in test_cost_calculation_uses_debug_level and test_batch_cost_calculation_uses_debug_level
• Both tests properly save/restore logger state in finally blocks
• Minor: import pytest is now unused and should be removed; file is missing a trailing newline

Confidence Score: 4/5

• This PR is safe to merge — it only modifies test infrastructure with no impact on production code.
• Score of 4 reflects that the core change is correct and well-implemented. The custom handler approach properly solves the pytest-xdist compatibility issue. Minor style issues (unused import, missing trailing newline) are the only items preventing a 5.
• No files require special attention. The only changed file is a test file with minor style issues.

Important Files Changed

Filename	Overview
tests/test_litellm/test_cost_calculation_log_level.py	Replaces caplog with custom LogRecordHandler for pytest-xdist compatibility. Correct approach with proper cleanup in finally blocks. Minor issues: unused pytest import and missing trailing newline.

Sequence Diagram

sequenceDiagram
    participant Test as Test Function
    participant Handler as LogRecordHandler
    participant Logger as verbose_logger
    participant CostCalc as completion_cost / batch_cost_calculator

    Test->>Handler: Create handler instance
    Test->>Logger: Save original level
    Test->>Logger: setLevel(DEBUG)
    Test->>Logger: addHandler(handler)
    Test->>CostCalc: Call cost calculation
    CostCalc->>Logger: verbose_logger.debug(...)
    Logger->>Handler: emit(record)
    Handler->>Handler: Append to records[]
    Test->>Handler: Filter records by message
    Test->>Test: Assert records are DEBUG level
    Test->>Logger: removeHandler(handler)
    Test->>Logger: Restore original level

Loading

_{Last reviewed commit: 4f2c7d3}

[greptile-apps]

_{1 file reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Additional Comments (1)

tests/test_litellm/test_cost_calculation_log_level.py
Unused import pytest

With the removal of the caplog fixture parameter, pytest is no longer used anywhere in this file (only referenced in docstring comments). This will trigger linting warnings.