fix: improve Langfuse test isolation to prevent flaky failures

[jquinter]

Problem

The test_log_langfuse_v2_handles_null_usage_values test was intermittently failing in CI with:

AssertionError: Expected 'generation' to have been called once. Called 0 times.

The test passed consistently locally but failed randomly in CI, blocking builds.

Root Cause

The test was creating fresh mocks to avoid state pollution, but this didn't fully isolate from the setUp method's mock configuration, leading to inconsistent behavior in CI.

Fix

Instead of creating entirely new mocks, properly reset the existing setUp mocks using .reset_mock():

self.mock_langfuse_client.reset_mock()
self.mock_langfuse_trace.reset_mock()
self.mock_langfuse_generation.reset_mock()

This ensures clean mock state while maintaining proper mock chain configuration.

Testing

poetry run pytest tests/test_litellm/integrations/test_langfuse.py::TestLangfuseUsageDetails::test_log_langfuse_v2_handles_null_usage_values -v

✅ Test passes consistently

Note

This is a clean version of PR #21073, which accidentally included unrelated changes from other branches.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 13, 2026 5:45pm

[greptile-apps]

Greptile Overview

Greptile Summary

This PR fixes a flaky test (test_log_langfuse_v2_handles_null_usage_values) by replacing the pattern of creating entirely fresh mock objects with resetting the existing setUp mocks via reset_mock(). The original approach created new mocks that bypassed the setUp mock chain, but this didn't fully isolate from setUp's configuration, causing intermittent CI failures where generation was never called.

• Replaces fresh mock creation with reset_mock() on self.mock_langfuse_client, self.mock_langfuse_trace, and self.mock_langfuse_generation
• Re-establishes the mock chain (trace -> generation, trace -> span) after reset
• Updates assertions to reference self.mock_langfuse_* instead of local mock_* variables
• Minor style note: reset_mock() does not clear the side_effect on child mocks set during setUp, so the return_value set on self.mock_langfuse_client.trace is technically shadowed by the surviving side_effect — but this is harmless since both return the same object

Confidence Score: 4/5

• This PR is safe to merge — it only modifies test code and the functional behavior is correct.
• Score of 4 reflects that this is a test-only change that correctly addresses the flaky test. The mock reset approach is sound and the test assertions are properly updated. One minor point: the side_effect from setUp survives the reset and shadows the newly-set return_value, but this is harmless since both return the same mock object. No production code is affected.
• No files require special attention — tests/test_litellm/integrations/test_langfuse.py has a minor style improvement opportunity but no functional issues.

Important Files Changed

Filename	Overview
tests/test_litellm/integrations/test_langfuse.py	Refactors test mock setup in test_log_langfuse_v2_handles_null_usage_values to use reset_mock() on existing setUp mocks instead of creating fresh mocks. The change is functionally correct and should fix the CI flakiness. Minor note: reset_mock() does not clear side_effect on child mocks, but this is harmless since the side_effect returns the same object.

Sequence Diagram

sequenceDiagram
    participant Test as Test Method
    participant Client as mock_langfuse_client
    participant Trace as mock_langfuse_trace
    participant Gen as mock_langfuse_generation

    Note over Test,Gen: Before (Original - Fresh Mocks)
    Test->>Test: Create new mock_client, mock_trace, mock_generation
    Test->>Test: logger.Langfuse = mock_client
    Test->>Test: call _log_langfuse_v2()
    Test->>Client: trace() → sometimes fails due to setUp interference

    Note over Test,Gen: After (Fixed - Reset Existing Mocks)
    Test->>Client: reset_mock()
    Test->>Trace: reset_mock()
    Test->>Gen: reset_mock()
    Test->>Trace: Re-wire generation.return_value
    Test->>Client: Re-wire trace.return_value
    Test->>Test: call _log_langfuse_v2()
    Test->>Client: trace() → reliably returns mock_langfuse_trace
    Client->>Trace: generation() → returns mock_langfuse_generation
    Test->>Trace: assert generation.assert_called_once()

Loading

_{Last reviewed commit: 610664f}

[greptile-apps]

_{1 file reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Redundant return_value due to active side_effect

reset_mock() on a parent MagicMock resets call counts recursively but does not clear side_effect or return_value on child mocks. The setUp method sets self.mock_langfuse_client.trace.side_effect = _trace_side_effect (line 56), and this side_effect survives the reset_mock() call on line 260.

When both side_effect and return_value are set on a mock, Python's unittest.mock gives side_effect precedence. So the return_value set on line 272 is effectively ignored — the side_effect from setUp is what actually runs when .trace() is called.

This happens to work correctly because _trace_side_effect returns self.mock_langfuse_trace, which is the same object being set as return_value. However, for clarity and to avoid confusion for future maintainers, consider explicitly clearing the side_effect:

Suggested change

	self.mock_langfuse_client.reset_mock()
	self.mock_langfuse_trace.reset_mock()
	self.mock_langfuse_generation.reset_mock()
	# Re-setup the trace and generation chain with clean state
	self.mock_langfuse_generation.trace_id = "test-trace-id"
	mock_span = MagicMock()
	mock_span.end = MagicMock()
	mock_trace.generation.return_value = mock_generation
	mock_trace.span.return_value = mock_span
	mock_client = MagicMock()
	mock_client.trace.return_value = mock_trace
	# Use our fresh mock client
	self.logger.Langfuse = mock_client
	self.mock_langfuse_trace.span.return_value = mock_span
	self.mock_langfuse_trace.generation.return_value = self.mock_langfuse_generation
	# Ensure trace returns our mock
	self.mock_langfuse_client.trace.return_value = self.mock_langfuse_trace
	# Reset the mock to ensure clean state
	self.mock_langfuse_client.reset_mock()
	self.mock_langfuse_trace.reset_mock()
	self.mock_langfuse_generation.reset_mock()
	# Re-setup the trace and generation chain with clean state
	self.mock_langfuse_generation.trace_id = "test-trace-id"
	mock_span = MagicMock()
	mock_span.end = MagicMock()
	self.mock_langfuse_trace.span.return_value = mock_span
	self.mock_langfuse_trace.generation.return_value = self.mock_langfuse_generation
	# Clear side_effect from setUp so return_value takes effect
	self.mock_langfuse_client.trace.side_effect = None
	self.mock_langfuse_client.trace.return_value = self.mock_langfuse_trace
	self.logger.Langfuse = self.mock_langfuse_client