Litellm oss staging 02 16 2026

[krrishdholakia]

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 18, 2026 0:09am

[greptile-apps]

Greptile Summary

This staging PR bundles four distinct fixes/features:

• Fix: Prevent double-counting of litellm_proxy_total_requests_metric — The Prometheus metric was being incremented in both async_log_success_event (for streaming) and async_post_call_success_hook (for all successes), causing non-streaming requests to be counted twice. The fix consolidates the increment into async_log_success_event only (for both streaming and non-streaming) and makes async_post_call_success_hook a no-op. A complementary fix in _init_litellm_callbacks replaces string callbacks in-place in litellm.callbacks instead of appending initialized instances alongside the original strings, which was another source of duplicate callbacks.

• Feature: Generic Guardrail API fail-open fallback — Adds a configurable unreachable_fallback parameter (fail_closed default, fail_open option) to the generic guardrail API. When set to fail_open, network errors (httpx.RequestError, litellm.Timeout) and upstream proxy errors (HTTP 502/503/504) allow the request to proceed with a critical log instead of blocking. Includes documentation and example config updates.

• Fix: finish_reason='tool_calls' for response.completed with function calls — When streaming responses from codex models (GPT-5.1-codex), response.completed events that contained function_call items in their output always returned finish_reason='stop'. Agents like OpenCode need finish_reason='tool_calls' to know tools should be executed. The fix inspects the response.completed output items and returns the correct finish reason.

• Fix: Preserve metadata for custom callbacks on codex/responses path — When codex models route through the responses API bridge, metadata was passed as litellm_metadata in kwargs but not forwarded to update_environment_variables. The fix falls back to kwargs.get('litellm_metadata') when metadata is None.

Confidence Score: 4/5

• This PR is safe to merge — each fix is well-scoped with corresponding tests, and the changes align with the existing codebase patterns.
• The PR bundles four independent changes, each with targeted tests. The Prometheus double-counting fix and callback deduplication fix work together correctly. The guardrail fail-open feature is well-implemented with proper error categorization. The only minor concerns are stylistic (lost exception chains when re-raising). All test changes are mock-based and verify the intended behavior.
• Pay close attention to litellm/proxy/utils.py (callback initialization logic change) and litellm/integrations/prometheus.py (metric increment consolidation) as they affect metric accuracy across the entire proxy.

Important Files Changed

Filename	Overview
litellm/proxy/utils.py	Fixes double-counting of metrics by replacing string callbacks in-place in litellm.callbacks instead of appending initialized instances alongside the original strings. No longer calls add_litellm_callback for each entry.
litellm/integrations/prometheus.py	Moves litellm_proxy_total_requests_metric increment to async_log_success_event for ALL requests (not just streaming), and removes it from async_post_call_success_hook to prevent double-counting. The hook method is now a no-op pass.
litellm/proxy/guardrails/guardrail_hooks/generic_guardrail_api/generic_guardrail_api.py	Adds configurable unreachable_fallback (fail_closed/fail_open) for the generic guardrail API. Adds httpx.HTTPStatusError (502/503/504), httpx.RequestError, and litellm.Timeout exception handling with fail-open support. Extracts helper methods _fail_open_passthrough and _build_guardrail_return_inputs.
litellm/completion_extras/litellm_responses_transformation/transformation.py	Fixes finish_reason in response.completed streaming event to return 'tool_calls' when the response contains function_call items (instead of always 'stop'). Also includes formatting/whitespace changes.
litellm/responses/main.py	Preserves metadata for custom callbacks when called from completion bridge (codex models) by falling back to kwargs.get('litellm_metadata') when metadata is None.
litellm/types/guardrails.py	Adds unreachable_fallback field to BaseLitellmParams with 'fail_closed' default. Also adds it to LitellmParams validator.
litellm/types/proxy/guardrails/guardrail_hooks/generic_guardrail_api.py	Adds unreachable_fallback field to GenericGuardrailAPIOptionalParams. Minor formatting change for litellm_trace_id.
tests/litellm/proxy/test_init_litellm_callbacks.py	New test file validating that _init_litellm_callbacks replaces string callbacks in-place rather than creating duplicates. Tests cover replacement, no-duplication of instances, unrecognized strings, and multiple callbacks. All mock-based.
tests/test_litellm/responses/test_metadata_codex_callback.py	New test file validating metadata preservation for custom callbacks when using codex models (responses API bridge). Tests use HTTP mocking with AsyncMock.

Flowchart

flowchart TD
    subgraph Before["Before (Double-Counting Bug)"]
        A1[Successful Request] --> B1[async_log_success_event]
        B1 -->|streaming only| C1[Inc total_requests_metric]
        A1 --> D1[async_post_call_success_hook]
        D1 -->|all requests| E1[Inc total_requests_metric]
        C1 -.- F1[Non-streaming: 1x count ✓]
        E1 -.- F1
        C1 -.- G1[Streaming: 2x count ✗]
        E1 -.- G1
    end

    subgraph After["After (Fix)"]
        A2[Successful Request] --> B2[async_log_success_event]
        B2 -->|all requests| C2[Inc total_requests_metric]
        A2 --> D2[async_post_call_success_hook]
        D2 -->|pass / no-op| E2[No metric increment]
        C2 -.- F2[All requests: 1x count ✓]
    end

    subgraph Guardrail["Generic Guardrail API - Fail Open/Closed"]
        G[Guardrail API Call] -->|Success| H[Process Response]
        G -->|Timeout / RequestError| I{unreachable_fallback?}
        G -->|HTTP 502/503/504| I
        I -->|fail_closed| J[Raise Exception]
        I -->|fail_open| K[Log Critical + Passthrough]
    end

Loading

_{Last reviewed commit: 504c70f}

[greptile-apps]

_{16 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Non-5xx errors swallowed as generic Exception
When unreachable_fallback="fail_closed" (or any HTTP error that isn't 502/503/504), the original httpx.HTTPStatusError is caught and re-raised as a bare Exception(f"Generic Guardrail API failed: {str(e)}"). This discards the HTTP status code, response body, and exception chain, making debugging harder.

Consider preserving the original exception with raise ... from e, and ideally including the status code in the message:

Suggested change

	except httpx.HTTPStatusError as e:
	# Common reverse-proxy/LB failures can present as HTTP errors even when the backend is unreachable.
	status_code = getattr(getattr(e, "response", None), "status_code", None)
	if self.unreachable_fallback == "fail_open" and status_code in (
	502,
	503,
	504,
	):
	return self._fail_open_passthrough(
	inputs=inputs,
	input_type=input_type,
	logging_obj=logging_obj,
	error=e,
	http_status_code=status_code,
	)
	verbose_proxy_logger.error(
	"Generic Guardrail API: failed to make request: %s", str(e)
	)
	raise Exception(f"Generic Guardrail API failed: {str(e)}")
	status_code = getattr(getattr(e, "response", None), "status_code", None)
	if self.unreachable_fallback == "fail_open" and status_code in (
	502,
	503,
	504,
	):
	return self._fail_open_passthrough(
	inputs=inputs,
	input_type=input_type,
	logging_obj=logging_obj,
	error=e,
	http_status_code=status_code,
	)
	verbose_proxy_logger.error(
	"Generic Guardrail API: failed to make request: %s", str(e)
	)
	raise Exception(f"Generic Guardrail API failed (HTTP {status_code}): {str(e)}") from e

[greptile-apps]

Exception chain lost when re-raising
Same as the httpx.HTTPStatusError handler above — use raise ... from e to preserve the exception chain for httpx.RequestError, which aids debugging (e.g., distinguishing DNS resolution failures from connection resets).

Suggested change

	verbose_proxy_logger.error(
	"Generic Guardrail API: failed to make request: %s", str(e)
	)
	raise Exception(f"Generic Guardrail API failed: {str(e)}")
	raise Exception(f"Generic Guardrail API failed: {str(e)}") from e