Bump litellm 1.86.2 -> 1.94.0, retire four absorbed patches (#3697)

config/litellm/Dockerfile

@@ -5,20 +5,32 @@
 # stock image's Anthropic->OpenAI translation drops prompt-cache hits on
 # Qwen/DeepSeek, mis-streams reasoning models, silently drops params it does
 # not recognise, manufactures a reasoning ceiling from the caller's thinking
-# budget, never sends prior-turn reasoning back, and discards the provider's
-# own bill during stream reassembly while its rate card cannot price the route
-# either.
-# patch_litellm_cache.py closes the twelve gaps at build time
+# budget, never sends prior-turn reasoning back, loses the BYOK half of the
+# provider's bill during stream reassembly while its rate card cannot price
+# the route either, and seeds its own cost calculator with a BYOK zero that
+# suppresses the estimate.
+# patch_litellm_cache.py closes the ten gaps at build time
 # (see that script for the diagnosis);
 # cost_callback.py surfaces the resulting cost/cache stats into the pod log
 # stream. Both mirror the host-side dotfiles setup that took Qwen cache hit
 # rate from 0% to ~99.99%.
 #
-# Pinned to v1.86.2 — the version the patches and cost_callback were
-# validated against, and the same release the dotfiles host install pins.
+# Pinned to v1.94.0 — the newest stable published release, and the version
+# the patches and cost_callback are validated against (#3697). The bump from
+# v1.86.2 retired four patches upstream absorbed, narrowed a fifth, and added
+# a tenth for a regression the bump itself brings in (1.94.0 hands its cost
+# calculator the provider's `cost` even when that is a BYOK zero, which
+# suppresses the independent estimate and makes the rate-card patch dead code
+# on the streaming path).
+#
 # The patch script fails the build loudly if a needle stops matching, so a
-# future bump can't silently ship an unpatched (full-input-rate) image.
-FROM ghcr.io/berriai/litellm:v1.86.2
+# future bump can't silently ship an unpatched (full-input-rate) image. Read
+# that failure as "look at this", not as "upstream fixed it": at this bump,
+# two needles stopped matching purely because upstream reflowed the code
+# around them, and deleting those patches would have shipped an image paying
+# full input rate on every Qwen turn. Confirm each miss against the new
+# source before retiring anything.
+FROM ghcr.io/berriai/litellm:v1.94.0
 
 # Patches edit the installed litellm package (root-owned site-packages); the
 # build runs as root. The runtime pod still drops to UID 1000 via the

config/litellm/anthropic_thinking_policy.py

@@ -28,15 +28,15 @@
 Historically this never mattered: the model-cost map did not carry these slugs,
 ``OpenrouterConfig`` advertised no reasoning knobs, and ``drop_params`` silently
 discarded the synthesized param — which is precisely why these models have been
-running at full depth. Patch 7 makes the OpenRouter param gate accurate, which
+running at full depth. Patch 4 makes the OpenRouter param gate accurate, which
 is right for an *operator-configured* ``reasoning_effort`` and wrong for this
 adapter-manufactured one: it would turn a knob nobody set into the effective
-setting, with no config file mentioning it and nothing in the logs (Patch 8
+setting, with no config file mentioning it and nothing in the logs (Patch 5
 only fires on drops, and this param would no longer be dropped).
 
-So Patch 9 gates the synthesis, defaulting it off. ``thinking`` stays out of
+So Patch 6 gates the synthesis, defaulting it off. ``thinking`` stays out of
 the OpenAI body for non-Claude models and only an explicitly configured
-``reasoning_effort`` reaches the wire — exactly the property Patch 7 exists to
+``reasoning_effort`` reaches the wire — exactly the property Patch 4 exists to
 restore, without the adapter's bucket riding along.
 
 The gate covers the *derived* value only. On an adaptive request

config/litellm/cost_callback.py

@@ -28,32 +28,45 @@
 calculate_usage``, a rebuild that enumerates the token/cache counts and
 originally DROPPED the provider's ``cost`` / ``cost_details`` outright.
 That is why this module recorded ``cost: null`` on 1252 of 1252 sampled
-calls in run 6. The egg-litellm image's **patch 11** now carries those two
-fields across the rebuild (``config/litellm/stream_cost_preservation.py``),
-so the billed figure reaches ``_extract_cost`` on the streaming path too
-and this module needs no change to read it — the value simply stops being
-absent (#3691).
+calls in run 6. litellm 1.94.0 carries ``cost`` across that rebuild, and the
+egg-litellm image's **cost_details preservation patch** carries the
+``cost_details`` it still drops (``config/litellm/stream_cost_preservation.py``)
+— that is where the BYOK number lives. Between them the billed figure reaches
+``_extract_cost`` on the streaming path too, and this module needs no change to
+read it — the value simply stops being absent (#3691).
 
 ``cost: null`` therefore no longer means "streaming"; it means the cost was
-genuinely unavailable — a stock (unpatched) LiteLLM under this callback, or
-a provider that does not report one. It is still emitted as null rather
-than ``0.0``: a zero would read in the logs as "this route is free", the
-exact opposite of the cost-visibility signal this module exists to provide
-(#2799).
+genuinely unavailable — a LiteLLM older than 1.94.0 under this callback, a
+BYOK route on an unpatched image, or a provider that does not report one. It
+is still emitted as null rather than ``0.0``: a zero would read in the logs as
+"this route is free", the exact opposite of the cost-visibility signal this
+module exists to provide (#2799).
 
 Each line also carries ``cost_estimated``: LiteLLM's own ``response_cost``,
 computed at logging time from the assembled usage and its pricing map
 (issue #3175). It is kept strictly separate from ``cost`` — an estimate
 from a possibly-stale rate card must never be mistaken for a bill — and
 follows the same null-not-zero rule when LiteLLM cannot price the model.
-That was the case for every route egg uses until the image's **patch 12**
-taught the model-info lookup to read OpenRouter's published rate card; it
-remains the case for a model whose prompt-length surcharge lands on a
+That was the case for every route egg uses until the image's **live pricing
+patch** taught the model-info lookup to read OpenRouter's published rate card;
+it remains the case for a model whose prompt-length surcharge lands on a
 boundary or component LiteLLM's map has no slot for, which is declined whole
-rather than translated in part (see ``openrouter_capabilities``). With
-both patches in place the two fields are independent measurements of the
-same turn, and a persistent gap between them is a signal in its own right:
-a stale rate card, an unexpected provider, or a surcharge tier.
+rather than translated in part (see ``openrouter_capabilities``).
+
+The two fields are independent measurements ONLY WHERE THE PROVIDER REPORTS NO
+CHARGE, which on egg's BYOK routes is every call. From 1.94.0 litellm copies a
+reported ``usage.cost`` into ``_hidden_params`` and ``response_cost_calculator``
+returns that value directly, so where a provider does bill a positive figure
+``cost_estimated`` mirrors ``cost`` rather than pricing the turn independently.
+The image's **billable-cost gate patch** narrows that propagation to a positive,
+finite charge — without it a BYOK ``cost: 0.0`` would be handed to the
+calculator, ``cost_estimated`` would read null on ~100% of egg's traffic, and
+the live pricing patch would be dead code on the streaming path (it lives inside
+``completion_cost()``, which the short circuit returns before). So on a BYOK
+route the two fields remain independent and a persistent gap between them is a
+signal in its own right — a stale rate card, an unexpected provider, or a
+surcharge tier — while on a route that reports a real charge, agreement between
+them is arithmetic, not corroboration.
 
 Each line also carries ``request_params``: the decoding configuration that
 actually went upstream on that call (issue #3599). Repetition and
@@ -165,9 +178,10 @@ def _usage_from_response_obj(response_obj):
     counts (the final usage chunk's counts are folded into
     ``response_obj.usage`` by ``stream_chunk_builder``) and, on the egg-litellm
     image, for cost as well: that reassembly rebuilds a fresh ``Usage`` and
-    stock drops ``cost`` / ``cost_details`` with it, which patch 11 restores.
-    Under a stock LiteLLM the counts still arrive and ``_extract_cost`` returns
-    None — see the module docstring."""
+    older releases dropped ``cost`` / ``cost_details`` with it; 1.94.0 carries
+    ``cost`` and the image's cost_details preservation patch supplies
+    ``cost_details``. Under a stock LiteLLM the counts still arrive and
+    ``_extract_cost`` returns None — see the module docstring."""
     if response_obj is None:
         return None
     usage = getattr(response_obj, "usage", None)
@@ -195,8 +209,8 @@ def _extract_cost(usage):
     provider, so fall back to ``cost_details.upstream_inference_cost`` (what
     the upstream provider will bill for the same request). Either way, the
     number we record matches real spend on that turn. Returns None when no
-    positive cost is present — a provider that reports none, or a stock LiteLLM
-    whose chunk reassembly drops it (see ``_usage_from_response_obj``).
+    positive cost is present — a provider that reports none, or a LiteLLM whose
+    chunk reassembly drops it (see ``_usage_from_response_obj``).
     Callers must treat None as "unknown", not "$0".
 
     ``_positive`` rejects non-finite values as well as non-positive ones: a
@@ -322,11 +336,22 @@ def _extract_estimated_cost(mcd):
     reported a bill. It is an estimate, not a bill: the pricing map may lag the
     provider's rates or lack cache-discount entries for a model. Returns None —
     never 0.0 — when LiteLLM couldn't price the call, mirroring the billed-cost
-    "unknown ≠ free" discipline. On the egg-litellm image patch 12 supplies
-    OpenRouter's published rates for slugs the bundled map does not carry, so a
-    None here now means a genuinely unpriceable model (an inexpressible
-    prompt-length surcharge, or a provider with no live card to read) rather
-    than the routine case it was.
+    "unknown ≠ free" discipline. On the egg-litellm image the live pricing patch
+    supplies OpenRouter's published rates for slugs the bundled map does not
+    carry, so a None here now means a genuinely unpriceable model (an
+    inexpressible prompt-length surcharge, or a provider with no live card to
+    read) rather than the routine case it was.
+
+    "Independently of whether the provider reported a bill" holds only where the
+    provider reported no charge — which on egg's BYOK routes is every call. From
+    1.94.0, litellm copies a reported ``usage.cost`` into ``_hidden_params`` and
+    ``response_cost_calculator`` returns it before consulting the pricing map at
+    all, so on a route that does bill a positive figure this returns ``cost``
+    itself and the two fields stop being independent. The image's billable-cost
+    gate patch narrows that propagation to a positive, finite charge — it is
+    also what keeps the live pricing patch reachable on the streaming path,
+    since that patch lives inside ``completion_cost()`` and the short circuit
+    returns before it.
 
     Reads the top-level ``response_cost`` first, then falls back to
     ``standard_logging_object.response_cost`` — the latter is LiteLLM's
@@ -376,10 +401,11 @@ def _extract_model(mcd):
 #
 # ``stream`` is included because it selects which of the two paths the cost on
 # this line came through (see the module docstring), and it was the reason
-# ``cost`` read null on every line before patch 11 — worth having next to the
-# number rather than inferred, and worth keeping now that the null case is rare
-# enough to need explaining when it happens. ``max_tokens`` and ``n`` are not
-# sampling knobs but shape the generation, and are cheap to carry.
+# ``cost`` read null on every line before 1.94.0 and the cost_details
+# preservation patch — worth having next to the number rather than inferred, and
+# worth keeping now that the null case is rare enough to need explaining when it
+# happens. ``max_tokens`` and ``n`` are not sampling knobs but shape the
+# generation, and are cheap to carry.
 _REQUEST_PARAM_KEYS = (
     "temperature",
     "top_p",
@@ -643,7 +669,7 @@ def _extract_request_params(mcd):
 
     Source is ``model_call_details['optional_params']``: LiteLLM's
     POST-mapping parameter set, i.e. the dict that becomes the upstream
-    request body. Verified against the pinned litellm 1.86.2 on the
+    request body. Verified against the pinned litellm 1.94.0 on the
     ``anthropic_messages`` route (the route every Claude Code agent request
     takes), streaming and non-streaming — the streaming case matters most,
     since that is essentially all real agent traffic.
@@ -845,7 +871,7 @@ def _record(self, mcd, response_obj):
                     # /v1/messages route (#3624): stock LiteLLM rewrites the
                     # caller's ``thinking`` block into a bucketed
                     # ``reasoning_effort``, but that bucket is a cap below the
-                    # model default, so egg-litellm's patch 9 gates the
+                    # model default, so egg-litellm's patch 6 gates the
                     # synthesis off by default. A missing key here means the
                     # request ran at the model's own reasoning depth, not that
                     # the field failed to record.

config/litellm/drop_params_visibility.py

@@ -2,7 +2,7 @@
 
 ``drop_params`` exists so an unsupported parameter does not fail the whole
 request, and that tradeoff is right. But dropping a parameter *changes
-generation behaviour*, and in stock LiteLLM 1.86.2 it happens with no signal
+generation behaviour*, and in stock LiteLLM it happens with no signal
 at all: the branch that pops them is a bare loop with no logging. A
 ``reasoning_effort``, ``temperature`` or penalty set in a proxy config simply
 never reaches the provider, and nothing in the logs or the response says so.
@@ -15,14 +15,14 @@
 to notice (jwbron/egg#3620, #3624). One log line would have made it a
 five-minute question.
 
-Patch 7 fixes the OpenRouter false-negative specifically; this covers the rest.
+Patch 4 fixes the OpenRouter false-negative specifically; this covers the rest.
 A drop can still be *correct* and worth knowing about: ``poolside/laguna-s-2.1``
 genuinely does not accept ``reasoning_effort``, so the knob is dropped on
 purpose, and without this the operator has no way to learn why their config
 line does nothing.
 
 Mirrors jwbron/litellm#7 (merged into the fork the host proxy runs). The
-cluster image pins stock 1.86.2, which predates it, hence this patch.
+cluster image pins a stock release, which does not carry it, hence this patch.
 """
 
 # Warn-once bookkeeping, keyed by (provider, model, sorted dropped param

config/litellm/openrouter_capabilities.py

@@ -33,7 +33,7 @@
   cost: ``supports_reasoning: true`` alone makes stock
   ``get_supported_openai_params`` admit ``thinking``, which Patch 2's notes
   explain would forward an Anthropic-shaped block verbatim to a provider that
-  expects ``reasoning``. Patch 7 remains the only path by which a parameter
+  expects ``reasoning``. Patch 4 remains the only path by which a parameter
   becomes admissible, and it admits exactly ``reasoning_effort``.
 * **Tiered rate cards are translated only where LiteLLM can hold them.**
   OpenRouter expresses a long-context surcharge as ``pricing.overrides`` — a
@@ -79,7 +79,7 @@
   both, since one fetch serves both.
 * ``LITELLM_OPENROUTER_PRICING=0`` disables only the pricing half, leaving the
   parameter lookup running. For an operator who wants LiteLLM's bundled map to
-  be the sole authority on cost while keeping Patch 7's parameter fix.
+  be the sole authority on cost while keeping Patch 4's parameter fix.
 * ``LITELLM_OPENROUTER_CAPABILITY_TTL`` seconds between refreshes
   (default 3600). ``0`` disables caching and re-fetches on every lookup —
   a debugging aid, not a production setting.
@@ -584,7 +584,7 @@ def _log_fetch_failure(message: str, *args: object) -> None:
     A permanently unreachable endpoint is the exact case where behaviour
     silently reverts to the model-cost map, and litellm's default log level is
     INFO — so debug-only reporting reproduces, one file over, the silence
-    Patch 8 exists to remove. Warning once is enough to be findable without
+    Patch 5 exists to remove. Warning once is enough to be findable without
     turning an offline deployment into a log flood.
     """
     global _WARNED_FETCH_FAILURE