feat(typesafe): backport #41607 to stable/1.99.x for v1.99.2

[devin-ai-integration]

TLDR

Problem this solves:

• TypeSafe Jev only ships in v1.103.0-rc, users on stable 1.99.x cannot use it
• Customers want it on a stable line without moving to an rc in prod

How it solves it:

• Cherry-picks feat(proxy): add TypeSafe AI Jev evaluate passthrough with registry-priced spend tracking #41607 (main sha deb9d8a) onto stable/1.99.x for v1.99.2
• Adds the /typesafe/* passthrough route with logging and cost tracking
• Adds typesafe/jev-latest, typesafe/jev-preview, typesafe/jev-1.13.0 pricing entries
• No version bump in this PR; the release bump is handled separately

Included PRs:

• feat(proxy): add TypeSafe AI Jev evaluate passthrough with registry-priced spend tracking #41607 feat(typesafe): TypeSafe Jev passthrough with logging and cost tracking

User Flow

Before: a developer on stable 1.99.x points their TypeSafe client at the proxy and every call is refused

1. The admin sets TYPESAFE_API_KEY and starts the proxy from stable/1.99.x
2. The developer sends POST http://localhost:4000/typesafe/v1/systemone with a model: jev-latest payload
3. The proxy answers 404 or 401, the request never reaches TypeSafe
4. http://localhost:4000/ui/?page=logs shows no TypeSafe request

After: the same call is forwarded to TypeSafe and logged with spend

1. The admin sets TYPESAFE_API_KEY and starts the proxy from this branch
2. The developer sends POST http://localhost:4000/typesafe/v1/systemone with the same payload
3. The proxy answers 200 with the TypeSafe response body
4. http://localhost:4000/ui/?page=logs shows the request under typesafe/jev-latest with non-zero spend

Relevant issues

Backport of #41607

Affected release

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Live proxy started with python litellm/proxy/proxy_cli.py --config config.yaml --port 4000 --detailed_debug (master key sk-1234, TYPESAFE_API_KEY set from the shared vault) at each sha, one real call to https://api.typesafe.ai per leg

Before (3c08f26, stable/1.99.x merge base)

POST /typesafe/v1/systemone

curl -s -w "\nHTTP %{http_code}\n" -X POST http://localhost:4000/typesafe/v1/systemone
-H "Authorization: Bearer sk-1234" -H "Content-Type: application/json"
-d '{"state":"Help! My payouts have been failing for 3 days.","model":"jev-latest","questions":{"department":{"type":"choice","instructions":"Which team should handle this?","criteria":{"billing":"Payments, invoicing, refunds","technical":"Bugs, outages, integrations","sales":"Pricing, upgrades, new accounts"}}}}'

2. ```
{"detail":"Not Found"}
HTTP 404

After (7d1db43, PR tip)

POST /typesafe/v1/systemone

1. Same curl as above

{"model":"jev-1.13.0","answers":{"department":{"type":"choice","choice":"billing","confidence":0.83,"probabilities":{"sales":0.0,"billing":0.88,"technical":0.12}}},"usage":{"input_tokens":349,"output_tokens":38}}
HTTP 200

3. Proxy log after the call shows spend tracked against `typesafe/jev-1.13.0`:

response_cost: 1.4658e-05

## Type

🆕 New Feature

## Caveats (if any)

## Final Attestation

- [x] The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR


Link to Devin session: https://app.devin.ai/sessions/b6d5b6c20aec42babbd9213ec27ef3bd
Open in Devin Desktop: https://app.devin.ai/desktop/session/b6d5b6c20aec42babbd9213ec27ef3bd?variant=devin
Requested by: @mateo-berri

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Introduces a new authenticated passthrough path and spend attribution logic; misconfigured `TYPESAFE_API_KEY` or pricing keys could affect billing visibility, but scope is isolated to the new provider route.
> 
> **Overview**
> Adds **TypeSafe AI** as a first-class proxy passthrough provider so stable `1.99.x` can forward `/typesafe/*` to the TypeSafe API (backport of #41607).
> 
> The proxy registers **`/typesafe/{endpoint:path}`** (GET/POST), whitelists `/typesafe/` on the gateway and in mapped passthrough routes, and proxies to `TYPESAFE_API_BASE` (default `https://api.typesafe.ai`) with **`TYPESAFE_API_KEY`** injected as `Authorization: Bearer`. Passthrough success logging routes **`custom_llm_provider=typesafe`** through a new handler that parses response/request `model` and `usage`, maps models as `typesafe/{model}`, and computes **spend** from the model cost registry.
> 
> The cost map gains **`typesafe/jev-1.13.0`**, **`jev-latest`**, and **`jev-preview`** with **`mode: evaluation`**, and types/schema enums are extended for that mode. Tests cover the route wiring, logging/cost edge cases, and metadata; OpenAPI/UI schema includes the new endpoint.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 7d1db43c6883bf4ad61806547b64eef13aa36010. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge because no outstanding correctness, security, or repository-rule issue remains.

Summary

Backports TypeSafe AI passthrough support to the stable 1.99.x line.

• Adds the authenticated /typesafe/{endpoint} GET/POST passthrough route.
• Adds TypeSafe response normalization, usage logging, and registry-based cost attribution.
• Registers TypeSafe Jev model pricing and the evaluation model mode.
• Updates generated API types and adds route, logging, pricing, and metadata tests.

_{Reviews (2) · Last reviewed commit: "feat(typesafe): add TypeSafe Jev passthr..."}

[mateo-berri]

bugbot run

[devin-ai-integration]

@greptileai

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 7d1db43. Configure here.}

[mateo-berri]

LGTM