diff --git a/docs/pass_through/azure_passthrough.md b/docs/pass_through/azure_passthrough.md index cac063335..2212bad88 100644 --- a/docs/pass_through/azure_passthrough.md +++ b/docs/pass_through/azure_passthrough.md @@ -6,7 +6,7 @@ Pass-through endpoints for `/azure` | Feature | Supported | Notes | |-------|-------|-------| -| Cost Tracking | ❌ | Not supported | +| Cost Tracking | ✅ | v1 API paths only: `/v1/chat/completions`, `/v1/embeddings`, `/v1/images/generations`, `/v1/images/edits`, `/v1/responses`. Deployments-style paths (`/openai/deployments/...`) are logged without cost | | Logging | ✅ | Works across all integrations | | Streaming | ✅ | Fully supported | diff --git a/docs/pass_through/intro.md b/docs/pass_through/intro.md index 38218224f..f940b061e 100644 --- a/docs/pass_through/intro.md +++ b/docs/pass_through/intro.md @@ -49,5 +49,5 @@ graph LR - **Unified Authentication**: One API key for all providers - **Centralized Logging**: All requests logged through LiteLLM -- **Cost Tracking**: Usage tracked across all endpoints +- **Cost Tracking**: Tracked where the provider page's overview table says so; other passthrough requests are logged without cost - **Access Control**: Same permissions apply to passthrough endpoints diff --git a/docs/pass_through/openai_passthrough.md b/docs/pass_through/openai_passthrough.md index 49026f8aa..ae6ba0281 100644 --- a/docs/pass_through/openai_passthrough.md +++ b/docs/pass_through/openai_passthrough.md @@ -6,7 +6,7 @@ Pass-through endpoints for direct OpenAI API access | Feature | Supported | Notes | |-------|-------|-------| -| Cost Tracking | ❌ | Not supported | +| Cost Tracking | ✅ | Chat completions, embeddings, image generations, image edits, and the Responses API. Other endpoints are logged without cost | | Logging | ✅ | Works across all integrations | | Streaming | ✅ | Fully supported | @@ -37,12 +37,26 @@ Simply replace `https://api.openai.com` with `LITELLM_PROXY_BASE_URL/openai_pass Requirements: Set `OPENAI_API_KEY` in your environment variables. +### Embeddings + +Spend from passthrough embeddings calls is tracked and attributed to the calling key, just like the native `/embeddings` route + +```bash +curl http://0.0.0.0:4000/openai_passthrough/v1/embeddings \ + -H "Authorization: Bearer sk-anything" \ + -H "Content-Type: application/json" \ + -d '{ + "model": "text-embedding-3-small", + "input": "hello world" + }' +``` + ### Assistants API #### Create OpenAI Client Make sure you do the following: -- Point `base_url` to your `LITELLM_PROXY_BASE_URL/openai` +- Point `base_url` to your `LITELLM_PROXY_BASE_URL/openai_passthrough` - Use your `LITELLM_API_KEY` as the `api_key` ```python