[April 6th] - Ishaan

docs/my-website/docs/proxy/users.md

@@ -333,6 +333,67 @@ curl 'http://0.0.0.0:4000/key/generate' \
 }'
 ```
 
+#### **Set multiple budget windows on a key**
+
+Apply multiple concurrent budget limits at different time scales on the same key — for example, cap a key at **$10/day** AND **$100/month**.
+
+**When is this useful?**
+
+A single `budget_duration` window can't prevent a bad day from burning your entire month. Multiple budget windows let you:
+
+- Block a runaway usage spike within the day while still allowing normal monthly spend.
+- Give Claude Code rollouts a daily guardrail (`24h`) and a monthly ceiling (`30d`) so a single heavy session doesn't exhaust the whole month.
+- Layer fine-grained hourly limits for bursty workloads on top of a weekly cap.
+
+:::info
+
+See [User Budget docs](https://docs.litellm.ai/docs/proxy/users) for more on how budgets work across keys, teams, and users.
+
+:::
+
+**Via API**
+
+Pass `budget_limits` as a list of `{budget_duration, max_budget}` objects:
+
+```bash
+curl 'http://0.0.0.0:4000/key/generate' \
+--header 'Authorization: Bearer <your-master-key>' \
+--header 'Content-Type: application/json' \
+--data-raw '{
+  "budget_limits": [
+    {"budget_duration": "24h",  "max_budget": 10},
+    {"budget_duration": "30d",  "max_budget": 100}
+  ]
+}'
+```
+
+Each window is tracked independently and resets on its own schedule:
+
+| `budget_duration` | Resets |
+|---|---|
+| `1h`  | Every hour |
+| `24h` | Daily at midnight UTC |
+| `7d`  | Every Sunday at midnight UTC |
+| `30d` | 1st of every month at midnight UTC |
+
+**Via Dashboard**
+
+Open **Virtual Keys → Create Key → Optional Settings → Budget Windows**.
+
+![Step 1 - open key settings](https://colony-recorder.s3.amazonaws.com/files/2026-04-01/18930ba5-67c0-4031-afc0-57f37b4e59e4/ascreenshot_ef79d8a000bb41cdacf1bd9827732ee8_text_export.jpeg)
+
+Click **+ Add Budget Window** to add a row, choose the period from the dropdown, and enter the spend cap.
+
+![Step 2 - add a window](https://colony-recorder.s3.amazonaws.com/files/2026-04-01/5ae8c0b3-2d03-41ad-a63c-47b20c350dfe/ascreenshot_1a7dc6c7d65544f38fd8a65604674f22_text_export.jpeg)
+
+Add a second row for a different time period (e.g. monthly $100 on top of a daily $10).
+
+![Step 3 - add second window](https://colony-recorder.s3.amazonaws.com/files/2026-04-01/cbded3a7-1086-4e20-8f0f-de154b76146c/ascreenshot_c51c18752c3b4f8b976d28799b2638b6_text_export.jpeg)
+
+Each window shows the reset schedule below the input so it's always clear when spend resets.
+
+![Step 4 - reset hints](https://colony-recorder.s3.amazonaws.com/files/2026-04-01/8754f121-1640-4892-9dd0-fd4a870418bf/ascreenshot_8079eb0df2194e8f99e5258ba4b3c082_text_export.jpeg)
+
 
 ### ✨ Virtual Key (Model Specific)
 

docs/my-website/docs/skills_gateway.md

@@ -0,0 +1,111 @@
+# Skills Gateway
+
+<iframe width="840" height="500" src="https://www.loom.com/embed/cb74eb79df3e4c2b83a6efae54a589f9" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
+
+LiteLLM acts as a **Skills Registry** — a central place to register, manage, and discover Claude Code skills across your organization. Teams can publish skills once and have agents and developers find them through a single hub.
+
+## How it works
+
+```mermaid
+graph TD
+    Dev["👨‍💻 Developer<br/>registers a skill<br/>(GitHub URL or subdir)"] -->|POST /claude-code/plugins| Proxy["LiteLLM Proxy<br/>(Skills Registry)"]
+
+    Admin["🔑 Admin<br/>publishes skill<br/>(marks as public)"] -->|enable via UI or API| Proxy
+
+    Proxy -->|GET /public/skill_hub| SkillHub["🗂️ Skill Hub<br/>(AI Hub → Skill Hub tab)"]
+    Proxy -->|GET /claude-code/marketplace.json| Marketplace["📦 Claude Code<br/>Marketplace endpoint"]
+
+    SkillHub --> Human["🧑 Human<br/>browses & discovers skills<br/>in AI Hub UI"]
+    Marketplace --> Agent["🤖 Agent / Claude Code<br/>installs skill with<br/>/plugin marketplace add &lt;name&gt;"]
+
+    style Proxy fill:#1a73e8,color:#fff
+    style SkillHub fill:#e8f0fe,color:#1a73e8
+    style Marketplace fill:#e8f0fe,color:#1a73e8
+```
+
+## Quick start
+
+### 1. Register a skill
+
+Paste any GitHub URL into the Skills UI — LiteLLM auto-detects the source type and skill name.
+
+```bash
+curl -X POST https://your-proxy/claude-code/plugins \
+  -H "Authorization: Bearer $LITELLM_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "name": "grill-me",
+    "source": {
+      "source": "git-subdir",
+      "url": "https://github.com/mattpocock/skills",
+      "path": "grill-me"
+    },
+    "description": "Interview skill for relentless questioning",
+    "domain": "Productivity",
+    "namespace": "interviews"
+  }'
+```
+
+Skills nested in subdirectories (e.g. `github.com/org/repo/tree/main/skill-name`) are supported — LiteLLM parses the URL automatically in the UI.
+
+### 2. Publish to hub
+
+In the Admin UI: **AI Hub → Skill Hub → Select Skills to Make Public**.
+
+Or via API:
+
+```bash
+curl -X POST https://your-proxy/claude-code/plugins/grill-me/enable \
+  -H "Authorization: Bearer $LITELLM_KEY"
+```
+
+### 3. Browse the hub
+
+Public skills appear at:
+- **Admin UI**: AI Hub → Skill Hub tab
+- **Public page**: `/ui/model_hub` → Skill Hub tab (no login required)
+- **API**: `GET /public/skill_hub`
+
+### 4. Install in Claude Code
+
+Point Claude Code at your proxy marketplace once:
+
+```json title="~/.claude/settings.json"
+{
+  "extraKnownMarketplaces": {
+    "my-org": {
+      "source": "url",
+      "url": "https://your-proxy/claude-code/marketplace.json"
+    }
+  }
+}
+```
+
+Then install any skill:
+
+```
+/plugin marketplace add grill-me
+```
+
+## Skill fields
+
+| Field | Description |
+|-------|-------------|
+| `name` | Unique skill identifier (used in `/plugin marketplace add`) |
+| `source` | Git source — `github`, `url`, or `git-subdir` |
+| `description` | Short description shown in the hub |
+| `domain` | Category for grouping (e.g. `Engineering`, `Productivity`) |
+| `namespace` | Subcategory within a domain (e.g. `quality`, `meetings`) |
+| `keywords` | Tags for search and filtering |
+| `version` | Semver string |
+
+## API reference
+
+| Endpoint | Auth | Description |
+|----------|------|-------------|
+| `POST /claude-code/plugins` | Required | Register a skill |
+| `GET /claude-code/plugins` | Required | List all skills (admin) |
+| `POST /claude-code/plugins/{name}/enable` | Required | Publish a skill |
+| `POST /claude-code/plugins/{name}/disable` | Required | Unpublish a skill |
+| `GET /public/skill_hub` | None | List public skills |
+| `GET /claude-code/marketplace.json` | None | Claude Code marketplace manifest |

docs/my-website/sidebars.js

@@ -333,6 +333,13 @@ const sidebars = {
                 },
               ],
             },
+            {
+              type: "category",
+              label: "Skills Gateway",
+              items: [
+                "skills_gateway",
+              ],
+            },
           ],
         },
         {

enterprise/litellm_enterprise/proxy/common_utils/check_responses_cost.py

@@ -11,6 +11,7 @@
 from litellm.constants import (
     MANAGED_OBJECT_STALENESS_CUTOFF_DAYS,
     MAX_OBJECTS_PER_POLL_CYCLE,
+    STALE_OBJECT_CLEANUP_BATCH_SIZE,
 )
 
 if TYPE_CHECKING:
@@ -32,21 +33,49 @@ def __init__(
         self.prisma_client: PrismaClient = prisma_client
         self.llm_router: Router = llm_router
 
+    async def _expire_stale_rows(
+        self, cutoff: datetime, batch_size: int
+    ) -> int:
+        """Execute the bounded UPDATE that marks stale rows as 'stale_expired'.
+
+        Isolated so it can be swapped / mocked in tests without touching the
+        orchestration logic in ``_cleanup_stale_managed_objects``.
+
+        Uses PostgreSQL syntax (``$1::timestamptz``, ``LIMIT``, double-quoted
+        identifiers) which is the only dialect the proxy supports — every
+        ``schema.prisma`` in the repo sets ``provider = "postgresql"``.
+        Same pattern as ``spend_log_cleanup.py``.
+        """
+        return await self.prisma_client.db.execute_raw(
+            """
+            UPDATE "LiteLLM_ManagedObjectTable"
+            SET "status" = 'stale_expired'
+            WHERE "id" IN (
+                SELECT "id" FROM "LiteLLM_ManagedObjectTable"
+                WHERE "file_purpose" = 'response'
+                AND "status" NOT IN ('completed', 'complete', 'failed', 'expired', 'cancelled', 'stale_expired')
+                AND "created_at" < $1::timestamptz
+                ORDER BY "created_at" ASC
+                LIMIT $2
+            )
+            """,
+            cutoff,
+            batch_size,
+        )
+
     async def _cleanup_stale_managed_objects(self) -> None:
         """
         Mark managed objects older than MANAGED_OBJECT_STALENESS_CUTOFF_DAYS days
         in non-terminal states as 'stale_expired'. These will never complete and
         should not be polled.
+
+        Runs as a single DB query with a subquery LIMIT so no rows are loaded
+        into Python memory. Processes at most STALE_OBJECT_CLEANUP_BATCH_SIZE
+        rows per invocation to avoid overwhelming the DB when there is a large
+        backlog.
         """
         cutoff = datetime.now(timezone.utc) - timedelta(days=MANAGED_OBJECT_STALENESS_CUTOFF_DAYS)
-        result = await self.prisma_client.db.litellm_managedobjecttable.update_many(
-            where={
-                "file_purpose": "response",
-                "status": {"not_in": ["completed", "complete", "failed", "expired", "cancelled", "stale_expired"]},
-                "created_at": {"lt": cutoff},
-            },
-            data={"status": "stale_expired"},
-        )
+        result = await self._expire_stale_rows(cutoff, STALE_OBJECT_CLEANUP_BATCH_SIZE)
         if result > 0:
             verbose_proxy_logger.warning(
                 f"CheckResponsesCost: marked {result} stale managed objects "

litellm-proxy-extras/litellm_proxy_extras/migrations/20260401000000_add_budget_limits/migration.sql

@@ -0,0 +1,5 @@
+-- AlterTable: add budget_limits column to LiteLLM_VerificationToken
+ALTER TABLE "LiteLLM_VerificationToken" ADD COLUMN IF NOT EXISTS "budget_limits" JSONB;
+
+-- AlterTable: add budget_limits column to LiteLLM_TeamTable
+ALTER TABLE "LiteLLM_TeamTable" ADD COLUMN IF NOT EXISTS "budget_limits" JSONB;

litellm-proxy-extras/litellm_proxy_extras/migrations/20260401000000_add_team_member_model_scope/migration.sql

@@ -0,0 +1,9 @@
+-- Add per-member model scope to LiteLLM_BudgetTable
+-- allowed_models: empty array = inherit team models; non-empty = enforce member-level restriction
+ALTER TABLE "LiteLLM_BudgetTable"
+  ADD COLUMN IF NOT EXISTS "allowed_models" TEXT[] DEFAULT ARRAY[]::TEXT[];
+
+-- Add default_team_member_models to LiteLLM_TeamTable
+-- Seeds allowed_models for newly added team members; empty = no per-member restriction
+ALTER TABLE "LiteLLM_TeamTable"
+  ADD COLUMN IF NOT EXISTS "default_team_member_models" TEXT[] DEFAULT ARRAY[]::TEXT[];

litellm-proxy-extras/litellm_proxy_extras/schema.prisma

@@ -17,8 +17,9 @@ model LiteLLM_BudgetTable {
   tpm_limit     BigInt?
   rpm_limit     BigInt?
   model_max_budget Json?
-  budget_duration String? 
+  budget_duration String?
   budget_reset_at DateTime?
+  allowed_models String[] @default([]) // per-member model scope; empty = inherit team models
   created_at    DateTime               @default(now()) @map("created_at")
   created_by String
   updated_at    DateTime               @default(now()) @updatedAt @map("updated_at")
@@ -140,6 +141,7 @@ model LiteLLM_TeamTable {
     team_member_permissions String[] @default([])
     access_group_ids String[] @default([])
     policies         String[] @default([])
+    default_team_member_models String[] @default([]) // default allowed_models for newly added team members; empty = no per-member restriction
     model_id Int? @unique // id for LiteLLM_ModelTable -> stores team-level model aliases
     allow_team_guardrail_config Boolean @default(false) // if true, team admin can configure guardrails for this team
     litellm_organization_table LiteLLM_OrganizationTable?   @relation(fields: [organization_id], references: [organization_id])

litellm/constants.py

@@ -1367,6 +1367,9 @@
 MANAGED_OBJECT_STALENESS_CUTOFF_DAYS = max(
     1, int(os.getenv("MANAGED_OBJECT_STALENESS_CUTOFF_DAYS", 7))
 )
+STALE_OBJECT_CLEANUP_BATCH_SIZE = max(
+    1, int(os.getenv("STALE_OBJECT_CLEANUP_BATCH_SIZE", 1000))
+)
 # Set PROXY_BATCH_POLLING_ENABLED=false to disable the CheckBatchCost and
 # CheckResponsesCost background polling jobs entirely (e.g. to avoid DB load on
 # installations with large numbers of stale managed objects).

litellm/exceptions.py

@@ -281,7 +281,7 @@ def __repr__(self):
         return _message
 
 
-class PermissionDeniedError(openai.PermissionDeniedError):  # type:ignore
+class PermissionDeniedError(openai.PermissionDeniedError):  # type: ignore
     def __init__(
         self,
         message,
@@ -847,6 +847,7 @@ def __init__(
     ):
         self.current_cost = current_cost
         self.max_budget = max_budget
+        self.status_code = 429
         message = (
             message
             or f"Budget has been exceeded! Current cost: {current_cost}, Max budget: {max_budget}"

litellm/llms/bedrock/common_utils.py

@@ -432,12 +432,16 @@ def strip_bedrock_routing_prefix(model: str) -> str:
 
 
 def strip_bedrock_throughput_suffix(model: str) -> str:
-    """Strip throughput tier suffixes from Bedrock model names."""
+    """Strip throughput tier suffixes and context window suffixes from Bedrock model names."""
     import re
 
     # Pattern matches model:version:throughput where throughput is like 51k, 18k, etc.
     # Keep the model:version part, strip the :throughput suffix
-    return re.sub(r"(:\d+):\d+k$", r"\1", model)
+    model = re.sub(r"(:\d+):\d+k$", r"\1", model)
+    # Strip context window suffixes like [1m], [200k], etc.
+    # e.g. "us.anthropic.claude-opus-4-6-v1[1m]" -> "us.anthropic.claude-opus-4-6-v1"
+    model = re.sub(r"\[\w+\]$", "", model)
+    return model
 
 
 def get_bedrock_base_model(model: str) -> str:
@@ -1062,9 +1066,11 @@ def sign_aws_request(
 
         return (
             dict(prepped.headers),
-            request_data.encode("utf-8")
-            if isinstance(request_data, str)
-            else request_data,
+            (
+                request_data.encode("utf-8")
+                if isinstance(request_data, str)
+                else request_data
+            ),
         )
 
     def generate_unique_job_name(self, model: str, prefix: str = "litellm") -> str: