BerriAI · krrishdholakia · Feb 22, 2026 · Feb 21, 2026 · Feb 22, 2026 · Feb 22, 2026
diff --git a/docs/my-website/blog/server_root_path/index.md b/docs/my-website/blog/server_root_path/index.md
@@ -0,0 +1,154 @@
+---
+slug: server-root-path-incident
+title: "Incident Report: SERVER_ROOT_PATH regression broke UI routing"
+date: 2026-02-21T10:00:00
+authors:
+  - name: Yuneng Jiang
+    title: SWE @ LiteLLM (Full Stack)
+    url: https://www.linkedin.com/in/yunengjiang/
+  - name: Ishaan Jaff
+    title: "CTO, LiteLLM"
+    url: https://www.linkedin.com/in/reffajnaahsi/
+    image_url: https://pbs.twimg.com/profile_images/1613813310264340481/lz54oEiB_400x400.jpg
+  - name: Krrish Dholakia
+    title: "CEO, LiteLLM"
+    url: https://www.linkedin.com/in/krish-d/
+    image_url: https://pbs.twimg.com/profile_images/1298587542745358340/DZv3Oj-h_400x400.jpg
+tags: [incident-report, ui, stability]
+hide_table_of_contents: false
+---
+
+**Date:** January 22, 2026
+**Duration:** ~4 days (until fix merged January 26, 2026)
+**Severity:** High
+**Status:** Resolved
+
+> **Note:** This fix is available starting from LiteLLM `v1.81.3.rc.6` or higher.
+
+## Summary
+
+A PR ([`#19467`](https://github.com/BerriAI/litellm/pull/19467)) accidentally removed the `root_path=server_root_path` parameter from the FastAPI app initialization in `proxy_server.py`. This caused the proxy to ignore the `SERVER_ROOT_PATH` environment variable when serving the UI. Users who deploy LiteLLM behind a reverse proxy with a path prefix (e.g., `/api/v1` or `/llmproxy`) found that all UI pages returned 404 Not Found.
+
+- **LLM API calls:** No impact. API routing was unaffected.
+- **UI pages:** All UI pages returned 404 for deployments using `SERVER_ROOT_PATH`.
+- **Swagger/OpenAPI docs:** Broken when accessed through the configured root path.
+
+{/* truncate */}
+
+---
+
+## Background
+
+Many LiteLLM deployments run behind a reverse proxy (e.g., Nginx, Traefik, AWS ALB) that routes traffic to LiteLLM under a path prefix. FastAPI's `root_path` parameter tells the application about this prefix so it can correctly serve static files, generate URLs, and handle routing.
+
+```mermaid
+sequenceDiagram
+    participant User as User Browser
+    participant RP as Reverse Proxy
+    participant LP as LiteLLM Proxy
+
+    User->>RP: GET /llmproxy/ui/
+    RP->>LP: GET /ui/ (X-Forwarded-Prefix: /llmproxy)
+
+    Note over LP: Before regression:<br/>FastAPI root_path="/llmproxy"<br/>→ Serves UI correctly
+
+    Note over LP: After regression:<br/>FastAPI root_path=""<br/>→ UI assets resolve to wrong paths<br/>→ 404 Not Found
+```
+
+The `root_path` parameter was present in `proxy_server.py` since early versions of LiteLLM. It was removed as a side effect of PR [#19467](https://github.com/BerriAI/litellm/pull/19467), which was intended to fix a different UI 404 issue.
+
+---
+
+## Root cause
+
+PR [#19467](https://github.com/BerriAI/litellm/pull/19467) (`73d49f8`) removed the `root_path=server_root_path` line from the `FastAPI()` constructor in `proxy_server.py`:
+
+```diff
+ app = FastAPI(
+     docs_url=_get_docs_url(),
+     redoc_url=_get_redoc_url(),
+     title=_title,
+     description=_description,
+     version=version,
+-    root_path=server_root_path,
+     lifespan=proxy_startup_event,
+ )
+```
+
+Without `root_path`, FastAPI treated all requests as if the application was mounted at `/`, causing path mismatches for any deployment using `SERVER_ROOT_PATH`.
+
+The regression went undetected because:
+
+1. **No automated test** verified that `root_path` was set on the FastAPI app.
+2. **No manual test procedure** existed for `SERVER_ROOT_PATH` functionality.
+3. **Default deployments** (without `SERVER_ROOT_PATH`) were unaffected, so most CI tests passed.
+
+---
+
+## Remediation
+
+| #   | Action                                                                                            | Status  | Code                                                                                                                       |
+| --- | ------------------------------------------------------------------------------------------------- | ------- | -------------------------------------------------------------------------------------------------------------------------- |
+| 1   | Restore `root_path=server_root_path` in FastAPI app initialization                                | ✅ Done | [`#19790`](https://github.com/BerriAI/litellm/pull/19790) (`5426b3c`)                                                      |
+| 2   | Add unit tests for `get_server_root_path()` and FastAPI app initialization                        | ✅ Done | [`test_server_root_path.py`](https://github.com/BerriAI/litellm/blob/main/tests/proxy_unit_tests/test_server_root_path.py) |
+| 3   | Add CI workflow that builds Docker image and tests UI routing with `SERVER_ROOT_PATH` on every PR | ✅ Done | [`test_server_root_path.yml`](https://github.com/BerriAI/litellm/blob/main/.github/workflows/test_server_root_path.yml)    |
+| 4   | Document manual test procedure for `SERVER_ROOT_PATH`                                             | ✅ Done | [Discussion #8495](https://github.com/BerriAI/litellm/discussions/8495)                                                    |
+
+---
+
+## CI workflow details
+
+The new [`test_server_root_path.yml`](https://github.com/BerriAI/litellm/blob/main/.github/workflows/test_server_root_path.yml) workflow runs on every PR against `main`. It:
+
+1. Builds the LiteLLM Docker image
+2. Starts a container with `SERVER_ROOT_PATH` set (tests both `/api/v1` and `/llmproxy`)
+3. Verifies the UI returns valid HTML at `{ROOT_PATH}/ui/`
+4. Fails the workflow if the UI is unreachable
+
+```mermaid
+flowchart TD
+    A["PR opened/updated"] --> B["Build Docker image"]
+    B --> C["Start container with SERVER_ROOT_PATH=/api/v1"]
+    B --> D["Start container with SERVER_ROOT_PATH=/llmproxy"]
+    C --> E["curl {ROOT_PATH}/ui/ → expect HTML"]
+    D --> F["curl {ROOT_PATH}/ui/ → expect HTML"]
+    E -->|"HTML found"| G["✅ Pass"]
+    E -->|"404 or no HTML"| H["❌ Fail Workflow"]
+    F -->|"HTML found"| G
+    F -->|"404 or no HTML"| H
+
+    style G fill:#d4edda,stroke:#28a745
+    style H fill:#f8d7da,stroke:#dc3545
+```
+
+This prevents future regressions where changes to `proxy_server.py` accidentally break `SERVER_ROOT_PATH` support.
+
+---
+
+## Timeline
+
+| Time (UTC)         | Event                                                                                                                                                        |
+| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| Jan 22, 2026 04:20 | PR [#19467](https://github.com/BerriAI/litellm/pull/19467) merged, removing `root_path=server_root_path`                                                     |
+| Jan 22–26          | Users on nightly builds report UI 404 errors when using `SERVER_ROOT_PATH`                                                                                   |
+| Jan 26, 2026 17:48 | Fix PR [#19790](https://github.com/BerriAI/litellm/pull/19790) merged, restoring `root_path=server_root_path`                                                |
+| Feb 18, 2026       | CI workflow [`test_server_root_path.yml`](https://github.com/BerriAI/litellm/blob/main/.github/workflows/test_server_root_path.yml) added to run on every PR |
+
+---
+
+## Resolution steps for users
+
+For users still experiencing issues, update to the latest LiteLLM version:
+
+```bash
+pip install --upgrade litellm
+```
+
+Verify your `SERVER_ROOT_PATH` is correctly set:
+
+```bash
+# In your environment or docker-compose.yml
+SERVER_ROOT_PATH="/your-prefix"
+```
+
+Then confirm the UI is accessible at `http://your-host:4000/your-prefix/ui/`.
diff --git a/docs/my-website/docs/proxy/config_settings.md b/docs/my-website/docs/proxy/config_settings.md
@@ -758,7 +758,7 @@ router_settings:
 | LITELLM_CLI_JWT_EXPIRATION_HOURS | Expiration time in hours for CLI-generated JWT tokens. Default is 24 hours
 | LITELLM_DD_AGENT_HOST | Hostname or IP of DataDog agent for LiteLLM-specific logging. When set, logs are sent to agent instead of direct API
 | LITELLM_DEPLOYMENT_ENVIRONMENT | Environment name for the deployment (e.g., "production", "staging"). Used as a fallback when OTEL_ENVIRONMENT_NAME is not set. Sets the `environment` tag in telemetry data
-| LITELLM_DETAILED_TIMING | When true, adds detailed per-phase timing headers to responses (x-litellm-timing-{pre-processing,llm-api,post-processing,message-copy}-ms). Default is false. See [latency overhead docs](../troubleshoot/latency_overhead.md)
+| LITELLM_DETAILED_TIMING | When true, adds detailed per-phase timing headers to responses (`x-litellm-timing-{pre-processing,llm-api,post-processing,message-copy}-ms`). Default is false. See [latency overhead docs](../troubleshoot/latency_overhead.md)
 | LITELLM_DD_AGENT_PORT | Port of DataDog agent for LiteLLM-specific log intake. Default is 10518
 | LITELLM_DD_LLM_OBS_PORT | Port for Datadog LLM Observability agent. Default is 8126
 | LITELLM_DONT_SHOW_FEEDBACK_BOX | Flag to hide feedback box in LiteLLM UI

diff --git a/docs/my-website/docs/proxy/ui_store_model_db_setting.md b/docs/my-website/docs/proxy/ui_store_model_db_setting.md
@@ -0,0 +1,92 @@
+import Image from '@theme/IdealImage';
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Store Model in DB Settings
+
+Enable or disable storing model definitions in the database directly from the Admin UI—no config file edits or proxy restart required. This is especially useful for cloud deployments where updating the config is difficult or requires a long release process.
+
+## Overview
+
+Previously, the `store_model_in_db` setting had to be configured in `proxy_config.yaml` under `general_settings`. Changing it required editing the config and restarting the proxy, which was problematic for cloud users who don't have direct access to the config file or who want to avoid the downtime caused by restarts.
+
+<Image img={require('../../img/ui_store_model_in_db.png')} />
+
+**Store Model in DB Settings** lets you:
+
+- **Enable or disable storing models in the database** – Control whether model definitions are cached in your database (useful for reducing config file size and improving scalability)
+- **Apply changes immediately** – No proxy restart needed; settings take effect for new model operations as soon as you save
+
+:::warning UI overrides config
+Settings changed in the UI **override** the values in your config file. For example, if `store_model_in_db` is set to `false` in `general_settings`, enabling it in the UI will still persist model definitions to the database. Use the UI when you want runtime control without redeploying.
+:::
+
+## How Store Model in DB Works
+
+When `store_model_in_db` is enabled, the LiteLLM proxy stores model definitions in the database instead of relying solely on your `proxy_config.yaml`. This provides several benefits:
+
+- **Reduced config size** – Move model definitions out of YAML for easier maintenance
+- **Scalability** – Database storage scales better than large YAML files
+- **Dynamic updates** – Models can be added or updated without editing config files
+- **Persistence** – Model definitions persist across proxy instances and restarts
+
+The setting applies to all new model operations from the moment you save it.
+
+## How to Configure Store Model in DB in the UI
+
+### 1. Access Models + Endpoints Settings
+
+Navigate to the Admin UI (e.g. `http://localhost:4000/ui` or your `PROXY_BASE_URL/ui`) and go to the **Models + Endpoints** page.
+
+![](https://colony-recorder.s3.amazonaws.com/files/2026-02-22/55bc71f5-730f-4b2c-8539-8a4f46b8bd10/ascreenshot_0f7ba8f1c2694e94938996fd1b4adfcc_text_export.jpeg)
+
+### 2. Open Settings
+
+Click **Models + Endpoints** from the navigation menu.
+
+![](https://colony-recorder.s3.amazonaws.com/files/2026-02-22/55bc71f5-730f-4b2c-8539-8a4f46b8bd10/ascreenshot_fc2b9e4812a9480087f4eb350fa0a792_text_export.jpeg)
+
+### 3. Click the Settings Icon
+
+Look for the settings (gear) icon on the Models + Endpoints page to open the configuration panel.
+
+![](https://colony-recorder.s3.amazonaws.com/files/2026-02-22/7b394364-c281-4db8-8cad-ee322c76c935/ascreenshot_d7c8a6b234bc4e4d92aa7f09aefb13d3_text_export.jpeg)
+
+### 4. Enable or Disable Store Model in DB
+
+Toggle the **Store Model in DB** setting based on your preference:
+
+- **Enabled**: Model definitions will be stored in the database
+- **Disabled**: Models are read from the config file only
+
+![](https://colony-recorder.s3.amazonaws.com/files/2026-02-22/54a263ec-ad67-4b16-ba9f-2be57c3e4cb8/ascreenshot_501abda2a6c847f79d085efce814265d_text_export.jpeg)
+
+### 5. Save Settings
+
+Click **Save Settings** to apply the change. No proxy restart is required; the new setting takes effect immediately for subsequent model operations.
+
+![](https://colony-recorder.s3.amazonaws.com/files/2026-02-22/7d13559a-d4e4-41f7-993b-cb20fbfa1f6e/ascreenshot_3245f3c5bd0d43cb96c5f5ff0ccb461d_text_export.jpeg)
+
+## Use Cases
+
+### Cloud and Managed Deployments
+
+When the proxy runs in a managed or cloud environment, config may be in a separate repo, require a long release cycle, or be controlled by another team. Using the UI lets you change the `store_model_in_db` setting without going through a deployment process.
+
+### Reducing Configuration Complexity
+
+For large deployments with hundreds of models, storing model definitions in the database reduces the size and complexity of your `proxy_config.yaml`, making it easier to maintain and version control.
+
+### Dynamic Model Management
+
+Enable `store_model_in_db` to support dynamic model additions and updates without editing your config file. Teams can manage models through the UI or API without needing to redeploy the proxy.
+
+### Zero-Downtime Updates
+
+Change the setting from the UI and have it take effect immediately—perfect for production environments where downtime must be minimized.
+
+## Related Documentation
+
+- [Admin UI Overview](./ui_overview.md) – General guide to the LiteLLM Admin UI
+- [Models and Endpoints](./models_and_endpoints.md) – Managing models and API endpoints
+- [Config Settings](./config_settings.md) – `store_model_in_db` in `general_settings`
diff --git a/docs/my-website/img/release_notes/compliance_playground.png b/docs/my-website/img/release_notes/compliance_playground.png
diff --git a/docs/my-website/img/release_notes/guardrail_garden.png b/docs/my-website/img/release_notes/guardrail_garden.png
diff --git a/docs/my-website/img/ui_store_model_in_db.png b/docs/my-website/img/ui_store_model_in_db.png