Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 92 additions & 0 deletions docs/my-website/docs/proxy/ui_store_model_db_setting.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
import Image from '@theme/IdealImage';
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
Comment on lines +2 to +3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused imports

Tabs and TabItem are imported but never used in this document. Remove these unused imports to keep the file clean.

Suggested change
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';


# Store Model in DB Settings

Enable or disable storing model definitions in the database directly from the Admin UI—no config file edits or proxy restart required. This is especially useful for cloud deployments where updating the config is difficult or requires a long release process.

## Overview

Previously, the `store_model_in_db` setting had to be configured in `proxy_config.yaml` under `general_settings`. Changing it required editing the config and restarting the proxy, which was problematic for cloud users who don't have direct access to the config file or who want to avoid the downtime caused by restarts.

<Image img={require('../../img/ui_store_model_in_db.png')} />

**Store Model in DB Settings** lets you:

- **Enable or disable storing models in the database** – Control whether model definitions are cached in your database (useful for reducing config file size and improving scalability)
- **Apply changes immediately** – No proxy restart needed; settings take effect for new model operations as soon as you save

:::warning UI overrides config
Settings changed in the UI **override** the values in your config file. For example, if `store_model_in_db` is set to `false` in `general_settings`, enabling it in the UI will still persist model definitions to the database. Use the UI when you want runtime control without redeploying.
:::

## How Store Model in DB Works

When `store_model_in_db` is enabled, the LiteLLM proxy stores model definitions in the database instead of relying solely on your `proxy_config.yaml`. This provides several benefits:

- **Reduced config size** – Move model definitions out of YAML for easier maintenance
- **Scalability** – Database storage scales better than large YAML files
- **Dynamic updates** – Models can be added or updated without editing config files
- **Persistence** – Model definitions persist across proxy instances and restarts

The setting applies to all new model operations from the moment you save it.

## How to Configure Store Model in DB in the UI

### 1. Access Models + Endpoints Settings

Navigate to the Admin UI (e.g. `http://localhost:4000/ui` or your `PROXY_BASE_URL/ui`) and go to the **Models + Endpoints** page.

![](https://colony-recorder.s3.amazonaws.com/files/2026-02-22/55bc71f5-730f-4b2c-8539-8a4f46b8bd10/ascreenshot_0f7ba8f1c2694e94938996fd1b4adfcc_text_export.jpeg)

### 2. Open Settings

Click **Models + Endpoints** from the navigation menu.

![](https://colony-recorder.s3.amazonaws.com/files/2026-02-22/55bc71f5-730f-4b2c-8539-8a4f46b8bd10/ascreenshot_fc2b9e4812a9480087f4eb350fa0a792_text_export.jpeg)

### 3. Click the Settings Icon

Look for the settings (gear) icon on the Models + Endpoints page to open the configuration panel.

![](https://colony-recorder.s3.amazonaws.com/files/2026-02-22/7b394364-c281-4db8-8cad-ee322c76c935/ascreenshot_d7c8a6b234bc4e4d92aa7f09aefb13d3_text_export.jpeg)

### 4. Enable or Disable Store Model in DB

Toggle the **Store Model in DB** setting based on your preference:

- **Enabled**: Model definitions will be stored in the database
- **Disabled**: Models are read from the config file only

![](https://colony-recorder.s3.amazonaws.com/files/2026-02-22/54a263ec-ad67-4b16-ba9f-2be57c3e4cb8/ascreenshot_501abda2a6c847f79d085efce814265d_text_export.jpeg)

### 5. Save Settings

Click **Save Settings** to apply the change. No proxy restart is required; the new setting takes effect immediately for subsequent model operations.

![](https://colony-recorder.s3.amazonaws.com/files/2026-02-22/7d13559a-d4e4-41f7-993b-cb20fbfa1f6e/ascreenshot_3245f3c5bd0d43cb96c5f5ff0ccb461d_text_export.jpeg)

## Use Cases

### Cloud and Managed Deployments

When the proxy runs in a managed or cloud environment, config may be in a separate repo, require a long release cycle, or be controlled by another team. Using the UI lets you change the `store_model_in_db` setting without going through a deployment process.

### Reducing Configuration Complexity

For large deployments with hundreds of models, storing model definitions in the database reduces the size and complexity of your `proxy_config.yaml`, making it easier to maintain and version control.

### Dynamic Model Management

Enable `store_model_in_db` to support dynamic model additions and updates without editing your config file. Teams can manage models through the UI or API without needing to redeploy the proxy.

### Zero-Downtime Updates

Change the setting from the UI and have it take effect immediately—perfect for production environments where downtime must be minimized.

## Related Documentation

- [Admin UI Overview](./ui_overview.md) – General guide to the LiteLLM Admin UI
- [Models and Endpoints](./models_and_endpoints.md) – Managing models and API endpoints
- [Config Settings](./config_settings.md)`store_model_in_db` in `general_settings`
Comment on lines +90 to +92
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Broken links to non-existent docs

Two of the three "Related Documentation" links point to files that do not exist in the repository:

  • ./ui_overview.md — no file at docs/my-website/docs/proxy/ui_overview.md
  • ./models_and_endpoints.md — no file at docs/my-website/docs/proxy/models_and_endpoints.md

Only ./config_settings.md exists. These broken links will cause 404 errors in the published docs. Either create the referenced pages, or update these links to point to existing documentation pages.

Binary file added docs/my-website/img/ui_store_model_in_db.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
8 changes: 8 additions & 0 deletions docs/my-website/release_notes/v1.81.14.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ pip install litellm==1.81.14
- **Guardrail Garden** — [Browse built-in and partner guardrails by use case — competitor blocking, topic filtering, GDPR, prompt injection, and more. Pick a template, customize it, attach it to a team or key.](../../docs/proxy/guardrails/policy_templates)
- **Compliance Playground** — [Test any guardrail policy against your own traffic before it goes live. See precision, recall, and false positive rate — so you know how it'll behave in production.](../../docs/proxy/guardrails/policy_templates)
- **3 new zero-cost built-in guardrails** — [Competitor name blocker, topic blocker, and insults filter — all gateway-level, &lt;0.1ms latency, no external API, configurable per-team or key](../../docs/proxy/guardrails)
- **Store Model in DB Settings via UI** - [Configure model storage directly in the Admin UI without editing config files or restarting the proxy—perfect for cloud deployments](../../docs/proxy/ui_store_model_db_setting)
- **Claude Sonnet 4.6 — day 0** — [Full support across Anthropic and Vertex AI: reasoning, computer use, prompt caching, 200K context](../../docs/providers/anthropic)
- **20+ performance optimizations** — Faster routing, lower logging overhead, reduced cost-calculator latency, and connection pool fixes — meaningfully less CPU and latency on every request

Expand All @@ -67,6 +68,13 @@ This release brings 3 new built-in guardrails that run directly on the gateway.

These guardrails are built for production and on our benchmarks had a 100% Recall and Precision.

### Store Model in DB Settings via UI

Previously, the `store_model_in_db` setting could only be configured in `proxy_config.yaml` under `general_settings`, requiring a proxy restart to take effect. Now you can enable or disable this setting directly from the Admin UI without any restarts. This is especially useful for cloud deployments where you don't have direct access to config files or want to avoid downtime. Enable `store_model_in_db` to move model definitions from your YAML into the database—reducing config complexity, improving scalability, and enabling dynamic model management across multiple proxy instances.

![Store model in DB Setting](../img/ui_store_model_in_db.png)


#### Eval results

We benchmarked our new built-in guardrails against labeled datasets before shipping. You can see the results for Denied Financial Advice (207 cases) and Denied Insults (299 cases):
Expand Down
1 change: 1 addition & 0 deletions docs/my-website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -334,6 +334,7 @@ const sidebars = {
"proxy/ui_credentials",
"proxy/ai_hub",
"proxy/model_compare_ui",
"proxy/ui_store_model_db_setting",
]
},
{
Expand Down
Loading