feat: support custom LLM provider configuration via web UI

[italic-jinxin]

Users can now define custom LLM providers through the web UI and have them take effect without modifying environment variables or config files.

• Add CustomLlmProviderSettings struct and llm_custom_providers field to Settings so custom provider definitions are persisted and loaded from the DB settings table
• Add LlmConfig::resolve_custom_provider() to build a RegistryProviderConfig from user-defined provider data (base_url, adapter, model, api_key)
• Flip resolution priority to db > env > default so active provider set through the UI takes precedence over deployment env vars
• Warn when a custom provider is missing base_url or model
• Add startup info logs for backend source and provider creation
• Add regression tests for custom provider resolution and DB priority

Summary

Change Type

• Bug fix
• New feature
• Refactor
• Documentation
• CI/Infrastructure
• Security
• Dependencies

Linked Issue

Validation

• cargo fmt
• cargo clippy --all --benches --tests --examples --all-features
• Relevant tests pass:
• Manual testing:

Security Impact

Database Impact

Blast Radius

Rollback Plan

---

Review track:

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the flexibility of LLM integration by enabling users to define and manage custom LLM providers directly through the web user interface. This change eliminates the need for manual environment variable or configuration file modifications, streamlining the setup process and improving user control over their language model backends. The update also includes robust backend logic to ensure proper resolution priority and prevent accidental deletion of active providers.

Highlights

• Custom LLM Provider Management UI: Introduced a new 'Config' tab in the web UI, allowing users to add, manage, activate, and delete custom LLM providers directly from the interface without manual configuration file edits.
• Database Persistence for Custom Providers: Implemented persistence for custom LLM provider configurations by adding a new llm_custom_providers field to the Settings struct, ensuring user-defined providers are stored in the database.
• LLM Backend Resolution Priority: Adjusted the LLM backend resolution logic to prioritize database settings over environment variables, followed by default values (db > env > default), giving precedence to UI-configured providers.
• Active Provider Deletion Guard: Added a server-side guard to prevent users from deleting a custom LLM provider that is currently active, enhancing system stability and user experience.
• Enhanced LLM Configuration Logging: Integrated new logging statements to provide clearer insights into LLM backend resolution and provider creation processes at startup.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a valuable feature for configuring custom LLM providers through the web UI. The backend changes to support this, including persistence and configuration resolution, are well-implemented and include relevant tests. The frontend implementation is also comprehensive. I have identified a couple of areas for improvement: one is a performance optimization in the backend to avoid unnecessary data cloning, and the other is a bug fix in the frontend to correctly handle API errors and prevent UI state inconsistencies.

[Copilot]

Pull request overview

This PR adds end-user configuration of LLM providers from the web UI by persisting custom provider definitions and built-in provider overrides into per-user settings, and updating backend resolution so UI-selected backends can take precedence.

Changes:

• Persist custom LLM providers (llm_custom_providers) and per-provider overrides (llm_builtin_overrides) in Settings, and resolve custom providers into RegistryProviderConfig.
• Add a new “Config” tab UI to add/edit/delete providers, set the active backend, test connectivity, and list available models.
• Improve robustness for strict OpenAI-compatible providers by filtering empty chat messages and adding startup logs + regression tests for config resolution.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 13 comments.

Show a summary per file

File	Description
src/settings.rs	Adds settings structs/fields for custom providers and built-in overrides.
src/config/llm.rs	Changes LLM backend resolution priority and adds custom provider resolution + tests.
src/llm/mod.rs	Adds an info log when creating LLM providers.
src/llm/rig_adapter.rs	Skips empty user/assistant messages and adds unit tests.
src/channels/web/server.rs	Adds /api/llm/test_connection and /api/llm/list_models endpoints.
src/channels/web/handlers/settings.rs	Adds a guard to prevent deleting the active custom provider (currently not wired into server routes).
src/channels/web/static/index.html	Adds the Config tab panel and provider add/edit dialog markup.
src/channels/web/static/style.css	Adds styling for the Config tab and provider dialog/cards.
src/channels/web/static/app.js	Implements Config tab behavior (provider CRUD, activation, test, list models).
src/channels/web/static/i18n/en.js	Adds English i18n strings for the Config tab.
src/channels/web/static/i18n/zh-CN.js	Adds Simplified Chinese i18n strings for the Config tab.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[think-in-universe]

A few recommended enhancements and bug fixing. Let's resolve the issues.

1. Should we add a URL input for OpenAI Compatible?

2. The API URL for NEAR AI (https://api.near.ai/v1) seems incorrect

3. Should we add a dropdown selector for selecting models from one provider?

4. It seems there's no way to configure NEAR AI?

5. Should we move Providers tab into Inference tab as a sub-section?

[ilblackdragon]

It should be in "Inference" settings menu

[ilblackdragon]

Code Review

Overview

This PR adds the ability to define and manage custom LLM providers through the web UI. It includes new settings types (CustomLlmProviderSettings, LlmBuiltinOverride), backend resolution priority flipped to DB > env > default, a full web UI with provider cards/modal/test-connection/model-listing, guard preventing deletion of the active provider, and empty message filtering in rig_adapter.rs. Good regression test coverage for the config resolution logic.

---

Security Issues

1. SSRF via test_connection / list_models endpoints (server.rs): These endpoints accept an arbitrary base_url from the user and make HTTP requests to it from the server. An attacker with web UI access could probe internal network services (e.g., http://169.254.169.254/ for cloud metadata, internal APIs). These endpoints need URL validation — at minimum, reject private/link-local IP ranges and non-HTTP(S) schemes.

2. API keys stored in plaintext (settings.rs): CustomLlmProviderSettings.api_key and LlmBuiltinOverride.api_key are stored as plain strings in the DB settings table. The project has src/secrets/ with AES-256-GCM encryption specifically for this purpose. API keys should be encrypted at rest, or at least a follow-up issue should be filed.

3. API keys in llm_custom_providers exposed via settings export (settings_export_handler): Calling GET /api/settings/export will return all custom providers including their api_key values. The export handler has no field-level redaction.

Breaking Change

4. Resolution priority flip (DB > env) (config/llm.rs:58): Changing from env > db to db > env is a silent breaking change for any deployment that uses environment variables to override DB-persisted settings. This should be prominently documented in release notes. Consider making this opt-in or at least logging a warning when a DB value overrides a set env var.

Code Quality

5. ~200 lines of handler logic inlined in server.rs: The test_provider_connection, fetch_provider_models, and related types should be extracted to a handler module (e.g., handlers/llm.rs), consistent with the existing handler module pattern.

6. Duplicate HTML escape function: escHtml() (line ~702 in new JS) duplicates the existing escapeHtml() at app.js:5436. Use the existing function.

7. Hardcoded BUILTIN_PROVIDERS in JavaScript: This 25-entry array duplicates data from the Rust provider registry and will drift out of sync as providers are added or removed. Consider serving this from a backend endpoint (e.g., GET /api/llm/providers).

8. Inconsistent model resolution: resolve_registry_provider now inlines the model resolution chain (env > selected_model > builtin_override > default) while resolve_model() is still used for NearAI with a different chain (env > selected_model > default). The helper should either be updated to support the override step or the logic should be unified.

9. Separate concern bundled in: The empty-message filtering in rig_adapter.rs (Kimi compatibility fix) is unrelated to custom LLM provider config. It should be a separate PR/commit for cleaner bisectability.

Functional Issues

10. Guard only protects PUT, not DELETE (handlers/settings.rs): guard_active_provider_not_removed runs when key == "llm_custom_providers" on PUT, but DELETE /api/settings/llm_custom_providers would wipe all custom providers including the active one, bypassing the guard entirely.

11. Race condition in guard: The guard reads llm_backend, then reads llm_custom_providers, then validates — all as separate DB calls with no transaction. A concurrent request could change llm_backend between reads.

12. No validation on adapter values: The CustomLlmProviderSettings.adapter field accepts any string. Invalid values silently default to OpenAiCompletions. Consider validating against a known set (open_ai_completions, anthropic, ollama) and returning an error.

Minor

• Stale/speculative model names in BUILTIN_PROVIDERS (e.g., gpt-5-mini) — verify these are current defaults.
• nearai base_url uses staging (private-chat-stg.near.ai) — intentional?
• No Escape key handler for the provider dialog.
• Test db_llm_backend_takes_priority_over_env_var cleans up LLM_BACKEND env var manually at test end rather than using an RAII guard — if an assertion panics before cleanup, the env var leaks.
• No tests for the settings guard logic or the test-connection/list-models handlers.

Verdict

The feature is well-scoped and the Rust config changes are clean. The SSRF issue (#1) and plaintext API keys (#2) should be addressed before merge. The priority flip (#4) needs explicit documentation. The rest are improvements that could be follow-ups.

[ilblackdragon]

Hey, I've been looking into this.
We def need the flip of DB preferred over env for multi tenancy.

I'm going to push few changes here that make sure it's universally addressed across code base.

[ilblackdragon]

Review fixes pushed — db50fb17

Reviewed this PR and pushed fixes for the issues found. Here's what was addressed:

Security fixes

1. Server-side provider ID validation — The [a-z0-9-]+ regex was only enforced in the frontend JS. Added is_valid_provider_id() and validate_custom_providers() server-side in the settings handler. Rejects uppercase, spaces, dots, path traversal chars, and IDs over 64 chars. (7 new tests)

2. Tightened is_nearai_private_endpoint — The old check host.contains("private") matched hostnames like private-evil.near.ai or myprivate.near.ai. Now matches private.near.ai exactly or *.private.near.ai subdomains only. (5 new tests)

Doc/correctness fixes

3. Fixed misleading priority doc comments — The module-level comments in config/mod.rs and the TOML template header claimed "DB > env > default" universally, but only LLM settings were flipped. Updated to document the split model: LLM uses DB > env, other subsystems still use env > DB.

4. Cleaned up PR fix(agent): persist /model selection to .env, TOML, and DB #1581 artifacts — PR fix(agent): persist /model selection to .env, TOML, and DB #1581 (merged) added .env and TOML write-through in persist_selected_model as a workaround for the old env > db priority. With DB now authoritative for LLM:

   • Removed TOML file creation when none exists (DB persistence is sufficient)
   • Kept best-effort updates to .env/TOML when they already contain the var (avoids user confusion)
   • Updated stale priority comments in commands.rs
   • Fixed contradictory test stale_toml_overwrites_db_model → replaced with db_model_wins_over_stale_toml that tests the correct merge direction

5. Added missing validation tests — 6 new tests for adapter validation (validate_custom_providers_adapters): rejects unknown, rejects missing, accepts all valid adapters, handles non-array input.

Related issue/PR comments

• feat: Unify configuration sources before hot-reload #1119 — Commented with status update on what this PR addresses vs. what remains
• PR feat(config): add conflict detection warnings when env vars override user-configured settings #1401 — Commented confirming compatibility (its warn_if_env_shadows is correct for non-LLM subsystems)

Verification

• cargo fmt — clean
• cargo clippy --all --all-features — zero warnings
• cargo test --lib — 3770 passed, 0 failed

[Copilot]

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

optional_env(env_var).ok().flatten() silently discards ConfigError (e.g., non-unicode env var values). Since optional_env() already returns Ok(None) when unset, this should typically be optional_env(env_var)? so real env parsing errors are surfaced instead of being ignored.

Suggested change

	Ok(settings
	.selected_model
	.clone()
	.or_else(|| optional_env(env_var).ok().flatten())
	.unwrap_or_else(|| default.to_string()))
	if let Some(model) = settings.selected_model.clone() {
	Ok(model)
	} else if let Some(model) = optional_env(env_var)? {
	Ok(model)
	} else {
	Ok(default.to_string())
	}