feat(ui): add Chat UI — ChatGPT-like interface with MCP tools and streaming#22937
Merged
ishaan-jaff merged 26 commits intomainfrom Mar 6, 2026
Merged
feat(ui): add Chat UI — ChatGPT-like interface with MCP tools and streaming#22937ishaan-jaff merged 26 commits intomainfrom
ishaan-jaff merged 26 commits intomainfrom
Conversation
… caused scroll-lock bypass
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Greptile SummaryThis PR adds a ChatGPT-style chat interface ( Functional issues found:
Maintainability issue:
Confidence Score: 2/5
Sequence DiagramsequenceDiagram
participant User
participant ChatPage
participant useChatHistory
participant localStorage
participant LiteLLMProxy
User->>ChatPage: Types message & clicks Send
ChatPage->>useChatHistory: createConversation(selectedModel)
useChatHistory->>localStorage: save conversation {id, model, messages:[]}
ChatPage->>useChatHistory: appendMessage(convId, {role:user})
ChatPage->>useChatHistory: appendMessage(convId, {role:assistant, content:""})
ChatPage->>LiteLLMProxy: makeOpenAIChatCompletionRequest(history, ..., mcpServers)
LiteLLMProxy-->>ChatPage: stream chunk (onChunk callback)
ChatPage->>useChatHistory: updateLastAssistantMessage(convId, {content: accumulated})
LiteLLMProxy-->>ChatPage: stream complete
ChatPage->>ChatPage: setIsStreaming(false)
User->>ChatPage: Navigates to existing conversation (?id=X)
ChatPage->>localStorage: getItem(LOCALSTORAGE_MODEL_KEY)
Note over ChatPage: ⚠️ activeConversation.model is IGNORED
ChatPage->>ChatPage: setSelectedModel(lastGlobalModel)
User->>ChatPage: Switches MCP servers
ChatPage->>ChatPage: setSelectedMCPServers(newServers)
Note over ChatPage: ⚠️ NOT saved to Conversation.mcpServerNames
User->>ChatPage: Clicks to different conversation (?id=Y)
Note over ChatPage: ⚠️ MCP servers still show the previous conversation's tools
Last reviewed commit: 8f09c09 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Member
Author
|
@greptile review |
Member
Author
|
@greptile review |
Member
Author
|
@greptile review |
… before async state update
Member
Author
|
@greptile review |
…error, inline detection, remove unused ref
Member
Author
|
@greptile review |
Comment on lines
+94
to
+111
| useEffect(() => { | ||
| if (!accessToken) return; | ||
| setIsLoadingModels(true); | ||
| fetchAvailableModels(accessToken) | ||
| .then((data) => { | ||
| const names = (data || []).map((m: { model_group?: string }) => m.model_group ?? "").filter(Boolean); | ||
| setModels(names); | ||
| const saved = localStorage.getItem(LOCALSTORAGE_MODEL_KEY); | ||
| if (saved && names.includes(saved)) { | ||
| setSelectedModel(saved); | ||
| } else if (names.length > 0) { | ||
| setSelectedModel(names[0]); | ||
| localStorage.setItem(LOCALSTORAGE_MODEL_KEY, names[0]); | ||
| } | ||
| }) | ||
| .catch(() => message.error("Could not load models")) | ||
| .finally(() => setIsLoadingModels(false)); | ||
| }, [accessToken]); |
Contributor
There was a problem hiding this comment.
Conversation model is never restored when navigating to an existing chat
The Conversation type has a model field that is populated when a conversation is created (line 131), but it is never read back. When a user navigates to an existing conversation via URL (e.g. ?id=abc123), the model selector is set exclusively from LOCALSTORAGE_MODEL_KEY (lines 101–107) — never from activeConversation?.model. This causes:
- User creates conversation A with
gpt-4o - User changes model to
claude-3-5-sonnetand creates conversation B - User navigates back to conversation A — the selector still shows
claude-3-5-sonnet, so messages are sent with the wrong model
Fix: After models are loaded and an active conversation exists, prefer the conversation's own stored model. Add a separate useEffect that reacts to activeConversationId changes to restore the model:
// In the model-loading effect, prefer conversation model if available
const preferredModel = activeConversation?.model ?? saved ?? "";
if (preferredModel && names.includes(preferredModel)) {
setSelectedModel(preferredModel);
}
Comment on lines
+154
to
+171
| try { | ||
| await makeOpenAIChatCompletionRequest( | ||
| history, | ||
| (chunk: string) => { | ||
| accumulatedContent += chunk; | ||
| updateLastAssistantMessage(convId!, { content: accumulatedContent }); | ||
| }, | ||
| selectedModel, | ||
| accessToken, | ||
| undefined, | ||
| abortControllerRef.current.signal, | ||
| (rc: string) => { | ||
| accumulatedReasoning += rc; | ||
| updateLastAssistantMessage(convId!, { reasoningContent: accumulatedReasoning }); | ||
| }, | ||
| undefined, undefined, undefined, undefined, undefined, undefined, | ||
| selectedMCPServers.length > 0 ? selectedMCPServers : undefined, | ||
| ); |
Contributor
There was a problem hiding this comment.
Fragile positional undefined arguments to makeOpenAIChatCompletionRequest
Six consecutive undefined arguments are passed (line 169) to fill positional slots before selectedMCPServers. This is extremely brittle — if any parameter is inserted or reordered in the function's signature, MCP server support will silently break with no compile-time error.
Consider refactoring to use named options, or at minimum add comments documenting which parameter each undefined corresponds to so future maintainers understand the intent.
Comment on lines
+113
to
+115
| useEffect(() => { | ||
| if (staleId) router.replace("/chat"); | ||
| }, [staleId, router]); |
Contributor
There was a problem hiding this comment.
router.replace hardcodes /chat path without serverRootPath
When a stale conversation ID is detected, the code redirects to /chat using an absolute path (line 114). This will fail on sub-path deployments (e.g. server_root_path = "/litellm"), since the correct path would be /litellm/ui/chat. The file already has getDashboardUrl() and imports serverRootPath for exactly this reason.
Suggested change
| useEffect(() => { | |
| if (staleId) router.replace("/chat"); | |
| }, [staleId, router]); | |
| useEffect(() => { | |
| if (staleId) router.replace(`${serverRootPath && serverRootPath !== "/" ? serverRootPath : ""}/ui/chat`); | |
| }, [staleId, router, serverRootPath]); |
Comment on lines
+63
to
+70
| const [selectedModel, setSelectedModel] = useState<string>(""); | ||
| const [models, setModels] = useState<string[]>([]); | ||
| const [isLoadingModels, setIsLoadingModels] = useState(true); | ||
| const [selectedMCPServers, setSelectedMCPServers] = useState<string[]>([]); | ||
| const [isStreaming, setIsStreaming] = useState(false); | ||
| const [inputText, setInputText] = useState(""); | ||
| const [mcpPopoverOpen, setMcpPopoverOpen] = useState(false); | ||
| const [sidebarCollapsed, setSidebarCollapsed] = useState(false); |
Contributor
There was a problem hiding this comment.
MCP server selection is global, not per-conversation
selectedMCPServers is component-level state (line 66) and is never persisted to or restored from the Conversation object. When a user navigates between conversations, the MCP server selection does not change to reflect what was active in each conversation. This means:
- Switching to an old conversation silently re-uses whichever servers were toggled on last, which may be incorrect
- The
Conversation.mcpServerNamesfield is initialized duringcreateConversationbut never written after creation and never read back
Fix: Save the active MCP server selection to the conversation when sending a message, and restore it when activeConversationId changes.
Comment on lines
+453
to
+469
| <Avatar | ||
| size={28} | ||
| icon={<UserOutlined />} | ||
| style={{ backgroundColor: "#e0e7ff", color: "#4f46e5", flexShrink: 0 }} | ||
| /> | ||
| <Text | ||
| style={{ | ||
| fontSize: 13, | ||
| color: "#555", | ||
| overflow: "hidden", | ||
| whiteSpace: "nowrap", | ||
| textOverflow: "ellipsis", | ||
| }} | ||
| > | ||
| My Account | ||
| </Text> | ||
| </div> |
Contributor
There was a problem hiding this comment.
Hardcoded "My Account" placeholder never displays actual user identity
The footer renders the static string "My Account" (line 467) with a generic UserOutlined avatar. The ConversationList component does not receive userId or userEmail as props, so there is no way to display the logged-in user's actual identity here. Showing "My Account" without the user's real name/email is misleading.
Fix: Either:
- Pass
userId/userEmailfromChatPagetoConversationListas props and render the actual user info, or - Remove the footer entirely to avoid confusing users into thinking this section is functional
2 tasks
krrishdholakia
added a commit
that referenced
this pull request
Mar 6, 2026
…verflow (#22950) * feat(spend-logs): add truncation note when error logs are truncated for DB storage (#22936) When the messages or response JSON fields in spend logs are truncated before being written to the database, the truncation marker now includes a note explaining: - This is a DB storage safeguard - Full, untruncated data is still sent to logging callbacks (OTEL, Datadog, etc.) - The MAX_STRING_LENGTH_PROMPT_IN_DB env var can be used to increase the limit Also emits a verbose_proxy_logger.info message when truncation occurs in the request body or response spend log paths. Adds 3 new tests: - test_truncation_includes_db_safeguard_note - test_response_truncation_logs_info_message - test_request_body_truncation_logs_info_message Co-authored-by: Cursor Agent <cursoragent@cursor.com> * Fix admin viewer unable to see all organizations The /organization/list endpoint only checked for PROXY_ADMIN role, causing PROXY_ADMIN_VIEW_ONLY users to fall into the else branch which restricts results to orgs the user is a member of. Use the existing _user_has_admin_view() helper to include both roles. * feat(ui): add Chat UI — ChatGPT-like interface with MCP tools and streaming (#22937) * feat(ui): add chat message and conversation types * feat(ui): add useChatHistory hook for localStorage-backed conversations * feat(ui): add ConversationList sidebar component * feat(ui): add MCPConnectPicker for attaching MCP servers to chat * feat(ui): add ModelSelector dropdown for chat * feat(ui): add ChatInputBar with MCP tool attachment support * feat(ui): add MCPAppsPanel with list/detail view for MCP servers * feat(ui): add ChatMessages component; remove auto-scrollIntoView that caused scroll-lock bypass * feat(ui): add ChatPage — ChatGPT-like UI with scroll lock, MCP tools, streaming * feat(ui): add /chat route wired to ChatPage * feat(ui): remove chat from leftnav — chat accessible via navbar button * feat(ui): add Chat button to top navbar * feat(ui): add dismissible Chat UI announcement banner to Playground page * feat(proxy): add Chat UI link to Swagger description * feat(ui): add react-markdown and syntax-highlighter deps for chat UI * fix(ui): replace missing BorderOutlined import with inline stop icon div * fix(ui): apply remark-gfm plugin to ReactMarkdown for GFM support * fix(ui): remove unused isEvenRow variable in MCPAppsPanel * fix(ui): add ellipsis when truncating conversation title * fix(ui): wire search button to chats view; remove non-functional keyboard hint * fix(ui): use serverRootPath in navbar chat link for sub-path deployments * fix(ui): remove unused ChatInputBar and ModelSelector files * fix(ui): correct grid bottom-border condition for odd server count * fix(chat): move localStorage writes out of setConversations updater (React purity) * fix(chat): fix stale closure in handleEditAndResend - compute history before async state update * fix(chat): fix 4 issues in ChatMessages - array redaction, clipboard error, inline detection, remove unused ref * docs: add PayGo/priority cost tracking for Gemini Vertex AI - Add PayGo / Priority Cost Tracking section to Vertex AI provider docs - Document trafficType to service_tier mapping (ON_DEMAND_PRIORITY, FLEX, etc.) - Add service tier cost keys to custom pricing docs - Add provider-specific cost tracking note to spend tracking overview Made-with: Cursor * fix: normalize response images missing index + guard audio duration overflow 1. convert_dict_to_response.py (#22640): Providers like OpenRouter/Gemini return images without the required `index` field, causing pydantic ValidationError when constructing Message. Added _normalize_images() to backfill index from enumeration position. 2. audio_utils/utils.py (#22622): libsndfile can report 2^63-1 frames for malformed audio files, causing astronomically large duration values used for cost calculation. Added guards for sentinel frame counts and implausible durations (>24h). Co-Authored-By: claude-flow <ruv@ruv.net> * fix: add type annotations to _normalize_images + guard samplerate==0 Address review feedback: - Add type hints to _normalize_images() for consistency with codebase - Guard against samplerate <= 0 to prevent ZeroDivisionError on malformed audio files Co-Authored-By: claude-flow <ruv@ruv.net> --------- Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Ryan Crabbe <rcrabbe@berkeley.edu> Co-authored-by: ryan-crabbe <128659760+ryan-crabbe@users.noreply.github.com> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: Sameer Kankute <sameer@berri.ai> Co-authored-by: claude-flow <ruv@ruv.net>
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat(ui): add Chat UI — ChatGPT-like interface with MCP tools and streaming
Closes #chat-ui
What
Adds a new
/chatroute to the LiteLLM dashboard — a ChatGPT-style chat interface for users to chat with any model on the proxy, with MCP tool support and streaming.New components:
ChatPage— main layout with collapsible sidebar, scroll-lock during streaming, transparent scroll-to-bottom arrowChatMessages— markdown rendering, code highlighting, reasoning content, edit+resendMCPAppsPanel— list/detail view for connecting MCP servers (LiteLLM design, not copy-paste)ConversationList,useChatHistory— localStorage-backed conversation historyMCPConnectPicker,ModelSelector,ChatInputBar— supporting componentsOther changes:
ChatMessageswas callingscrollIntoViewon every streamed token, bypassing the scroll lock. Removed it — scrolling is now managed entirely byChatPageviauseLayoutEffect+streamScrollLockrefPre-Submission checklist
Type