[Feat] AI Gateway - Add Tracing for MCP Calls running through AI Gateway - #21018
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Greptile OverviewGreptile SummaryThis PR adds tracing for MCP (Model Context Protocol) tool calls flowing through the AI Gateway by threading a Backend changes:
Frontend changes:
Critical issue:
Missing from the PR checklist:
Confidence Score: 1/5
|
| Filename | Overview |
|---|---|
| litellm/proxy/_experimental/mcp_server/server.py | Adds litellm_trace_id parameter to _get_tools_from_mcp_servers and threads it into spend log metadata; refactors user metadata attachment to use standard LiteLLMProxyRequestSetup helper instead of manual key extraction. |
| litellm/proxy/spend_tracking/spend_management_endpoints.py | Extracts _build_ui_spend_logs_response helper that enriches spend log rows with session_total_count via a GROUP BY query on session_id. Adds an extra DB query per page load for the v1/UI endpoint. No new tests cover the enrichment logic. |
| litellm/responses/mcp/litellm_proxy_mcp_handler.py | Threads litellm_trace_id through _get_mcp_tools_from_manager, _process_mcp_tools_to_openai_format, and _process_mcp_tools_without_openai_transform. Mostly import reordering and parameter addition. |
| ui/litellm-dashboard/src/components/view_logs/LogDetailsDrawer/LogDetailsDrawer.tsx | Major rewrite: adds session-mode sidebar with trace event list, collapsible panel, and session metrics. Imports missing LogDetailContent component — file does not exist, causing a build failure. Also adds sessionId and accessToken props for fetching session logs via React Query. |
| ui/litellm-dashboard/src/components/view_logs/LogDetailsDrawer/index.ts | Re-exports LogDetailContent and LogDetailContentProps from ./LogDetailContent — but the source file does not exist, causing build failure. |
| ui/litellm-dashboard/src/components/view_logs/columns.tsx | Adds a new "Type" column showing LLM/MCP badges; extends LogEntry type with session composition fields; enhances Cost column to show MCP sub-cost breakdown. |
| ui/litellm-dashboard/src/components/view_logs/index.tsx | Replaces separate SessionView page with in-drawer session mode; adds session composition counting and O(n^2) deduplication filter for multi-call sessions; passes sessionId/accessToken to LogDetailsDrawer. |
Sequence Diagram
sequenceDiagram
participant Client
participant Proxy as LiteLLM Proxy
participant MCP as MCP Server Manager
participant DB as Spend Logs DB
participant UI as Dashboard UI
Client->>Proxy: responses API / chat completions (with litellm_trace_id)
Proxy->>MCP: _process_mcp_tools_without_openai_transform(litellm_trace_id)
MCP->>MCP: _get_mcp_tools_from_manager(litellm_trace_id)
MCP->>MCP: _get_tools_from_mcp_servers(litellm_trace_id)
MCP->>DB: Log list_mcp_tools with litellm_trace_id + user metadata
MCP-->>Proxy: MCP tools list
Proxy->>Proxy: Execute LLM call + MCP tool calls
Proxy->>DB: Log LLM call & MCP tool calls (same trace_id / session_id)
UI->>Proxy: GET /spend/logs/ui
Proxy->>DB: find_many (paginated spend logs)
Proxy->>DB: GROUP BY session_id (session_total_count enrichment)
Proxy-->>UI: Enriched rows with session_total_count
UI->>UI: Deduplicate multi-call sessions, show Type badges
UI->>UI: Open drawer → fetch session logs → show trace sidebar
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Contributor
Author
|
@greptile review again |
Contributor
Greptile OverviewGreptile SummaryAdds end-to-end tracing for MCP tool calls flowing through the AI Gateway by threading
Confidence Score: 3/5
|
| Filename | Overview |
|---|---|
| litellm/proxy/_experimental/mcp_server/server.py | Adds litellm_trace_id parameter to _get_tools_from_mcp_servers and uses LiteLLMProxyRequestSetup.add_user_api_key_auth_to_request_metadata for consistent metadata enrichment. Clean, targeted change. |
| litellm/proxy/spend_tracking/spend_management_endpoints.py | Adds _build_ui_spend_logs_response helper that enriches spend log rows with session_total_count via a GROUP BY query. The query is bounded by page_size. Note: response format changes from raw Prisma objects to explicitly model_dump()-ed dicts — should be functionally equivalent but is a behavioral change worth noting. |
| litellm/responses/mcp/litellm_proxy_mcp_handler.py | Threads litellm_trace_id through _get_mcp_tools_from_manager, _process_mcp_tools_to_openai_format, and _process_mcp_tools_without_openai_transform. Also sorts imports alphabetically. Clean plumbing. |
| ui/litellm-dashboard/src/components/view_logs/LogDetailsDrawer/LogDetailContent.tsx | New file — extracts the log detail rendering logic from LogDetailsDrawer into a standalone component. Well-structured with proper sub-components for errors, tags, metrics, request/response, and metadata sections. |
| ui/litellm-dashboard/src/components/view_logs/LogDetailsDrawer/LogDetailsDrawer.tsx | Major refactor — adds session mode with sidebar trace event list, session log fetching via useQuery, and MCP/LLM type differentiation. Delegates detail rendering to LogDetailContent. Complex but well-organized. Uses navigator.clipboard without error handling. |
| ui/litellm-dashboard/src/components/view_logs/columns.tsx | Adds a new "Type" column with LLM/MCP badges and session composition indicators. Extends LogEntry type with session count fields. Cost column updated to show MCP cost breakdown when applicable. |
| ui/litellm-dashboard/src/components/view_logs/index.tsx | Replaces the old full-page SessionView with inline drawer-based session viewing. Adds session deduplication with efficient O(n) single-pass map. Passes sessionId and accessToken to drawer. Removes sessionSpendLogsCall import as fetching moved to drawer. |
| ui/litellm-dashboard/src/components/view_logs/table.tsx | Adds renderChildRows prop for MCP child row rendering alongside existing renderSubComponent. Renames isLegacyMode to supportsExpansion and fixes row click behavior to be based on onRowClick presence rather than mode. |
Sequence Diagram
sequenceDiagram
participant Client
participant Proxy as LiteLLM Proxy
participant MCP as MCP Server
participant DB as SpendLogs DB
participant UI as Dashboard UI
Client->>Proxy: responses API / chat completion (with trace_id)
Proxy->>MCP: list_mcp_tools (litellm_trace_id)
MCP-->>Proxy: available tools
Proxy->>DB: log list_tools (linked by trace_id)
Proxy->>MCP: call_mcp_tool (litellm_trace_id)
MCP-->>Proxy: tool result
Proxy->>DB: log call_tool (linked by trace_id)
Proxy-->>Client: response with MCP metadata
UI->>DB: GET /spend/logs/ui (paginated)
DB-->>UI: rows + session_total_count (GROUP BY)
Note over UI: Deduplicate sessions, show Type badges
UI->>UI: Click session row → open drawer
UI->>DB: GET /spend/logs/session/ui?session_id=X
DB-->>UI: all session logs
Note over UI: Sidebar trace timeline (LLM + MCP events)
Contributor
Author
|
@greptile review this again |
Contributor
Greptile OverviewGreptile SummaryAdds end-to-end tracing for MCP (Model Context Protocol) tool calls flowing through the AI Gateway by threading
Confidence Score: 3/5
|
| Filename | Overview |
|---|---|
| litellm/proxy/_experimental/mcp_server/server.py | Adds litellm_trace_id parameter to _get_tools_from_mcp_servers and switches to using the standard LiteLLMProxyRequestSetup.add_user_api_key_auth_to_request_metadata helper for consistent metadata enrichment. Clean, safe changes. |
| litellm/proxy/spend_tracking/spend_management_endpoints.py | Extracts _build_ui_spend_logs_response helper to enrich v1 UI spend-log rows with session_total_count via a GROUP BY query. Bounded by page_size; v2 endpoint is unaffected. |
| litellm/responses/mcp/litellm_proxy_mcp_handler.py | Threads litellm_trace_id through _get_mcp_tools_from_manager, _process_mcp_tools_to_openai_format, and _process_mcp_tools_without_openai_transform. Import reordering throughout. |
| ui/litellm-dashboard/src/components/view_logs/LogDetailsDrawer/LogDetailContent.tsx | New file extracting log detail rendering (error alerts, metrics, request/response, guardrails, metadata) from LogDetailsDrawer into a reusable component for both single-log and session views. |
| ui/litellm-dashboard/src/components/view_logs/LogDetailsDrawer/LogDetailsDrawer.tsx | Major refactor to support session mode: adds sidebar with trace event list, fetches session logs via react-query, and delegates content rendering to LogDetailContent. Session duration calculation uses sort-order indices instead of min/max timestamps (bug). |
| ui/litellm-dashboard/src/components/view_logs/columns.tsx | Adds "Type" column showing LLM/MCP badges, enriches LogEntry type with session fields, and shows MCP cost breakdown in the Cost column. |
| ui/litellm-dashboard/src/components/view_logs/index.tsx | Replaces full-page SessionView with drawer-based session support. Adds session composition calculation, representative-row deduplication for multi-call sessions, and passes sessionId/accessToken to LogDetailsDrawer. |
Sequence Diagram
sequenceDiagram
participant UI as Dashboard UI
participant Proxy as LiteLLM Proxy
participant DB as Spend Logs DB
participant MCP as MCP Server
UI->>Proxy: POST /responses (with MCP tools)
Proxy->>Proxy: Generate litellm_trace_id
Proxy->>MCP: list_tools (litellm_trace_id in metadata)
MCP-->>Proxy: Available tools
Proxy->>DB: Log list_mcp_tools (with trace_id)
Proxy->>MCP: call_mcp_tool
MCP-->>Proxy: Tool result
Proxy->>DB: Log call_mcp_tool (with trace_id)
Proxy-->>UI: Response
UI->>Proxy: GET /spend/logs (v1)
Proxy->>DB: Fetch paginated logs
Proxy->>DB: GROUP BY session_id (counts)
Proxy-->>UI: Enriched logs with session_total_count
UI->>UI: Show LLM/MCP badges, session grouping
UI->>Proxy: GET /session/logs/{session_id}
Proxy->>DB: Fetch session logs
Proxy-->>UI: Session log entries
UI->>UI: Render trace sidebar in drawer
Comment on lines
+191
to
+192
| const sessionStart = sessionLogs.length > 0 ? new Date(sessionLogs[0].startTime) : null; | ||
| const sessionEnd = sessionLogs.length > 0 ? new Date(sessionLogs[sessionLogs.length - 1].endTime) : null; |
Contributor
There was a problem hiding this comment.
sessionLogs is sorted by type (LLM first, MCP second), not strictly by time. Using sessionLogs[0] / sessionLogs[sessionLogs.length - 1] for the session time range will produce wrong values when a session has both LLM and MCP calls — the last MCP call might start before the last LLM call chronologically.
Use Math.min / Math.max across all entries instead:
Suggested change
| const sessionStart = sessionLogs.length > 0 ? new Date(sessionLogs[0].startTime) : null; | |
| const sessionEnd = sessionLogs.length > 0 ? new Date(sessionLogs[sessionLogs.length - 1].endTime) : null; | |
| const sessionStart = sessionLogs.length > 0 ? new Date(Math.min(...sessionLogs.map(r => new Date(r.startTime).getTime()))) : null; | |
| const sessionEnd = sessionLogs.length > 0 ? new Date(Math.max(...sessionLogs.map(r => new Date(r.endTime).getTime()))) : null; |
ghost
pushed a commit
that referenced
this pull request
Feb 12, 2026
- Fix session time range calculation: use Math.min/Math.max across all entries instead of relying on array order (sessionLogs is sorted by type, not time). Other Greptile comments were already addressed in the branch: - LogDetailContent.tsx exists - Clipboard call already wrapped in try/catch - Dedup already uses O(1) Map lookup - model_dump() serialization is documented - GROUP BY performance comment already present
ghost
mentioned this pull request
- Fix session time range calculation: use Math.min/Math.max across all entries instead of relying on array order (sessionLogs is sorted by type, not time). Other Greptile comments were already addressed in the branch: - LogDetailContent.tsx exists - Clipboard call already wrapped in try/catch - Dedup already uses O(1) Map lookup - model_dump() serialization is documented - GROUP BY performance comment already present
|
|
Contributor
Author
|
@greptile review again |
Contributor
Greptile OverviewGreptile SummaryAdds end-to-end tracing for MCP (Model Context Protocol) tool calls running through the AI Gateway by threading Backend changes:
Frontend changes:
Note: No tests were added in the Confidence Score: 4/5
|
| Filename | Overview |
|---|---|
| litellm/proxy/_experimental/mcp_server/server.py | Added litellm_trace_id parameter to _get_tools_from_mcp_servers and propagated it into request data for spend log linkage. Switched to standard LiteLLMProxyRequestSetup helper for attaching user auth metadata. |
| litellm/proxy/spend_tracking/spend_management_endpoints.py | Added _build_ui_spend_logs_response helper that enriches rows with session_total_count via a GROUP BY query for v1 UI. New direct DB query is in the UI endpoint, not the LLM request critical path. |
| litellm/responses/mcp/litellm_proxy_mcp_handler.py | Threaded litellm_trace_id parameter through _get_mcp_tools_from_manager, _process_mcp_tools_to_openai_format, and _process_mcp_tools_without_openai_transform. Import re-ordering is style-only. |
| ui/litellm-dashboard/src/components/view_logs/LogDetailsDrawer/LogDetailContent.tsx | New file: extracts the scrollable detail content from LogDetailsDrawer into a reusable component for both single-log and session-mode views. |
| ui/litellm-dashboard/src/components/view_logs/LogDetailsDrawer/LogDetailsDrawer.tsx | Major refactor: added session mode with collapsible sidebar, trace event navigation, session metrics. Detail content extracted to LogDetailContent. Session logs fetched via react-query. |
| ui/litellm-dashboard/src/components/view_logs/columns.tsx | Added "Type" column with LLM/MCP badges, enriched "Cost" column with MCP spend breakdown, and added new LogEntry fields for session composition. |
| ui/litellm-dashboard/src/components/view_logs/index.tsx | Removed full-page SessionView, replaced with drawer-based session mode. Added efficient single-pass session deduplication. Multi-call sessions open in the drawer. |
| ui/litellm-dashboard/src/components/view_logs/table.tsx | Added renderChildRows prop for MCP child rows as real table rows alongside existing colspan sub-component pattern for audit logs. |
Sequence Diagram
sequenceDiagram
participant UI as Dashboard UI
participant API as LiteLLM Proxy
participant DB as Prisma DB
participant MCP as MCP Server
UI->>API: POST /responses (with mcp_tools)
API->>API: Generate litellm_trace_id
API->>MCP: list_mcp_tools (with litellm_trace_id)
MCP-->>API: Tool list
API->>DB: Log list_mcp_tools (trace_id links to parent)
API->>MCP: call_mcp_tool (with litellm_trace_id)
MCP-->>API: Tool result
API->>DB: Log call_mcp_tool (trace_id links to parent)
API-->>UI: Response
Note over UI,DB: UI Log Viewing Flow
UI->>API: GET /spend/logs/ui (v1)
API->>DB: Fetch paginated spend logs
API->>DB: GROUP BY session_id (bounded by page_size)
DB-->>API: Session counts
API-->>UI: Enriched logs with session_total_count
UI->>API: GET /spend/logs/session/ui?session_id=X
API->>DB: Fetch all logs for session
DB-->>API: Session logs
API-->>UI: Session logs (displayed in drawer sidebar)
Comment on lines
+3153
to
+3157
| counts = await prisma_client.db.litellm_spendlogs.group_by( | ||
| by=["session_id"], | ||
| where={"session_id": {"in": session_ids}}, | ||
| count={"session_id": True}, | ||
| ) |
Contributor
There was a problem hiding this comment.
This GROUP BY query runs on every v1 UI page load. The IN clause is bounded by page_size (typically 25–50 distinct session IDs), so it should be fine at moderate scale. Consider a window function in the main query if performance becomes a concern at scale.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Merged
1 task
fzowl
pushed a commit
to fzowl/litellm
that referenced
this pull request
Jun 24, 2026
…way (BerriAI#21018) * commit new expansion * fix MCP * fix: LiteLLMProxyRequestSetup * _process_mcp_tools_without_openai_transform * UI fixes * UI refactor view logs/sessions * index * _add_mcp_tool_metadata_to_final_chunk * add badges * add getEventDisplayName * ui fixes * backend fix * fix * UI fix * UI fix * fix row * fix: address Greptile review feedback on PR BerriAI#21018 (BerriAI#21057) - Fix session time range calculation: use Math.min/Math.max across all entries instead of relying on array order (sessionLogs is sorted by type, not time). Other Greptile comments were already addressed in the branch: - LogDetailContent.tsx exists - Clipboard call already wrapped in try/catch - Dedup already uses O(1) Map lookup - model_dump() serialization is documented - GROUP BY performance comment already present --------- Co-authored-by: shin-bot-litellm <shin-bot-litellm@berri.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[Feat] AI Gateway - Add Tracing for MCP Calls running through AI Gateway
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewCI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Type
🆕 New Feature
✅ Test
Changes