LLM error truncation note by krrishdholakia · Pull Request #22936 · BerriAI/litellm

krrishdholakia · 2026-03-05T23:44:17Z

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
My PR passes all unit tests on make test-unit
My PR's scope is as isolated as possible, it only solves 1 specific problem
I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

CI (LiteLLM team)

CI status guideline:

50-55 passing tests: main is stable with minor issues.

45-49 passing tests: acceptable but needs attention

<= 40 passing tests: unstable; be careful with your merges and assess the risk.

Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
✅ Test

Changes

Adds a note to truncated log messages explaining that truncation is a DB safeguard and that full, untruncated data is sent to logging callbacks (e.g., OTEL, Datadog). This clarifies why certain log fields might appear truncated in the database while being complete in other logging systems.

Introduced LITELLM_TRUNCATION_DB_SAFEGUARD_NOTE constant.
Modified _sanitize_request_body_for_spend_logs_payload to include this safeguard note in the truncation marker.
Added verbose info logs when request or response bodies are truncated.
Updated existing tests and added new tests to verify the presence of the safeguard note and the new logging behavior.

…or DB storage When the messages or response JSON fields in spend logs are truncated before being written to the database, the truncation marker now includes a note explaining: - This is a DB storage safeguard - Full, untruncated data is still sent to logging callbacks (OTEL, Datadog, etc.) - The MAX_STRING_LENGTH_PROMPT_IN_DB env var can be used to increase the limit Also emits a verbose_proxy_logger.info message when truncation occurs in the request body or response spend log paths. Adds 3 new tests: - test_truncation_includes_db_safeguard_note - test_response_truncation_logs_info_message - test_request_body_truncation_logs_info_message Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>

cursor · 2026-03-05T23:44:18Z

Cursor Agent can help with this pull request. Just @cursor in comments and I'll start working on changes in this branch.
_{Learn more about Cursor Agents}

vercel · 2026-03-05T23:44:22Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Mar 5, 2026 11:45pm

CLAassistant · 2026-03-05T23:44:24Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

greptile-apps · 2026-03-06T01:02:30Z

Greptile Summary

This PR adds a human-readable explanation (LITELLM_TRUNCATION_DB_SAFEGUARD_NOTE) to the truncation marker embedded in spend-log fields, and emits an info-level log whenever a request or response body is truncated before DB storage. The change is narrowly scoped to spend-tracking utilities and fits naturally alongside the existing LITELLM_TRUNCATED_PAYLOAD_FIELD constant.

Key observations:

The safeguard note (~201 chars) is appended inside every truncated field value, which means each truncated DB column will store approximately MAX_STRING_LENGTH_PROMPT_IN_DB + 270 characters — about 13% beyond the intended default limit of 2048. The new verbose_proxy_logger.info calls already surface this note in logs, so embedding it in the stored value is redundant.
Constants are imported both at module level (lines 14–17) and again inside _sanitize_request_body_for_spend_logs_payload (lines 635–638), making the local import redundant.
Tests are all mock-based and follow the project's no-network-calls rule for this directory.
One test assertion contains a redundant condition that can be simplified.

Confidence Score: 4/5

Safe to merge with minor concerns around redundant imports and embedded marker size.
The PR safely adds informative logging and clarifies truncation behavior via the safeguard note. The main concern is that the note (~201 chars) is embedded in every truncated field, inflating DB storage by ~13% beyond the intended limit—though this does not cause functional regression or data loss. The code contains a redundant local import that should be removed. No architectural flaws or security issues identified.
litellm/proxy/spend_tracking/spend_tracking_utils.py — redundant local import and marker size inflation require attention.

Important Files Changed

Filename	Overview
litellm/constants.py	Adds `LITELLM_TRUNCATION_DB_SAFEGUARD_NOTE` constant (~201 chars) placed alongside existing truncation constants. Straightforward and self-contained.
litellm/proxy/spend_tracking/spend_tracking_utils.py	Appends safeguard note to the in-value truncation marker, inflating stored DB field sizes beyond the intended MAX_STRING_LENGTH_PROMPT_IN_DB limit. Contains redundant local import of constants already imported at module level. The truncation-detection logic and info-log feature work as intended.
tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py	New tests cover safeguard note presence and info-log emission on truncation; all mock-based and locally runnable. One test assertion contains a redundant branch that can be simplified.

_{Last reviewed commit: b211407}

greptile-apps · 2026-03-06T01:02:34Z

litellm/proxy/spend_tracking/spend_tracking_utils.py

+    from litellm.constants import (
+        LITELLM_TRUNCATED_PAYLOAD_FIELD,
+        LITELLM_TRUNCATION_DB_SAFEGUARD_NOTE,
+    )


These constants are already imported at the module level (lines 14–17). This local import is now redundant and can be removed.

Suggested change

from litellm.constants import (

LITELLM_TRUNCATED_PAYLOAD_FIELD,

LITELLM_TRUNCATION_DB_SAFEGUARD_NOTE,

)

if visited is None:

greptile-apps · 2026-03-06T01:02:35Z

litellm/proxy/spend_tracking/spend_tracking_utils.py

                truncated_value = (
                    f"{value[:start_chars]}"
-                    f"... ({LITELLM_TRUNCATED_PAYLOAD_FIELD} skipped {skipped_chars} chars) ..."
+                    f"... ({LITELLM_TRUNCATED_PAYLOAD_FIELD} skipped {skipped_chars} chars. "
+                    f"{LITELLM_TRUNCATION_DB_SAFEGUARD_NOTE}) ..."
                    f"{value[-end_chars:]}"
                )
                return truncated_value


The truncation marker now includes LITELLM_TRUNCATION_DB_SAFEGUARD_NOTE (~201 characters), which means every truncated field stored in the database will exceed the intended MAX_STRING_LENGTH_PROMPT_IN_DB limit by approximately 270+ characters. With a default limit of 2048, this represents a ~13% size increase per truncated field—the opposite of the intended safeguard.

The safeguard note is already surfaced in logs via the new verbose_proxy_logger.info() calls (lines 803–806 and 886–889), making the embedded text redundant. Consider either:

Storing the note in a separate metadata field

Keeping the in-value marker minimal and relying on the log-level info messages

greptile-apps · 2026-03-06T01:02:36Z

tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py

+    assert LITELLM_TRUNCATED_PAYLOAD_FIELD in truncated
+    assert LITELLM_TRUNCATION_DB_SAFEGUARD_NOTE in truncated
+    assert "DB storage safeguard" in truncated
+    assert "logging callbacks" in truncated.lower() or "logging integrations" in truncated.lower() or "logging callbacks" in truncated


The third or condition is redundant. Since LITELLM_TRUNCATION_DB_SAFEGUARD_NOTE already contains the lowercase phrase "logging callbacks", the expression "logging callbacks" in truncated will always be True if "logging callbacks" in truncated.lower() is True.

Suggested change

assert "logging callbacks" in truncated.lower() or "logging integrations" in truncated.lower() or "logging callbacks" in truncated

assert "logging callbacks" in truncated.lower() or "logging integrations" in truncated.lower()

…verflow (#22950) * feat(spend-logs): add truncation note when error logs are truncated for DB storage (#22936) When the messages or response JSON fields in spend logs are truncated before being written to the database, the truncation marker now includes a note explaining: - This is a DB storage safeguard - Full, untruncated data is still sent to logging callbacks (OTEL, Datadog, etc.) - The MAX_STRING_LENGTH_PROMPT_IN_DB env var can be used to increase the limit Also emits a verbose_proxy_logger.info message when truncation occurs in the request body or response spend log paths. Adds 3 new tests: - test_truncation_includes_db_safeguard_note - test_response_truncation_logs_info_message - test_request_body_truncation_logs_info_message Co-authored-by: Cursor Agent <cursoragent@cursor.com> * Fix admin viewer unable to see all organizations The /organization/list endpoint only checked for PROXY_ADMIN role, causing PROXY_ADMIN_VIEW_ONLY users to fall into the else branch which restricts results to orgs the user is a member of. Use the existing _user_has_admin_view() helper to include both roles. * feat(ui): add Chat UI — ChatGPT-like interface with MCP tools and streaming (#22937) * feat(ui): add chat message and conversation types * feat(ui): add useChatHistory hook for localStorage-backed conversations * feat(ui): add ConversationList sidebar component * feat(ui): add MCPConnectPicker for attaching MCP servers to chat * feat(ui): add ModelSelector dropdown for chat * feat(ui): add ChatInputBar with MCP tool attachment support * feat(ui): add MCPAppsPanel with list/detail view for MCP servers * feat(ui): add ChatMessages component; remove auto-scrollIntoView that caused scroll-lock bypass * feat(ui): add ChatPage — ChatGPT-like UI with scroll lock, MCP tools, streaming * feat(ui): add /chat route wired to ChatPage * feat(ui): remove chat from leftnav — chat accessible via navbar button * feat(ui): add Chat button to top navbar * feat(ui): add dismissible Chat UI announcement banner to Playground page * feat(proxy): add Chat UI link to Swagger description * feat(ui): add react-markdown and syntax-highlighter deps for chat UI * fix(ui): replace missing BorderOutlined import with inline stop icon div * fix(ui): apply remark-gfm plugin to ReactMarkdown for GFM support * fix(ui): remove unused isEvenRow variable in MCPAppsPanel * fix(ui): add ellipsis when truncating conversation title * fix(ui): wire search button to chats view; remove non-functional keyboard hint * fix(ui): use serverRootPath in navbar chat link for sub-path deployments * fix(ui): remove unused ChatInputBar and ModelSelector files * fix(ui): correct grid bottom-border condition for odd server count * fix(chat): move localStorage writes out of setConversations updater (React purity) * fix(chat): fix stale closure in handleEditAndResend - compute history before async state update * fix(chat): fix 4 issues in ChatMessages - array redaction, clipboard error, inline detection, remove unused ref * docs: add PayGo/priority cost tracking for Gemini Vertex AI - Add PayGo / Priority Cost Tracking section to Vertex AI provider docs - Document trafficType to service_tier mapping (ON_DEMAND_PRIORITY, FLEX, etc.) - Add service tier cost keys to custom pricing docs - Add provider-specific cost tracking note to spend tracking overview Made-with: Cursor * fix: normalize response images missing index + guard audio duration overflow 1. convert_dict_to_response.py (#22640): Providers like OpenRouter/Gemini return images without the required `index` field, causing pydantic ValidationError when constructing Message. Added _normalize_images() to backfill index from enumeration position. 2. audio_utils/utils.py (#22622): libsndfile can report 2^63-1 frames for malformed audio files, causing astronomically large duration values used for cost calculation. Added guards for sentinel frame counts and implausible durations (>24h). Co-Authored-By: claude-flow <ruv@ruv.net> * fix: add type annotations to _normalize_images + guard samplerate==0 Address review feedback: - Add type hints to _normalize_images() for consistency with codebase - Guard against samplerate <= 0 to prevent ZeroDivisionError on malformed audio files Co-Authored-By: claude-flow <ruv@ruv.net> --------- Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Ryan Crabbe <rcrabbe@berkeley.edu> Co-authored-by: ryan-crabbe <128659760+ryan-crabbe@users.noreply.github.com> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: Sameer Kankute <sameer@berri.ai> Co-authored-by: claude-flow <ruv@ruv.net>

vercel bot deployed to Preview March 5, 2026 23:45 View deployment

krrishdholakia marked this pull request as ready for review March 6, 2026 00:58

krrishdholakia merged commit 53a1e31 into main Mar 6, 2026
31 of 42 checks passed

greptile-apps bot reviewed Mar 6, 2026

View reviewed changes

krrishdholakia mentioned this pull request Mar 6, 2026

fix: normalize response images missing index + guard audio duration o… #22955

Open

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

LLM error truncation note#22936

LLM error truncation note#22936
krrishdholakia merged 1 commit intomainfrom
cursor/llm-error-truncation-note-32be

krrishdholakia commented Mar 5, 2026

Uh oh!

cursor bot commented Mar 5, 2026

Uh oh!

vercel bot commented Mar 5, 2026 •

edited

Loading

Uh oh!

CLAassistant commented Mar 5, 2026

Uh oh!

Uh oh!

greptile-apps bot commented Mar 6, 2026

Important Files Changed

Uh oh!

greptile-apps bot Mar 6, 2026

Uh oh!

greptile-apps bot Mar 6, 2026

Uh oh!

greptile-apps bot Mar 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	assert "logging callbacks" in truncated.lower() or "logging integrations" in truncated.lower() or "logging callbacks" in truncated
	assert "logging callbacks" in truncated.lower() or "logging integrations" in truncated.lower()

Uh oh!

Conversation

krrishdholakia commented Mar 5, 2026

Relevant issues

Pre-Submission checklist

CI (LiteLLM team)

Type

Changes

Uh oh!

cursor bot commented Mar 5, 2026

Uh oh!

vercel bot commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CLAassistant commented Mar 5, 2026

Uh oh!

Uh oh!

greptile-apps bot commented Mar 6, 2026

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Uh oh!

greptile-apps bot Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vercel bot commented Mar 5, 2026 •

edited

Loading