fix(proxy): honor MAX_STRING_LENGTH_PROMPT_IN_DB from config env vars by gavksingh · Pull Request #22106 · BerriAI/litellm

gavksingh · 2026-02-25T17:45:55Z

Fixes #22088

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
My PR passes all unit tests on make test-unit
My PR's scope is as isolated as possible, it only solves 1 specific problem
I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

CI (LiteLLM team)

CI status guideline:

50-55 passing tests: main is stable with minor issues.

45-49 passing tests: acceptable but needs attention

<= 40 passing tests: unstable; be careful with your merges and assess the risk.

Branch creation CI run
Link: <add link>
CI run for the last commit
Link: <add link>
Merge / cherry-pick CI run
Links: <add link(s)>

Type

🐛 Bug Fix
✅ Test

Changes

Issue #22088 root cause was import-time evaluation of MAX_STRING_LENGTH_PROMPT_IN_DB in spend-tracking sanitization paths. Proxy config environment_variables are loaded later via _load_environment_variables(), so the imported value could become stale.

Updated litellm/proxy/spend_tracking/spend_tracking_utils.py

Resolve max prompt length at runtime from os.environ.
Use fallback from litellm.constants default constant when env is missing or invalid.
_sanitize_request_body_for_spend_logs_payload() resolves the limit once per call and threads it through recursive calls.
Truncation behavior and formatting are unchanged. Only value resolution timing changed.

Added regression coverage in tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py

Added test that sets MAX_STRING_LENGTH_PROMPT_IN_DB after import path setup.
Verifies sanitization uses runtime env override, not stale import-time value.

Validation (targeted)

tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py -> 32 passed
tests/test_litellm/proxy/test_proxy_server.py -k load_environment_variables -> 2 passed, 84 deselected

Scope and impact

Scope remains isolated to 2 files.
No public API changes.
No schema changes.
Fix ensures config-loaded env overrides are honored for spend-log truncation threshold.

vercel · 2026-02-25T17:45:59Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 25, 2026 11:56pm

greptile-apps · 2026-02-25T17:48:53Z

Greptile Summary

This PR fixes issue #22088 where MAX_STRING_LENGTH_PROMPT_IN_DB was resolved at import time in the spend-tracking sanitization path, causing proxy config environment_variables (loaded later via _load_environment_variables()) to be ignored.

Adds a new _get_max_string_length_prompt_in_db() helper in spend_tracking_utils.py that reads os.environ at runtime, falling back to the imported constant from litellm.constants
Threads the resolved value through _sanitize_request_body_for_spend_logs_payload() via a new optional parameter to avoid redundant lookups on recursive calls
Adds a regression test using monkeypatch.setenv to verify the runtime override is honored
No public API changes, no schema changes, and truncation behavior is unchanged

Confidence Score: 5/5

This PR is safe to merge — it's a minimal, well-scoped fix with no behavioral changes beyond when the config value is resolved.
The change is isolated to a single function's value resolution timing. The approach is clean (runtime os.getenv with constant fallback), the recursive threading is correct, and it includes a proper regression test. No public API surface is changed, no new dependencies are introduced, and all custom rules are satisfied.
No files require special attention.

Important Files Changed

Filename	Overview
litellm/proxy/spend_tracking/spend_tracking_utils.py	Adds `_get_max_string_length_prompt_in_db()` to resolve the truncation limit from `os.environ` at runtime, replacing the stale import-time constant. The limit is threaded through `_sanitize_request_body_for_spend_logs_payload` via a new optional parameter. No behavioral changes beyond resolution timing.
tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py	Adds a regression test that sets `MAX_STRING_LENGTH_PROMPT_IN_DB` via `monkeypatch.setenv` after module import, verifying that the sanitization function honors the runtime env override. Also removes a trailing blank line at end of file.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["_sanitize_request_body_for_spend_logs_payload()"] -->|max_string_length is None| B["_get_max_string_length_prompt_in_db()"]
    B --> C{"os.getenv('MAX_STRING_LENGTH_PROMPT_IN_DB')"}
    C -->|Set & valid int| D["Use runtime env value"]
    C -->|Not set| E["Use DEFAULT_MAX_STRING_LENGTH_PROMPT_IN_DB from constants"]
    C -->|Invalid value| E
    D --> F["Sanitize strings using resolved limit"]
    E --> F
    F -->|Nested dict| G["Recurse with same limit (no re-lookup)"]
    G --> F

_{Last reviewed commit: 078be19}

greptile-apps

_{2 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps · 2026-02-25T17:48:57Z

+    Resolve prompt truncation threshold at runtime so values loaded later via
+    proxy config environment_variables are honored.
+    """
+    default_max = 2048


Hardcoded default duplicates constant

The default value 2048 is duplicated here and in litellm/constants.py:348 (MAX_STRING_LENGTH_PROMPT_IN_DB = int(os.getenv("MAX_STRING_LENGTH_PROMPT_IN_DB", 2048))). If the default ever changes in constants.py, this location could silently drift out of sync.

Consider importing the constant as the fallback default instead of hardcoding it:

Suggested change

default_max = 2048

default_max = 2048 # keep in sync with litellm.constants.MAX_STRING_LENGTH_PROMPT_IN_DB

Alternatively, you could import the constant and use it as the fallback (since by the time this function is called — not imported — the constant is already resolved from the env at module load time of constants.py):

from litellm.constants import MAX_STRING_LENGTH_PROMPT_IN_DB as _DEFAULT_MAX_STRING_LENGTH def _get_max_string_length_prompt_in_db() -> int: max_length_str = os.getenv("MAX_STRING_LENGTH_PROMPT_IN_DB") if max_length_str is None: return _DEFAULT_MAX_STRING_LENGTH ...

This is a minor style nit — the current approach is functionally correct.

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

ghost · 2026-02-25T18:08:29Z

🚅 Shin's PR Review

1. Does this PR fix the issue it describes?
Yes. Issue #22088 reports that MAX_STRING_LENGTH_PROMPT_IN_DB is imported at module load time, before _load_environment_variables() runs, so config values are ignored. The fix moves the import to runtime, which correctly addresses the timing issue.

2. Has this issue already been solved elsewhere?
Not exactly. PR #14042 added the original MAX_STRING_LENGTH_PROMPT_IN_DB config support, but that didn't account for the proxy's environment variable loading order. This is a new fix for a timing bug introduced by the config loading flow.

3. Are there other PRs addressing the same problem?
PR #22093 (docs) updates the default value documentation, but doesn't fix the actual loading bug. This is the only PR fixing the runtime issue.

4. Are there other issues this potentially closes?
No other related issues found.

✅ Looks good — straightforward fix with tests.

gavksingh · 2026-02-25T19:38:32Z

@greptileai review

greptile-apps

_{2 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

…DB runtime resolver

gavksingh · 2026-02-25T20:48:24Z

@greptileai review

greptile-apps

_{2 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps · 2026-02-25T20:51:17Z

    ), "max_retries should be None when not provided"
-
-
 def test_get_request_duration_ms_normal():


Missing blank line between functions

The two blank lines separating test_get_logging_payload_handles_missing_retry_info_gracefully and test_get_request_duration_ms_normal were removed, violating PEP 8's two-blank-lines-between-top-level-definitions convention. This appears to be an accidental whitespace change.

Suggested change

), "max_retries should be None when not provided"

def test_get_request_duration_ms_normal():

), "max_retries should be None when not provided"

def test_get_request_duration_ms_normal():

gavksingh · 2026-02-26T00:00:29Z

@greptileai review

gavksingh · 2026-02-26T00:05:40Z

@greptileai review

gavksingh · 2026-02-26T00:08:30Z

Hi, @krrishdholakia @ishaan-jaff the Greptile review passed with a 5/5 (safe to merge) and ready for maintainer review whenever you have a moment! Fixes #22088.

…#22106) * fix(proxy): honor MAX_STRING_LENGTH_PROMPT_IN_DB from config env vars * fix(proxy): reuse constants fallback for MAX_STRING_LENGTH_PROMPT_IN_DB runtime resolver * test(proxy): restore PEP8 spacing between spend tracking tests

…BerriAI#22106) * fix(proxy): honor MAX_STRING_LENGTH_PROMPT_IN_DB from config env vars * fix(proxy): reuse constants fallback for MAX_STRING_LENGTH_PROMPT_IN_DB runtime resolver * test(proxy): restore PEP8 spacing between spend tracking tests

vercel Bot deployed to Preview February 25, 2026 17:47 View deployment

greptile-apps Bot reviewed Feb 25, 2026

View reviewed changes

vercel Bot deployed to Preview February 25, 2026 19:08 View deployment

greptile-apps Bot reviewed Feb 25, 2026

View reviewed changes

gavksingh added 2 commits February 25, 2026 15:18

fix(proxy): honor MAX_STRING_LENGTH_PROMPT_IN_DB from config env vars

9e58260

fix(proxy): reuse constants fallback for MAX_STRING_LENGTH_PROMPT_IN_…

43ffbd7

…DB runtime resolver

gavksingh force-pushed the fix/issue-22088-max-string-env branch from 0ba75bf to 43ffbd7 Compare February 25, 2026 20:19

vercel Bot deployed to Preview February 25, 2026 20:26 View deployment

greptile-apps Bot reviewed Feb 25, 2026

View reviewed changes

test(proxy): restore PEP8 spacing between spend tracking tests

078be19

vercel Bot deployed to Preview February 25, 2026 23:56 View deployment

ghost changed the base branch from main to litellm_oss_staging_02_26_2026 February 26, 2026 08:06

ghost merged commit f3e31bc into BerriAI:litellm_oss_staging_02_26_2026 Feb 26, 2026
22 of 30 checks passed

This pull request was closed.

	default_max = 2048
	default_max = 2048 # keep in sync with litellm.constants.MAX_STRING_LENGTH_PROMPT_IN_DB

		), "max_retries should be None when not provided"


		def test_get_request_duration_ms_normal():

Uh oh!

Uh oh!

Conversation

gavksingh commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pre-Submission checklist

CI (LiteLLM team)

Type

Changes

Uh oh!

vercel Bot commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps Bot commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Flowchart

Uh oh!

greptile-apps Bot left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

ghost commented Feb 25, 2026

🚅 Shin's PR Review

Uh oh!

gavksingh commented Feb 25, 2026

Uh oh!

greptile-apps Bot left a comment

Choose a reason for hiding this comment

Uh oh!

gavksingh commented Feb 25, 2026

Uh oh!

greptile-apps Bot left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

gavksingh commented Feb 26, 2026

Uh oh!

gavksingh commented Feb 26, 2026

Uh oh!

gavksingh commented Feb 26, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

gavksingh commented Feb 25, 2026 •

edited

Loading

vercel Bot commented Feb 25, 2026 •

edited

Loading

greptile-apps Bot commented Feb 25, 2026 •

edited

Loading