fix(proxy): return persisted DB value for general_settings in /config/list

[devin-ai-integration]

Relevant issues

The "Store Prompts in Spend Logs" toggle (Admin Settings -> Logging Settings) turns on, saves with a green success popup, then flips back to off. Reported to happen after clicking around the UI and running a few LLM requests

Linear ticket

Resolves LIT-4204

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review

Root cause

This is a backend bug, separate from the UI-only load-display issue in #32145

GET /config/list reads the fresh row from the DB into db_general_settings_dict, but when resolving each field's value it preferred the per-process in-memory general_settings global and only fell back to the DB when the in-memory value was None

_field_value = general_settings.get(field_name, None)
if _field_value is None and field_name in db_general_settings_dict:
    _field_value = db_general_settings_dict[field_name]

For a boolean, a stale False is not None, so the DB fallback never fires. The in-memory general_settings global is only refreshed by the background add_deployment job (30s interval, see the add_deployment_job scheduler), and /config/update writes the DB without updating the globals of other workers. So in a multi-worker / multi-pod deployment, right after saving true the DB holds true while a sibling worker's global still holds a stale false. When the UI's post-save refetch lands on that worker, /config/list returns false and the toggle snaps back off. "Click around and run a few requests" is just what gives the background job time to seed that stale false into the other workers

The fix makes the persisted DB value authoritative for /config/list, falling back to the in-memory global only for settings that live purely in config.yaml / env and were never written to the DB

- _field_value = general_settings.get(field_name, None)
- if _field_value is None and field_name in db_general_settings_dict:
-     _field_value = db_general_settings_dict[field_name]
+ if field_name in db_general_settings_dict:
+     _field_value = db_general_settings_dict[field_name]
+ else:
+     _field_value = general_settings.get(field_name, None)

Screenshots / Proof of Fix

Reproduced live on a local proxy running 2 workers against Postgres, hitting the real /config/update and /config/list endpoints (no mocks)

Before the fix, save OFF, wait for the background job to seed both workers, save ON, then poll /config/list. The DB is true but the stale worker still answers false

$ curl -s -X POST $P/config/update -H "$H" -d '{"general_settings":{"store_prompts_in_spend_logs":false}}'
$ sleep 35   # let the 30s add_deployment job seed both workers' in-memory globals with False
$ curl -s -X POST $P/config/update -H "$H" -d '{"general_settings":{"store_prompts_in_spend_logs":true}}'
# DB value -> true
# 40x GET /config/list store_prompts_in_spend_logs:
#      1 False      <-- stale worker; UI refetch landing here shows the toggle OFF
#     39 True

After the fix, same sequence, DB true, 60/60 responses agree

# DB value -> true
# 60x GET /config/list store_prompts_in_spend_logs:
#     60 True

Regression test test_get_config_list_prefers_db_over_stale_in_memory_general_settings pins a fresh DB row of True against a stale in-memory general_settings of False and asserts /config/list returns True. It fails on the old code and passes with the fix

Type

🐛 Bug Fix

Changes

litellm/proxy/proxy_server.py: in get_config_list, prefer the DB-persisted value over the in-memory general_settings global when a field is present in the DB row

tests/test_litellm/proxy/test_proxy_server.py: add the regression test described above

Link to Devin session: https://app.devin.ai/sessions/2edea14c59e54828bbe342d5983ef34e
Requested by: @krrish-berri-2

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This PR fixes a multi-worker staleness bug in GET /config/list: for scalar general settings fields (Boolean, Integer, String, List), the handler now prefers the persisted DB value over the per-process in-memory general_settings global, falling back to the in-memory value only when the field is absent from the DB.

• litellm/proxy/proxy_server.py: In get_config_list, the value-resolution order for scalar fields in the else branch is swapped so the DB is authoritative; the PydanticModel branch (currently covering pass_through_endpoints) was not updated and still reads from the in-memory global.
• tests/test_litellm/proxy/test_proxy_server.py: Adds a regression test that patches a stale False into the in-memory global and a fresh True into the mocked DB row, then asserts /config/list returns True.

Confidence Score: 3/5

The scalar-field fix is correct and well-tested, but the same stale-global read for PydanticModel-type fields (currently pass_through_endpoints) was left behind, leaving a partial fix.

The PydanticModel branch on line 14613 still uses general_settings.get(field_name, None) for field_value instead of preferring db_general_settings_dict, meaning pass_through_endpoints remains vulnerable to the exact multi-worker staleness that motivated this change. The regression test only covers the Boolean path and would not catch this gap.

litellm/proxy/proxy_server.py — specifically the PydanticModel branch in get_config_list around line 14613.

Important Files Changed

Filename	Overview
litellm/proxy/proxy_server.py	DB-preference logic correctly applied to scalar fields in get_config_list, but the parallel PydanticModel branch still reads field_value from the stale in-memory general_settings global rather than from db_general_settings_dict.
tests/test_litellm/proxy/test_proxy_server.py	New regression test correctly mocks the DB and the in-memory global with conflicting values and asserts the DB value wins; uses only mock objects, no real network calls.

Comments Outside Diff (1)

1. litellm/proxy/proxy_server.py, line 14609-14621 (link)

   Same staleness bug remains in the PydanticModel branch

   The fix was only applied to the else branch (scalar types like Boolean/Integer/String). The PydanticModel branch at line 14613 still reads field_value from the in-memory general_settings global instead of preferring db_general_settings_dict. Currently pass_through_endpoints is the only PydanticModel field in allowed_args, so if it is saved via /config/update, a stale worker will return the old in-memory value for pass_through_endpoints from /config/list — the same multi-worker race that this PR fixes for booleans.

_{Reviews (1): Last reviewed commit: "fix(proxy): return persisted DB value fo..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!