fix(proxy): persist periodic reload schedule state so status survives restarts and fires without store_model_in_db

[ryan-crabbe-berri]

TLDR

Problem this solves:

• Reload status showed "Never ran" after every proxy restart
• Schedules never fired when store_model_in_db was false
• Manual "Reload Now" only refreshed one other pod per fleet

How it solves it:

• Persist last run and a reload revision as LiteLLM_Config columns
• Serve the status endpoint from the row, not process memory
• Run the reload check on its own ungated scheduler job
• Each pod reloads on a revision it has not applied

Relevant issues

Fixes #24930

Linear ticket

Resolves LIT-4882

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Reproduction before the fix is documented in #24930 and matches the base branch (b930e2f): schedule a reload from Models + Endpoints -> Price Data Reload, restart the proxy, and GET /schedule/model_cost_map_reload/status returns "last_run": null, "next_run": null so the card shows "Never"; with store_model_in_db: false the reload additionally never executes, because the check only ran from the store_model_in_db-gated add_deployment job

Everything below ran at commit d66cfab against a live proxy: fresh Postgres 16, general_settings.store_model_in_db: false, a config-file model list, migrations applied with prisma migrate deploy

The migration adds both columns, and scheduling writes only the admin-owned param_value

$ psql ... -c '\d "LiteLLM_Config"'
     Column      |              Type              | Nullable | Default
-----------------+--------------------------------+----------+---------
 param_name      | text                           | not null |
 param_value     | jsonb                          |          |
 last_run_at     | timestamp(3) without time zone |          |
 reload_revision | bigint                         | not null | 0

$ curl -s -X POST "http://localhost:4127/schedule/model_cost_map_reload?hours=6" -H "Authorization: Bearer sk-1234"
{"message":"Model cost map reload scheduled for every 6 hours","status":"success","interval_hours":6,"timestamp":"2026-08-04T05:55:30.259398+00:00"}

One scheduler tick later, with store_model_in_db false, the new periodic_reload_job fires and stamps the run

$ psql ... -c "select param_value, last_run_at, reload_revision from \"LiteLLM_Config\" where param_name='model_cost_map_reload_config'"
      param_value      |       last_run_at       | reload_revision
-----------------------+-------------------------+-----------------
 {"interval_hours": 6} | 2026-08-04 05:57:20.046 |               0

Two manual reloads each publish a distinct revision, so a second click is never swallowed by the first

$ curl -s -X POST "http://localhost:4127/reload/model_cost_map" -H "Authorization: Bearer sk-1234"
{"message":"Price data reloaded successfully! 2985 models updated.","status":"success","models_count":2985,...}
$ curl -s -X POST "http://localhost:4127/reload/model_cost_map" -H "Authorization: Bearer sk-1234"
{"message":"Price data reloaded successfully! 2985 models updated.","status":"success","models_count":2985,...}

$ psql ... -c "select last_run_at, reload_revision from ..."
       last_run_at       | reload_revision
-------------------------+-----------------
 2026-08-04 05:57:33.604 |               2

Cancelling clears the interval and keeps the row, so the counter carries on. With a delete the re-created row restarted at 1, which is a number pods had already applied, and their next Reload Now would have been skipped everywhere but the pod that served it

$ curl -s -X DELETE "http://localhost:4127/schedule/model_cost_map_reload" -H "Authorization: Bearer sk-1234"
{"message":"Model cost map reload schedule cancelled","status":"success","timestamp":"2026-08-04T06:00:46.005933+00:00"}

$ psql ... -c "select param_value, last_run_at, reload_revision from ..."
       param_value        |       last_run_at       | reload_revision
--------------------------+-------------------------+-----------------
 {"interval_hours": null} | 2026-08-04 06:00:45.886 |               1

$ curl -s "http://localhost:4127/schedule/model_cost_map_reload/status" -H "Authorization: Bearer sk-1234"
{"scheduled":false,"interval_hours":null,"last_run":"2026-08-04T06:00:45.886000+00:00","next_run":null}

$ curl -s -X POST "http://localhost:4127/reload/model_cost_map" -H "Authorization: Bearer sk-1234" -o /dev/null
$ psql ... -c "select last_run_at, reload_revision from ..."
       last_run_at       | reload_revision
-------------------------+-----------------
 2026-08-04 06:00:54.276 |               2

Re-schedule, kill the proxy, start it again and query status immediately; this is the exact customer flow that used to report "Never"

$ curl -s "http://localhost:4127/schedule/model_cost_map_reload/status" -H "Authorization: Bearer sk-1234"
{"scheduled":true,"interval_hours":6,"last_run":"2026-08-04T06:00:54.276000+00:00","next_run":"2026-08-04T12:00:54.276000+00:00"}

Type

🐛 Bug Fix

Changes

The interval an admin configures and the runtime state the reload job produces used to share one JSON blob in LiteLLM_Config, with the last-run time living only in a module-level global. This PR splits them by owner. The schedule endpoints own param_value ({"interval_hours": N}); the reload job and the manual reload endpoint own two new columns, last_run_at and reload_revision, written with field-scoped updates so no writer can clobber another's data. The migration is additive and schema-only, with IF NOT EXISTS guards, so prisma db push and prisma migrate deploy leave the database in the same state

The check itself moves off the store_model_in_db-gated add_deployment job onto its own periodic_reload_job, registered whenever a database is configured, so config-file deployments execute their schedules. The status endpoint now derives last_run and next_run purely from the row, which is what makes the UI survive restarts and multi-pod routing. The logic lives in litellm/proxy/common_utils/periodic_reload_schedule.py. Scope note: this PR only changes the model cost map path; the Anthropic beta headers reload keeps its previous implementation and can adopt the shared module in a follow-up

The old force_reload boolean is replaced by reload_revision, a counter the manual reload endpoint increments atomically in the database. Each pod records the revision it last applied and reloads whenever the row's differs, which identifies a request rather than ordering one: there is no clock comparison, so neither skew between pods nor the gap between Postgres TIMESTAMP(3) and Python's microsecond stamps can make a pod miss a reload or serve one twice. Nothing ever clears the counter, so a request reaches every pod exactly once, where the boolean was consumed by the first poller and starved the rest. Concurrent requests publish distinct revisions instead of overwriting each other. A booting pod starts unapplied rather than adopting whatever revision it finds, because it cannot prove that an outstanding request predates the prices it fetched at import; it serves that request on its first poll and is quiet from then on. The cost is one redundant fetch per pod boot, and only once someone has used Reload Now at least once, which is the cheaper side of the trade against a pod sitting on stale prices with no interval configured to rescue it. Interval reloads still key off the pod's own data age, where hour-scale comparisons make precision irrelevant

Cancelling a schedule clears param_value instead of deleting the row, because the row also carries the counter. Deleting it restarted the count, and a reissued revision matches what pods already applied, so the next Reload Now would reach only the pod that served it

A schedule whose row has never recorded a run fires on the next tick rather than one interval later. The job stamps last_run_at with update_many so a schedule cancelled mid-poll is not resurrected. The fetch itself goes through refetch_model_cost_map, so a failed reload keeps the pod's current pricing and leaves the revision unapplied, which is what makes the next poll retry it rather than record it as served

One visible output change worth noting: timestamps from these endpoints now carry an explicit +00:00 offset, which fixes the dashboard rendering them shifted by the viewer's UTC offset (the previous naive strings were parsed as local time). Response shapes are unchanged. A follow-up PR ports the dashboard card to react-query and stops it rendering fetch errors as "not scheduled"

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Medium Risk
Touches pricing reload paths and shared LiteLLM_Config schema; behavior changes for multi-pod fleets and deployments without store_model_in_db, though failures keep current pricing and retry.

Overview
Fixes proxy model cost map reload scheduling so last run and status survive restarts, interval jobs run without store_model_in_db, and manual reload reaches every pod.

Database: LiteLLM_Config gains last_run_at and reload_revision. Admins still own param_value (interval_hours); the reload job owns the new columns via scoped writes. Cancelling a schedule nulls the interval in JSON instead of deleting the row so the revision counter is never reset.

Coordination: Replaces the old force_reload flag and in-memory last_model_cost_map_reload with a monotonic reload_revision that manual reload bumps atomically. Each pod tracks model_cost_map_applied_revision and reloads when it lags the row; interval timing uses per-pod model_cost_map_loaded_at (from get_model_cost_map_loaded_at()). Failed fetches leave the revision unapplied for retry.

Runtime: New periodic_reload_schedule module and a dedicated periodic_reload_job (not gated on store_model_in_db). Status API reads the DB row; reload logic is centralized in _swap_in_model_cost_map.

^{Reviewed by Cursor Bugbot for commit d4df611. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

This PR persists model-cost-map reload state and coordinates fleet-wide reloads through a database-backed revision

• Adds last-run and revision columns consistently across the Prisma schemas and migration
• Moves periodic reload polling into a database-enabled scheduler job independent of store_model_in_db
• Replaces the shared force_reload flag with per-pod revision tracking and retry-safe state updates
• Serves schedule status from persisted state and retains revision history when schedules are cancelled
• Adds regression coverage for startup, failure, cancellation, scheduling, and multi-pod behavior

Confidence Score: 4/5

The PR is not yet safe to merge because database upgrades can discard a pending legacy fleet-wide reload request

Existing rows carrying force_reload are initialized with reload_revision 0, while the migration performs no conversion and the new parser intentionally ignores force_reload, so an unconsumed request can disappear during upgrade and leave other pods on stale pricing data

Files Needing Attention: litellm-proxy-extras/litellm_proxy_extras/migrations/20260729000000_add_reload_tracking_to_litellm_config/migration.sql

Important Files Changed

Filename	Overview
litellm/proxy/common_utils/periodic_reload_schedule.py	Introduces typed persistence helpers and revision-based due checks for periodic and fleet-wide model-cost-map reloads
litellm/proxy/proxy_server.py	Registers the independent reload job, applies revisions only after successful reload bookkeeping, and updates the management endpoints
litellm/litellm_core_utils/get_model_cost_map.py	Records the actual initial cost-map load time used to seed per-pod scheduling state
litellm-proxy-extras/litellm_proxy_extras/migrations/20260729000000_add_reload_tracking_to_litellm_config/migration.sql	Adds the persisted reload columns but leaves the previously reported legacy-request upgrade gap outstanding
schema.prisma	Adds nullable last-run state and a monotonic reload revision to LiteLLM_Config
tests/test_litellm/proxy/common_utils/test_periodic_reload_schedule.py	Covers parsing, status derivation, revision checks, cancellation, atomic increments, and field-scoped persistence
tests/test_litellm/proxy/test_proxy_server.py	Adds integration-level regression coverage for scheduler registration, retries, startup state, and fleet revision propagation

_{Reviews (15): Last reviewed commit: "fix(ui): match the CI-generated user_rol..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 84.61538% with 26 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/proxy_server.py	63.38%	26 Missing ⚠️

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_lit_4882_persist_reload_schedule (fd10f95) with litellm_internal_staging (e64536c)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (e4fd790) during the generation of this report, so c93a336 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[ryan-crabbe-berri]

On the db-push backfill note: acknowledged but intentionally left as-is. The backfill only matters for a manual reload clicked in the seconds before an upgrade and not yet consumed, the legacy boolean only ever reached one other pod anyway, and the remedy is clicking Reload Now again. Handling the legacy JSON flag in code would reintroduce the unclearable-flag loop this PR removes, so migrate-deploy paths get the carry-over and db-push paths degrade to exactly the old behavior's worst case

@greptileai

[greptile-apps]

Acknowledged — that's a reasonable trade-off. The backfill window is narrow (only a pending force_reload flag set in the seconds before upgrade), the legacy path was already lossy in multi-pod setups (first poller consumed the flag, others missed it), and avoiding the carry-over logic keeps the new reload_requested_at semantics clean. Degrading db-push to the old behavior's worst case rather than reintroducing the consumption-loop is the right call.

No further concerns on this point.

[ryan-crabbe-berri]

@greptileai re review

[ryan-crabbe-berri]

@greptileai re review

[ryan-crabbe-berri]

@greptileai re review

[ryan-crabbe-berri]

@greptileai

[ryan-crabbe-berri]

Resolved the db-push gap differently in 2887eb3: the legacy force_reload backfill is gone and the migration is now schema-only, so db push and migrate deploy leave the database in the same state and there is no data statement for one path to skip. Dropping it costs nothing in practice; that flag only meant "other pods, re-fetch your prices", and the rolling restart an upgrade performs already does that because every booting pod fetches the cost map at import

@greptileai

[ryan-crabbe-berri]

@greptileai

[ryan-crabbe-berri]

@greptileai

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 2 · PR risk: 0/10

[ryan-crabbe-berri]

Fixed the first-poll race in 9839b95: the pod now seeds model_cost_map_applied_revision from the row once at startup, right after its boot-time cost map fetch, instead of adopting whatever it finds on the first tick. A missing row seeds 0, so the very first Reload Now still reaches a pod that booted before it, and a database hiccup at boot falls back to the previous adopt-on-first-poll behavior rather than blocking startup

@greptileai

[ryan-crabbe-berri]

@greptileai

[ryan-crabbe-berri]

Closed the startup window in 19b6570 rather than narrowing it again. Seeding the applied revision was still a guess about ordering, so it is gone: a pod starts unapplied and serves any outstanding request on its first poll, then stays quiet. That costs one redundant cost map fetch per pod boot, and only after someone has used Reload Now at least once, which is cheaper than a pod holding stale prices indefinitely when no interval is configured. seed_model_cost_map_revision and its three tests are deleted and the per-pod field is a plain int again

Also merged litellm_internal_staging in. Worth flagging one interaction: invalidate_config_param now publishes on the config sync channel, so the reload job stamping last_run_at would have made every pod broadcast a config change and put the whole fleet through add_deployment plus get_credentials on every price reload. The module now calls evict_config_param, which is what this path did before and what test_model_cost_map_reload_does_not_publish_config_change pins

@greptileai

[ryan-crabbe-berri]

@greptileai re review

[ryan-crabbe-berri]

@greptileai re review

[ryan-crabbe-berri]

@greptileai

[ryan-crabbe-berri]

@greptileai

[devin-ai-integration]

Distributed QA: 3 pods, one shared Postgres, mutable cost map

QA only; no source changes. I ran this branch as a fleet to check that the original report in #24930 (a manual "Reload Now" refreshing a single pod, status showing nothing after a restart, and schedules never firing with store_model_in_db: false) cannot recur

Fleet setup

One Postgres 16 in docker on 5433, with the branch migration applied through prisma migrate deploy so LiteLLM_Config really has the two new columns

$ docker run -d --name qa-pg16 -e POSTGRES_PASSWORD=qa -e POSTGRES_USER=qa -e POSTGRES_DB=litellm_qa -p 5433:5432 postgres:16
$ cd litellm-proxy-extras/litellm_proxy_extras && DATABASE_URL="postgresql://qa:qa@127.0.0.1:5433/litellm_qa" python -m prisma migrate deploy --schema=./schema.prisma
  └─ 20260729000000_add_reload_tracking_to_litellm_config/
    └─ migration.sql
All migrations have been successfully applied.

$ docker exec qa-pg16 psql -U qa -d litellm_qa -c '\d "LiteLLM_Config"'
     Column      |              Type              | Collation | Nullable | Default
-----------------+--------------------------------+-----------+----------+---------
 param_name      | text                           |           | not null |
 param_value     | jsonb                          |           |          |
 last_run_at     | timestamp(3) without time zone |           |          |
 reload_revision | bigint                         |           | not null | 0

Three pods on 4001, 4002 and 4003, same DATABASE_URL, own log file each, master key sk-1234, and this config file, so the periodic job is exercised in exactly the mode where it used to be skipped

model_list:
  - model_name: qa-sentinel-model
    litellm_params:
      model: openai/qa-sentinel-model
      api_key: sk-fake-qa
      api_base: http://127.0.0.1:9/v1

general_settings:
  master_key: sk-1234
  store_model_in_db: false
  proxy_config_reload_interval_seconds: 10

To make each pod's reload observable on its own, the cost map source is a local mutable file rather than the GitHub raw URL. cost_map.json is the bundled model_prices_and_context_window.json plus a qa-sentinel-model entry (2988 entries total, so the shrink and minimum-count integrity checks in refetch_model_cost_map still pass), served by python3 -m http.server 4999, and every pod runs with LITELLM_MODEL_COST_MAP_URL=http://127.0.0.1:4999/cost_map.json. Changing the sentinel's input_cost_per_token in that file and then reading GET /model/info per pod says exactly which pods have refetched

$ ./probe.sh          # GET /model/info on each pod, printing the sentinel's input_cost_per_token
pod 4001: 1e-06
pod 4002: 1e-06
pod 4003: 1e-06

A. Manual reload fan-out

Sentinel changed from 1e-06 to 7.7e-06, then one POST /reload/model_cost_map against pod 4001 only

$ ./set_sentinel.sh 0.0000077 && ./probe.sh
sentinel input_cost_per_token = 7.7e-06
pod 4001: 1e-06
pod 4002: 1e-06
pod 4003: 1e-06

$ curl -s -X POST -H "Authorization: Bearer sk-1234" http://127.0.0.1:4001/reload/model_cost_map
{"message":"Price data reloaded successfully! 2987 models updated.","status":"success","models_count":2987,"timestamp":"2026-08-04T16:15:13.926273+00:00"}

$ ./probe.sh          # immediately after, only the serving pod has it
pod 4001: 7.7e-06
pod 4002: 1e-06
pod 4003: 1e-06

One poll interval later every pod serves the new price, and the row carries revision 1

$ ./probe.sh
pod 4001: 7.7e-06
pod 4002: 7.7e-06
pod 4003: 7.7e-06

$ docker exec qa-pg16 psql -U qa -d litellm_qa -c 'select param_name, param_value, last_run_at, reload_revision from "LiteLLM_Config";'
          param_name          | param_value |       last_run_at       | reload_revision
------------------------------+-------------+-------------------------+-----------------
 model_cost_map_reload_config |             | 2026-08-04 16:15:14.834 |               1

Per-pod logs show the fan-out landing exactly once on each pod that did not serve the request, and no pod repeating it over the following minute of ten-second polls

pod_4001.log: 16:15:13 proxy_server.py:15854 - Model cost map reloaded successfully in current pod. Models count: 2987
pod_4002.log: 16:15:14 periodic_reload_schedule.py:154 - Model cost map reload triggered by manual reload request
pod_4002.log: 16:15:14 proxy_server.py:6566 - Model cost map reloaded successfully. Models count: 2987
pod_4003.log: 16:15:14 periodic_reload_schedule.py:154 - Model cost map reload triggered by manual reload request
pod_4003.log: 16:15:14 proxy_server.py:6566 - Model cost map reloaded successfully. Models count: 2987

B. Scheduled reload with store_model_in_db: false

Scheduled through the API at the shortest granularity it accepts, which is one hour, and then left to fire for real rather than forced; no DB column was hand-edited for this scenario

$ curl -s -X POST -H "Authorization: Bearer sk-1234" "http://127.0.0.1:4002/schedule/model_cost_map_reload?hours=1"
{"message":"Model cost map reload scheduled for every 1 hours","status":"success","interval_hours":1,"timestamp":"2026-08-04T16:17:35.604869+00:00"}

The last reload on every pod was at 16:18:5x, so the interval was due at 17:18:5x. Sentinel changed to 4.56e-06 at 16:30 and left alone; the fleet still served the old price for the next 48 minutes, which is also the evidence that nothing reloads spuriously

$ date -u; ./probe.sh
Tue Aug  4 17:17:48 UTC 2026
pod 4001: 0.000123
pod 4002: 0.000123
pod 4003: 0.000123

$ date -u; ./probe.sh
Tue Aug  4 17:20:55 UTC 2026
pod 4001: 4.56e-06
pod 4002: 4.56e-06
pod 4003: 4.56e-06

$ docker exec qa-pg16 psql -U qa -d litellm_qa -c 'select param_name, param_value, last_run_at, reload_revision from "LiteLLM_Config";'
          param_name          |      param_value      |      last_run_at       | reload_revision
------------------------------+-----------------------+------------------------+-----------------
 model_cost_map_reload_config | {"interval_hours": 1} | 2026-08-04 17:18:55.98 |               3

All three pods ran their own periodic_reload_job at the tick, with store_model_in_db false the whole time

pod_4001.log: 17:18:54 periodic_reload_schedule.py:164 - Model cost map reload triggered by interval. Hours since data loaded: 1.00, Interval: 1
pod_4001.log: 17:18:54 proxy_server.py:6566 - Model cost map reloaded successfully. Models count: 2987
pod_4002.log: 17:18:54 periodic_reload_schedule.py:164 - Model cost map reload triggered by interval. Hours since data loaded: 1.00, Interval: 1
pod_4002.log: 17:18:54 proxy_server.py:6566 - Model cost map reloaded successfully. Models count: 2987
pod_4003.log: 17:18:55 periodic_reload_schedule.py:164 - Model cost map reload triggered by interval. Hours since data loaded: 1.00, Interval: 1
pod_4003.log: 17:18:55 proxy_server.py:6566 - Model cost map reloaded successfully. Models count: 2987

Note that next_run is the shared last_run_at plus the interval, while each pod decides for itself from when its own copy was loaded; the three pods here were within a second of each other, so the tick looks synchronized, and a pod that booted later would fire later. That matches what the module documents and it does not leave any pod stale

C. Restart survival

Pod 4003 killed and restarted, then asked for status as soon as it was up

$ curl -s -H "Authorization: Bearer sk-1234" "http://127.0.0.1:4003/schedule/model_cost_map_reload/status"
{"scheduled":true,"interval_hours":1,"last_run":"2026-08-04T16:18:15.988000+00:00","next_run":"2026-08-04T17:18:15.988000+00:00"}

Real timestamps out of the row, no null and no "Never ran". The 16:18:15 stamp is the restarted pod's own boot reload, which is the documented one redundant fetch per boot; it starts at revision 0 and so serves the published revision 1 once

pod_4003.log: 16:18:15 periodic_reload_schedule.py:154 - Model cost map reload triggered by manual reload request
pod_4003.log: 16:18:16 proxy_server.py:6566 - Model cost map reloaded successfully. Models count: 2987

D. Concurrency and idempotence

Sentinel changed to 0.000123, then two manual reloads fired at the same instant against two different pods

$ (curl -s -X POST -H "Authorization: Bearer sk-1234" http://127.0.0.1:4001/reload/model_cost_map &
   curl -s -X POST -H "Authorization: Bearer sk-1234" http://127.0.0.1:4002/reload/model_cost_map & wait)
{"message":"Price data reloaded successfully! 2987 models updated.","status":"success","models_count":2987,"timestamp":"2026-08-04T16:18:54.820336+00:00"}
{"message":"Price data reloaded successfully! 2987 models updated.","status":"success","models_count":2987,"timestamp":"2026-08-04T16:18:54.821767+00:00"}

$ docker exec qa-pg16 psql -U qa -d litellm_qa -c 'select param_name, last_run_at, reload_revision from "LiteLLM_Config";'
          param_name          |       last_run_at       | reload_revision
------------------------------+-------------------------+-----------------
 model_cost_map_reload_config | 2026-08-04 16:18:54.833 |               3

The counter went from 1 to 3, so both requests published a distinct revision instead of overwriting each other. Pod 4001 published and adopted 3, pod 4002 published 2 and then applied 3 on its next poll, and pod 4003 collapsed the two into one fetch because it only ever observed 3

pod_4001.log: 16:18:54 proxy_server.py:15854 - Model cost map reloaded successfully in current pod. Models count: 2987
pod_4002.log: 16:18:54 proxy_server.py:15854 - Model cost map reloaded successfully in current pod. Models count: 2987
pod_4002.log: 16:18:54 periodic_reload_schedule.py:154 - Model cost map reload triggered by manual reload request
pod_4002.log: 16:18:54 proxy_server.py:6566 - Model cost map reloaded successfully. Models count: 2987
pod_4003.log: 16:18:55 periodic_reload_schedule.py:154 - Model cost map reload triggered by manual reload request
pod_4003.log: 16:18:55 proxy_server.py:6566 - Model cost map reloaded successfully. Models count: 2987

Nothing was swallowed and nothing was served twice; every pod ended on 0.000123 and then sat still for the next 60 minutes of polling, with the reload counts per log staying at 2, 3 and 2 until the scheduled tick in B

$ ./probe.sh
pod 4001: 0.000123
pod 4002: 0.000123
pod 4003: 0.000123

E. Same fleet on the base commit, reproducing #24930

Second fleet from a worktree at 368dd0be5b (litellm_internal_staging) on ports 5001/5002/5003, its own database on the same server, store_model_in_db: true so the pre-fix polling path exists at all. Sentinel changed from 0.000123 to 0.00099, then one manual reload against pod 5001

$ curl -s -X POST -H "Authorization: Bearer sk-1234" http://127.0.0.1:5001/reload/model_cost_map
{"message":"Price data reloaded successfully! 2987 models updated.","status":"success","models_count":2987,"timestamp":"2026-08-04T16:22:00.184252"}

$ date -u; ./probe_base.sh
Tue Aug  4 16:23:35 UTC 2026
base pod 5001: 0.00099
base pod 5002: 0.000123
base pod 5003: 0.000123

Ninety seconds and nine poll cycles later the other two pods are still on the old price, permanently, and the boolean was already consumed by the pod that served the request

$ docker exec qa-pg16 psql -U qa -d litellm_qa_base -c "select param_name, param_value from \"LiteLLM_Config\" where param_name = 'model_cost_map_reload_config';"
          param_name          |                   param_value
------------------------------+-------------------------------------------------
 model_cost_map_reload_config | {"force_reload": false, "interval_hours": null}

basepod_5001.log: 16:22:00 proxy_server.py:15893 - Model cost map reloaded successfully in current pod. Models count: 2987
basepod_5001.log: 16:22:03 proxy_server.py:6514 - Model cost map reload triggered by force reload flag
basepod_5001.log: 16:22:03 proxy_server.py:6589 - Model cost map reloaded successfully. Models count: 2987
basepod_5002.log: (no reload lines)
basepod_5003.log: (no reload lines)

Same base commit with store_model_in_db: false on ports 5011/5012/5013 is worse, which is the other half of the report. An hourly schedule plus a manual reload on pod 5011 leaves the flag set forever because no pod ever polls it, the two other pods never move off the old price, and their status card has nothing to show

$ curl -s -X POST -H "Authorization: Bearer sk-1234" "http://127.0.0.1:5011/schedule/model_cost_map_reload?hours=1"
$ ./set_sentinel.sh 0.00888
$ curl -s -X POST -H "Authorization: Bearer sk-1234" http://127.0.0.1:5011/reload/model_cost_map

$ date -u; ./probe_nodb.sh
Tue Aug  4 16:29:51 UTC 2026
base-nodb pod 5011: 0.00888
base-nodb pod 5012: 0.00099
base-nodb pod 5013: 0.00099

$ docker exec qa-pg16 psql -U qa -d litellm_qa_base -c "select param_name, param_value from \"LiteLLM_Config\" where param_name = 'model_cost_map_reload_config';"
          param_name          |                 param_value
------------------------------+---------------------------------------------
 model_cost_map_reload_config | {"force_reload": true, "interval_hours": 1}

$ for p in 5011 5012 5013; do curl -s -H "Authorization: Bearer sk-1234" "http://127.0.0.1:$p/schedule/model_cost_map_reload/status"; echo; done
{"scheduled":true,"interval_hours":1,"last_run":"2026-08-04T16:28:04.035114","next_run":"2026-08-04T17:28:04.035114"}
{"scheduled":true,"interval_hours":1,"last_run":null,"next_run":null}
{"scheduled":true,"interval_hours":1,"last_run":null,"next_run":null}

Verdict

A pass, B pass, C pass, D pass, E reproduces #24930 on the base commit and is fixed on this branch. Manual "Reload Now" and the hourly schedule both reach every pod exactly once, the status endpoint survives a restart because it reads the row, concurrent reloads publish distinct revisions with no pod starved or double-served, and the single-pod-only behaviour does not recur

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit d4df611. Configure here.}

[cursor]

Failed stamp causes reload loop

Medium Severity

When an interval schedule has never persisted last_run_at, pod_reload_is_due keeps returning true on every tick without looking at pod_data_loaded_at. If _check_and_reload_model_cost_map fetches and swaps pricing but record_reload_run errors, the row stays unstamped and the pod refetches on each periodic_reload_job interval.

Additional Locations (1)

• litellm/proxy/proxy_server.py#L6557-L6560

 

^{Reviewed by Cursor Bugbot for commit d4df611. Configure here.}

[mateo-berri]

I think worst case of refetching is no big deal. Brief DB outages or failovers are not frequent anyways

[mateo-berri]

LGTM. Thanks!

[mateo-berri]

Good to have explicit checking (input is object -> output is verified value) instead of force casting. Love it