Repository navigation
fix(proxy): look up hashed key names with two spend log rows per key - #43656
Conversation
The spend-log fallback for keys missing from the key table read every row per key to check that all named rows agreed, which passed the 5s statement timeout on busy keys even with the (api_key, startTime) index. Probe only the oldest and newest named row per key, so the lookup stays two index reads per key however much the key logged.
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| ORDER BY "startTime" {direction} | ||
| LIMIT 1 |
There was a problem hiding this comment.
If a key’s owner, team, or alias changes and then changes back within the window, the oldest and newest named spend logs agree while a row between them does not. The two LIMIT 1 lookups miss that conflict, so recovery can report a misleading owner or label in daily-activity data. The previous all-row comparison withheld fields with conflicting values. Logs tied at an endpoint timestamp can also make the selected value arbitrary.
There was a problem hiding this comment.
Deliberate, see the Low caveat: the all-row check is what timed out, and a mid-window flip back still names one of that key's real owners
There was a problem hiding this comment.
That’s fair. The behavior is deliberate and explicitly documented in the Low caveat: the endpoint now treats agreement between the oldest and newest named rows as sufficient, and a flip-back still resolves to a real owner/alias rather than inventing metadata. Given the timeout/performance goal and the accepted semantics, I’m withdrawing this concern.
|
bugbot run |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
| FROM unnest($1::text[]) AS keys(digest) | ||
| CROSS JOIN LATERAL ({_named_spend_log_edge_row_sql("ASC")}) first_row | ||
| CROSS JOIN LATERAL ({_named_spend_log_edge_row_sql("DESC")}) last_row |
There was a problem hiding this comment.
When both edge rows have aliases but no owner, this lookup ignores an owner recorded between them. Daily activity then returns a null user ID and email.
There was a problem hiding this comment.
Covering interior rows needs a probe per field, and a field missing on every row reads them all again, the timeout this PR removes
There was a problem hiding this comment.
That’s fair. Recovering interior values independently would require additional per-field probes, and a field absent from every row would still force a full scan—reintroducing the timeout this change is intended to eliminate. Given the documented trade-off of using only the oldest/newest named rows and the flat-cost requirement, I’m withdrawing the finding; no change is needed for this PR.
|
bugbot run |
The newest-row probe now starts at the row where the oldest-row probe gave up, so a key with under 200 rows in the window is read once instead of twice, and the lookup transaction turns bitmap scans off so the planner walks the (api_key, startTime) index instead of every row of a busy key when statistics or the visibility map are stale.
|
bugbot run |
|
bugbot run |
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 54da4e5. Configure here.
…erriAI#43656) Backport of BerriAI#43656 to stable/1.103.x. Cherry-picked from 61a73c5 (main).
![renovate[bot]](https://github.githubassets.com/images/gravatars/gravatar-user-420.png){kind=avatar}
…03.2) (#328) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/berriai/litellm](https://images.chainguard.dev/directory/image/wolfi-base/overview) ([source](https://github.com/BerriAI/litellm)) | patch | `v1.103.1` → `v1.103.2` | --- ### Release Notes <details> <summary>BerriAI/litellm (ghcr.io/berriai/litellm)</summary> ### [`v1.103.2`](https://github.com/BerriAI/litellm/releases/tag/v1.103.2) [Compare Source](BerriAI/litellm@v1.103.1...v1.103.2) #### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](BerriAI/litellm@0112e53). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.103.2 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.103.2/cosign.pub \ ghcr.io/berriai/litellm:v1.103.2 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** #### What's Changed - chore(release): sync stable/1.103.x to v1.103.1 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​43824](BerriAI/litellm#43824) - fix(proxy): backport [#​40541](BerriAI/litellm#40541), [#​43642](BerriAI/litellm#43642), and [#​43656](BerriAI/litellm#43656) to stable/1.103.x for v1.103.2 by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43897](BerriAI/litellm#43897) - fix(anthropic): backport [#​42152](BerriAI/litellm#42152) and [#​42288](BerriAI/litellm#42288) to stable/1.103.x by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43662](BerriAI/litellm#43662) - fix(proxy): backport [#​43962](BerriAI/litellm#43962) to stable/1.103.x by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​43984](BerriAI/litellm#43984) **Full Changelog**: <BerriAI/litellm@v1.103.1...v1.103.2> </details> --- ### Configuration 📅 **Schedule**: (in timezone America/New_York) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xMTUuMTMiLCJ1cGRhdGVkSW5WZXIiOiI0NC4xMTUuMTMiLCJ0YXJnZXRCcmFuY2giOiJtYWluIiwibGFiZWxzIjpbInJlbm92YXRlL2NvbnRhaW5lciIsInR5cGUvcGF0Y2giXX0=--> Reviewed-on: https://git.greyrock.io/todd/home-ops/pulls/328
1 participant
TLDR
Problem this solves:
How it solves it:
(api_key, "startTime")index instead of reading every row of a busy key when the table's statistics or visibility map are staleUser Flow
Before: an admin's daily BI export gets every CLI session key back with no name, only an owner, so that spend can't be tied to the session it came from
breakdown.api_keysshowskey_aliasas null, withuser_idanduser_emailfilled in from its daily spend rowsThe Admin UI Usage page, Key Activity tab, at /ui/?page=new_usage on the same data as the After screenshot, labels the same keys by their owner's email since the alias is missing, and 4 of them
key-hash-<digest>(boxed in red)After: the same export comes back fast with every CLI session key named
key_alias,user_id, anduser_email, e.g.cli-session-cli.user13@example.comandcli.user13@example.comThe same Key Activity tab now shows every key by its alias (boxed in red)
Linear ticket
Resolves LIT-8924
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Last updated: 54da4e5. QA, /live-pr-risk, and /caveats ran on 59d6c46, the last commit with a product diff; the commit since adds the /audit cells below and touches no product code
Audit round 1 of 3
Merge base e7460f1 and head 59d6c46, each booted as a 2-worker proxy on its own database, ran the same 72 cells from
tests/integration/spend/(run ids base8ca7f467a4654f5197c704d3f8fcdb45, head run 12a9b0e74d6c74d018fd50fa9c0f308e8, head run 2f5f8b671c29f4289a5e11b9515855737, seeds 4106601/4106601). Base passed 70, head passed 72 then 72 with the same collected and passed selections, 0 skips and no retries. The 2 cells red on base are the intended behavior changes: 100 nameless + named + 100 nameless (middle-only name), shipped deliberately: the second Low caveat in this PR's Caveats section, called deliberate by mateo-berri in #43656 (comment), where Greptile withdrew its finding; A + 100 nameless + B + 100 nameless + A (rename and back), shipped deliberately: the first Low caveat in this PR's Caveats section, called deliberate by mateo-berri in #43656 (comment), where Greptile withdrew its finding. Every inventory row is verified by a cell below; the latency differential is evidenced by the clone-data legs above rather than by a cell, since a deterministic cell reproducing it would seed hundreds of thousands of rows against a 5 s statement timeout. The rig is the CircleCI shape: owned Postgres and Redis per leg, the scripted upstream, a 2-worker proxy in production mode with a real license, nothing mocked inside the proxy. Runs 1 to 3 of this round failed on the cells' own fixtures, on a convergence predicate, and on the rig's 60-connection Postgres under three concurrent legs, and none of them touched the productMatrix, one row per cell
tests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_named_only_by_a_spend_log_is_reported_on_every_daily_activity_route[user_daily_activity]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_named_only_by_a_spend_log_is_reported_on_every_daily_activity_route[user_daily_activity_aggregated]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_named_only_by_a_spend_log_is_reported_on_every_daily_activity_route[team_daily_activity]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_named_only_by_a_spend_log_is_reported_on_every_daily_activity_route[team_daily_activity_aggregated]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_named_only_by_a_spend_log_is_reported_on_every_daily_activity_route[tag_daily_activity]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_named_only_by_a_spend_log_is_reported_on_every_daily_activity_route[organization_daily_activity]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_named_only_by_a_spend_log_is_reported_on_every_daily_activity_route[customer_daily_activity]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_named_only_by_a_spend_log_is_reported_on_every_daily_activity_route[end_user_daily_activity]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_named_only_by_a_spend_log_is_reported_on_every_daily_activity_route[agent_daily_activity]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_on_an_edge_of_the_window_is_reported_whatever_surrounds_it[named_between_50_and_50_nameless]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_on_an_edge_of_the_window_is_reported_whatever_surrounds_it[oldest_named_150_nameless_newer]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_on_an_edge_of_the_window_is_reported_whatever_surrounds_it[newest_named_150_nameless_older]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_on_an_edge_of_the_window_is_reported_whatever_surrounds_it[both_edges_named_150_nameless_between]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_on_an_edge_of_the_window_is_reported_whatever_surrounds_it[100_nameless_named_99_nameless]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_on_an_edge_of_the_window_is_reported_whatever_surrounds_it[99_nameless_named_100_nameless]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_team_named_only_by_a_spend_log_is_reported_next_to_the_daily_owner[team_id_column]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_team_named_only_by_a_spend_log_is_reported_next_to_the_daily_owner[team_id_in_metadata]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_user_named_by_a_spend_log_beats_the_owner_the_daily_rows_name[user_column]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_user_named_by_a_spend_log_beats_the_owner_the_daily_rows_name[user_id_in_metadata]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_hashed_jwt_digest_is_named_by_its_spend_logtests/integration/spend/test_daily_activity_key_owner_traffic.py::test_key_purged_from_the_key_tables_is_reported_with_the_alias_its_spend_logs_nametests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_named_only_in_the_middle_of_two_hundred_nameless_rows_is_not_picked_uptests/integration/spend/test_daily_activity_key_alias_probes.py::test_key_renamed_and_renamed_back_is_reported_with_the_alias_on_both_edgestests/integration/spend/test_daily_activity_key_alias_probes.py::test_spend_log_names_the_key_only_from_one_day_before_to_two_days_after_the_read[second_before_the_window]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_spend_log_names_the_key_only_from_one_day_before_to_two_days_after_the_read[first_second_after_the_window]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_spend_log_names_the_key_only_from_one_day_before_to_two_days_after_the_read[first_second_of_the_window]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_spend_log_names_the_key_only_from_one_day_before_to_two_days_after_the_read[last_second_of_the_window]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_two_aliases_on_the_two_edges_leave_the_key_unnamedtests/integration/spend/test_daily_activity_key_alias_probes.py::test_rows_without_a_usable_alias_do_not_hide_the_named_row_after_them[empty_string_alias]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_rows_without_a_usable_alias_do_not_hide_the_named_row_after_them[array_then_string_metadata]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_found_once_is_served_from_the_cache_for_the_same_window_onlytests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_logged_after_a_cached_miss_shows_once_the_miss_expirestests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_of_an_unexpected_shape_is_reported_as_postgres_renders_it[json_int]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_of_an_unexpected_shape_is_reported_as_postgres_renders_it[json_list]tests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_of_an_unexpected_shape_is_reported_as_postgres_renders_it[five_kb_string]tests/integration/spend/test_daily_activity_key_owner.py::test_key_missing_from_the_key_tables_is_reported_with_the_one_user_its_daily_spend_names[user_daily_activity]tests/integration/spend/test_daily_activity_key_owner.py::test_key_missing_from_the_key_tables_is_reported_with_the_one_user_its_daily_spend_names[user_daily_activity_aggregated]tests/integration/spend/test_daily_activity_key_owner.py::test_key_missing_from_the_key_tables_is_reported_with_the_one_user_its_daily_spend_names[team_daily_activity]tests/integration/spend/test_daily_activity_key_owner.py::test_key_missing_from_the_key_tables_is_reported_with_the_one_user_its_daily_spend_names[team_daily_activity_aggregated]tests/integration/spend/test_daily_activity_key_owner.py::test_key_missing_from_the_key_tables_is_reported_with_the_one_user_its_daily_spend_names[tag_daily_activity]tests/integration/spend/test_daily_activity_key_owner.py::test_key_missing_from_the_key_tables_is_reported_with_the_one_user_its_daily_spend_names[organization_daily_activity]tests/integration/spend/test_daily_activity_key_owner.py::test_key_missing_from_the_key_tables_is_reported_with_the_one_user_its_daily_spend_names[customer_daily_activity]tests/integration/spend/test_daily_activity_key_owner.py::test_key_missing_from_the_key_tables_is_reported_with_the_one_user_its_daily_spend_names[end_user_daily_activity]tests/integration/spend/test_daily_activity_key_owner.py::test_key_missing_from_the_key_tables_is_reported_with_the_one_user_its_daily_spend_names[agent_daily_activity]tests/integration/spend/test_daily_activity_key_owner.py::test_key_whose_daily_spend_names_two_users_is_reported_with_no_ownertests/integration/spend/test_daily_activity_key_owner.py::test_daily_spend_rows_naming_no_user_do_not_hide_the_one_user_the_others_name[blank_user]tests/integration/spend/test_daily_activity_key_owner.py::test_daily_spend_rows_naming_no_user_do_not_hide_the_one_user_the_others_name[null_user]tests/integration/spend/test_daily_activity_key_owner.py::test_key_whose_daily_spend_names_no_user_at_all_is_reported_with_no_ownertests/integration/spend/test_daily_activity_key_owner.py::test_owner_the_user_table_does_not_hold_is_reported_by_id_with_no_emailtests/integration/spend/test_daily_activity_key_owner.py::test_live_key_keeps_its_own_user_when_its_daily_spend_names_anothertests/integration/spend/test_daily_activity_key_owner.py::test_live_key_with_no_user_is_not_given_the_user_its_daily_spend_namestests/integration/spend/test_daily_activity_key_owner.py::test_deleted_key_keeps_its_own_user_when_its_daily_spend_names_anothertests/integration/spend/test_daily_activity_key_owner.py::test_deleted_key_with_no_user_keeps_its_alias_and_gains_the_one_user_its_daily_spend_namestests/integration/spend/test_daily_activity_key_owner.py::test_key_named_only_by_a_spend_log_alias_keeps_that_alias_and_gains_the_one_user_its_daily_spend_namestests/integration/spend/test_daily_activity_key_alias_probes.py::test_alias_lookup_gives_up_while_spend_logs_are_locked_and_answers_once_they_are_nottests/integration/spend/test_daily_activity_key_alias_probes.py::test_concurrent_reads_over_every_route_all_name_a_fresh_keytests/integration/spend/test_daily_activity_key_owner_faults.py::test_user_reading_a_key_shared_with_another_user_is_shown_no_owner_and_nothing_of_the_other_usertests/integration/spend/test_daily_activity_key_owner_faults.py::test_user_reading_a_key_only_they_spent_with_is_shown_themselves_as_its_ownertests/integration/spend/test_daily_activity_key_owner_faults.py::test_user_reading_a_key_only_another_user_spent_with_is_shown_nothing_of_ittests/integration/spend/test_daily_activity_key_owner_faults.py::test_invalid_key_is_refused_without_naming_the_ownertests/integration/spend/test_daily_activity_key_owner_faults.py::test_five_kilobyte_key_is_reported_with_the_one_user_its_daily_spend_namestests/integration/spend/test_daily_activity_key_owner_faults.py::test_key_with_no_daily_spend_is_reported_as_no_activitytests/integration/spend/test_daily_activity_key_owner_faults.py::test_every_key_of_a_team_is_reported_with_its_own_usertests/integration/spend/test_daily_activity_key_owner_faults.py::test_reading_the_same_activity_twice_gives_the_same_answertests/integration/spend/test_daily_activity_key_owner_faults.py::test_key_stops_being_reported_with_an_owner_once_a_second_user_spends_with_ittests/integration/spend/test_daily_activity_key_owner_faults.py::test_owner_lookup_gives_up_while_daily_user_spend_is_locked_and_answers_once_it_is_nottests/integration/spend/test_daily_activity_key_owner_faults.py::test_owner_is_reported_while_a_worker_is_killed_and_after_it_is_replacedtests/integration/spend/test_daily_activity_key_owner_faults.py::test_owner_is_reported_again_after_the_proxy_restartstests/integration/spend/test_daily_activity_key_owner_traffic.py::test_key_used_on_every_unified_endpoint_is_reported_with_its_own_alias_and_usertests/integration/spend/test_daily_activity_key_owner_traffic.py::test_cli_session_spend_is_reported_with_the_user_and_team_of_the_sessiontests/integration/spend/test_daily_activity_key_owner_traffic.py::test_usage_ai_chat_hands_the_model_the_usage_summary_without_any_key_ownertests/integration/spend/test_daily_activity_key_owner_traffic.py::test_owner_is_reported_on_every_route_while_a_burst_of_requests_waits_on_the_providerShared setup: a local proxy on PostgreSQL 18.6 with every migration applied at boot, including the
(api_key, "startTime")index, and onegpt-5.4-nanodeployment in the config. A key with aliascli-session-templatemade three realgpt-5.4-nanocalls, and the seed below clones its spend log row into 186 hashed CLI session keys that are no longer in the key table (23 keys with 4,500 rows each and 163 with 300 each), plus 250k rows from other keys. Each row carries about 3 KB of metadata like real spend logs, 402,403 rows in total. The endpoint reads spend that is already logged, so the proof itself makes no LLM callseed.sql, run as
psql -v key_from=1 -v hashed_keys=23 -v rows_per_key=4500 -v background_rows=250000 -f seed.sql, then again withkey_from=24 hashed_keys=186 rows_per_key=300 background_rows=0(the rig itself was seeded in three smaller passes with the same totals)The count line in each run comes from this jq filter over the response
Both runs use the same topology: two proxy processes, instance A and instance B, on two ports with two workers each and one shared Postgres. There is no Redis in the rig, since the lookup keeps its cache inside each worker. The same request goes to A, then B, then to each of them again. The Before run is a worktree at the merge base e7460f1 and the After run is one at the tip 59d6c46
Before (e7460f1)
Instance A on port 35782:
curl -sS -o before_a1.json -w 'HTTP %{http_code} in %{time_total}s\n' -H "Authorization: Bearer $LITELLM_MASTER_KEY" "http://localhost:35782/user/daily/activity/aggregated?start_date=2026-09-24&end_date=2026-09-24", thenjq -r -f count_nulls.jq before_a1.jsonInstance B on port 50376, the same curl and jq
Instance A again
Instance B again
On this merge base the owner comes from the key's daily spend rows through #43642, which is why
user_emailis no longer null there, whilekey_aliasstill isAfter (59d6c46)
Instance A on port 52726, the same curl and jq
Instance B on port 37802
Instance A again
Instance B again
A sample key entry from instance B's last response
Spend per key is the same in every response of both runs, only the names differ
Admin UI, Key Activity tab
The before and after screenshots are under User Flow above. Both come from instance A of the same two-instance rig, signed in as the proxy admin, on the page's default date range of 2026-09-22 to 2026-09-29
Before (e7460f1, instance A on port 40046), the page says "Showing 191 of 191 keys", the keys are labeled by their owner's email since the alias is missing, 4 of them are labeled
key-hash-<digest>, and the request behind the page took 5.4 seconds. After (59d6c46, instance A on port 32321), the page says "Showing 191 of 191 keys", the keys are labeled by their aliascli-session-<email>, none of them is labeledkey-hash-<digest>, and that request took 0.9 seconds (rows)Type
🐛 Bug Fix
Caveats (if any)
Low
test_recover_key_metadata_from_spend_logs_walks_a_bounded_number_of_nameless_rows_per_keyassertsSET LOCAL enable_bitmapscan = offbefore its querySET LOCALlasts for the transaction alone, like theSET LOCAL statement_timeoutline that shipped before this PR, so it works under PgBouncer transaction pooling the same wayEXISTS ... OFFSET 100without anORDER BYis not bound to the window's oldest rows, and skipping the newest-row lookup when the oldest one finds nothing misses a key named only at its newest end(api_key, "startTime")index from migration20260823000000_add_spend_logs_api_key_starttime_indexwalks the"startTime"index with anapi_keyfilter insteadmisc / Run testsjob is red on 5 tests, 4 intests/unit/interactions/test_openapi_compliance.pyandtests/unit/test_utils.py::test_aaamodel_prices_and_context_window_json_is_validtest_aaparallel_function_call[gpt-3.5-turbo-1106], whose retired model test(ci): refresh retired OpenAI tool-call models #43676 replaces (rows)Final Attestation
/live-pr-risk passes at 59d6c46. It ran on two proxy instances sharing one Postgres database, 2 workers each, with requests alternating between them and no mocks. All 144 requests returned 200 on the merge base, the tip, and the tip merged into main, and every instance and round returned the same body for the same side. The only values that differ between the sides are key names and owners (rows)
Breaking
GET /user/daily/activity/aggregated81da1932dd7ccame back withuser_id"u-cli-02" anduser_email"cli.user02@example.com" before, and after withuser_id"u-risk", the owner its daily spend rows carry, which fix(proxy): recover session key owners from daily spend for usage attribution #43642 on main falls back to, anduser_emailnull (rows)Backward incompatible
47fc8df03190came back withkey_aliasnull before and "alias-a" after (rows)test_recover_key_metadata_from_spend_logs_walks_a_bounded_number_of_nameless_rows_per_keycovers the bound on rows read, and the null result is read from the SQLRegression risk
(api_key, "startTime")index from migration20260823000000_add_spend_logs_api_key_starttime_indexwould sort each key's rows without itDependency graph
GET /user/daily/activityandGET /user/daily/activity/aggregatedGET /team/daily/activityandGET /team/daily/activity/aggregatedGET /tag/daily/activity,GET /organization/daily/activity,GET /customer/daily/activity,GET /end_user/daily/activity, andGET /agent/daily/activityPOST /usage/ai/chat, which returned the same tool call and the same totals on every side (rows)GET /gateway/daily/activityreads a table with no key column and imports nothing from the daily activity module, and the Usage page called it on both sides (rows)fill_missing_api_key_aliases, which uses none of the changed code, and this run did not drive them liveNot verified
(api_key, "startTime")indexThe merged tree is 2c12d4e31d, which is 59d6c46 merged with main at ffb15f9
59d6c46 passes /live-pr-risk
Note
Medium Risk
Changes spend metadata SQL and semantics on daily activity paths (including deliberate tradeoffs: names only on window edges within the 100-row cap, aliases when oldest/newest named rows agree). Mis-tuned probes or planner settings could still yield null names or slow batches on very large miss sets.
Overview
Replaces the spend-log fallback that named missing API keys so daily activity routes stop timing out on busy hashed keys and can return
key_alias(and related fields) again.The old query aggregated every
LiteLLM_SpendLogsrow per key in the window. The new path probes each digest with at mostSPEND_LOG_KEY_METADATA_ROWS_PER_PROBE(100) rows from the oldest and newest ends, picks the first named row on each side, and merges alias/team/user only when those edge values agree (with updated_unanimousnull handling). The newest-side scan starts after the oldest probe’s stop time so keys with fewer than ~200 rows are not scanned twice. The lookup transaction also runsSET LOCAL enable_bitmapscan = offalongside the existing statement timeout so Postgres tends to use the(api_key, "startTime")index.Tests add PostgreSQL-backed unit coverage for row bounds and edge cases, integration probes for daily activity naming/cache/timeout behavior, and a traffic test that purged keys still get aliases from spend logs.
Reviewed by Cursor Bugbot for commit 54da4e5. Bugbot is set up for automated code reviews on this repo. Configure here.