Skip to content

fix(proxy): recover session key owners from daily spend for usage attribution - #43642

Merged
mateo-berri merged 6 commits into
mainfrom
litellm_usage_daily_spend_owner_fallback
Sep 29, 2026
Merged

mateo-berri merged 6 commits into
mainfrom
litellm_usage_daily_spend_owner_fallback

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • Usage reports show no owner for CLI session keys
  • Happens when spend logs are off, purged, or out of window
  • Model Activity key list never showed who owns a key

How it solves it:

  • Falls back to the user on the key's daily spend rows
  • Only for keys that are missing from the keys table
  • Only when every row that names a user names the same one
  • Model Activity key rows name the owner, by email or id

This completes the fix started in #40729, which only recovered the owner from spend logs

Intentional product change: on the Usage page's Top Virtual Keys (Global, Team, and Tag views), a session key with no alias now shows its owner's email in the Key Alias cell and a User column appears, Model Activity key rows name the owner, and the Key Activity search finds these keys by email. This is what makes the keys attributable. Keys with no known owner still show the truncated hash, and no column goes away, but on the Tag view the wider table puts Spend (USD) behind a sideways scroll in a browser window 1500 px wide (it fits at 1728 px), the same as for any key with an owner today

User Flow

Before: an admin on a proxy with spend logs off cannot tell which user a CLI session key belongs to

  1. A developer runs lite login, signs in with SSO, and sends chat completions through the proxy with the session key
  2. The admin calls GET http://localhost:4000/user/daily/activity/aggregated?start_date=2026-09-29&end_date=2026-09-29
  3. Each session key comes back with "user_id": null and "user_email": null
  4. The admin opens http://localhost:4000/ui/?page=new_usage, Cost tab. Top Virtual Keys has no User column and Key Alias shows -
  5. On the Model Activity tab, "Top Virtual Keys by Spend" lists only truncated key hashes

pr43642-005927502f-base_cost_top_keys_boxed.png
pr43642-005927502f-base_model_activity_top_keys_boxed.png

After: the same report and the same page name the owner of every session key

  1. A developer runs lite login, signs in with SSO, and sends chat completions through the proxy with the session key
  2. The admin calls GET http://localhost:4000/user/daily/activity/aggregated?start_date=2026-09-29&end_date=2026-09-29
  3. Each session key comes back with the developer's user_id and user_email
  4. The admin opens http://localhost:4000/ui/?page=new_usage, Cost tab. Top Virtual Keys shows the developer's email in Key Alias and a new User column
  5. On the Model Activity tab, each session key row shows the developer's email, or a "User:" line under an aliased key

pr43642-005927502f-tip_cost_top_keys_boxed.png
pr43642-005927502f-tip_model_activity_top_keys_boxed.png

Linear ticket

Resolves LIT-7572

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Everything below ran against one Postgres and one Redis, with real SSO logins and real Anthropic calls, and nothing mocked. Setup shared by both sides:

  1. A v1.103.0-rc.1 proxy on :58712 (2 workers, the config below) recorded the spend, since that release still records CLI session spend under a key hash that is not in the keys table

  2. One developer signed in twice with lite login through Google SSO, which gave them two CLI session keys

  3. They sent 5 real claude-haiku-4-5 chat completions through that proxy, 3 on one key and 2 on the other

  4. POST http://localhost:58712/user/update set their email to dev.one@example.com

  5. Two more proxies read the same database with the same config, 2 workers each, each serving its own dashboard build: base dab2deb on :21988 and tip 0059275 on :38554

  6. For the timeout case only, seed_timeout.sql and seed_timeout_wide.sql below added 8,400,000 daily spend rows for the two keys and the rig dropped LiteLLM_DailyUserSpend_api_key_idx, so the owner lookup had to scan an 8,400,002-row table (10220 MB with its remaining indexes, 6642 MB for the table alone, measured after the drop). The rows were deleted and the index was put back after the case

  7. For the two-user case only, seed_probe_keys.sql below added 4 daily spend rows for two made-up keys, one used by two users and one with a row that names no user. The rows were deleted afterwards

Rows and filter for the two-user case
INSERT INTO "LiteLLM_DailyUserSpend"
  (id, user_id, date, api_key, model, model_group, custom_llm_provider,
   prompt_tokens, completion_tokens, spend, updated_at, api_requests, successful_requests)
VALUES
  (gen_random_uuid()::text, 'probe-user-a', '2026-09-29', 'probe-key-with-two-users', 'probe-model-1', 'probe', 'anthropic', 1, 1, 0.0, now(), 1, 1),
  (gen_random_uuid()::text, 'probe-user-b', '2026-09-29', 'probe-key-with-two-users', 'probe-model-2', 'probe', 'anthropic', 1, 1, 0.0, now(), 1, 1),
  (gen_random_uuid()::text, 'probe-user-c', '2026-09-29', 'probe-key-with-one-user-and-a-blank-row', 'probe-model-1', 'probe', 'anthropic', 1, 1, 0.0, now(), 1, 1),
  (gen_random_uuid()::text, '', '2026-09-29', 'probe-key-with-one-user-and-a-blank-row', 'probe-model-2', 'probe', 'anthropic', 1, 1, 0.0, now(), 1, 1);

probes.jq

[.results[].breakdown.api_keys // {} | to_entries[] | select(.key | startswith("probe-")) | {key: .key, user_id: .value.metadata.user_id, user_email: .value.metadata.user_email, key_exists: .value.metadata.key_exists, requests: .value.metrics.api_requests}] | sort_by(.key) | .[]
Proxy config, the same file on all three proxies
model_list:
  - model_name: gpt-5.4-nano
    litellm_params:
      model: openai/gpt-5.4-nano
      api_key: os.environ/OPENAI_API_KEY
  - model_name: claude-haiku-4-5
    litellm_params:
      model: anthropic/claude-haiku-4-5
      api_key: os.environ/ANTHROPIC_API_KEY
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL
  disable_spend_logs: true
seed_timeout.sql and seed_timeout_wide.sql from step 6
INSERT INTO "LiteLLM_DailyUserSpend"
  (id, user_id, date, api_key, model, model_group, custom_llm_provider,
   prompt_tokens, completion_tokens, spend, updated_at, api_requests, successful_requests)
SELECT gen_random_uuid()::text, k.user_id, '2024-01-01', k.api_key, 'seed-' || i, 'seed', 'anthropic',
       1, 1, 0.0, now(), 1, 1
FROM (SELECT DISTINCT api_key, user_id FROM "LiteLLM_DailyUserSpend" WHERE date = '2026-09-29') k,
     generate_series(1, 3000000) i;
INSERT INTO "LiteLLM_DailyUserSpend"
  (id, user_id, date, api_key, model, model_group, custom_llm_provider,
   prompt_tokens, completion_tokens, spend, updated_at, api_requests, successful_requests)
SELECT gen_random_uuid()::text, k.user_id, '2024-01-02', k.api_key, 'wide-' || i || repeat('x', 1500), 'seed', 'anthropic',
       1, 1, 0.0, now(), 1, 1
FROM (SELECT DISTINCT api_key, user_id FROM "LiteLLM_DailyUserSpend" WHERE date = '2026-09-29') k,
     generate_series(1, 1200000) i;

Every case prints with owners.jq, which cuts key hashes to 12 characters and user ids to 4:

[.results[].breakdown.api_keys // {} | to_entries[] | {key: .key[0:12], key_alias: .value.metadata.key_alias, user_id: (.value.metadata.user_id | if . then .[0:4] + "..." else null end), user_email: .value.metadata.user_email, key_exists: .value.metadata.key_exists, requests: .value.metrics.api_requests}] | sort_by(.key) | .[]

Before (dab2deb)

Aggregated usage API

  1. Ask for the day's usage twice. Both session keys come back with no user_id and no user_email

    for i in 1 2; do
      curl -sS -o agg.json -w "run $i: HTTP %{http_code} in %{time_total}s\n" \
        -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
        "http://localhost:21988/user/daily/activity/aggregated?start_date=2026-09-29&end_date=2026-09-29"
      jq -c -f owners.jq agg.json
    done
    run 1: HTTP 200 in 0.527830s
    {"key":"eb82a3b8f88f","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":3}
    {"key":"fed050208fd3","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":2}
    run 2: HTTP 200 in 0.805786s
    {"key":"eb82a3b8f88f","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":3}
    {"key":"fed050208fd3","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":2}
    

Per-day usage API

  1. Ask for the same day from the per-day route twice. Both session keys come back with no owner

    for i in 1 2; do
      curl -sS -o day.json -w "run $i: HTTP %{http_code} in %{time_total}s\n" \
        -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
        "http://localhost:21988/user/daily/activity?start_date=2026-09-29&end_date=2026-09-29"
      jq -c '.results[] | {date, keys: (.breakdown.api_keys | length)}' day.json
      jq -c -f owners.jq day.json
    done
    run 1: HTTP 200 in 0.081014s
    {"date":"2026-09-29","keys":2}
    {"key":"eb82a3b8f88f","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":3}
    {"key":"fed050208fd3","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":2}
    run 2: HTTP 200 in 0.038918s
    {"date":"2026-09-29","keys":2}
    {"key":"eb82a3b8f88f","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":3}
    {"key":"fed050208fd3","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":2}
    

Team and tag usage APIs

  1. Ask the team and tag routes for the same day. Both session keys come back with no owner on all three

    for route in team/daily/activity team/daily/activity/aggregated tag/daily/activity; do
      curl -sS -o route.json -w "$route: HTTP %{http_code} in %{time_total}s\n" \
        -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
        "http://localhost:21988/$route?start_date=2026-09-29&end_date=2026-09-29"
      jq -c -f owners.jq route.json
    done
    team/daily/activity: HTTP 200 in 0.068074s
    {"key":"eb82a3b8f88f","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":3}
    {"key":"fed050208fd3","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":2}
    team/daily/activity/aggregated: HTTP 200 in 0.054143s
    {"key":"eb82a3b8f88f","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":3}
    {"key":"fed050208fd3","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":2}
    tag/daily/activity: HTTP 200 in 0.101265s
    {"key":"eb82a3b8f88f","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":6}
    {"key":"fed050208fd3","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":4}
    

Admin UI

  1. Open http://localhost:21988/ui/?page=new_usage and log in as admin. The Cost and Model Activity tabs look like the Before screenshots under User Flow

  2. Pick "Team Usage" in the view selector at the top. Top Virtual Keys shows Key ID, Key Alias (-), and Spend

    pr43642-005927502f-base_team_top_keys_boxed.png

  3. Pick "Tag Usage" with the browser 1500 px wide. The keys table has four columns and all of them fit

    pr43642-005927502f-base_tag_table_1500_boxed.png

  4. Pick "Global Usage", open the Key Activity tab, and type dev.one@example.com into "Search keys". It reads "Showing 0 of 2 keys"

    pr43642-005927502f-base_key_search_boxed.png

Owner lookup timeout

  1. With the table from setup step 6 in place, ask for the day's usage twice. Both answers are fast and carry no owner (captured 2026-09-29 21:51 UTC, times rounded to the millisecond)

    for i in 1 2; do
      curl -sS -o base_timeout_$i.json -w "base run $i: HTTP %{http_code} in %{time_total}s\n" \
        -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
        "http://localhost:21988/user/daily/activity/aggregated?start_date=2026-09-29&end_date=2026-09-29"
      jq -c -f owners.jq base_timeout_$i.json
    done
    base run 1: HTTP 200 in 0.351s
    {"key":"eb82a3b8f88f","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":3}
    {"key":"fed050208fd3","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":2}
    base run 2: HTTP 200 in 0.019s
    {"key":"eb82a3b8f88f","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":3}
    {"key":"fed050208fd3","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":2}
    

Keys with two users or a blank user

  1. With the rows from setup step 7 in place, ask for the day's usage twice. Neither made-up key gets an owner (captured 2026-09-29 21:24 UTC)

    for i in 1 2; do
      curl -sS -o probe_base_$i.json -w "base run $i: HTTP %{http_code} in %{time_total}s\n" \
        -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
        "http://localhost:21988/user/daily/activity/aggregated?start_date=2026-09-29&end_date=2026-09-29"
      jq -c -f probes.jq probe_base_$i.json
    done
    base run 1: HTTP 200 in 0.279707s
    {"key":"probe-key-with-one-user-and-a-blank-row","user_id":null,"user_email":null,"key_exists":false,"requests":2}
    {"key":"probe-key-with-two-users","user_id":null,"user_email":null,"key_exists":false,"requests":2}
    base run 2: HTTP 200 in 0.011177s
    {"key":"probe-key-with-one-user-and-a-blank-row","user_id":null,"user_email":null,"key_exists":false,"requests":2}
    {"key":"probe-key-with-two-users","user_id":null,"user_email":null,"key_exists":false,"requests":2}
    

After (0059275)

Aggregated usage API

  1. Ask for the day's usage twice. Both session keys come back with the developer's user_id and user_email

    for i in 1 2; do
      curl -sS -o agg.json -w "run $i: HTTP %{http_code} in %{time_total}s\n" \
        -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
        "http://localhost:38554/user/daily/activity/aggregated?start_date=2026-09-29&end_date=2026-09-29"
      jq -c -f owners.jq agg.json
    done
    run 1: HTTP 200 in 0.573903s
    {"key":"eb82a3b8f88f","key_alias":null,"user_id":"1018...","user_email":"dev.one@example.com","key_exists":false,"requests":3}
    {"key":"fed050208fd3","key_alias":null,"user_id":"1018...","user_email":"dev.one@example.com","key_exists":false,"requests":2}
    run 2: HTTP 200 in 0.023237s
    {"key":"eb82a3b8f88f","key_alias":null,"user_id":"1018...","user_email":"dev.one@example.com","key_exists":false,"requests":3}
    {"key":"fed050208fd3","key_alias":null,"user_id":"1018...","user_email":"dev.one@example.com","key_exists":false,"requests":2}
    

Per-day usage API

  1. Ask for the same day from the per-day route twice. Both session keys come back with their owner

    for i in 1 2; do
      curl -sS -o day.json -w "run $i: HTTP %{http_code} in %{time_total}s\n" \
        -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
        "http://localhost:38554/user/daily/activity?start_date=2026-09-29&end_date=2026-09-29"
      jq -c '.results[] | {date, keys: (.breakdown.api_keys | length)}' day.json
      jq -c -f owners.jq day.json
    done
    run 1: HTTP 200 in 0.031053s
    {"date":"2026-09-29","keys":2}
    {"key":"eb82a3b8f88f","key_alias":null,"user_id":"1018...","user_email":"dev.one@example.com","key_exists":false,"requests":3}
    {"key":"fed050208fd3","key_alias":null,"user_id":"1018...","user_email":"dev.one@example.com","key_exists":false,"requests":2}
    run 2: HTTP 200 in 0.033822s
    {"date":"2026-09-29","keys":2}
    {"key":"eb82a3b8f88f","key_alias":null,"user_id":"1018...","user_email":"dev.one@example.com","key_exists":false,"requests":3}
    {"key":"fed050208fd3","key_alias":null,"user_id":"1018...","user_email":"dev.one@example.com","key_exists":false,"requests":2}
    

Team and tag usage APIs

  1. Ask the team and tag routes for the same day. Both session keys come back with their owner on all three

    for route in team/daily/activity team/daily/activity/aggregated tag/daily/activity; do
      curl -sS -o route.json -w "$route: HTTP %{http_code} in %{time_total}s\n" \
        -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
        "http://localhost:38554/$route?start_date=2026-09-29&end_date=2026-09-29"
      jq -c -f owners.jq route.json
    done
    team/daily/activity: HTTP 200 in 0.071374s
    {"key":"eb82a3b8f88f","key_alias":null,"user_id":"1018...","user_email":"dev.one@example.com","key_exists":false,"requests":3}
    {"key":"fed050208fd3","key_alias":null,"user_id":"1018...","user_email":"dev.one@example.com","key_exists":false,"requests":2}
    team/daily/activity/aggregated: HTTP 200 in 0.039450s
    {"key":"eb82a3b8f88f","key_alias":null,"user_id":"1018...","user_email":"dev.one@example.com","key_exists":false,"requests":3}
    {"key":"fed050208fd3","key_alias":null,"user_id":"1018...","user_email":"dev.one@example.com","key_exists":false,"requests":2}
    tag/daily/activity: HTTP 200 in 0.086461s
    {"key":"eb82a3b8f88f","key_alias":null,"user_id":"1018...","user_email":"dev.one@example.com","key_exists":false,"requests":6}
    {"key":"fed050208fd3","key_alias":null,"user_id":"1018...","user_email":"dev.one@example.com","key_exists":false,"requests":4}
    

Admin UI

  1. Open http://localhost:38554/ui/?page=new_usage and log in as admin. The Cost and Model Activity tabs look like the After screenshots under User Flow

  2. Pick "Team Usage" in the view selector at the top. Top Virtual Keys shows the developer's email in Key Alias and in a new User column

    pr43642-005927502f-tip_team_top_keys_boxed.png

  3. Pick "Tag Usage" with the browser 1500 px wide. The keys table has five columns, and Spend (USD) shows after a sideways scroll of the table

    pr43642-005927502f-tip_tag_table_1500_boxed.png
    pr43642-005927502f-tip_tag_table_1500_scrolled_boxed.png

  4. Pick "Global Usage", open the Key Activity tab, and type dev.one@example.com into "Search keys". It reads "Showing 2 of 2 keys"

    pr43642-005927502f-tip_key_search_boxed.png

Owner lookup timeout

  1. With the table from setup step 6 in place, ask for the day's usage twice. Both answers are HTTP 200 after 5 to 6 s, and neither carries an owner since the lookup hit its 5 s limit both times (captured 2026-09-29 21:51 UTC, times rounded to the millisecond)

    for i in 1 2; do
      curl -sS -o tip_timeout_$i.json -w "tip run $i: HTTP %{http_code} in %{time_total}s\n" \
        -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
        "http://localhost:38554/user/daily/activity/aggregated?start_date=2026-09-29&end_date=2026-09-29"
      jq -c -f owners.jq tip_timeout_$i.json
    done
    tip run 1: HTTP 200 in 5.913s
    {"key":"eb82a3b8f88f","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":3}
    {"key":"fed050208fd3","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":2}
    tip run 2: HTTP 200 in 5.085s
    {"key":"eb82a3b8f88f","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":3}
    {"key":"fed050208fd3","key_alias":null,"user_id":null,"user_email":null,"key_exists":false,"requests":2}
    
  2. The proxy log names the timeout, once per answer

    14:51:16 - LiteLLM Proxy:WARNING: key_metadata_recovery.py:143 - Failed daily-spend key owner recovery for 2 keys: ERROR: canceling statement due to statement timeout
    14:51:22 - LiteLLM Proxy:WARNING: key_metadata_recovery.py:143 - Failed daily-spend key owner recovery for 2 keys: ERROR: canceling statement due to statement timeout
    

Keys with two users or a blank user

  1. With the rows from setup step 7 in place, ask for the day's usage twice. The key with two users still gets no owner, and the key with a blank row gets the one user its other row names (captured 2026-09-29 21:24 UTC)

    for i in 1 2; do
      curl -sS -o probe_tip_$i.json -w "tip run $i: HTTP %{http_code} in %{time_total}s\n" \
        -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
        "http://localhost:38554/user/daily/activity/aggregated?start_date=2026-09-29&end_date=2026-09-29"
      jq -c -f probes.jq probe_tip_$i.json
    done
    tip run 1: HTTP 200 in 0.214160s
    {"key":"probe-key-with-one-user-and-a-blank-row","user_id":"probe-user-c","user_email":null,"key_exists":false,"requests":2}
    {"key":"probe-key-with-two-users","user_id":null,"user_email":null,"key_exists":false,"requests":2}
    tip run 2: HTTP 200 in 0.013963s
    {"key":"probe-key-with-one-user-and-a-blank-row","user_id":"probe-user-c","user_email":null,"key_exists":false,"requests":2}
    {"key":"probe-key-with-two-users","user_id":null,"user_email":null,"key_exists":false,"requests":2}
    

Audit matrix

The three test files this PR adds under tests/integration/spend/ ran through tests/integration/run.py accounting on a rig shaped like CircleCI's: an owned Postgres and Redis, the scripted upstream from tests/integration/_support/upstream.py, and a proxy with 2 workers per side, base dab2deb on :52094 and tip 0059275 on :52097, each on its own database. The base ran once and the tip six times. Every tip run collected the same 35 tests and passed all of them with no skip, retry, or sleep, and the base failed exactly the 22 cells the fix changes and passed the 13 it must not touch

run side pytest workers started (UTC) result
v3 base 1 21:02:31 22 failed, 13 passed in 189 s
v3_run1 tip 1 21:05:51 35 passed in 220 s
v3_run2 tip 1 21:09:40 35 passed in 194 s
v3_w4 tip 4 21:13:03 35 passed in 101 s
v3_w8a tip 8 21:14:45 35 passed in 84 s
v3_w8b tip 8 21:16:11 35 passed in 98 s
v3_w8c tip 8 21:17:50 35 passed in 97 s
The 35 cells, base dab2deb against tip 0059275 six times
id surface scenario test base tip verdict
A1 GET /user/daily/activity ownerless key, one user test_daily_activity_key_owner.py::test_key_missing_from_the_key_tables_is_reported_with_the_one_user_its_daily_spend_names[user_daily_activity] red, as expected green x6 PASS
A2 GET /user/daily/activity/aggregated same test_daily_activity_key_owner.py::test_key_missing_from_the_key_tables_is_reported_with_the_one_user_its_daily_spend_names[user_daily_activity_aggregated] red, as expected green x6 PASS
A3 GET /team/daily/activity same, team rows test_daily_activity_key_owner.py::test_key_missing_from_the_key_tables_is_reported_with_the_one_user_its_daily_spend_names[team_daily_activity] red, as expected green x6 PASS
A4 GET /team/daily/activity/aggregated same test_daily_activity_key_owner.py::test_key_missing_from_the_key_tables_is_reported_with_the_one_user_its_daily_spend_names[team_daily_activity_aggregated] red, as expected green x6 PASS
A5 GET /tag/daily/activity same, tag rows test_daily_activity_key_owner.py::test_key_missing_from_the_key_tables_is_reported_with_the_one_user_its_daily_spend_names[tag_daily_activity] red, as expected green x6 PASS
A6 GET /organization/daily/activity same, org rows test_daily_activity_key_owner.py::test_key_missing_from_the_key_tables_is_reported_with_the_one_user_its_daily_spend_names[organization_daily_activity] red, as expected green x6 PASS
A7 GET /customer/daily/activity same, end user rows test_daily_activity_key_owner.py::test_key_missing_from_the_key_tables_is_reported_with_the_one_user_its_daily_spend_names[customer_daily_activity] red, as expected green x6 PASS
A8 GET /end_user/daily/activity same test_daily_activity_key_owner.py::test_key_missing_from_the_key_tables_is_reported_with_the_one_user_its_daily_spend_names[end_user_daily_activity] red, as expected green x6 PASS
A9 GET /agent/daily/activity same, agent rows test_daily_activity_key_owner.py::test_key_missing_from_the_key_tables_is_reported_with_the_one_user_its_daily_spend_names[agent_daily_activity] red, as expected green x6 PASS
A10 POST /usage/ai/chat ownerless key in range test_daily_activity_key_owner_traffic.py::test_usage_ai_chat_hands_the_model_the_usage_summary_without_any_key_owner green green x6 PASS
B1 user aggregated key with two users test_daily_activity_key_owner.py::test_key_whose_daily_spend_names_two_users_is_reported_with_no_owner green green x6 PASS
B2 user aggregated one user plus blank-user row test_daily_activity_key_owner.py::test_daily_spend_rows_naming_no_user_do_not_hide_the_one_user_the_others_name[blank_user] red, as expected green x6 PASS
B3 user aggregated one user plus NULL-user row test_daily_activity_key_owner.py::test_daily_spend_rows_naming_no_user_do_not_hide_the_one_user_the_others_name[null_user] red, as expected green x6 PASS
B4 user aggregated only blank and NULL rows test_daily_activity_key_owner.py::test_key_whose_daily_spend_names_no_user_at_all_is_reported_with_no_owner green green x6 PASS
B5 user aggregated owner missing from user table test_daily_activity_key_owner.py::test_owner_the_user_table_does_not_hold_is_reported_by_id_with_no_email red, as expected green x6 PASS
B6 user aggregated live key with its own user, daily rows name another test_daily_activity_key_owner.py::test_live_key_keeps_its_own_user_when_its_daily_spend_names_another green green x6 PASS
B7 user aggregated live key with no user, daily rows name one test_daily_activity_key_owner.py::test_live_key_with_no_user_is_not_given_the_user_its_daily_spend_names green green x6 PASS
B8 user aggregated deleted key with a user test_daily_activity_key_owner.py::test_deleted_key_keeps_its_own_user_when_its_daily_spend_names_another green green x6 PASS
B9 user aggregated deleted key with no user, daily rows name one test_daily_activity_key_owner.py::test_deleted_key_with_no_user_keeps_its_alias_and_gains_the_one_user_its_daily_spend_names red, as expected green x6 PASS
B10 user aggregated spend logs give alias only test_daily_activity_key_owner.py::test_key_named_only_by_a_spend_log_alias_keeps_that_alias_and_gains_the_one_user_its_daily_spend_names red, as expected green x6 PASS
B11 /v1/chat/completions with CLI session token, then usage current writer test_daily_activity_key_owner_traffic.py::test_cli_session_spend_is_reported_with_the_user_and_team_of_the_session green green x6 PASS
B12 /v1/chat/completions, /v1/messages, /v1/responses with a user's virtual key, then usage current writer test_daily_activity_key_owner_traffic.py::test_key_used_on_every_unified_endpoint_is_reported_with_its_own_alias_and_user green green x6 PASS
C1 team daily, LiteLLM_DailyUserSpend locked lookup cannot finish test_daily_activity_key_owner_faults.py::test_owner_lookup_gives_up_while_daily_user_spend_is_locked_and_answers_once_it_is_not red, as expected green x6 PASS
C2 user daily as a non-admin user key shared with another user test_daily_activity_key_owner_faults.py::test_user_reading_a_key_shared_with_another_user_is_shown_no_owner_and_nothing_of_the_other_user green green x6 PASS
C2b user daily as a non-admin user key only the reader spent with test_daily_activity_key_owner_faults.py::test_user_reading_a_key_only_they_spent_with_is_shown_themselves_as_its_owner red, as expected green x6 PASS
C2c user daily as a non-admin user key only another user spent with test_daily_activity_key_owner_faults.py::test_user_reading_a_key_only_another_user_spent_with_is_shown_nothing_of_it green green x6 PASS
C3 user aggregated, invalid key unauthenticated test_daily_activity_key_owner_faults.py::test_invalid_key_is_refused_without_naming_the_owner green green x6 PASS
C4 user aggregated 5 KB api_key value, one user test_daily_activity_key_owner_faults.py::test_five_kilobyte_key_is_reported_with_the_one_user_its_daily_spend_names red, as expected green x6 PASS
D1 user aggregated no rows in range test_daily_activity_key_owner_faults.py::test_key_with_no_daily_spend_is_reported_as_no_activity green green x6 PASS
D2 team aggregated, no api_key filter 300 ownerless keys of one team, one user each test_daily_activity_key_owner_faults.py::test_every_key_of_a_team_is_reported_with_its_own_user red, as expected green x6 PASS
D3 user aggregated same read twice test_daily_activity_key_owner_faults.py::test_reading_the_same_activity_twice_gives_the_same_answer green green x6 PASS
D4 user aggregated second user's row lands between reads test_daily_activity_key_owner_faults.py::test_key_stops_being_reported_with_an_owner_once_a_second_user_spends_with_it red, as expected green x6 PASS
E1 all nine routes 30 concurrent reads while 21 requests across the three unified endpoints wait on the provider test_daily_activity_key_owner_traffic.py::test_owner_is_reported_on_every_route_while_a_burst_of_requests_waits_on_the_provider red, as expected green x6 PASS
E2 owned two-worker proxy kill one worker, then read test_daily_activity_key_owner_faults.py::test_owner_is_reported_while_a_worker_is_killed_and_after_it_is_replaced red, as expected green x6 PASS
E3 owned proxy restart, then read test_daily_activity_key_owner_faults.py::test_owner_is_reported_again_after_the_proxy_restarts red, as expected green x6 PASS
  • 0059275 passes /audit (this tip is the audit's tests commit)

Seen along the way, none of it caused by this PR:

Type

🐛 Bug Fix

Caveats (if any)

Low

  • The owner lookup shares the spend-log lookup's 5 s statement timeout
    • On a timeout the keys come back with no owner, at HTTP 200
    • Not taken: its own setting, per-key probes, a date bound, a cache
  • The lookup runs on every usage page load, with no cache
    • Only for keys with no owner that are missing from the keys table
    • 41 ms for 50 keys, 416 ms for 170, 1.3 s cold for 500, on 2,730,007 rows (seed below)
    • Worst case adds 5 s, or 10 s when the spend-log lookup times out too
  • Daily spend rows with no user are ignored when picking the owner
    • A key with such rows still gets the one user its other rows name
  • A key used by more than one user gets no owner, same rule as the spend-log lookup
    • Shown live in the two-user case, on made-up rows since the rig has one SSO identity
  • Session keys now make the Tag usage keys table five columns wide
    • Spend (USD) needs a sideways scroll at 1280 and 1500 px wide, and fits at 1728 px
    • Base lays out any key with an owner the same way (screenshot in the risk report)
    • Left as is: the layout lives in a shared table this PR does not touch
  • The timeout proof needed a dropped index on an 8,400,002-row table
    • The timings two rows up were taken with the index in place
  • misc / Run tests is red on 4 test_openapi_compliance.py tests
  • CircleCI pipeline 90599, at the tip before the tests commit (same product code as this tip), is red on 31 tests after one rerun from failed per workflow, none in usage reporting
  • CircleCI pipeline 90631, at this tip, was canceled during a release with 12 failures recorded and no rerun
    • 11 in local_testing_part2 and 1 in langfuse_logging_unit_tests, all 12 among the 31 listed for pipeline 90599 above
    • Its integration workflow was canceled too, so no integration-accounting result exists at this tip
    • No new pipeline is started while the release holds the runners; the three test files ran seven times on the local rig instead (audit matrix above)
  • osv-scan is red on 3 Medium advisories against 2 packages in uv.lock, which this PR does not touch
Seed behind the lookup timings
\set ON_ERROR_STOP on
INSERT INTO "LiteLLM_DailyUserSpend"(id, user_id, date, api_key, model, model_group, custom_llm_provider, mcp_namespaced_tool_name, endpoint, prompt_tokens, completion_tokens, spend, api_requests, successful_requests)
SELECT gen_random_uuid()::text,
       'u-long-' || k,
       to_char(date '2025-09-29' + d, 'YYYY-MM-DD'),
       encode(digest('long-lived-key-' || k, 'sha256'), 'hex'),
       m.model, m.model, m.provider, '', '/v1/chat/completions',
       1000, 200, 0.01, 10, 10
FROM generate_series(1, 500) AS k
CROSS JOIN generate_series(0, 364) AS d
CROSS JOIN (VALUES ('gpt-5.4-nano','openai'), ('gpt-5.4-mini','openai'), ('claude-haiku-4-5','anthropic'), ('claude-sonnet-5-5','anthropic')) AS m(model, provider);
INSERT INTO "LiteLLM_DailyUserSpend"(id, user_id, date, api_key, model, model_group, custom_llm_provider, mcp_namespaced_tool_name, endpoint, prompt_tokens, completion_tokens, spend, api_requests, successful_requests)
SELECT gen_random_uuid()::text,
       'u-bg-' || (g % 3000),
       to_char(date '2025-09-29' + (g % 365), 'YYYY-MM-DD'),
       encode(digest('background-key-' || (g % 40000), 'sha256'), 'hex'),
       'gpt-5.4-nano', 'gpt-5.4-nano', 'openai', '', '/v1/chat/completions',
       1000, 200, 0.01, 10, 10
FROM generate_series(1, 2000000) AS g
ON CONFLICT DO NOTHING;
ANALYZE "LiteLLM_DailyUserSpend";
SELECT count(*) AS total_rows, pg_size_pretty(pg_total_relation_size('"LiteLLM_DailyUserSpend"')) AS size FROM "LiteLLM_DailyUserSpend";

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/44c211256d5d4fd3886c33c80dce006e
Open in Devin Desktop: https://app.devin.ai/desktop/session/44c211256d5d4fd3886c33c80dce006e?variant=devin
Requested by: @jesus-berri

Risk report

/live-pr-risk at 0059275 against base dab2deb, both builds live on the same database. Verdict: nothing broke and no regression was observed. Five changes a client or an admin can see are listed under Backward incompatible, each with its decision

Breaking

None observed. Every route below answered HTTP 200 on both builds, and a whole-body jq -S diff of base against tip shows only user_id and user_email changing

Backward incompatible

  • metadata.user_id and metadata.user_email go from null to the owner for session keys, on the user, team, and tag daily activity routes. Observed in the API cases above. A client that groups keys by a null owner sees them move. This is the change LIT-7572 asks for

  • Top Virtual Keys on the Global, Team, and Tag views shows the email in Key Alias and gains a User column, and Model Activity rows name the owner. Observed in the screenshots above. This is the intended product change named in the TLDR

  • On the Tag view the five-column keys table puts Spend (USD) behind a sideways scroll at 1280 and 1500 px wide. Observed on tip, and on base with one extra key that has an owner (below, Spend cut off at the right edge), so the layout predates this PR and session keys now reach it. Accepted as a Low caveat by @mateo-berri on 2026-09-29, in this description

    pr43642-005927502f-base_owned_tag_table_1500_boxed.png

  • The Key Activity search by email goes from "Showing 0 of 2 keys" to "Showing 2 of 2 keys". Observed in the Admin UI case. This follows from the change LIT-7572 asks for

  • Usage routes can take up to 5 s longer when the owner lookup is slow, and then answer with no owner. Observed in the timeout case. Accepted as a Low caveat by @mateo-berri on 2026-09-29, in this description

Regression risk

  • The AI usage chat runs the same lookup and never reads key owners. POST /usage/ai/chat gave the same answer on both builds, HTTP 200 in 2.9 s on the base and 2.5 s on the tip

    curl -sS -N -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" \
      -d '{"model":"claude-haiku-4-5","messages":[{"role":"user","content":"How many requests and how much spend were recorded on 2026-09-29? Answer in one sentence."}]}' \
      http://localhost:38554/usage/ai/chat
    base: On 2026-09-29, there were 5 requests recorded with a total spend of $0.0003.
    tip:  On 2026-09-29, there were 5 requests recorded with a total spend of $0.0003.
    
  • The old usage page calls none of the changed routes and imports none of the changed components, so it cannot differ

  • A key with two users gets no owner on both builds, shown in the two-user case

Dependency graph

Dependent Status
GET /user/daily/activity verified-live
GET /user/daily/activity/aggregated verified-live
GET /team/daily/activity verified-live
GET /team/daily/activity/aggregated verified-live
GET /tag/daily/activity verified-live
GET /organization/daily/activity verified-live, no rows
GET /customer/daily/activity verified-live, no rows
GET /end_user/daily/activity verified-live, no rows
GET /agent/daily/activity verified-live, no rows
AI usage chat verified-live
Usage page, Cost tab verified-live
Usage page, Model Activity tab verified-live
Usage page, Key Activity search verified-live
Usage page, Team view verified-live
Usage page, Tag view verified-live
Old usage page unreachable
Owner lookup and its fallbacks tested

Not verified

  • The top cases were not run again on a tree merged into today's main. git merge-tree against main at 2d034bb is clean, and no commit on main since the base touches this PR's files (checked 2026-09-29 22:07 UTC)

  • The organization, customer, end user, and agent routes returned the same empty body on both builds, since the rig has no such spend

  • No outbound recorder was used, since the change adds one database read and no outbound call

  • 0059275 passes /live-pr-risk


Note

Medium Risk
Changes usage API metadata and adds an uncached DB lookup on daily-activity loads (up to ~5s timeout per request); behavior is gated but affects reporting and UI attribution for shared or ambiguous keys.

Overview
Adds a daily-spend fallback so usage APIs and the dashboard can attribute ownerless session keys (e.g. CLI session hashes not in the keys table) when spend logs are unavailable.

Proxy: get_api_key_metadata now calls recover_key_owner_from_daily_spend, which reads LiteLLM_DailyUserSpend (with the same statement timeout as spend-log recovery) for keys that still lack user_id and are not active in the keys table. An owner is set only when all non-empty daily rows agree on one user; conflicting or all-blank rows leave owner null. Live/deleted keys keep their table-backed user and are not overridden by daily spend.

Dashboard: Model Activity “Top Virtual Keys” and related aggregations expose owner email or user id on each key row (with UI to avoid duplicating the alias label).

Tests: New integration coverage across daily-activity routes, auth/timeout/worker fault cases, and live traffic; unit tests for recovery and metadata wiring.

Reviewed by Cursor Bugbot for commit 0059275. Bugbot is set up for automated code reviews on this repo. Configure here.

jesus-berri and others added 2 commits September 28, 2026 23:36
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

devin-ai-integration Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@codecov

codecov Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@greptile-apps

greptile-apps Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Medium risk] Adds owner recovery for API keys from daily spend records.

The PR appears safe to merge; no new actionable issue was established.

Summary

The PR recovers missing session-key owners from daily spend when earlier metadata lookups cannot identify them, and displays recovered owners in usage reporting.

  • Adds integration and unit coverage for attribution, conflicting owners, timeouts, and usage routes.

Reviews (3) · Last reviewed commit: "test(integration): audit the daily activ..."

Comment thread litellm/proxy/spend_tracking/key_metadata_recovery.py
Comment thread litellm/proxy/spend_tracking/key_metadata_recovery.py
@codspeed

codspeed Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_usage_daily_spend_owner_fallback (0059275) with main (d2a574b)

Open in CodSpeed

jesus-berri and others added 2 commits September 29, 2026 00:19
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@mateo-berri

Copy link
Copy Markdown
Contributor

@greptileai

@mateo-berri mateo-berri added run-ci and removed run-ci labels Sep 29, 2026
@mateo-berri

Copy link
Copy Markdown
Contributor

bugbot run

@cursor cursor Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

…ry usage route

Thirty five integration cells under tests/integration/spend cover the daily
spend owner fallback on all nine daily activity routes and /usage/ai/chat:
the happy path per route, the unanimity rules (two users, blank and null
rows, an owner the user table lacks, live and deleted keys with and without
their own user, a spend log alias), a non admin reader, an invalid key, a 5 KB
key, a locked LiteLLM_DailyUserSpend, 300 keys of one team, repeated reads, a
second user landing between reads, a concurrent burst across the unified
endpoints, a killed worker, and a proxy restart

The traffic cells ignore the GET /v1/models call the proxy's five minute token
limit refresh makes to every registered OpenAI compatible deployment, since it
lands on a test's provider wire whenever the refresh instant falls inside the
test
@mateo-berri

Copy link
Copy Markdown
Contributor

@greptileai

@mateo-berri

Copy link
Copy Markdown
Contributor

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 0059275. Configure here.

@mateo-berri mateo-berri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mateo-berri
mateo-berri merged commit f5a1c9f into main Sep 29, 2026
131 of 149 checks passed
@mateo-berri
mateo-berri deleted the litellm_usage_daily_spend_owner_fallback branch September 29, 2026 22:36
mateo-berri added a commit that referenced this pull request Sep 30, 2026
…tion_stable_1_103_x

fix(proxy): backport #40541, #43642, and #43656 to stable/1.103.x for v1.103.2
stvnksslr pushed a commit to stvnksslr/litellm that referenced this pull request Oct 1, 2026
Hand-ported prerequisite for the BerriAI#43642 backport to stable/1.103.x.
Taken from dd636373229b1ab3bfb2ed5b9b17b0ebd68b3ac7 (BerriAI#43288, main), scratch_database only.
stvnksslr pushed a commit to stvnksslr/litellm that referenced this pull request Oct 1, 2026
…ribution (BerriAI#43642)

Backport of BerriAI#43642 to stable/1.103.x.
Cherry-picked from f5a1c9f (main).
doonga pushed a commit to greyrock-labs/home-ops that referenced this pull request Oct 2, 2026
…03.2) (#328)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [ghcr.io/berriai/litellm](https://images.chainguard.dev/directory/image/wolfi-base/overview) ([source](https://github.com/BerriAI/litellm)) | patch | `v1.103.1` → `v1.103.2` |

---

### Release Notes

<details>
<summary>BerriAI/litellm (ghcr.io/berriai/litellm)</summary>

### [`v1.103.2`](https://github.com/BerriAI/litellm/releases/tag/v1.103.2)

[Compare Source](BerriAI/litellm@v1.103.1...v1.103.2)

#### Verify Docker Image Signature

All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](BerriAI/litellm@0112e53).

**Verify using the pinned commit hash (recommended):**

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

```bash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.103.2
```

**Verify using the release tag (convenience):**

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

```bash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.103.2/cosign.pub \
  ghcr.io/berriai/litellm:v1.103.2
```

Expected output:

```
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key
```

***

#### What's Changed

- chore(release): sync stable/1.103.x to v1.103.1 by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;43824](BerriAI/litellm#43824)
- fix(proxy): backport [#&#8203;40541](BerriAI/litellm#40541), [#&#8203;43642](BerriAI/litellm#43642), and [#&#8203;43656](BerriAI/litellm#43656) to stable/1.103.x for v1.103.2 by [@&#8203;devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#&#8203;43897](BerriAI/litellm#43897)
- fix(anthropic): backport [#&#8203;42152](BerriAI/litellm#42152) and [#&#8203;42288](BerriAI/litellm#42288) to stable/1.103.x by [@&#8203;devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#&#8203;43662](BerriAI/litellm#43662)
- fix(proxy): backport [#&#8203;43962](BerriAI/litellm#43962) to stable/1.103.x by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;43984](BerriAI/litellm#43984)

**Full Changelog**: <BerriAI/litellm@v1.103.1...v1.103.2>

</details>

---

### Configuration

📅 **Schedule**: (in timezone America/New_York)

- Branch creation
  - At any time (no schedule defined)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xMTUuMTMiLCJ1cGRhdGVkSW5WZXIiOiI0NC4xMTUuMTMiLCJ0YXJnZXRCcmFuY2giOiJtYWluIiwibGFiZWxzIjpbInJlbm92YXRlL2NvbnRhaW5lciIsInR5cGUvcGF0Y2giXX0=-->

Reviewed-on: https://git.greyrock.io/todd/home-ops/pulls/328
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants