fix: use group_by instead of find_many(distinct) in /tag/list to avoid full table scan

[CAFxX]

Summary

• The /tag/list endpoint used find_many(distinct=["tag"]) on LiteLLM_DailyTagSpend to discover dynamic tags. Prisma's distinct is a client-side post-processing filter that fetches all rows with all columns before deduplicating, causing a full table scan of 3M+ rows on every usage page load.
• Replaced with Prisma's group_by() which generates proper GROUP BY SQL, letting the database handle deduplication efficiently.
• NULL tags are filtered out via where={"tag": {"not": None}}.
• created_at and updated_at are preserved in the response via _min/_max aggregates.
• Added unit tests for the list_tags endpoint.

Alternative: single raw SQL query

If raw SQL is acceptable, the entire endpoint could be simplified to a single query using UNION ALL, eliminating the need for two separate queries and Python-side deduplication:

-- Stored tags (with budget info)
SELECT
    t.tag_name AS name,
    t.description,
    t.models,
    t.model_info,
    t.created_at,
    t.updated_at,
    t.created_by,
    b.budget_id,
    b.max_budget,
    b.soft_budget,
    b.budget_duration,
    b.budget_reset_at
FROM "LiteLLM_TagTable" t
LEFT JOIN "LiteLLM_BudgetTable" b ON t.budget_id = b.budget_id

UNION ALL

-- Dynamic-only tags (not in TagTable)
SELECT
    dts.tag AS name,
    'This is just a spend tag that was passed dynamically in a request. It does not control any LLM models.' AS description,
    NULL::text[] AS models,
    NULL::jsonb AS model_info,
    MIN(dts.created_at) AS created_at,
    MAX(dts.updated_at) AS updated_at,
    NULL AS created_by,
    NULL AS budget_id,
    NULL::double precision AS max_budget,
    NULL::double precision AS soft_budget,
    NULL AS budget_duration,
    NULL::timestamptz AS budget_reset_at
FROM "LiteLLM_DailyTagSpend" dts
WHERE dts.tag IS NOT NULL
  AND dts.tag NOT IN (SELECT tag_name FROM "LiteLLM_TagTable")
GROUP BY dts.tag

This pushes all filtering and deduplication into the database. The NOT IN subquery is safe here because tag_name is the primary key of LiteLLM_TagTable (never NULL). This approach goes against the project's CLAUDE.md guideline to prefer Prisma model methods over raw SQL, so the current PR uses group_by() instead.

Test plan

• test_list_tags_with_dynamic_tags — verifies group_by is called, dynamic tags are merged, duplicates with stored tags are excluded, and created_at/updated_at are present
• test_list_tags_no_dynamic_tags — verifies correct behavior when no dynamic tags exist
• All existing tests in test_tag_management_endpoints.py continue to pass (12/12)

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Error		Mar 11, 2026 6:55pm

[greptile-apps]

Greptile Summary

This PR fixes a significant performance regression in the /tag/list endpoint by replacing find_many(distinct=["tag"]) — which caused a full client-side table scan on LiteLLM_DailyTagSpend — with a proper group_by(by=["tag"]) that delegates deduplication to the database via a GROUP BY SQL clause. NULL tags are correctly excluded with where={"tag": {"not": None}}, and created_at/updated_at timestamps are preserved through _min/_max aggregates with proper defensive null-checking.

Key improvements:

• Core fix: group_by replaces find_many(distinct=...), eliminating full table scans on 3M+ row tables for every UI page load.
• NULL filtering: where={"tag": {"not": None}} correctly guards against NULL tag rows being surfaced in the response.
• Defensive aggregate access: row.get("_min", {}).get("created_at") pattern with proper None checks matches how other group_by callers in the codebase handle aggregates.
• Tests added: Two new unit tests verify the group_by call signature (including the NULL filter) and correct merging/deduplication behavior. Both tests validate call arguments with assert_called_once_with(...) and are mock-only with no real network calls.
• The LiteLLM_DailyTagSpendTable import is cleanly removed as it is no longer needed.

Confidence Score: 5/5

• This PR is safe to merge — it is a well-scoped performance fix with no functional regressions, proper NULL filtering, and comprehensive unit test coverage.
• The change is narrowly scoped to one query in list_tags. The new group_by approach is consistent with how other endpoints in the codebase use the same Prisma API. Aggregate fields are accessed defensively with proper null checks. Both new tests use assert_called_once_with(...) to fully validate the critical call arguments including NULL filtering. All previously flagged issues have been verified to be false positives—they either critique code that already implements the fix, misread the actual implementation, or misunderstood what the tests assert. No real issues remain.
• No files require special attention.

_{Last reviewed commit: 2c1c6f1}

[RheagalFire]

Nitpick:
I don't think the if-else guard is needed here. The group_by query returns "dynamically" created tags which are only created by the spend tracking system and are guaranteed to have a spend row corresponding to them.

[RheagalFire]

Thank you for adding this test but let's update it to only test the response and not the exact db call that's being made.