feat: forward-port dd user_email span tag and model hub description from internal staging by devin-ai-integration[bot] · Pull Request #42749 · BerriAI/litellm

devin-ai-integration · 2026-09-23T15:06:20Z

TLDR

Problem this solves:

feat(dd_span_tagger): emit litellm.user_email span tag for JWT-authenticated requests #34939 and feat(model_hub): surface model_info.description in Model Hub #37762 merged into litellm_internal_staging but never reached main
that branch is being deleted (see chore(ci): drop litellm_internal_staging and litellm_oss_staging references, main is the only trunk #42745), so the work would be lost

How it solves it:

cherry-picks the four commits onto main with -x, original author kept
Datadog span tagger emits litellm.user_email so JWT-authenticated requests are filterable per person
model_info.description surfaces on /model_group/info and the Model Hub table
one conflict in tests/test_litellm/test_router.py, resolved by keeping both main's test and the ported one

User Flow

Before: an admin cannot tell which person made a JWT-authenticated request in Datadog, and model descriptions from the config never show up anywhere

A developer authenticated with a JWT sends POST https://litellm-domain/v1/chat/completions
The admin opens the request span in Datadog LLM Observability and sees litellm.key_alias empty, no per-user tag at all
The admin sets model_info.description: "Best for long documents" on a model in the config and restarts
GET https://litellm-domain/model_group/info returns the group with no description field
https://litellm-domain/ui/?page=model-hub lists the model without any description

After: the same span carries the user's email, and the description shows on the API and in the Model Hub

The same developer sends the same POST https://litellm-domain/v1/chat/completions with the JWT
The span in Datadog carries litellm.user_email:"dev@example.com", so @litellm.user_email:"dev@example.com" filters that person's requests
The admin sets the same model_info.description and restarts
GET https://litellm-domain/model_group/info returns "description": "Best for long documents" on the group
https://litellm-domain/ui/?page=model-hub shows the text under Description when the model row is expanded

Relevant issues

Forward-port of #34939 and #37762. Companion to #42745

Affected release

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have added meaningful tests
The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

The commits are byte-for-byte the ones reviewed and merged in #34939 and #37762, whose proof runs live there. Local checks on this branch:

$ LITELLM_LOCAL_MODEL_COST_MAP=True uv run --no-sync pytest tests/test_litellm/test_router.py tests/test_litellm/proxy/test_common_request_processing.py -q
396 passed
$ LITELLM_LOCAL_MODEL_COST_MAP=True uv run --no-sync pytest tests/test_litellm/integrations/datadog/ tests/test_litellm/proxy/public_v1/test_model_hub.py -q
209 passed
$ make check
PASS (all gates, schema.d.ts regenerated via npm run gen:api and byte-identical)

Caveats

The tests/test_litellm/test_router.py conflict was the only manual edit in the cherry-picks, and everything else applied cleanly. Main was merged in afterwards, which needed one more manual edit in litellm/router.py so the description lookup sits on top of main's newer deployment lookup for a model group, and the ported test now lives in tests/unit/test_router/test_router.py. The two ported user_email tests were also rewritten to read the tags that land on the active span, because the originals only inspected the patched tag setter and the test quality gate counted them as two new mock-echo tests over main. No new live proxy run on this branch, the proof lives on the original PRs

Link to Devin session: https://app.devin.ai/sessions/184c56e57d1f4eb8876c97a72ac57526
Open in Devin Desktop: https://app.devin.ai/desktop/session/184c56e57d1f4eb8876c97a72ac57526?variant=devin
Requested by: @mateo-berri

Note

Low Risk
Adds optional tracing metadata and surfaces existing config descriptions; no changes to auth logic or request routing behavior.

Overview
Forward-ports observability and Model Hub work from internal staging: Datadog request spans can now carry litellm.user_email when the authenticated identity has an email (JWT claim / virtual key user row), so admins can filter traces by person when key_alias is empty.

Model group descriptions from config model_info.description are aggregated into ModelGroupInfo (first string description found across deployments in the group), exposed on the model-group info API / OpenAPI schema, and shown in the Model Hub detail panels (admin and public UI).

^{Reviewed by Cursor Bugbot for commit 1fd0c65. Bugbot is set up for automated code reviews on this repo. Configure here.}

…ticated requests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> (cherry picked from commit b4f9e14)

…stency Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> (cherry picked from commit 819faa0)

…nd Model Hub UI Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> (cherry picked from commit b224b15)

…mutation Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> (cherry picked from commit 342b93f)

devin-ai-integration · 2026-09-23T15:06:24Z

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

Disable automatic comment, CI, and merge conflict monitoring

greptile-apps · 2026-09-23T15:11:00Z

Confidence Score: 5/5

The PR appears safe to merge, with no outstanding findings from this re-review.

Findings

Description Column Is Missing ▶

Summary

This PR forward-ports Datadog user-email tagging and model-group descriptions.

Adds the authenticated user’s email to Datadog request-span tags when available.
Propagates configured model descriptions through ModelGroupInfo.
Displays model descriptions in the private and public Model Hub expanded-detail views.
Adds backend tests and updates generated API types.
No code changed since the previous review.

_{Reviews (2) · Last reviewed commit: "refactor(router): aggregate model group ..."}

greptile-apps · 2026-09-23T15:11:06Z

  supports_function_calling: boolean;
  supported_openai_params?: string[];
  is_public_model_group: boolean;
+  description?: string;


Description Column Is Missing

Both model tables use explicit column lists, but neither list includes description. The new field appears only after opening a model's details, so users cannot see the promised Description column in the Model Hub table.

Description renders in each rows expanded details panel, matching the merged upstream implementation. The tables explicit column list was intentionally left unchanged.

You’re right — the stated user flow promises the description in the expanded model details, and this change implements that in both model hub tables. The explicit column lists are intentionally unchanged, so my previous finding does not apply; no change is needed.

codspeed · 2026-09-23T15:12:57Z

Merging this PR will not alter performance

⚠️

1 benchmark spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

test_completion_streaming

✅ 31 untouched benchmarks

_{Comparing litellm_forward_port_staging_features (1fd0c65) with main (b2cc5f2)¹}

No successful run was found on main (f6bb877) during the generation of this report, so b2cc5f2 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

mateo-berri · 2026-09-23T15:16:49Z

@greptileai

codecov · 2026-09-23T15:18:48Z

Codecov Report

❌ Patch coverage is 66.66667% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/router.py	33.33%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

mateo-berri · 2026-09-23T15:20:45Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

…staging_features # Conflicts: # litellm/router.py # tests/unit/test_router/test_router.py

ryan-crabbe-berri · 2026-10-10T23:04:34Z

bugbot run

devin-ai-integration · 2026-10-10T23:04:37Z

❌ Cannot revive Devin session - the session is too old. Please start a new session instead.

View session

cursor

Stale Bugbot comment from a previous run.

…stead of on the patched setter The two ported tests only inspected the mock that replaced the tag setter, which the test quality gate counts as two new mock-echo tests over main. They now read the tags that land on the active span

ryan-crabbe-berri · 2026-10-10T23:12:00Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

…model_info reader The ported lookup read model_info off the untyped deployment dict, which added three basedpyright errors over main. It now goes through the existing validated model_info reader

ryan-crabbe-berri · 2026-10-10T23:36:08Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 1fd0c65. Configure here.}

milan-berri and others added 4 commits September 23, 2026 14:45

feat(dd_span_tagger): emit litellm_user_email span tag for JWT-authen…

f4c1069

…ticated requests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> (cherry picked from commit b4f9e14)

refactor(dd_span_tagger): use dotted litellm.user_email tag for consi…

506f493

…stency Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> (cherry picked from commit 819faa0)

feat(model_hub): surface model_info.description in model group info a…

f60f88b

…nd Model Hub UI Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> (cherry picked from commit b224b15)

refactor(router): aggregate model group description without in-place …

3a0780c

…mutation Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> (cherry picked from commit 342b93f)

devin-ai-integration Bot requested a review from a team September 23, 2026 15:06

devin-ai-integration Bot requested a review from yuneng-berri as a code owner September 23, 2026 15:06

devin-ai-integration Bot assigned mateo-berri Sep 23, 2026

devin-ai-integration Bot requested a review from ryan-crabbe-berri as a code owner September 23, 2026 15:06

greptile-apps Bot reviewed Sep 23, 2026

View reviewed changes

cursor Bot reviewed Sep 23, 2026

View reviewed changes

Merge remote-tracking branch 'origin/main' into litellm_forward_port_…

41fe9aa

…staging_features # Conflicts: # litellm/router.py # tests/unit/test_router/test_router.py

cursor Bot reviewed Oct 10, 2026

View reviewed changes

refactor(router): read the model group description through the typed …

1fd0c65

…model_info reader The ported lookup read model_info off the untyped deployment dict, which added three basedpyright errors over main. It now goes through the existing validated model_info reader

cursor Bot reviewed Oct 10, 2026

View reviewed changes

ryan-crabbe-berri approved these changes Oct 10, 2026

View reviewed changes

ryan-crabbe-berri merged commit 31e2ce1 into main Oct 10, 2026
86 checks passed

ryan-crabbe-berri deleted the litellm_forward_port_staging_features branch October 10, 2026 23:48

mateo-berri mentioned this pull request Oct 12, 2026

fix(bedrock_mantle): map the OpenAI-shaped context overflow envelope too #41747

Merged

8 tasks done

Conversation

devin-ai-integration Bot commented Sep 23, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

User Flow

Relevant issues

Affected release

Linear ticket

Pre-Submission checklist

Delays in PR merge?

Screenshots / Proof of Fix

Caveats

Uh oh!

devin-ai-integration Bot commented Sep 23, 2026

Uh oh!

greptile-apps Bot commented Sep 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Findings

Summary

Uh oh!

greptile-apps Bot Sep 23, 2026

Choose a reason for hiding this comment

Uh oh!

mateo-berri Sep 23, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Sep 23, 2026

Choose a reason for hiding this comment

Uh oh!

codspeed Bot commented Sep 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

mateo-berri commented Sep 23, 2026

Uh oh!

codecov Bot commented Sep 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mateo-berri commented Sep 23, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ryan-crabbe-berri commented Oct 10, 2026

Uh oh!

devin-ai-integration Bot commented Oct 10, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ryan-crabbe-berri commented Oct 10, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ryan-crabbe-berri commented Oct 10, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

devin-ai-integration Bot commented Sep 23, 2026 •

edited by cursor Bot

Loading

greptile-apps Bot commented Sep 23, 2026 •

edited

Loading

codspeed Bot commented Sep 23, 2026 •

edited

Loading

codecov Bot commented Sep 23, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading