[Feature] UI E2E Tests: Proxy Admin Team and Key Management by yuneng-berri · Pull Request #25365 · BerriAI/litellm

yuneng-berri · 2026-04-08T18:51:40Z

Summary

Add Playwright E2E tests covering proxy admin team and key management workflows, with a self-contained test runner and CircleCI integration.

Test runner (run_e2e.sh) builds the UI from source before starting the proxy, so tests always run against the latest UI changes
Added data-testid attributes to key UI components for reliable Playwright selectors
Added ui_e2e_tests job to CircleCI workflow

Testing

10 tests covering the stable release QA checklist items:

Teams (5 tests): create team, invite user, edit member (non-member team), delete team, edit member (org team)

Keys (5 tests): create key in team, regenerate key, update TPM/RPM limits, delete key, see internal user keys

Run locally:

bash ui/litellm-dashboard/e2e_tests/run_e2e.sh

Type

🆕 New Feature
✅ Test
🚄 Infrastructure

…ntegration Add Playwright E2E tests covering proxy admin team and key management workflows, with a self-contained test runner and CircleCI integration. Tests cover: create team, invite user, edit/delete team members, create key in team, regenerate key, update TPM/RPM limits, delete key, and verify internal user keys are visible. Infrastructure: run_e2e.sh builds the UI from source before starting the proxy, ensuring tests always run against the latest UI changes. Added data-testid attributes to key UI components for reliable selectors.

vercel · 2026-04-08T18:51:47Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Apr 8, 2026 8:34pm

codspeed-hq · 2026-04-08T18:54:02Z

Merging this PR will not alter performance

✅ 16 untouched benchmarks

_{Comparing litellm_e2e_ui_tests (467dbc4) with main (2dac54b)}

greptile-apps · 2026-04-08T18:56:20Z

Greptile Summary

This PR adds a self-contained Playwright E2E test suite for proxy admin team and key management workflows, replacing the previous machine-based CircleCI job with a docker-based one. It introduces a seed SQL file, a mock LLM server, a consolidated test runner script, and data-testid attributes on key UI components. The previously flagged blocking issues (missing constants exports, missing seed.sql, broken globalSetup imports, missing STORAGE_PATHS) have all been resolved in this revision.

Confidence Score: 5/5

Safe to merge — all previously blocking issues are resolved; remaining findings are minor P2 style suggestions.

All P0/P1 findings from prior review rounds (missing constants, broken globalSetup imports, absent seed.sql) are addressed. The three remaining comments are P2: redundant popup-dismiss calls, a fragile SVG selector for delete, and an unused TeamAdmin session. None of these block correctness or CI reliability.

No files require special attention beyond the P2 style suggestions in teams.spec.ts, navigation.ts, and fixtures/users.ts.

Vulnerabilities

No security concerns identified. The hardcoded credentials (sk-1234, e2epassword) are test-only values scoped to the CI environment and are not used in production paths.

Important Files Changed

Filename	Overview
ui/litellm-dashboard/e2e_tests/constants.ts	Replaced hardcoded key-ID prefixes with stable key/team aliases that match seed.sql; all exports now present and consistent with spec imports.
ui/litellm-dashboard/e2e_tests/fixtures/users.ts	Inlines Role enum and adds STORAGE_PATHS; TeamAdmin role is set up in globalSetup but has no consuming test yet (unused overhead).
ui/litellm-dashboard/e2e_tests/globalSetup.ts	Now loops over all roles using STORAGE_PATHS correctly; includes screenshot on failure and proper page cleanup.
ui/litellm-dashboard/e2e_tests/fixtures/seed.sql	New idempotent seed file; deletes and re-inserts deterministic e2e test data for users, teams, memberships, and API keys.
ui/litellm-dashboard/e2e_tests/run_e2e.sh	Self-contained runner: starts Postgres (local) or uses CI service, builds UI from source, seeds DB, starts mock + proxy, then runs Playwright.
ui/litellm-dashboard/e2e_tests/tests/proxy-admin/keys.spec.ts	Five key-management tests using seeded data; all constants now correctly imported from updated constants.ts.
ui/litellm-dashboard/e2e_tests/tests/proxy-admin/teams.spec.ts	Five team-management tests; delete-team uses a fragile svg/img locator instead of a dedicated testid for the delete button.
ui/litellm-dashboard/e2e_tests/helpers/navigation.ts	navigateToPage now internally calls dismissFeedbackPopup, but every test also calls it explicitly — redundant ~1.5 s per navigation.
.circleci/config.yml	Replaces the old machine-based e2e job with a docker-based one; workflow entry omits the context key needed to inject DockerHub credentials.
ui/litellm-dashboard/src/components/OldTeams.tsx	Adds data-testid attributes to team-id cells, create-team button, team-name input, and create-team submit button.
ui/litellm-dashboard/src/components/organisms/create_key_button.tsx	Adds data-testid=create-key-button to the create-key trigger button.

_{Reviews (5): Last reviewed commit: "[Fix] Remove old broken key tests supers..." | Re-trigger Greptile}

greptile-apps · 2026-04-08T18:56:24Z

+import { test, expect } from "@playwright/test";
+import {
+  ADMIN_STORAGE_PATH,
+  E2E_DELETE_KEY_ALIAS,
+  E2E_REGENERATE_KEY_ALIAS,
+  E2E_UPDATE_LIMITS_KEY_ALIAS,
+  E2E_INTERNAL_USER_KEY_ALIAS,
+  E2E_TEAM_CRUD_ALIAS,
+} from "../../constants";


Imported constants are missing from constants.ts

All five named imports here — E2E_DELETE_KEY_ALIAS, E2E_REGENERATE_KEY_ALIAS, E2E_UPDATE_LIMITS_KEY_ALIAS, E2E_INTERNAL_USER_KEY_ALIAS, and E2E_TEAM_CRUD_ALIAS — are not exported from e2e_tests/constants.ts. That file only exports ADMIN_STORAGE_PATH, E2E_UPDATE_LIMITS_KEY_ID_PREFIX, E2E_DELETE_KEY_ID_PREFIX, E2E_DELETE_KEY_NAME, and E2E_REGENERATE_KEY_ID_PREFIX. Every one of these imports resolves to undefined at runtime, so selectors like page.locator("tr", { hasText: E2E_DELETE_KEY_ALIAS }) match everything (or nothing), making the tests useless. The same gap exists in teams.spec.ts for E2E_TEAM_CRUD_ID, E2E_TEAM_DELETE_ALIAS, E2E_TEAM_NO_ADMIN_ID, and E2E_TEAM_ORG_ID.

greptile-apps · 2026-04-08T18:56:25Z

+import { chromium, expect } from "@playwright/test";
+import { users, Role, STORAGE_PATHS } from "./fixtures/users";
+import * as fs from "fs";


STORAGE_PATHS and Role are not exported from fixtures/users

globalSetup.ts now imports { users, Role, STORAGE_PATHS } from ./fixtures/users, but users.ts only exports users. Role is defined in ./fixtures/roles.ts and is merely imported (not re-exported) by users.ts. STORAGE_PATHS does not exist anywhere in the visible codebase. As a result, STORAGE_PATHS is undefined and storagePath on line 11 is always undefined, making page.context().storageState({ path: undefined }) throw at runtime and abort global setup entirely before any test runs.

greptile-apps · 2026-04-08T18:56:26Z

+  for (const role of Object.values(Role)) {
+    const { email, password } = users[role];
+    const storagePath = STORAGE_PATHS[role];


Loop over all roles but users only defines ProxyAdmin

Object.values(Role) yields four values (proxy_admin, proxy_admin_viewer, internal_user, internal_user_viewer), but users.ts only contains an entry for Role.ProxyAdmin. For every other role the destructure on line 9 (const { email, password } = users[role]) hits undefined and throws TypeError: Cannot destructure property 'email' of undefined, crashing global setup before any test has a chance to run.

greptile-apps · 2026-04-08T18:56:27Z

+PGPASSWORD="$DB_PASS" psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" \
+  -f "$SCRIPT_DIR/fixtures/seed.sql"


seed.sql is referenced but does not exist

run_e2e.sh and the CircleCI ui_e2e_tests job both attempt to run ui/litellm-dashboard/e2e_tests/fixtures/seed.sql, but no such file is present in the repository. The seeding step will fail with a "file not found" error, leaving the database empty. All tests that rely on pre-seeded teams, keys, or users (i.e. the majority of the 10 test cases) will then fail against the missing data.

greptile-apps · 2026-04-08T18:56:28Z

+async function clickTeamId(page: import("@playwright/test").Page, teamId: string) {
+  const idPrefix = teamId.slice(0, 7);


Unused idPrefix variable

idPrefix is computed but never referenced in the function body — only teamId itself is used in the locator filter.

Suggested change

async function clickTeamId(page: import("@playwright/test").Page, teamId: string) {

const idPrefix = teamId.slice(0, 7);

async function clickTeamId(page: import("@playwright/test").Page, teamId: string) {

- Add constants.ts with all required exports (key aliases, team IDs) - Add fixtures/users.ts with all role definitions and storage paths - Add fixtures/seed.sql for deterministic test database seeding - Remove Firefox project from playwright config (only Chromium installed) - Remove unused variable in teams.spec.ts - Rename CircleCI job to e2e_ui_testing

codecov · 2026-04-08T19:44:12Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

…finition

…ork in CI

greptile-apps · 2026-04-08T20:29:24Z

              only:
                - main
                - /litellm_.*/
-      # - e2e_ui_testing:
-      #     name: e2e_ui_testing_chromium
-      #     browser: chromium
-      #     context: e2e_ui_tests
-      #     requires:
-      #       - ui_build
-      #       - build_docker_database_image
-      #       - prisma_schema_sync
-      #     filters:
-      #       branches:
-      #         only:
-      #           - main
-      #           - /litellm_.*/
-      # - e2e_ui_testing:
-      #     name: e2e_ui_testing_firefox
-      #     browser: firefox
-      #     context: e2e_ui_tests
-      #     requires:
-      #       - ui_build
-      #       - build_docker_database_image
-      #       - prisma_schema_sync
-      #     filters:
-      #       branches:
-      #         only:
-      #           - main
-      #           - /litellm_.*/
+      - e2e_ui_testing:
+          filters:
+            branches:
+              only:
+                - main
+                - /litellm_.*/


Missing context causes DockerHub auth to fail

The new e2e_ui_testing workflow entry omits the context: key that the old (commented-out) entries used (context: e2e_ui_tests). The job definition's auth block references ${DOCKERHUB_USERNAME} and ${DOCKERHUB_PASSWORD}, but without a context those variables are never injected, so the credentials resolve to empty strings. CircleCI will then attempt an unauthenticated pull and hit Docker Hub's rate limit or an auth error, causing every run on main/litellm_* to fail before any test executes.

Suggested change

only:

- main

- /litellm_.*/

# - e2e_ui_testing:

# name: e2e_ui_testing_chromium

# browser: chromium

# context: e2e_ui_tests

# requires:

# - ui_build

# - build_docker_database_image

# - prisma_schema_sync

# filters:

# branches:

# only:

# - main

# - /litellm_.*/

# - e2e_ui_testing:

# name: e2e_ui_testing_firefox

# browser: firefox

# context: e2e_ui_tests

# requires:

# - ui_build

# - build_docker_database_image

# - prisma_schema_sync

# filters:

# branches:

# only:

# - main

# - /litellm_.*/

- e2e_ui_testing:

filters:

branches:

only:

- main

- /litellm_.*/

- e2e_ui_testing:

context: e2e_ui_tests

filters:

branches:

only:

- main

- /litellm_.*/

Resolved conflicts: - streaming_handler.py: combined role check (PR #24354, Azure streaming) with reasoning_items check (new in main) — both are independent OR conditions in is_chunk_non_empty() - CI/CD: accepted main's versions throughout - Redis tests migrated to CircleCI (PR #25354): removed enable-redis from GH Actions workflows - E2E UI tests restructured (PR #25365): simplified CircleCI job - Coverage via Codecov added to all GH Actions unit test workflows - Deleted test-litellm-matrix.yml and test-proxy-e2e-azure-batches.yml (removed in main)

[Feature] UI E2E Tests: Proxy Admin Team and Key Management

Resolved conflicts: - streaming_handler.py: combined role check (PR BerriAI#24354, Azure streaming) with reasoning_items check (new in main) — both are independent OR conditions in is_chunk_non_empty() - CI/CD: accepted main's versions throughout - Redis tests migrated to CircleCI (PR BerriAI#25354): removed enable-redis from GH Actions workflows - E2E UI tests restructured (PR BerriAI#25365): simplified CircleCI job - Coverage via Codecov added to all GH Actions unit test workflows - Deleted test-litellm-matrix.yml and test-proxy-e2e-azure-batches.yml (removed in main)

yuneng-berri temporarily deployed to integration-redis-postgres April 8, 2026 18:51 — with GitHub Actions Inactive

yuneng-berri requested a review from a team April 8, 2026 18:51

vercel Bot deployed to Preview April 8, 2026 18:52 View deployment

github-advanced-security Bot found potential problems Apr 8, 2026

View reviewed changes

Comment thread ui/litellm-dashboard/e2e_tests/globalSetup.ts Fixed

Comment thread ui/litellm-dashboard/e2e_tests/tests/proxy-admin/teams.spec.ts Fixed

greptile-apps Bot reviewed Apr 8, 2026

View reviewed changes

yuneng-berri temporarily deployed to integration-postgres April 8, 2026 19:40 — with GitHub Actions Inactive

yuneng-berri temporarily deployed to integration-redis-postgres April 8, 2026 19:41 — with GitHub Actions Inactive

yuneng-berri had a problem deploying to integration-postgres April 8, 2026 19:41 — with GitHub Actions Error

yuneng-berri temporarily deployed to integration-postgres April 8, 2026 19:41 — with GitHub Actions Inactive

vercel Bot deployed to Preview April 8, 2026 19:42 View deployment

[Fix] Rename CI job to e2e_ui_testing and remove duplicate old job de…

ac9ebdf

…finition

yuneng-berri temporarily deployed to integration-postgres April 8, 2026 20:17 — with GitHub Actions Inactive

yuneng-berri temporarily deployed to integration-redis-postgres April 8, 2026 20:17 — with GitHub Actions Inactive

yuneng-berri had a problem deploying to integration-postgres April 8, 2026 20:17 — with GitHub Actions Error

vercel Bot deployed to Preview April 8, 2026 20:19 View deployment

[Fix] Restructure HTML files after UI build so extensionless routes w…

4ee7d42

…ork in CI

yuneng-berri temporarily deployed to integration-redis-postgres April 8, 2026 20:24 — with GitHub Actions Inactive

yuneng-berri temporarily deployed to integration-postgres April 8, 2026 20:24 — with GitHub Actions Inactive

yuneng-berri had a problem deploying to integration-postgres April 8, 2026 20:25 — with GitHub Actions Error

yuneng-berri temporarily deployed to integration-postgres April 8, 2026 20:25 — with GitHub Actions Inactive

yuneng-berri had a problem deploying to integration-postgres April 8, 2026 20:25 — with GitHub Actions Error

vercel Bot deployed to Preview April 8, 2026 20:26 View deployment

greptile-apps Bot reviewed Apr 8, 2026

View reviewed changes

[Fix] Remove old broken key tests superseded by proxy-admin/keys.spec.ts

467dbc4

yuneng-berri temporarily deployed to integration-postgres April 8, 2026 20:32 — with GitHub Actions Inactive

yuneng-berri temporarily deployed to integration-redis-postgres April 8, 2026 20:32 — with GitHub Actions Inactive

yuneng-berri temporarily deployed to integration-postgres April 8, 2026 20:33 — with GitHub Actions Inactive

yuneng-berri had a problem deploying to integration-postgres April 8, 2026 20:33 — with GitHub Actions Error

yuneng-berri temporarily deployed to integration-postgres April 8, 2026 20:33 — with GitHub Actions Inactive

vercel Bot deployed to Preview April 8, 2026 20:34 View deployment

yuneng-berri requested a review from ryan-crabbe-berri April 8, 2026 22:12

ryan-crabbe-berri approved these changes Apr 8, 2026

View reviewed changes

yuneng-berri enabled auto-merge April 8, 2026 22:23

shin-berri self-requested a review April 8, 2026 22:28

shin-berri approved these changes Apr 8, 2026

View reviewed changes

yuneng-berri merged commit 072d410 into main Apr 8, 2026
104 of 108 checks passed

yuneng-berri deleted the litellm_e2e_ui_tests branch April 8, 2026 22:29

avianion mentioned this pull request Apr 8, 2026

feat: add Avian as a new provider #21905

Closed

5 tasks

fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026

Merge pull request BerriAI#25365 from BerriAI/litellm_e2e_ui_tests

36de574

[Feature] UI E2E Tests: Proxy Admin Team and Key Management

Navigation Menu

Uh oh!

Uh oh!

[Feature] UI E2E Tests: Proxy Admin Team and Key Management - #25365

[Feature] UI E2E Tests: Proxy Admin Team and Key Management#25365
yuneng-berri merged 5 commits into
mainfrom
litellm_e2e_ui_tests

yuneng-berri commented Apr 8, 2026

Uh oh!

vercel Bot commented Apr 8, 2026 •

edited

Loading

Uh oh!

codspeed-hq Bot commented Apr 8, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot commented Apr 8, 2026 •

edited

Loading

Vulnerabilities

Important Files Changed

Uh oh!

greptile-apps Bot Apr 8, 2026

Uh oh!

greptile-apps Bot Apr 8, 2026

Uh oh!

greptile-apps Bot Apr 8, 2026

Uh oh!

greptile-apps Bot Apr 8, 2026

Uh oh!

greptile-apps Bot Apr 8, 2026

Uh oh!

codecov Bot commented Apr 8, 2026

Uh oh!

greptile-apps Bot Apr 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		PGPASSWORD="$DB_PASS" psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" \
		-f "$SCRIPT_DIR/fixtures/seed.sql"

		async function clickTeamId(page: import("@playwright/test").Page, teamId: string) {
		const idPrefix = teamId.slice(0, 7);

Uh oh!

Uh oh!

Conversation

yuneng-berri commented Apr 8, 2026

Summary

Testing

Type

Uh oh!

vercel Bot commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codspeed-hq Bot commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Vulnerabilities

Important Files Changed

Uh oh!

greptile-apps Bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Apr 8, 2026

Codecov Report

Uh oh!

greptile-apps Bot Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

vercel Bot commented Apr 8, 2026 •

edited

Loading

codspeed-hq Bot commented Apr 8, 2026 •

edited

Loading

greptile-apps Bot commented Apr 8, 2026 •

edited

Loading