Skip to content

fix(e2e): make ui login readiness robust to httpOnly token cookies - #33564

Merged
mubashir1osmani merged 1 commit into
litellm_e2e_stagingfrom
litellm_e2e_ui_login_readiness
Jul 16, 2026
Merged

fix(e2e): make ui login readiness robust to httpOnly token cookies#33564
mubashir1osmani merged 1 commit into
litellm_e2e_stagingfrom
litellm_e2e_ui_login_readiness

Conversation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Relevant issues

Linear ticket

Resolves LIT-4484

Pre-Submission checklist

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Follow-up hardening on top of #33562 (already merged into litellm_e2e_staging). Repro against the compose stack in tests/e2e/ with ghcr.io/berriai/litellm:main-latest; the four browser tests need no LLM traffic

Ran the file 3 times back to back at this branch's HEAD, 4 passed every time:

============================== 4 passed in 6.96s ===============================
============================== 4 passed in 6.79s ===============================
============================== 4 passed in 6.90s ===============================

Type

🐛 Bug Fix
✅ Test

Changes

#33562 fixed the ui_page login for the react admin UI and used the token cookie on document.cookie as the login-complete signal. That signal only holds while loginCall sets the cookie from JavaScript; document.cookie returns empty for an httpOnly cookie, so if the server ever switches to delivering token via a Set-Cookie header the wait_for_function would spin to the 30s timeout and silently reproduce the original hang with a different cause

This accepts a second, delivery-agnostic readiness signal: the login form detaching (#username gone once the post-login redirect lands). Readiness now holds whether the cookie is JS-set or httpOnly

page.click('button[type="submit"]')
page.wait_for_function(
    "() => document.cookie.includes('token=') || !document.querySelector('#username')"
)

QA runbook

All four cases share the ui_page fixture; this PR only changes how the fixture decides login is complete, so the manual check is the login plus each test's dropdown assertion. Prereqs: compose stack up from tests/e2e/ on an image whose bundled UI has #32382; uv sync --inexact --group e2e-dev && uv run playwright install chromium

  • tests/e2e/management/test_key_models_dropdown_e2e.py::TestKeyModelsDropdownUI::test_create_teamless_key_offers_proxy_scope_and_persists - a teamless key's create Models dropdown offers All Proxy Models but not All Team Models, and persists as all-proxy-models
    • Open http://localhost:4000/ui/, log in as admin / your master key, confirm you land on the dashboard rather than bouncing back to the login card
    • Open http://localhost:4000/ui/api-keys/?create=true, open the Models dropdown, expect gpt-5.5 and All Proxy Models present and All Team Models absent
    • Pick All Proxy Models, name and create the key, then GET /key/info for it and expect models == ["all-proxy-models"] with no team
    • Sanity check: this test makes sense to add and is not hand-wavey or potentially flaky
  • tests/e2e/management/test_key_models_dropdown_e2e.py::TestKeyModelsDropdownUI::test_create_team_key_offers_team_scope_and_persists - a team key's create dropdown offers All Team Models plus the team's own model but never the proxy sentinel, and persists as all-team-models
    • With a team whose models are ["all-proxy-models", "gpt-5.5"], open the create modal and select that team
    • Open the Models dropdown, expect All Team Models and gpt-5.5 present and neither All Proxy Models nor the raw all-proxy-models present
    • Pick All Team Models, create, then GET /key/info and expect models == ["all-team-models"] on that team
    • Sanity check: this test makes sense to add and is not hand-wavey or potentially flaky
  • tests/e2e/management/test_key_models_dropdown_e2e.py::TestKeyModelsDropdownUI::test_edit_teamless_key_offers_proxy_scope - editing a teamless key shows All Proxy Models but not All Team Models
    • Create a teamless key with models ["gpt-5.5"], open it on the api-keys page, Settings tab, Edit Settings
    • Open the Models dropdown and expect All Proxy Models present and All Team Models absent
    • Sanity check: this test makes sense to add and is not hand-wavey or potentially flaky
  • tests/e2e/management/test_key_models_dropdown_e2e.py::TestKeyModelsDropdownUI::test_edit_team_key_offers_team_scope_only - editing a team key shows All Team Models plus the team model but never the proxy sentinel
    • Create a key on a team whose models are ["all-proxy-models", "gpt-5.5"], open it, Settings tab, Edit Settings
    • Open the Models dropdown and expect All Team Models and gpt-5.5 present and neither All Proxy Models nor raw all-proxy-models present
    • Sanity check: this test makes sense to add and is not hand-wavey or potentially flaky

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/5d5f93d9ba36469b910db0271edb31f2

The login readiness check waited only on document.cookie including token=, which is empty when the token cookie is httpOnly. If the server ever sets it via a Set-Cookie header, the wait would spin to the 30s timeout and silently reproduce the original hang. Also accept the login form detaching (#username gone after the post-login redirect) so readiness holds regardless of how the cookie is delivered.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Mubashir Osmani seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps

greptile-apps Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR hardens the ui_page e2e fixture so that login-complete detection works regardless of whether the token cookie is set from JavaScript or via an httpOnly Set-Cookie header. The original single-condition wait (document.cookie.includes('token=')) would spin to timeout if the server-side ever delivers the cookie as httpOnly, since document.cookie does not expose those values.

  • The readiness condition is extended to document.cookie.includes('token=') || !document.querySelector('#username'), so an httpOnly token delivery still resolves as soon as the post-login redirect causes the login form to unmount.
  • The fix is confined to the shared ui_page session fixture in tests/e2e/management/conftest.py and carries no production-code changes.

Confidence Score: 4/5

Safe to merge; the change is test-only, isolated to the login fixture, and corrects a real gap in the readiness signal.

The fix correctly addresses the httpOnly cookie blind spot. The one open question is whether the #username disappearance check could silently accept an error-page redirect as a successful login, leaving every subsequent test in a confusing broken state rather than producing a clear login-failure error. This is unlikely in practice given the application's routing behavior, but a lightweight post-wait URL or dashboard-selector assertion would remove the ambiguity entirely.

tests/e2e/management/conftest.py — the ui_page fixture's new second branch lacks a post-login state guard.

Important Files Changed

Filename Overview
tests/e2e/management/conftest.py Login readiness condition widened from cookie-only to cookie-or-form-detach; no post-login state assertion guards against false positives on error pages.

Reviews (1): Last reviewed commit: "fix(e2e): make ui login readiness robust..." | Re-trigger Greptile

Comment on lines +54 to +56
page.wait_for_function(
"() => document.cookie.includes('token=') || !document.querySelector('#username')"
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 No post-login assertion after #username disappears

The !document.querySelector('#username') branch resolves the wait as soon as the login form unmounts — but it does not verify that the resulting page is the dashboard rather than an error page (e.g., a 500 or a redirect back to /ui/login with a different form structure). If a misconfigured or unreachable backend causes the login to fail with a page that happens not to contain #username, every downstream test in the session will receive a page in a broken state and produce confusing assertion failures instead of a clear "fixture login failed" diagnostic. A lightweight guard after the wait_for_function — checking that the final URL contains /ui/ or that a known dashboard selector is present — would make the failure mode much more actionable.

@codecov

codecov Bot commented Jul 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@mubashir1osmani
mubashir1osmani merged commit 0947a65 into litellm_e2e_staging Jul 16, 2026
92 of 95 checks passed
@mubashir1osmani
mubashir1osmani deleted the litellm_e2e_ui_login_readiness branch July 16, 2026 18:42
mubashir1osmani added a commit that referenced this pull request Jul 16, 2026
* chore(e2e): establish litellm_e2e_staging integration line

Long-lived berri branch for e2e suite recovery work (LIT-4479 through LIT-4486) before merge to litellm_internal_staging

* test(e2e): remove langfuse_otel logging e2e suite (#33558)

* test(e2e): remove langfuse_otel logging e2e suite

Removes the LIT-4483 dynamic per-team/key/org langfuse_otel logging e2e tests (tests/e2e/logging/test_langfuse_e2e.py, added in #32857). The shared logging_client harness and the langfuse coverage-registry cells are left in place; only the test module is removed. The otel and prometheus logging e2e suites are unaffected.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(e2e): drop orphaned langfuse coverage-registry cells

The three logging.langfuse.*.logs_spend P0 cells were only exercised by the deleted langfuse_otel e2e suite. Remove them so the coverage registry has no orphaned rows.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(e2e): log into the react admin ui in the management browser fixture (#33562)

The management ui_page fixture drove the old server-rendered login form: it clicked input[type="submit"] and treated wait_for_url("**/ui/**") as the done signal. /ui/ now serves the react (antd) dashboard whose submit is a <button type="submit">, so the click waited out the full 30s timeout and errored every browser test in the suite. wait_for_url also matched instantly because the login page already lives at /ui/, so on the fast path the fixture navigated before the auth cookie landed and got bounced back to login.

Click the antd submit button and wait for the token cookie loginCall sets on document.cookie, the real post-login signal.

Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(e2e): make ui login readiness robust to httpOnly token cookies (#33564)

The login readiness check waited only on document.cookie including token=, which is empty when the token cookie is httpOnly. If the server ever sets it via a Set-Cookie header, the wait would spin to the 30s timeout and silently reproduce the original hang. Also accept the login form detaching (#username gone after the post-login redirect) so readiness holds regardless of how the cookie is delivered.

Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants