Skip to content
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ jobs:
env:
E2E_SCREENSHOT_DIR: ${{ runner.temp }}/e2e-screenshots
E2E_GITHUB_PASSWORD: ${{ secrets.E2E_GITHUB_PASSWORD }}
E2E_GITHUB_TOTP_SECRET: ${{ secrets.E2E_GITHUB_TOTP_SECRET }}
Comment thread
ggallen marked this conversation as resolved.
Resolved
Comment thread
ggallen marked this conversation as resolved.
Resolved
Comment thread
ggallen marked this conversation as resolved.
Resolved
Comment thread
ggallen marked this conversation as resolved.
Resolved
Comment thread
ggallen marked this conversation as resolved.
Resolved

- name: Upload debug screenshots
if: always() && steps.secrets-check.outputs.available == 'true'
Expand Down
1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ The e2e tests require GitHub credentials. There are three ways to provide them:
- **`E2E_GITHUB_PASSWORD` env var:** Set directly with the password.
- **`E2E_GITHUB_PASSWORD_FILE` env var:** Set to a file path containing the password (used in devaipod environments where secrets are mounted as files).
- **`E2E_GITHUB_SESSION_FILE` env var:** Set to a pre-exported Playwright session file (skips login).
- **`E2E_GITHUB_TOTP_SECRET` env var:** Optional. The TOTP secret (base32) for the test account's 2FA. Required only when the test account has 2FA enabled — used during session export and sudo confirmation.

If only `E2E_GITHUB_USERNAME` and a password source are available, `make e2e-test` will automatically generate a session file before running tests. See `make help` for all available targets.

Expand Down
13 changes: 8 additions & 5 deletions docs/ADRs/0010-stored-session-for-e2e-browser-auth.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ Date: 2026-04-03

Accepted

Extended by [ADR 0039](0039-totp-automation-for-e2e-2fa.md).

## Context

The admin CLI e2e tests use Playwright to automate browser interactions with
Expand Down Expand Up @@ -86,8 +88,8 @@ The e2e tests need both because:
baked into the stored session).

The `handleSudoIfPresent()` function detects the "Confirm access" page by
title and enters the password automatically. It is called before PAT creation
(both classic and fine-grained).
title and enters either the password or a TOTP code (see [ADR 0039](0039-totp-automation-for-e2e-2fa.md)).
It is called before PAT creation (both classic and fine-grained).

**Local development:** When `E2E_GITHUB_SESSION_FILE` is not set but
`E2E_GITHUB_USERNAME` and `E2E_GITHUB_PASSWORD` are, `make e2e-test`
Expand Down Expand Up @@ -131,12 +133,13 @@ it does expire, a developer runs `make e2e-upload-session` to refresh it.

- Two repo secrets are required in CI: `E2E_GITHUB_SESSION` (base64-encoded
storageState JSON for login bypass) and `E2E_GITHUB_PASSWORD` (for sudo
confirmation on sensitive pages).
confirmation on sensitive pages). A third secret, `E2E_GITHUB_TOTP_SECRET`,
is required when the test account has 2FA enabled (see [ADR 0039](0039-totp-automation-for-e2e-2fa.md)).
- If the test account's password changes, the stored session must be
re-exported (password change invalidates all sessions) and the
`E2E_GITHUB_PASSWORD` secret must be updated.
- If the test account enables 2FA, the session export must happen after the 2FA
step.
- If the test account has 2FA enabled, set `E2E_GITHUB_TOTP_SECRET` so
`make e2e-export-session` and sudo confirmation can handle TOTP automatically.
- The login function becomes a session-loading function -- simpler and more
reliable.
- Session refresh is `make e2e-upload-session`, expected to be needed at most
Expand Down
56 changes: 56 additions & 0 deletions docs/ADRs/0039-totp-automation-for-e2e-2fa.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
---
title: "39. TOTP automation for e2e 2FA"
status: Accepted
relates_to:
- testing-agents
topics:
- e2e
- ci
- authentication
- 2fa
---

# 39. TOTP automation for e2e 2FA

Date: 2026-05-19

## Status

Accepted

Extends [ADR 0010](0010-stored-session-for-e2e-browser-auth.md).

## Context

[ADR 0010](0010-stored-session-for-e2e-browser-auth.md) established stored
Playwright sessions for e2e CI authentication. When that ADR was written, the
test account did not have 2FA enabled. With 2FA on, two new authentication
prompts appear that the existing infrastructure could not handle:

1. A TOTP challenge during `make e2e-export-session` (local login).
2. A TOTP challenge on sudo pages in CI (GitHub may present TOTP instead of
a password field for 2FA-enabled accounts).

## Decision

Automate TOTP entry in Playwright using a shared `e2e/internal/otp` package
that wraps `pquerna/otp` to generate time-based codes. The TOTP secret is
supplied via the `E2E_GITHUB_TOTP_SECRET` environment variable (base32-encoded).

- `handleTOTPIfPresent()` in the login helpers detects TOTP form fields and
enters the current code. `handleSudoIfPresent()` now handles both password
and TOTP sudo prompts.
- The `export-session` command detects and completes the 2FA prompt after
password login.
- `E2E_GITHUB_TOTP_SECRET` is optional — omitting it preserves the existing
non-2FA flow.

## Consequences

- Three repo secrets are now required in CI when the test account has 2FA:
`E2E_GITHUB_SESSION`, `E2E_GITHUB_PASSWORD`, and `E2E_GITHUB_TOTP_SECRET`.
- Session export and sudo confirmation work automatically for 2FA accounts.
- The TOTP secret is a long-lived credential that must be protected like the
password.
- If GitHub changes its TOTP form markup, the Playwright selectors will need
updating.
2 changes: 1 addition & 1 deletion e2e/admin/admin_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ func setupE2ETest(t *testing.T) *e2eEnv {
// Generate a PAT for API access.
patNote := fmt.Sprintf("fullsend-e2e-%d", time.Now().Unix())
t.Logf("Creating PAT: %s", patNote)
token, err := createPAT(page, patNote, cfg.password, screenshotDir, t.Logf)
token, err := createPAT(page, patNote, cfg.password, cfg.totpSecret, screenshotDir, t.Logf)
require.NoError(t, err, "creating PAT")
t.Cleanup(func() {
t.Log("Deleting PAT...")
Expand Down
25 changes: 25 additions & 0 deletions e2e/admin/browser.go
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ func (b *PlaywrightBrowserOpener) Open(_ context.Context, url string) error {
case strings.Contains(pageURL, "/settings/apps/new"),
strings.Contains(pageURL, "/settings/apps/manifest"):
return b.handleCreateAppPage()
case strings.Contains(pageURL, "/installations/select_target"):
Comment thread
ggallen marked this conversation as resolved.
Resolved
return b.handleSelectTargetPage()
case strings.Contains(pageURL, "/installations/new"):
return b.handleInstallAppPage()
default:
Expand Down Expand Up @@ -204,6 +206,29 @@ func (b *PlaywrightBrowserOpener) handleCreateAppPage() error {
return nil
}

// handleSelectTargetPage selects the target organization on GitHub's
// "select installation target" page, then proceeds to the install page.
func (b *PlaywrightBrowserOpener) handleSelectTargetPage() error {
b.logf("[browser] handleSelectTargetPage at URL: %s", b.page.URL())

orgLink := b.page.Locator(fmt.Sprintf("a:has-text('%s')", testOrg))
if err := orgLink.First().Click(playwright.LocatorClickOptions{
Timeout: playwright.Float(5000),
}); err != nil {
saveDebugScreenshot(b.page, b.screenshotDir, "browser-select-target-failed", b.logf)
return fmt.Errorf("selecting target org %s: %w", testOrg, err)
}

if err := b.page.WaitForLoadState(playwright.PageWaitForLoadStateOptions{
State: playwright.LoadStateDomcontentloaded,
}); err != nil {
return fmt.Errorf("waiting for install page after selecting org: %w", err)
}

b.logf("[browser] Selected org %s, now at: %s", testOrg, b.page.URL())
return b.handleInstallAppPage()
}

// handleInstallAppPage clicks "Install" on the GitHub App installation page.
// Retries navigation if the page 404s (GitHub needs time to provision the app).
func (b *PlaywrightBrowserOpener) handleInstallAppPage() error {
Expand Down
110 changes: 79 additions & 31 deletions e2e/admin/login.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"path/filepath"
"strings"

"github.com/fullsend-ai/fullsend/e2e/internal/otp"
"github.com/playwright-community/playwright-go"
)

Expand Down Expand Up @@ -34,37 +35,63 @@ func verifyGitHubSession(page playwright.Page, screenshotDir string, logf func(s
}

// handleSudoIfPresent detects GitHub's "Confirm access" sudo page and
// enters the password to proceed. GitHub requires sudo confirmation when
// accessing sensitive settings pages (token management, app settings)
// even with a valid session. Returns true if sudo was handled.
func handleSudoIfPresent(page playwright.Page, password, screenshotDir string, logf func(string, ...any)) (bool, error) {
// enters the password (or TOTP code if 2FA is enabled) to proceed.
// GitHub requires sudo confirmation when accessing sensitive settings pages
// (token management, app settings) even with a valid session.
// Returns true if sudo was handled.
func handleSudoIfPresent(page playwright.Page, password, totpSecret, screenshotDir string, logf func(string, ...any)) (bool, error) {
pageTitle, _ := page.Title()
if !strings.Contains(pageTitle, "Confirm access") && !strings.Contains(pageTitle, "Sudo") {
return false, nil
}

logf("[sudo] Detected sudo confirmation page (title: %s)", pageTitle)

if password == "" {
saveDebugScreenshot(page, screenshotDir, "sudo-no-password", logf)
return false, fmt.Errorf("sudo confirmation required but no password available — set E2E_GITHUB_PASSWORD")
}

// GitHub's sudo form uses #sudo_password for the password field.
// GitHub may show a password field or a TOTP field (or both with a toggle).
Comment thread
ggallen marked this conversation as resolved.
Resolved
// Try password first, fall back to TOTP.
passwordInput := page.Locator("#sudo_password")
if err := passwordInput.WaitFor(playwright.LocatorWaitForOptions{
passwordVisible := passwordInput.WaitFor(playwright.LocatorWaitForOptions{
State: playwright.WaitForSelectorStateVisible,
Timeout: playwright.Float(5000),
}); err != nil {
saveDebugScreenshot(page, screenshotDir, "sudo-password-field-missing", logf)
return false, fmt.Errorf("sudo password field not found: %w", err)
Timeout: playwright.Float(2000),
}) == nil
Comment thread
ggallen marked this conversation as resolved.
Resolved

if passwordVisible && password != "" {
if err := passwordInput.Fill(password); err != nil {
return false, fmt.Errorf("filling sudo password: %w", err)
}
} else if passwordVisible && totpSecret != "" {
if handled, err := handleTOTPIfPresent(page, totpSecret, screenshotDir, logf); err != nil {
return false, fmt.Errorf("TOTP on sudo page (password field present but empty): %w", err)
} else if handled {
if err := waitForPageToLeave(page, "Confirm access", "Sudo"); err != nil {
saveDebugScreenshot(page, screenshotDir, "sudo-totp-still-on-page", logf)
return false, err
}
return true, nil
}
saveDebugScreenshot(page, screenshotDir, "sudo-password-not-set", logf)
return false, fmt.Errorf("sudo page shows password field but neither password nor TOTP succeeded")
} else if passwordVisible {
Comment thread
ggallen marked this conversation as resolved.
Resolved
saveDebugScreenshot(page, screenshotDir, "sudo-password-not-set", logf)
Comment thread
ggallen marked this conversation as resolved.
Resolved
return false, fmt.Errorf("sudo page shows password field but E2E_GITHUB_PASSWORD is not set")
} else if totpSecret != "" {
if handled, err := handleTOTPIfPresent(page, totpSecret, screenshotDir, logf); err != nil {
return false, fmt.Errorf("TOTP on sudo page: %w", err)
} else if !handled {
saveDebugScreenshot(page, screenshotDir, "sudo-no-auth-method", logf)
return false, fmt.Errorf("sudo page has no visible password or TOTP field")
}
Comment thread
ggallen marked this conversation as resolved.
Resolved
if err := waitForPageToLeave(page, "Confirm access", "Sudo"); err != nil {
saveDebugScreenshot(page, screenshotDir, "sudo-totp-still-on-page", logf)
return false, err
}
return true, nil
} else {
saveDebugScreenshot(page, screenshotDir, "sudo-no-credentials", logf)
return false, fmt.Errorf("sudo confirmation required but no password or TOTP secret available — set E2E_GITHUB_PASSWORD or E2E_GITHUB_TOTP_SECRET")
}

if err := passwordInput.Fill(password); err != nil {
return false, fmt.Errorf("filling sudo password: %w", err)
}

// Click the confirm button.
// Click the confirm button (password path).
confirmBtn := page.Locator("button[type='submit']:has-text('Confirm'), button[type='submit']:has-text('Confirm password'), button[type='submit']")
if err := confirmBtn.First().Click(playwright.LocatorClickOptions{
Timeout: playwright.Float(5000),
Expand All @@ -73,24 +100,45 @@ func handleSudoIfPresent(page playwright.Page, password, screenshotDir string, l
return false, fmt.Errorf("clicking sudo confirm button: %w", err)
}

// Wait for the page to navigate away from the sudo page.
if err := page.WaitForLoadState(playwright.PageWaitForLoadStateOptions{
State: playwright.LoadStateDomcontentloaded,
}); err != nil {
return false, fmt.Errorf("waiting for post-sudo navigation: %w", err)
}

// Verify we're past sudo.
newTitle, _ := page.Title()
if strings.Contains(newTitle, "Confirm access") || strings.Contains(newTitle, "Sudo") {
if err := waitForPageToLeave(page, "Confirm access", "Sudo"); err != nil {
saveDebugScreenshot(page, screenshotDir, "sudo-still-on-page", logf)
return false, fmt.Errorf("sudo confirmation failed — still on confirmation page (title: %s)", newTitle)
return false, err
}

logf("[sudo] Sudo confirmation succeeded")
return true, nil
}

// handleTOTPIfPresent detects a GitHub 2FA/TOTP input on the current page
// and fills in a generated code. Works on both the post-login 2FA page
// (/sessions/two-factor) and the sudo TOTP prompt. Returns true if a TOTP
// form was found and submitted.
func handleTOTPIfPresent(page playwright.Page, totpSecret, screenshotDir string, logf func(string, ...any)) (bool, error) {
handled, err := otp.EnterTOTPCode(page, totpSecret, logf)
if err != nil {
saveDebugScreenshot(page, screenshotDir, "totp-failed", logf)
}
return handled, err
}

// waitForPageToLeave waits until the page title no longer contains any of
// the given substrings, or until the timeout (10s) is reached.
func waitForPageToLeave(page playwright.Page, titleSubstrings ...string) error {
checks := make([]string, len(titleSubstrings))
for i, sub := range titleSubstrings {
checks[i] = fmt.Sprintf("!document.title.includes(%q)", sub)
}
jsExpr := "() => " + strings.Join(checks, " && ")
_, err := page.WaitForFunction(jsExpr, nil, playwright.PageWaitForFunctionOptions{
Timeout: playwright.Float(10000),
})
if err != nil {
title, _ := page.Title()
return fmt.Errorf("still on page after 10s (title: %s)", title)
}
return nil
}

// saveDebugScreenshot saves a screenshot to dir for debugging.
func saveDebugScreenshot(page playwright.Page, dir, name string, logf func(string, ...any)) {
path := filepath.Join(dir, fmt.Sprintf("e2e-debug-%s.png", name))
Expand Down
4 changes: 2 additions & 2 deletions e2e/admin/pat.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ var patScopes = []string{
// createPAT creates a classic GitHub Personal Access Token via the browser.
// The token is created with a 7-day expiry and the scopes needed for e2e tests.
// Returns the token string.
func createPAT(page playwright.Page, note, password, screenshotDir string, logf func(string, ...any)) (string, error) {
func createPAT(page playwright.Page, note, password, totpSecret, screenshotDir string, logf func(string, ...any)) (string, error) {
url := "https://github.com/settings/tokens/new"
if _, err := page.Goto(url, playwright.PageGotoOptions{
WaitUntil: playwright.WaitUntilStateDomcontentloaded,
Expand All @@ -39,7 +39,7 @@ func createPAT(page playwright.Page, note, password, screenshotDir string, logf
}

// Handle sudo confirmation if GitHub requires re-authentication.
if handled, err := handleSudoIfPresent(page, password, screenshotDir, logf); err != nil {
if handled, err := handleSudoIfPresent(page, password, totpSecret, screenshotDir, logf); err != nil {
return "", fmt.Errorf("sudo confirmation for PAT creation: %w", err)
} else if handled {
// After sudo, we may need to re-navigate to the token page.
Expand Down
3 changes: 3 additions & 0 deletions e2e/admin/testutil.go
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ var defaultRoles = []string{"fullsend", "triage", "coder", "review"}
type envConfig struct {
sessionFile string
password string
totpSecret string
lockTimeout time.Duration
}

Expand All @@ -62,6 +63,7 @@ func loadEnvConfig(t *testing.T) envConfig {
}

password := os.Getenv("E2E_GITHUB_PASSWORD")
totpSecret := os.Getenv("E2E_GITHUB_TOTP_SECRET")

lockTimeout := defaultLockTimeout
if v := os.Getenv("E2E_LOCK_TIMEOUT"); v != "" {
Expand All @@ -75,6 +77,7 @@ func loadEnvConfig(t *testing.T) envConfig {
return envConfig{
sessionFile: sessionFile,
password: password,
totpSecret: totpSecret,
lockTimeout: lockTimeout,
}
}
Expand Down
Loading
Loading