Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 36 additions & 3 deletions docs/guides/dev/e2e-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,9 +108,10 @@ Each pool org must be provisioned before e2e can use it:

1. Org exists with `botsend` as owner
2. `test-repo` and `e2e-lock` repos (lock created at runtime)
3. All role apps installed, including `fullsend-ai-e2e` with **Repository → Variables: Read and write** (`actions_variables`) and **Organization → Variables: Read and write** (`organization_actions_variables`)
4. `FULLSEND_FOREIGN_E2E_REPOS` includes `fullsend-ai/fullsend` with org-wide visibility (`visibility: all`)
5. Mint enrolled: org in `ALLOWED_ORGS`, `${ORG}/e2e` in `ROLE_APP_IDS`, e2e app PEM enrolled
3. Test actor permissions granted (see [Test actor permissions](#test-actor-permissions) below)
4. All role apps installed, including `fullsend-ai-e2e` with **Repository → Variables: Read and write** (`actions_variables`) and **Organization → Variables: Read and write** (`organization_actions_variables`)
5. `FULLSEND_FOREIGN_E2E_REPOS` includes `fullsend-ai/fullsend` with org-wide visibility (`visibility: all`)
6. Mint enrolled: org in `ALLOWED_ORGS`, `${ORG}/e2e` in `ROLE_APP_IDS`, e2e app PEM enrolled

Use the idempotent setup script:

Expand Down Expand Up @@ -142,6 +143,38 @@ go run ./cmd/fullsend admin foreign list --org halfsend-NN --repo target-repo
See [ADR 0083](../../ADRs/0083-repo-level-foreign-allow-list.md) for details
on repo-level foreign grants.

### Test actor permissions

Pool orgs grant three test actor accounts specific access levels for
e2e testing of permission-sensitive behaviour:

| Actor | Org membership | Repo permission on base `test-repo*` |
|-------|----------------|--------------------------------------|
| `fstest-write` | member | push (write) |
| `fstest-triage` | member | triage |
| `fstest-outsider` | none | public read only (no collaborator grant) |

Elevated access uses direct collaborator grants (not team membership). Fork repos
(`test-repo-fork`) are intentionally excluded — they are not base/enrolled
targets for permission grants.

The setup script (`hack/setup-new-e2e-org.sh`) creates or verifies this
model idempotently. To auto-accept org membership invitations, pass the
actor PATs as environment variables:

```bash
TEST_ACTOR_WRITE_PAT=ghp_... TEST_ACTOR_TRIAGE_PAT=ghp_... \
MINT_PROJECT=... MINT_FUNCTION=... hack/setup-new-e2e-org.sh 07
```

Without the PAT variables, the script pauses for manual acceptance.

Accounts and PATs are managed under
[#6024](https://github.com/fullsend-ai/fullsend/issues/6024). PATs are
stored as repository secrets `TEST_ACTOR_WRITE_PAT`,
`TEST_ACTOR_TRIAGE_PAT`, and `TEST_ACTOR_OUTSIDER_PAT` on
`fullsend-ai/fullsend`.

## CI authorization

Pull requests trigger e2e via `pull_request_target` in
Expand Down
115 changes: 115 additions & 0 deletions hack/setup-new-e2e-org.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@ set -euo pipefail
APP_SET="fullsend-ai"
ROLES=(fullsend triage coder review retro prioritize e2e)
BOT_USER="botsend"
TEST_WRITE_USER="fstest-write"
TEST_TRIAGE_USER="fstest-triage"
TEST_OUTSIDER_USER="fstest-outsider"

# open_browser tries to open a URL in the default browser.
open_browser() {
Expand Down Expand Up @@ -141,6 +144,118 @@ else
fi
echo

# --- 3b. test actor org membership ---
echo "==> Checking test actor org membership..."

for actor_info in "${TEST_WRITE_USER}:TEST_ACTOR_WRITE_PAT" "${TEST_TRIAGE_USER}:TEST_ACTOR_TRIAGE_PAT"; do
actor="${actor_info%%:*}"
pat_var="${actor_info##*:}"

membership_state=$(gh api "/orgs/${ORG}/memberships/${actor}" --jq '.state' 2>/dev/null || echo "none")

if [[ "${membership_state}" == "active" ]]; then
echo " OK: ${actor} is an active org member"
continue
fi

# Send or refresh the membership invitation.
if [[ "${membership_state}" != "pending" ]]; then
echo " Inviting ${actor} as org member..."
gh api "/orgs/${ORG}/memberships/${actor}" \
-X PUT \
-f role="member" \
--silent 2>/dev/null || true
fi

# Accept the invitation using the actor's PAT if available.
if [[ -n "${!pat_var:-}" ]]; then
echo " Accepting invitation for ${actor} using ${pat_var}..."
GH_TOKEN="${!pat_var}" gh api "/user/memberships/orgs/${ORG}" \
-X PATCH \
-f state="active" \
--silent 2>/dev/null || true

membership_state=$(gh api "/orgs/${ORG}/memberships/${actor}" --jq '.state' 2>/dev/null || echo "none")
if [[ "${membership_state}" == "active" ]]; then
echo " OK: ${actor} is now an active org member"
continue
fi
fi

echo " PENDING: ${actor} invitation sent but not yet accepted."
if [[ -z "${!pat_var:-}" ]]; then
echo " Set ${pat_var} env var to auto-accept, or accept manually."
fi
wait_for_user "Press Enter after ${actor} has accepted the invitation..."

membership_state=$(gh api "/orgs/${ORG}/memberships/${actor}" --jq '.state' 2>/dev/null || echo "none")
if [[ "${membership_state}" != "active" ]]; then
echo " ERROR: ${actor} is still not an active member (state: ${membership_state})."
exit 1
fi
echo " OK: ${actor} is now an active org member"
done

# Verify outsider has no org membership.
outsider_state=$(gh api "/orgs/${ORG}/memberships/${TEST_OUTSIDER_USER}" --jq '.state' 2>/dev/null || echo "none")
if [[ "${outsider_state}" == "none" ]]; then
echo " OK: ${TEST_OUTSIDER_USER} has no org membership"
else
echo " WARNING: ${TEST_OUTSIDER_USER} has org membership (state: ${outsider_state})."
echo " Remove from ${ORG} to preserve the outsider test model."
fi
echo

# --- 3c. test actor repo permissions ---
echo "==> Granting test actor collaborator permissions on base repos..."
if ! base_repos=$(gh api --paginate "/orgs/${ORG}/repos" \
--jq '.[] | select(.fork == false) | select(.name | startswith("test-repo")) | .name' \
2>&1); then
echo " WARNING: could not list repos for ${ORG}: ${base_repos}"
echo " Check authentication and permissions, then re-run."
base_repos=""
fi

if [[ -z "${base_repos}" ]]; then
echo " No base test-repo* repos found. Skipping collaborator grants."
echo " Re-run after test repos are created to apply permissions."
else
ok_count=0
fail_count=0
while IFS= read -r repo; do
# fstest-write → push
if gh api "/repos/${ORG}/${repo}/collaborators/${TEST_WRITE_USER}" \
-X PUT -f permission="push" --silent 2>/dev/null; then
ok_count=$((ok_count + 1))
else
echo " WARNING: ${TEST_WRITE_USER} → push on ${repo}"
fail_count=$((fail_count + 1))
fi

# fstest-triage → triage
if gh api "/repos/${ORG}/${repo}/collaborators/${TEST_TRIAGE_USER}" \
-X PUT -f permission="triage" --silent 2>/dev/null; then
ok_count=$((ok_count + 1))
else
echo " WARNING: ${TEST_TRIAGE_USER} → triage on ${repo}"
fail_count=$((fail_count + 1))
fi
done <<< "${base_repos}"

echo " Collaborator grants: OK=${ok_count} WARNING=${fail_count}"

# Verify outsider has no collaborator access on the first base repo.
first_repo=$(echo "${base_repos}" | head -1)
if gh api "/repos/${ORG}/${first_repo}/collaborators/${TEST_OUTSIDER_USER}" \
--silent 2>/dev/null; then
echo " WARNING: ${TEST_OUTSIDER_USER} is a collaborator on ${first_repo}."
echo " Remove to preserve the outsider test model."
else
echo " OK: ${TEST_OUTSIDER_USER} is not a collaborator on ${first_repo}"
fi
fi
echo

# --- 4. check app installations ---
echo "==> Checking app installations..."
org_id=$(gh api "/orgs/${ORG}" --jq '.id')
Expand Down
Loading