Repository navigation
feat: implement issue #1280 — Verify persona addressing teams in CI #1284
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,153 @@ | ||
| #!/usr/bin/env bash | ||
| set -euo pipefail | ||
| # verify-persona-teams.sh — LIVE persona addressing-team check (#1280). | ||
| # | ||
| # The persona standard (§4.1) requires a persona's `address.handle` to name an org | ||
| # TEAM with specific LIVE properties. The hermetic personas/validate-personas.py | ||
| # checks the handle SHAPE (team slug == id) but deliberately cannot reach the | ||
| # network. This script is the separate, network-touching half: for every | ||
| # personas/<id>/persona.yml that declares an `address` block, it verifies the live | ||
| # team named by `address.handle`: | ||
| # | ||
| # 1. the team EXISTS, | ||
| # 2. privacy == "closed" (secret teams are unmentionable), | ||
| # 3. notification_setting == "notifications_disabled" (routes a webhook, not a page). | ||
| # | ||
| # The slug==id invariant is already enforced hermetically by validate-personas.py | ||
| # and is NOT re-checked here — this check is about LIVE state only. A persona with | ||
| # no `address` block passes (address is optional unless the `mention` surface is on). | ||
| # | ||
| # FAIL CLOSED: any API/parse error is a failure, never a pass. Conflating "could | ||
| # not read the team" with "the team is fine" is the exact systemic bug this check | ||
| # exists to prevent (petry-projects/.github#755, finding 3). If the team cannot be | ||
| # read, the check fails. | ||
| # | ||
| # Requires: gh (authenticated with an ORG-scoped token — github.token cannot read | ||
| # teams; GH_PAT_WORKFLOWS is available in CI), jq, and python3 + PyYAML. Diagnostics | ||
| # use the `::error::persona team invalid: ...` style of validate-personas.py. | ||
| # | ||
| # Usage: verify-persona-teams.sh [personas_root] (default: personas) | ||
| # VPT_ORG — org whose teams are read (default: petry-projects) | ||
|
|
||
| VPT_ORG="${VPT_ORG:-petry-projects}" | ||
|
don-petry marked this conversation as resolved.
Outdated
|
||
|
|
||
| vpt_fail() { | ||
| echo "::error::persona team invalid: $*" >&2 | ||
| exit 1 | ||
| } | ||
|
don-petry marked this conversation as resolved.
don-petry marked this conversation as resolved.
|
||
|
|
||
| # vpt_list_addresses <root> — print a "<slug>\t<manifest>" line for every | ||
| # personas/<id>/persona.yml under <root> that declares an address.handle. Personas | ||
| # with no `address` block are omitted (they pass). A malformed handle (not an | ||
| # 'org/team-slug' string) is a hard error — printed as a `::error::` diagnostic and | ||
| # a non-zero exit, never a traceback. | ||
| vpt_list_addresses() { | ||
| local root="$1" | ||
| python3 - "$root" <<'PY' | ||
| import glob | ||
| import os | ||
| import sys | ||
|
|
||
| try: | ||
| import yaml | ||
| except ImportError: | ||
| print("::error::persona team invalid: PyYAML not installed (pip install pyyaml)", | ||
| file=sys.stderr) | ||
| sys.exit(1) | ||
|
Copilot marked this conversation as resolved.
|
||
|
|
||
| root = sys.argv[1] | ||
| for manifest in sorted(glob.glob(os.path.join(root, "*", "persona.yml"))): | ||
| try: | ||
| with open(manifest, encoding="utf-8") as fh: | ||
| data = yaml.safe_load(fh) | ||
| except (OSError, yaml.YAMLError) as exc: | ||
| print(f"::error::persona team invalid: could not read/parse {manifest}: {exc}", | ||
| file=sys.stderr) | ||
| sys.exit(1) | ||
|
Copilot marked this conversation as resolved.
|
||
| if not isinstance(data, dict): | ||
| continue | ||
| address = data.get("address") | ||
| if not address: | ||
| continue | ||
| if not isinstance(address, dict): | ||
| print(f"::error::persona team invalid: {manifest}: address must be a mapping, " | ||
| f"got {address!r}", file=sys.stderr) | ||
| sys.exit(1) | ||
| handle = address.get("handle") | ||
| if not isinstance(handle, str) or "/" not in handle: | ||
| print(f"::error::persona team invalid: {manifest}: address.handle must be an " | ||
| f"'org/team-slug' string (e.g. 'petry-projects/qa-lead'), got {handle!r}", | ||
| file=sys.stderr) | ||
| sys.exit(1) | ||
| slug = handle.split("/", 1)[1] | ||
| print(f"{slug}\t{manifest}") | ||
|
don-petry marked this conversation as resolved.
|
||
| PY | ||
| } | ||
|
|
||
| # vpt_team_api <slug> — echo the JSON for org team <slug>, or return non-zero on | ||
| # any API error (a nonexistent team returns HTTP 404, which is such an error). | ||
| # Isolated so the hermetic test suite can stub `gh`. | ||
| vpt_team_api() { | ||
| local slug="$1" | ||
| gh api "orgs/${VPT_ORG}/teams/${slug}" | ||
| } | ||
|
|
||
| # vpt_check_team <manifest> <slug> — fetch and verify the live team. Fails closed: | ||
| # a read/parse error, a secret team, or an enabled notification setting all fail. | ||
| vpt_check_team() { | ||
| local manifest="$1" slug="$2" json privacy notif | ||
| if ! json="$(vpt_team_api "$slug")"; then | ||
| vpt_fail "$manifest: could not read team '${VPT_ORG}/${slug}' from the org teams API — failing closed (an API error is never treated as a valid team). The team may not exist, or the token may not be org-scoped (use GH_PAT_WORKFLOWS)." | ||
| fi | ||
| if ! privacy="$(printf '%s' "$json" | jq -er '.privacy')"; then | ||
| vpt_fail "$manifest: team '${VPT_ORG}/${slug}' API response has no 'privacy' field (or is not JSON) — failing closed." | ||
| fi | ||
| if ! notif="$(printf '%s' "$json" | jq -er '.notification_setting')"; then | ||
| vpt_fail "$manifest: team '${VPT_ORG}/${slug}' API response has no 'notification_setting' field (or is not JSON) — failing closed." | ||
| fi | ||
|
don-petry marked this conversation as resolved.
Outdated
|
||
| if [ "$privacy" != "closed" ]; then | ||
| vpt_fail "$manifest: team '${VPT_ORG}/${slug}' has privacy '${privacy}', expected 'closed' — a secret team is unmentionable, so every mention silently no-ops." | ||
| fi | ||
| if [ "$notif" != "notifications_disabled" ]; then | ||
| vpt_fail "$manifest: team '${VPT_ORG}/${slug}' has notification_setting '${notif}', expected 'notifications_disabled' — the handle routes a webhook, it must not page the team's members." | ||
| fi | ||
| echo "verify-persona-teams: OK — ${manifest} → ${VPT_ORG}/${slug} (exists, closed, notifications disabled)." | ||
| } | ||
|
|
||
| vpt_run() { | ||
| local root="${1:-personas}" | ||
| command -v gh >/dev/null 2>&1 || vpt_fail "gh is required but not installed" | ||
| command -v jq >/dev/null 2>&1 || vpt_fail "jq is required but not installed" | ||
| command -v python3 >/dev/null 2>&1 || vpt_fail "python3 is required but not installed" | ||
|
|
||
| if [ ! -d "$root" ]; then | ||
| echo "verify-persona-teams: no personas root at '${root}' — nothing to verify." | ||
| return 0 | ||
| fi | ||
|
|
||
| local lines | ||
| if ! lines="$(vpt_list_addresses "$root")"; then | ||
| # vpt_list_addresses already printed a ::error:: diagnostic; fail closed. | ||
| exit 1 | ||
| fi | ||
|
don-petry marked this conversation as resolved.
|
||
|
|
||
| if [ -z "$lines" ]; then | ||
| echo "verify-persona-teams: no personas declare an address block — nothing to verify." | ||
| return 0 | ||
| fi | ||
|
|
||
| local count=0 slug manifest | ||
| while IFS=$'\t' read -r slug manifest; do | ||
| [ -n "$slug" ] || continue | ||
| vpt_check_team "$manifest" "$slug" | ||
| count=$((count + 1)) | ||
| done <<< "$lines" | ||
|
|
||
| echo "verify-persona-teams: OK — ${count} persona team(s) verified live." | ||
| } | ||
|
|
||
| # Only run main when executed directly, so the hermetic tests can source the | ||
| # functions and stub `gh`. | ||
| if [ "${BASH_SOURCE[0]}" = "${0}" ]; then | ||
| vpt_run "$@" | ||
| fi | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,163 @@ | ||
| #!/usr/bin/env bats | ||
| # Hermetic unit + fixture tests for scripts/verify-persona-teams.sh (#1280). | ||
| # | ||
| # verify-persona-teams.sh is the LIVE, network-touching half of persona addressing | ||
| # validation: for every personas/<id>/persona.yml that declares an `address` block, | ||
| # it verifies the team named by address.handle EXISTS, is `privacy: closed` (secret | ||
| # teams are unmentionable), and is `notification_setting: notifications_disabled` | ||
| # (the handle routes a webhook, it must not page members). It FAILS CLOSED — an API | ||
| # error is never read as "the team is fine" (#755). | ||
| # | ||
| # These tests stay hermetic (the sibling of the hermetic validate-personas suite): | ||
| # the network call goes through `gh`, which is stubbed here to return canned team | ||
| # JSON or an error. NO real teams API is ever contacted. | ||
| # | ||
| # Run with: bats tests/verify_persona_teams.bats | ||
|
|
||
| setup() { | ||
| REPO_ROOT="$(cd "$(dirname "$BATS_TEST_FILENAME")/.." && pwd)" | ||
| SCRIPT="$REPO_ROOT/scripts/verify-persona-teams.sh" | ||
| TMP="$(mktemp -d)" | ||
| # shellcheck source=/dev/null | ||
| source "$SCRIPT" | ||
|
|
||
| # Stub `gh` so `vpt_team_api` never touches the network. It emulates | ||
| # `gh api orgs/<org>/teams/<slug>`: echo VPT_TEST_TEAM_JSON, or fail if | ||
| # VPT_TEST_GH_FAIL=1 (an API/HTTP error, e.g. 404 for a nonexistent team). | ||
| gh() { | ||
| if [ "${VPT_TEST_GH_FAIL:-0}" = "1" ]; then | ||
| return 1 | ||
| fi | ||
| printf '%s' "${VPT_TEST_TEAM_JSON:-}" | ||
| } | ||
| export -f gh | ||
| } | ||
|
|
||
| teardown() { rm -rf "$TMP"; } | ||
|
don-petry marked this conversation as resolved.
Outdated
|
||
|
|
||
| # make_persona <id> [handle] — write a minimal personas/<id>/persona.yml fixture. | ||
| # With a handle, it carries an `address` block; without, it has none. | ||
| make_persona() { | ||
| local id="$1" handle="${2:-}" | ||
| mkdir -p "$TMP/personas/$id" | ||
| { | ||
| printf 'id: %s\nname: %s\n' "$id" "$id" | ||
| if [ -n "$handle" ]; then | ||
| printf 'address:\n handle: %s\n aliases: []\n' "$handle" | ||
| fi | ||
| } > "$TMP/personas/$id/persona.yml" | ||
| } | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # vpt_check_team — verify one live team (the core assertion, fails closed) | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
| @test "vpt_check_team passes a closed, notifications-disabled team" { | ||
| export VPT_TEST_TEAM_JSON='{"slug":"qa-lead","privacy":"closed","notification_setting":"notifications_disabled"}' | ||
| run vpt_check_team "personas/qa-lead/persona.yml" qa-lead | ||
| [ "$status" -eq 0 ] | ||
| } | ||
|
|
||
| @test "vpt_check_team fails a secret team (unmentionable)" { | ||
| export VPT_TEST_TEAM_JSON='{"slug":"qa-lead","privacy":"secret","notification_setting":"notifications_disabled"}' | ||
| run vpt_check_team "personas/qa-lead/persona.yml" qa-lead | ||
| [ "$status" -ne 0 ] | ||
| [[ "$output" == *"privacy"* ]] | ||
| [[ "$output" == *"closed"* ]] | ||
| [[ "$output" != *"Traceback"* ]] | ||
| } | ||
|
|
||
| @test "vpt_check_team fails a team that pages its members" { | ||
| export VPT_TEST_TEAM_JSON='{"slug":"qa-lead","privacy":"closed","notification_setting":"notifications_enabled"}' | ||
| run vpt_check_team "personas/qa-lead/persona.yml" qa-lead | ||
| [ "$status" -ne 0 ] | ||
| [[ "$output" == *"notification_setting"* ]] | ||
| [[ "$output" != *"Traceback"* ]] | ||
| } | ||
|
|
||
| @test "vpt_check_team fails CLOSED when the teams API errors" { | ||
| export VPT_TEST_GH_FAIL=1 | ||
| run vpt_check_team "personas/qa-lead/persona.yml" qa-lead | ||
| [ "$status" -ne 0 ] | ||
| [[ "$output" == *"could not read team"* ]] | ||
| [[ "$output" != *"Traceback"* ]] | ||
| } | ||
|
|
||
| @test "vpt_check_team fails a team JSON missing privacy" { | ||
| export VPT_TEST_TEAM_JSON='{"slug":"qa-lead","notification_setting":"notifications_disabled"}' | ||
| run vpt_check_team "personas/qa-lead/persona.yml" qa-lead | ||
| [ "$status" -ne 0 ] | ||
| [[ "$output" != *"Traceback"* ]] | ||
| } | ||
|
|
||
| @test "vpt_check_team fails when the API returns non-JSON" { | ||
| export VPT_TEST_TEAM_JSON='not json at all' | ||
| run vpt_check_team "personas/qa-lead/persona.yml" qa-lead | ||
| [ "$status" -ne 0 ] | ||
| [[ "$output" != *"Traceback"* ]] | ||
| } | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # vpt_list_addresses — extract (slug, manifest) only for address-carrying personas | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
| @test "vpt_list_addresses emits slug+manifest only for personas with an address" { | ||
| make_persona qa-lead petry-projects/qa-lead | ||
| make_persona no-addr | ||
| run vpt_list_addresses "$TMP/personas" | ||
| [ "$status" -eq 0 ] | ||
| [[ "$output" == *"qa-lead"* ]] | ||
| [[ "$output" != *"no-addr"* ]] | ||
| } | ||
|
|
||
| @test "vpt_list_addresses reports a malformed handle without a traceback" { | ||
| make_persona bad "no-slash-handle" | ||
| run vpt_list_addresses "$TMP/personas" | ||
| [ "$status" -ne 0 ] | ||
| [[ "$output" != *"Traceback"* ]] | ||
| } | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # vpt_run — end-to-end orchestration over a fixture personas tree | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
| @test "vpt_run verifies a persona whose team is closed + notifications-disabled" { | ||
| make_persona qa-lead petry-projects/qa-lead | ||
| export VPT_TEST_TEAM_JSON='{"slug":"qa-lead","privacy":"closed","notification_setting":"notifications_disabled"}' | ||
| run vpt_run "$TMP/personas" | ||
| [ "$status" -eq 0 ] | ||
| [[ "$output" == *"OK"* ]] | ||
| } | ||
|
|
||
| @test "vpt_run fails when a persona's team is secret" { | ||
| make_persona qa-lead petry-projects/qa-lead | ||
| export VPT_TEST_TEAM_JSON='{"slug":"qa-lead","privacy":"secret","notification_setting":"notifications_disabled"}' | ||
| run vpt_run "$TMP/personas" | ||
| [ "$status" -ne 0 ] | ||
| } | ||
|
|
||
| @test "vpt_run fails CLOSED when the teams API errors for a declared address" { | ||
| make_persona qa-lead petry-projects/qa-lead | ||
| export VPT_TEST_GH_FAIL=1 | ||
| run vpt_run "$TMP/personas" | ||
| [ "$status" -ne 0 ] | ||
| [[ "$output" == *"could not read team"* ]] | ||
| } | ||
|
|
||
| @test "vpt_run passes a persona with no address block (address is optional)" { | ||
| make_persona no-addr | ||
| run vpt_run "$TMP/personas" | ||
| [ "$status" -eq 0 ] | ||
| } | ||
|
|
||
| @test "vpt_run is a no-op when there is no personas root" { | ||
| run vpt_run "$TMP/nonexistent" | ||
| [ "$status" -eq 0 ] | ||
| } | ||
|
|
||
| @test "vpt_run reports a malformed handle without a traceback" { | ||
| make_persona bad "no-slash-handle" | ||
| run vpt_run "$TMP/personas" | ||
| [ "$status" -ne 0 ] | ||
| [[ "$output" != *"Traceback"* ]] | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.