Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/pr-limits-report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,4 +48,6 @@ jobs:
env:
# `gh search prs` needs a token; github.token has the read scope it needs.
GH_TOKEN: ${{ github.token }}
# Org-level Actions variable; overrides the cap in pr-limits.json when set.
PR_LIMITS_ORG_CAP: ${{ vars.PR_LIMITS_ORG_CAP }}
Comment thread
don-petry marked this conversation as resolved.
Comment thread
don-petry marked this conversation as resolved.
run: bash scripts/pr-limits-report.sh
8 changes: 6 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -609,8 +609,12 @@ auto-rebase fan-out. Full standard, rationale, and operator runbook:
so this is enforced **source-side** by [`scripts/lib/pr-limit-gate.sh`](https://github.com/petry-projects/.github/blob/main/scripts/lib/pr-limit-gate.sh):
a PR-creating workflow asks the gate before opening a PR and **defers** (never
fails or closes) when the queue is at the cap. Live-path wiring is Story 3 (#508).
- **The configured value lives only in [`standards/pr-limits.json`](https://github.com/petry-projects/.github/blob/main/standards/pr-limits.json)** —
the single source of truth. Read it with `jq`; never hardcode or restate it.
- **The signed-off default value lives only in [`standards/pr-limits.json`](https://github.com/petry-projects/.github/blob/main/standards/pr-limits.json)** —
the single source of truth for the default. Read it with `jq`; never hardcode or restate it.
- **Runtime override (the operational lever):** the org-level Actions variable `PR_LIMITS_ORG_CAP`
(positive integer) overrides the file value without a code change; invalid or
unset falls back to the file. Pass it via `vars.PR_LIMITS_ORG_CAP` (see
[`standards/pr-limits.md` §2.1](https://github.com/petry-projects/.github/blob/main/standards/pr-limits.md)).

> **Exempt actors are sanctioned policy, not drift.** `dependabot[bot]`,
> `OrganizationAdmin`, `@petry-projects/org-leads`, the `dependabot-automerge-petry`
Expand Down
31 changes: 28 additions & 3 deletions scripts/lib/pr-limit-gate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,11 @@
# - $PR_LIMITS_CONFIG — path to the pr-limits.json single source of truth
# (default: <repo>/standards/pr-limits.json, resolved
# relative to this file)
# - $PR_LIMITS_ORG_CAP — runtime override for org_wide.automation_open_pr_cap.
# Wire it from the org-level Actions variable
# (`vars.PR_LIMITS_ORG_CAP`) so the ceiling can change on
# the fly with no code change. Must be a positive
# integer; unset/empty/invalid falls back to the config.
# - $DRY_RUN / $DEV_LEAD_DRY_RUN — "true" forces an allow result with no
# side effects, after printing the computed decision
# - `gh` CLI on PATH, `jq` on PATH
Expand All @@ -53,6 +58,26 @@
printf '%s' "${PR_LIMITS_CONFIG:-$PLG_DEFAULT_CONFIG}"
}

# ---------------------------------------------------------------------------
# plg_effective_org_cap <config> — echo the org-wide cap in force.
#
# Precedence: $PR_LIMITS_ORG_CAP (org-level variable) when set to a positive
# integer, else org_wide.automation_open_pr_cap from the config. An invalid
# override is ignored with a warning (never silently disables or zeroes the cap).
# Echoes nothing if neither source yields an integer.
# ---------------------------------------------------------------------------
plg_effective_org_cap() {
local config="$1" override="${PR_LIMITS_ORG_CAP:-}"
if [ -n "$override" ]; then

Check failure on line 71 in scripts/lib/pr-limit-gate.sh

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.

See more on https://sonarcloud.io/project/issues?id=petry-projects_.github&issues=AaD5J7q5a-7KIkAgkiPz&open=AaD5J7q5a-7KIkAgkiPz&pullRequest=1221
if [[ "$override" =~ ^[1-9][0-9]{0,8}$ ]]; then
printf '%s' "$override"
return 0
fi
plg_log "warning: ignoring invalid PR_LIMITS_ORG_CAP='$override' (need a positive integer of at most 9 digits); using config value"
fi
Comment thread
don-petry marked this conversation as resolved.
jq -er 'try .org_wide.automation_open_pr_cap catch empty' "$config" 2>/dev/null || printf ''
}

# ---------------------------------------------------------------------------
# plg_is_dry_run — 0 (true) when DRY_RUN or DEV_LEAD_DRY_RUN is "true".
# ---------------------------------------------------------------------------
Expand Down Expand Up @@ -193,9 +218,9 @@
fi

local org_cap org_count
org_cap="$(jq -er '.org_wide.automation_open_pr_cap' "$config" 2>/dev/null || printf '')"
org_cap="$(plg_effective_org_cap "$config")"
if ! [[ "$org_cap" =~ ^[0-9]+$ ]]; then
plg_log "error: org_wide.automation_open_pr_cap is missing or not an integer in $config"
plg_log "error: org cap is missing or not an integer (PR_LIMITS_ORG_CAP / org_wide.automation_open_pr_cap in $config)"
return 2
fi
org_count="$(plg_count_open_automation_prs "$config")"
Expand All @@ -213,7 +238,7 @@
else
# 3. Per-source sub-cap, only when one is configured for this source.
local sub_cap
sub_cap="$(jq -r --arg s "$source" '.per_source_caps[$s] // empty' "$config" 2>/dev/null || printf '')"
sub_cap="$(jq -r --arg s "$source" 'try .per_source_caps[$s] catch empty // empty' "$config" 2>/dev/null || printf '')"
if [[ "$sub_cap" =~ ^[0-9]+$ ]]; then
local src_count
src_count="$(plg_count_source_prs "$source")"
Expand Down
10 changes: 8 additions & 2 deletions scripts/pr-limits-report.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@
#
# Environment (all optional):
# ORG — GitHub org slug to scope the search (default: petry-projects)
# PR_LIMITS_ORG_CAP — org-variable override of the cap (positive integer); wins
# over the config value, same as the admission gate
# PR_LIMITS_CONFIG — path to pr-limits.json (default: <repo>/standards/pr-limits.json,
# resolved relative to this script so it works from any CWD)
# GH_TOKEN / GITHUB_TOKEN — token for `gh search prs` (github.token in CI)
Expand All @@ -37,15 +39,19 @@ DEFAULT_CONFIG="${SCRIPT_DIR}/../standards/pr-limits.json"
CONFIG="${PR_LIMITS_CONFIG:-$DEFAULT_CONFIG}"
ORG="${ORG:-petry-projects}"

# Reuse the gate's cap resolution so report and gate always agree.
# shellcheck source=scripts/lib/pr-limit-gate.sh
source "${SCRIPT_DIR}/lib/pr-limit-gate.sh"

if [ ! -f "$CONFIG" ]; then
echo "pr-limits-report: error: config not found at $CONFIG" >&2
exit 1
fi

# ── Read the cap + exempt lists from the single source of truth ───────────────
CAP="$(jq -er '.org_wide.automation_open_pr_cap' "$CONFIG" 2>/dev/null || printf '')"
CAP="$(plg_effective_org_cap "$CONFIG")"
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
if ! [[ "$CAP" =~ ^[0-9]+$ ]]; then
echo "pr-limits-report: error: org_wide.automation_open_pr_cap missing or not an integer in $CONFIG" >&2
echo "pr-limits-report: error: org cap missing or not an integer (PR_LIMITS_ORG_CAP / org_wide.automation_open_pr_cap in $CONFIG)" >&2
exit 1
fi
EXEMPT_ACTORS="$(jq -c '.exempt_actors // []' "$CONFIG" 2>/dev/null || echo '[]')"
Expand Down
58 changes: 46 additions & 12 deletions standards/pr-limits.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,11 @@ double-cap it and risk starving a security PR. See §5 and ADR §7.4.

## 2. Source of truth — the configured value

The cap value, any per-source sub-caps, and the exempt lists live **only** in
[`standards/pr-limits.json`](pr-limits.json). This document deliberately does
**not** restate the number: a changeable value stated in prose is a second place
to forget to update. To read the current cap, read the config:
The signed-off **default** cap value, any per-source sub-caps, and the exempt
lists live **only** in [`standards/pr-limits.json`](pr-limits.json). The cap in
force can be overridden at runtime by the org variable in §2.1. This document
deliberately does **not** restate the number: a changeable value stated in prose
is a second place to forget to update. To read the default cap, read the config:

```bash
jq '.org_wide.automation_open_pr_cap' standards/pr-limits.json
Expand All @@ -54,7 +55,31 @@ jq '.org_wide.automation_open_pr_cap' standards/pr-limits.json
The config carries its own inline `_note` fields recording the human sign-off
(epic [#505](https://github.com/petry-projects/.github/issues/505) gate) and the
rationale for the number. Consumers (the apply path in §3, and any future
PR-creating workflow) **must** read the value from this file — never hardcode it.
PR-creating workflow) **must** resolve the cap via `plg_effective_org_cap`
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
(override, then this file) — never hardcode it.

### 2.1 Runtime override — the org-level variable

The ceiling can be changed **on the fly, with no code change or PR**, via the
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
org-level Actions variable **`PR_LIMITS_ORG_CAP`** (Org → Settings → Secrets and
variables → Actions → Variables). Consumers pass it to the gate as the
`PR_LIMITS_ORG_CAP` environment variable (`env: PR_LIMITS_ORG_CAP: ${{ vars.PR_LIMITS_ORG_CAP }}`).

- **Precedence:** a positive-integer `PR_LIMITS_ORG_CAP` (at most 9 digits, no
leading zeros) wins; if it is unset or empty (the normal "variable not set"
case, silent), or invalid (zero, negative, non-integer, leading zeros, more than
9 digits — logged as a warning), the gate falls back to `org_wide.automation_open_pr_cap` in the config. A bad value can
never silently disable or zero the cap.
- The file value is the **default/fallback**; the variable is the **operational
lever**. Exempt actors/labels and sub-caps still come only from the config.
- Resolution lives in one place (`plg_effective_org_cap` in the gate library) and
is reused by [`scripts/pr-limits-report.sh`](../scripts/pr-limits-report.sh), so
the gate and the daily report always agree.
- Set or change it with `gh variable set PR_LIMITS_ORG_CAP --org petry-projects --visibility all --body <N>`.
Take effect on the next run of each consumer; remove it to revert to the file.
- **Wiring:** this repo's report workflow already passes it. The PR-creation
engine in `petry-projects/.github-private` must add the same `env:` line to the
step that sources the gate for the variable to take effect there.

Its contract (parseable JSON, positive-integer cap, required keys,
`dependabot[bot]` and the `security` label present on the exempt lists) is
Expand All @@ -76,7 +101,9 @@ workflow sources it and calls `plg_admission_gate <source>` before opening a PR:
1. If `<source>` is an exempt actor (§4) → **allow** (never blocked, never counted).
2. Else it counts the open, non-draft, non-exempt automation queue org-wide via
`gh search prs` (the enumeration lives in the gate library itself) and, if
the queue is at or over `org_wide.automation_open_pr_cap` → **defer**.
the queue is at or over the effective cap (`plg_effective_org_cap`: the
`PR_LIMITS_ORG_CAP` override (§2.1) if set and valid, else
`org_wide.automation_open_pr_cap`) → **defer**.
3. If a per-source sub-cap is configured for `<source>` and its own queue is at
or over it → **defer**. (No sub-caps are configured under the current
signed-off policy; the map is intentionally empty.)
Expand Down Expand Up @@ -133,10 +160,16 @@ This keeps a single source of truth *per actor*: Dependabot's number lives in

## 6. Operator runbook

All changes here are edits to the single source of truth
[`standards/pr-limits.json`](pr-limits.json), gated by the config tests. None of
them touches a GitHub setting or ruleset — there is no such surface to apply
(§3).
Changes here are either edits to the single source of truth
[`standards/pr-limits.json`](pr-limits.json) (the default, gated by the config
tests) or a change to the `PR_LIMITS_ORG_CAP` org Actions variable (the runtime
override, §2.1, not gated by tests). Neither touches a repo setting or ruleset —
there is no such surface to apply (§3).

### 6.0 Quick change (no PR)

Set the org variable (§2.1): `gh variable set PR_LIMITS_ORG_CAP --org petry-projects --visibility all --body <N>`.
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
Use §6.1 only to change the signed-off default in the file.
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.

### 6.1 Change the limit and re-apply it

Expand All @@ -153,8 +186,9 @@ them touches a GitHub setting or ruleset — there is no such surface to apply
3. Open a PR. CI ([`.github/workflows/pr-limits-tests.yml`](../.github/workflows/pr-limits-tests.yml))
re-runs the config + gate tests.
4. **Re-apply = nothing to deploy.** Because enforcement is source-side and reads
the file at run time (§2–§3), the new ceiling takes effect for every consumer
as soon as the change merges to `main` — there is no `apply-*.sh` run and no
the file at run time (§2–§3), the new default takes effect for every consumer
as soon as the change merges to `main` — unless a valid `PR_LIMITS_ORG_CAP`
override (§2.1) is set, in which case the file value is only the fallback — there is no `apply-*.sh` run and no
GitHub setting to push. (Per-repo Dependabot caps, if you also changed those,
are applied through `dependabot.yml`, not this file.)

Expand Down
49 changes: 49 additions & 0 deletions test/scripts/pr-limits/pr-limit-gate.bats
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ LIB="$(cd "$BATS_TEST_DIRNAME/../../.." && pwd)/scripts/lib/pr-limit-gate.sh"
GH_STUB_SRC="$(cd "$BATS_TEST_DIRNAME/../.." && pwd)/scripts/compliance-remediate/stubs/gh"

setup() {
# Quarantine the override so ambient env never skews config-driven tests.
unset PR_LIMITS_ORG_CAP
TMP="$(mktemp -d)"
export TMP

Expand Down Expand Up @@ -84,6 +86,53 @@ run_gate() {
[[ "$output" == *"decision=allow"* ]]
}

# --------------------------------------------------------------------------
# PR_LIMITS_ORG_CAP — org-variable override of the config cap
# --------------------------------------------------------------------------
@test "PR_LIMITS_ORG_CAP raises the cap above the config value" {
write_config 5 9
stub_open_prs 7
PR_LIMITS_ORG_CAP=20 run_gate "claude"
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
[ "$status" -eq 0 ]
[[ "$output" == *"decision=allow"* ]]
}

@test "PR_LIMITS_ORG_CAP can lower the cap below the config value" {
write_config 50 9
stub_open_prs 7
PR_LIMITS_ORG_CAP=7 run_gate "claude"
[ "$status" -eq 1 ]
[[ "$output" == *"decision=defer"* ]]
}

@test "invalid PR_LIMITS_ORG_CAP is ignored and the config cap applies" {
write_config 5 9
stub_open_prs 7
for bad in 0 -3 abc 1.5 007 1000000000 99999999999999999999; do
PR_LIMITS_ORG_CAP="$bad" run_gate "claude"
[ "$status" -eq 1 ]
[[ "$output" == *"decision=defer"* ]]
[[ "$output" == *"ignoring invalid PR_LIMITS_ORG_CAP"* ]]
done
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.

@test "9-digit maximum PR_LIMITS_ORG_CAP is accepted as a valid override" {
write_config 5 9
stub_open_prs 7
PR_LIMITS_ORG_CAP=999999999 run_gate "claude"
[ "$status" -eq 0 ]
[[ "$output" == *"decision=allow"* ]]
[[ "$output" != *"ignoring invalid PR_LIMITS_ORG_CAP"* ]]
}

@test "empty PR_LIMITS_ORG_CAP (unset org variable) falls back to config" {
write_config 5 9
stub_open_prs 7
PR_LIMITS_ORG_CAP="" run_gate "claude"
[ "$status" -eq 1 ]
[[ "$output" == *"decision=defer"* ]]
}

# --------------------------------------------------------------------------
# AC #1 — at / over the org-wide cap: defer
# --------------------------------------------------------------------------
Expand Down
24 changes: 24 additions & 0 deletions test/scripts/pr-limits/pr-limits-report.bats
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ SCRIPT="$(cd "$BATS_TEST_DIRNAME/../../.." && pwd)/scripts/pr-limits-report.sh"
GH_STUB_SRC="$(cd "$BATS_TEST_DIRNAME/../.." && pwd)/scripts/compliance-remediate/stubs/gh"

setup() {
# Quarantine the override so ambient env never skews config-driven tests.
unset PR_LIMITS_ORG_CAP
TMP="$(mktemp -d)"
export TMP

Expand Down Expand Up @@ -188,6 +190,28 @@ run_report() {
[[ "$output" == *"| Cap | 4 |"* ]]
}

# --------------------------------------------------------------------------
# PR_LIMITS_ORG_CAP override is honored by the report (same as the gate)
# --------------------------------------------------------------------------
@test "PR_LIMITS_ORG_CAP overrides the config cap in the report" {
write_config 4
stub_open_prs 4
PR_LIMITS_ORG_CAP=20 run_report
[ "$status" -eq 0 ]
[[ "$output" != *"AT OR OVER CAP"* ]]
[[ "$output" == *"| Cap | 20 |"* ]]
}

@test "invalid or empty PR_LIMITS_ORG_CAP falls back to the config cap in the report" {
write_config 4
stub_open_prs 4
for bad in "" 0 abc 99999999999999999999; do
PR_LIMITS_ORG_CAP="$bad" run_report
[ "$status" -eq 0 ]
[[ "$output" == *"| Cap | 4 |"* ]]
done
}

# --------------------------------------------------------------------------
# A null / missing author does not crash jq: it is counted as "unknown".
# --------------------------------------------------------------------------
Expand Down
Loading