fix(budget): exempt the Dependabot merge pipeline, count marked agent comments, and set the cap from measured PR peaks (#2224)

docs/agentic-interaction-model.md

@@ -551,16 +551,34 @@ More numbers would be more false comfort.
 
 The ceiling is therefore the **existing** two controls, and they remain authoritative:
 
-1. **The per-PR automation budget** — `MAX_PR_AUTOMATION_CYCLES=10` in
+1. **The per-PR automation budget** — `MAX_PR_AUTOMATION_CYCLES=50` in
    `scripts/lib/pr-automation-budget.sh`. It counts agent-authored commits + review cycles +
    acks **since the last human interaction** (action-counting, not failure-counting; per-PR,
    not per-worker — §7 rules 3, 4). On exhaustion it stops all automated writes, adds
    `needs-human-review`, disables auto-merge, and posts one deduped escalation. Re-engagement
    is **human-gated** (§7 rule 2); `FORCE_REVIEW` does not bypass it.
+
+   Who is "automated" is decided by login (`*[bot]` or `AUTOMATION_BOT_LOGINS`), with two
+   corrections (#2224). On a PR authored by `dependabot[bot]`, the merge pipeline's own
+   events (Dependabot's commit, the dependabot-automerge App's branch updates and
+   re-approvals) neither count nor reset: a push to `main` causes them, not an agent
+   answering an agent. And a comment or review that carries one of our automation markers
+   is an agent event whatever login posted it: it counts and it never resets, because
+   dev-lead, the personas and auto-rebase post as the owner account. **Known limit:** a
+   commit, an unmarked comment or a review with no marker in its body that an agent makes
+   under the owner account still reads as a human interaction and resets the budget. That
+   includes review-thread replies: the reply text (`/pulls/{pr}/comments`) is not read, but
+   each reply arrives as a review with an empty body, and that review is collected.
+
+   The cap of 50 is set from measurement so that only extreme cases trip it (#2224). Across
+   332 agent-authored PRs (2026-09-10 to 2026-10-10, plus those still open), the peak count
+   between two human interactions was median 10, p95 25, p99 35; no PR that merged went
+   above 35 and one PR exceeded 50. #860 peaks at 1,487 under the same rule.
 2. **The runaway detector** — the pushed detection signal in
    `daily-pr-review-health` / `scripts/fleet_monitor.sh` that flags open PRs over soft
    thresholds (commit/comment/cycle counts, or open >48h with active agent churn), so a
-   runaway is surfaced rather than waited-on (§7 rule 7).
+   runaway is surfaced rather than waited-on (§7 rule 7). Its cycle threshold is 25, half
+   the budget cap, so a PR is reported well before it is stopped.
 
 **Rule.** A new agentic role or timer must obey these two ceilings and declare which via its
 contract's `budget` field (§8.1). It must **not** invent its own numeric cap as a substitute

docs/postmortems/2026-06-pr-860-runaway.md

@@ -99,7 +99,8 @@ This work spans two repos.
 - **Per-PR automated-churn circuit breaker** (`scripts/lib/pr-automation-budget.sh`):
   `compute_pr_automation_cycles` / `pr_budget_exhausted` count agent-authored
   commits + review cycles + acks **since the last human interaction**; default
-  `MAX_PR_AUTOMATION_CYCLES=10` (env-overridable). Consulted by
+  `MAX_PR_AUTOMATION_CYCLES=10` (env-overridable; raised to 50 in #2224, when agent
+  comments posted under the owner login began to count). Consulted by
   `scripts/dev-lead-fix-reviews.sh`, `scripts/dev-lead-fix-ci.sh`, and
   `scripts/review-one-pr.sh` before any automated write. On exhaustion it **stops
   all automated writes**, adds `needs-human-review`, **disables auto-merge**, and

scripts/lib/pr-automation-budget.sh

@@ -13,7 +13,7 @@
 #     the count to zero; a machine action (bot comment, bot commit, machine
 #     approval) never does — so the runaway can't re-arm its own budget.
 #   pr_budget_exhausted <events_json>
-#     Exit 0 when the count has reached MAX_PR_AUTOMATION_CYCLES (default 10).
+#     Exit 0 when the count has reached MAX_PR_AUTOMATION_CYCLES (default 50).
 #   gather_pr_automation_events <pr> <repo>   -> normalized {when, login}[] JSON
 #   enforce_pr_budget <pr> <repo>
 #     The orchestration entrypoint scripts call before doing automated writes:
@@ -25,12 +25,59 @@
 # compare correctly); items with null/empty `when` are ignored. A login is a
 # bot when it ends in `[bot]` or is listed in AUTOMATION_BOT_LOGINS; an empty /
 # unattributed login is treated as a machine (conservative — it must not reset).
+#
+# Login alone misreads two kinds of event, so gather_pr_automation_events flags
+# them (#2224) and compute_pr_automation_cycles honours the flags:
+#   mechanical: true    On a PR authored by dependabot[bot], an event by a login
+#                       in PR_BUDGET_MECHANICAL_LOGINS — Dependabot's own commit,
+#                       the dependabot-automerge App's `update-branch` merge
+#                       commits and re-approvals. A push to `main` causes these,
+#                       not an agent answering an agent. Neither counts nor
+#                       resets. Both keys are required: any other bot on a
+#                       Dependabot PR, and the App on anyone else's PR, count.
+#   agent_marked: true  A comment or review whose body carries one of our
+#                       automation markers (PR_BUDGET_AGENT_MARKER_RE). dev-lead,
+#                       the personas and auto-rebase post as the owner account,
+#                       so their login reads as a human. Such an event is an
+#                       agent event whatever its login: it counts, and it never
+#                       resets.
+# Still read as human, because nothing in the event tells them apart: a commit,
+# an unmarked comment, or a review-thread reply an agent makes under a human login.
 
 # Space-separated bot logins beyond any `*[bot]` GitHub App identity. Includes
 # the agent identity (donpetry-bot) and repair-pr-approvals, whose machine
 # approvals must not reset the budget.
 : "${AUTOMATION_BOT_LOGINS:=donpetry-bot github-actions[bot] repair-pr-approvals}"
-: "${MAX_PR_AUTOMATION_CYCLES:=10}"
+# The cap is set from measurement, to trip only on extreme cases (#2224). The
+# sample is 332 agent-authored PRs: every one created in the org from 2026-09-10
+# to 2026-10-10, plus those still open. The peak count a PR reached between two
+# human interactions was: median 10, p95 25, p99 35. The highest on a PR that
+# merged was 35; one PR (93) was above 50. #860 peaks at 1,487 under the same
+# rule, and its ack storm (one every ~9 seconds) reaches 50 in under ten
+# minutes. Before #2224 the cap was 10 and agent comments posted under the owner
+# login were not counted; at 10 with them counted, half the sample would trip.
+: "${MAX_PR_AUTOMATION_CYCLES:=50}"
+
+# Space-separated logins of the Dependabot merge pipeline. Their events are
+# flagged `mechanical` on a PR authored by dependabot[bot], and nowhere else.
+# Set but empty turns the exemption off (hence `=`, not `:=`).
+: "${PR_BUDGET_MECHANICAL_LOGINS=dependabot[bot] petry-projects-dependabot-automrg[bot]}"
+_PR_BUDGET_DEPENDABOT_AUTHOR="dependabot[bot]"
+
+# Regex (jq `test`) for a body one of our agents wrote. The registered list is
+# scripts/lib/maintainer-comment-gate.sh's _MAINTAINER_GATE_AGENT_MARKERS (kept
+# in step by tests/test_pr_automation_budget.bats), with two differences:
+# `maintainer-resolve` is left out, because a maintainer posts it through
+# maintainer-resolve-comment.sh and that is a human act; and this breaker's own
+# escalation marker is added, because dev-lead posts it as the owner account.
+# Ensure agent markers are anchored to prevent matching arbitrary text: most need
+# a delimiter (colon, dash, or space) after the keyword (#2224 false-positive fix).
+# The match is not anchored to the start of the body: about a fifth of real agent
+# comments carry their marker further down (a disposition marker closing a
+# reply, for one). So a marker a human quotes is told apart by backticks alone:
+# agent_marked drops `code spans` before testing. A complete marker pasted
+# outside backticks still reads as an agent's.
+PR_BUDGET_AGENT_MARKER_RE='<!-- (pr-review-agent[ ]|pr-review-claim[ ]|persona:|dev-lead[ :-]|dependency-advisory[ ]|auto-rebase-|pr-automation-budget[ ])[^>]*-->'
 
 # The label this breaker adds on exhaustion (and the cycle cap adds on escalation).
 # It is the human-controlled re-engagement gate: a human removing it resumes
@@ -57,19 +104,23 @@ _PR_BUDGET_JQ_DEFS='
     | ($login == null) or ($login == "")
       or ($login | endswith("[bot]"))
       or (($bots | index($login)) != null);
+  def agent_marked($re): (. // "" | tostring | gsub("`[^`]*`"; "") | test($re));
 '
 
 # compute_pr_automation_cycles <events_json>
-#   Prints the number of bot-authored events newer than the latest human event.
+#   Prints the number of automated events newer than the latest human event.
+#   An event is automated when its login is a bot or it is `agent_marked`; every
+#   other event is human. `mechanical` events are left out altogether (#2224).
 compute_pr_automation_cycles() {
   local events_json="${1:-[]}"
   local bots_json count
   bots_json=$(printf '%s' "${AUTOMATION_BOT_LOGINS}" \
     | jq -R 'split(" ") | map(select(. != ""))' 2>/dev/null) || bots_json='[]'
   count=$(jq -r --argjson bots "$bots_json" "$_PR_BUDGET_JQ_DEFS"'
-    map(select(.when != null and .when != ""))
-    | ([.[] | select((.login | is_bot($bots)) | not) | .when] | max // "") as $last_human
-    | [.[] | select((.login | is_bot($bots)) and (.when > $last_human))]
+    map(select(.when != null and .when != "" and .mechanical != true))
+    | ([.[] | select(((.login | is_bot($bots)) | not) and (.agent_marked != true)) | .when]
+        | max // "") as $last_human
+    | [.[] | select(((.login | is_bot($bots)) or (.agent_marked == true)) and (.when > $last_human))]
     | length
   ' <<<"$events_json" 2>/dev/null) || count=0
   # Defensive: malformed input degrades to 0 so the integer comparison at the
@@ -80,6 +131,38 @@ compute_pr_automation_cycles() {
   esac
 }
 
+# pr_budget_flag_mechanical <events_json> [pr_author]
+#   Print the events with `mechanical: true` added to each Dependabot-pipeline
+#   event (see the header). Pure. No event is dropped: the stall and merge-ready
+#   scans read `.when` from every event. An empty, missing or other PR author
+#   flags nothing, and input that is not a JSON array is printed unchanged, so a
+#   failure here can only leave the count where it was (fail closed).
+pr_budget_flag_mechanical() {
+  local events_json="${1:-[]}" pr_author="${2:-}" logins_json out
+  if [ "$pr_author" != "$_PR_BUDGET_DEPENDABOT_AUTHOR" ] || [ -z "${PR_BUDGET_MECHANICAL_LOGINS}" ]; then
+    printf '%s\n' "$events_json"
+    return 0
+  fi
+  logins_json=$(printf '%s' "${PR_BUDGET_MECHANICAL_LOGINS}" \
+    | jq -R 'split(" ") | map(select(. != ""))' 2>/dev/null) || logins_json='[]'
+  if out=$(jq -c --argjson logins "$logins_json" '
+      if type != "array" then error("not an array") else . end
+      | map(if (.login as $l | $logins | index($l)) != null then . + {mechanical: true} else . end)
+    ' <<<"$events_json" 2>/dev/null); then
+    printf '%s\n' "$out"
+  else
+    printf '%s\n' "$events_json"
+  fi
+}
+
+# pr_budget_body_is_agent_marked <body>
+#   Exit 0 when the body carries one of our automation markers. Pure; the same
+#   jq test gather_pr_automation_events applies to comment and review bodies.
+pr_budget_body_is_agent_marked() {
+  jq -en --arg b "${1:-}" --arg re "$PR_BUDGET_AGENT_MARKER_RE" "$_PR_BUDGET_JQ_DEFS"'
+    $b | agent_marked($re)' >/dev/null 2>&1
+}
+
 # pr_has_escalation_label <labels_json>
 #   Exit 0 if the JSON array of label NAMES contains NEEDS_HUMAN_REVIEW_LABEL —
 #   i.e. the PR is currently escalated to a human.
@@ -249,24 +332,31 @@ pr_budget_exhausted() {
   local events_json="${1:-[]}"
   local n
   n=$(compute_pr_automation_cycles "$events_json")
-  if [ "${n:-0}" -ge "${MAX_PR_AUTOMATION_CYCLES:-10}" ]; then
+  if [ "${n:-0}" -ge "${MAX_PR_AUTOMATION_CYCLES:-50}" ]; then
     return 0
   fi
   return 1
 }
 
 # gather_pr_automation_events <pr> <repo>
-#   Assemble {when, login}[] from the PR's commits, issue comments, and reviews.
+#   Assemble {when, login}[] from the PR's commits, issue comments, and reviews,
+#   with the `agent_marked` and `mechanical` flags the header describes (#2224).
+#   Bodies are tested here and not carried in the result.
 #   Exits 1 (failure) when any GitHub API call fails so callers can distinguish
 #   an unavailable result from a genuinely empty event list — critical for the
 #   fail-closed budget gate in pr_resume_suppressed. Each gh call is captured
 #   separately from jq so gh failures are not masked by jq succeeding on
-#   empty/partial input.
+#   empty/partial input. The PR-author lookup is the exception: without it no
+#   event is flagged `mechanical`, which is the count as it was before #2224,
+#   so its failure is not an error.
 gather_pr_automation_events() {
   local pr="$1" repo="$2"
   local comments commits reviews api_error=0 _raw
+  local mark_defs="$_PR_BUDGET_JQ_DEFS"'
+    def flagged($re): if (.body | agent_marked($re)) then {agent_marked: true} else {} end;'
   if _raw=$(gh api --paginate "repos/${repo}/issues/${pr}/comments?per_page=100" 2>/dev/null); then
-    comments=$(jq -s '[.[][] | {when: .created_at, login: (.user?.login // "")}]' \
+    comments=$(jq -s --arg re "$PR_BUDGET_AGENT_MARKER_RE" "$mark_defs"'
+      [.[][] | {when: .created_at, login: (.user?.login // "")} + flagged($re)]' \
       <<< "$_raw" 2>/dev/null) || comments='[]'
   else
     api_error=1; comments='[]'
@@ -278,7 +368,8 @@ gather_pr_automation_events() {
     api_error=1; commits='[]'
   fi
   if _raw=$(gh api --paginate "repos/${repo}/pulls/${pr}/reviews?per_page=100" 2>/dev/null); then
-    reviews=$(jq -s '[.[][] | {when: .submitted_at, login: (.user?.login // "")}]' \
+    reviews=$(jq -s --arg re "$PR_BUDGET_AGENT_MARKER_RE" "$mark_defs"'
+      [.[][] | {when: .submitted_at, login: (.user?.login // "")} + flagged($re)]' \
       <<< "$_raw" 2>/dev/null) || reviews='[]'
   else
     api_error=1; reviews='[]'
@@ -287,8 +378,35 @@ gather_pr_automation_events() {
     echo "  [warn] budget event collection incomplete for PR ${pr} in ${repo} — API unavailable" >&2
     return 1
   fi
-  jq -n --argjson a "${comments:-[]}" --argjson b "${commits:-[]}" --argjson c "${reviews:-[]}" \
-    '$a + $b + $c' 2>/dev/null || echo '[]'
+  local events pr_author
+  # Merged on stdin: three --argjson values would put a long PR's whole history
+  # on jq's command line.
+  events=$(printf '%s\n%s\n%s\n' "${comments:-[]}" "${commits:-[]}" "${reviews:-[]}" \
+    | jq -c -s 'add // []' 2>/dev/null) || events='[]'
+  pr_author=$(gh api "repos/${repo}/pulls/${pr}" --jq '.user.login // ""' 2>/dev/null) || pr_author=''
+  events=$(pr_budget_flag_mechanical "$events" "$pr_author")
+  _pr_budget_log_flags "$pr" "$repo" "$events"
+  printf '%s\n' "$events"
+}
+
+# _pr_budget_log_flags <pr> <repo> <events_json>
+#   One stderr line per kind of flagged event, so a run log shows why a count is
+#   not what the logins alone suggest. Silent when nothing is flagged.
+_pr_budget_log_flags() {
+  local pr="$1" repo="$2" events_json="${3:-[]}" bots_json n_mech n_marked
+  bots_json=$(printf '%s' "${AUTOMATION_BOT_LOGINS}" \
+    | jq -R 'split(" ") | map(select(. != ""))' 2>/dev/null) || bots_json='[]'
+  read -r n_mech n_marked < <(jq -r --argjson bots "$bots_json" "$_PR_BUDGET_JQ_DEFS"'
+    [ ([.[] | select(.mechanical == true)] | length),
+      ([.[] | select(.agent_marked == true and ((.login | is_bot($bots)) | not))] | length) ]
+    | @tsv' <<<"$events_json" 2>/dev/null) || true
+  if [ "${n_mech:-0}" -gt 0 ] 2>/dev/null; then
+    echo "  [budget] PR #${pr} in ${repo}: ${n_mech} Dependabot-pipeline event(s) left out of the automation budget (#2224)" >&2
+  fi
+  if [ "${n_marked:-0}" -gt 0 ] 2>/dev/null; then
+    echo "  [budget] PR #${pr} in ${repo}: ${n_marked} marker-carrying agent event(s) under a human login counted as automated (#2224)" >&2
+  fi
+  return 0
 }
 
 # pr_automation_already_escalated <pr> <repo>

scripts/lib/pr-runaway-detect.sh

@@ -32,7 +32,7 @@ set -euo pipefail
 # Soft thresholds — a PR is a candidate when it exceeds ANY of these.
 : "${RUNAWAY_MAX_COMMITS:=50}"       # > this many commits
 : "${RUNAWAY_MAX_COMMENTS:=200}"     # > this many issue comments
-: "${RUNAWAY_MAX_CYCLES:=10}"        # > this many automated cycles (see #926)
+: "${RUNAWAY_MAX_CYCLES:=25}"        # > this many automated cycles: half the #926 cap, p95 of PRs (#2224)
 : "${RUNAWAY_MIN_AGE_HOURS:=48}"     # open longer than this, AND churning
 : "${RUNAWAY_CHURN_MIN_CYCLES:=1}"   # min cycles that count as "active churn"
 
@@ -80,7 +80,7 @@ pr_runaway_reasons() {
   local max_commits max_comments max_cycles min_age churn_floor
   max_commits=$(_runaway_threshold "${RUNAWAY_MAX_COMMITS}" 50)
   max_comments=$(_runaway_threshold "${RUNAWAY_MAX_COMMENTS}" 200)
-  max_cycles=$(_runaway_threshold "${RUNAWAY_MAX_CYCLES}" 10)
+  max_cycles=$(_runaway_threshold "${RUNAWAY_MAX_CYCLES}" 25)
   min_age=$(_runaway_threshold "${RUNAWAY_MIN_AGE_HOURS}" 48)
   churn_floor=$(_runaway_threshold "${RUNAWAY_CHURN_MIN_CYCLES}" 1)
 
@@ -154,7 +154,7 @@ generate_runaway_report() {
   local max_commits max_comments max_cycles min_age smell_commits smell_files
   max_commits=$(_runaway_threshold "${RUNAWAY_MAX_COMMITS}" 50)
   max_comments=$(_runaway_threshold "${RUNAWAY_MAX_COMMENTS}" 200)
-  max_cycles=$(_runaway_threshold "${RUNAWAY_MAX_CYCLES}" 10)
+  max_cycles=$(_runaway_threshold "${RUNAWAY_MAX_CYCLES}" 25)
   min_age=$(_runaway_threshold "${RUNAWAY_MIN_AGE_HOURS}" 48)
   smell_commits=$(_runaway_threshold "${REBASE_SMELL_MIN_COMMITS}" 50)
   smell_files=$(_runaway_threshold "${REBASE_SMELL_MAX_FILES}" 20)

tests/dev-lead/unit/test_dev_lead_resume.bats

@@ -68,8 +68,8 @@ teardown() {
 
 # ── (b) an exhausted budget suppresses the resume ─────────────────────────────
 @test "resume: exhausted per-PR automation budget suppresses the resume" {
-  # 10 bot commits since the last human interaction → budget exhausted (#926).
-  COMMITS_JSON="$(jq -nc '[range(10) | {commit:{author:{date:"2026-08-08T00:00:00Z"}}, author:{login:"donpetry-bot"}}]')"
+  # 50 bot commits since the last human interaction → budget exhausted (#926; cap set in #2224).
+  COMMITS_JSON="$(jq -nc '[range(50) | {commit:{author:{date:"2026-08-08T00:00:00Z"}}, author:{login:"donpetry-bot"}}]')"
 
   run resume_main
 

tests/dev-lead/unit/test_dev_lead_retry_pr_escalation.bats

@@ -82,12 +82,12 @@ teardown() {
 }
 
 @test "retry PR: exhausted automation budget (no label) → skips (no dispatch)" {
-  # No needs-human-review label, but 10 bot commits since the last human
+  # No needs-human-review label, but 50 bot commits since the last human
   # interaction → the per-PR automation budget is exhausted (#926). The
   # safety-net cron must consult the budget, not just the label, so it can
   # never re-ignite the #860 amplifier (#1407 AC #3).
   LABELS_JSON='["bug"]'
-  COMMITS_JSON="$(jq -nc '[range(10) | {commit:{author:{date:"2026-06-18T00:00:00Z"}}, author:{login:"donpetry-bot"}}]')"
+  COMMITS_JSON="$(jq -nc '[range(50) | {commit:{author:{date:"2026-06-18T00:00:00Z"}}, author:{login:"donpetry-bot"}}]')"
 
   run scan_pr_for_rate_limits "petry-projects/demo" 860
 

tests/test_persona_pr_pregate.bats

@@ -149,8 +149,8 @@ STUB
 }
 
 @test "skip: budget-exhausted (>= MAX automated actions since last human)" {
-  # 10 bot comments, no human -> compute_pr_automation_cycles >= MAX (10).
-  bots="$(jq -n '[range(10) | {created_at: "2026-09-20T00:00:0\(.)Z", user: {login: "github-actions[bot]"}}]')"
+  # 50 bot comments, no human -> compute_pr_automation_cycles >= MAX (50, #2224).
+  bots="$(jq -n '[range(50) | {created_at: "2026-09-20T00:00:\(. + 100 | tostring | .[1:])Z", user: {login: "github-actions[bot]"}}]')"
   STUB_RAW_COMMENTS="$bots" \
     run persona_event_pregate qa-lead pull_request petry-projects/.github-private 5 opened
   [ "$status" -eq 1 ]