diff --git a/.changeset/retire-thumbs-followup.md b/.changeset/retire-thumbs-followup.md new file mode 100644 index 00000000..a81fc426 --- /dev/null +++ b/.changeset/retire-thumbs-followup.md @@ -0,0 +1,5 @@ +--- +"review": minor +--- + +Delete the thumbs sweep entirely (`thumbs-sweep.ts`, `thumbs-sweep-github.ts`, `run-thumbs-sweep.ts`, and their tests), along with its `octokit` dependency; the lib scripts consumers run are dependency-free again. The 2026-08-20 version audit measured 2 reason replies across the 31 "why?" follow-ups ever posted, 26 of them landing as bursts on one PR, each follow-up also registered as an implicit empty COMMENTED review event that pollutes run counts and the PR timeline, and nobody consumed the read-side tallies. Nothing depends on the sweep: a bare thumbs-down has adjudicated the finding directly since v1.17.0 (staging reads thread-opener reactions itself, identity-filtered), and feedback reporting is done manually today via the `review-feedback-audit` skill (the reconciler and claim validation still read thread replies on re-review; no automated job turns them into feedback reports, the weekly feedback report is the planned instrument). Consumer repos should delete their `review-feedback.yml` sweep workflow when bumping; the `review-counters.yml` weekly counters workflow is unaffected. Anyone counting review runs should key on the v1.14.0+ version footer rather than review events: the autofix workflow's thread replies still carry the implicit-review shape. diff --git a/.claude/skills/review-feedback-audit/SKILL.md b/.claude/skills/review-feedback-audit/SKILL.md index 987508a1..b2770983 100644 --- a/.claude/skills/review-feedback-audit/SKILL.md +++ b/.claude/skills/review-feedback-audit/SKILL.md @@ -1,6 +1,6 @@ --- name: review-feedback-audit -description: Audit the PR review workflow's posted output and human feedback in a consumer repo over a time window (typically since a reviewer deploy). Computes volume, verdict mix, label mix, verbosity, duplication at three grains, suppression notes, and human feedback (reactions, thumbs-sweep replies, thread replies), then maps each finding to an open Khan/actions PR or a new-PR candidate. Invoke with a consumer repo, a cutoff timestamp, and optionally the bot login. +description: Audit the PR review workflow's posted output and human feedback in a consumer repo over a time window (typically since a reviewer deploy). Computes volume, verdict mix, label mix, verbosity, duplication at three grains, suppression notes, and human feedback (reactions, thread replies, and, in windows predating the follow-up retirement, thumbs-sweep follow-up replies), then maps each finding to an open Khan/actions PR or a new-PR candidate. Invoke with a consumer repo, a cutoff timestamp, and optionally the bot login. --- # Review quality and feedback audit @@ -49,7 +49,8 @@ jq --arg bot "$BOT" --arg c "$CUTOFF" \ "$WORK/pc.json" > "$WORK/bot_inline_raw.json" ``` -**Issue comments** (guidance comments, sweep follow-ups, CI noise). The +**Issue comments** (guidance comments, CI noise, and, in windows predating +the follow-up retirement, sweep follow-ups). The reviewer's guidance comment is identified by the engine-appended `gh-aw-agentic-workflow` marker, not by any bot-authored marker: the ingest sanitizer strips agent-written HTML comments (see Known constraints). @@ -80,13 +81,15 @@ while read -r pr; do gh api --paginate "repos/$REPO/pulls/$pr/reviews?per_page=100" \ --jq "[.[] | select(.user.login==\"$BOT\" and .submitted_at >= \"$CUTOFF\")] - | .[] | [$pr, .id, .state, .submitted_at, (.body|length)] | @tsv" \ + | .[] | [$pr, .id, .state, .submitted_at, (.body|length), + (.body|test(\"review-v[0-9]\"))] | @tsv" \ < /dev/null >> "$WORK/reviews.tsv" done < "$WORK/prs.txt" ``` Redirecting stdin from `/dev/null` inside the loop matters: `gh` can consume -the loop's stdin and truncate the PR list. +the loop's stdin and truncate the PR list. The last column marks rows whose +body carries the v1.14.0+ version footer; Step 3's run count keys on it. **Reactions.** The listing's `reactions` object gives counts; fetch the detail endpoint only for comments with `total_count > 0`, and exclude the @@ -150,8 +153,10 @@ Every later step reads `bot_replies.tsv`; `replies.tsv` is an intermediate. **Conventional-Comment labels.** Parse the label prefix off each bot body: `issue`, `suggestion`, `question`, `note`, `nitpick`, `thought`, plus variants like `(non-blocking, documentation)` and -`(non-blocking, best-practice)`. Bodies with no label are usually -thumbs-sweep follow-ups (next paragraph): +`(non-blocking, best-practice)`. In windows predating the follow-up +retirement, bodies with no label are usually thumbs-sweep follow-ups (next +paragraph); in windows after it, an unlabeled bot body is an anomaly worth +reading rather than bucketing: ```sh jq '[.[] | {id, pr: (.pull_request_url|split("/")|last|tonumber), path, @@ -163,25 +168,43 @@ jq '[.[] | {id, pr: (.pull_request_url|split("/")|last|tonumber), path, "$WORK/bot_inline_raw.json" > "$WORK/bot_inline.json" ``` -**Thumbs-sweep follow-ups** carry the `review-thumbs-followup` marker (a -sweep-posted comment survives the sanitizer because the sweep posts through -the plain API, not through safe outputs). Count them separately from -findings, and note that a follow-up posted as a review shows up in -`reviews.tsv` as a `COMMENTED` review; do not count it as a review run. - -**Reason replies** use the closed vocabulary the follow-up offers: -`incorrect`, `unimportant`, `unclear`, `duplicate`. Match replies to -follow-ups by thread and record the latency from downvote to follow-up and -from follow-up to reply. +**Thumbs-sweep follow-ups (historical only).** The sweep's "why?" follow-up +was retired and the sweep itself then deleted entirely (see the +`workflows/review` CHANGELOG entries for both); PRs reviewed after the +retirement release never carry follow-ups. +When the audit window predates the retirement, follow-ups carry the +`review-thumbs-followup` marker (a sweep-posted comment survived the +sanitizer because the sweep posted through the plain API, not through safe +outputs). Count them separately from findings, and note that a follow-up +posted as a review shows up in `reviews.tsv` as a `COMMENTED` review; the +run-count rule in Step 3 keeps it out of run totals. + +**Reason replies (historical only).** Follow-ups offered a closed +vocabulary: `incorrect`, `unimportant`, `unclear`, `duplicate`. It survives +in code only as the permanently-unpopulated `DownvoteReason` type in +`workflows/review/eval/judge.ts`, kept to type historical labels. For +pre-retirement windows, match replies to follow-ups by thread and record +the latency from downvote to follow-up and from follow-up to reply; skip +both metrics for windows after the retirement (there is nothing to measure). **Human replies**: classify each thread's outcome by reading the exchange: `accepted` (author changed code or agreed), `declined` (author rejected with a reason), `answered` (bot asked, author answered, no change requested). +Thread replies reach a maintainer only through this skill or manual reading; +no automated job consumes them (the weekly report is `counters-report.ts` +and has no reply handling), so treat an unaddressed reply as unseen, not +triaged. ## Step 3: compute the metrics -- **Runs and verdicts**: rows of `reviews.tsv` minus sweep follow-ups; - verdict mix (`APPROVED` / `CHANGES_REQUESTED` / `COMMENTED`). +- **Runs and verdicts**: count review runs by the v1.14.0+ version footer + (the `review-v` segment in the review body's collapsed + `
` block), not by review events: the autofix workflow's thread + replies arrive as implicit empty `COMMENTED` review events (as did sweep + follow-ups, pre-retirement), so a raw `reviews.tsv` row count overstates + runs. Verdict mix (`APPROVED` / `CHANGES_REQUESTED` / `COMMENTED`) over + the footer-bearing rows. For windows predating v1.14.0 no footer exists; + fall back to review events minus sweep follow-ups and say so in Caveats. - **Volume**: bot inline comments per PR and per run; guidance comments. - **Label mix**: count per label; blocking vs non-blocking split. - **Verbosity**: mean / median / p90 / max body chars over top-level bot @@ -210,8 +233,9 @@ jq '[.[] | select(.body - **Suppression notes**, parsed from review bodies: `not re-posted (already tracked)`, `shed under the ... run budget`, and the `N of M prior review threads` accountability lines. These show which mitigations fired. -- **Feedback**: reactions by kind and reactor; sweep follow-up latency; - reason-reply latency; human replies by outcome class. +- **Feedback**: reactions by kind and reactor; human replies by outcome + class; for pre-retirement windows only, sweep follow-up latency and + reason-reply latency. - **Attribution**: check each posted review body and guidance comment for a version marker or footer, and report presence per body. @@ -241,13 +265,16 @@ jq '[.[] | select(.body a lower bound. - **Sentinel strings live in lib code.** The markers this audit greps for are defined in Khan/actions source, and a zero count is indistinguishable - from a renamed marker: `review-thumbs-followup` and the reason vocabulary - in `workflows/review/lib/thumbs-sweep.ts`, "A sketch, not a committable - replacement" in `workflows/review/lib/submission.ts`, the suppression - note phrasing in `workflows/review/lib/dispatch.ts`, and the - `gh-aw-agentic-workflow` marker appended by the gh-aw engine. Before - trusting any zero measurement, re-derive the string from the checkout - being audited. + from a renamed marker: "A sketch, not a committable replacement" in + `workflows/review/lib/submission.ts`, the suppression note phrasing in + `workflows/review/lib/dispatch.ts`, and the `gh-aw-agentic-workflow` + marker appended by the gh-aw engine. Before trusting any zero + measurement, re-derive the string from the checkout being audited. The + `review-thumbs-followup` marker and the downvote-reason vocabulary were + deleted from the lib with the follow-up retirement; when auditing a + pre-retirement window, re-derive them from the `workflows/review` + CHANGELOG entry for the retirement or from a pre-retirement tag's + `lib/thumbs-sweep.ts`. ## Step 4: report @@ -260,7 +287,8 @@ Use these sections, in order: 4. **Verbosity**: the stats above, sketch-block share, review-body and guidance-comment sizes. 5. **Duplication**: one subsection per grain, with ids. -6. **Human feedback**: reactions, sweep loop latencies, replies by outcome. +6. **Human feedback**: reactions, replies by outcome, and (pre-retirement + windows only) sweep follow-up loop latencies. 7. **Caveats**: GraphQL availability, marker expectations, sample-size limits, anything unverifiable. diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 14d14744..eaa57b42 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -120,9 +120,6 @@ importers: '@anthropic-ai/claude-agent-sdk': specifier: 0.3.205 version: 0.3.205(@anthropic-ai/sdk@0.110.0(zod@4.4.3))(@modelcontextprotocol/sdk@1.29.0(zod@4.4.3))(zod@4.4.3) - octokit: - specifier: 5.0.5 - version: 5.0.5 zod: specifier: 4.4.3 version: 4.4.3 diff --git a/workflows/autofix/README.md b/workflows/autofix/README.md index e71f8d7a..07eabf40 100644 --- a/workflows/autofix/README.md +++ b/workflows/autofix/README.md @@ -435,14 +435,13 @@ by the command at all; the label is the only surface available to it. Per-comment triggering was considered and dropped for v1. GitHub emits **no webhook for reactions** โ€” the feature request has been open since 2022 โ€” which -is why the review workflow's own thumbs sweep is a two-hourly cron. A -reaction-triggered autofix would inherit that latency, or need a second poll to -shave a delay it still could not bound. `pull_request: labeled` and +is why the review workflow's since-deleted thumbs sweep had to be a two-hourly +cron. A reaction-triggered autofix would inherit that latency, or need its own +poll to shave a delay it still could not bound. `pull_request: labeled` and `issue_comment: created` both fire immediately. -Note also that ๐Ÿš€ is already live signal: `thumbs-sweep.ts` counts it as a -positive reaction feeding the reviewer's tuning loop, so overloading it would -corrupt that channel. +Note also that ๐Ÿš€ is already live signal: gh-aw's outcome evaluation counts it +as a positive reaction, so overloading it would corrupt that channel. The command surface makes per-comment autofix nearly free when it lands: an `/autofix` posted as a **reply inside a review thread** fires diff --git a/workflows/review/README.md b/workflows/review/README.md index 3d6818e1..0c52db15 100644 --- a/workflows/review/README.md +++ b/workflows/review/README.md @@ -149,14 +149,15 @@ you can pick the one that says what you mean: do not join the corpus; a fixed defect that reappears is a fresh finding. - **๐Ÿ‘Ž the finding's comment.** Same adjudication as resolving, through the reaction channel: a ๐Ÿ‘Ž on a thread's OPENING comment puts its defect in the - adjudicated corpus whether or not you also resolve. The feedback sweep may - additionally ask one follow-up ("why?"), which calibrates the eval suite; - answering it is welcome but the ๐Ÿ‘Ž alone is what suppresses. Reactions on - replies are conversation, not adjudication. ๐Ÿ‘Ž is the ONLY adjudicating - reaction: a ๐Ÿ˜• triggers the sweep's follow-up question like a ๐Ÿ‘Ž does, but - it does not suppress (๐Ÿ˜• reads as "unclear", not "wrong", and ambiguity is - worth a question, not a standing suppression). The bot's own seeded nudge - reactions never count as adjudication either. + adjudicated corpus whether or not you also resolve. The ๐Ÿ‘Ž alone is what + suppresses; if you want to say why, reply in the thread: the reconciler and + claim validation read replies as described above, nothing prompts you for a + reason any more, and replies only reach a feedback report when a maintainer + runs the `review-feedback-audit` skill. Reactions on replies are + conversation, not adjudication. ๐Ÿ‘Ž is the ONLY adjudicating reaction: a ๐Ÿ˜• + does not suppress (๐Ÿ˜• reads as "unclear", not "wrong", and ambiguity is + worth a conversation, not a standing suppression). The bot's own seeded + nudge reactions never count as adjudication either. - **Hide the comment.** Reads as nothing. The reviewer does not see hidden state; resolve or ๐Ÿ‘Ž instead. @@ -709,42 +710,28 @@ analysis, and a refused security lens would be a silent coverage hole. Any further per-role promotion (or Sonnet step-down) earns its line through its own eval-suite arm. -### Feedback signal: thumbs sweep and live counters - -Two small scheduled workflows in each consumer repo turn on the tuning loop's -production signal. Both are plain GitHub Actions YAML (not gh-aw), both check -out this repo at the pinned `review-v*` tag and run lib scripts with -`npx -y tsx`, and neither touches review semantics: - -- **Thumbs sweep** (`lib/run-thumbs-sweep.ts`, every 1-2 hours): collects - reactions on the reviewer's comments at both grains (inline review comments, - identified by the code-owned Conventional-Comment label prefixes; the - risks/patterns summary comment, identified by its hidden marker) and posts - exactly one "why?" follow-up per newly-downvoted comment, offering the closed - reason vocabulary (`incorrect` / `unimportant` / `unclear` / `duplicate`). - Reactions are tallied with the same sets gh-aw's outcome evaluation uses - (๐Ÿ‘/โค๏ธ/๐ŸŽ‰/๐Ÿš€ positive, ๐Ÿ‘Ž/๐Ÿ˜• negative; a ๐Ÿ˜• triggers the follow-up like a ๐Ÿ‘Ž), - and resolved inline threads are counted as their own positive column: threads - also get resolved just to clear noise, so resolution is reported alongside - the reaction tallies rather than folded into them. Idempotent across restarts - via the hidden follow-up markers; bounded to PRs updated in the last 14 days - (`REVIEW_SWEEP_LOOKBACK_DAYS`), skipping PRs closed or merged more than 3 - days ago (`REVIEW_SWEEP_CLOSED_GRACE_DAYS`; feedback lands around merge time, - after which a landed PR stops changing). Needs only `pull-requests: write`. - The sweep run needs `npm ci --omit=dev` in the checked-out - `workflows/review/` first (the sweep's `octokit` dependency is pinned exactly - in `package.json`, with the transitive tree locked by the committed - `package-lock.json`); the other lib scripts remain dependency-free. Each run's - `SweepResult` and API-request count land in the job summary. +### Feedback signal: live counters + +One small scheduled workflow in each consumer repo turns on the tuning loop's +production signal. It is plain GitHub Actions YAML (not gh-aw), checks out +this repo at the pinned `review-v*` tag, runs a lib script with `npx -y tsx`, +and never touches review semantics: + - **Live counters** (`lib/counters-report.ts`, weekly): the workflow downloads the review runs' per-run artifacts (bounded window), and the script aggregates them with `lib/counters.ts` into the job summary โ€” verdict mix, - comments/run, validator drop rate, cost/run. Needs only `actions: read`. - -The reviewer posts as `github-actions[bot]` (gh-aw safe outputs use the -workflow's own token), so that login is both the sweep's `botLogin` filter and -the author of its follow-ups; every count in the sweep excludes that login's -own reactions, so the seeded nudge pair (below) is never live signal. + comments/run, validator drop rate, cost/run. Needs only `actions: read`, + and no `npm ci`: the lib scripts consumers run are dependency-free. + +There used to be a second workflow here, the thumbs sweep (a 2-hourly poll +that tallied reactions on the reviewer's comments and posted a "why?" +follow-up per newly-downvoted comment). Both halves are retired: the +2026-08-20 audit measured 2 reason replies across the 31 follow-ups ever +posted, each follow-up also registered as an implicit empty review event, and +nobody consumed the read-side tallies. A bare ๐Ÿ‘Ž adjudicates directly since +v1.17.0 (the staging reads thread-opener reactions itself, excluding the +bot's own seeded nudges), so no scheduled collector is needed for feedback to +act on the reviewer. ### Relationship to the gh-aw outcome-collector @@ -753,29 +740,28 @@ classifies every agentic safe output as accepted / rejected / ignored / pending and exports the results to Sentry over OTLP. The two systems answer different questions and neither replaces the other: -- **Outcome-collector**: passive fleet-wide acceptance telemetry. It never - writes to GitHub, so it can observe engagement but cannot ask *why* a - comment was downvoted. Its data lives in Sentry. -- **Thumbs sweep**: active reason elicitation for the reviewer's tuning loop. - Its "why?" follow-ups produce the closed reason labels that calibrate the - eval-suite judge and feed dismissal learning. Its data lives in each run's - job summary and stdout JSON (not exported to OTel today). +- **Outcome-collector**: passive fleet-wide acceptance telemetry across every + agentic workflow. Its data lives in Sentry. It counts any reaction with no + reactor identity, so it cannot exclude the seeded nudges. +- **Reviewer-side signal**: the adjudication path reads thread-opener + reactions identity-filtered at review time, and the live-counters report + aggregates the per-run artifacts. Neither is exported to OTel today. Two known interactions: - **Nudge seeding** is planned as a post-time step in the consumer repos' review workflow (a custom safe-output job that reacts ๐Ÿ‘/๐Ÿ‘Ž to each posted - comment seconds after posting), not in the sweep: gh-aw cannot react to its - own safe outputs natively, and comments posted via `GITHUB_TOKEN` emit no - workflow events, so post-time is the only immediate option. + comment seconds after posting): gh-aw cannot react to its own safe outputs + natively, and comments posted via `GITHUB_TOKEN` emit no workflow events, + so post-time is the only immediate option. - Once seeding is live, the outcome-collector's `add_comment` metric for the review workflow is **inflated by design**: its evaluator counts any reaction as acceptance with no reactor identity, so every seeded summary comment reads as `accepted`. The inflation is bounded to that one metric (inline - comments and submitted reviews are evaluated by other means), and the sweep's - identity-filtered tallies are the authoritative reviewer-comment engagement - numbers. An upstream gh-aw change to identity-aware reaction counting would - retire this caveat. + comments and submitted reviews are evaluated by other means); the + adjudication path's identity-filtered reads are the authoritative + reviewer-comment engagement signal. An upstream gh-aw change to + identity-aware reaction counting would retire this caveat. ### Required secrets / variables diff --git a/workflows/review/eval/judge.ts b/workflows/review/eval/judge.ts index 0a737c3c..7bd8bac7 100644 --- a/workflows/review/eval/judge.ts +++ b/workflows/review/eval/judge.ts @@ -1,7 +1,7 @@ /** * LLM-judge: an Opus-4.8 judge that scores the *quality* of the * comments a run posted, a human-audit sample surfaced from its output, and a - * calibration pass against the thumbs-sweep labels. + * calibration pass against the (historical) thumbs-sweep labels. * * Why a judge at all: the deterministic metrics (recall/precision/noise) score * whether the reviewer posted the *right findings* against corpus ground truth. @@ -265,16 +265,12 @@ export const selectAuditSample = ( /* -------------------------------------------------------------------------- */ /** - * The fixed downvote-reason vocabulary the thumbs sweep offers on a ๐Ÿ‘Ž. - * - * Declared locally rather than imported from `../lib/thumbs-sweep` on purpose: - * the judge consumes thumbs labels as *data* (see {@link ThumbsLabel}) and never - * needs the sweep module at build time, so importing its type would create a - * build dependency on the thumbs sweep for a field this module only carries - * through (calibration keys off `direction`, not `reason`). This union is - * structurally identical to the thumbs sweep's `DownvoteReason`, so a value produced there - * is assignable here and vice versa; keep the two in sync if the sweep's - * vocabulary changes. + * The downvote-reason vocabulary the thumbs sweep USED to elicit via its + * "why?" follow-up, retired with that surface (the sweep no longer defines or + * produces it). Retained only to type historical thumbs labels: calibration + * keys off `direction`, so `reason` is carry-through data and stays + * permanently unpopulated for labels mined after the retirement. There is no + * longer a sweep-side counterpart to keep in sync with. */ export type DownvoteReason = | "incorrect" @@ -283,9 +279,10 @@ export type DownvoteReason = | "duplicate"; /** - * A human thumbs signal on a posted comment, mined by the thumbs sweep. `up` - * means ๐Ÿ‘ (the human agreed with the comment), `down` means ๐Ÿ‘Ž (disagreed); - * `reason` is the sweep's fixed downvote vocabulary when the human gave one. + * A human thumbs signal on a posted comment, mined by the (now deleted) + * thumbs sweep. `up` means ๐Ÿ‘ (the human agreed with the comment), `down` + * means ๐Ÿ‘Ž (disagreed); `reason` is set only on labels mined before the + * "why?" follow-up was retired (see {@link DownvoteReason}). */ export type ThumbsLabel = { findingId: string; diff --git a/workflows/review/eval/metrics.ts b/workflows/review/eval/metrics.ts index bfe83357..fb23f826 100644 --- a/workflows/review/eval/metrics.ts +++ b/workflows/review/eval/metrics.ts @@ -21,8 +21,9 @@ * 5. **calibration** โ€” do the model's `confidence` numbers mean anything? We * bucket posted findings by confidence and compare each bucket's mean * confidence to its empirical correctness, reporting the expected - * calibration error (ECE). The thumbs-sweep labels calibrate the same axis - * at runtime; here we measure it against the corpus ground truth. + * calibration error (ECE). The historical thumbs-sweep labels calibrated + * the same axis at runtime; here we measure it against the corpus ground + * truth. * * Determinism boundary: this module reads structured findings and * case expectations and emits numbers. It authors no prose about code under diff --git a/workflows/review/lib/counters-report.test.ts b/workflows/review/lib/counters-report.test.ts index f993714a..dd0d0f01 100644 --- a/workflows/review/lib/counters-report.test.ts +++ b/workflows/review/lib/counters-report.test.ts @@ -225,7 +225,9 @@ describe("renderCountersMarkdown", () => { expect(markdown).toContain("*1 run(s) submitted no review"); expect(markdown).toContain("| (unknown) | 2 | 1 | 50.0% |"); expect(markdown).toContain("$4.20 total"); - expect(markdown).toContain("thumbs-sweep run's job summary"); + expect(markdown).toContain( + "Thumbs: none recorded in per-run artifacts", + ); }); }); diff --git a/workflows/review/lib/counters-report.ts b/workflows/review/lib/counters-report.ts index 5d4ba5fd..154353be 100644 --- a/workflows/review/lib/counters-report.ts +++ b/workflows/review/lib/counters-report.ts @@ -327,8 +327,7 @@ export const renderCountersMarkdown = ( lines.push("", "### Thumbs and cost", ""); lines.push( counters.thumbs.agreeRate === null - ? "- Thumbs: none recorded in per-run artifacts (live tallies are" + - " reported by each thumbs-sweep run's job summary)" + ? "- Thumbs: none recorded in per-run artifacts" : `- Thumbs: ${counters.thumbs.up} ๐Ÿ‘ / ${counters.thumbs.down} ๐Ÿ‘Ž` + ` (agree rate ${pct(counters.thumbs.agreeRate)})`, ); diff --git a/workflows/review/lib/counters.ts b/workflows/review/lib/counters.ts index c37acb9d..558e2b9d 100644 --- a/workflows/review/lib/counters.ts +++ b/workflows/review/lib/counters.ts @@ -10,7 +10,7 @@ * source; * - the run summary (verdict, posted-comment count, model cost) -> comments/PR, * verdict mix, cost/run; - * - the thumbs reactions the thumbs sweep collects -> thumbs agree rate. + * - the thumbs reactions recorded in per-run artifacts -> thumbs agree rate. * * The module is split into a pure core and a thin, best-effort filesystem * loader. The core ({@link computeRunCounters}, {@link normalizeRunArtifacts}) @@ -48,7 +48,7 @@ export type ValidatorDecision = { decision: "keep" | "drop"; }; -/** Thumbs reactions collected on a run's comments (thumbs sweep). */ +/** Thumbs reactions collected on a run's comments. */ export type ThumbsTally = { /** ๐Ÿ‘ count โ€” a human agreed with the bot's comment. */ up: number; diff --git a/workflows/review/lib/run-thumbs-sweep.test.ts b/workflows/review/lib/run-thumbs-sweep.test.ts deleted file mode 100644 index 835b6fad..00000000 --- a/workflows/review/lib/run-thumbs-sweep.test.ts +++ /dev/null @@ -1,77 +0,0 @@ -import {describe, it, expect} from "vitest"; - -import {renderSweepSummary} from "./run-thumbs-sweep.ts"; -import type {SweepResult} from "./thumbs-sweep.ts"; -import type {SweepTraversalStats} from "./thumbs-sweep-github.ts"; - -const stats: SweepTraversalStats = { - pullsScanned: 4, - apiRequests: 31, - reactions: {positive: 5, negative: 2}, - resolvedInlineThreads: 3, -}; - -const result: SweepResult = { - actions: [ - { - grain: "inline", - commentId: 11, - downvotes: 0, - posted: false, - reason: "no-downvote", - }, - { - grain: "summary", - commentId: 22, - downvotes: 2, - posted: true, - reason: "posted", - }, - { - grain: "inline", - commentId: 33, - downvotes: 1, - posted: false, - reason: "already-followed-up", - }, - ], - followupsPosted: 1, -}; - -describe("renderSweepSummary", () => { - it("renders the tallies and the downvoted-comment table", () => { - const markdown = renderSweepSummary(result, stats); - - expect(markdown).toContain("## Thumbs feedback sweep"); - expect(markdown).not.toContain("DRY RUN"); - expect(markdown).toContain( - "Reviewer comments swept: **3** across 4 recently-active PRs", - ); - expect(markdown).toContain("**5 positive / 2 negative**"); - expect(markdown).toContain("threads resolved: **3**"); - expect(markdown).toContain( - "Follow-ups posted this sweep: **1** (already followed up: 1)", - ); - expect(markdown).toContain("API requests used: 31"); - // Only the two downvoted comments appear in the table. - expect(markdown).toContain("| summary | 22 | 2 | posted |"); - expect(markdown).toContain("| inline | 33 | 1 | already-followed-up |"); - expect(markdown).not.toContain("| inline | 11 |"); - }); - - it("marks a dry run in the header", () => { - const markdown = renderSweepSummary(result, stats, {dryRun: true}); - expect(markdown).toContain( - "## Thumbs feedback sweep (DRY RUN โ€” nothing was posted)", - ); - }); - - it("omits the table when nothing is downvoted", () => { - const markdown = renderSweepSummary( - {actions: [], followupsPosted: 0}, - {...stats, reactions: {positive: 0, negative: 0}}, - ); - expect(markdown).toContain("(already followed up: 0)"); - expect(markdown).not.toContain("| Grain |"); - }); -}); diff --git a/workflows/review/lib/run-thumbs-sweep.ts b/workflows/review/lib/run-thumbs-sweep.ts deleted file mode 100644 index 3b2191ea..00000000 --- a/workflows/review/lib/run-thumbs-sweep.ts +++ /dev/null @@ -1,195 +0,0 @@ -/** - * CLI entry for the thumbs feedback sweep โ€” the script the consumer repos' - * scheduled `review-feedback` workflows run: - * - * cd gh-aw-review-lib/workflows/review && npm ci --omit=dev && - * npx -y tsx lib/run-thumbs-sweep.ts - * - * (The `npm ci` supplies `octokit`, this package's one runtime - * dependency โ€” unlike the router/investigation-cap scripts, the sweep talks to - * the GitHub API. It is pinned exactly in `package.json` and its transitive - * tree is locked by the committed `package-lock.json`, so a consumer run - * resolves the same client version this release was tested with.) - * - * All configuration is environment variables, so the consumer workflow is pure - * YAML with no arguments to quote: - * - * GITHUB_TOKEN required; the workflow's token - * (`pull-requests: write` is the only scope the - * sweep needs). - * GITHUB_REPOSITORY `owner/repo`; provided by Actions. - * REVIEW_SWEEP_BOT_LOGIN login the reviewer posts as - * (default `github-actions[bot]`). - * REVIEW_SWEEP_LOOKBACK_DAYS PR-activity window (default 14). - * REVIEW_SWEEP_MAX_PULLS traversal cap (default 200). - * REVIEW_SWEEP_CLOSED_GRACE_DAYS days a closed/merged PR stays in the - * sweep after closing (default 3). - * REVIEW_SWEEP_DRY_RUN `true` to traverse and decide without - * posting anything (first-run - * audit; default `false`). - * REVIEW_SWEEP_WORKFLOW_IDS comma-separated gh-aw workflow ids whose - * call-id markers identify the summary - * comment (default `review`). - * - * Output: the full {@link SweepResult} as JSON on stdout, and โ€” when - * `GITHUB_STEP_SUMMARY` is set โ€” a Markdown digest appended to the job summary - * so every sweep run is auditable from the Actions UI. - */ - -import {appendFileSync} from "node:fs"; - -import { - sweepThumbs, - type SweepResult, - type ThumbsSweepPort, -} from "./thumbs-sweep.ts"; -import { - GithubThumbsSweepPort, - type OctokitRequestFn, - type SweepTraversalStats, -} from "./thumbs-sweep-github.ts"; - -const env = (name: string): string | undefined => { - const value = process.env[name]; - return value === undefined || value.trim() === "" ? undefined : value; -}; - -const intEnv = (name: string): number | undefined => { - const raw = env(name); - if (raw === undefined) { - return undefined; - } - const value = Number.parseInt(raw, 10); - if (!Number.isInteger(value) || value <= 0) { - throw new Error(`${name} must be a positive integer, got: ${raw}`); - } - return value; -}; - -/** Render the auditable Markdown digest for the job summary. */ -export const renderSweepSummary = ( - result: SweepResult, - stats: SweepTraversalStats, - options: {dryRun?: boolean} = {}, -): string => { - const byReason = new Map(); - for (const action of result.actions) { - byReason.set(action.reason, (byReason.get(action.reason) ?? 0) + 1); - } - const downvoted = result.actions.filter((a) => a.downvotes > 0); - - const lines = [ - options.dryRun === true - ? "## Thumbs feedback sweep (DRY RUN โ€” nothing was posted)" - : "## Thumbs feedback sweep", - "", - `- Reviewer comments swept: **${result.actions.length}** across ${stats.pullsScanned} recently-active PRs`, - `- Live reactions observed (bot's own excluded; ๐Ÿ‘/โค๏ธ/๐ŸŽ‰/๐Ÿš€ vs ๐Ÿ‘Ž/๐Ÿ˜•): **${stats.reactions.positive} positive / ${stats.reactions.negative} negative**`, - `- Reviewer inline threads resolved: **${stats.resolvedInlineThreads}**`, - `- Follow-ups posted this sweep: **${result.followupsPosted}**` + - ` (already followed up: ${ - byReason.get("already-followed-up") ?? 0 - })`, - `- GitHub API requests used: ${stats.apiRequests}`, - ]; - - if (downvoted.length > 0) { - lines.push( - "", - "| Grain | Comment | ๐Ÿ‘Ž | Action |", - "| --- | --- | --- | --- |", - ); - for (const action of downvoted) { - lines.push( - `| ${action.grain} | ${action.commentId} | ${action.downvotes} | ${action.reason} |`, - ); - } - } - - lines.push(""); - return lines.join("\n"); -}; - -const main = async (): Promise => { - const token = env("GITHUB_TOKEN"); - if (token === undefined) { - throw new Error("GITHUB_TOKEN is required"); - } - const repository = env("GITHUB_REPOSITORY"); - if (repository === undefined || !repository.includes("/")) { - throw new Error("GITHUB_REPOSITORY must be set to owner/repo"); - } - const [owner, repo] = repository.split("/", 2) as [string, string]; - - const botLogin = env("REVIEW_SWEEP_BOT_LOGIN") ?? "github-actions[bot]"; - const lookbackDays = intEnv("REVIEW_SWEEP_LOOKBACK_DAYS"); - const maxPulls = intEnv("REVIEW_SWEEP_MAX_PULLS"); - const closedGraceDays = intEnv("REVIEW_SWEEP_CLOSED_GRACE_DAYS"); - const dryRun = env("REVIEW_SWEEP_DRY_RUN") === "true"; - const reviewWorkflowIds = (env("REVIEW_SWEEP_WORKFLOW_IDS") ?? "review") - .split(",") - .map((id) => id.trim()) - .filter((id) => id !== ""); - - // `Octokit` from the `octokit` package ships the throttling/retry plugins, - // so secondary-rate-limit pauses are handled by the client instead of - // failing the scheduled run. Loaded with a dynamic import because the - // package is ESM-only while tsx runs this script as CJS (no `"type": - // "module"` in this package); `import()` crosses that boundary, a static - // import cannot. - const {Octokit} = await import("octokit"); - const octokit = new Octokit({auth: token}); - const request: OctokitRequestFn = (route, params) => - octokit.request(route, params); - - const port = new GithubThumbsSweepPort(request, { - owner, - repo, - botLogin, - reviewWorkflowIds, - ...(lookbackDays !== undefined ? {lookbackDays} : {}), - ...(maxPulls !== undefined ? {maxPulls} : {}), - ...(closedGraceDays !== undefined ? {closedGraceDays} : {}), - }); - - // Dry-run mode (`REVIEW_SWEEP_DRY_RUN=true`): traverse and decide exactly - // as a real sweep would, but swallow the write call. Useful for a - // first-run audit of what a repo's sweep WOULD post. - const effectivePort: ThumbsSweepPort = dryRun - ? { - listBotComments: (grain) => port.listBotComments(grain), - listExistingFollowups: () => port.listExistingFollowups(), - postFollowup: async () => {}, - } - : port; - - const result = await sweepThumbs(effectivePort, { - owner, - repo, - botLogin, - }); - const stats = port.stats(); - - process.stdout.write( - `${JSON.stringify({dryRun, result, stats}, null, 2)}\n`, - ); - - const summaryPath = env("GITHUB_STEP_SUMMARY"); - if (summaryPath !== undefined) { - appendFileSync( - summaryPath, - renderSweepSummary(result, stats, {dryRun}), - ); - } -}; - -// Run only when invoked directly (`npx tsx lib/run-thumbs-sweep.ts`), never -// on import (tests import `renderSweepSummary`). Same guard as the other lib -// CLIs (`investigation-cap.ts`). -if (typeof require !== "undefined" && require.main === module) { - main().catch((error: unknown) => { - // eslint-disable-next-line no-console - console.error(error); - process.exitCode = 1; - }); -} diff --git a/workflows/review/lib/thumbs-sweep-github.test.ts b/workflows/review/lib/thumbs-sweep-github.test.ts deleted file mode 100644 index 2d42c8bb..00000000 --- a/workflows/review/lib/thumbs-sweep-github.test.ts +++ /dev/null @@ -1,336 +0,0 @@ -import {describe, it, expect} from "vitest"; - -import { - GithubThumbsSweepPort, - INLINE_COMMENT_PREFIXES, - isReviewerInlineBody, - isReviewerSummaryBody, - type OctokitRequestFn, -} from "./thumbs-sweep-github.ts"; -import {buildFollowupMarker, sweepThumbs} from "./thumbs-sweep.ts"; - -/** - * Tests for the octokit-backed port. A fake `request` function dispatches on - * the route template and records writes, so the traversal, the two-grain - * classification, the reaction resolution, and the write routing are all - * exercised without a network. - */ - -const BOT = "github-actions[bot]"; -const NOW = Date.parse("2026-07-08T00:00:00Z"); -const RECENT = "2026-07-07T12:00:00Z"; // inside the 14-day window -const STALE = "2026-05-01T00:00:00Z"; // far outside it -const CLOSED_RECENTLY = "2026-07-07T00:00:00Z"; // inside the 3-day closed grace -const CLOSED_LONG_AGO = "2026-07-03T00:00:00Z"; // past the closed grace - -// Production summary comments carry gh-aw's engine-emitted call-id marker, -// not (yet) the pr-reviewer marker โ€” mirror that shape here. -const SUMMARY_BODY = [ - "## Guidance for reviewers", - "", -].join("\n"); - -const PR_REVIEWER_SUMMARY_BODY = [ - "", - "## Guidance for reviewers", - "", -].join("\n"); - -type RecordedWrite = {route: string; params: Record}; - -/** - * A fake GitHub: one recent PR (#7) carrying reviewer comments at both grains - * plus decoys, and one stale PR (#1) that must never be traversed. - */ -const makeFakeGithub = () => { - const writes: RecordedWrite[] = []; - - const inlineComments = [ - { - id: 101, - user: {login: BOT}, - body: "**issue (blocking):** off-by-one in the prune loop.", - reactions: {total_count: 3}, - }, - { - // The sweep's own earlier follow-up reply: idempotency source, - // never a candidate. - id: 102, - user: {login: BOT}, - body: `${buildFollowupMarker("inline", 999)}\nThanks!`, - reactions: {total_count: 0}, - }, - { - // Bot-authored but not templated like a finding -> ignored. - id: 103, - user: {login: BOT}, - body: "some other workflow's inline note", - reactions: {total_count: 5}, - }, - { - // Human comment -> ignored. - id: 104, - user: {login: "human-dev"}, - body: "**issue (blocking):** looks reviewer-shaped but human.", - reactions: {total_count: 1}, - }, - ]; - - const issueComments = [ - { - id: 201, - user: {login: BOT}, - body: SUMMARY_BODY, - reactions: {total_count: 0}, - }, - { - // Bot-authored, no risks/patterns marker -> another workflow's. - id: 202, - user: {login: BOT}, - body: "Test results: all green", - reactions: {total_count: 3}, - }, - ]; - - const reactionsByComment: Record = { - 101: [ - {content: "-1", user: {login: "human-dev"}}, - {content: "heart", user: {login: "another-dev"}}, // positive set - {content: "+1", user: {login: BOT}}, // the bot's own; must not count - ], - }; - - // PR #7's review threads: the candidate 101's thread is resolved (counts); - // the resolved thread rooted at the human comment 104 must not count. - const reviewThreads = [ - {isResolved: true, comments: {nodes: [{databaseId: 101}]}}, - {isResolved: false, comments: {nodes: [{databaseId: 103}]}}, - {isResolved: true, comments: {nodes: [{databaseId: 104}]}}, - ]; - - const request: OctokitRequestFn = async (route, params = {}) => { - if (route === "GET /repos/{owner}/{repo}/pulls") { - const page = params["page"] as number; - return { - data: - page === 1 - ? [ - {number: 7, updated_at: RECENT, state: "open"}, - { - // Closed within the grace window -> swept. - number: 6, - updated_at: RECENT, - state: "closed", - closed_at: CLOSED_RECENTLY, - }, - { - // Closed past the grace window -> skipped. - number: 5, - updated_at: RECENT, - state: "closed", - closed_at: CLOSED_LONG_AGO, - }, - {number: 1, updated_at: STALE}, - ] - : [], - }; - } - if ( - route === "GET /repos/{owner}/{repo}/pulls/{pull_number}/comments" - ) { - expect([6, 7]).toContain(params["pull_number"]); - const items = - params["pull_number"] === 7 && params["page"] === 1 - ? inlineComments - : []; - return {data: items}; - } - if ( - route === "GET /repos/{owner}/{repo}/issues/{issue_number}/comments" - ) { - expect([6, 7]).toContain(params["issue_number"]); - const items = - params["issue_number"] === 7 && params["page"] === 1 - ? issueComments - : []; - return {data: items}; - } - if (route === "POST /graphql") { - const variables = params["variables"] as Record; - expect(variables["number"]).toBe(7); // only PRs with inline candidates - return { - data: { - data: { - repository: { - pullRequest: { - reviewThreads: { - pageInfo: { - hasNextPage: false, - endCursor: null, - }, - nodes: reviewThreads, - }, - }, - }, - }, - }, - }; - } - if ( - route === - "GET /repos/{owner}/{repo}/pulls/comments/{comment_id}/reactions" - ) { - const id = params["comment_id"] as number; - return { - data: params["page"] === 1 ? reactionsByComment[id] ?? [] : [], - }; - } - if ( - route === - "GET /repos/{owner}/{repo}/issues/comments/{comment_id}/reactions" - ) { - return {data: []}; - } - if (route.startsWith("POST ")) { - writes.push({route, params}); - return {data: {}}; - } - throw new Error(`unexpected route: ${route}`); - }; - - return {request, writes}; -}; - -const makePort = (request: OctokitRequestFn) => - new GithubThumbsSweepPort(request, { - owner: "Khan", - repo: "webapp", - botLogin: BOT, - now: NOW, - }); - -describe("comment identification", () => { - it("recognises every code-owned conventional label as an inline prefix", () => { - expect(INLINE_COMMENT_PREFIXES).toContain("**issue (blocking):**"); - expect(INLINE_COMMENT_PREFIXES).toContain( - "**suggestion (non-blocking, best-practice):**", - ); - expect(isReviewerInlineBody("**todo (blocking):** add the test.")).toBe( - true, - ); - expect(isReviewerInlineBody("regular prose")).toBe(false); - }); - - it("identifies the summary comment by either hidden marker", () => { - // The spec'd pr-reviewer marker matches unconditionally. - expect(isReviewerSummaryBody(PR_REVIEWER_SUMMARY_BODY)).toBe(true); - // The engine-emitted call-id marker matches when configured. - expect(isReviewerSummaryBody(SUMMARY_BODY)).toBe(false); - expect( - isReviewerSummaryBody(SUMMARY_BODY, [ - "", - ]), - ).toBe(true); - expect(isReviewerSummaryBody("Test results: all green")).toBe(false); - }); -}); - -describe("traversal and classification", () => { - it("lists reviewer comments at both grains, excluding decoys and follow-ups", async () => { - const {request} = makeFakeGithub(); - const port = makePort(request); - - const inline = await port.listBotComments("inline"); - expect(inline.map((c) => c.id)).toEqual([101]); - // Reactor logins survive so the sweep can exclude the bot's own. - expect(inline[0]?.reactions).toEqual([ - {content: "-1", user: "human-dev"}, - {content: "heart", user: "another-dev"}, - {content: "+1", user: BOT}, - ]); - - const summary = await port.listBotComments("summary"); - expect(summary.map((c) => c.id)).toEqual([201]); - - const followups = await port.listExistingFollowups(); - expect(followups).toHaveLength(1); - expect(followups[0]).toContain("comment-id=999"); - }); - - it("stays within the lookback + closed-grace windows and reports auditable stats", async () => { - const {request} = makeFakeGithub(); - const port = makePort(request); - await port.listBotComments("inline"); - - const stats = port.stats(); - // The open PR and the recently-closed PR are traversed; the PR closed - // past the grace window and the stale PR are not. - expect(stats.pullsScanned).toBe(2); - // The human ๐Ÿ‘Ž and โค๏ธ count (shared reaction sets); the bot's own ๐Ÿ‘ - // is excluded. - expect(stats.reactions).toEqual({positive: 1, negative: 1}); - // Candidate 101's thread is resolved; the non-candidate threads on the - // same PR are not counted. - expect(stats.resolvedInlineThreads).toBe(1); - expect(stats.apiRequests).toBeGreaterThan(0); - }); -}); - -describe("writes", () => { - it("routes follow-ups per grain (inline reply vs PR comment)", async () => { - const {request, writes} = makeFakeGithub(); - const port = makePort(request); - await port.listBotComments("inline"); // populate the comment->PR map - - await port.postFollowup({ - grain: "inline", - commentId: 101, - body: "why?", - }); - await port.postFollowup({ - grain: "summary", - commentId: 201, - body: "why?", - }); - - expect(writes.map((w) => w.route)).toEqual([ - "POST /repos/{owner}/{repo}/pulls/{pull_number}/comments/{comment_id}/replies", - "POST /repos/{owner}/{repo}/issues/{issue_number}/comments", - ]); - expect(writes[0]?.params["pull_number"]).toBe(7); - expect(writes[0]?.params["comment_id"]).toBe(101); - expect(writes[1]?.params["issue_number"]).toBe(7); - }); - - it("rejects a follow-up for a comment the traversal never saw", async () => { - const {request} = makeFakeGithub(); - const port = makePort(request); - await port.listBotComments("inline"); - await expect( - port.postFollowup({grain: "inline", commentId: 555, body: "?"}), - ).rejects.toThrow(/unknown comment/); - }); -}); - -describe("end-to-end with the sweep core", () => { - it("follows up the human ๐Ÿ‘Ž exactly once", async () => { - const {request, writes} = makeFakeGithub(); - const port = makePort(request); - - const result = await sweepThumbs(port, { - owner: "Khan", - repo: "webapp", - botLogin: BOT, - }); - - // The human ๐Ÿ‘Ž on 101 draws exactly one follow-up, threaded inline. - expect(result.followupsPosted).toBe(1); - const followupWrites = writes.filter((w) => - w.route.includes("replies"), - ); - expect(followupWrites).toHaveLength(1); - expect(String(followupWrites[0]?.params["body"])).toContain( - "review-thumbs-followup grain=inline comment-id=101", - ); - }); -}); diff --git a/workflows/review/lib/thumbs-sweep-github.ts b/workflows/review/lib/thumbs-sweep-github.ts deleted file mode 100644 index 60ca5860..00000000 --- a/workflows/review/lib/thumbs-sweep-github.ts +++ /dev/null @@ -1,685 +0,0 @@ -/** - * The octokit-backed {@link ThumbsSweepPort} โ€” the production GitHub - * implementation of the side-effect boundary `thumbs-sweep.ts` defines. - * - * Division of labour (unchanged from the sweep module): `thumbs-sweep.ts` owns - * all control flow and idempotency; this module owns only the GitHub traversal โ€” - * which comments are the reviewer's, at which grain, with which reactions and - * thread state โ€” and the write call (`postFollowup`). It holds no sweep logic: - * a bug here can mis-list or mis-post, but it cannot re-ping, because that rule - * lives in the sweep core. - * - * How the reviewer's comments are identified (per grain): - * - * - `summary` โ€” issue comments authored by `botLogin` that carry one of the - * workflow's hidden markers: the risks/patterns marker - * (`pr-reviewer:risks-and-patterns`, the exact marker line `review.md` - * Step 7 requires the comment to begin with) or gh-aw's engine-emitted - * `gh-aw-workflow-call-id` marker for the repo's review install (what - * observed production comments actually carry). The marker, not the - * author, is what scopes the sweep to the reviewer: - * `github-actions[bot]` authors many other workflows' comments. - * - `inline` โ€” pull-request review comments authored by `botLogin` whose body - * starts with one of the reviewer's code-owned Conventional-Comment labels - * (`**issue (blocking):** โ€ฆ`, `render-comment.ts`'s taxonomy). Inline - * comments carry no hidden marker at current releases, so the label grammar - * is the identifying signature; it is code-owned and templated, so the match - * is exact, not heuristic prose-sniffing. - * - * Comments containing a thumbs-followup marker are never candidates at either - * grain โ€” they are returned through `listExistingFollowups` instead, which is - * what makes the sweep idempotent across restarts with no state store. - * - * API-call bounding: the traversal reads only pull requests updated within - * `lookbackDays` (default 14), newest first, capped at `maxPulls`; closed or - * merged PRs are skipped once closed for more than `closedGraceDays` (default - * 3) โ€” feedback lands around merge time, and a landed PR's reactions stop - * changing shortly after. Reactions are fetched per comment only when the - * comment's reaction summary shows any reactions at all. Resolved inline - * threads are counted with one GraphQL query per PR that has reviewer inline - * comments (thread resolution is not on the REST comment listing). Every - * request is counted and reported via {@link GithubThumbsSweepPort.stats} so - * each run's API budget is auditable. - */ - -import type { - BotComment, - FeedbackGrain, - PostedFollowup, - Reaction, - ThumbsSweepPort, -} from "./thumbs-sweep.ts"; -import { - NEGATIVE_REACTIONS, - parseFollowupMarkers, - POSITIVE_REACTIONS, -} from "./thumbs-sweep.ts"; -import {BLOCKING_LABELS, NON_BLOCKING_LABELS} from "./render-comment.ts"; - -/** - * The one octokit surface this module needs: `octokit.request`. Kept this - * narrow so tests fake a single function and the module never depends on - * octokit's types; the real client (constructed in `run-thumbs-sweep.ts`) - * satisfies it directly. - */ -export type OctokitRequestFn = ( - route: string, - params?: Record, -) => Promise<{data: unknown}>; - -/** - * The hidden marker that identifies the reviewer's risks/patterns summary - * comment (`review.md` Step 7 requires the comment to begin with this exact - * marker line). Matched as a substring so surrounding whitespace or the - * version marker never break identification. - */ -export const SUMMARY_COMMENT_MARKER = ""; - -/** - * The engine-emitted marker gh-aw appends to every comment a workflow posts: - * ``. Observed - * production summary comments carry this but not the `pr-reviewer` marker - * (the orchestrator's marker line is not reliably emitted on older pins), so - * the sweep accepts either. The reviewer's only `add-comment` output is the - * risks/patterns comment (`max: 1`, status comments disabled), so scoping by - * the review workflow's call id is exact, not heuristic. - */ -export const workflowCallIdMarker = ( - owner: string, - repo: string, - workflowId: string, -): string => ``; - -/** - * Body prefixes that identify a reviewer inline comment: the code-owned - * Conventional-Comment label taxonomy, exactly as `renderComment` templates it - * (`**