Skip to content

Build: Add agent scan to detect automated accounts#34293

Merged
huang-julien merged 16 commits into
nextfrom
ci/agent-scan
Apr 27, 2026
Merged

Build: Add agent scan to detect automated accounts#34293
huang-julien merged 16 commits into
nextfrom
ci/agent-scan

Conversation

@huang-julien
Copy link
Copy Markdown
Contributor

@huang-julien huang-julien commented Mar 24, 2026

Closes #

What I did

This PR adds AgentScan into our workflow for newly created PRs.

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

  • stories
  • unit tests
  • integration tests
  • end-to-end tests

Manual testing

  • no test needed but report any strange behavior if it happens from the ci

Caution

This section is mandatory for all contributions. If you believe no manual test is necessary, please state so explicitly. Thanks!

Documentation

  • Add or update documentation reflecting your changes
  • If you are deprecating/removing a feature, make sure to update
    MIGRATION.MD

Checklist for Maintainers

  • When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli-storybook/src/sandbox-templates.ts

  • Make sure this PR contains one of the labels below:

    Available labels
    • bug: Internal changes that fixes incorrect behavior.
    • maintenance: User-facing maintenance tasks.
    • dependencies: Upgrading (sometimes downgrading) dependencies.
    • build: Internal-facing build tooling & test updates. Will not show up in release changelog.
    • cleanup: Minor cleanup style change. Will not show up in release changelog.
    • documentation: Documentation only changes. Will not show up in release changelog.
    • feature request: Introducing a new feature.
    • BREAKING CHANGE: Changes that break compatibility in some way with current major version.
    • other: Changes that don't fit in the above categories.

🦋 Canary release

This PR does not have a canary release associated. You can request a canary release of this pull request by mentioning the @storybookjs/core team here.

core team members can create a canary release here or locally with gh workflow run --repo storybookjs/storybook publish.yml --field pr=<PR_NUMBER>

Summary by CodeRabbit

  • Chores
    • Added an automated workflow to scan incoming pull requests when opened or reopened.
    • Added supporting scripts to apply classification-based labels (maps classifications to human/automated) after scans.
    • PRs flagged by the community are automatically closed; workflow runs conditional post-scan actions for labeling or closing.

@huang-julien huang-julien self-assigned this Mar 24, 2026
@huang-julien huang-julien changed the title Ci: add agent scan to detect automated accounts Ci: Add agent scan to detect automated accounts Mar 24, 2026
@huang-julien huang-julien added the ci:merged Run the CI jobs that normally run when merged. label Mar 24, 2026
@huang-julien huang-julien changed the title Ci: Add agent scan to detect automated accounts Other: Add agent scan to detect automated accounts Mar 24, 2026
@nx-cloud
Copy link
Copy Markdown

nx-cloud Bot commented Mar 24, 2026

View your CI Pipeline Execution ↗ for commit 1ff1d08

Command Status Duration Result
nx run-many -t compile,check,knip,test,lint,fmt... ✅ Succeeded 1m 53s View ↗

☁️ Nx Cloud last updated this comment at 2026-04-14 09:24:15 UTC

@huang-julien huang-julien changed the title Other: Add agent scan to detect automated accounts Build: Add agent scan to detect automated accounts Mar 24, 2026
@huang-julien huang-julien marked this pull request as ready for review March 24, 2026 14:25
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Mar 24, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds a new GitHub Actions workflow agent-scan triggered on pull_request_target (opened,reopened) that runs an Agentscan action with caching and then conditionally labels or closes the PR via two new Node ESM scripts based on classification and community-flagged outputs.

Changes

Cohort / File(s) Summary
Workflow
.github/workflows/agent-scan.yml
New agent-scan workflow on pull_request_target (opened,reopened). Concurrency scoped to PR, gated by github.repository_owner == 'storybookjs', sets issues/pull-requests write permissions, checks out code, caches .agentscan-cache, runs MatteoGabriele/agentscan-action, and exposes outputs used by post-scan steps.
Post-scan scripts
.github/scripts/agent-scan-label-pr.mjs, .github/scripts/agent-scan-close-pr.mjs
New Node ESM scripts using @actions/core/@actions/github: one maps classification (organichuman, automationautomated) and adds an agent-scan:<mapped-classification> label to the PR; the other closes the PR. Both read inputs and PR number from GitHub context and call Octokit; errors call core.setFailed.
Scripts manifest
.github/scripts/package.json
New package manifest ("type":"module", private) declaring @actions/core and @actions/github dependencies for the scripts.

Sequence Diagram(s)

sequenceDiagram
    rect rgba(200,200,255,0.5)
    participant PR as Pull Request
    end
    rect rgba(200,255,200,0.5)
    participant GH as GitHub Actions
    end
    rect rgba(255,200,200,0.5)
    participant Agent as Agentscan Action
    end
    rect rgba(255,255,200,0.5)
    participant Cache as Cache
    end
    rect rgba(200,255,255,0.5)
    participant API as GitHub API
    end

    PR->>GH: opens/reopens PR (pull_request_target)
    GH->>Cache: restore `.agentscan-cache` (key: github.actor)
    GH->>Agent: run Agentscan with inputs + cache-path
    Agent-->>GH: outputs (classification, community-flagged)
    GH->>API: if community-flagged != 'true' -> run `agent-scan-label-pr.mjs` to add label
    GH->>API: if community-flagged == 'true' -> run `agent-scan-close-pr.mjs` to close PR
    GH->>Cache: save `.agentscan-cache`
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (2)
.github/workflows/agent-scan.yml (2)

58-70: Consider adding a comment explaining why the PR was closed.

When closing community-flagged PRs, adding a comment would help the contributor understand why their PR was closed and what recourse they have if it's a false positive.

💬 Proposed fix to add explanatory comment before closing
       - name: Close community flagged accounts
         if: steps.agentscan.outputs.community-flagged == 'true'
         uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
         with:
           script: |
             const prNumber = context.payload.pull_request.number;
 
+            await github.rest.issues.createComment({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              issue_number: prNumber,
+              body: '🚫 This PR has been automatically closed because the account has been community-flagged as automated/suspicious. If you believe this is a mistake, please reach out to the maintainers.',
+            });
+
             await github.rest.pulls.update({
               owner: context.repo.owner,
               repo: context.repo.repo,
               pull_number: prNumber,
               state: 'closed',
             });
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/agent-scan.yml around lines 58 - 70, Add an explanatory
comment before closing the PR in the "Close community flagged accounts" step:
use the same prNumber/owner/repo values to call github.rest.issues.createComment
with a concise body explaining the reason for closure (community-flagged), links
or steps for appeal if it's a false positive, and any contact or next steps,
then proceed to call github.rest.pulls.update (state: 'closed') as currently
implemented.

21-27: Consider adding version comments to SHA-pinned actions for maintainability.

Lines 38 and 60 include version comments (e.g., # v8.0.0), but the checkout, cache, and agentscan actions lack them. Adding version comments helps maintainers quickly identify which version is pinned and when updates are needed.

📝 Suggested version comments
-      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
+      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4.x.x
       - name: Cache AgentScan analysis
-        uses: actions/cache@668228422ae6a00e4ad889ee87cd7109ec5666a7
+        uses: actions/cache@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v4.x.x
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/agent-scan.yml around lines 21 - 27, The workflow uses
SHA-pinned actions (actions/checkout@de0fac2e..., actions/cache@6682284..., and
the AgentScan action) without human-readable version comments; update the YAML
to add trailing comments (e.g., "# v2.7.0" or appropriate semver) next to those
pinned SHAs so maintainers can quickly see the intended major/minor version for
actions/checkout, actions/cache and the AgentScan action; confirm the semver
used matches the upstream release corresponding to each SHA before adding the
comment.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/workflows/agent-scan.yml:
- Line 37: The conditional uses contains(...) to test
steps.agentscan.outputs.community-flagged which can match substrings like "not
true"; change both checks to use strict equality against the literal string
'true' instead of contains — e.g., replace
!contains(steps.agentscan.outputs.community-flagged, 'true') with
steps.agentscan.outputs.community-flagged == 'true' (or the inverted strict
check as needed) so both occurrences (the condition with fromJSON(...) && ...
and the later line 59 usage) consistently and explicitly compare
steps.agentscan.outputs.community-flagged to 'true'.
- Around line 36-57: The createComment call currently posts an empty body
because the script's body array is empty; update the
github.rest.issues.createComment invocation in the actions/github-script step to
pass a non-empty message (use the prNumber and the classification variable) —
e.g., construct a clear comment string explaining the PR was flagged (include
the ${classification}), instruct maintainers and contributors to review and to
ping if this is a false positive, and replace the empty body array in the
createComment call so the workflow posts that meaningful message instead of a
blank comment.

---

Nitpick comments:
In @.github/workflows/agent-scan.yml:
- Around line 58-70: Add an explanatory comment before closing the PR in the
"Close community flagged accounts" step: use the same prNumber/owner/repo values
to call github.rest.issues.createComment with a concise body explaining the
reason for closure (community-flagged), links or steps for appeal if it's a
false positive, and any contact or next steps, then proceed to call
github.rest.pulls.update (state: 'closed') as currently implemented.
- Around line 21-27: The workflow uses SHA-pinned actions
(actions/checkout@de0fac2e..., actions/cache@6682284..., and the AgentScan
action) without human-readable version comments; update the YAML to add trailing
comments (e.g., "# v2.7.0" or appropriate semver) next to those pinned SHAs so
maintainers can quickly see the intended major/minor version for
actions/checkout, actions/cache and the AgentScan action; confirm the semver
used matches the upstream release corresponding to each SHA before adding the
comment.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 95d61e1a-1b95-4d11-892a-680f1818b608

📥 Commits

Reviewing files that changed from the base of the PR and between 96c8c23 and 48795ac.

📒 Files selected for processing (1)
  • .github/workflows/agent-scan.yml

Comment thread .github/workflows/agent-scan.yml Outdated
Comment thread .github/workflows/agent-scan.yml Outdated
@huang-julien huang-julien marked this pull request as draft March 27, 2026 13:32
@huang-julien huang-julien added the build Internal-facing build tooling & test updates label Apr 8, 2026
@huang-julien huang-julien marked this pull request as ready for review April 8, 2026 15:03
@storybook-app-bot
Copy link
Copy Markdown

storybook-app-bot Bot commented Apr 8, 2026

Package Benchmarks

Commit: c969bcd, ran on 27 April 2026 at 10:34:21 UTC

The following packages have significant changes to their size or dependencies:

storybook

Before After Difference
Dependency count 50 50 0
Self size 20.54 MB 20.52 MB 🎉 -21 KB 🎉
Dependency size 16.56 MB 16.56 MB 🎉 -95 B 🎉
Bundle Size Analyzer Link Link

@storybook/cli

Before After Difference
Dependency count 184 184 0
Self size 782 KB 782 KB 🎉 -7 B 🎉
Dependency size 68.20 MB 68.18 MB 🎉 -20 KB 🎉
Bundle Size Analyzer Link Link

@storybook/codemod

Before After Difference
Dependency count 177 177 0
Self size 32 KB 32 KB 🚨 +34 B 🚨
Dependency size 66.73 MB 66.71 MB 🎉 -21 KB 🎉
Bundle Size Analyzer Link Link

create-storybook

Before After Difference
Dependency count 51 51 0
Self size 1.04 MB 1.04 MB 🚨 +1 KB 🚨
Dependency size 37.10 MB 37.07 MB 🎉 -21 KB 🎉
Bundle Size Analyzer node node

id: agentscan
uses: MatteoGabriele/agentscan-action@a584774dd15cabe6df4c6ab45fc43514a3b56b2d
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same security considerations as #34013, we should ideally make a dedicated API token

  • with exactly the right privileges so it can't be abused more than necessary
  • and unique to ensure any breach allows us to trace back which workflow was abused

I have yet to create a token for my own PR, whoever figures it out first can ping the other :) Jeppe shared a solution that I believe is mentioned on my PR.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Sidnioulz I think secrets.GITHUB_TOKEN is a unique generated token per job, it's already unique.

uses: MatteoGabriele/agentscan-action@a584774dd15cabe6df4c6ab45fc43514a3b56b2d
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
skip-members: "dependabot[bot],AriPerkkio,huang-julien,MichaelArestad,yannbf,vanessayuenn,jonniebigodes,Sidnioulz,kasperpeulen,valentinpalkovic,github-actions[bot],ndelangen,shilman,JReinhold,ghengeveld,storybook-bot,kylegach"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Can we use a team identifier instead? This will age poorly 😅
  • What is the rationale behind skipping ourselves? Perf / cost? If the agent scan job ever also detects agent-made PRs made by human accounts, I think we'll want to flag ourselves and hold ourselves to account to (e.g. if this informs the number of requested approvals)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agent scan scans by accounts activity, not PR content.

It's goal is to detect highly automated accounts so it probably doesn't makes sense to scan us. Otherwise if someone's flagged, all hist future PRs will be flagged too 😂

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for clarifying :) Let's see if it's not too painful to add group support to agent-scan, otherwise feel free to resolve this.

Comment thread .github/workflows/agent-scan.yml Outdated
Comment thread .github/workflows/agent-scan.yml Outdated
Comment thread .github/workflows/agent-scan.yml Outdated
Comment thread .github/workflows/agent-scan.yml Outdated
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (5)
.github/scripts/agent-scan-close-pr.mjs (2)

8-8: Add defensive null check for pull_request payload.

If the workflow context is somehow missing the pull_request object (e.g., misconfigured trigger), accessing .number will throw an unclear TypeError. Adding a guard provides a clearer error message.

🛡️ Proposed defensive check
   const octokit = github.getOctokit(token);
+  const pullRequest = github.context.payload.pull_request;
+  if (!pullRequest) {
+    throw new Error('This script must be run in a pull_request context');
+  }
-  const prNumber = github.context.payload.pull_request.number;
+  const prNumber = pullRequest.number;
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/scripts/agent-scan-close-pr.mjs at line 8, Add a defensive null
check around github.context.payload.pull_request before using it to derive
prNumber: validate that github.context && github.context.payload &&
github.context.payload.pull_request exist and if not, log or throw a clear error
explaining the missing pull_request payload (instead of letting the TypeError
occur) and exit gracefully; then safely assign prNumber from
github.context.payload.pull_request.number. Reference the variable prNumber and
the object github.context.payload.pull_request when implementing this guard.

10-14: Consider adding a label or comment explaining why the PR was closed.

The PR is closed silently without any record of why. Per the earlier discussion between maintainers, adding an agent-scan:community-flagged label before closing would provide a clear audit trail and help contributors understand the closure reason.

📝 Proposed fix to add label before closing
+  await octokit.rest.issues.addLabels({
+    ...github.context.repo,
+    issue_number: prNumber,
+    labels: ['agent-scan:community-flagged'],
+  });
+
   await octokit.rest.pulls.update({
     ...github.context.repo,
     pull_number: prNumber,
     state: 'closed',
   });
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/scripts/agent-scan-close-pr.mjs around lines 10 - 14, Before calling
octokit.rest.pulls.update to close the PR, add a label and/or comment explaining
the reason; use the existing prNumber and github.context.repo values to call
octokit.rest.issues.addLabels (or octokit.rest.issues.createComment to add a
text explanation) to apply the "agent-scan:community-flagged" label and a short
explanatory comment, then proceed with the pull update call that sets state:
'closed' so the closure is recorded with an audit trail.
.github/workflows/agent-scan.yml (2)

35-35: Hardcoded skip-members list will require manual maintenance.

This list of usernames will need updates whenever team membership changes. As noted in past discussion, consider whether a GitHub team identifier could be used instead to reduce maintenance burden.

If a team-based approach isn't feasible with the current agentscan-action, consider adding a comment documenting when/how to update this list.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/agent-scan.yml at line 35, Replace the hardcoded
skip-members CSV in the agentscan-action configuration with a less brittle
mechanism: either switch the skip-members value to reference a GitHub team
identifier (if agentscan-action supports team names/IDs) or, if that isn’t
supported, add an in-file comment adjacent to the skip-members line documenting
the canonical owner(s) responsible for updates and the process/timestamp to
update the list; update the entry referencing the exact key "skip-members" so
reviewers can find and change behavior in the action configuration.

44-48: Consider adding a label before closing for audit trail.

When closing PRs from community-flagged accounts, there's no record explaining why. Adding an agent-scan:community-flagged label (as discussed in past reviews) would provide transparency and help contributors understand the closure reason.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/agent-scan.yml around lines 44 - 48, Update the "Close
community flagged accounts" step to add an `agent-scan:community-flagged` label
before running the close script: call the GitHub API or gh CLI within that step
to add the label to the target PR (using the same INPUT_TOKEN env) and only then
invoke node .github/scripts/agent-scan-close-pr.mjs; ensure the label name
matches `agent-scan:community-flagged` and that the step handles API errors so
closure still proceeds or logs failures appropriately.
.github/scripts/agent-scan-label-pr.mjs (1)

15-15: Add defensive null check for pull_request payload.

Same issue as in agent-scan-close-pr.mjs — accessing .number without a guard can produce an unclear error if the context is malformed.

🛡️ Proposed defensive check
   const octokit = github.getOctokit(token);
-  const prNumber = github.context.payload.pull_request.number;
+  const pullRequest = github.context.payload.pull_request;
+  if (!pullRequest) {
+    throw new Error('This script must be run in a pull_request context');
+  }
+  const prNumber = pullRequest.number;
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/scripts/agent-scan-label-pr.mjs at line 15, The pull request
accesses github.context.payload.pull_request.number directly via the prNumber
variable which can throw if payload.pull_request is null; update the code that
defines prNumber to defensively check github.context.payload and
github.context.payload.pull_request exist (e.g., guard with an if/early return
or throw a clear error) before reading .number, and ensure the failure path logs
a helpful message and exits/returns so downstream code (using prNumber) won't
run with undefined.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/scripts/agent-scan-label-pr.mjs:
- Around line 4-8: The CLASSIFICATION_MAP constant uses the wrong key
'automated' so lookups for the agentscan-action output 'automation' fail; update
CLASSIFICATION_MAP (symbol name CLASSIFICATION_MAP) so the key is 'automation'
mapping to 'automated' (i.e., 'automation': 'automated') instead of the current
'automated' key; after updating, ensure any code that reads the incoming
classification (the variable used where CLASSIFICATION_MAP is indexed) still
uses the same lookup (+ fallback) and then also verify the workflow condition
that mentions 'suspicious' is valid (remove or adjust it if agentscan-action
never emits 'suspicious').

In @.github/workflows/agent-scan.yml:
- Around line 38-43: The workflow and labeling script disagree on classification
names: the action emits "automation" but CLASSIFICATION_MAP in
agent-scan-label-pr.mjs expects "automated", and the workflow condition checks
for "suspicious" which is not emitted; fix by adding a "automation": "automated"
entry to CLASSIFICATION_MAP in agent-scan-label-pr.mjs (so
CLASSIFICATION_MAP["automation"] -> "automated") and update the workflow
condition used in the step (the contains(...) array) to check for "automation"
instead of "suspicious" so the condition and labeling script use the same
classification token.

---

Nitpick comments:
In @.github/scripts/agent-scan-close-pr.mjs:
- Line 8: Add a defensive null check around github.context.payload.pull_request
before using it to derive prNumber: validate that github.context &&
github.context.payload && github.context.payload.pull_request exist and if not,
log or throw a clear error explaining the missing pull_request payload (instead
of letting the TypeError occur) and exit gracefully; then safely assign prNumber
from github.context.payload.pull_request.number. Reference the variable prNumber
and the object github.context.payload.pull_request when implementing this guard.
- Around line 10-14: Before calling octokit.rest.pulls.update to close the PR,
add a label and/or comment explaining the reason; use the existing prNumber and
github.context.repo values to call octokit.rest.issues.addLabels (or
octokit.rest.issues.createComment to add a text explanation) to apply the
"agent-scan:community-flagged" label and a short explanatory comment, then
proceed with the pull update call that sets state: 'closed' so the closure is
recorded with an audit trail.

In @.github/scripts/agent-scan-label-pr.mjs:
- Line 15: The pull request accesses github.context.payload.pull_request.number
directly via the prNumber variable which can throw if payload.pull_request is
null; update the code that defines prNumber to defensively check
github.context.payload and github.context.payload.pull_request exist (e.g.,
guard with an if/early return or throw a clear error) before reading .number,
and ensure the failure path logs a helpful message and exits/returns so
downstream code (using prNumber) won't run with undefined.

In @.github/workflows/agent-scan.yml:
- Line 35: Replace the hardcoded skip-members CSV in the agentscan-action
configuration with a less brittle mechanism: either switch the skip-members
value to reference a GitHub team identifier (if agentscan-action supports team
names/IDs) or, if that isn’t supported, add an in-file comment adjacent to the
skip-members line documenting the canonical owner(s) responsible for updates and
the process/timestamp to update the list; update the entry referencing the exact
key "skip-members" so reviewers can find and change behavior in the action
configuration.
- Around line 44-48: Update the "Close community flagged accounts" step to add
an `agent-scan:community-flagged` label before running the close script: call
the GitHub API or gh CLI within that step to add the label to the target PR
(using the same INPUT_TOKEN env) and only then invoke node
.github/scripts/agent-scan-close-pr.mjs; ensure the label name matches
`agent-scan:community-flagged` and that the step handles API errors so closure
still proceeds or logs failures appropriately.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 92f8490c-6daf-4f9f-939b-0db72161b81c

📥 Commits

Reviewing files that changed from the base of the PR and between 84e6785 and f77b608.

📒 Files selected for processing (4)
  • .github/scripts/agent-scan-close-pr.mjs
  • .github/scripts/agent-scan-label-pr.mjs
  • .github/scripts/package.json
  • .github/workflows/agent-scan.yml
✅ Files skipped from review due to trivial changes (1)
  • .github/scripts/package.json

Comment thread .github/scripts/agent-scan-label-pr.mjs Outdated
Comment thread .github/workflows/agent-scan.yml Outdated
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
.github/scripts/agent-scan-label-pr.mjs (1)

5-9: ⚠️ Potential issue | 🟡 Minor

Clarify the classification comment to match actual input/output terms.

Line 7 says automated, but the incoming classification value is automation and it maps to automated. Updating this comment will avoid future confusion.

✏️ Suggested comment fix
 /**
  * agent scan classification rename
  * - organic -> human
- * - automated
+ * - automation -> automated
  * - mixed
  */
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/scripts/agent-scan-label-pr.mjs around lines 5 - 9, Update the
top-of-file classification comment in .github/scripts/agent-scan-label-pr.mjs to
reflect the actual input/output terms: change the list to show "human" (was
organic), and explicitly note that incoming value "automation" maps to output
"automated" (so display as "automation -> automated"), plus "mixed"; this
clarifies the mapping used by the script.
🧹 Nitpick comments (1)
.github/scripts/agent-scan-label-pr.mjs (1)

1-31: Add a short AGENTS.md note for this new CI gate.

Since this introduces automated account scanning/labeling behavior for PRs, contributor/maintainer guidance should briefly document what agent-scan:* labels mean and expected maintainer actions.

Based on learnings: Keep AGENTS.md up to date when Storybook's architecture, tooling, workflows, or contributor guidance changes.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/scripts/agent-scan-label-pr.mjs around lines 1 - 31, Add a short
AGENTS.md documenting the new CI gate: describe the meaning of agent-scan:*
labels produced by the .github/scripts/agent-scan-label-pr.mjs script (reference
CLASSIFICATION_MAP and the addLabels usage that emits labels like
agent-scan:human, agent-scan:automated, agent-scan:mixed), list expected
maintainer actions when each label appears (e.g., review for sensitive scans,
verify automation, or investigate mixed results), and include guidance on
keeping AGENTS.md updated when agent detection or workflows change.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In @.github/scripts/agent-scan-label-pr.mjs:
- Around line 5-9: Update the top-of-file classification comment in
.github/scripts/agent-scan-label-pr.mjs to reflect the actual input/output
terms: change the list to show "human" (was organic), and explicitly note that
incoming value "automation" maps to output "automated" (so display as
"automation -> automated"), plus "mixed"; this clarifies the mapping used by the
script.

---

Nitpick comments:
In @.github/scripts/agent-scan-label-pr.mjs:
- Around line 1-31: Add a short AGENTS.md documenting the new CI gate: describe
the meaning of agent-scan:* labels produced by the
.github/scripts/agent-scan-label-pr.mjs script (reference CLASSIFICATION_MAP and
the addLabels usage that emits labels like agent-scan:human,
agent-scan:automated, agent-scan:mixed), list expected maintainer actions when
each label appears (e.g., review for sensitive scans, verify automation, or
investigate mixed results), and include guidance on keeping AGENTS.md updated
when agent detection or workflows change.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: f3cc3c12-5405-4884-b2f6-7a3b0a06cd6c

📥 Commits

Reviewing files that changed from the base of the PR and between a0c2ec2 and a8658a3.

📒 Files selected for processing (3)
  • .github/scripts/agent-scan-close-pr.mjs
  • .github/scripts/agent-scan-label-pr.mjs
  • .github/workflows/agent-scan.yml
✅ Files skipped from review due to trivial changes (1)
  • .github/workflows/agent-scan.yml
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/scripts/agent-scan-close-pr.mjs

@huang-julien huang-julien requested a review from Sidnioulz April 13, 2026 09:27
@huang-julien huang-julien merged commit c7e0fe1 into next Apr 27, 2026
179 checks passed
@huang-julien huang-julien deleted the ci/agent-scan branch April 27, 2026 11:15
@github-actions github-actions Bot mentioned this pull request Apr 27, 2026
16 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Internal-facing build tooling & test updates ci:merged Run the CI jobs that normally run when merged. other

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants