ci: harden Mintlify docs automation (AI triage classifier + scoped generator) #2405
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| name: Mintlify PR Triage | ||
|
|
||
| # Auto-triages documentation PRs opened by the Mintlify bot. A Claude classifier | ||
| # verifies each PR against the actual code, then takes exactly ONE action: | ||
| # - MERGE (enable auto-merge; it lands only after CI passes) when it can | ||
| # positively confirm the docs are accurate, user-facing, and apply cleanly. | ||
| # - CLOSE in every other case — stale / backend-internal / inaccurate / | ||
| # duplicate, or anything it cannot confidently verify (bias to close). | ||
| # Replaces the previous "auto-merge every bot PR" workflow. | ||
|
|
||
| on: | ||
| pull_request: | ||
| types: [opened] | ||
|
|
||
| concurrency: | ||
| group: mintlify-triage-${{ github.event.pull_request.number }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| triage: | ||
| # Gate on the PR author, not github.actor (which is spoofable and is the | ||
| # human, not the bot, on reopen). | ||
| if: github.event.pull_request.user.login == 'mintlify[bot]' | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: write # enable auto-merge | ||
| pull-requests: write # close / comment | ||
| id-token: write # claude-code-action auth | ||
| actions: read # read CI status | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 1 # full file tree (code + docs) so feature greps work | ||
|
Comment on lines
+32
to
+35
Contributor
There was a problem hiding this comment. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
FILE=".github/workflows/mintlify-triage.yml"
echo "== File =="
wc -l "$FILE"
echo
echo "== Uses lines =="
rg -n "^\s*uses:" "$FILE" || true
echo
echo "== Lines 1-120 =="
nl -ba "$FILE" | sed -n '1,120p'Repository: MCPJam/inspector Length of output: 298 🏁 Script executed: #!/bin/bash
set -euo pipefail
FILE=".github/workflows/mintlify-triage.yml"
echo "== Full file (numbered) =="
cat -n "$FILE"
echo
echo "== Permissions occurrences =="
rg -n "permissions:" "$FILE" || true
echo
echo "== Checkout step block (lines containing checkout until blank line or next - name/uses) =="
# crude but sufficient for small file
awk '
/uses: actions\/checkout@/ {start=1}
start {print}
start && NR>1 && NF==0 {exit}
' "$FILE"Repository: MCPJam/inspector Length of output: 5171 🌐 Web query:
💡 Result: In actions/checkout (v4 and later), the persist-credentials input defaults to true [1][2][3]. While earlier iterations of the action directly wrote the authentication token into the local.git/config file, recent updates have improved credential security by storing them in a separate file within the runner's temporary directory ($RUNNER_TEMP) [3][4][5][6]. The action then references this file using git config include.path mechanisms rather than embedding the token directly into the repository's primary configuration [6]. Key points regarding this behavior: - Default behavior: The action automatically configures the token to enable authenticated git commands (like git fetch or git push) during the job [7][8][3]. - Security: Credentials are kept in a separate file under $RUNNER_TEMP, not directly inside.git/config [3][4][5][6]. - Cleanup: The token and the associated temporary configuration file are removed during post-job cleanup [7][8][3]. - Opt-out: You can disable this behavior by setting persist-credentials: false in your workflow step [7][8][3]. Citations:
Pin third-party actions to commit SHAs and disable checkout credential persistence
🧰 Tools🪛 zizmor (1.25.2)[warning] 29-32: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false (artipacked) [error] 30-30: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy) (unpinned-uses) 🤖 Prompt for AI Agents |
||
| persist-credentials: false # job has write perms; Claude uses gh, not git auth | ||
|
|
||
| - name: Classify and act | ||
| uses: anthropics/claude-code-action@v1 | ||
| with: | ||
| anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }} | ||
| additional_permissions: | | ||
| actions: read | ||
| claude_args: >- | ||
| --model claude-sonnet-4-6 | ||
| --allowed-tools "Bash(gh pr view:*),Bash(gh pr diff:*),Bash(gh pr merge:*),Bash(gh pr close:*),Bash(gh pr comment:*),Bash(git grep:*),Bash(git log:*),Bash(git show:*),Bash(git ls-files:*),Grep,Glob,Read" | ||
| prompt: | | ||
| You are the documentation-PR triager for ${{ github.repository }}. PR #${{ github.event.pull_request.number }} was opened by the Mintlify docs bot. Decide and execute EXACTLY ONE outcome: MERGE or CLOSE. Bias toward CLOSE — only MERGE when you can positively verify the PR is good. Everything published must be user-facing, accurate, and free of backend/internal detail. Do not ask questions; take the action yourself with the gh CLI. | ||
|
|
||
| Investigate first (read-only): | ||
| - Diff + metadata: `gh pr diff ${{ github.event.pull_request.number }}` and `gh pr view ${{ github.event.pull_request.number }} --json files,mergeable,mergeStateStatus,title,body`. A non-mergeable / conflicting PR usually means it targets a page that was renamed or deleted (stale). | ||
| - Confirm any feature the PR documents actually exists in the code: `git grep -i "<symbol or exact UI string>" -- mcpjam-inspector/ sdk/`. If you cannot find it, treat the feature as non-existent. | ||
|
There was a problem hiding this comment. Triage skips CLI code pathsMedium Severity The triage prompt tells the classifier to verify features only under   Reviewed by Cursor Bugbot for commit 7b97cd4. Configure here. |
||
| - Check whether the content is already documented: `git grep -i "<phrase>" -- docs/`. | ||
|
|
||
| MERGE only if you can confirm ALL of: the PR is mergeable; every target page still exists; the feature is present in the code; the content is user-facing and accurate; and it is not a duplicate or editing anything under `docs/contributing/**`. | ||
|
|
||
| CLOSE in every other case — stale (deleted/renamed page or not mergeable); backend/internal content (Convex, internal HTTP endpoints/routes, environment variables, secrets/keys, server-side persistence or runtime, internal source-file paths); a feature you cannot find in the code; a duplicate; a clear factual error versus the code; OR anything you are not confident enough to merge. Closing is safe and cheap — the bot regenerates a fresh PR when the code changes again. | ||
|
There was a problem hiding this comment. Early mergeable check closes PRsMedium Severity Triage runs immediately on Reviewed by Cursor Bugbot for commit 7b97cd4. Configure here. There was a problem hiding this comment.
Because this workflow runs immediately on Useful? React with 👍 / 👎. |
||
|
|
||
| Execute exactly one, then stop: | ||
| - MERGE → `gh pr merge ${{ github.event.pull_request.number }} --auto --squash` | ||
| - CLOSE → `gh pr close ${{ github.event.pull_request.number }} --comment "<one concise sentence on why>"` | ||
|
Comment on lines
+46
to
+61
Contributor
There was a problem hiding this comment. Restore the promised REVIEW fallback. The workflow now forces every uncertain case into 🤖 Prompt for AI Agents |
||
|
|
||
| Keep the comment to a single sentence. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the triage workflow I checked, the
GITHUB_TOKENpermissions explicitly omitissues, so unspecified permissions arenone; GitHub's Labels REST API documentsCreate a labelas requiringIssuesrepository permission with write access. Because the very next step runsgh label create ... --force, every Mintlify PR run will fail before Claude can merge/close/review anything unless this job grantsissues: write.Useful? React with 👍 / 👎.