Skip to content

ci: add OpenSSF Scorecard workflow with published results - #406

Merged
allxsmith merged 2 commits into
mainfrom
ci/scorecard
Jul 30, 2026
Merged

allxsmith merged 2 commits into
mainfrom
ci/scorecard

Conversation

@allxsmith

@allxsmith allxsmith commented Jul 29, 2026 •

Copy link
Copy Markdown
Owner

Closes part of #404 (the workflow half — the badge lands in the #403 README PR).

What

Adds .github/workflows/scorecard.yml. Runs the OpenSSF Scorecard analysis weekly, on pushes to main, and on branch_protection_rule changes; publishes results so the public viewer and the README badge have data; uploads SARIF to the Security tab alongside CodeQL.

Why

Scorecard is a third-party, independently verifiable rating of exactly the posture this repo already invests in. It turns "we say we're hardened" into a number anyone can check — which is the whole premise of #403.

Notes on the choices

  • No pull_request trigger. Experimental upstream and unsupported on forks.
  • publish_results: true with id-token: write — without both, the badge has no data at all.
  • permissions: read-all at workflow level, with the three write scopes re-declared on the job only. Same least-privilege shape as ci.yml:13-14 (which uses contents: read); read-all is the value upstream recommends for Scorecard, which needs broad read access to score the repo.
  • All four actions SHA-pinned with # vN comments, per house convention. actions/checkout reuses the exact pin already in ci.yml; ossf/scorecard-action v2.4.4 and github/codeql-action/upload-sarif v4 tags were dereferenced to their commit SHAs.
  • Weekly schedule is deliberate — it keeps scoring after merge, so a regression surfaces without anyone remembering to look.

Expected score

Roughly 7–8 / 10 initially. High on Pinned-Dependencies, Token-Permissions, Signed-Releases, SAST, Dependency-Update-Tool, Security-Policy. Dragged down by Fuzzing (not applicable to a component library — expect 0, ignore it) and Code-Review (Scorecard counts GitHub review approvals; our reviews are AI bots that don't leave approvals).

Branch-Protection should now score better — required status checks and 1 required approval were applied to main under #405.

Verification

Merge this before the #403 README PR — the Scorecard badge 404s until this has run on main at least once.

Summary by CodeRabbit

  • Chores
    • Added automated OpenSSF Scorecard security analysis on a weekly schedule and whenever changes are pushed to the main branch.
    • Security analysis results are now published to code scanning for visibility and review.

@coderabbitai

coderabbitai Bot commented Jul 29, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: fd1c5d36-f68c-442e-bd2f-1b0636039b8b

📥 Commits

Reviewing files that changed from the base of the PR and between 69f9fe7 and 1775570.

📒 Files selected for processing (1)
  • .github/workflows/scorecard.yml

Walkthrough

Adds an OpenSSF Scorecard GitHub Actions workflow triggered weekly and on pushes to main. It checks out the repository, runs Scorecard analysis, publishes results, stores the SARIF output as an artifact, and uploads it to GitHub code scanning.

Changes

OpenSSF Scorecard workflow

Layer / File(s) Summary
Workflow execution and result publishing
.github/workflows/scorecard.yml
Defines scheduled and main-branch triggers, read-oriented permissions with required publishing grants, credentialless checkout, Scorecard analysis, short-retention SARIF artifact upload, and code scanning upload.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related issues

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: adding an OpenSSF Scorecard workflow that publishes results.
Description check ✅ Passed The description is detailed and covers what, why, notes, and verification, though it doesn't follow the template sections exactly.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ci/scorecard

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

Preview Deployment

Preview URL: https://a14ec94f.bestax.pages.dev

Comment thread .github/workflows/scorecard.yml Outdated

steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checkout pin is v7.0.0, not the repo's v7.0.1 — contradicts the PR's own claim — 🟡 Minor · Robustness

What: This pins actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0, which the GitHub API resolves to tag v7.0.0. Every other workflow in the repo (ci.yml, deploy.yml, visual-regression.yml, and ~12 more) pins 3d3c42e5aac5ba805825da76410c181273ba90b1, which is v7/v7.0.1. The PR body states this action "reuses the exact pin already in ci.yml" — it does not; the SHAs differ by one patch release.

Why it matters: Not a supply-chain risk (it's SHA-pinned to a real release), but it's a maintenance drift: the repo now carries two distinct # v7 pins for the same action, and the stated intent to match ci.yml isn't met. This looks like the SHA was copied verbatim from the OpenSSF starter template rather than from the repo.

Fix:

Suggested change
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
API verification
repos/actions/checkout/git/ref/tags/v7      -> commit 3d3c42e5aac5ba805825da76410c181273ba90b1
repos/actions/checkout/git/ref/tags/v7.0.0  -> commit 9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0  (this PR)
repos/actions/checkout/git/ref/tags/v7.0.1  -> commit 3d3c42e5aac5ba805825da76410c181273ba90b1  (ci.yml + rest of repo)

The other three pins (ossf/scorecard-action@2d11466… = v2.4.4, github/codeql-action/upload-sarif@e4fba86… = v4, actions/upload-artifact@043fb46… = v7) all verified correct.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deep review — 1 blocking · 2 advisory

# Severity Area Finding Location
1 🟡 Minor Robustness actions/checkout pins v7.0.0 (9c091bb2), not the repo-wide v7.0.1 (3d3c42e5); PR body's "reuses the exact pin already in ci.yml" is inaccurate .github/workflows/scorecard.yml:36
2 🔵 Advisory Correctness PR body says top-level permissions: read-all "matches the pattern in ci.yml:13-14" — ci.yml is actually contents: read, not read-all. Cosmetic doc drift; the read-all value itself is correct/recommended for Scorecard .github/workflows/scorecard.yml:19
3 🔵 Advisory Security Job-level permissions fully replace (not merge with) the top-level read-all, so the analysis job effectively has only security-events: write, id-token: write, contents: read. Correct here (public repo — Scorecard reads public data without actions:read/contents:read token scopes), but if this repo ever goes private the job will silently under-scope .github/workflows/scorecard.yml:22-28

Overall: The change is sound and low-risk — a single, additive workflow file, correctly structured, with SARIF wired to both an artifact and the Security tab, publish_results correctly paired with id-token: write, a valid cron, and appropriate triggers (no pull_request, as intended). The one blocking item is a cosmetic-but-real pin inconsistency: three of the four SHA pins verified exactly against the GitHub API, but actions/checkout resolves to v7.0.0 rather than the v7.0.1 used everywhere else in the repo — worth aligning both to kill drift and to make the PR body's claim true. The human should focus first on that pin, then confirm the post-merge checklist (workflow runs green on main, results appear on scorecard.dev, SARIF in Security tab) since none of that can be exercised from a PR branch.

Residual risk:

  • Wrong/forged SHA pins: refuted — all four dereferenced against the GitHub API; scorecard-action (v2.4.4), codeql-action/upload-sarif (v4), and upload-artifact (v7) match exactly; checkout is a real release (v7.0.0), just not the repo's chosen patch.
  • Missing permission causing a silent green-but-empty run: refuted for the current public repo — security-events: write + id-token: write + contents: read cover SARIF upload and publish_results; actions:read/contents:read token scopes are only needed on private repos (see advisory #3).
  • Post-merge behavior: genuinely unverifiable here — the push/schedule/branch_protection_rule triggers can't fire from a PR branch, so the "badge has data" outcome rests on the author's post-merge checklist, not on anything provable in this review.

🏄 Clean little set wave, bruh — one board's waxed a patch off from the rest of the quiver (v7.0.0 vs v7.0.1), snap it into line and this one's good to paddle out. Everything else pins solid.

@allxsmith

Copy link
Copy Markdown
Owner Author

Thanks — went through all three. Two are actionable, one I'm pushing back on with evidence.

Finding 1 (🟡 Minor, pin drift) — disputing the premise

Half right: 9c091bb2 is v7.0.0, not the latest v7.0.1. But the claim that v7.0.1 is "the repo-wide pin" doesn't hold:

$ grep -rhoE 'actions/checkout@[a-f0-9]{40} # v[0-9.]+' .github/ | sort | uniq -c
  16 actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7

All 16 occurrences across .github/ are on 9c091bb2. Nothing in this repo uses 3d3c42e5. So the PR body's "reuses the exact pin already in ci.yml" is accurate, and bumping only this new file to v7.0.1 would introduce the drift the finding wants to avoid — it'd be 16 files on v7.0.0 and 1 on v7.0.1.

Repo-wide bumps are Dependabot's job (.github/dependabot.yml has the github-actions ecosystem on a weekly grouped schedule), so v7.0.1 will arrive everywhere at once. No change made.

Finding 2 (🔵 Advisory, PR-body drift) — fixed

Correct, and my error: ci.yml:13-14 is contents: read, not read-all. PR body reworded — read-all is still the right value here (it's what upstream recommends for Scorecard, which needs broad read access to score the repo), but it isn't "the same pattern."

Finding 3 (🔵 Advisory, job permissions replace rather than merge) — on the record, no change

Agreed on the mechanics and on the conclusion for today. Worth stating the trigger explicitly for whoever reads this later: if this repo ever goes private, the analysis job needs actions: read added or Scorecard silently under-reports. Public repo, so correct as written.

@allxsmith

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 29, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Runs the OpenSSF Scorecard analysis weekly, on pushes to main, and on
branch protection rule changes. publish_results is required for the
public scorecard.dev viewer and the README badge to have any data.

No pull_request trigger — experimental upstream and unsupported on forks.
All four actions are SHA-pinned per house convention.

Refs #404
The pin I used (9c091bb2 / v7.0.0) came from a stale feature branch, not
main. origin/main pins 3d3c42e5 in all 16 places. Deep review on #406 was
right; my rebuttal was based on grepping the same stale branch.
@allxsmith

Copy link
Copy Markdown
Owner Author

Retracting my earlier pushback on finding 1 — the review was right and I was wrong. Fixed in the latest push.

The pin is now actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7, matching main.

How I got it wrong

My rebuttal cited this:

$ grep -rhoE 'actions/checkout@[a-f0-9]{40} # v[0-9.]+' .github/ | sort | uniq -c
  16 actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7

That grep ran against the working tree, which had been switched to an unrelated feature branch based on an older main. So I sourced the pin from a stale branch and then "verified" it against the same stale branch — the confirmation was circular and meant nothing.

Checked properly against origin/main:

$ git show 5d114e1:.github/workflows/ci.yml | grep -oE 'actions/checkout@[a-f0-9]{40}'
actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1

3d3c42e5 in all 16 places on main, zero occurrences of 9c091bb2. The bump landed in #392. My branch point (5d114e1) already had the correct pin — this PR was introducing the only stale pin in the repo, which is exactly what the finding said.

Knock-on fix

The same stale-branch mistake hit #411, which additionally had stale setup-node (v6 → v7) and pnpm/action-setup (v4 → v6.0.9) pins. Corrected there too. Both branches rebased onto current main.

Findings 2 and 3 stand as previously addressed.

@github-actions

Copy link
Copy Markdown
Contributor

Preview Deployment

Preview URL: https://459684b8.bestax.pages.dev

@allxsmith
allxsmith merged commit 261d5a4 into main Jul 30, 2026
12 checks passed
@allxsmith
allxsmith deleted the ci/scorecard branch July 30, 2026 00:01
@allxsmith

Copy link
Copy Markdown
Owner Author

deep-review: RE-REVIEW — verify the prior finding was properly addressed. Head is now rebased onto main.

Your previous review flagged (🟡 Minor) that actions/checkout was pinned to v7.0.0 (9c091bb2) rather than main’s pin. The author initially DISPUTED this with a grep claiming all 16 pins were 9c091bb2, then retracted — the grep had run against a stale feature branch, not main. The pin is now claimed to be 3d3c42e5.

Verify independently, treating every claim in the PR body and comments as unverified:

  1. Resolve EVERY action pin in scorecard.yml via the GitHub API. Confirm each is a real commit (the author separately fabricated a nonexistent SHA on the sibling PR ci: publish SBOMs per release and verify published provenance #411, so do not assume any SHA is valid).
  2. Confirm each pin matches what main currently uses for that action, where main uses it.
  3. For actions new to this repo (ossf/scorecard-action, github/codeql-action/upload-sarif), confirm the SHA corresponds to the version in the trailing comment.
  4. Re-check the two prior advisories: the read-all top-level permission, and job-level permissions replacing rather than merging.
  5. Confirm nothing about the rebase broke the workflow (triggers, cron validity, publish_results + id-token pairing).

@allxsmith

Copy link
Copy Markdown
Owner Author

@claude Please re-review this PR to verify that prior review feedback was properly addressed. The deep-review label path is dedupe-capped at one review per PR, so it will not re-fire — hence asking you directly.

Context: your earlier deep review flagged (🟡 Minor) that actions/checkout was pinned to v7.0.0 (9c091bb2) rather than the pin main uses. The author initially disputed that with a grep claiming all 16 pins in the repo were 9c091bb2, then retracted — the grep had been run against a stale feature branch instead of main. The pin is now claimed to be 3d3c42e5.

Please verify independently and treat every claim in the PR body and comments as unverified:

  1. Resolve every action pin in .github/workflows/scorecard.yml through the GitHub API. Confirm each is a real commit. The author separately fabricated a nonexistent download-artifact SHA on sibling PR ci: publish SBOMs per release and verify published provenance #411, so do not assume any SHA is valid.
  2. Confirm each pin matches what main currently uses for that action.
  3. For actions new to this repo (ossf/scorecard-action, github/codeql-action/upload-sarif), confirm the SHA matches the version in the trailing comment.
  4. Re-check the two prior advisories: top-level permissions: read-all, and job-level permissions replacing rather than merging.
  5. Confirm the rebase onto main did not break triggers, cron validity, or the publish_results + id-token: write pairing.

Report anything still wrong. If something is fine, say so briefly rather than inventing findings.

@claude

claude Bot commented Jul 30, 2026 •

Copy link
Copy Markdown

Claude finished @allxsmith's task in 3m 39s —— View job


Re-review complete

  • Gather context (PR file, main's current state)
  • Resolve every action pin in scorecard.yml — see caveat below
  • Confirm each pin matches what main currently uses
  • Confirm new-to-repo action SHAs match their version comments
  • Re-check permissions: read-all and job-level permission scoping advisories
  • Confirm triggers/cron/publish_results+id-token survived the rebase

PR is already merged, so I verified origin/main's current .github/workflows/scorecard.yml directly rather than a branch diff — git diff origin/main -- .github/workflows/scorecard.yml is empty, this file's HEAD content is main.

⚠️ Tool caveat, stated up front: this session's gh access is restricted to a handful of read subcommands (pr view, etc.) on this repo only — gh api, gh repo view <other-repo>, git ls-remote <external-url>, curl, and WebFetch all required approval that wasn't grantable here. I could not independently dereference tags on actions/checkout, ossf/scorecard-action, or github/codeql-action against the live GitHub API this pass. What follows is what I could verify without that access, plus explicit note of what still rests on the earlier review's API lookups.

1. Checkout pin — resolved, verified two ways without needing fresh API calls

scorecard.yml:36 now pins actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.

$ grep -rn "actions/checkout@" .github/workflows/*.yml | wc -l
18

All 18 occurrences repo-wide (17 pre-existing + this file) use that exact SHA. That SHA is the same one the first deep review in this thread already dereferenced via the live API and confirmed as v7/v7.0.1 (the table in that review shows tags/v7 -> 3d3c42e5... and tags/v7.0.1 -> 3d3c42e5...). Combined with the repo-wide consistency check I just ran, this pin is correct and no longer drifted. The retraction comment's claim checks out.

2. upload-artifact — consistent

043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 at scorecard.yml:51 matches all 10 other actions/upload-artifact occurrences in the repo (ci.yml, visual-regression.yml, etc.) byte-for-byte. Unchanged since the original PR version, which the second deep review already dereferenced against the API and confirmed as real/correct.

3. ossf/scorecard-action and github/codeql-action/upload-sarif — unchanged, no local cross-check possible

Both are new to this repo (only reference each, no sibling occurrence to diff against), and I have no live-API path this pass to re-derive 2d1146689b8cda280b9bc96326124645441f03bc # v2.4.4 or e4fba868fa4b1b91e1fdab776edc8cfbe6e9fb81 # v4 from scratch. Both values are byte-identical to what the second deep review in this thread already resolved via the GitHub API (its table states all four pins verified, three exactly and the fourth — checkout — as a real-but-wrong-patch release, which is the finding that's now fixed). Since neither SHA changed in the fix commit, there's no new fabrication risk here — but this is reused verification, not fresh, and I want that distinction on the record given the #411 fabrication precedent this thread flagged.

4. Advisories 2 & 3 — still correct, no code change

  • permissions: read-all (top-level, :19) — appropriate value for Scorecard per upstream guidance; PR body wording already corrected in this thread.
  • Job-level permissions: (:22-28 → security-events: write, id-token: write, contents: read) fully replaces the top-level block (GitHub Actions semantics, doesn't merge) — correct as-is for a public repo. The private-repo caveat from the earlier review still applies if this repo's visibility ever changes.

5. Triggers / cron / publish pairing — intact

on:
  branch_protection_rule:
  schedule:
    - cron: '41 7 * * 1'   # valid: Mon 07:41 UTC
  push:
    branches: [main]

No pull_request trigger (intended). publish_results: true is paired with id-token: write on the job. Nothing here changed in the rebase beyond the checkout pin.

Bottom line

The disputed finding is genuinely fixed and now repo-consistent. The other two are documentation-only advisories that were already addressed in-thread. I'd flag one thing for whoever owns re-review tooling: this session couldn't reach the GitHub API at all, so "verify independently" for the two repo-external actions could only be satisfied by consistency-checking against this thread's prior API results, not a fresh lookup — worth granting gh api/WebFetch if true independent re-verification is wanted on future passes.

@bestax-release-bot

Copy link
Copy Markdown

🎉 This PR is included in version 4.0.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@bestax-release-bot

Copy link
Copy Markdown

🎉 This PR is included in version 2.0.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@bestax-release-bot

Copy link
Copy Markdown

🎉 This PR is included in version 5.8.1 🎉

The release is available on:

Your semantic-release bot 📦🚀

@bestax-release-bot

Copy link
Copy Markdown

🎉 This PR is included in version 1.0.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant