ci(skills): require NVSkills validation for skill changes

[ngoncharenko]

Summary

• add a required PR workflow that watches only skills/ changes
• require PR HEAD to be the trusted NVSkills signature commit before passing
• document the /nvskills-ci maintainer/admin flow and required branch protection check

Linear

https://linear.app/nvidia/issue/AALGO-233/block-merges-when-skills-changes-lack-nvskill-pipeline-validation

Validation

• Tested making dummy change in skills
  • job failed: https://github.com/NVIDIA-NeMo/nemo-platform/actions/runs/26849416402/job/79177491876
  • then run /nvskills-ci ---> triggerred nvskills pipeline ---> successful nvskills pipeline committed sig file ---> auto-triggered Require NVSkills CI passed
• uv run --frozen python -c "from pathlib import Path; import yaml; yaml.safe_load(Path('.github/workflows/require-nvskills-ci.yml').read_text()); print('yaml ok')"
• git diff --check
• local Node scenario harness for unsigned, signed, stale-signature, no-skills, and non-skills changes

Notes

• actionlint was not installed locally, so it was not run.
• Repo admin must require Require NVSkills CI for skill changes / require-nvskills-ci in branch protection or rulesets.

Summary by CodeRabbit

• Documentation

  • Updated CI guidance and several skill docs: benchmark dates/scores, license and use-case wording, output types, and evaluation details.

• Chores

  • Added a new workflow that enforces NVSkills validation for changes to skills.
  • Documented how to request NVSkills validation and updated CI onboarding guidance.
  • Added temporary test content for validation and refreshed the validation signature bundle.

[github-actions]

Suite	Lines Covered	Line Rate	Branch Rate
Unit Tests	18641/24678	75.5%	62.0%
Integration Tests	11964/23455	51.0%	26.1%

[Copilot]

Pull request overview

This PR introduces a merge-blocking GitHub Actions check that enforces NVSkills validation for pull requests that modify skills/, and documents the maintainer/admin flow for triggering NVSkills CI and configuring branch protection.

Changes:

• Added a new PR workflow (require-nvskills-ci.yml) that fails when skills/ changes are present but the PR HEAD is not the trusted NVSkills signature commit.
• Documented the /nvskills-ci flow and the required branch protection check in .github/CI_README.md.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
.github/workflows/require-nvskills-ci.yml	Adds the new required check that validates the PR HEAD commit matches the trusted NVSkills signature commit when skills/ files change.
.github/CI_README.md	Documents the NVSkills CI request workflow and the new required NVSkills enforcement workflow and branch protection requirements.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Adds a merge-blocking GitHub Actions workflow (require-nvskills-ci.yml) that blocks merge of changes under skills/ unless the PR head commit is authored by a trusted NVSkills signature actor and has a specific commit-title prefix. Updates CI README to document request/require workflows. Updates skill docs/metrics, adds a temporary SKILL test comment, and replaces the skill DSSE signature bundle.

Changes

NVSkills CI Validation

Layer / File(s)	Summary
Workflow trigger and permissions .github/workflows/require-nvskills-ci.yml	Workflow name, PR triggers targeting main, and repository/PR read-only permissions are defined.
Validation env and script logic .github/workflows/require-nvskills-ci.yml	Env defaults for trusted NVSkills actor/title and onboarding URL are set; script paginates PR files, detects skills/ changes, fetches PR head commit, derives author/title, and fails the check on mismatch.
Request workflow actor fallback .github/workflows/request-nvskills-ci.yml	Changes push-job default github.actor fallback from nv-skills-ci[bot] to svc-nvskills-signing, preserving repo var override.
CI README updates .github/CI_README.md	Documents request-nvskills-ci.yml (comment dispatch /nvskills-ci) and require-nvskills-ci.yml (merge-blocking signed commit/title requirement) and adds onboarding/debug link.
Replace skill DSSE sigstore bundle skills/nemo-evaluator-plugin/skill.oms.sig	Replaced the DSSE JSON bundle: updated payload subject/resource digests, serialization ignore-paths, and signature value.
Temporary SKILL test comment skills/nemo-evaluator-plugin/SKILL.md	Inserted a short temporary comment for NVSkills CI signature testing.
Skill docs, card, and benchmark updates skills/nemo-evaluator-plugin/BENCHMARK.md, skills/nemo-evaluator-plugin/skill-card.md	Updated benchmark date and per-dimension metrics; revised license/use-case/output/evaluation text and results table rows in skill-card.

Sequence Diagram

sequenceDiagram
  participant PR as PullRequest
  participant Actions as GitHubActions
  participant GHAPI as GitHubAPI
  PR->>Actions: PR opened or synchronized
  Actions->>GHAPI: list changed files
  GHAPI->>Actions: return file list (skills/ detected)
  Actions->>GHAPI: get PR head commit
  GHAPI->>Actions: return commit author and message
  Actions->>Actions: validate actor == SIGNATURE_ACTOR and title startsWith SIGNATURE_TITLE
  Actions-->>PR: set check success or failure

Loading

Possibly related PRs

• NVIDIA-NeMo/nemo-platform#103: Updates the request-nvskills-ci.yml workflow trigger/actor logic, related to the request workflow change here.

Suggested reviewers

• gabwow
• philipmattingly
• svvarom

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	Title accurately describes the main change: adding a required NVSkills validation workflow for skill-related PRs, which is the central feature of the changeset.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch ngoncharenko/aalgo-233-skills-update-guard

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/require-nvskills-ci.yml:
- Line 17: The workflow is using the unpinned action reference "uses:
actions/github-script@v7"; replace this with a specific commit SHA to prevent
supply-chain breakage by locating the line with uses: actions/github-script@v7
(and the other occurrence in docs.yaml) and change it to uses:
actions/github-script@<commit-sha> where <commit-sha> is the full 40-character
commit hash from the actions/github-script repository (fetch the current
recommended pinned SHA from the action's repo/tags) so both workflows use the
same immutable ref.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 186fe18f-fc35-418e-a8ef-d21673ebbf85

📥 Commits

Reviewing files that changed from the base of the PR and between 8e65012 and 2e41be6.

📒 Files selected for processing (2)

• .github/CI_README.md
• .github/workflows/require-nvskills-ci.yml

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@skills/.nvskills-ci-signature-test.md`:
- Line 1: The file's first line must be a top-level heading to satisfy MD041;
replace or prepend the current first line "Temporary NVSkills CI signature test
file." with an H1 heading (e.g., "# Temporary NVSkills CI signature test file.")
so the document begins with a single leading "#" heading on line 1.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 2bcbc78e-634e-4d50-a26c-44a0d7065249

📥 Commits

Reviewing files that changed from the base of the PR and between 2e41be6 and 77d2b4e.

📒 Files selected for processing (1)

• skills/.nvskills-ci-signature-test.md

[svvarom]

/nvskills-ci

[svvarom]

/nvskills-ci

[svvarom]

/nvskills-ci