[Infra] feat: update build-validator job and README for Node.js version requirement - #86
Closed
ViktorHofer wants to merge 9 commits into
Closed
[Infra] feat: update build-validator job and README for Node.js version requirement#86ViktorHofer wants to merge 9 commits into
ViktorHofer wants to merge 9 commits into
Conversation
…rement - Update an existing comment if it exists with the total summary - Build the skill-validator just once and upload - Clean-up when passing CLI args to skill-validator
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the evaluation workflow to avoid rebuilding eng/skill-validator in every matrix job, and clarifies the Node.js version requirement for running the validator.
Changes:
- Adds a dedicated
build-validatorjob that buildseng/skill-validatoronce and uploads it as an artifact for downstream jobs. - Cleans up CLI argument construction for
skill-validatorexecution inevaluate. - Updates the workflow to consolidate per-component summaries into a single PR comment, and updates the README Node.js prerequisite.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
eng/skill-validator/README.md |
Updates documented Node.js prerequisite to match the validator’s runtime requirements. |
.github/workflows/evaluation.yml |
Builds validator once, reuses it across matrix runs, and consolidates/upserts PR comments for evaluation summaries. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Reduces artifact size by removing devDependencies (typescript, vitest, etc.) from node_modules/ since the evaluate job only needs the built dist/. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…hofer/skills into ImproveEvaluationWorkflow
Move GITHUB_STEP_SUMMARY writing from per-matrix Generate Summary step into the existing Consolidate summaries step in comment-on-pr job. This produces a single unified summary instead of fragmented per-component summaries. The job now runs for all event types, with the PR comment step scoped to pull_request events only. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Remove --no-save-results option and saveRunResults function - Reporters no longer accept inline paths (e.g., json:path) - Add --results-dir option: when specified, file reporters (json, junit, markdown) write to that directory; when omitted, they write to console - Enhance markdown reporter with per-scenario judge reports including rubric scores, reasoning, and agent output - Update workflow, README, and CONTRIBUTING docs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…kdown reporters by default Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Resolve conflicts in evaluation.yml and CONTRIBUTING.md. - evaluation.yml: adopt entry-based matrix from #76, keep build-validator job and consolidated comment-on-pr from this branch - CONTRIBUTING.md: keep full paths (eng/skill-validator/dist/index.js) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
moesac0970
pushed a commit
to moesac0970/skills
that referenced
this pull request
Jul 4, 2026
…he only mechanism
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.