Skip to content

Add public API governance instruction evaluation - #68698

Draft
PureWeen wants to merge 1 commit into
mainfrom
pureween-public-api-instruction-eval
Draft

Add public API governance instruction evaluation#68698
PureWeen wants to merge 1 commit into
mainfrom
pureween-public-api-instruction-eval

Conversation

@PureWeen

@PureWeen PureWeen commented Aug 21, 2026

Copy link
Copy Markdown
Member

Summary

  • adds a specialized Vally regression suite for the repository-wide public API governance instructions in Add public API governance instructions #68168
  • covers an unapproved new public member, shipped interface signature change, and shipped default/convention change
  • loads .github/copilot-instructions.md into Vally's isolated execution environment and checks the complete API-review gate

This is the top layer over #68168 and targets pureween-public-api-instructions. It follows the evaluation infrastructure merged in #68628 and adds no Vally onboarding infrastructure.

Validation

Validated in a synthesized checkout containing current main's #68628 infrastructure, #68168's instruction changes, and this stack layer:

  • pwsh -NoProfile -File ./eng/skill-evals/run.ps1 Test
  • pwsh -NoProfile -File ./eng/skill-evals/run.ps1 Validate -Eval eng/skill-evals/copilot-instructions/public-api-governance.vally.yaml
  • one current-behavior run with --runs 1 --workers 3: 100%, all graders passed for all three stimuli
  • git diff --check

@PureWeen
PureWeen changed the base branch from main to pureween-public-api-instructions August 21, 2026 16:38
Base automatically changed from pureween-public-api-instructions to main August 25, 2026 15:52
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@PureWeen
PureWeen force-pushed the pureween-public-api-instruction-eval branch from c80cac4 to ba87f27 Compare August 25, 2026 15:52
@Youssef1313 Youssef1313 added the area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework label Sep 2, 2026
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Hey @dotnet/aspnet-build, looks like this PR is something you want to take a look at.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants