Fix infra change detection to use merge-base instead of base tip by ViktorHofer · Pull Request #285 · dotnet/skills

ViktorHofer · 2026-03-09T07:39:55Z

The evaluation workflow was using a two-point diff (base_sha..head_sha) to detect changed files in PRs. This includes files changed on main that the PR branch hasn't merged yet, causing false Infrastructure changes detected triggers.

For example, this run on PR #75 triggered infra detection because evaluation-run.yml was modified on main (by PR #284) after the PR branch diverged — even though the PR itself never touched any evaluation workflow files.

Fix: compute the merge-base between the base branch tip and the PR head, then diff against that. This ensures only changes actually introduced by the PR are considered.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

The evaluation workflow was using a two-point diff (base_sha..head_sha) to detect changed files, which includes files changed on main that the PR branch hasn't merged yet. This caused false 'Infrastructure changes detected' triggers when evaluation workflow files were modified on main after a PR branch diverged. Switch to computing the merge-base first, so only changes actually introduced by the PR are considered. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR fixes false “Infrastructure changes detected” triggers in the evaluation workflow by diffing PR changes against the PR’s merge-base instead of the base branch tip, ensuring only files actually introduced by the PR are considered.

Changes:

Compute git merge-base between PR base and head commits.
Diff changed files from merge-base → head when determining infra vs. skill changes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

github-actions · 2026-03-09T08:12:48Z

Skill Validation Results

Skill	Scenario	Baseline	With Skill	Δ	Skills Loaded	Overfit	Verdict
csharp-scripts	Test a C# language feature with a script	3.0/5	5.0/5	+2.0	✅ csharp-scripts; tools: skill, create, edit	🟡 0.32	✅
nuget-trusted-publishing	Set up trusted publishing for a new NuGet library	3.0/5	4.0/5	+1.0	✅ nuget-trusted-publishing; tools: skill	✅ 0.11	✅
nuget-trusted-publishing	Set up NuGet publishing without mentioning trusted publishing	2.0/5	5.0/5	+3.0	✅ nuget-trusted-publishing; tools: skill, glob, view	✅ 0.11	✅
nuget-trusted-publishing	Migrate existing workflow from API key to trusted publishing	2.0/5	5.0/5	+3.0	✅ nuget-trusted-publishing; tools: skill	✅ 0.11	✅
dotnet-pinvoke	Generate LibraryImport declaration from C header (.NET 8+)	3.0/5	5.0/5	+2.0	✅ dotnet-pinvoke; tools: skill	✅ 0.09	✅
dotnet-pinvoke	Generate LibraryImport declaration from C header (.NET Framework)	5.0/5	5.0/5	0.0	✅ dotnet-pinvoke; tools: skill	✅ 0.09	❌
dotnet-trace-collect	High CPU in Kubernetes on Linux (.NET 8)	4.0/5	5.0/5	+1.0	✅ dotnet-trace-collect; tools: skill, report_intent, view, glob	✅ 0.12	✅
dotnet-trace-collect	.NET Framework on Windows without admin privileges	4.0/5	5.0/5	+1.0	✅ dotnet-trace-collect; tools: skill	✅ 0.12	✅
dotnet-trace-collect	.NET 10 on Linux with root access and native call stacks	1.0/5	4.0/5	+3.0	✅ dotnet-trace-collect; tools: skill	✅ 0.12	✅
dotnet-trace-collect	Memory leak on Linux (.NET 8)	3.0/5	3.0/5	0.0	✅ dotnet-trace-collect; tools: skill, report_intent, view, glob	✅ 0.12	✅
dotnet-trace-collect	Slow requests on Windows with PerfView	5.0/5	5.0/5	0.0	✅ dotnet-trace-collect; tools: skill, report_intent, view	✅ 0.12	✅
dotnet-trace-collect	Excessive GC on Linux (.NET 8)	3.0/5	4.0/5	+1.0	✅ dotnet-trace-collect; tools: skill	✅ 0.12	✅
dotnet-trace-collect	Hang or deadlock diagnosis on Linux	3.0/5	5.0/5	+2.0	✅ dotnet-trace-collect; tools: skill, report_intent, view	✅ 0.12	✅
dotnet-trace-collect	Windows container high CPU with PerfView	1.0/5	5.0/5	+4.0	✅ dotnet-trace-collect; tools: report_intent, skill, view	✅ 0.12	✅
dotnet-trace-collect	Long-running intermittent issue with PerfView triggers	3.0/5	5.0/5	+2.0	✅ dotnet-trace-collect; tools: skill, report_intent, view	✅ 0.12	✅
dotnet-trace-collect	Linux pre-.NET 10 needing native call stacks	4.0/5	5.0/5	+1.0	✅ dotnet-trace-collect; tools: skill, report_intent, view, glob	✅ 0.12	✅
dotnet-trace-collect	Windows modern .NET with admin high CPU	2.0/5	5.0/5	+3.0	✅ dotnet-trace-collect; tools: skill, report_intent, view	✅ 0.12	✅
dotnet-trace-collect	Memory leak on .NET Framework Windows	3.0/5	5.0/5	+2.0	✅ dotnet-trace-collect; tools: skill, report_intent, view	✅ 0.12	✅
dotnet-trace-collect	Kubernetes with console access prefers console tools	4.0/5	5.0/5	+1.0	✅ dotnet-trace-collect; tools: skill	✅ 0.12	✅
dotnet-trace-collect	Container installation without .NET SDK	4.0/5	5.0/5	+1.0	✅ dotnet-trace-collect; tools: skill	✅ 0.12	✅
dotnet-trace-collect	HTTP 500s from downstream service on Linux (.NET 8)	4.0/5	5.0/5	+1.0	✅ dotnet-trace-collect; tools: skill, report_intent, view	✅ 0.12	✅
dotnet-trace-collect	Networking timeouts on Windows with admin (.NET 8)	2.0/5	5.0/5	+3.0	✅ dotnet-trace-collect; tools: skill, report_intent, view	✅ 0.12	✅
microbenchmarking	Investigate runtime upgrade performance impact	5.0/5	5.0/5	0.0	✅ microbenchmarking; tools: skill, glob	✅ 0.11	❌
clr-activation-debugging	Diagnose unexpected FOD dialog from native build tool	1.0/5	5.0/5	+4.0	✅ clr-activation-debugging; tools: skill	✅ 0.08	✅
clr-activation-debugging	Diagnose FOD suppressed but activation still failing	1.0/5	5.0/5	+4.0	✅ clr-activation-debugging; tools: skill	✅ 0.08	✅
clr-activation-debugging	Explain why same binary behaves differently under different launch methods	1.0/5	5.0/5	+4.0	✅ clr-activation-debugging; tools: skill	✅ 0.08	✅
clr-activation-debugging	Analyze healthy managed EXE activation	1.0/5	5.0/5	+4.0	✅ clr-activation-debugging; tools: skill	✅ 0.08	✅
clr-activation-debugging	Identify multiple activation sequences in a single log	1.0/5	5.0/5	+4.0	✅ clr-activation-debugging; tools: skill	✅ 0.08	✅
clr-activation-debugging	Explain useLegacyV2RuntimeActivationPolicy in activation log	2.0/5	3.0/5	+1.0	✅ clr-activation-debugging; tools: skill	✅ 0.08	✅
clr-activation-debugging	Decline non-CLR-activation issue	1.0/5	5.0/5	+4.0	ℹ️ not activated (expected)	✅ 0.08	✅
analyzing-dotnet-performance	Detects compiled regex startup budget and regex chain allocations	1.0/5	4.0/5 ⏰ timeout	+3.0	✅ analyzing-dotnet-performance; tools: skill	✅ 0.13	✅
analyzing-dotnet-performance	Detects CurrentCulture comparer and compiled regex budget in inflection rules	1.0/5	5.0/5	+4.0	✅ analyzing-dotnet-performance; tools: skill	✅ 0.13	✅
analyzing-dotnet-performance	Finds per-call Dictionary allocation not hoisted to static	1.0/5	5.0/5	+4.0	✅ analyzing-dotnet-performance; tools: skill	✅ 0.13	✅
analyzing-dotnet-performance	Catches compound allocations in recursive number converter with ToLower	1.0/5	5.0/5	+4.0	✅ analyzing-dotnet-performance; tools: skill	✅ 0.13	✅
analyzing-dotnet-performance	Finds StringComparison.Ordinal missing and FrozenDictionary opportunities	1.0/5	5.0/5	+4.0	✅ analyzing-dotnet-performance; tools: skill, task, glob, grep	✅ 0.13	✅
analyzing-dotnet-performance	Detects Aggregate+Replace chain and struct missing IEquatable	1.0/5	5.0/5	+4.0	✅ analyzing-dotnet-performance; tools: skill	✅ 0.13	✅
analyzing-dotnet-performance	Finds branched Replace chain in format string manipulation	1.0/5	4.0/5	+3.0	✅ analyzing-dotnet-performance; tools: skill	✅ 0.13	✅
analyzing-dotnet-performance	Catches LINQ on hot-path string processing and All(char.IsUpper)	1.0/5	5.0/5	+4.0	✅ analyzing-dotnet-performance; tools: skill	✅ 0.13	✅
analyzing-dotnet-performance	Detects LINQ pipeline in TimeSpan formatting and collection processing	1.0/5	5.0/5	+4.0	✅ analyzing-dotnet-performance; tools: skill	✅ 0.13	✅
analyzing-dotnet-performance	Flags Span inconsistencies and compound method chains in truncation library	1.0/5	5.0/5	+4.0	✅ analyzing-dotnet-performance; tools: skill	✅ 0.13	✅
analyzing-dotnet-performance	Identifies unsealed leaf classes and locale hierarchy patterns	1.0/5	5.0/5	+4.0	✅ analyzing-dotnet-performance; tools: skill	✅ 0.13	✅
android-tombstone-symbolication	Symbolicate .NET frames in an Android tombstone	4.0/5	5.0/5	+1.0	✅ android-tombstone-symbolication; tools: skill, glob, stop_bash	✅ 0.14	✅
android-tombstone-symbolication	Recognize tombstone with no .NET frames	5.0/5	5.0/5	0.0	✅ android-tombstone-symbolication; tools: skill	✅ 0.14	❌
android-tombstone-symbolication	Symbolicate CoreCLR frames in an Android tombstone	1.0/5 ⏰ timeout	4.0/5	+3.0	✅ android-tombstone-symbolication; tools: skill, stop_bash	✅ 0.14	✅
android-tombstone-symbolication	Recognize NativeAOT tombstone with app binary and libSystem.Native.so	3.0/5	4.0/5	+1.0	✅ android-tombstone-symbolication; tools: skill, bash, glob	✅ 0.14	✅
android-tombstone-symbolication	Symbolicate multi-thread tombstone	4.0/5	5.0/5	+1.0	✅ android-tombstone-symbolication; tools: skill	✅ 0.14	✅
android-tombstone-symbolication	Handle .NET frames with no BuildId metadata	3.0/5	5.0/5	+2.0	✅ android-tombstone-symbolication; tools: skill	✅ 0.14	✅
android-tombstone-symbolication	Symbolicate tombstone with multiple .NET libraries and different BuildIds	3.0/5	3.0/5	0.0	✅ android-tombstone-symbolication; tools: skill, read_bash	✅ 0.14	❌
android-tombstone-symbolication	Reject iOS crash log as wrong format	5.0/5	5.0/5	0.0	ℹ️ not activated (expected)	✅ 0.14	❌
dump-collect	Configure automatic crash dumps for CoreCLR app on Linux	3.0/5	5.0/5	+2.0	✅ dump-collect; tools: skill, report_intent, view, glob	✅ 0.07	✅
dump-collect	Set up NativeAOT crash dumps with createdump in Kubernetes	2.0/5	5.0/5	+3.0	✅ dump-collect; tools: skill	✅ 0.07	✅
dump-collect	Recover crash dump from macOS NativeAOT without createdump	4.0/5	4.0/5	0.0	✅ dump-collect; tools: skill, report_intent, view, glob, bash	✅ 0.07	✅
dump-collect	Configure CoreCLR dump collection in Alpine Docker as non-root	4.0/5	4.0/5	0.0	✅ dump-collect; tools: skill, report_intent, view, glob	✅ 0.07	✅
dump-collect	Advisory: macOS NativeAOT crash dump recovery steps	4.0/5	5.0/5	+1.0	✅ dump-collect; tools: skill	✅ 0.07	✅
dump-collect	Advisory: CoreCLR Alpine Docker non-root configuration	4.0/5	5.0/5	+1.0	✅ dump-collect; tools: skill, report_intent, view, glob, bash	✅ 0.07	✅
dump-collect	Advisory: NativeAOT Kubernetes dump collection setup	2.0/5	5.0/5	+3.0	✅ dump-collect; tools: skill	✅ 0.07	✅
dump-collect	Detect runtime and configure crash dumps for unknown .NET app on Linux	4.0/5	4.0/5	0.0	✅ dump-collect; tools: skill	✅ 0.07	❌
dump-collect	Decline dump analysis request	4.0/5	4.0/5	0.0	ℹ️ not activated (expected)	✅ 0.07	✅
optimizing-ef-core-queries	Optimize bulk operations with EF Core 7+ ExecuteUpdate and ExecuteDelete	5.0/5	5.0/5	0.0	✅ optimizing-ef-core-queries; tools: skill	🟡 0.21	❌
build-parallelism	Analyze build parallelism bottlenecks	4.0/5	4.0/5	0.0	✅ build-parallelism; binlog-generation; tools: skill, glob	✅ 0.15	✅
including-generated-files	Diagnose generated file inclusion failure	3.0/5	5.0/5	+2.0	✅ including-generated-files; tools: skill	🟡 0.24	✅
msbuild-antipatterns	Review MSBuild files for anti-patterns and style issues	5.0/5	5.0/5	0.0	✅ msbuild-antipatterns; tools: skill, task	✅ 0.07	❌
build-perf-baseline	Establish build performance baseline and recommend optimizations	4.0/5	4.0/5	0.0	✅ build-perf-baseline; tools: skill	🟡 0.30	✅
msbuild-modernization	Modernize legacy project to SDK-style	5.0/5	5.0/5	0.0	✅ msbuild-modernization; tools: skill, glob, edit, bash	✅ 0.05	✅
directory-build-organization	Organize build infrastructure for a multi-project repo	3.0/5	5.0/5	+2.0	✅ msbuild-antipatterns; directory-build-organization; tools: skill	✅ 0.15	✅
check-bin-obj-clash	Diagnose bin/obj output path clashes	3.0/5	5.0/5	+2.0	✅ check-bin-obj-clash; tools: skill, glob	✅ 0.15	❌
incremental-build	Analyze incremental build issues	3.0/5	5.0/5	+2.0	✅ incremental-build; tools: skill, bash	✅ 0.13	✅
eval-performance	Analyze MSBuild evaluation performance issues	4.0/5	5.0/5	+1.0	✅ eval-performance; tools: skill	✅ 0.07	✅
build-perf-diagnostics	Analyze analyzer performance impact on builds	5.0/5	3.0/5 ⏰ timeout	-2.0	✅ binlog-generation; build-perf-diagnostics; tools: skill, edit	🟡 0.29	❌
binlog-generation	Build project with /bl flag	1.0/5	5.0/5	+4.0	✅ binlog-generation; tools: skill	✅ 0.00	❌
binlog-generation	Build with /bl in PowerShell	3.0/5	5.0/5	+2.0	✅ binlog-generation; tools: skill	✅ 0.00	✅
binlog-generation	Build multiple configurations with unique binlogs	3.0/5	5.0/5	+2.0	✅ binlog-generation; tools: skill	✅ 0.00	❌
binlog-failure-analysis	Diagnose build failures from binlog only (no source files)	4.0/5	5.0/5	+1.0	✅ binlog-failure-analysis; tools: skill	✅ 0.04	✅
thread-abort-migration	Worker thread with abort-based cancellation	5.0/5	5.0/5	0.0	✅ thread-abort-migration; tools: skill	✅ 0.09	✅
thread-abort-migration	Timeout enforcement via Thread.Abort	4.0/5	5.0/5	+1.0	✅ thread-abort-migration; tools: skill	✅ 0.09	❌
thread-abort-migration	Blocking WaitHandle with Thread.Interrupt	4.0/5	4.0/5	0.0	✅ thread-abort-migration; tools: skill	✅ 0.09	✅
thread-abort-migration	ASP.NET Response.End and Response.Redirect with Thread.Abort	4.0/5	5.0/5	+1.0	✅ thread-abort-migration; tools: skill	✅ 0.09	✅
thread-abort-migration	Thread.Join and Thread.Sleep only — should not migrate	3.0/5	5.0/5	+2.0	✅ thread-abort-migration; tools: skill	✅ 0.09	✅
migrate-nullable-references	Enable NRT in a small library with mixed nullability	5.0/5	5.0/5	0.0	✅ migrate-nullable-references; tools: skill, glob	✅ 0.13	❌
migrate-nullable-references	File-by-file migration: only modify the targeted file	5.0/5	5.0/5	0.0	⚠️ NOT ACTIVATED	✅ 0.13	❌
migrate-nullable-references	Enable NRT in ASP.NET Core Web API with EF Core	3.0/5	3.0/5	0.0	✅ migrate-nullable-references; tools: skill	✅ 0.13	❌
dotnet-aot-compat	Make Azure.ResourceManager AOT-compatible	2.0/5	5.0/5	+3.0	✅ dotnet-aot-compat; tools: skill, create, read_agent	✅ 0.14	✅

⏰ timeout — run hit the scenario timeout limit; scoring may be impacted by aborting model execution before it could produce its full output

Model: claude-opus-4.6 | Judge: claude-opus-4.6

Full results

ViktorHofer requested review from a team and Copilot March 9, 2026 07:39

ViktorHofer requested a review from JanKrivanek as a code owner March 9, 2026 07:39

Copilot started reviewing on behalf of ViktorHofer March 9, 2026 07:40 View session

Copilot AI reviewed Mar 9, 2026

View reviewed changes

ViktorHofer enabled auto-merge (squash) March 9, 2026 08:21

JanKrivanek approved these changes Mar 9, 2026

View reviewed changes

ViktorHofer merged commit 7884803 into main Mar 9, 2026
14 checks passed

ViktorHofer deleted the fix/evaluation-merge-base branch March 9, 2026 08:53

danmoseley added the infrastructure label Mar 9, 2026

Navigation Menu

Uh oh!

Fix infra change detection to use merge-base instead of base tip - #285

Fix infra change detection to use merge-base instead of base tip#285
ViktorHofer merged 1 commit into
mainfrom
fix/evaluation-merge-base

ViktorHofer commented Mar 9, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

github-actions Bot commented Mar 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

ViktorHofer commented Mar 9, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

github-actions Bot commented Mar 9, 2026

Skill Validation Results

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants