Add performance-fixer skill + make the benchmark harness AI-ready#4361
Merged
Conversation
Add a `performance-fixer` skill that periodically scans the managed C# binding for a hot-path performance opportunity, proves it with a BenchmarkDotNet New-vs-Old measurement AND a behaviour-parity test, fixes it (managed-only, ABI-safe), and opens a linked issue + draft PR. Modelled on `memory-leak-fixer` and grounded in real perf PRs (#4241 SKMatrix native->managed, #4345 SKColor alloc-free parse, #4247 SKSurface.Canvas caching). The skill's defining discipline is two proofs: every fix must be shown FASTER (benchmark) and behaviour-IDENTICAL (equivalence test, bit-exact vs SkiaApi.sk_* for numeric ports) - a faster-but-different answer is a rendering regression, not a win. References are split by area (mirroring dotnet/aspnetcore's optimize-perf skill): hot-paths/ (SkiaSharp code areas) + bcl-patterns/ (.NET techniques), plus decision-framework, measuring, signals, and repo-helpers. A gh-aw workflow runs the skill on a schedule. Make the benchmark harness AI-ready so a run never has to repair infra: - Replace the broken TheBenchmark.cs (its [SimpleJob(Net48/70/80)] attributes silently skip off-Windows and its empty bodies taught none of the conventions) with a working TemplateBenchmark.cs: a New-vs-Old + [MemoryDiagnoser] + [Params] + return-sink skeleton that runs out of the box. - Add benchmarks/README.md: the on-ramp (add/run commands, --list flat and --job short for fast iteration, single-TFM warning, New-vs-Old + sink conventions, InternalsVisibleTo native-oracle note). Also register /performance-fixer in AGENTS.md's command tables. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
📦 Try the packages from this PRWarning Do not run these scripts without first reviewing the code in this PR. Step 1 — Download the packages bash / macOS / Linux: curl -fsSL https://raw.githubusercontent.com/mono/SkiaSharp/main/scripts/get-skiasharp-pr.sh | bash -s -- 4361PowerShell / Windows: iex "& { $(irm https://raw.githubusercontent.com/mono/SkiaSharp/main/scripts/get-skiasharp-pr.ps1) } 4361"Step 2 — Add the local NuGet source dotnet nuget add source ~/.skiasharp/hives/pr-4361/packages --name skiasharp-pr-4361More options
Or download manually from Azure Pipelines — look for the Remove the source when you're done: dotnet nuget remove source skiasharp-pr-4361 |
Contributor
|
📖 Documentation Preview The documentation for this PR has been deployed and is available at: 🔗 View Staging Site This preview will be updated automatically when you push new commits to this PR. This comment is automatically updated by the documentation staging workflow. |
…gory Every issue/PR the workflow emits now gets tenet/performance automatically, plus exactly one agent-chosen perf/* category label (perf/interop, perf/allocations, perf/rendering, perf/throughput, perf/startup, perf/memory-leak, perf/size) decided by the dominant measured driver of the win. Wired via safe-outputs allowed-labels (both create-issue and create-pull-request) and a Step 2 guardrail decision table; documented in the skill's Phase 4. Recompiled the workflow lock. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
mattleibow
added a commit
that referenced
this pull request
Jul 7, 2026
Align the memory-leak-fixer's terminology with the performance-fixer skill (PR #4361): the 11 rotating leak categories are now 'focus areas' rather than 'families', and the workflow's focus_family input is renamed to focus_area. Reword the single overarching 'leak family' to 'class of leaks' to drop the term entirely. - types-of-leaks.md: intro, table header (Family -> Focus area), and cross-references (family N -> area N). - SKILL.md: catalogue framing, Phase 1.1 round-robin, and all references. - memory-leak-fixer workflow: focus_family -> focus_area input + body. - Recompile the .lock.yml. No behavioural change — terminology only. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
mattleibow
added a commit
that referenced
this pull request
Jul 7, 2026
… areas (#4362) [skills] Add perf/* label taxonomy and rework memory-leak-fixer focus areas (#4362) Two related improvements to the agentic-workflow skills. Memory-leak findings previously landed with only the `agentic-workflows` label — a label read by nothing and present on zero issues/PRs — so leaks and performance reports had no meaningful classification, and the triage skill had no rule mapping either to a quality tenet. ~~ perf/* label taxonomy ~~ There is no dedicated "memory-leak" label, and grouping every speed, memory, or size concern under the single `tenet/performance` umbrella loses the sub-type. Add a `perf/*` axis beneath that umbrella, grounded in the repo's real corpus (61 tenet/performance + ~25 memory-leak issues): perf/memory-leak, perf/allocations, perf/interop, perf/rendering, perf/throughput, perf/startup, perf/size. Any perf/* implies tenet/performance. * memory-leak-fixer workflow: drop agentic-workflows; label the finding issue and fix PR with tenet/performance + perf/memory-leak. * issue-triage skill: add the perf axis to labels.md, the schema (classifiedPerf enum + validated perf[] field), the cheatsheet, SKILL.md, and the report template. * auto-triage workflow: apply classification.perf[] labels; raise the add-labels cap 10 -> 12 to fit the new axis. Type classification (type/bug vs type/enhancement) is left to triage on purpose — a leak is usually a bug, but a "make it faster" report may be an enhancement. ~~ "family" -> "focus area" rework ~~ Align the memory-leak-fixer's vocabulary with the new performance-fixer skill (#4361), which routes over "focus areas" and a focus_area knob. The 11 rotating leak categories are now focus areas rather than families, and the workflow's focus_family dispatch input is renamed to focus_area. The single overarching "leak family" becomes "class of leaks" so the term is dropped entirely. Terminology only — no behavioural change. Both .lock.yml files were regenerated with `gh aw compile` (0 errors); the perf/* labels were created in mono/SkiaSharp. Co-authored-by: Matthew Leibowitz <mattleibow@live.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Merge main and match the perf/* label taxonomy the memory-leak-fixer now uses: apply tenet/performance automatically and let the agent add the matching perf/* sub-type from .agents/skills/issue-triage/references/labels.md (chosen by the win's dominant driver — a removed P/Invoke maps to perf/interop, whose taxonomy example is literally "remove native interop in SKMatrix"). Drop the agentic-workflows label to match the sibling workflow, reference the shared taxonomy instead of duplicating a table, and recompile. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
mattleibow
marked this pull request as ready for review
July 8, 2026 07:47
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a
performance-fixerskill — a periodic AI scanner that finds a hot-path performance opportunity in the managed C# binding, proves it, fixes it (managed-only, ABI-safe), and opens a linked issue + draft PR — plus source changes that make the benchmark harness easy for that skill (and humans) to write and run benchmarks against.Modelled on the existing
memory-leak-fixerskill and on a colleague'soptimize-aspnetcore-performanceskill, and grounded in our real perf PRs:ReadOnlySpanoverloadThe defining discipline: two proofs
Unlike a generic perf pass, every fix must be shown faster (a BenchmarkDotNet
NewvsOldmeasurement) and behaviour-identical (an equivalence test — bit-exact vsSkiaApi.sk_*for numeric ports). A faster answer that differs from Skia is a rendering regression, not a win. The skill also hard-codes the traps we learned the hard way: the x87 float-determinism fallback, the ARM64Vector2565.7–6.5× regression, benchmark statistical rigor, and full behaviour parity (exceptions / ownership /GC.KeepAlive/ rendered pixels).Structure
References are split by area (the aspnetcore pattern):
hot-paths/(SkiaSharp code areas) +bcl-patterns/(.NET techniques), plusdecision-framework.md(impact×complexity rubric),measuring.md,signals.md(detection routing), andrepo-helpers.md. A lean routerSKILL.mdties them together.A gh-aw workflow (
.github/workflows/performance-fixer.md) runs the skill on a schedule (every 12h, dry-run on PR, focus-area knob), compiled to.lock.yml.Benchmark harness: ready in the source
So a skill run never wastes time repairing infra:
TheBenchmark.cs→TemplateBenchmark.cs. The old template was a trap:[SimpleJob(Net48/Net70/Net80)]silently skips on macOS/Linux (verified) and its empty bodies taught none of the conventions. The new one is a workingNewvsOld+[MemoryDiagnoser]+[Params]+ return-sink skeleton that runs out of the box (verified:Old88/1048 B →New0 B).benchmarks/README.md— the on-ramp: how to add/run a benchmark,--list flatfor discovery and--job shortfor fast iteration, the single-TFM warning, the New-vs-Old + sink conventions, and theInternalsVisibleTonative-oracle note.Labels
Every finding is labelled
tenet/performance(the quality-tenet umbrella) plus one agent-chosenperf/*sub-type, decided by the win's dominant measured driver (a removed P/Invoke →perf/interop, removed managed allocations →perf/allocations, etc.). This aligns to the canonical taxonomy in.agents/skills/issue-triage/references/labels.md(#4362) and matches thememory-leak-fixerconvention, so agentic perf findings triage the same way as human-filed ones.Verification
quick_validate; all cross-reference links resolve; the gh-aw workflow compiles with 0 errors / 0 warnings.externals-download→ built the benchmark project → ranSKColorParseBenchmark's full New-vs-Old comparison and the newTemplateBenchmark(clean,Ratio/Allocatedpopulated).StringUtilitiesvsUtilsnaming, TFM guards).Validated end to end on real CI
The workflow self-tests on any PR touching the skill via a forced dry-run. Those runs demonstrated the full range of correct behaviour:
SKFourByteTag.Parse'schar[4]is already stack-allocated by .NET 9/10 escape analysis, so it emitted anooprather than a bogus "removes per-frame allocation" PR.SKShaper.Shapethrowaway glyph-array extraction (~30% faster, 55–71% fewer allocations →perf/allocations) and theSKColor→SKColorFoperator P/Invoke (→perf/interop), each with a New-vs-Old benchmark run twice and a mutation-checked bit-exact equivalence test.SKColor→SKColorFport), both carryingtenet/performance+perf/interop, confirming the labels land on the issue and the PR.Notes
AGENTS.md. No binding/runtime code changes.SKColor→SKColorFport is a legitimate optimization).Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com