simd modifying existing code skill - #26
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces two complementary skills for identifying and optimizing SIMD (Single Instruction, Multiple Data) opportunities in existing .NET 8+ codebases. The skills are designed to help AI agents scan codebases for scalar operations that would benefit from vectorization using System.Runtime.Intrinsics.
Changes:
- Added
simd-vector-mathskill (355 lines) for optimizing floating-point vector operations like cosine similarity, dot products, L2 distance, batch normalization, and reduction operations - Added
simd-pattern-matchingskill (274 lines) for optimizing byte/string search, character class membership checks, and pattern matching operations - Included comprehensive test scenarios (eval.yaml) for both skills with positive optimization cases and negative "no opportunity" cases to teach appropriate application boundaries
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| skills/simd-vector-math/SKILL.md | Defines skill for identifying and optimizing scalar floating-point vector math operations (Category A-D patterns: distance computations, batch arithmetic, reductions, lookups) with platform-adaptive SIMD implementations |
| skills/simd-vector-math/tests/eval.yaml | Test scenarios covering cosine similarity, dot product/L2 distance, min-max normalization optimization, plus a negative case (string processing) with no SIMD opportunity |
| skills/simd-pattern-matching/SKILL.md | Defines skill for identifying and optimizing scalar byte/string pattern matching operations (byte search, character class checks, fuzzy matching, bulk scanning) with SIMD acceleration |
| skills/simd-pattern-matching/tests/eval.yaml | Test scenarios covering byte pattern search, alphanumeric counting, plus a negative case (config parser) with no SIMD opportunity |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
jeffschwMSFT
changed the title
|
simd-pattern-matching
|
|
simd-vector-math
|
|
thoguhts about my comment above? |
I was not intentionally ignoring, but I am not sure so when I ask an expert I am hoping they would answer :) |
|
A bunch of @tannergooding feedback I'd expect copilot could infer from instructing it to read our past blog posts, etc. I found that super useful for the NRT annotation one to gather more "team wisdom" |
There is also https://github.com/dotnet/runtime/blob/e5f78931886de441a0c9ffd26ad373a0c1d1bc77/docs/coding-guidelines/vectorization-guidelines.md, which is the general "coding guidelines" for anyone adding SIMD to the repo. We also have some other documents such as https://github.com/dotnet/runtime/blob/e5f78931886de441a0c9ffd26ad373a0c1d1bc77/docs/design/coreclr/botr/vectors-and-intrinsics.md which cover other details and how SIMD support is setup in the JIT/VM |
|
the reason I started this skill was at copilot's request :) when exploring areas of potential need, SIMD'ifying existing code was elevated as a weak spot.
It likely does, but with such a large corpus of information it needs to be guided.
This is a great point, I am curious if you feel there are patterns that would benefit from SIMD? eg. are there are parts of a skill that make sense, or is our strategy to keep improving the framework and less about moving customer code. |
|
I would expect us to evaluate on a per scenario basis. In most cases we want to use existing centralized helper APIs where possible, like those exposed on Span (often via MemoryExtensions) or on TensorPrimitives. There are then special cases that warrant their own vectorization support, like many of the transcoding APIs. In many cases, if we find an API that would benefit from vectorization but where we do not have an existing centralized helper, we want to consider if exposing a new centralized API is appropriate and vectorize that instead; only vectorizing the specific path if its one of the special cases that warrants that support. SIMD code can be complex and hard to maintain, it also involves a bit of unsafe code (such as due to JIT limitations) and so its not something we really want to do "everywhere" or want customers to feel the need to go and do themselves. We rather want to try to ensure they get the benefits while keeping most paths readable and maintainable first and foremost. |
|
really appreciate the feedback Tanner. Trying to find out what skills mater and which do not. I am going to take another run at this and see what you think. |
In addition, I ran this against a real piece of code doing 'maths' which I have been meaning to vectorize. Both with and skillout the skill it did a fine job, but overall raw thoughput is higher with the skill. without skill
with skill
In addition, the run without the skill choose to add the nuget reference for Numerics, while the Skill knew it was part of the framework (less wasted time). Finally, interestingly, the non-skill run actually found 1 optimization which the skill one did not (a copy to move to Array.Copy). |
Add SKILL.md and eval.yaml for both SIMD skills under the new plugins/dotnet/skills/ and tests/dotnet/ directory structure. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
I cannot comment on the implementation - I'd be curious how this does when merged with the other Perf skill and seeing how they interact in the real world needle-in-a-haystack eval. (You can advise copilot to build such a scenario. Your eval will balloon, but you might get some more real-world results.) I'm also curious from a general applicability if we'd want this loaded by default in all sessions - that's a fair amount of tokens for a somewhat narrow opportunity. (It fits the standard, but I am curious if it's worth it by default.) |
| - **Bitwise:** operators `&`, `|`, `^`, `~` | ||
| - **Mask extraction:** `vec.ExtractMostSignificantBits()` → `uint` bitmask | ||
| - **Population count:** `BitOperations.PopCount(mask)` for counting matches | ||
| - **Shuffle:** `Vector128.Shuffle(vec, indices)` for nibble-lookup tables |
There was a problem hiding this comment.
Does there need to be an explicit callout for arithmetic operations?
Can we simplify this to just indicate that we prefer using the APIs on these types without listing them all?
We notably also have a number of other helpers so that things like ExtractMostSignificantBits is ideally needed/used less, such as All/Any/None, IndexOf, LastIndexOf, Count, etc
| Build two 16-entry lookup tables (high/low nibble). `Vector128.Shuffle` with nibble index → AND results → `ExtractMostSignificantBits` + `PopCount` for counting. | ||
|
|
||
| ### Memory access pattern | ||
| Head (scalar for pre-vector bytes) → Body (`Vector256.LoadUnsafe` / `Vector128.LoadUnsafe`) → Tail (overlapping last-vector for idempotent operations, or scalar remainder). Always use `ref MemoryMarshal.GetReference(span)` and `LoadUnsafe(ref T, nuint elementOffset)`. |
There was a problem hiding this comment.
Backtracking for idempotent operations is great, but we can also do the same thing for non-idempotent scenarios.
See for example here: https://source.dot.net/#System.Numerics.Tensors/System/Numerics/Tensors/netcore/Common/TensorPrimitives.IAggregationOperator.cs,00a250d202e0c1bb,references
Where the Vectrorized128 path is doing an aggregation operation and so can't include already processed elements twice, so we just get a mask based on what the remainder was so that we can appropriately adjust those to some identity value for the operation.
In other non-aggregation cases you can just merge it with the last stored vector so that the final store preserves those how they should've been, rather than potentially changing them.
| @@ -0,0 +1,122 @@ | |||
| --- | |||
| name: simd-vector-math | |||
| description: Optimizes scalar float/double vector math in .NET 8+ with cross-platform Vector128/Vector256 SIMD intrinsics. Transforms hot-path scalar code into vectorized implementations — never generates greenfield. | |||
There was a problem hiding this comment.
Not sure I understand the difference between pattern matching and vector-math here.
Seems like these are both just "optimize loop code" scenarios where if we don't have an existing built-in, we want to vectorize it. All the general vectorization steps should then be essentially the same.
|
Was this closed because the plan is to integrate it with another skill as well or is it just not moving forward? |
|
Apologies, this was closed as I moved from private to public. Still iterating on feedback
…________________________________
From: Tanner Gooding ***@***.***>
Sent: Thursday, March 5, 2026 7:33:39 AM
To: dotnet/skills ***@***.***>
Cc: Jeff Schwartz ***@***.***>; state_change ***@***.***>
Subject: Re: [dotnet/skills] simd modifying existing code skill (PR #26)
[https://avatars.githubusercontent.com/u/10487869?s=20&v=4]tannergooding left a comment (dotnet/skills#26)<#26 (comment)>
Was this closed because the plan is to integrate it with another skill as well or is it just not moving forward?
—
Reply to this email directly, view it on GitHub<#26 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACI6L36F4FHARCRYE4OUTPT4PGM5HAVCNFSM6AAAAACVG42JH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DAMBVHEYTQMJQGA>.
You are receiving this because you modified the open/close state.Message ID: ***@***.***>
|
…olication skills removed, diagnostics skills kept
while doing some broad .NET experience analysis, the model suggested it might not think of identifying common SIMD patterns and correctly modify the code without a skill. here are some common simd patterns and ways to optimize.