-
Notifications
You must be signed in to change notification settings - Fork 369
simd modifying existing code skill #26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,120 @@ | ||
| --- | ||
| name: simd-pattern-matching | ||
| description: Optimizes scalar byte/string pattern matching in .NET 8+ with cross-platform Vector128/Vector256 SIMD intrinsics. Transforms hot-path scalar code into vectorized implementations — never generates greenfield. | ||
| --- | ||
|
|
||
| # SIMD Pattern Matching Optimization | ||
|
|
||
| > **STOP — Not everything is a SIMD opportunity.** String processing on small collections (`ToLower`, `Trim`, `Sort`, dedup on < 20 items), operations covered by framework APIs (`Span<T>.IndexOf`, `SearchValues<T>`, `System.Text.Ascii`), and code without large byte/char buffer loops are NOT candidates for manual SIMD. Using `Vector256` for ASCII lowercasing when `System.Text.Ascii.ToLower()` exists is harmful — it adds complexity for zero benefit. If the code does not contain a scalar loop over a byte/char buffer ≥ 64 bytes, report "no optimization opportunity" and stop immediately. | ||
|
|
||
| > **Early exit:** If the code is simply reimplementing a framework API (`Span<T>.IndexOf`, `MemoryExtensions.IndexOfAny`, `SearchValues<T>`, etc.), replace with the API call and stop. Those are already SIMD-optimized internally. This skill is for cases that need manual vectorization. | ||
|
|
||
| Scan an existing .NET 8+ codebase for scalar pattern matching code on hot paths that would benefit from manual SIMD vectorization using cross-platform `Vector128`/`Vector256` intrinsics. Focus on byte-level operations over large buffers where no existing framework API covers the operation. If no SIMD-eligible candidates exist, report that and stop. | ||
|
jeffschwMSFT marked this conversation as resolved.
|
||
|
|
||
| ## Decision Gate (mandatory — do this FIRST, before writing any code) | ||
|
|
||
| 1. Does the code contain a scalar loop over a byte or char buffer? **If NO → stop, report "no optimization opportunity"** | ||
|
jeffschwMSFT marked this conversation as resolved.
|
||
| 2. Are the buffers ≥ 64 bytes on the hot path? **If NO → stop** | ||
|
jeffschwMSFT marked this conversation as resolved.
|
||
| 3. Is the operation already covered by a framework API (`IndexOf`, `Contains`, `SearchValues<T>`, `System.Text.Ascii`)? **If YES → use that API instead and stop** | ||
| 4. Is this string/object processing on small collections (< 20 items), not bulk buffer scanning? **If YES → stop, this skill does not apply** | ||
|
|
||
| State your assessment before implementing: `[SIMD CANDIDATE: <method name>, Category <A/B/C/D>]` or `[NO SIMD OPPORTUNITY: <reason>]`. Do NOT proceed to implementation without stating one of these. | ||
|
|
||
| ## When to Use | ||
|
|
||
| - Character-class membership counting/classification in tight loops over large buffers (e.g., counting alphanumeric bytes, multi-range byte classification) | ||
| - Custom byte-range validation on large buffers (e.g., verifying all bytes are printable ASCII, valid hex digits) | ||
| - Approximate matching (Levenshtein distance in loops, edit-distance thresholding, pattern ≤ 63 bytes) | ||
| - Bulk byte scanning with custom logic that no framework API covers (e.g., multi-range classification, nibble-based lookup) | ||
|
|
||
| ## When NOT to Use — Report "no opportunity" instead | ||
|
|
||
| - No existing scalar code to optimize (greenfield) | ||
| - Operation is already covered by a framework API (`IndexOf`, `Contains`, `SequenceEqual`, `SearchValues<T>`) — use the API instead | ||
| - Code already uses `Vector256`/`Vector128`/`TensorPrimitives`/`Vector<T>`/`SearchValues<T>` | ||
| - Buffers consistently < 64 bytes | ||
| - Regex with back-tracking/capture groups/lookahead | ||
| - String processing on small collections (< 20 items) where HashSet/sort dominates | ||
| - `ReadOnlySpan<char>` UTF-16 that can't convert to byte processing | ||
| - Code that doesn't process large byte/char buffers in loops | ||
|
|
||
| ## Pattern Categories | ||
|
|
||
| **A — Character class membership:** `if` chains testing byte/char ranges in tight loops (`c >= 'a' && c <= 'z'`), lookup table arrays indexed by byte value. SIMD range comparison or nibble-lookup approach. | ||
|
|
||
| **B — Byte-range validation:** Loops checking if every byte satisfies a condition (e.g., all printable ASCII, all valid Base64). SIMD subtract-and-compare for unsigned range checks across full vectors. | ||
|
|
||
| **C — Approximate matching:** Levenshtein distance in loops, edit-distance thresholding (only if pattern ≤ 63 bytes). | ||
|
|
||
| **D — Bulk byte counting/scanning:** Counting byte occurrences, scanning with custom multi-condition logic that no single framework API covers. | ||
|
|
||
| ## Transformation Rules | ||
|
|
||
| ### Assessment (do this quickly) | ||
| - **First:** Check if an existing framework API covers the operation — if so, use it and stop (early exit, no manual SIMD needed) | ||
| - Skip if buffer typically < 64 bytes | ||
| - Skip if already uses `Vector256`/`Vector128`/`TensorPrimitives`/`SearchValues<T>` | ||
|
|
||
| ### Required imports (for manual SIMD only) | ||
| ```csharp | ||
| using System.Runtime.CompilerServices; | ||
| using System.Runtime.InteropServices; | ||
| using System.Runtime.Intrinsics; | ||
| ``` | ||
|
|
||
| Do NOT use platform-specific imports (`System.Runtime.Intrinsics.X86`, `System.Runtime.Intrinsics.Arm`). Use the cross-platform `Vector128`/`Vector256` APIs instead. | ||
|
|
||
| ### Dispatch pattern | ||
|
|
||
| Use cross-platform hardware acceleration checks. Do NOT use platform-specific checks like `Avx2.IsSupported`, `Sse42.IsSupported`, or `AdvSimd.IsSupported`. | ||
|
|
||
| ```csharp | ||
| if (!Vector128.IsHardwareAccelerated || buffer.Length < Vector128<byte>.Count) | ||
| { | ||
| // scalar fallback (never delete this path) | ||
| } | ||
| else if (Vector256.IsHardwareAccelerated && buffer.Length >= Vector256<byte>.Count) | ||
| { | ||
| // Vector256 code path | ||
| } | ||
| else | ||
| { | ||
| // Vector128 code path | ||
| } | ||
|
jeffschwMSFT marked this conversation as resolved.
|
||
| ``` | ||
|
|
||
| ### SIMD operations reference | ||
|
|
||
| Use cross-platform `Vector128`/`Vector256` operations: | ||
| - **Create/broadcast:** `Vector128.Create(value)`, `Vector256.Create(value)` | ||
| - **Load/store:** `Vector128.LoadUnsafe(ref src, offset)`, `Vector256.StoreUnsafe(vec, ref dst, offset)` | ||
| - **Comparison:** `Vector128.Equals(a, b)`, `Vector128.GreaterThan(a, b)`, `Vector128.LessThan(a, b)` | ||
| - **Bitwise:** operators `&`, `|`, `^`, `~` | ||
| - **Mask extraction:** `vec.ExtractMostSignificantBits()` → `uint` bitmask | ||
| - **Population count:** `BitOperations.PopCount(mask)` for counting matches | ||
| - **Shuffle:** `Vector128.Shuffle(vec, indices)` for nibble-lookup tables | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does there need to be an explicit callout for arithmetic operations? Can we simplify this to just indicate that we prefer using the APIs on these types without listing them all? We notably also have a number of other helpers so that things like |
||
|
|
||
| ### Vectorized range comparison | ||
| Broadcast range bounds → subtract lower bound (wraps for out-of-range bytes in unsigned arithmetic) → compare against range width. For validation: check all lanes pass. For counting: `ExtractMostSignificantBits` → `PopCount`. | ||
|
jeffschwMSFT marked this conversation as resolved.
|
||
|
|
||
| ### Nibble-lookup for character classes | ||
| Build two 16-entry lookup tables (high/low nibble). `Vector128.Shuffle` with nibble index → AND results → `ExtractMostSignificantBits` + `PopCount` for counting. | ||
|
|
||
| ### Memory access pattern | ||
| Head (scalar for pre-vector bytes) → Body (`Vector256.LoadUnsafe` / `Vector128.LoadUnsafe`) → Tail (overlapping last-vector for idempotent operations, or scalar remainder). Always use `ref MemoryMarshal.GetReference(span)` and `LoadUnsafe(ref T, nuint elementOffset)`. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Backtracking for idempotent operations is great, but we can also do the same thing for non-idempotent scenarios. See for example here: https://source.dot.net/#System.Numerics.Tensors/System/Numerics/Tensors/netcore/Common/TensorPrimitives.IAggregationOperator.cs,00a250d202e0c1bb,references Where the Vectrorized128 path is doing an aggregation operation and so can't include already processed elements twice, so we just get a mask based on what the remainder was so that we can appropriately adjust those to some identity value for the operation. In other non-aggregation cases you can just merge it with the last stored vector so that the final store preserves those how they should've been, rather than potentially changing them. |
||
|
|
||
| ## Validation (required) | ||
|
|
||
| ```bash | ||
| dotnet build -c Release -warnaserror | ||
| dotnet test -c Release | ||
| ``` | ||
|
|
||
| If no tests exist for the method, add tests for: empty input, input < vector width, match at start/end, no match, input of exactly one vector width. **All existing tests must pass.** | ||
|
|
||
| ## Key Rules | ||
| - Preserve original method signature — drop-in replacement | ||
| - Never delete scalar code — it's the fallback | ||
| - Use cross-platform `Vector128`/`Vector256` APIs — never platform-specific intrinsics (`Avx2`, `Sse42`, `AdvSimd`) | ||
| - If no SIMD candidates found, report "no optimization opportunity" and explain why | ||
| - Skip categories with no matching code — don't generate from scratch | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,122 @@ | ||
| --- | ||
| name: simd-vector-math | ||
| description: Optimizes scalar float/double vector math in .NET 8+ with cross-platform Vector128/Vector256 SIMD intrinsics. Transforms hot-path scalar code into vectorized implementations — never generates greenfield. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not sure I understand the difference between pattern matching and vector-math here. Seems like these are both just "optimize loop code" scenarios where if we don't have an existing built-in, we want to vectorize it. All the general vectorization steps should then be essentially the same. |
||
| --- | ||
|
|
||
| # SIMD Vector Math Optimization | ||
|
|
||
| > **STOP — Not everything is a SIMD opportunity.** String processing (`ToLower`, `Trim`, `Sort`), small collections (< 20 items), and operations covered by framework APIs (`System.Text.Ascii`, `TensorPrimitives`, `SearchValues<T>`) are NOT candidates for manual SIMD. Using `Vector256` for ASCII lowercasing when `System.Text.Ascii.ToLower()` exists is harmful — it adds complexity for zero benefit. If the code does not contain a scalar loop over `float[]`/`double[]`/`byte[]` arrays ≥ 16 elements, report "no optimization opportunity" and stop immediately. | ||
|
|
||
| > **Early exit:** If the code is computing a standard operation covered by `TensorPrimitives` (dot product, cosine similarity, softmax, element-wise add/multiply, etc.), replace with the `TensorPrimitives` call and stop. Those are already SIMD-optimized internally. This skill is for cases that need manual vectorization. | ||
|
|
||
| Scan an existing .NET 8+ codebase for scalar floating-point vector math on hot paths that would benefit from manual SIMD vectorization using cross-platform `Vector128`/`Vector256` intrinsics. Focus on operations where no existing framework API covers the computation. If no SIMD-eligible candidates exist, report that and stop. | ||
|
|
||
| ## Decision Gate (mandatory — do this FIRST, before writing any code) | ||
|
|
||
| 1. Does the code contain a scalar loop over numeric arrays (`float[]`, `double[]`, `byte[]`, `int[]`)? **If NO → stop, report "no optimization opportunity"** | ||
| 2. Are the arrays ≥ 16 elements on the hot path? **If NO → stop** | ||
| 3. Is the operation already covered by `TensorPrimitives` or another framework API? **If YES → use that API instead and stop** | ||
| 4. Is this string/object processing, not numeric array math? **If YES → stop, this skill does not apply** | ||
|
|
||
| State your assessment before implementing: `[SIMD CANDIDATE: <method name>, Category <A/B/C/D>]` or `[NO SIMD OPPORTUNITY: <reason>]`. Do NOT proceed to implementation without stating one of these. | ||
|
|
||
| ## When to Use | ||
|
|
||
| - Multi-array fused computations with 3+ input arrays where no single `TensorPrimitives` call applies (e.g., weighted distance, fused multiply-accumulate with per-element parameters) | ||
| - Cross-type conversions combined with arithmetic (e.g., quantized int8→float dequantization with scale/offset) | ||
| - Custom distance metrics or similarity functions not covered by `TensorPrimitives` | ||
| - Domain-specific float processing with no suitable framework API (e.g., specialized quantization, custom activation functions) | ||
|
|
||
| ## When NOT to Use — Report "no opportunity" instead | ||
|
|
||
| - No existing scalar code to optimize (greenfield) | ||
| - Operation is covered by `TensorPrimitives` (dot product, cosine similarity, softmax, element-wise arithmetic, etc.) — use it instead | ||
| - Code already uses `TensorPrimitives`, `Vector256<float>`, `Vector128<float>`, or `Vector<T>` | ||
| - Vectors consistently < 16 floats (64 bytes) | ||
| - Bottleneck is memory bandwidth, not compute | ||
| - String/object processing with small collections — no float array math | ||
| - Code requires `decimal` or arbitrary-precision arithmetic | ||
|
|
||
| ## Pattern Categories | ||
|
|
||
| **A — Multi-input fused computations:** Operations on 3+ parallel arrays that require a single pass for efficiency (e.g., weighted Euclidean distance: `sum += w[i] * (a[i] - b[i])²`). No `TensorPrimitives` method handles these directly. | ||
|
|
||
| **B — Cross-type conversions:** Converting between integer and float types combined with arithmetic (e.g., quantized dequantization: `output[i] = (quantized[i] - zeroPoint) * scale`). Requires SIMD widening and type conversion intrinsics. | ||
|
|
||
| **C — Custom distance/similarity metrics:** Domain-specific distance functions not in `TensorPrimitives` (e.g., Mahalanobis distance, weighted Minkowski distance). | ||
|
|
||
| **D — Batch lookup/gather:** Embedding table lookups, scatter/gather, quantized dequantization. No framework API typically exists. | ||
|
|
||
| ## Transformation Rules | ||
|
|
||
| ### Assessment (do this quickly) | ||
| - **First:** Check if `TensorPrimitives` or another existing API covers the operation — if so, use it and stop (early exit, no manual SIMD needed) | ||
| - Skip if vectors typically < 16 floats | ||
| - Skip if already uses `TensorPrimitives`/`Vector256`/`Vector<T>` | ||
|
|
||
| ### Required imports | ||
| ```csharp | ||
| using System.Numerics; | ||
| using System.Runtime.CompilerServices; | ||
| using System.Runtime.InteropServices; | ||
| using System.Runtime.Intrinsics; | ||
| ``` | ||
|
|
||
| Do NOT use platform-specific imports (`System.Runtime.Intrinsics.X86`, `System.Runtime.Intrinsics.Arm`). Use the cross-platform `Vector128`/`Vector256` APIs instead. | ||
|
|
||
| ### Dispatch pattern | ||
|
|
||
| Use cross-platform hardware acceleration checks. Do NOT use platform-specific checks like `Avx2.IsSupported`, `Fma.IsSupported`, or `AdvSimd.IsSupported`. | ||
|
|
||
| ```csharp | ||
| if (!Vector128.IsHardwareAccelerated || data.Length < Vector128<float>.Count) | ||
| { | ||
| // scalar fallback (never delete this path) | ||
| } | ||
| else if (Vector256.IsHardwareAccelerated && data.Length >= Vector256<float>.Count) | ||
| { | ||
| // Vector256 code path | ||
| } | ||
| else | ||
| { | ||
| // Vector128 code path | ||
| } | ||
| ``` | ||
|
|
||
| ### SIMD operations reference | ||
|
|
||
| Use cross-platform `Vector128`/`Vector256` operations: | ||
| - **Create/broadcast:** `Vector128.Create(value)` to broadcast scalar to all lanes | ||
| - **Load/store:** `Vector128.LoadUnsafe(ref src, offset)`, `Vector128.StoreUnsafe(vec, ref dst, offset)` | ||
| - **Arithmetic:** operators `+`, `-`, `*`, `/` on vector types | ||
| - **FMA:** `Vector128.MultiplyAddEstimate(a, b, c)` for fused multiply-add | ||
| - **Min/Max:** `Vector128.Min(a, b)`, `Vector128.Max(a, b)` | ||
| - **Horizontal sum:** `Vector128.Sum(vec)` for final reduction | ||
| - **Type conversion:** `Vector128.WidenLower(v)` / `Vector128.WidenUpper(v)` for widening, `Vector128.ConvertToSingle(intVec)` for int→float | ||
|
|
||
| ### Multi-input fused computation pattern | ||
| Load from multiple arrays in the same loop, perform fused arithmetic, accumulate into vector accumulator(s). Use `Vector128.Sum` for final horizontal reduction. Always handle the loop remainder with scalar code for accumulations (do NOT use overlapping-vector to avoid double-counting). | ||
|
|
||
| ### Cross-type conversion pattern | ||
| For byte→float: Load `Vector128<byte>` (16 bytes) → `WidenLower`/`WidenUpper` to `Vector128<ushort>` → widen again to `Vector128<uint>` → `ConvertToSingle` to `Vector128<float>`. Process 4 float vectors per byte vector load. | ||
|
|
||
| ### Memory access pattern | ||
| Process data in vector-width chunks. For accumulations (reductions): use scalar remainder for remaining elements after the last full vector. For element-wise transforms: overlapping last-vector is safe if the operation is idempotent. Always use `ref MemoryMarshal.GetReference(span)` and `LoadUnsafe(ref T, nuint elementOffset)`. | ||
|
|
||
| Use `Assert.Equal(expected, actual, precision: 5)` for float comparisons in tests, since SIMD may reorder floating-point additions. | ||
|
|
||
| ## Validation (required) | ||
|
|
||
| ```bash | ||
| dotnet build -c Release -warnaserror | ||
| dotnet test -c Release | ||
| ``` | ||
|
|
||
| If no tests exist, add tests for: empty array, array < vector width, exactly one vector width, large array, special values (0, NaN). **All existing tests must pass.** | ||
|
|
||
| ## Key Rules | ||
| - Preserve original method signature — drop-in replacement | ||
| - Never delete scalar code — it's the fallback | ||
| - Use cross-platform `Vector128`/`Vector256` APIs — never platform-specific intrinsics (`Avx2`, `Fma`, `Sse`, `AdvSimd`) | ||
| - If no SIMD candidates found, report "no optimization opportunity" and explain why | ||
| - Skip categories with no matching code — don't generate from scratch | ||
Uh oh!
There was an error while loading. Please reload this page.