[efficiency-improver] perf: eliminate closure allocations and redundant dict lookups in FastFilter.Evaluate

[Jakub Jareš (nohwnd)]

Goal and Rationale

FastFilter.Evaluate is called once per test case whenever a test filter is active. The previous implementation had two independent inefficiencies in the hot loop:

1. Redundant dictionary lookups: iterating over FilterProperties.Keys required 1–3 additional FilterProperties[name] hash-and-compare lookups per property per call (for hasNoneFilter, singleValue, and the multi-value branch).

2. Closure allocation in multi-value path: the Select().Any(lambda) pattern allocated a Func<string,bool> delegate, a captured-variable closure object (capturing FilterProperties + name), and (in the regex path) an intermediate Select iterator — all per invocation.

This PR addresses both issues with no behaviour change. ValidForProperties is intentionally left unchanged.

Focus Area

Code-Level Efficiency — eliminating per-test-case allocations and redundant computation in the filter evaluation hot path.

Approach

Change 1 – cache the filter value set:

// Before: iterates .Keys, then looks up FilterProperties[name] up to 3 times
foreach (var name in FilterProperties.Keys)
{
    bool hasNoneFilter = FilterProperties[name].Contains(Condition.NoneFilterValue);
    ...
    matched = value != null && FilterProperties[name].Contains(value);
    ...
    matched = values?.Any(result => ... && FilterProperties[name].Contains(result)) == true;
}

// After: obtains both key and value once from the iterator
foreach (var kvp in FilterProperties)
{
    var filterValues = kvp.Value;  // cached — no further dict lookups
    bool hasNoneFilter = filterValues.Contains(Condition.NoneFilterValue);
    ...
    matched = value != null && filterValues.Contains(value);
}

Change 2 – replace Select().Any(lambda) with foreach:

// Before: allocates delegate + closure + (in regex path) Select iterator
var values = PropertyValueRegex == null ? multiValues : multiValues?.Select(value => ApplyRegex(value));
matched = values?.Any(result => result != null && FilterProperties[name].Contains(result)) == true;

// After: zero allocations, early exit on first match
foreach (var result in multiValues)
{
    if (result != null && filterValues.Contains(result))
    {
        matched = true;
        break;
    }
}

The foreach-based replacement was suggested by the maintainer in review comments on PR #16139.

Energy Efficiency Evidence

Proxy metric: heap allocation count → GC pressure → CPU energy; instruction count → CPU energy.

Per Evaluate() call, the savings are:

Path	Before	After	Saving
singleValue (common)	2 dict lookups	0 dict lookups	2 × hash+compare
multiValues, no regex	1 closure + delegate + 3 dict lookups	0 allocs, 0 extra lookups	~2 heap allocs + lookups
multiValues, with regex	2 closures + iterator + 3 lookups	0 allocs, 0 extra lookups	~3 heap allocs + lookups
hasNoneFilter	1 dict lookup	0 (reuses filterValues)	1 × hash+compare

For a 10,000-test run with a single-property filter: up to 20,000 fewer dictionary lookups and — for runs with multi-category filters — up to 20,000 fewer heap allocations, reducing GC pause time and CPU energy.

Green Software Foundation Context

Hardware Efficiency: Eliminating allocation-per-call noise makes better use of CPU cache lines for actual test evaluation work.

SCI (Software Carbon Intensity): dotnet test is invoked millions of times daily across .NET CI/CD pipelines. Reducing per-invocation work lowers the energy term in the SCI equation at ecosystem scale.

Trade-offs

The multi-value branch is slightly more verbose (explicit foreach instead of LINQ one-liner). This is offset by:

• Zero allocation cost vs. two heap allocations per call
• The foreach pattern is idiomatic C# and consistent with the rest of the codebase

Test Status

• Microsoft.TestPlatform.Common.UnitTests (Filtering namespace): ✅ 108/108 passed
• Microsoft.TestPlatform.Filter.Source.UnitTests: ✅ 45/45 passed
• Build: ✅ 0 errors (Debug, net11.0)

Reproducibility

.dotnet/dotnet run --project test/Microsoft.TestPlatform.Common.UnitTests/Microsoft.TestPlatform.Common.UnitTests.csproj -f net11.0 -- --filter "Filtering"
.dotnet/dotnet run --project test/Microsoft.TestPlatform.Filter.Source.UnitTests/Microsoft.TestPlatform.Filter.Source.UnitTests.csproj -f net11.0

Generated by Efficiency Improver · 2K AIC · ⌖ 27.3 AIC · ⊞ 45.6K · ◷

[Copilot]

Pull request overview

This PR improves the hot-path performance of FastFilter.Evaluate (used per test case when filters are enabled) by removing redundant dictionary lookups and avoiding LINQ/closure allocations in the multi-value evaluation path.

Changes:

• Iterate FilterProperties as key/value pairs to reuse the per-property value set (filterValues) and avoid repeated dictionary indexing.
• Replace the Select(...).Any(...) pattern with allocation-free foreach loops (with early-exit) for multi-value comparisons, including the regex-transformation path.

[Jakub Jareš (nohwnd)]

Code Review — FastFilter.Evaluate optimization

Dimensions activated: Algorithmic Correctness · Performance & Allocations · Null Safety & Boundary Validation · Backward Compatibility

---

Algorithmic Correctness ✅

All execution paths are semantically equivalent to the original:

• singleValue path: filterValues.Contains(value) is identical to FilterProperties[name].Contains(value) since filterValues = kvp.Value and kvp.Key == name.
• multiValues no-regex path: The inner break exits the inner foreach only; the outer loop's if (matched) { break; } guard fires immediately after, preserving early-exit behavior.
• multiValues regex path: ApplyRegex(result) is called for each element, and transformed != null mirrors the old result != null check on the post-Select enumerable.
• hasNoneFilter + empty-array path (else if (hasNoneFilter) at the bottom): unchanged and still reachable when multiValues is non-null but Length == 0.

Enumerating an ImmutableDictionary<K,V> as KeyValuePairs vs. via .Keys yields the same iteration order, so outer-loop behavior is preserved.

Performance ✅

The savings are genuine:

• Eliminates 1–3 FilterProperties[name] hash-and-compare lookups per outer iteration.
• Eliminates the Func delegate + captured-variable closure in the Any(lambda) call.
• In the regex path, eliminates the intermediate Select iterator object.

using System.Linq is still needed for ValidForProperties (.All, .Where, .Contains, .ToArray), so no spurious IDE0005 warning.

Null Safety ✅

• filterValues = kvp.Value is always non-null: AddProperty always creates the set before inserting.
• multiValues is { Length: > 0 } guarantees non-null, non-empty array before either inner loop.
• result != null (no-regex) and transformed != null (regex) null guards are preserved.

No issues found.

---

🧠 Reviewed by expert-reviewer workflow · PR #16160

🧠 Reviewed by Expert Code Reviewer 🧠