Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/arm64: 414 Improvements on 3/6/2023 9:06:14 PM #13808

Closed
performanceautofiler bot opened this issue Mar 8, 2023 · 18 comments
Closed

[Perf] Linux/arm64: 414 Improvements on 3/6/2023 9:06:14 PM #13808

performanceautofiler bot opened this issue Mar 8, 2023 · 18 comments
Labels
ampere arch-arm64 branch-refs/heads/main kind-micro_mono mono-aot os-linux perf-improvement runtime-mono untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Mar 8, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SubtractBenchmark - Duration of single invocation 20.37 ns 16.58 ns 0.81 0.01 False
GreaterThanAllBenchmark - Duration of single invocation 11.20 ns 9.49 ns 0.85 0.08 False
SubtractionOperatorBenchmark - Duration of single invocation 18.24 ns 16.59 ns 0.91 0.01 False
LessThanAnyBenchmark - Duration of single invocation 10.96 ns 9.06 ns 0.83 0.06 False
DotBenchmark - Duration of single invocation 10.65 ns 8.10 ns 0.76 0.14 False
BitwiseAndOperatorBenchmark - Duration of single invocation 14.59 ns 12.94 ns 0.89 0.02 False
BitwiseAndBenchmark - Duration of single invocation 15.42 ns 13.02 ns 0.84 0.02 False
MultiplyOperatorBenchmark - Duration of single invocation 18.67 ns 16.88 ns 0.90 0.01 False
EqualsAnyBenchmark - Duration of single invocation 11.01 ns 9.17 ns 0.83 0.08 False
XorBenchmark - Duration of single invocation 15.70 ns 13.16 ns 0.84 0.01 False
DivisionOperatorBenchmark - Duration of single invocation 31.76 ns 29.98 ns 0.94 0.00 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 12.42 ns 10.27 ns 0.83 0.04 False
LessThanOrEqualBenchmark - Duration of single invocation 20.41 ns 18.62 ns 0.91 0.01 False
BitwiseOrOperatorBenchmark - Duration of single invocation 14.30 ns 13.25 ns 0.93 0.03 False
MultiplyBenchmark - Duration of single invocation 20.49 ns 16.84 ns 0.82 0.01 False
LessThanBenchmark - Duration of single invocation 19.72 ns 17.94 ns 0.91 0.01 False
DivideBenchmark - Duration of single invocation 33.67 ns 29.71 ns 0.88 0.00 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 12.16 ns 10.39 ns 0.85 0.04 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 14.42 ns 12.84 ns 0.89 0.04 False
EqualsBenchmark - Duration of single invocation 11.06 ns 8.36 ns 0.76 0.09 False
ConditionalSelectBenchmark - Duration of single invocation 16.96 ns 15.46 ns 0.91 0.05 False
EqualsAllBenchmark - Duration of single invocation 8.00 ns 4.39 ns 0.55 0.31 False
AddBenchmark - Duration of single invocation 20.22 ns 16.86 ns 0.83 0.03 False
BitwiseOrBenchmark - Duration of single invocation 15.47 ns 13.18 ns 0.85 0.02 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.SubtractBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.576085573362093 < 19.1589181395493.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 106.7016724800388 (T) = (0 -16.625117069765476) / Math.Sqrt((0.17930635444954707 / (299)) + (0.0020434747226783603 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1867913638145138 = (20.44385208173493 - 16.625117069765476) / 20.44385208173493 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.491120280078013 < 10.620185911121592.
IsChangePoint: Marked as a change because one of 11/21/2022 12:46:21 PM, 11/24/2022 8:21:24 PM, 12/15/2022 2:06:03 PM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 15.782132979641517 (T) = (0 -9.52894032112919) / Math.Sqrt((0.43236013952045804 / (299)) + (0.03609758723105373 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1612735080702987 = (11.36120107426852 - 9.52894032112919) / 11.36120107426852 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.59454732367262 < 17.326894997935145.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 18.109405588895775 (T) = (0 -16.689802041253788) / Math.Sqrt((0.36819272504466805 / (299)) + (0.03256047891986513 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1065700499301892 = (18.680593861835145 - 16.689802041253788) / 18.680593861835145 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.06015696975417 < 10.442971069593026.
IsChangePoint: Marked as a change because one of 11/21/2022 12:46:21 PM, 11/24/2022 8:21:24 PM, 12/14/2022 12:01:27 PM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 34.71568499351236 (T) = (0 -8.958073542086353) / Math.Sqrt((0.4978120429807613 / (299)) + (0.008038837433715106 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.20346423550758358 = (11.246291681321788 - 8.958073542086353) / 11.246291681321788 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.095909813411543 < 10.01202517082308.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 10.547451413775972 (T) = (0 -8.458341062235183) / Math.Sqrt((0.6344640292577112 / (299)) + (0.15061348182718598 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.22194018765242168 = (10.871067915350235 - 8.458341062235183) / 10.871067915350235 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.942700693322255 < 13.578245029569802.
IsChangePoint: Marked as a change because one of 11/24/2022 9:42:44 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 39.07567133554189 (T) = (0 -12.993187906251785) / Math.Sqrt((0.12157522979092558 / (299)) + (0.0034138230329297673 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10569990310617623 = (14.52889019176122 - 12.993187906251785) / 14.52889019176122 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.017921299058747 < 14.721104573591383.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 63.34276158110158 (T) = (0 -12.978018105266921) / Math.Sqrt((0.5127425111424811 / (299)) + (0.0016138486293054413 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1880874815393909 = (15.984503022410985 - 12.978018105266921) / 15.984503022410985 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.876623055812807 < 17.74180101206577.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 29.752389259820568 (T) = (0 -16.944414353386346) / Math.Sqrt((0.48729668078209776 / (299)) + (0.011111258084770164 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11365870938526589 = (19.117257125225787 - 16.944414353386346) / 19.117257125225787 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.171935903571105 < 10.451528654730906.
IsChangePoint: Marked as a change because one of 11/21/2022 12:46:21 PM, 11/24/2022 8:21:24 PM, 12/15/2022 1:21:30 AM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 31.847938578651334 (T) = (0 -9.066113405029462) / Math.Sqrt((0.5314152560939454 / (299)) + (0.008404118534293546 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.19205020212900373 = (11.221134566676419 - 9.066113405029462) / 11.221134566676419 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.160202344005226 < 14.825845264852829.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 66.59861611028667 (T) = (0 -13.147310518137642) / Math.Sqrt((0.45137003702681594 / (299)) + (0.0009215304848264089 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.17757278414711564 = (15.98598668029662 - 13.147310518137642) / 15.98598668029662 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.976184150248766 < 30.198228934212786.
IsChangePoint: Marked as a change because one of 11/28/2022 8:15:19 AM, 1/5/2023 9:29:47 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 12.893359387152769 (T) = (0 -29.720193248084176) / Math.Sqrt((0.10964056338003618 / (299)) + (0.049857783202621554 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0535147212926228 = (31.400586904714768 - 29.720193248084176) / 31.400586904714768 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.269092725595351 < 11.714178681435124.
IsChangePoint: Marked as a change because one of 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 26.83337137325293 (T) = (0 -10.292397953525276) / Math.Sqrt((0.053830782166554764 / (299)) + (0.015503491427443807 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.16012638204485838 = (12.254698484974917 - 10.292397953525276) / 12.254698484974917 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.617289880704675 < 19.40737351167177.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 17.600204217101467 (T) = (0 -18.824154500561107) / Math.Sqrt((0.2808389927570428 / (299)) + (0.032113372228080674 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09164402395346372 = (20.72332323115219 - 18.824154500561107) / 20.72332323115219 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.24856999724897 < 13.581564292197948.
IsChangePoint: Marked as a change because one of 1/5/2023 11:12:07 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 14.089836003207843 (T) = (0 -13.154402339866543) / Math.Sqrt((0.11024244057944708 / (299)) + (0.017735095393788085 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.09333207952148032 = (14.508511928958438 - 13.154402339866543) / 14.508511928958438 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.83503272859422 < 19.469463290537682.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 2:25:12 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 67.93831346204063 (T) = (0 -16.934958175084734) / Math.Sqrt((0.06512765283080023 / (299)) + (0.008066087374992943 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.17782105941787416 = (20.597655010592106 - 16.934958175084734) / 20.597655010592106 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.935828273976135 < 18.70829914909539.
IsChangePoint: Marked as a change because one of 12/14/2022 8:02:51 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 25.348030804242235 (T) = (0 -17.869458452832898) / Math.Sqrt((0.4844540239243368 / (299)) + (0.019014647625520416 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11233074200214084 = (20.130761870854375 - 17.869458452832898) / 20.130761870854375 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.708362910920286 < 32.071111594812336.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 34.59271979529993 (T) = (0 -29.56882278209537) / Math.Sqrt((0.06164569002203434 / (299)) + (0.04213986393454087 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1225524127213929 = (33.69867694753451 - 29.56882278209537) / 33.69867694753451 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.389399067449782 < 11.472918663574971.
IsChangePoint: Marked as a change because one of 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 33.06514532439056 (T) = (0 -10.315907230014048) / Math.Sqrt((0.05717686529102786 / (299)) + (0.008265632321179484 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.14820031243867643 = (12.110719668785245 - 10.315907230014048) / 12.110719668785245 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.837390354934778 < 13.664107037357406.
IsChangePoint: Marked as a change because one of 12/12/2022 5:25:39 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 33.36109889056714 (T) = (0 -12.919296849352838) / Math.Sqrt((0.2717112779335731 / (299)) + (0.005316480988913009 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11793479540500214 = (14.646646055247992 - 12.919296849352838) / 14.646646055247992 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.357192906499202 < 10.48636474087684.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 4.194196211496869 (T) = (0 -7.347424794734592) / Math.Sqrt((1.4065108428452702 / (299)) + (1.2827963936589386 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.27289920098329207 = (10.105097951578179 - 7.347424794734592) / 10.105097951578179 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.462594479634793 < 15.952375356094048.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 12.907017891761415 (T) = (0 -15.38348764673954) / Math.Sqrt((0.13582002102096688 / (299)) + (0.03864990634703264 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08833715076863234 = (16.87409732634111 - 15.38348764673954) / 16.87409732634111 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.3921054872301895 < 7.598861837881436.
IsChangePoint: Marked as a change because one of 1/5/2023 9:29:47 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 38.418186231070905 (T) = (0 -4.505968080406466) / Math.Sqrt((0.30502159304494636 / (299)) + (0.019967086827145993 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.42758505532087243 = (7.871856111185789 - 4.505968080406466) / 7.871856111185789 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.855776255279718 < 19.14700418060722.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 55.62933533046058 (T) = (0 -16.78838994897373) / Math.Sqrt((0.19897654982826668 / (299)) + (0.011016008702713864 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1791365816607151 = (20.452111245179935 - 16.78838994897373) / 20.452111245179935 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.181823450855791 < 14.763094028340255.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 60.21446443059153 (T) = (0 -13.127751401363463) / Math.Sqrt((0.21700586088922103 / (299)) + (0.0036947247116032684 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1686958330511095 = (15.791754598735906 - 13.127751401363463) / 15.791754598735906 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 8, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GreaterThanOrEqualBenchmark - Duration of single invocation 16.17 ns 14.38 ns 0.89 0.05 False
EqualsAnyBenchmark - Duration of single invocation 8.07 ns 6.27 ns 0.78 0.14 False
SubtractBenchmark - Duration of single invocation 17.31 ns 13.76 ns 0.79 0.09 False
BitwiseOrOperatorBenchmark - Duration of single invocation 14.40 ns 13.06 ns 0.91 0.02 False
BitwiseAndOperatorBenchmark - Duration of single invocation 14.36 ns 13.12 ns 0.91 0.02 False
DivisionOperatorBenchmark - Duration of single invocation 17.44 ns 15.88 ns 0.91 0.03 False
MaxBenchmark - Duration of single invocation 15.71 ns 14.51 ns 0.92 0.02 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 8.69 ns 7.50 ns 0.86 0.14 False
BitwiseAndBenchmark - Duration of single invocation 15.61 ns 11.93 ns 0.76 0.01 False
EqualsStaticBenchmark - Duration of single invocation 15.33 ns 13.55 ns 0.88 0.02 False
AddBenchmark - Duration of single invocation 17.77 ns 13.66 ns 0.77 0.06 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 14.51 ns 13.21 ns 0.91 0.02 False
EqualsAllBenchmark - Duration of single invocation 8.64 ns 4.90 ns 0.57 0.34 False
BitwiseOrBenchmark - Duration of single invocation 15.51 ns 13.12 ns 0.85 0.03 False
AddOperatorBenchmark - Duration of single invocation 15.49 ns 13.75 ns 0.89 0.04 False
SubtractionOperatorBenchmark - Duration of single invocation 15.79 ns 13.96 ns 0.88 0.04 False
XorBenchmark - Duration of single invocation 15.54 ns 12.94 ns 0.83 0.02 False
ConditionalSelectBenchmark - Duration of single invocation 16.81 ns 15.35 ns 0.91 0.05 False
GreaterThanBenchmark - Duration of single invocation 15.76 ns 13.72 ns 0.87 0.02 False
AndNotBenchmark - Duration of single invocation 14.52 ns 13.02 ns 0.90 0.05 False
LessThanBenchmark - Duration of single invocation 15.08 ns 13.31 ns 0.88 0.02 False
DivideBenchmark - Duration of single invocation 19.80 ns 15.34 ns 0.78 0.01 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 9.11 ns 7.31 ns 0.80 0.18 False
MultiplyBenchmark - Duration of single invocation 18.51 ns 14.55 ns 0.79 0.04 False
GreaterThanAllBenchmark - Duration of single invocation 7.76 ns 6.24 ns 0.80 0.13 False
LessThanAnyBenchmark - Duration of single invocation 7.72 ns 6.41 ns 0.83 0.19 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GreaterThanOrEqualBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.384199688229463 < 15.351125991398392.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 5.757931264328238 (T) = (0 -13.997238846292015) / Math.Sqrt((0.24097173255862614 / (299)) + (0.5269515833384997 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.14733945037969354 = (16.415956915709362 - 13.997238846292015) / 16.415956915709362 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.270371501525982 < 7.5304846445342655.
IsChangePoint: Marked as a change because one of 11/11/2022 5:53:56 PM, 11/21/2022 12:46:21 PM, 11/24/2022 8:21:24 PM, 12/15/2022 2:06:03 PM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 8.777777858141837 (T) = (0 -6.456897001471936) / Math.Sqrt((0.279578238598325 / (299)) + (0.08982855755962488 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.19282103815057455 = (7.999337577725957 - 6.456897001471936) / 7.999337577725957 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.759690883711833 < 16.64412141461513.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 47.453465843120476 (T) = (0 -13.827905676969513) / Math.Sqrt((0.1632094585162487 / (299)) + (0.018626179539003307 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.21999310386395082 = (17.727927465089543 - 13.827905676969513) / 17.727927465089543 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.062133541915292 < 13.685348188690059.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 4:05:11 PM, 3/6/2023 6:03:21 AM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 40.322438202776965 (T) = (0 -13.118657729370778) / Math.Sqrt((0.09370729374336689 / (299)) + (0.003563449267082559 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.09638346191374374 = (14.5179477980277 - 13.118657729370778) / 14.5179477980277 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.115505723630141 < 13.638828616942423.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 19.47232325887347 (T) = (0 -13.035290077741061) / Math.Sqrt((0.09756105605223124 / (299)) + (0.015887278782810384 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10072493649279901 = (14.495331413842413 - 13.035290077741061) / 14.495331413842413 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.88273820455317 < 16.55810877602034.
IsChangePoint: Marked as a change because one of 12/9/2022 1:11:26 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 10.044398651247775 (T) = (0 -15.726249336721247) / Math.Sqrt((0.15163304665528515 / (299)) + (0.12224119276061572 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11483097854160977 = (17.766380155070195 - 15.726249336721247) / 17.766380155070195 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.5134320172385 < 14.933417431068237.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 9.851480461359202 (T) = (0 -14.372993966882012) / Math.Sqrt((0.2524317755239559 / (299)) + (0.08226717187505336 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10332948637856669 = (16.0292925311361 - 14.372993966882012) / 16.0292925311361 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.5007273176259845 < 8.157284644013073.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 2/6/2023 2:52:08 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 5.241846373853523 (T) = (0 -7.389554155475463) / Math.Sqrt((0.07531385691909818 / (299)) + (0.10707980195559655 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11854575213084517 = (8.383366661785475 - 7.389554155475463) / 8.383366661785475 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.933695045163516 < 14.776872341152732.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 6.494486787402397 (T) = (0 -12.453351553154974) / Math.Sqrt((0.2079547334984926 / (299)) + (0.5400857725957527 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.2134359614351787 = (15.832597147306124 - 12.453351553154974) / 15.832597147306124 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.548129453642842 < 14.40674683272816.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 22.483729986551474 (T) = (0 -13.461914056537985) / Math.Sqrt((0.23081629823299094 / (299)) + (0.02000398307048754 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.12592080501574424 = (15.40125212198932 - 13.461914056537985) / 15.40125212198932 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.655431520998725 < 16.755956947785354.
IsChangePoint: Marked as a change because one of 11/24/2022 1:04:57 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 31.838383926040986 (T) = (0 -13.876427421535773) / Math.Sqrt((0.10632513723228905 / (299)) + (0.03992003770236746 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.21147169724847692 = (17.59788123408481 - 13.876427421535773) / 17.59788123408481 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.208858107027606 < 13.689275477713583.
IsChangePoint: Marked as a change because one of 12/12/2022 5:25:39 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 14.183422005171426 (T) = (0 -13.001004967134074) / Math.Sqrt((0.15514038835597185 / (299)) + (0.03402197212424768 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10618984119739477 = (14.545599911899524 - 13.001004967134074) / 14.545599911899524 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.895448036377271 < 8.191656459851494.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 13.752120485628517 (T) = (0 -4.679781144003298) / Math.Sqrt((0.3433879201466459 / (299)) + (0.15775411494111308 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4051815966422773 = (7.867579613519266 - 4.679781144003298) / 7.867579613519266 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.120621818643352 < 14.730437747407825.
IsChangePoint: Marked as a change because one of 12/5/2022 11:02:29 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 36.64101272756858 (T) = (0 -13.043174662497153) / Math.Sqrt((0.21374500905418814 / (299)) + (0.014709196110069659 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.17393487537236257 = (15.789523457217227 - 13.043174662497153) / 15.789523457217227 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.75268853153915 < 14.917314823054536.
IsChangePoint: Marked as a change because one of 11/25/2022 4:06:25 PM, 12/5/2022 3:49:45 AM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 97.27785188955019 (T) = (0 -13.739300921234237) / Math.Sqrt((0.1173105379955183 / (299)) + (0.00020642463909446602 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.13197804581712078 = (15.828287354975785 - 13.739300921234237) / 15.828287354975785 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.962666206239959 < 14.957753997880223.
IsChangePoint: Marked as a change because one of 12/14/2022 7:00:38 AM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 22.781609432771493 (T) = (0 -13.852850157607962) / Math.Sqrt((0.10259860248433245 / (299)) + (0.021156742028869135 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.12390201879031155 = (15.811987305895151 - 13.852850157607962) / 15.811987305895151 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.93654800775347 < 14.775395262732937.
IsChangePoint: Marked as a change because one of 11/25/2022 6:33:31 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 21.41933471949165 (T) = (0 -12.970674514503363) / Math.Sqrt((0.18878936364306229 / (299)) + (0.05020223636457155 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.17872166292968117 = (15.793274860716068 - 12.970674514503363) / 15.793274860716068 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.346885796856293 < 15.881362442445793.
IsChangePoint: Marked as a change because one of 12/14/2022 8:02:51 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 49.761185413132914 (T) = (0 -15.36285187407593) / Math.Sqrt((0.2912879398502072 / (299)) + (0.00019495373817515668 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09454401871106327 = (16.966978176240627 - 15.36285187407593) / 16.966978176240627 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.717327676462295 < 14.96981913097308.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 27.686646032731712 (T) = (0 -13.797121170800855) / Math.Sqrt((0.6142300335288363 / (299)) + (0.017318478175081656 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.15076790715751384 = (16.246584752373362 - 13.797121170800855) / 16.246584752373362 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.023666428649044 < 13.770369310253846.
IsChangePoint: Marked as a change because one of 11/17/2022 5:10:53 PM, 11/21/2022 12:46:21 PM, 12/5/2022 5:12:51 PM, 12/15/2022 5:36:17 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 33.51742983422947 (T) = (0 -13.082933608388478) / Math.Sqrt((0.13553030200081803 / (299)) + (0.004891917534835644 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10470599278100849 = (14.613002547651762 - 13.082933608388478) / 14.613002547651762 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.307827924830365 < 14.339212105001392.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 21.590801848427535 (T) = (0 -13.364832400137649) / Math.Sqrt((0.46319497390060993 / (299)) + (0.025846304198394867 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.14006112068627316 = (15.541607341679256 - 13.364832400137649) / 15.541607341679256 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.34358468185546 < 18.60727064222209.
IsChangePoint: Marked as a change because one of 12/14/2022 8:02:51 PM, 1/5/2023 9:23:01 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 13.238965697569842 (T) = (0 -15.625905608840847) / Math.Sqrt((0.12383408411776987 / (299)) + (0.2576183758846229 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1992794077783057 = (19.514804240871236 - 15.625905608840847) / 19.514804240871236 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.311750104386026 < 8.245579694510408.
IsChangePoint: Marked as a change because one of 1/5/2023 9:23:01 PM, 2/12/2023 3:00:19 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 7.328370775136652 (T) = (0 -7.43485873554823) / Math.Sqrt((0.10786828247646982 / (299)) + (0.05868659387119135 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.12213481673878701 = (8.469248897567853 - 7.43485873554823) / 8.469248897567853 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.551384812416886 < 17.61254530511271.
IsChangePoint: Marked as a change because one of 1/5/2023 9:23:01 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 27.105844478270125 (T) = (0 -14.77265500472273) / Math.Sqrt((0.07782516888035473 / (299)) + (0.06396178574012895 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.21231918184407159 = (18.754620734966725 - 14.77265500472273) / 18.754620734966725 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.242614945729717 < 7.701481746254397.
IsChangePoint: Marked as a change because one of 12/14/2022 12:01:27 PM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 31.81235872378857 (T) = (0 -6.147237490800916) / Math.Sqrt((0.26547182423778487 / (299)) + (0.009061956794494792 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.24444791614792477 = (8.136087004697409 - 6.147237490800916) / 8.136087004697409 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.409913152465916 < 7.403000891074231.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 29.527454834213835 (T) = (0 -6.355783901750212) / Math.Sqrt((0.34196648184794626 / (299)) + (0.005676419144218499 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.20380536065218116 = (7.982701198486314 - 6.355783901750212) / 7.982701198486314 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 8, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
XorBenchmark - Duration of single invocation 14.82 ns 8.63 ns 0.58 0.30 False
MultiplyBenchmark - Duration of single invocation 43.99 ns 40.75 ns 0.93 0.00 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 3.58 ns 1.08 ns 0.30 0.40 False
OneBenchmark - Duration of single invocation 22.22 ns 0.83 ns 0.04 0.09 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 13.60 ns 8.44 ns 0.62 0.02 False
BitwiseOrOperatorBenchmark - Duration of single invocation 13.88 ns 7.99 ns 0.58 0.04 False
BitwiseOrBenchmark - Duration of single invocation 16.30 ns 7.99 ns 0.49 0.28 False
BitwiseAndOperatorBenchmark - Duration of single invocation 13.78 ns 7.97 ns 0.58 0.03 False
OnesComplementBenchmark - Duration of single invocation 8.40 ns 6.34 ns 0.76 0.05 False
ConditionalSelectBenchmark - Duration of single invocation 61.99 ns 32.45 ns 0.52 0.36 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 3.79 ns 1.07 ns 0.28 0.44 False
OnesComplementOperatorBenchmark - Duration of single invocation 8.41 ns 6.32 ns 0.75 0.03 False
AndNotBenchmark - Duration of single invocation 24.08 ns 16.13 ns 0.67 0.01 False
EqualsAllBenchmark - Duration of single invocation 5.56 ns 1.56 ns 0.28 0.16 False
BitwiseAndBenchmark - Duration of single invocation 16.16 ns 8.09 ns 0.50 0.28 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<SByte>.XorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.633590072754371 < 14.115990111119231.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 49.860316339977764 (T) = (0 -8.404120147972423) / Math.Sqrt((0.412444096289158 / (299)) + (0.05831603615735516 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.46121230794049417 = (15.598203655781058 - 8.404120147972423) / 15.598203655781058 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 40.74625682452059 < 41.82844504342909.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 15.547380930560807 (T) = (0 -40.987777854080896) / Math.Sqrt((0.5511093200436843 / (299)) + (0.07228900607456235 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.05757457573190624 = (43.491799773878995 - 40.987777854080896) / 43.491799773878995 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0785503064371618 < 3.339855492954083.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 64.91789953228432 (T) = (0 -1.0701878604299597) / Math.Sqrt((0.36715068519875516 / (299)) + (0.0013777738897390642 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.7136019670631099 = (3.736715121454006 - 1.0701878604299597) / 3.736715121454006 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.8334072778158286 < 21.062457212845082.
IsChangePoint: Marked as a change because one of 12/5/2022 9:44:53 PM, 12/13/2022 5:38:37 AM, 1/11/2023 1:25:06 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 140.31868495872504 (T) = (0 -0.8619145010584269) / Math.Sqrt((7.528285328811811 / (299)) + (0.008617298228127302 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.9646219100983968 = (24.36294620358711 - 0.8619145010584269) / 24.36294620358711 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.44148065886328 < 12.88742055740692.
IsChangePoint: Marked as a change because one of 1/5/2023 9:23:01 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 58.638366999787 (T) = (0 -8.2547626096112) / Math.Sqrt((0.146600017711946 / (299)) + (0.026258048663126143 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4058022115126736 = (13.89228093666536 - 8.2547626096112) / 13.89228093666536 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.985302613527896 < 13.020713030838207.
IsChangePoint: Marked as a change because one of 12/12/2022 3:40:04 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 33.775179357452956 (T) = (0 -8.143741252982453) / Math.Sqrt((0.08495882681346016 / (299)) + (0.08209368281024072 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4081516349276198 = (13.759844131674695 - 8.143741252982453) / 13.759844131674695 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.991391496323965 < 14.083646640295687.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 66.52058209179825 (T) = (0 -8.099842395528055) / Math.Sqrt((0.4168533696461243 / (299)) + (0.034126013536121755 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.481338365837679 = (15.61681424269974 - 8.099842395528055) / 15.61681424269974 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.972593855501605 < 12.971447220266308.
IsChangePoint: Marked as a change because one of 12/12/2022 5:25:39 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 43.52460086739372 (T) = (0 -8.190577142293522) / Math.Sqrt((0.15971272250483465 / (299)) + (0.05031576765055422 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.41144215367020753 = (13.916350267640498 - 8.190577142293522) / 13.916350267640498 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.343068386834007 < 7.984583410068535.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 60.99781681509847 (T) = (0 -6.329521351779152) / Math.Sqrt((0.475450170620735 / (299)) + (0.0004972266061776292 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.28765918261294476 = (8.885523891494152 - 6.329521351779152) / 8.885523891494152 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.4498003498365 < 56.726237390685334.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 105.90417858069941 (T) = (0 -32.47180412729963) / Math.Sqrt((14.287691577649749 / (299)) + (0.010951478763509643 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4251782231419 = (56.49021215721197 - 32.47180412729963) / 56.49021215721197 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.067568506286509 < 3.3398990709949525.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 31.83445844914557 (T) = (0 -1.1630779568731795) / Math.Sqrt((0.3210560267170296 / (299)) + (0.017131942577341023 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.6927304708520046 = (3.7852043451825215 - 1.1630779568731795) / 3.7852043451825215 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.323759654111822 < 8.006122221716046.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 11.981292437997341 (T) = (0 -6.685803379692384) / Math.Sqrt((0.4704594763702457 / (299)) + (0.09858277333279794 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.24955444626893192 = (8.909111855552853 - 6.685803379692384) / 8.909111855552853 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.129203598630376 < 22.902637611852136.
IsChangePoint: Marked as a change because one of 1/5/2023 11:12:07 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 39.87679536884198 (T) = (0 -15.934076515319255) / Math.Sqrt((0.15089550181442382 / (299)) + (0.12031181911763701 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.33524587254630994 = (23.969879775480294 - 15.934076515319255) / 23.969879775480294 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.560213168837031 < 5.362606930720633.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 18.818994126736094 (T) = (0 -1.3372876149810964) / Math.Sqrt((0.19398067435917604 / (299)) + (0.12969266755492434 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.746696611640664 = (5.279390945548747 - 1.3372876149810964) / 5.279390945548747 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.091199623161248 < 14.71114287355818.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 57.669968261417424 (T) = (0 -8.302146252094861) / Math.Sqrt((0.39122508788527804 / (299)) + (0.043933999102195975 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.46733952034957266 = (15.586187767381139 - 8.302146252094861) / 15.586187767381139 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 8, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
XorBenchmark - Duration of single invocation 15.59 ns 13.17 ns 0.85 0.03 False
EqualsAllBenchmark - Duration of single invocation 8.46 ns 4.44 ns 0.52 0.26 False
SubtractBenchmark - Duration of single invocation 56.65 ns 49.77 ns 0.88 0.00 False
BitwiseAndOperatorBenchmark - Duration of single invocation 14.16 ns 12.96 ns 0.91 0.02 False
LessThanAnyBenchmark - Duration of single invocation 38.75 ns 34.39 ns 0.89 0.01 False
GreaterThanBenchmark - Duration of single invocation 61.89 ns 57.77 ns 0.93 0.00 False
AddBenchmark - Duration of single invocation 55.09 ns 49.93 ns 0.91 0.01 False
AndNotBenchmark - Duration of single invocation 14.56 ns 13.36 ns 0.92 0.07 False
EqualsAnyBenchmark - Duration of single invocation 39.40 ns 34.48 ns 0.87 0.01 False
ConditionalSelectBenchmark - Duration of single invocation 16.65 ns 15.60 ns 0.94 0.02 False
EqualsStaticBenchmark - Duration of single invocation 60.25 ns 52.67 ns 0.87 0.01 False
BitwiseAndBenchmark - Duration of single invocation 15.46 ns 12.91 ns 0.83 0.02 False
MultiplyBenchmark - Duration of single invocation 56.37 ns 50.90 ns 0.90 0.00 False
DotBenchmark - Duration of single invocation 44.61 ns 39.51 ns 0.89 0.01 False
SubtractionOperatorBenchmark - Duration of single invocation 55.56 ns 49.95 ns 0.90 0.01 False
MaxBenchmark - Duration of single invocation 64.63 ns 60.81 ns 0.94 0.01 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 14.29 ns 13.03 ns 0.91 0.01 False
BitwiseOrOperatorBenchmark - Duration of single invocation 14.38 ns 13.22 ns 0.92 0.02 False
BitwiseOrBenchmark - Duration of single invocation 15.67 ns 13.00 ns 0.83 0.03 False
LessThanBenchmark - Duration of single invocation 58.15 ns 54.12 ns 0.93 0.01 False
EqualityOperatorBenchmark - Duration of single invocation 6.62 ns 5.22 ns 0.79 0.24 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 43.24 ns 38.34 ns 0.89 0.01 False
GreaterThanAllBenchmark - Duration of single invocation 39.61 ns 35.00 ns 0.88 0.02 False
DivideBenchmark - Duration of single invocation 105.33 ns 97.69 ns 0.93 0.00 False
MultiplyOperatorBenchmark - Duration of single invocation 57.06 ns 51.89 ns 0.91 0.01 False
AddOperatorBenchmark - Duration of single invocation 55.86 ns 49.22 ns 0.88 0.01 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.XorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.173317061108923 < 14.76330285025517.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 37.351429928643974 (T) = (0 -13.173832052063894) / Math.Sqrt((0.41273240441868003 / (299)) + (0.01195929725209544 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.17198525813307292 = (15.910141916508417 - 13.173832052063894) / 15.910141916508417 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.436656337550505 < 8.0065696543159.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 35.60129639719627 (T) = (0 -4.613921844758646) / Math.Sqrt((0.2566968773457997 / (299)) + (0.027584276457032778 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.436193663169331 = (8.18352250294833 - 4.613921844758646) / 8.18352250294833 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.770361732333335 < 52.57876134824698.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 23.770373026742366 (T) = (0 -49.34052930370974) / Math.Sqrt((0.2594131770941796 / (299)) + (0.1795431060109249 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10611197576793709 = (55.1976623090997 - 49.34052930370974) / 55.1976623090997 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.957786956764702 < 13.51934256131762.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 11.439112348176193 (T) = (0 -13.04111193507334) / Math.Sqrt((0.10034181797629041 / (299)) + (0.047076719167177455 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09994977608844272 = (14.489315805508665 - 13.04111193507334) / 14.489315805508665 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.38515767630318 < 37.48962078476888.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 31.931791375532274 (T) = (0 -34.298835963073465) / Math.Sqrt((4.191136862094142 / (299)) + (0.005725685713721026 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10513585434103169 = (38.32853973361082 - 34.298835963073465) / 38.32853973361082 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 57.767088339547506 < 59.440207280277235.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 20.14812977648825 (T) = (0 -57.69249912168177) / Math.Sqrt((2.394329264581769 / (299)) + (0.05993126475365204 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.05519734694178756 = (61.06301557784379 - 57.69249912168177) / 61.06301557784379 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.932808223273554 < 52.43090200462407.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 26.661483878568124 (T) = (0 -49.71261249805121) / Math.Sqrt((0.2406131817405395 / (299)) + (0.12480153237766888 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09945629427455809 = (55.20288708031635 - 49.71261249805121) / 55.20288708031635 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.355731897823041 < 13.806597238740416.
IsChangePoint: Marked as a change because one of 11/17/2022 5:10:53 PM, 11/21/2022 12:46:21 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 17.589453193832576 (T) = (0 -13.232843494316453) / Math.Sqrt((0.07742408776758275 / (299)) + (0.013697643543567877 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08452522055559213 = (14.45462375527956 - 13.232843494316453) / 14.45462375527956 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.47554403147015 < 37.50934202038816.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 15.578591535365538 (T) = (0 -34.556432575906506) / Math.Sqrt((4.1260060086802754 / (299)) + (0.12858026038366313 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09690956866051283 = (38.26464258363529 - 34.556432575906506) / 38.26464258363529 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.603101204075946 < 15.90583316419128.
IsChangePoint: Marked as a change because one of 11/11/2022 10:59:41 PM, 12/10/2022 9:07:41 PM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 18.169224988137085 (T) = (0 -15.466384460384681) / Math.Sqrt((0.1008598114842201 / (299)) + (0.016045835977439818 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0813742831009765 = (16.83643749120597 - 15.466384460384681) / 16.83643749120597 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 52.67006606410033 < 56.56240436077192.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 20.303269072888682 (T) = (0 -52.911590139328176) / Math.Sqrt((2.226821724231888 / (299)) + (0.17820349235998756 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0902569169038039 = (58.16102493382004 - 52.911590139328176) / 58.16102493382004 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.905650964361604 < 14.710145480394113.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 27.833503011981062 (T) = (0 -13.0843278183077) / Math.Sqrt((0.48859009547217824 / (299)) + (0.02798184302688421 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.18214766286012285 = (15.998398762379388 - 13.0843278183077) / 15.998398762379388 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 50.895844400279564 < 53.76589414418647.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 16.157146067762078 (T) = (0 -51.207031544871846) / Math.Sqrt((0.12116411296029443 / (299)) + (0.33489542615271567 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09552408176236518 = (56.61514089247218 - 51.207031544871846) / 56.61514089247218 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 39.51202369007215 < 42.380382651844855.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 17.819707171968815 (T) = (0 -39.406854730026716) / Math.Sqrt((2.810935783171276 / (299)) + (0.17531778231976178 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1053698731678643 = (44.04820891686877 - 39.406854730026716) / 44.04820891686877 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.952072952122386 < 52.2446331328948.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 17.30617514613589 (T) = (0 -49.50750873637273) / Math.Sqrt((3.3344348355681235 / (299)) + (0.1657340436246672 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0826317183354321 = (53.96688519308863 - 49.50750873637273) / 53.96688519308863 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 60.813910863639876 < 61.8184770661607.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 11.255467622747332 (T) = (0 -60.19331097266508) / Math.Sqrt((2.828025709777721 / (299)) + (0.29276680098467023 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.057651991153067385 = (63.87588280291299 - 60.19331097266508) / 63.87588280291299 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.028427528960739 < 13.593856602593679.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 32.93725327809305 (T) = (0 -13.019086260421703) / Math.Sqrt((0.09575441587154387 / (299)) + (0.0052917829379340925 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10353933333551125 = (14.522763512717791 - 13.019086260421703) / 14.522763512717791 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.220319538933223 < 13.596727085855132.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 14.298402485453371 (T) = (0 -13.178084104653935) / Math.Sqrt((0.0895214367907364 / (299)) + (0.024101872740654257 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09012155966289183 = (14.483345818999164 - 13.178084104653935) / 14.483345818999164 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.99747287648328 < 14.817912593206096.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 45.05895604589647 (T) = (0 -13.098608733236754) / Math.Sqrt((0.22752503880875044 / (299)) + (0.008541447070866773 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.17124618974343425 = (15.805186740778524 - 13.098608733236754) / 15.805186740778524 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 54.11672512114 < 56.25436284558706.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 11.634044147288478 (T) = (0 -53.41768969340834) / Math.Sqrt((2.18499183188652 / (299)) + (0.48005098923195383 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08180165487069653 = (58.17663468548933 - 53.41768969340834) / 58.17663468548933 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.2174601960703155 < 5.960062725036106.
IsChangePoint: Marked as a change because one of 2/4/2023 8:08:12 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 7.197190143998295 (T) = (0 -5.354512650400846) / Math.Sqrt((0.14349654452447147 / (299)) + (0.03908521618983452 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.13511462567900315 = (6.191008438088758 - 5.354512650400846) / 6.191008438088758 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.34462457981702 < 40.93089438891144.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 24.12634969460578 (T) = (0 -38.46992212795537) / Math.Sqrt((3.795592389511109 / (299)) + (0.026497820789717026 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08426208692237365 = (42.00975145679516 - 38.46992212795537) / 42.00975145679516 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.9978846306914 < 37.663497142831226.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 13.828666081769644 (T) = (0 -34.735198148031245) / Math.Sqrt((3.7125300378265256 / (299)) + (0.1655944326445138 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09381019280937751 = (38.33104044252893 - 34.735198148031245) / 38.33104044252893 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 97.69492826110886 < 99.98490864739813.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 12.26135305687878 (T) = (0 -98.5325932600789) / Math.Sqrt((1.1436487853263093 / (299)) + (0.6951686871551417 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.05695482425003617 = (104.48342857140445 - 98.5325932600789) / 104.48342857140445 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.892560452169946 < 54.177640856304315.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 11.624668228124284 (T) = (0 -51.22776160234579) / Math.Sqrt((2.7554022119881236 / (299)) + (0.33363516962565026 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0729990966706597 = (55.26182490045087 - 51.22776160234579) / 55.26182490045087 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.22324295363071 < 52.02213587851941.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 54.7119802209005 (T) = (0 -49.21599674713922) / Math.Sqrt((2.3727057834041463 / (299)) + (0.00013021366496154395 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09032978286323674 = (54.103119811978964 - 49.21599674713922) / 54.103119811978964 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 8, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AddBenchmark - Duration of single invocation 13.49 ns 11.72 ns 0.87 0.01 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 2.99 ns 1.17 ns 0.39 0.36 False
MultiplyBenchmark - Duration of single invocation 13.83 ns 12.19 ns 0.88 0.02 False
AndNotBenchmark - Duration of single invocation 22.69 ns 15.50 ns 0.68 0.03 False
BitwiseAndBenchmark - Duration of single invocation 14.81 ns 8.43 ns 0.57 0.27 False
BitwiseAndOperatorBenchmark - Duration of single invocation 13.79 ns 8.41 ns 0.61 0.02 False
BitwiseOrOperatorBenchmark - Duration of single invocation 13.56 ns 7.97 ns 0.59 0.04 False
BitwiseOrBenchmark - Duration of single invocation 16.16 ns 8.00 ns 0.49 0.27 False
SubtractBenchmark - Duration of single invocation 13.68 ns 11.96 ns 0.87 0.02 False
OneBenchmark - Duration of single invocation 10.53 ns 0.88 ns 0.08 0.22 False
DivideBenchmark - Duration of single invocation 25.63 ns 23.37 ns 0.91 0.00 False
EqualsAllBenchmark - Duration of single invocation 5.50 ns 0.92 ns 0.17 0.16 False
ConditionalSelectBenchmark - Duration of single invocation 62.34 ns 32.45 ns 0.52 0.23 False
OnesComplementOperatorBenchmark - Duration of single invocation 8.49 ns 6.34 ns 0.75 0.04 False
EqualsBenchmark - Duration of single invocation 8.88 ns 5.45 ns 0.61 0.08 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 13.83 ns 7.97 ns 0.58 0.02 False
XorBenchmark - Duration of single invocation 14.78 ns 8.33 ns 0.56 0.29 False
OnesComplementBenchmark - Duration of single invocation 8.44 ns 6.30 ns 0.75 0.05 False
LessThanAllBenchmark - Duration of single invocation 8.19 ns 6.76 ns 0.83 0.05 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 3.30 ns 1.03 ns 0.31 0.41 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int32>.AddBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.719750541207077 < 12.820926898914914.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 46.88468008460323 (T) = (0 -11.79155167836988) / Math.Sqrt((0.11215303912334383 / (299)) + (0.003869035633401332 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.13958683726337323 = (13.70452265149654 - 11.79155167836988) / 13.70452265149654 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1673227521228564 < 2.855783405882802.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 6:03:21 AM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 18.28419349723953 (T) = (0 -0.8520837878901307) / Math.Sqrt((0.4538757766146451 / (299)) + (0.06847821789222543 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.745512173440503 = (3.3482300486028214 - 0.8520837878901307) / 3.3482300486028214 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.189807395874954 < 13.036307400261597.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 17.445101535697702 (T) = (0 -12.359063549596963) / Math.Sqrt((0.11050088796831121 / (299)) + (0.022280805464177787 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11082213525923514 = (13.899427819427537 - 12.359063549596963) / 13.899427819427537 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.500331811466573 < 22.95282016237008.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 19.452255429332453 (T) = (0 -15.12387948744167) / Math.Sqrt((0.26423872802041565 / (299)) + (0.598231215598205 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.3653318782256574 = (23.829587415167186 - 15.12387948744167) / 23.829587415167186 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.431178206657002 < 14.088084979089306.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 47.93040948971244 (T) = (0 -8.128704492242658) / Math.Sqrt((0.4283881935213691 / (299)) + (0.0686255883945905 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.47898047695680074 = (15.601535322062555 - 8.128704492242658) / 15.601535322062555 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.412413221546636 < 12.975752129026679.
IsChangePoint: Marked as a change because one of 1/5/2023 2:25:12 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 72.20979731210264 (T) = (0 -8.289292035660353) / Math.Sqrt((0.09421016891609484 / (299)) + (0.016756867699410316 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4008966622745765 = (13.836164003244859 - 8.289292035660353) / 13.836164003244859 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.970223257336106 < 12.898717913651655.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 34.139504325455924 (T) = (0 -8.13825215146131) / Math.Sqrt((0.2722313557701292 / (299)) + (0.05646741852187814 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.41729923758222565 = (13.966434706029254 - 8.13825215146131) / 13.966434706029254 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.996679942922413 < 15.339586839717867.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 47.18466515677412 (T) = (0 -8.26749300286182) / Math.Sqrt((0.3763163220397801 / (299)) + (0.06517273978665439 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4638714965760852 = (15.420730198194187 - 8.26749300286182) / 15.420730198194187 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.958446870137967 < 12.786381502842424.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 49.16417775594474 (T) = (0 -11.926875036021487) / Math.Sqrt((0.3495195060056005 / (299)) + (0.001138795626170013 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1395726572533457 = (13.861571388408628 - 11.926875036021487) / 13.861571388408628 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.8768871161924017 < 10.026496384102753.
IsChangePoint: Marked as a change because one of 12/5/2022 9:44:53 PM, 12/15/2022 2:06:03 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 119.92072326024676 (T) = (0 -0.9583898954216258) / Math.Sqrt((1.3720532353279917 / (299)) + (0.0091985098428596 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.9163020701169519 = (11.4505806387421 - 0.9583898954216258) / 11.4505806387421 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.37118737344351 < 24.353655122641175.
IsChangePoint: Marked as a change because one of 11/11/2022 8:39:50 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 48.74746736857108 (T) = (0 -23.439275254599305) / Math.Sqrt((0.025082086459690494 / (299)) + (0.005648123812170858 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08444068991657333 = (25.601045171463 - 23.439275254599305) / 25.601045171463 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.9201857833481156 < 5.238774000643605.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 42.739843355598104 (T) = (0 -0.7440677847838474) / Math.Sqrt((0.1948813248705364 / (299)) + (0.02949252968023729 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.8546734789547208 = (5.119972455351206 - 0.7440677847838474) / 5.119972455351206 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.45379498355859 < 58.410579629328005.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 91.67834580399229 (T) = (0 -32.46508571019296) / Math.Sqrt((20.0653130659876 / (299)) + (0.0007609953725647424 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4229394713649171 = (56.25941144680679 - 32.46508571019296) / 56.25941144680679 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.343974399733917 < 7.988758943551482.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 12.932738532077563 (T) = (0 -6.5112899212813575) / Math.Sqrt((0.40157022465627823 / (299)) + (0.09510498847310973 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2652763066657672 = (8.8622294072655 - 6.5112899212813575) / 8.8622294072655 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.453127927505573 < 8.574585642877965.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 32.23590673935884 (T) = (0 -5.385860414329397) / Math.Sqrt((1.0771834587031694 / (299)) + (0.01172829289895537 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.34156687127073204 = (8.17981383276347 - 5.385860414329397) / 8.17981383276347 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.970637669842174 < 13.128554696367416.
IsChangePoint: Marked as a change because one of 12/12/2022 5:25:39 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 40.031400030112444 (T) = (0 -8.172819514810902) / Math.Sqrt((0.10744595092063365 / (299)) + (0.058917378629650845 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4092179225455097 = (13.833898871856821 - 8.172819514810902) / 13.833898871856821 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.328273831000255 < 14.715695833206992.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 23.669566206479978 (T) = (0 -7.837296500197667) / Math.Sqrt((0.47676444139694873 / (299)) + (0.3256152594465522 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.5005656625859891 = (15.692346146598375 - 7.837296500197667) / 15.692346146598375 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.29856967640458 < 8.00911665389172.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 6:03:21 AM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 8.517488500484886 (T) = (0 -6.211372729322419) / Math.Sqrt((0.4094529175861283 / (299)) + (0.38808600234697826 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.3007636985932651 = (8.883081036877346 - 6.211372729322419) / 8.883081036877346 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.760725072762565 < 7.757607472186769.
IsChangePoint: Marked as a change because one of 2/14/2023 11:21:41 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 4.300677190710238 (T) = (0 -6.860498319408323) / Math.Sqrt((0.03280952601927689 / (299)) + (0.20397241250340548 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.14059067246193682 = (7.982806445749766 - 6.860498319408323) / 7.982806445749766 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0341821517045378 < 2.9725191344449335.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 39.219881742236375 (T) = (0 -1.0210770560473619) / Math.Sqrt((0.4127010693671611 / (299)) + (0.00696490493295961 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.7003304830147612 = (3.4073437509416697 - 1.0210770560473619) / 3.4073437509416697 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 8, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GreaterThanOrEqualBenchmark - Duration of single invocation 43.69 ns 32.49 ns 0.74 0.01 False
EqualsAllBenchmark - Duration of single invocation 13.23 ns 10.43 ns 0.79 0.05 False
AndNotBenchmark - Duration of single invocation 14.68 ns 13.02 ns 0.89 0.05 False
MultiplyBenchmark - Duration of single invocation 46.08 ns 36.67 ns 0.80 0.02 False
ConditionalSelectBenchmark - Duration of single invocation 16.50 ns 15.35 ns 0.93 0.03 False
BitwiseAndOperatorBenchmark - Duration of single invocation 14.59 ns 13.15 ns 0.90 0.01 False
GreaterThanBenchmark - Duration of single invocation 41.93 ns 31.22 ns 0.74 0.01 False
DivideBenchmark - Duration of single invocation 48.74 ns 39.36 ns 0.81 0.01 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 14.42 ns 13.26 ns 0.92 0.02 False
BitwiseAndBenchmark - Duration of single invocation 15.60 ns 13.04 ns 0.84 0.01 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 19.90 ns 18.22 ns 0.92 0.02 False
EqualsStaticBenchmark - Duration of single invocation 41.64 ns 30.88 ns 0.74 0.02 False
XorBenchmark - Duration of single invocation 15.40 ns 13.08 ns 0.85 0.02 False
BitwiseOrOperatorBenchmark - Duration of single invocation 14.31 ns 12.97 ns 0.91 0.03 False
LessThanOrEqualBenchmark - Duration of single invocation 42.98 ns 32.22 ns 0.75 0.02 False
LessThanOrEqualAllBenchmark - Duration of single invocation 12.22 ns 11.19 ns 0.92 0.06 False
LessThanBenchmark - Duration of single invocation 40.79 ns 31.45 ns 0.77 0.02 False
BitwiseOrBenchmark - Duration of single invocation 15.55 ns 13.10 ns 0.84 0.02 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.GreaterThanOrEqualBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.48793607475815 < 41.02874216779005.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 45.20420000374996 (T) = (0 -32.382938846644215) / Math.Sqrt((7.580858495715419 / (299)) + (0.049699222634395676 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.22227935197652285 = (41.63826552495835 - 32.382938846644215) / 41.63826552495835 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.42980344825657 < 12.58137047519909.
IsChangePoint: Marked as a change because one of 1/5/2023 11:12:07 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 82.11072616345957 (T) = (0 -10.452033189161272) / Math.Sqrt((0.1675226966670457 / (299)) + (0.0011171081250360745 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.19349314398944964 = (12.959633400840604 - 10.452033189161272) / 12.959633400840604 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.019701658343429 < 13.789872925668273.
IsChangePoint: Marked as a change because one of 11/17/2022 9:27:53 PM, 11/21/2022 12:46:21 PM, 12/15/2022 5:36:17 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 38.99998839070221 (T) = (0 -13.09084555925537) / Math.Sqrt((0.08822044055139293 / (299)) + (0.0038997674261340566 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10632889802313401 = (14.648393050080125 - 13.09084555925537) / 14.648393050080125 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.67029046710427 < 43.83627907086976.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 25.670716742172242 (T) = (0 -36.98114580572261) / Math.Sqrt((12.314843278857717 / (299)) + (0.08861053560938262 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1558356617490493 = (43.80799345580619 - 36.98114580572261) / 43.80799345580619 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.352674448657272 < 15.759476606507166.
IsChangePoint: Marked as a change because one of 11/21/2022 12:46:21 PM, 11/25/2022 10:05:08 AM, 12/21/2022 10:31:08 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 28.598584356158216 (T) = (0 -15.315069083188824) / Math.Sqrt((0.08132144035122316 / (299)) + (0.006795452758905127 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08597187673364326 = (16.755577529123645 - 15.315069083188824) / 16.755577529123645 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.146903046666443 < 13.716229553896243.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 24.321762973989504 (T) = (0 -13.02454081554294) / Math.Sqrt((0.2553762578800049 / (299)) + (0.011235151070529557 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11240516272483127 = (14.673970902677887 - 13.02454081554294) / 14.673970902677887 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.22301129158513 < 39.375291775279344.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 50.94626965789681 (T) = (0 -31.134835359124622) / Math.Sqrt((7.50928944762558 / (299)) + (0.011545265589766988 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2178189442559524 = (39.805151416647 - 31.134835359124622) / 39.805151416647 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 39.36166422042384 < 46.278562062854505.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 30.306098920714575 (T) = (0 -39.25961976200531) / Math.Sqrt((8.490857535476142 / (299)) + (0.09774652769732796 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.16010396713565705 = (46.74342802658098 - 39.25961976200531) / 46.74342802658098 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.264248868153777 < 13.669094291601649.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 12/14/2022 12:01:27 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 13.735381898903583 (T) = (0 -13.087932896250273) / Math.Sqrt((0.22644086586476528 / (299)) + (0.03426818790352103 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10380041103046968 = (14.603814883801789 - 13.087932896250273) / 14.603814883801789 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.041419341400436 < 14.825890571649065.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 65.54037740763542 (T) = (0 -13.025711315419551) / Math.Sqrt((0.17782961957447524 / (299)) + (0.0036022301589363915 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.17573757904379816 = (15.802869309884125 - 13.025711315419551) / 15.802869309884125 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.22299550256681 < 18.967398690369013.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 24.74442104593996 (T) = (0 -18.348254793460878) / Math.Sqrt((0.16674819400275784 / (299)) + (0.014530295670857098 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09017426113667519 = (20.166779208053573 - 18.348254793460878) / 20.166779208053573 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.882605971258204 < 39.39724858258719.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 45.45584055122005 (T) = (0 -31.089081690648907) / Math.Sqrt((7.340165907398661 / (299)) + (0.037008585227884076 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.21924232417925021 = (39.81911757443483 - 31.089081690648907) / 39.81911757443483 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.078353331480798 < 14.714506888244722.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 59.61830974281547 (T) = (0 -13.058224445449104) / Math.Sqrt((0.19492607172600107 / (299)) + (0.004339805523879449 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.17297113453214696 = (15.789321256715777 - 13.058224445449104) / 15.789321256715777 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.974650901806967 < 13.64334078921403.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 14.456842677285982 (T) = (0 -13.046501525460798) / Math.Sqrt((0.2825649303598841 / (299)) + (0.03387971043749766 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10919932158955634 = (14.64581453703105 - 13.046501525460798) / 14.64581453703105 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.22040343922818 < 40.90987617480315.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 59.91911380388401 (T) = (0 -32.29241914690688) / Math.Sqrt((6.091015242879856 / (299)) + (0.014527653510158932 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.22758150201701313 = (41.80689513680982 - 32.29241914690688) / 41.80689513680982 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.193361683320951 < 11.619386649198631.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 18.83854681792156 (T) = (0 -11.216338385207157) / Math.Sqrt((0.060780671317659345 / (299)) + (0.006752060590342232 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07681064573612174 = (12.149553429535274 - 11.216338385207157) / 12.149553429535274 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.44726340294409 < 38.79623079763619.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 39.02362982598357 (T) = (0 -31.16759533517255) / Math.Sqrt((7.221477092689791 / (299)) + (0.07312848931376645 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.21618880095662157 = (39.76416179459008 - 31.16759533517255) / 39.76416179459008 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.096025942777441 < 14.758201201381071.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 33.966768862549145 (T) = (0 -12.936338200474395) / Math.Sqrt((0.28400018111465897 / (299)) + (0.019212022482212848 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.18378309965256295 = (15.849142789089292 - 12.936338200474395) / 15.849142789089292 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 8, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultiplyOperatorBenchmark - Duration of single invocation 17.05 ns 14.65 ns 0.86 0.03 False
LessThanAnyBenchmark - Duration of single invocation 7.70 ns 6.38 ns 0.83 0.18 False
EqualsAnyBenchmark - Duration of single invocation 7.95 ns 6.55 ns 0.82 0.14 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 8.72 ns 7.58 ns 0.87 0.20 False
GreaterThanBenchmark - Duration of single invocation 15.72 ns 13.93 ns 0.89 0.03 False
GreaterThanOrEqualBenchmark - Duration of single invocation 16.12 ns 14.18 ns 0.88 0.04 False
ConditionalSelectBenchmark - Duration of single invocation 16.79 ns 15.60 ns 0.93 0.03 False
SubtractBenchmark - Duration of single invocation 17.75 ns 13.62 ns 0.77 0.05 False
BitwiseAndBenchmark - Duration of single invocation 15.58 ns 12.98 ns 0.83 0.02 False
BitwiseOrOperatorBenchmark - Duration of single invocation 14.59 ns 13.18 ns 0.90 0.02 False
MultiplyBenchmark - Duration of single invocation 18.87 ns 14.83 ns 0.79 0.06 False
AddBenchmark - Duration of single invocation 17.57 ns 13.79 ns 0.78 0.04 False
GreaterThanAllBenchmark - Duration of single invocation 8.14 ns 6.44 ns 0.79 0.16 False
XorBenchmark - Duration of single invocation 15.69 ns 12.97 ns 0.83 0.01 False
AddOperatorBenchmark - Duration of single invocation 15.64 ns 13.95 ns 0.89 0.02 False
DivideBenchmark - Duration of single invocation 23.06 ns 19.25 ns 0.83 0.01 False
LessThanBenchmark - Duration of single invocation 15.12 ns 13.48 ns 0.89 0.01 False
EqualsAllBenchmark - Duration of single invocation 8.59 ns 4.17 ns 0.49 0.29 False
BitwiseAndOperatorBenchmark - Duration of single invocation 14.36 ns 13.19 ns 0.92 0.04 False
AndNotBenchmark - Duration of single invocation 14.59 ns 12.94 ns 0.89 0.06 False
SubtractionOperatorBenchmark - Duration of single invocation 15.70 ns 13.57 ns 0.86 0.02 False
BitwiseOrBenchmark - Duration of single invocation 15.46 ns 13.18 ns 0.85 0.02 False
MaxBenchmark - Duration of single invocation 15.66 ns 14.35 ns 0.92 0.02 False
EqualsBenchmark - Duration of single invocation 10.88 ns 8.01 ns 0.74 0.09 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 14.47 ns 13.02 ns 0.90 0.03 False
DivisionOperatorBenchmark - Duration of single invocation 21.31 ns 19.83 ns 0.93 0.01 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.MultiplyOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.652769077617004 < 15.86412926662337.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 18.556095502956996 (T) = (0 -14.8760105494459) / Math.Sqrt((0.18532938793041726 / (299)) + (0.03739997064861374 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.12487657520547475 = (16.99875712153256 - 14.8760105494459) / 16.99875712153256 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.37525248418033 < 7.466120772356914.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 11.535779071682583 (T) = (0 -6.440393049113418) / Math.Sqrt((0.29527649038470183 / (299)) + (0.05096854022300915 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.19364999379227626 = (7.987093693224712 - 6.440393049113418) / 7.987093693224712 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.545122546566033 < 7.617518059861099.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 10.181146755986873 (T) = (0 -6.211572980801868) / Math.Sqrt((0.31375148698592653 / (299)) + (0.09009374672258566 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.22418152337101474 = (8.006477246832043 - 6.211572980801868) / 8.006477246832043 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.583304150785836 < 8.299077744133355.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 7.471535974326124 (T) = (0 -7.43226703847563) / Math.Sqrt((0.12775538341759188 / (299)) + (0.056137121307726676 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.12209628779779222 = (8.465925061225569 - 7.43226703847563) / 8.465925061225569 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.932566712718607 < 14.99224825808045.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 26.551279758496495 (T) = (0 -13.881155462997564) / Math.Sqrt((0.6309863000563106 / (299)) + (0.0172596817631918 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.14501920151546013 = (16.235634165822226 - 13.881155462997564) / 16.235634165822226 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.177691900953825 < 15.376285442813543.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 23.17853132198802 (T) = (0 -14.352024519548543) / Math.Sqrt((0.4261264245381693 / (299)) + (0.02284741516414733 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.13311896476441062 = (16.555933209045392 - 14.352024519548543) / 16.555933209045392 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.596761797261637 < 15.834851938666649.
IsChangePoint: Marked as a change because one of 12/15/2022 5:36:17 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 15.399361465087631 (T) = (0 -15.471487645718774) / Math.Sqrt((0.25921905648871046 / (299)) + (0.026293898951121168 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08899027758157767 = (16.982790924170615 - 15.471487645718774) / 16.982790924170615 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.619934463724208 < 16.911779303947256.
IsChangePoint: Marked as a change because one of 12/14/2022 12:01:27 PM, 12/30/2022 6:15:13 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 31.800572445102674 (T) = (0 -13.830350388639607) / Math.Sqrt((0.1534261807100764 / (299)) + (0.04365363515536259 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.22009831105678476 = (17.733453568205565 - 13.830350388639607) / 17.733453568205565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.983093839492513 < 14.736562108844367.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 43.86564427437511 (T) = (0 -13.036812634470834) / Math.Sqrt((0.4908483230295786 / (299)) + (0.005771417867845841 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.1846047678240703 = (15.988335619379745 - 13.036812634470834) / 15.988335619379745 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.180782445646392 < 13.64522242597758.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 19.55231899343511 (T) = (0 -13.033393065608747) / Math.Sqrt((0.2772718887436569 / (299)) + (0.01834706514636701 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11182039244971126 = (14.674276413029215 - 13.033393065608747) / 14.674276413029215 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.83295446869379 < 17.69594671110804.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 34.18157695750558 (T) = (0 -14.768281048610566) / Math.Sqrt((0.06112327091536339 / (299)) + (0.03952718657755685 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.21118684238481777 = (18.722153536661953 - 14.768281048610566) / 18.722153536661953 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.785473760856545 < 16.72671488643854.
IsChangePoint: Marked as a change because one of 11/24/2022 8:21:24 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 41.91412854474763 (T) = (0 -13.779839009729455) / Math.Sqrt((0.07116203718340271 / (299)) + (0.024778937486427812 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.21898918265913797 = (17.64359558635335 - 13.779839009729455) / 17.64359558635335 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.44303160110172 < 7.758915818584925.
IsChangePoint: Marked as a change because one of 12/14/2022 12:01:27 PM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 10.468557260793808 (T) = (0 -6.396998999269242) / Math.Sqrt((0.24788876342720018 / (299)) + (0.07757001425447733 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2109404549704703 = (8.107118201110971 - 6.396998999269242) / 8.107118201110971 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.968471060204998 < 14.749022299555506.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 95.8825861638607 (T) = (0 -12.992179102584695) / Math.Sqrt((0.16424109447654006 / (299)) + (0.0009549810928224529 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1785654975297422 = (15.816451662931106 - 12.992179102584695) / 15.816451662931106 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.949233955307054 < 14.934860469449156.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 12/2/2022 9:06:56 PM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 21.528724627977418 (T) = (0 -14.045578496538738) / Math.Sqrt((0.10121967701314782 / (299)) + (0.01917061575545659 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1116889871881039 = (15.811555067946628 - 14.045578496538738) / 15.811555067946628 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.252863606132944 < 22.08056765781755.
IsChangePoint: Marked as a change because one of 12/12/2022 5:25:39 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 24.00465379218265 (T) = (0 -19.463864192957747) / Math.Sqrt((0.07018979901644364 / (299)) + (0.06965647769205777 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.15886757793300096 = (23.140071268596735 - 19.463864192957747) / 23.140071268596735 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.48299899144004 < 14.346402765753814.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 23.891099088893174 (T) = (0 -13.420700238701224) / Math.Sqrt((0.42538853773607177 / (299)) + (0.019207092184053313 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.13604866894383347 = (15.534092901152933 - 13.420700238701224) / 15.534092901152933 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.172186472755573 < 7.833358639898948.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 42.671916402650105 (T) = (0 -4.312884959370401) / Math.Sqrt((0.3636378891556802 / (299)) + (0.017459486141439686 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.45352872745707856 = (7.89224461754603 - 4.312884959370401) / 7.89224461754603 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.193004178078292 < 13.551610235603892.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 5.0152333811891365 (T) = (0 -12.686955878168284) / Math.Sqrt((0.29740568954958135 / (299)) + (0.4646934564543963 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.13500770352480615 = (14.667131637896754 - 12.686955878168284) / 14.667131637896754 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.942245407127995 < 13.819093111610698.
IsChangePoint: Marked as a change because one of 11/17/2022 5:10:53 PM, 11/21/2022 12:46:21 PM, 11/25/2022 10:05:08 AM, 12/12/2022 5:25:39 PM, 12/15/2022 2:06:03 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 12.000648247467629 (T) = (0 -13.15859265135814) / Math.Sqrt((0.07303713607798563 / (299)) + (0.03554746232350457 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0911510430739627 = (14.478305279530616 - 13.15859265135814) / 14.478305279530616 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.573603547570464 < 14.897704879087389.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 6.900464390111486 (T) = (0 -13.890346124331009) / Math.Sqrt((0.19034801888342615 / (299)) + (0.2683650868361028 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1297614854210376 = (15.961539154643617 - 13.890346124331009) / 15.961539154643617 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.18364183164519 < 14.683360004134881.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 7.828062362776047 (T) = (0 -12.711502037550282) / Math.Sqrt((0.4702809078527165 / (299)) + (0.5204851567431393 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.20487766550495867 = (15.986850684584255 - 12.711502037550282) / 15.986850684584255 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.348806686234454 < 14.875151600492002.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 25.260101314557904 (T) = (0 -14.219210004710519) / Math.Sqrt((0.22305493546045865 / (299)) + (0.013026888254535293 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11246822728583333 = (16.021071517503717 - 14.219210004710519) / 16.021071517503717 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.00775282763401 < 10.319762009742034.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 10.278558159466325 (T) = (0 -7.633430923488402) / Math.Sqrt((1.1845862449062747 / (299)) + (0.14854551981638497 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.23744670124222222 = (10.010357224765128 - 7.633430923488402) / 10.010357224765128 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.016733636966302 < 13.688602363705373.
IsChangePoint: Marked as a change because one of 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 28.08380044569186 (T) = (0 -13.108042213948545) / Math.Sqrt((0.13085424418790517 / (299)) + (0.006508927961840864 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09861078784840357 = (14.54204469860465 - 13.108042213948545) / 14.54204469860465 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.828191803951295 < 20.253199879821.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 9.698354691747692 (T) = (0 -19.56896432662441) / Math.Sqrt((0.043772926912553564 / (299)) + (0.08750749569211938 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07821814232887293 = (21.229496071950482 - 19.56896432662441) / 21.229496071950482 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 8, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConditionalSelectBenchmark - Duration of single invocation 58.02 ns 32.44 ns 0.56 0.37 False
BitwiseOrBenchmark - Duration of single invocation 16.30 ns 8.31 ns 0.51 0.26 False
AddBenchmark - Duration of single invocation 26.09 ns 23.50 ns 0.90 0.01 False
LessThanAnyBenchmark - Duration of single invocation 3.19 ns 0.79 ns 0.25 0.41 False
MultiplyBenchmark - Duration of single invocation 27.11 ns 24.36 ns 0.90 0.00 False
BitwiseOrOperatorBenchmark - Duration of single invocation 13.81 ns 7.98 ns 0.58 0.03 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 13.52 ns 8.21 ns 0.61 0.02 False
OneBenchmark - Duration of single invocation 14.86 ns 0.96 ns 0.06 0.08 False
EqualityOperatorBenchmark - Duration of single invocation 3.04 ns 0.82 ns 0.27 0.42 False
OnesComplementBenchmark - Duration of single invocation 8.39 ns 6.30 ns 0.75 0.04 False
EqualsAllBenchmark - Duration of single invocation 4.63 ns 0.86 ns 0.19 0.21 False
BitwiseAndOperatorBenchmark - Duration of single invocation 13.55 ns 8.02 ns 0.59 0.03 False
GreaterThanAllBenchmark - Duration of single invocation 2.72 ns 1.03 ns 0.38 0.45 False
SubtractBenchmark - Duration of single invocation 26.03 ns 23.86 ns 0.92 0.01 False
LessThanOrEqualAllBenchmark - Duration of single invocation 20.07 ns 18.77 ns 0.94 0.02 False
OnesComplementOperatorBenchmark - Duration of single invocation 8.41 ns 6.33 ns 0.75 0.05 False
AndNotBenchmark - Duration of single invocation 24.00 ns 15.56 ns 0.65 0.02 False
BitwiseAndBenchmark - Duration of single invocation 14.87 ns 8.30 ns 0.56 0.26 False
XorBenchmark - Duration of single invocation 16.30 ns 8.18 ns 0.50 0.26 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt16>.ConditionalSelectBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.44278025455882 < 58.70954767346687.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 99.69068404071734 (T) = (0 -32.46719748658566) / Math.Sqrt((16.943789985489687 / (299)) + (0.0037854787089296347 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4249665356701055 = (56.46140529303066 - 32.46719748658566) / 56.46140529303066 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.310747444674586 < 15.408868071056403.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 83.56298123541121 (T) = (0 -8.301964932209954) / Math.Sqrt((0.4544383123575239 / (299)) + (0.0181696668248785 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4669858269685006 = (15.575505028305006 - 8.301964932209954) / 15.575505028305006 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.497472771626388 < 24.677361133022682.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 24.04951379603211 (T) = (0 -23.578871529671705) / Math.Sqrt((0.037708844607217434 / (299)) + (0.03356714638875703 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09787676469807968 = (26.137084831631 - 23.578871529671705) / 26.137084831631 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.785120987657617 < 2.77615605980854.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 18.376237197608056 (T) = (0 -0.7813022463021749) / Math.Sqrt((0.42703751769306203 / (299)) + (0.0492349220549394 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.7585399758181042 = (3.2357416054659502 - 0.7813022463021749) / 3.2357416054659502 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.361753733763088 < 25.743137881940147.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 18.54845612464818 (T) = (0 -24.490683385199976) / Math.Sqrt((0.09501320006400336 / (299)) + (0.049822608914357525 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08969383897626758 = (26.903787356176625 - 24.490683385199976) / 26.903787356176625 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.982934925690022 < 13.122815936863104.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 49.95285810753787 (T) = (0 -8.095200636243446) / Math.Sqrt((0.1955493107047355 / (299)) + (0.039205807048973196 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.41956807259572876 = (13.946856218688213 - 8.095200636243446) / 13.946856218688213 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.20713914873284 < 12.868785569783258.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 88.2310931020778 (T) = (0 -8.112027075237883) / Math.Sqrt((0.1534284180054967 / (299)) + (0.01146369246359814 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4172730465175469 = (13.920802919376458 - 8.112027075237883) / 13.920802919376458 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.9576690166114534 < 14.118339661651001.
IsChangePoint: Marked as a change because one of 12/5/2022 9:44:53 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 123.7026679962117 (T) = (0 -0.9192379429839841) / Math.Sqrt((4.501634209800545 / (299)) + (0.0029959204915520825 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.9446009964104561 = (16.593041091401204 - 0.9192379429839841) / 16.593041091401204 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.8156947663509415 < 2.8925084703871984.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 25.778420822176177 (T) = (0 -0.9278900946928226) / Math.Sqrt((0.5717426821140812 / (299)) + (0.01933951757763564 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.7175118517032882 = (3.2847045098621703 - 0.9278900946928226) / 3.2847045098621703 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.302784856665173 < 8.033360359101271.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 61.76675647862863 (T) = (0 -6.334612962887159) / Math.Sqrt((0.42588734547549256 / (299)) + (0.0007608678261378399 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.28541835506000574 = (8.864785441583932 - 6.334612962887159) / 8.864785441583932 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.8576496916525634 < 4.739647820611115.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 21.128492090570376 (T) = (0 -1.017478438333271) / Math.Sqrt((0.08670981986782435 / (299)) + (0.09858230248047226 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.7908324143730652 = (4.864417377499473 - 1.017478438333271) / 4.864417377499473 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.017993001043514 < 12.875621532812726.
IsChangePoint: Marked as a change because one of 12/14/2022 8:02:51 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 101.89076843283854 (T) = (0 -8.047851620138905) / Math.Sqrt((0.11631823005264554 / (299)) + (0.008579354822040057 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.41915644578137196 = (13.855454815135495 - 8.047851620138905) / 13.855454815135495 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.028301145546064 < 2.7211650792661515.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 7.646205896458128 (T) = (0 -1.123707600931665) / Math.Sqrt((0.5681990310022845 / (299)) + (0.226780264641725 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.6544814724045537 = (3.252235440894704 - 1.123707600931665) / 3.252235440894704 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.857943796511904 < 24.723396029345995.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 12/5/2022 3:49:45 AM, 12/14/2022 12:01:27 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 37.45000322349525 (T) = (0 -23.793403430802783) / Math.Sqrt((0.07641298714688813 / (299)) + (0.00976764608670079 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08531218215983909 = (26.01259464347716 - 23.793403430802783) / 26.01259464347716 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.7693641861575 < 18.980149914547287.
IsChangePoint: Marked as a change because one of 1/5/2023 9:29:47 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 11.491155977933493 (T) = (0 -18.7122698060934) / Math.Sqrt((0.05874418633420117 / (299)) + (0.024358095084079028 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.05303054867985498 = (19.76016204113778 - 18.7122698060934) / 19.76016204113778 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.331204485850102 < 8.039289209104455.
IsChangePoint: Marked as a change because one of 12/12/2022 5:25:39 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 13.126269804614704 (T) = (0 -6.515890489035954) / Math.Sqrt((0.41783753199837437 / (299)) + (0.09177629397230248 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2648713343909884 = (8.863605507258944 - 6.515890489035954) / 8.863605507258944 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.561817070669417 < 22.942872014612973.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 196.22916780032935 (T) = (0 -15.541189082102255) / Math.Sqrt((0.2758939047684509 / (299)) + (0.0025689574152704827 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.34749897205187175 = (23.817876779402912 - 15.541189082102255) / 23.817876779402912 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.300669262307974 < 14.091268546899059.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 97.91778696024775 (T) = (0 -8.35985258160738) / Math.Sqrt((0.4167594568343267 / (299)) + (0.012208821194932487 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4640248087386802 = (15.597461818958433 - 8.35985258160738) / 15.597461818958433 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.180073288382513 < 14.095692299068288.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 90.75668567775878 (T) = (0 -8.107194974994881) / Math.Sqrt((0.39838836498158503 / (299)) + (0.01577637784291809 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.47612315288897195 = (15.475383231197998 - 8.107194974994881) / 15.475383231197998 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 8, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Narrow_double - Duration of single invocation 26.83 μs 22.83 μs 0.85 0.00 False
Widen_float - Duration of single invocation 71.13 μs 64.14 μs 0.90 0.01 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorConvert* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorConvert* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorConvert.Narrow_double

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.828960859179194 < 25.48861979166666.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 37.93720511834243 (T) = (0 -22930.574053713895) / Math.Sqrt((27966.61777928998 / (299)) + (29979.46372533676 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1424842367625037 = (26740.70266316851 - 22930.574053713895) / 26740.70266316851 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_float

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 64.1441130952381 < 66.51188350142044.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 126.79101201891541 (T) = (0 -64125.63824829933) / Math.Sqrt((725938.5753579339 / (299)) + (289.40496343712664 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09036481521838961 = (70495.99589058873 - 64125.63824829933) / 70495.99589058873 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 8, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GreaterThanAllBenchmark - Duration of single invocation 8.26 ns 6.55 ns 0.79 0.10 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 13.58 ns 8.15 ns 0.60 0.02 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 9.61 ns 7.71 ns 0.80 0.10 False
BitwiseOrBenchmark - Duration of single invocation 16.33 ns 8.25 ns 0.51 0.26 False
BitwiseAndOperatorBenchmark - Duration of single invocation 13.52 ns 8.32 ns 0.62 0.01 False
OnesComplementBenchmark - Duration of single invocation 8.38 ns 6.34 ns 0.76 0.09 False
BitwiseOrOperatorBenchmark - Duration of single invocation 13.60 ns 8.45 ns 0.62 0.02 False
XorBenchmark - Duration of single invocation 16.19 ns 8.42 ns 0.52 0.27 False
BitwiseAndBenchmark - Duration of single invocation 14.73 ns 8.43 ns 0.57 0.26 False
EqualityOperatorBenchmark - Duration of single invocation 8.22 ns 6.48 ns 0.79 0.08 False
ConditionalSelectBenchmark - Duration of single invocation 60.30 ns 32.50 ns 0.54 0.30 False
LessThanAnyBenchmark - Duration of single invocation 8.39 ns 7.00 ns 0.83 0.10 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 9.66 ns 7.56 ns 0.78 0.08 False
OneBenchmark - Duration of single invocation 12.24 ns 0.96 ns 0.08 0.23 False
OnesComplementOperatorBenchmark - Duration of single invocation 8.54 ns 6.35 ns 0.74 0.04 False
EqualsAllBenchmark - Duration of single invocation 10.00 ns 6.78 ns 0.68 0.09 False
AndNotBenchmark - Duration of single invocation 24.00 ns 15.34 ns 0.64 0.02 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Single>.GreaterThanAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.547392650090684 < 7.841959737146368.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 11.09675523531491 (T) = (0 -6.698281896085525) / Math.Sqrt((0.10321723482297289 / (299)) + (0.048337693191579725 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.17527742901194593 = (8.121860770781 - 6.698281896085525) / 8.121860770781 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.149578882143382 < 12.867403376111172.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 93.58876739407472 (T) = (0 -8.030514714014684) / Math.Sqrt((0.08231905090871838 / (299)) + (0.01068451176526013 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4192399752011721 = (13.827595514681663 - 8.030514714014684) / 13.827595514681663 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.713642867923772 < 9.05406008018731.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 10.166027764976844 (T) = (0 -7.4305017702583145) / Math.Sqrt((0.2676831106728824 / (299)) + (0.06830590903635975 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.17387000902903685 = (8.994349377783916 - 7.4305017702583145) / 8.994349377783916 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.253121799873435 < 14.74265213863359.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 83.0924498113756 (T) = (0 -8.141355777799712) / Math.Sqrt((0.41573937343059036 / (299)) + (0.019668033132305154 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.47638800781644264 = (15.548451714882953 - 8.141355777799712) / 15.548451714882953 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.31805551119699 < 12.87523564984597.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 104.99300489616239 (T) = (0 -8.22162896272799) / Math.Sqrt((0.20224265340873168 / (299)) + (0.0070177383427841005 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.412207836329201 = (13.9873061787408 - 8.22162896272799) / 13.9873061787408 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.337748133044835 < 8.036328999318219.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 69.47335186992882 (T) = (0 -6.316226543820664) / Math.Sqrt((0.35502716220192165 / (299)) + (0.0003495114184752717 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.28426929432531606 = (8.824864566718107 - 6.316226543820664) / 8.824864566718107 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.450294653790065 < 13.032652714873878.
IsChangePoint: Marked as a change because one of 12/12/2022 5:25:39 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 19.439063579368327 (T) = (0 -7.948519001424397) / Math.Sqrt((0.0940843465132993 / (299)) + (0.27515061997313073 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4259210379174036 = (13.84568940235924 - 7.948519001424397) / 13.84568940235924 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.415300072225424 < 15.431496869893051.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 170.89728257848463 (T) = (0 -8.432558125861757) / Math.Sqrt((0.4297427022791482 / (299)) + (0.0005956808306290729 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.4577558008327211 = (15.551218692263715 - 8.432558125861757) / 15.551218692263715 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.432266708960224 < 15.476250297642832.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 38.55029267302078 (T) = (0 -8.320735381264088) / Math.Sqrt((0.43855448799128227 / (299)) + (0.10176625800019672 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.46568737651393466 = (15.572784575023407 - 8.320735381264088) / 15.572784575023407 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.4826259682537035 < 7.8114657838040715.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 10.135853246254944 (T) = (0 -6.73461720176142) / Math.Sqrt((0.11282031383596366 / (299)) + (0.055282814912543526 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1710789031600889 = (8.124557605586038 - 6.73461720176142) / 8.124557605586038 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.49550365287575 < 57.315994024951124.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 103.73665202927306 (T) = (0 -32.477590035334266) / Math.Sqrt((16.09154517439527 / (299)) + (0.0027154785078082203 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.42765379647410834 = (56.74465880136663 - 32.477590035334266) / 56.74465880136663 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.998307429437944 < 7.9265187208748555.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 8.383717560925822 (T) = (0 -6.662274964093833) / Math.Sqrt((0.09942608332132927 / (299)) + (0.08795962727950626 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.17809969826078395 = (8.105940525871388 - 6.662274964093833) / 8.105940525871388 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.563418886972566 < 8.97923970266782.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 23.431050462810664 (T) = (0 -7.50571184160582) / Math.Sqrt((0.2620353055986648 / (299)) + (0.009334634306004263 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.16467523329154615 = (8.985381663208214 - 7.50571184160582) / 8.985381663208214 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.9573753398400479 < 12.045939353448196.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 188.22640783033313 (T) = (0 -0.9918738260786211) / Math.Sqrt((0.6497464573863243 / (299)) + (0.004695558713010209 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.9206518073939198 = (12.500269930566974 - 0.9918738260786211) / 12.500269930566974 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.35001144567772 < 8.033229208532383.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 12.45825074194652 (T) = (0 -6.684797740405642) / Math.Sqrt((0.36307834559276775 / (299)) + (0.08406921458874458 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2416588260904655 = (8.81502675892302 - 6.684797740405642) / 8.81502675892302 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.7843318702885 < 9.241700041458751.
IsChangePoint: Marked as a change because one of 12/29/2022 3:42:58 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 20.35262707379724 (T) = (0 -7.025196173336719) / Math.Sqrt((0.08513396628174504 / (299)) + (0.0444511963299066 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2625485971415518 = (9.526317457809741 - 7.025196173336719) / 9.526317457809741 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.341974039219041 < 22.88771808293246.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 26.039892044023546 (T) = (0 -15.096977468009712) / Math.Sqrt((0.15597478585812186 / (299)) + (0.34385684354029955 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.36919646822509494 = (23.932931106981965 - 15.096977468009712) / 23.932931106981965 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 8, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SubtractBenchmark - Duration of single invocation 38.36 ns 34.61 ns 0.90 0.01 False
MultiplyBenchmark - Duration of single invocation 38.80 ns 34.58 ns 0.89 0.00 False
DotBenchmark - Duration of single invocation 25.43 ns 22.93 ns 0.90 0.01 False
ConditionalSelectBenchmark - Duration of single invocation 16.85 ns 15.53 ns 0.92 0.02 False
GreaterThanBenchmark - Duration of single invocation 40.06 ns 37.61 ns 0.94 0.00 False
AddBenchmark - Duration of single invocation 38.39 ns 33.52 ns 0.87 0.00 False
BitwiseAndBenchmark - Duration of single invocation 15.57 ns 13.04 ns 0.84 0.02 False
BitwiseOrOperatorBenchmark - Duration of single invocation 14.26 ns 13.08 ns 0.92 0.01 False
EqualsAllBenchmark - Duration of single invocation 8.29 ns 5.67 ns 0.68 0.08 False
BitwiseOrBenchmark - Duration of single invocation 15.57 ns 13.01 ns 0.84 0.01 False
LessThanBenchmark - Duration of single invocation 39.03 ns 36.51 ns 0.94 0.00 False
DivisionOperatorBenchmark - Duration of single invocation 62.44 ns 58.37 ns 0.93 0.01 False
DivideBenchmark - Duration of single invocation 63.26 ns 58.24 ns 0.92 0.00 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 14.40 ns 13.18 ns 0.92 0.01 False
GreaterThanAllBenchmark - Duration of single invocation 22.75 ns 21.55 ns 0.95 0.01 False
LessThanAnyBenchmark - Duration of single invocation 22.85 ns 20.97 ns 0.92 0.02 False
AndNotBenchmark - Duration of single invocation 14.55 ns 13.35 ns 0.92 0.04 False
EqualsStaticBenchmark - Duration of single invocation 38.65 ns 36.35 ns 0.94 0.00 False
XorBenchmark - Duration of single invocation 15.56 ns 12.96 ns 0.83 0.02 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.SubtractBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.611192277206904 < 36.375131222383985.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 13.892497665559764 (T) = (0 -34.35058155984904) / Math.Sqrt((0.13233101667020153 / (299)) + (0.2155913897265604 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09808450035036319 = (38.086252618114514 - 34.35058155984904) / 38.086252618114514 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.57502605367443 < 36.840899701253136.
IsChangePoint: Marked as a change because one of 12/16/2022 11:20:37 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 13.9304947207803 (T) = (0 -35.05990600878244) / Math.Sqrt((0.26007029023331507 / (299)) + (0.1834934162010836 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09005102847750265 = (38.52952979343593 - 35.05990600878244) / 38.52952979343593 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.92887417160146 < 24.28019382018359.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 33.24127526086867 (T) = (0 -22.977757900994302) / Math.Sqrt((0.27440826780605526 / (299)) + (0.014164682391640076 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09799259862608903 = (25.474023678736184 - 22.977757900994302) / 25.474023678736184 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.527552180429248 < 15.967581971044101.
IsChangePoint: Marked as a change because one of 11/17/2022 5:10:53 PM, 11/21/2022 12:46:21 PM, 12/5/2022 3:49:45 AM, 12/14/2022 12:01:27 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 13.495383096629855 (T) = (0 -15.318971905436653) / Math.Sqrt((0.07955394229160248 / (299)) + (0.0326339722153849 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08508589863015271 = (16.74361766039069 - 15.318971905436653) / 16.74361766039069 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 37.610454855269396 < 38.26627203947204.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 13.990804890234399 (T) = (0 -37.589655567888165) / Math.Sqrt((0.48529618764113236 / (299)) + (0.07299246076808381 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.05656988208848281 = (39.843603521054476 - 37.589655567888165) / 39.843603521054476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.52336086507386 < 36.39539697308398.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 152.4808593443043 (T) = (0 -33.518701489607615) / Math.Sqrt((0.10642404628089742 / (299)) + (0.0016110922239573884 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11967129365896953 = (38.07521127979985 - 33.518701489607615) / 38.07521127979985 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.042678593401563 < 14.761287879294434.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 38.9175297513454 (T) = (0 -12.914327279255913) / Math.Sqrt((0.275032898659448 / (299)) + (0.014562616442231793 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1863227356834528 = (15.871559702609332 - 12.914327279255913) / 15.871559702609332 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.078204907864267 < 13.648798008966839.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 21.770273956824578 (T) = (0 -12.97795597737) / Math.Sqrt((0.20854465165859032 / (299)) + (0.014830298866893467 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1118917258031101 = (14.613033516781357 - 12.97795597737) / 14.613033516781357 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.667061316962656 < 7.874734189457226.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 98.48773921941697 (T) = (0 -5.660442670358408) / Math.Sqrt((0.15379998100266234 / (299)) + (8.761296574384545E-05 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.29132084368252004 = (7.9873136099724595 - 5.660442670358408) / 7.9873136099724595 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.009746866939764 < 14.729881436100188.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 51.37655969830709 (T) = (0 -12.986961096497348) / Math.Sqrt((0.5614799914295832 / (299)) + (0.004885779413976284 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.18979557381899076 = (16.029239876796115 - 12.986961096497348) / 16.029239876796115 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.50787592661666 < 36.81246786607191.
IsChangePoint: Marked as a change because one of 11/25/2022 4:06:25 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 16.696806605219162 (T) = (0 -36.405402343204706) / Math.Sqrt((0.08428937140813088 / (299)) + (0.052581186510628014 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.05767500616376642 = (38.63359518354407 - 36.405402343204706) / 38.63359518354407 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 58.36835786452159 < 58.82898081629933.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 10.798261752915455 (T) = (0 -58.124889144934485) / Math.Sqrt((2.7632882079022267 / (299)) + (0.2266439857524905 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.05131960965005111 = (61.269200603475525 - 58.124889144934485) / 61.269200603475525 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 58.23883482921624 < 60.00069440920127.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 16.76081696547745 (T) = (0 -57.90051033556029) / Math.Sqrt((0.23606747078806092 / (299)) + (0.2557812397550831 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07826941164385912 = (62.817173550487105 - 57.90051033556029) / 62.817173550487105 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.182711956062823 < 13.638727948497634.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 13.16027590467862 (T) = (0 -13.080492394592978) / Math.Sqrt((0.18499510645943687 / (299)) + (0.038286187731986734 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10424834305014993 = (14.602811273756101 - 13.080492394592978) / 14.602811273756101 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.55052849583277 < 21.599818336921516.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 5.602788654584294 (T) = (0 -21.072220542719393) / Math.Sqrt((0.3692823902104769 / (299)) + (0.2350062002613446 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.06976872759779448 = (22.652668393208312 - 21.072220542719393) / 22.652668393208312 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.97168002233133 < 21.79509321586907.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 6.269431750614039 (T) = (0 -21.206323476650024) / Math.Sqrt((0.4202539123427807 / (299)) + (0.1308719156422764 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.059031837925358516 = (22.536706693555324 - 21.206323476650024) / 22.536706693555324 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.352848923406658 < 13.806895311062775.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 12/14/2022 8:02:51 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 22.418825047171165 (T) = (0 -13.248306036559251) / Math.Sqrt((0.08334368267191307 / (299)) + (0.008246789335362102 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0851808729706788 = (14.481885703002606 - 13.248306036559251) / 14.481885703002606 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.35466365511468 < 36.77485143945581.
IsChangePoint: Marked as a change because one of 12/12/2022 3:40:04 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 17.17542214049869 (T) = (0 -36.39250368360229) / Math.Sqrt((0.08690435122362018 / (299)) + (0.05023615162735134 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.05802549846165172 = (38.63427685586958 - 36.39250368360229) / 38.63427685586958 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.958375170565526 < 14.69719580551006.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 45.19691610706381 (T) = (0 -13.032544324743347) / Math.Sqrt((0.6267963101903139 / (299)) + (0.00677633471710869 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.18624048288834594 = (16.01522814872982 - 13.032544324743347) / 16.01522814872982 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 8, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsAllBenchmark - Duration of single invocation 5.56 ns 1.02 ns 0.18 0.13 False
LessThanAllBenchmark - Duration of single invocation 7.96 ns 6.41 ns 0.80 0.08 False
AndNotBenchmark - Duration of single invocation 24.38 ns 15.70 ns 0.64 0.02 False
OnesComplementBenchmark - Duration of single invocation 8.48 ns 6.85 ns 0.81 0.04 False
BitwiseOrOperatorBenchmark - Duration of single invocation 13.50 ns 7.97 ns 0.59 0.02 False
OnesComplementOperatorBenchmark - Duration of single invocation 7.40 ns 6.36 ns 0.86 0.03 False
SubtractBenchmark - Duration of single invocation 13.50 ns 11.56 ns 0.86 0.02 False
BitwiseOrBenchmark - Duration of single invocation 16.31 ns 8.18 ns 0.50 0.30 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 3.04 ns 1.07 ns 0.35 0.38 False
DivideBenchmark - Duration of single invocation 15.63 ns 13.56 ns 0.87 0.02 False
OneBenchmark - Duration of single invocation 10.58 ns 1.07 ns 0.10 0.19 False
ConditionalSelectBenchmark - Duration of single invocation 59.89 ns 32.46 ns 0.54 0.28 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 13.90 ns 8.30 ns 0.60 0.01 False
XorBenchmark - Duration of single invocation 16.36 ns 8.13 ns 0.50 0.26 False
MultiplyBenchmark - Duration of single invocation 13.64 ns 12.39 ns 0.91 0.02 False
BitwiseAndOperatorBenchmark - Duration of single invocation 13.95 ns 8.40 ns 0.60 0.03 False
EqualsBenchmark - Duration of single invocation 9.14 ns 5.50 ns 0.60 0.06 False
GreaterThanAnyBenchmark - Duration of single invocation 7.91 ns 6.52 ns 0.82 0.07 False
AddBenchmark - Duration of single invocation 13.44 ns 11.53 ns 0.86 0.01 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 3.45 ns 1.17 ns 0.34 0.40 False
BitwiseAndBenchmark - Duration of single invocation 15.04 ns 8.29 ns 0.55 0.28 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt32>.EqualsAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.017968064747111 < 5.2017701202602895.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 74.70031010787797 (T) = (0 -0.9272434129341359) / Math.Sqrt((0.19572365319373317 / (299)) + (0.007547831005178161 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.8193716836395473 = (5.133433293392247 - 0.9272434129341359) / 5.133433293392247 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.4078954701788335 < 7.686767291468114.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 5.836144720523423 (T) = (0 -6.736621696853671) / Math.Sqrt((0.048170374099494984 / (299)) + (0.13312227895080211 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.15456639950612702 = (7.96824457050012 - 6.736621696853671) / 7.96824457050012 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.697508386529698 < 23.035989633226762.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 258.8498503267428 (T) = (0 -15.70316087880999) / Math.Sqrt((0.27727946379536067 / (299)) + (0.00015814128895208976 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.34038988955657795 = (23.80673162855791 - 15.70316087880999) / 23.80673162855791 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.8502355800815025 < 8.021406311757072.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 11.715158860149504 (T) = (0 -6.691298007998835) / Math.Sqrt((0.15147267888901403 / (299)) + (0.08587181285320883 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.23007134242892888 = (8.690802637621227 - 6.691298007998835) / 8.690802637621227 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.9681904296517345 < 13.011543038184112.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 40.73731303764828 (T) = (0 -8.123461592485908) / Math.Sqrt((0.21536310087442861 / (299)) + (0.05988497229874988 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4190050148634548 = (13.9819822895317 - 8.123461592485908) / 13.9819822895317 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.363187503991555 < 7.994240552888796.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 12.103878065046366 (T) = (0 -6.522085351519659) / Math.Sqrt((0.14913723122558575 / (299)) + (0.09427831002264547 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.24901866923956303 = (8.68475031851385 - 6.522085351519659) / 8.68475031851385 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.564321750285053 < 12.78546521735245.
IsChangePoint: Marked as a change because one of 12/12/2022 5:25:39 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 22.775964049424775 (T) = (0 -11.732260157019033) / Math.Sqrt((0.16356163265688436 / (299)) + (0.021324526733906106 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.14519182033873518 = (13.725020930038573 - 11.732260157019033) / 13.725020930038573 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.177591754229935 < 14.940989281636137.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 20.527353690926883 (T) = (0 -7.720129175544372) / Math.Sqrt((0.39730344852118754 / (299)) + (0.4196727442706728 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4998019911272146 = (15.434146155323502 - 7.720129175544372) / 15.434146155323502 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0701093680503082 < 2.972374516347359.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 17.80508028309216 (T) = (0 -0.8143844749977497) / Math.Sqrt((0.4423965133880132 / (299)) + (0.0566518562933186 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.7572758337151754 = (3.3551849717432356 - 0.8143844749977497) / 3.3551849717432356 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.558747788178243 < 14.871915676515766.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 30.067247901472793 (T) = (0 -13.496783355157875) / Math.Sqrt((0.29546334066684327 / (299)) + (0.018502443812399016 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1585653925672651 = (16.040204712208514 - 13.496783355157875) / 16.040204712208514 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0708293562868927 < 10.096609208482414.
IsChangePoint: Marked as a change because one of 12/5/2022 9:44:53 PM, 12/15/2022 5:36:17 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 57.93693747920099 (T) = (0 -1.088263967413615) / Math.Sqrt((1.2180171461397167 / (299)) + (0.08034252078007624 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.9033954614487949 = (11.265143271056386 - 1.088263967413615) / 11.265143271056386 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.45688273173366 < 57.764485136688215.
IsChangePoint: Marked as a change because one of 1/5/2023 9:29:47 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 92.44565514557183 (T) = (0 -32.48041292755694) / Math.Sqrt((19.126832036327016 / (299)) + (0.00524756424648961 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4218451402252836 = (56.17943424399 - 32.48041292755694) / 56.17943424399 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.297314802731186 < 13.015500008555168.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 72.18708618202156 (T) = (0 -8.319546943715471) / Math.Sqrt((0.07463464254191472 / (299)) + (0.016415471483979722 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.39625016666692253 = (13.779791702444633 - 8.319546943715471) / 13.779791702444633 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.133251524911342 < 14.081639006681904.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 36.76540855576465 (T) = (0 -8.25484723383601) / Math.Sqrt((0.39506957812112525 / (299)) + (0.11070590865262428 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4654554931702129 = (15.442768802906375 - 8.25484723383601) / 15.442768802906375 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.389417535021353 < 12.994733641474848.
IsChangePoint: Marked as a change because one of 12/12/2022 5:25:39 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 44.33667240573094 (T) = (0 -12.368208920099882) / Math.Sqrt((0.3214033225395826 / (299)) + (0.0011323460814460253 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1201939936895722 = (14.05788188690306 - 12.368208920099882) / 14.05788188690306 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.40396737344457 < 13.146136955082453.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 40.59120662235362 (T) = (0 -8.12248863997752) / Math.Sqrt((0.22133164663945035 / (299)) + (0.059423115018142826 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.41736924735027636 = (13.941057184224437 - 8.12248863997752) / 13.941057184224437 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.50124361365619 < 8.636156489676184.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 18.590335534956672 (T) = (0 -5.400660335857701) / Math.Sqrt((1.4626105395746272 / (299)) + (0.04998844701311537 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.3357113730752674 = (8.12999066514144 - 5.400660335857701) / 8.12999066514144 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.5182913243275635 < 7.5104360598690745.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 12.173596423647954 (T) = (0 -6.707673237992044) / Math.Sqrt((0.050690492965803045 / (299)) + (0.02884929043364374 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.15220818839579248 = (7.91193444685395 - 6.707673237992044) / 7.91193444685395 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.528802213236622 < 12.80633379473299.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 26.736336950036986 (T) = (0 -11.668870467840575) / Math.Sqrt((0.34594665036659605 / (299)) + (0.017104998738686223 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.15949084427967986 = (13.883097392128109 - 11.668870467840575) / 13.883097392128109 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1718999727819728 < 2.956250251380072.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 6:03:21 AM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 10.842099602458147 (T) = (0 -1.1538655220608587) / Math.Sqrt((0.4267348057423676 / (299)) + (0.16960125379688903 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.662973256127002 = (3.4236616026402658 - 1.1538655220608587) / 3.4236616026402658 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.290400717514725 < 14.315660961496812.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 61.846386856345234 (T) = (0 -8.198686176993856) / Math.Sqrt((0.4100340726049704 / (299)) + (0.03736090967196829 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4700461513053803 = (15.47056634684101 - 8.198686176993856) / 15.47056634684101 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SubtractBenchmark - Duration of single invocation 9.92 ns 5.97 ns 0.60 0.05 False
LessThanBenchmark - Duration of single invocation 9.54 ns 7.02 ns 0.74 0.03 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 3.18 ns 0.88 ns 0.28 0.37 False
AddBenchmark - Duration of single invocation 9.92 ns 6.14 ns 0.62 0.05 False
GreaterThanOrEqualBenchmark - Duration of single invocation 9.68 ns 6.85 ns 0.71 0.02 False
MaxBenchmark - Duration of single invocation 9.70 ns 7.32 ns 0.75 0.04 False
MinBenchmark - Duration of single invocation 9.58 ns 6.51 ns 0.68 0.03 False
BitwiseOrOperatorBenchmark - Duration of single invocation 10.69 ns 8.30 ns 0.78 0.05 False
MultiplyBenchmark - Duration of single invocation 10.35 ns 6.12 ns 0.59 0.03 False
EqualsAllBenchmark - Duration of single invocation 5.36 ns 0.82 ns 0.15 0.19 False
GreaterThanBenchmark - Duration of single invocation 9.30 ns 6.48 ns 0.70 0.03 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 10.70 ns 7.99 ns 0.75 0.11 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 3.19 ns 1.01 ns 0.31 0.41 False
ConditionalSelectBenchmark - Duration of single invocation 36.91 ns 32.56 ns 0.88 0.02 False
DivideBenchmark - Duration of single invocation 11.20 ns 6.06 ns 0.54 0.03 False
UnaryNegateOperatorBenchmark - Duration of single invocation 7.87 ns 5.76 ns 0.73 0.05 False
SubtractionOperatorBenchmark - Duration of single invocation 8.19 ns 5.97 ns 0.73 0.05 False
BitwiseAndBenchmark - Duration of single invocation 11.98 ns 8.49 ns 0.71 0.05 False
NegateBenchmark - Duration of single invocation 8.77 ns 5.78 ns 0.66 0.03 False
AndNotBenchmark - Duration of single invocation 18.46 ns 15.50 ns 0.84 0.01 False
LessThanOrEqualBenchmark - Duration of single invocation 10.06 ns 7.58 ns 0.75 0.05 False
BitwiseOrBenchmark - Duration of single invocation 12.00 ns 8.41 ns 0.70 0.05 False
EqualsStaticBenchmark - Duration of single invocation 9.28 ns 6.48 ns 0.70 0.02 False
BitwiseAndOperatorBenchmark - Duration of single invocation 10.59 ns 7.97 ns 0.75 0.07 False
XorBenchmark - Duration of single invocation 11.82 ns 8.30 ns 0.70 0.05 False
DivisionOperatorBenchmark - Duration of single invocation 9.19 ns 7.08 ns 0.77 0.03 False
LessThanOrEqualAllBenchmark - Duration of single invocation 3.71 ns 0.97 ns 0.26 0.33 False
AddOperatorBenchmark - Duration of single invocation 8.20 ns 5.90 ns 0.72 0.06 False
OneBenchmark - Duration of single invocation 8.43 ns 0.96 ns 0.11 0.18 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt64>.SubtractBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.972333676891176 < 9.326505988271029.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 100.28236452816206 (T) = (0 -6.0099667053024355) / Math.Sqrt((0.16566251687642305 / (299)) + (0.0032706439626615023 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4035587254086112 = (10.076376269264323 - 6.0099667053024355) / 10.076376269264323 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.02177063741125 < 9.085846703503424.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 55.25582560138908 (T) = (0 -7.093433322830766) / Math.Sqrt((0.4453641457686282 / (299)) + (0.003851655472313102 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.29089591231844797 = (10.00337389962462 - 7.093433322830766) / 10.00337389962462 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.8767069962054351 < 2.858036036904316.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 37.01713898224704 (T) = (0 -0.9215261369952471) / Math.Sqrt((0.3844257502322514 / (299)) + (0.009184671179608068 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.7259161878127738 = (3.3622056320705083 - 0.9215261369952471) / 3.3622056320705083 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.141638358878235 < 9.418903507067443.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 70.46666410502014 (T) = (0 -6.084844535688119) / Math.Sqrt((0.16374214114411004 / (299)) + (0.00791282805132021 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.3952551934984411 = (10.061838432129527 - 6.084844535688119) / 10.061838432129527 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.848149561193766 < 9.273768716014883.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 30.06740637834575 (T) = (0 -6.956937658231506) / Math.Sqrt((0.117714823955095 / (299)) + (0.0281497136773554 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.29939879796610935 = (9.929953928190624 - 6.956937658231506) / 9.929953928190624 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.317324684210675 < 9.257326209045976.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 10.416561279999655 (T) = (0 -6.893295686265286) / Math.Sqrt((0.33870737448997057 / (299)) + (0.2790222162249816 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.31677394706531337 = (10.08933376684957 - 6.893295686265286) / 10.08933376684957 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.507492346246569 < 9.122348530234499.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 17.999783577810785 (T) = (0 -6.838987410634459) / Math.Sqrt((0.20330818153475544 / (299)) + (0.08280644625740864 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.30681693183177317 = (9.866062407881438 - 6.838987410634459) / 9.866062407881438 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.301999050060603 < 10.125154701096697.
IsChangePoint: Marked as a change because one of 11/21/2022 10:59:14 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 22.183224743420926 (T) = (0 -8.402095022668135) / Math.Sqrt((0.1527316816678601 / (299)) + (0.03498053357627877 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.225570323172212 = (10.849397013147433 - 8.402095022668135) / 10.849397013147433 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.121600068269913 < 9.838704461278697.
IsChangePoint: Marked as a change because one of 12/16/2022 5:41:28 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 37.43800695997364 (T) = (0 -6.294396879800753) / Math.Sqrt((0.12429014765046249 / (299)) + (0.03655758851009202 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.40036579211427187 = (10.497061036584943 - 6.294396879800753) / 10.497061036584943 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.8207646900241585 < 5.182246889741126.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 17.230072351763436 (T) = (0 -1.0545655432120067) / Math.Sqrt((0.21290238857334912 / (299)) + (0.10932567790273146 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.7935916965183564 = (5.109123641945887 - 1.0545655432120067) / 5.109123641945887 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.481336509958038 < 8.888122541691907.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 64.10282520739494 (T) = (0 -6.524699549228104) / Math.Sqrt((0.1695206632228372 / (299)) + (0.005049111356446887 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.31788254184350606 = (9.56536073253704 - 6.524699549228104) / 9.56536073253704 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.99405829304276 < 10.121726821649712.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 42.05818511567272 (T) = (0 -8.041151437335447) / Math.Sqrt((0.22037716744931699 / (299)) + (0.010952209218779142 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.25731255578838913 = (10.82710028290759 - 8.041151437335447) / 10.82710028290759 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0055147031000327 < 3.0432080583244043.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 32.02622006349951 (T) = (0 -0.9309287651330407) / Math.Sqrt((0.427598827062902 / (299)) + (0.013631713111692532 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.7267029005375887 = (3.406288493234004 - 0.9309287651330407) / 3.406288493234004 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.560815982485884 < 34.54973399187194.
IsChangePoint: Marked as a change because one of 12/14/2022 12:01:27 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 54.58613846721377 (T) = (0 -32.52555097057628) / Math.Sqrt((0.34530419825020814 / (299)) + (0.00878472362459826 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0968526651268033 = (36.01356026272604 - 32.52555097057628) / 36.01356026272604 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.0625664688236265 < 10.619659722885524.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 15.550390511232745 (T) = (0 -6.489515844873712) / Math.Sqrt((0.16789040946331804 / (299)) + (0.2869192595527198 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4263500632950773 = (11.312675953822733 - 6.489515844873712) / 11.312675953822733 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.UnaryNegateOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.76139370764235 < 7.450448694487443.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 66.15891046817627 (T) = (0 -5.781140526585998) / Math.Sqrt((0.3863807716469756 / (299)) + (0.0002936070758483803 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2990707606372634 = (8.247823320713563 - 5.781140526585998) / 8.247823320713563 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.970442137166554 < 7.788823362666914.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 41.7986473571819 (T) = (0 -6.022310729605116) / Math.Sqrt((0.5881040705480494 / (299)) + (0.006478994151826217 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.30836767775449786 = (8.707387633436012 - 6.022310729605116) / 8.707387633436012 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.4902000872117 < 11.30205529333677.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 20.66993740217929 (T) = (0 -8.356419526813527) / Math.Sqrt((0.41508795757883604 / (299)) + (0.10598287081783732 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.3215576433728189 = (12.317066358233824 - 8.356419526813527) / 12.317066358233824 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.NegateBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.776847139529254 < 8.26365727279251.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 84.98851883449981 (T) = (0 -5.780599269574914) / Math.Sqrt((0.4848372930044939 / (299)) + (3.516207405073644E-05 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.3727159553623167 = (9.215281847179398 - 5.780599269574914) / 9.215281847179398 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.495161876045424 < 17.029050046207136.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/6/2023 4:18:55 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 11.709029365470423 (T) = (0 -15.696779937752444) / Math.Sqrt((0.11546116602929093 / (299)) + (0.14468970450419208 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.14124399386488146 = (18.278509641401772 - 15.696779937752444) / 18.278509641401772 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.579692802710086 < 9.466004219502048.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 15.738947502497739 (T) = (0 -7.646338811970925) / Math.Sqrt((0.28296419807609796 / (299)) + (0.08129942111832396 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.25634746893078647 = (10.282139161117522 - 7.646338811970925) / 10.282139161117522 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.410792844432114 < 11.284438446110457.
IsChangePoint: Marked as a change because one of 1/4/2023 2:39:28 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 42.87002562795658 (T) = (0 -8.319433200295249) / Math.Sqrt((0.19989198719694 / (299)) + (0.021907634673036087 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.3150986832303349 = (12.146907877960915 - 8.319433200295249) / 12.146907877960915 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.48165156655419 < 8.815436054210457.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 28.838695529074695 (T) = (0 -6.579673068404001) / Math.Sqrt((0.38631063881498184 / (299)) + (0.0318538004522996 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.32356029232614253 = (9.726917260712263 - 6.579673068404001) / 9.726917260712263 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.968676096700505 < 10.042087045310117.
IsChangePoint: Marked as a change because one of 12/5/2022 9:44:53 PM, 12/12/2022 5:25:39 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 94.7084054702864 (T) = (0 -7.96637654060853) / Math.Sqrt((0.27555243752565617 / (299)) + (4.9058019584639456E-05 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2669126991787538 = (10.866886565466542 - 7.96637654060853) / 10.866886565466542 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.304723261584813 < 11.314150984093928.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 38.75615115849654 (T) = (0 -8.102138315204577) / Math.Sqrt((0.27348335808292235 / (299)) + (0.03106841203362384 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.3367952397507269 = (12.216646804765539 - 8.102138315204577) / 12.216646804765539 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.08177259833884 < 8.747327051155402.
IsChangePoint: Marked as a change because one of 12/4/2022 6:53:17 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 20.727692957228466 (T) = (0 -7.271863037010365) / Math.Sqrt((0.5076046014953524 / (299)) + (0.03574385618260312 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.24956513941900113 = (9.690198868667123 - 7.271863037010365) / 9.690198868667123 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.9656453078202927 < 3.6495412812275614.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 6.914730654775034 (T) = (0 -1.6804588703282899) / Math.Sqrt((0.5831808485252395 / (299)) + (0.4114812538170322 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.6054784123514995 = (4.2594852168786685 - 1.6804588703282899) / 4.2594852168786685 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.9018656284335105 < 7.786876371485339.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 46.41421291305603 (T) = (0 -5.906598748309539) / Math.Sqrt((0.6087087445572441 / (299)) + (0.004713906473207064 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.3206279272116936 = (8.694203051455201 - 5.906598748309539) / 8.694203051455201 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.9632505516835419 < 8.008144857272212.
IsChangePoint: Marked as a change because one of 12/5/2022 9:44:53 PM, 12/14/2022 8:02:51 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 157.80977398344683 (T) = (0 -0.9630778845780998) / Math.Sqrt((0.7569403225079809 / (299)) + (0.00028748975264941906 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.8936075074951275 = (9.052122587822572 - 0.9630778845780998) / 9.052122587822572 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GreaterThanOrEqualAllBenchmark - Duration of single invocation 3.29 ns 0.82 ns 0.25 0.35 False
BitwiseAndBenchmark - Duration of single invocation 16.12 ns 8.60 ns 0.53 0.30 False
OnesComplementBenchmark - Duration of single invocation 8.49 ns 6.33 ns 0.75 0.07 False
BitwiseAndOperatorBenchmark - Duration of single invocation 13.60 ns 8.31 ns 0.61 0.03 False
DivisionOperatorBenchmark - Duration of single invocation 14.21 ns 12.78 ns 0.90 0.02 False
DivideBenchmark - Duration of single invocation 16.21 ns 12.27 ns 0.76 0.02 False
BitwiseOrOperatorBenchmark - Duration of single invocation 13.88 ns 7.97 ns 0.57 0.03 False
MultiplyOperatorBenchmark - Duration of single invocation 8.34 ns 6.31 ns 0.76 0.02 False
SubtractBenchmark - Duration of single invocation 9.73 ns 5.79 ns 0.60 0.02 False
OnesComplementOperatorBenchmark - Duration of single invocation 8.43 ns 6.87 ns 0.82 0.07 False
MultiplyBenchmark - Duration of single invocation 10.44 ns 6.21 ns 0.59 0.04 False
EqualsBenchmark - Duration of single invocation 9.25 ns 5.97 ns 0.65 0.08 False
SubtractionOperatorBenchmark - Duration of single invocation 8.18 ns 5.98 ns 0.73 0.05 False
LessThanAllBenchmark - Duration of single invocation 3.77 ns 1.49 ns 0.40 0.35 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 3.19 ns 1.08 ns 0.34 0.38 False
LessThanBenchmark - Duration of single invocation 9.51 ns 6.93 ns 0.73 0.03 False
BitwiseOrBenchmark - Duration of single invocation 14.80 ns 8.14 ns 0.55 0.25 False
AddOperatorBenchmark - Duration of single invocation 8.21 ns 5.98 ns 0.73 0.04 False
OneBenchmark - Duration of single invocation 8.61 ns 0.94 ns 0.11 0.15 False
MaxBenchmark - Duration of single invocation 9.77 ns 7.20 ns 0.74 0.04 False
NegateBenchmark - Duration of single invocation 8.71 ns 5.78 ns 0.66 0.02 False
EqualsAllBenchmark - Duration of single invocation 5.50 ns 0.70 ns 0.13 0.16 False
GreaterThanOrEqualBenchmark - Duration of single invocation 9.82 ns 7.23 ns 0.74 0.04 False
GreaterThanBenchmark - Duration of single invocation 9.35 ns 6.46 ns 0.69 0.02 False
LessThanOrEqualBenchmark - Duration of single invocation 10.03 ns 7.79 ns 0.78 0.03 False
XorBenchmark - Duration of single invocation 14.80 ns 8.31 ns 0.56 0.28 False
AddBenchmark - Duration of single invocation 9.89 ns 6.14 ns 0.62 0.05 False
ConditionalSelectBenchmark - Duration of single invocation 54.36 ns 32.51 ns 0.60 0.28 False
UnaryNegateOperatorBenchmark - Duration of single invocation 7.84 ns 5.77 ns 0.74 0.04 False
AndNotBenchmark - Duration of single invocation 24.00 ns 15.51 ns 0.65 0.01 False
MinBenchmark - Duration of single invocation 8.51 ns 7.04 ns 0.83 0.03 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 13.86 ns 8.44 ns 0.61 0.02 False
EqualsStaticBenchmark - Duration of single invocation 9.31 ns 6.47 ns 0.70 0.04 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int64>.GreaterThanOrEqualAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.8212503217622753 < 3.02660316157282.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 37.86748874270484 (T) = (0 -0.8394656232003731) / Math.Sqrt((0.38480896278800736 / (299)) + (0.009683302150638382 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.7519203202029575 = (3.3838548319924953 - 0.8394656232003731) / 3.3838548319924953 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.601186166942302 < 14.701068877542204.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 90.25673659768087 (T) = (0 -8.469305975073576) / Math.Sqrt((0.4369262828485494 / (299)) + (0.013409971950440514 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.45077305391883055 = (15.420412336837366 - 8.469305975073576) / 15.420412336837366 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.330485668219313 < 8.026155891360991.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 13.280181429135979 (T) = (0 -6.516011651628439) / Math.Sqrt((0.5481865937099304 / (299)) + (0.09131621572637327 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2680055632190627 = (8.901722915113458 - 6.516011651628439) / 8.901722915113458 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.309740615829009 < 13.024668843490442.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 53.336045194138045 (T) = (0 -8.193942552634896) / Math.Sqrt((0.21472085121953885 / (299)) + (0.03293873842993453 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.41314869446570807 = (13.96255316357321 - 8.193942552634896) / 13.96255316357321 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.778585181155181 < 13.67711332177221.
IsChangePoint: Marked as a change because one of 12/5/2022 11:02:29 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 8.960614892145154 (T) = (0 -12.571369720547459) / Math.Sqrt((0.047483323714288976 / (299)) + (0.1386648430145494 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.13307702185309234 = (14.501137975855025 - 12.571369720547459) / 14.501137975855025 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.266060735927251 < 15.390249818312864.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 18.60063490201575 (T) = (0 -12.643481041755196) / Math.Sqrt((0.0186389667965117 / (299)) + (0.10808077859063618 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.21843224604839406 = (16.177076111226143 - 12.643481041755196) / 16.177076111226143 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.970837880823529 < 13.02251482294688.
IsChangePoint: Marked as a change because one of 1/5/2023 9:29:47 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 31.005983999769555 (T) = (0 -8.291093938441044) / Math.Sqrt((0.06300741091237684 / (299)) + (0.09380811829145846 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.39886379607863887 = (13.792371652807091 - 8.291093938441044) / 13.792371652807091 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.313847125971553 < 7.972051105342866.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 56.46324500114742 (T) = (0 -6.330512408933294) / Math.Sqrt((0.3677647800422042 / (299)) + (0.001875741316948112 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2775464777470031 = (8.762518575854356 - 6.330512408933294) / 8.762518575854356 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.7920715479695986 < 9.259888295199845.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 56.82874544052393 (T) = (0 -5.8429793564535055) / Math.Sqrt((0.36270342855500864 / (299)) + (0.01436106103117446 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4296718345968828 = (10.2449426679175 - 5.8429793564535055) / 10.2449426679175 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.870644925257441 < 8.046052730699605.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 11.972093975290694 (T) = (0 -6.680889768513144) / Math.Sqrt((0.4500166381280182 / (299)) + (0.09623452086973236 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2472121224090111 = (8.874863646705881 - 6.680889768513144) / 8.874863646705881 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.209652993318761 < 9.842862682850694.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 99.82320380113867 (T) = (0 -6.280423339829295) / Math.Sqrt((0.1467880154574921 / (299)) + (0.0038203238963102635 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4003475458661045 = (10.473438900371695 - 6.280423339829295) / 10.473438900371695 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.967531708044415 < 8.650071102241624.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 12.379449907339703 (T) = (0 -5.66919727141449) / Math.Sqrt((1.4444678624436835 / (299)) + (0.10291443108809081 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.3016678162156289 = (8.118195613858472 - 5.66919727141449) / 8.118195613858472 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.975550847649788 < 7.778556176270914.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 80.0864717544539 (T) = (0 -5.990025032595459) / Math.Sqrt((0.2900481255481697 / (299)) + (0.00015871480817544991 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2995349483735119 = (8.551497349777195 - 5.990025032595459) / 8.551497349777195 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.4946492659431165 < 3.594152707848558.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 38.86105654963525 (T) = (0 -1.574869893529028) / Math.Sqrt((0.5044753151099182 / (299)) + (0.005089998056132145 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.5893921183959526 = (3.8354594835753506 - 1.574869893529028) / 3.8354594835753506 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0798254843743906 < 3.0069901606575637.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 27.017973337840033 (T) = (0 -0.9285140455405196) / Math.Sqrt((0.401707419781512 / (299)) + (0.021535970917527337 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.728718280228286 = (3.4226930082936384 - 0.9285140455405196) / 3.4226930082936384 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.925624166823431 < 9.101487163351354.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 15.433135179580132 (T) = (0 -7.271771968264055) / Math.Sqrt((0.4646429195068487 / (299)) + (0.08986529213900214 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2736419025183754 = (10.01127679787175 - 7.271771968264055) / 10.01127679787175 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.144476070454457 < 14.06132610774357.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 46.972081776709416 (T) = (0 -8.2930625978285) / Math.Sqrt((0.4059001625504225 / (299)) + (0.04415591223415445 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.4644058216290029 = (15.483855002031099 - 8.2930625978285) / 15.483855002031099 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.977462754092701 < 7.829753516713754.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 8.286132869936395 (T) = (0 -5.704481949429244) / Math.Sqrt((0.28355653919928275 / (299)) + (0.34982190350485914 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.33245761443708316 = (8.545497743366273 - 5.704481949429244) / 8.545497743366273 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.9420575861453114 < 8.185415195052265.
IsChangePoint: Marked as a change because one of 12/5/2022 9:44:53 PM, 12/15/2022 5:36:17 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 116.10601947579227 (T) = (0 -0.9043537355455803) / Math.Sqrt((0.9521099648760304 / (299)) + (0.005181369925512106 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.8999747570629488 = (9.041255077128053 - 0.9043537355455803) / 9.041255077128053 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.200672878544046 < 9.31968828054195.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 28.486877592427238 (T) = (0 -7.226164145536488) / Math.Sqrt((0.3613027553938422 / (299)) + (0.027668645898800373 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.28705319292366277 = (10.13562873669306 - 7.226164145536488) / 10.13562873669306 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.NegateBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.78105856674589 < 8.259231957403896.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 15.287055959652996 (T) = (0 -5.563463710608539) / Math.Sqrt((0.18700138865529672 / (299)) + (0.15408626237584627 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.3851867412646667 = (9.049030142994225 - 5.563463710608539) / 9.049030142994225 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.6980026648552619 < 5.231102486251142.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 157.4204256197349 (T) = (0 -0.6821055904341812) / Math.Sqrt((0.19544370587286236 / (299)) + (0.0004172331837054001 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.8666294912687886 = (5.114365963834364 - 0.6821055904341812) / 5.114365963834364 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.231205764963141 < 9.276884161834214.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 21.35925490450336 (T) = (0 -7.13155641005729) / Math.Sqrt((0.3831003281315527 / (299)) + (0.05385283732050238 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2934620537363381 = (10.09366368469044 - 7.13155641005729) / 10.09366368469044 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.45834661263986 < 8.875144042855243.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 25.506457219650677 (T) = (0 -6.685270328016241) / Math.Sqrt((0.3730956133966779 / (299)) + (0.038620934305363605 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.3119529247222819 = (9.716297864238213 - 6.685270328016241) / 9.716297864238213 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.785107150159234 < 9.489826804704569.
IsChangePoint: Marked as a change because one of 12/14/2022 12:01:27 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 65.47414188555796 (T) = (0 -7.78600263852402) / Math.Sqrt((0.4097392663099842 / (299)) + (0.00048132967219445365 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.24756305603084702 = (10.347714450931075 - 7.78600263852402) / 10.347714450931075 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.313863916566572 < 14.068610035764388.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 63.94839907396215 (T) = (0 -8.202596133662267) / Math.Sqrt((0.4210547868111211 / (299)) + (0.03489292215266747 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4709646137563321 = (15.504815645515706 - 8.202596133662267) / 15.504815645515706 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.136768305114402 < 9.405415900188403.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 66.4008093510188 (T) = (0 -6.027132267853251) / Math.Sqrt((0.23021786917632905 / (299)) + (0.009019806902159524 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4037094254850062 = (10.107710108876956 - 6.027132267853251) / 10.107710108876956 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.51275006552931 < 55.17448891532883.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 100.04442349841891 (T) = (0 -32.474433279240735) / Math.Sqrt((16.627403900556736 / (299)) + (0.0016208253792098135 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.42196788404309105 = (56.18101898279582 - 32.474433279240735) / 56.18101898279582 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.UnaryNegateOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.774696965539276 < 7.450990965173961.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 54.64938600515658 (T) = (0 -5.768017465292703) / Math.Sqrt((0.6884232382926238 / (299)) + (8.567266067571026E-05 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.31386219532541326 = (8.406500015005426 - 5.768017465292703) / 8.406500015005426 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.508038159291317 < 22.820756644308855.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 203.24569365861774 (T) = (0 -15.44447595629562) / Math.Sqrt((0.08215528819878352 / (299)) + (0.004503948949717496 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.3567476950597541 = (24.009981523082633 - 15.44447595629562) / 24.009981523082633 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.037533310630753 < 9.122923222610137.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 28.448373325736114 (T) = (0 -7.2130858923679915) / Math.Sqrt((0.2099502652413178 / (299)) + (0.023909574179267434 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.268620691037765 = (9.862305104861042 - 7.2130858923679915) / 9.862305104861042 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.44268372728044 < 13.136708040288328.
IsChangePoint: Marked as a change because one of 1/5/2023 9:23:01 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 250.5888427421639 (T) = (0 -8.431311080887177) / Math.Sqrt((0.08137210996318067 / (299)) + (0.0005712700784478669 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.38995672144603477 = (13.820840876851546 - 8.431311080887177) / 13.820840876851546 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.4687996874886915 < 8.83484089795367.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 65.7438294819507 (T) = (0 -6.521739530763209) / Math.Sqrt((0.1833211732888066 / (299)) + (0.0045404963110730456 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.31734769455055456 = (9.553530367804761 - 6.521739530763209) / 9.553530367804761 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 8, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Int

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LessThanBenchmark - Duration of single invocation 19.64 ns 17.78 ns 0.91 0.04 False
MaxBenchmark - Duration of single invocation 20.71 ns 18.96 ns 0.92 0.12 False
EqualsStaticBenchmark - Duration of single invocation 19.64 ns 18.00 ns 0.92 0.04 False
LessThanOrEqualBenchmark - Duration of single invocation 20.62 ns 18.61 ns 0.90 0.02 False
MinBenchmark - Duration of single invocation 20.76 ns 19.13 ns 0.92 0.08 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Int* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Int* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.782474313768073 < 18.67977731668726.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 22.111094619529304 (T) = (0 -17.856197017948926) / Math.Sqrt((0.1791937779738052 / (299)) + (0.02264758460533438 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10054067051589012 = (19.85214498602227 - 17.856197017948926) / 19.85214498602227 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.96206051106812 < 19.691037418542432.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 17.527190820563913 (T) = (0 -19.066175135283583) / Math.Sqrt((0.39221394648981783 / (299)) + (0.03310074509678312 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09267508561305503 = (21.013613572119404 - 19.066175135283583) / 21.013613572119404 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.00169235772511 < 18.665413159853465.
IsChangePoint: Marked as a change because one of 11/22/2022 11:24:47 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 19.072741159855642 (T) = (0 -17.973255899080772) / Math.Sqrt((0.09576610484877666 / (299)) + (0.023297996536338034 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08711208084725121 = (19.688348944043152 - 17.973255899080772) / 19.688348944043152 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.613242453895193 < 19.55944284946102.
IsChangePoint: Marked as a change because one of 12/28/2022 6:20:56 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 22.035115073366875 (T) = (0 -18.679244295496655) / Math.Sqrt((0.14352672563681856 / (299)) + (0.021621200120268903 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09373312326736769 = (20.611196078180647 - 18.679244295496655) / 20.611196078180647 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.131961774720864 < 19.770810261289245.
IsChangePoint: Marked as a change because one of 12/8/2022 6:01:13 PM, 12/9/2022 1:11:26 PM, 12/15/2022 1:21:30 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 15.506955155395847 (T) = (0 -19.33060362254264) / Math.Sqrt((0.32686827679218633 / (299)) + (0.030618022779915946 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07857180019311372 = (20.978958128906804 - 19.33060362254264) / 20.978958128906804 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OnesComplementBenchmark - Duration of single invocation 8.42 ns 6.88 ns 0.82 0.07 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 12.82 ns 8.32 ns 0.65 0.02 False
MultiplyBenchmark - Duration of single invocation 43.88 ns 41.31 ns 0.94 0.01 False
XorBenchmark - Duration of single invocation 16.33 ns 8.60 ns 0.53 0.28 False
OnesComplementOperatorBenchmark - Duration of single invocation 8.45 ns 6.34 ns 0.75 0.04 False
EqualsAllBenchmark - Duration of single invocation 5.60 ns 0.83 ns 0.15 0.24 False
BitwiseOrOperatorBenchmark - Duration of single invocation 13.57 ns 7.99 ns 0.59 0.03 False
BitwiseAndOperatorBenchmark - Duration of single invocation 13.80 ns 8.26 ns 0.60 0.02 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 3.45 ns 1.06 ns 0.31 0.43 False
BitwiseOrBenchmark - Duration of single invocation 16.29 ns 7.99 ns 0.49 0.28 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 3.56 ns 1.36 ns 0.38 0.49 False
EqualsBenchmark - Duration of single invocation 8.78 ns 6.18 ns 0.70 0.07 False
OneBenchmark - Duration of single invocation 22.82 ns 0.96 ns 0.04 0.08 False
AndNotBenchmark - Duration of single invocation 23.95 ns 15.32 ns 0.64 0.01 False
ConditionalSelectBenchmark - Duration of single invocation 54.41 ns 32.69 ns 0.60 0.34 False
BitwiseAndBenchmark - Duration of single invocation 16.18 ns 8.46 ns 0.52 0.27 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Byte>.OnesComplementBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.880266123380068 < 8.016977115051843.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 6.596868339219565 (T) = (0 -6.547748581363973) / Math.Sqrt((0.24884600554186734 / (299)) + (0.328493980137505 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2507408899650722 = (8.738964256382202 - 6.547748581363973) / 8.738964256382202 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.321673502473034 < 13.15062191007443.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 95.50635765546788 (T) = (0 -8.37139718512631) / Math.Sqrt((0.0503270023630336 / (299)) + (0.009114244477402765 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.3924724131962877 = (13.779451940889471 - 8.37139718512631) / 13.779451940889471 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 41.31332271827614 < 41.82955970774298.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 11.007598044661908 (T) = (0 -40.92972632744244) / Math.Sqrt((0.6288358430021727 / (299)) + (0.15372625991146743 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.05848299827002449 = (43.47210539186946 - 40.92972632744244) / 43.47210539186946 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.600160982286067 < 14.086975254140526.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 40.173264764944655 (T) = (0 -8.301461940845527) / Math.Sqrt((0.4076324081717099 / (299)) + (0.09216187817168622 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.46432828700550055 = (15.497293845215177 - 8.301461940845527) / 15.497293845215177 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.335103692074997 < 8.029520699197413.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 12.383050563868856 (T) = (0 -6.516346180485406) / Math.Sqrt((0.22055913735417937 / (299)) + (0.09440748312866118 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.25430663005348375 = (8.738640362261476 - 6.516346180485406) / 8.738640362261476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.8314075287787189 < 5.380569038278472.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 43.56267176945964 (T) = (0 -0.8813397371828571) / Math.Sqrt((0.1888393070288684 / (299)) + (0.02858107169491403 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.8328267713418361 = (5.272014809171525 - 0.8813397371828571) / 5.272014809171525 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.991562050566451 < 12.87351621236921.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 37.28509144437641 (T) = (0 -8.137869190192744) / Math.Sqrt((0.20867144876460036 / (299)) + (0.07080722084477697 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4166433873139109 = (13.950076185339865 - 8.137869190192744) / 13.950076185339865 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.264084544922776 < 12.848176507966512.
IsChangePoint: Marked as a change because one of 12/18/2022 12:43:31 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 68.0071871821555 (T) = (0 -8.130697794264613) / Math.Sqrt((0.07630438709303498 / (299)) + (0.019967015896170448 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.41014567793755524 = (13.784247211812172 - 8.130697794264613) / 13.784247211812172 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.060607185670292 < 3.308291351878544.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 19.334227785865913 (T) = (0 -1.0348520148710187) / Math.Sqrt((0.33744062585462203 / (299)) + (0.057380581926814764 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.7267017051394113 = (3.786529350279714 - 1.0348520148710187) / 3.786529350279714 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.986556482820568 < 15.364866329370244.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 83.75920536239728 (T) = (0 -8.132248830637147) / Math.Sqrt((0.40618299156407317 / (299)) + (0.01904336996906794 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.47483317005838527 = (15.485077059305606 - 8.132248830637147) / 15.485077059305606 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3624291105543151 < 3.2623355635749003.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 25.244133748379454 (T) = (0 -1.2284601862960207) / Math.Sqrt((0.3576835472029317 / (299)) + (0.0261601779611739 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.6717356974053444 = (3.7422899065968096 - 1.2284601862960207) / 3.7422899065968096 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.18284766836157 < 8.46725077043316.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 12.682499710674714 (T) = (0 -6.358621423572829) / Math.Sqrt((0.8322069196096854 / (299)) + (0.04910310575540058 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2163120914891504 = (8.113716384441062 - 6.358621423572829) / 8.113716384441062 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.959817577659075 < 21.207274487239403.
IsChangePoint: Marked as a change because one of 12/5/2022 9:44:53 PM, 12/15/2022 5:36:17 AM, 1/11/2023 1:25:06 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 145.82773423609262 (T) = (0 -0.9547535689223046) / Math.Sqrt((7.8015382148980095 / (299)) + (4.343370132448951E-05 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.9610573514677383 = (24.516914100779378 - 0.9547535689223046) / 24.516914100779378 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.318411841627098 < 22.98863541247406.
IsChangePoint: Marked as a change because one of 1/5/2023 11:12:07 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 64.2312075179756 (T) = (0 -15.527024640123521) / Math.Sqrt((0.12771128429019277 / (299)) + (0.05057382227758206 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.3522761752110552 = (23.97167441723001 - 15.527024640123521) / 23.97167441723001 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.68990685127432 < 51.711264007892126.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 100.4848404378651 (T) = (0 -32.594138046027275) / Math.Sqrt((15.996312266715092 / (299)) + (0.007400499477803096 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.42174131142739907 = (56.366015228381734 - 32.594138046027275) / 56.366015228381734 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.459979399801977 < 14.087231078215876.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 67.20079664348582 (T) = (0 -8.445476958240947) / Math.Sqrt((0.4437094986024355 / (299)) + (0.02890563963904741 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4562404243280116 = (15.53163812849432 - 8.445476958240947) / 15.53163812849432 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 8, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultiplyOperatorBenchmark - Duration of single invocation 56.91 ns 51.40 ns 0.90 0.00 False
BitwiseOrBenchmark - Duration of single invocation 15.71 ns 13.03 ns 0.83 0.02 False
SubtractionOperatorBenchmark - Duration of single invocation 55.55 ns 49.28 ns 0.89 0.01 False
ConditionalSelectBenchmark - Duration of single invocation 16.52 ns 15.17 ns 0.92 0.04 False
GreaterThanAllBenchmark - Duration of single invocation 39.53 ns 34.90 ns 0.88 0.02 False
SubtractBenchmark - Duration of single invocation 55.50 ns 49.74 ns 0.90 0.01 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 42.95 ns 38.85 ns 0.90 0.01 False
EqualsAllBenchmark - Duration of single invocation 9.25 ns 5.56 ns 0.60 0.25 False
BitwiseAndOperatorBenchmark - Duration of single invocation 14.62 ns 12.92 ns 0.88 0.02 False
BitwiseOrOperatorBenchmark - Duration of single invocation 14.25 ns 13.02 ns 0.91 0.03 False
LessThanAnyBenchmark - Duration of single invocation 39.51 ns 34.53 ns 0.87 0.02 False
MultiplyBenchmark - Duration of single invocation 56.35 ns 51.38 ns 0.91 0.00 False
DotBenchmark - Duration of single invocation 44.49 ns 39.73 ns 0.89 0.01 False
NegateBenchmark - Duration of single invocation 50.12 ns 46.74 ns 0.93 0.01 False
XorBenchmark - Duration of single invocation 15.56 ns 12.85 ns 0.83 0.02 False
EqualsStaticBenchmark - Duration of single invocation 59.67 ns 53.87 ns 0.90 0.01 False
AddOperatorBenchmark - Duration of single invocation 54.74 ns 49.52 ns 0.90 0.01 False
LessThanBenchmark - Duration of single invocation 59.17 ns 54.05 ns 0.91 0.01 False
AddBenchmark - Duration of single invocation 54.93 ns 49.24 ns 0.90 0.00 False
LessThanOrEqualBenchmark - Duration of single invocation 64.48 ns 60.33 ns 0.94 0.01 False
DivideBenchmark - Duration of single invocation 105.65 ns 98.12 ns 0.93 0.00 False
BitwiseAndBenchmark - Duration of single invocation 15.72 ns 13.33 ns 0.85 0.03 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 14.24 ns 12.95 ns 0.91 0.01 False
EqualsAnyBenchmark - Duration of single invocation 39.44 ns 34.66 ns 0.88 0.01 False
AndNotBenchmark - Duration of single invocation 14.56 ns 13.26 ns 0.91 0.05 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.MultiplyOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.402836859443546 < 54.04905671090806.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 20.74915115953266 (T) = (0 -51.24688612866936) / Math.Sqrt((2.349152109127683 / (299)) + (0.0951426472321006 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07453827585437287 = (55.374398304781046 - 51.24688612866936) / 55.374398304781046 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.028271251340856 < 14.817958317846857.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 53.887965892089326 (T) = (0 -13.023441129883198) / Math.Sqrt((0.4490599402125833 / (299)) + (0.004493203908467191 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.18475159672775077 = (15.974813415898304 - 13.023441129883198) / 15.974813415898304 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.28244867614165 < 52.12763636138445.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 34.91190228767449 (T) = (0 -49.45227740025584) / Math.Sqrt((2.4967198382481457 / (299)) + (0.028424254832747808 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08613575980754117 = (54.113373984128366 - 49.45227740025584) / 54.113373984128366 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.174006503229991 < 15.807776262424559.
IsChangePoint: Marked as a change because one of 11/11/2022 1:18:55 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 15.468073780769407 (T) = (0 -15.347822321114966) / Math.Sqrt((0.10636874185865727 / (299)) + (0.025694426674608636 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08691535394348687 = (16.80876180253397 - 15.347822321114966) / 16.80876180253397 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.899573619795255 < 37.39367546612975.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 18.98152110811817 (T) = (0 -35.01795102902987) / Math.Sqrt((3.8525215538475135 / (299)) + (0.05578083290624499 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08773390125707281 = (38.385676150065706 - 35.01795102902987) / 38.385676150065706 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.740427840670854 < 52.73024517009423.
IsChangePoint: Marked as a change because one of 1/2/2023 5:13:32 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 166.89495300044226 (T) = (0 -49.74544235240787) / Math.Sqrt((0.30179785375241763 / (299)) + (0.00017171172037285456 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09874954349146349 = (55.19602458247042 - 49.74544235240787) / 55.19602458247042 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.8459089890242 < 41.22315978629139.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 27.926464119934103 (T) = (0 -38.81264820329287) / Math.Sqrt((3.889539524817637 / (299)) + (0.001457494594421483 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0771358111689418 = (42.056728035416285 - 38.81264820329287) / 42.056728035416285 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.560105977458371 < 8.328096350585843.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 10.918405492606587 (T) = (0 -5.20571216832749) / Math.Sqrt((0.23789188347025778 / (299)) + (0.2191203948624176 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.3630240873470456 = (8.172541637636044 - 5.20571216832749) / 8.172541637636044 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.921450090433702 < 13.561337064417268.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 16.08885509361225 (T) = (0 -13.088660704783448) / Math.Sqrt((0.27943364897766504 / (299)) + (0.02570573105109835 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10700684772865816 = (14.657067270328152 - 13.088660704783448) / 14.657067270328152 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.02455623363535 < 13.666625746897525.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 21.116182901871696 (T) = (0 -13.027325816962922) / Math.Sqrt((0.2983819501310992 / (299)) + (0.014938254134756019 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11136254179446115 = (14.659888233014081 - 13.027325816962922) / 14.659888233014081 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.53306124479446 < 37.55697455652957.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 15.655597797717634 (T) = (0 -34.58217935423292) / Math.Sqrt((4.148615436516157 / (299)) + (0.12936163335902684 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09753652163480037 = (38.31975496324579 - 34.58217935423292) / 38.31975496324579 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.38253077982813 < 53.59408684259102.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 22.39560485737969 (T) = (0 -51.405373528335616) / Math.Sqrt((0.13400621264556625 / (299)) + (0.1586708568274695 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09141933007924193 = (56.57766583656126 - 51.405373528335616) / 56.57766583656126 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 39.7303481357238 < 42.298545157936765.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 18.311732663600033 (T) = (0 -39.41167295252968) / Math.Sqrt((2.778173569590402 / (299)) + (0.16471638608854053 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10532409131996269 = (44.051340345886594 - 39.41167295252968) / 44.051340345886594 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.NegateBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 46.73730494168883 < 47.56535727464198.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 8.390725536251283 (T) = (0 -46.43097852395734) / Math.Sqrt((2.086995068132093 / (299)) + (0.3040888989093564 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.05614329996549946 = (49.19282611678252 - 46.43097852395734) / 49.19282611678252 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.851704134289092 < 14.783888123388692.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 33.94551467442452 (T) = (0 -12.994801767536023) / Math.Sqrt((0.42391122708295464 / (299)) + (0.018532362738758364 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.185440872604834 = (15.953171882182927 - 12.994801767536023) / 15.953171882182927 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 53.87202561454994 < 56.48126423024588.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 8.105207155562631 (T) = (0 -53.381045204471924) / Math.Sqrt((2.02033124232193 / (299)) + (1.1796967799588913 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08761515854045938 = (58.50715923675184 - 53.381045204471924) / 58.50715923675184 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.519826282933785 < 52.789667850962424.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 19.059090578239626 (T) = (0 -49.8555529220917) / Math.Sqrt((2.435422767017126 / (299)) + (0.1287472570111088 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07951514459090439 = (54.16227396803193 - 49.8555529220917) / 54.16227396803193 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 54.04998994412582 < 56.49150699614336.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 28.537817421021643 (T) = (0 -53.80274067358084) / Math.Sqrt((2.720851015511354 / (299)) + (0.04783981754401464 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07744306854402451 = (58.319154990976635 - 53.80274067358084) / 58.319154990976635 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.24358326470935 < 52.45351297985051.
IsChangePoint: Marked as a change because one of 12/30/2022 6:18:01 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 19.351264730553382 (T) = (0 -49.75376379192432) / Math.Sqrt((0.21946259038445767 / (299)) + (0.23346357914491783 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0982957926063797 = (55.17747769608149 - 49.75376379192432) / 55.17747769608149 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 60.327624554711996 < 61.3023418457264.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 10.0293504266196 (T) = (0 -59.690184027464) / Math.Sqrt((1.980824211997406 / (299)) + (0.38667608523857067 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.058250789462605305 = (63.382250135790095 - 59.690184027464) / 63.382250135790095 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 98.12464000948194 < 100.4500144794014.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 73.20268826487982 (T) = (0 -98.09970345561992) / Math.Sqrt((1.5259696254679893 / (299)) + (0.008682585702627466 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.06255873085538956 = (104.64623938002347 - 98.09970345561992) / 104.64623938002347 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.325838258059726 < 14.71105240940177.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 19.653309174312206 (T) = (0 -13.050738012244489) / Math.Sqrt((0.20642878074336893 / (299)) + (0.056762619288598035 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1741603233891472 = (15.802992253657703 - 13.050738012244489) / 15.802992253657703 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.947165165399326 < 13.577141885267165.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 21.107836851667923 (T) = (0 -12.916187932077806) / Math.Sqrt((0.09771400149863856 / (299)) + (0.01628285713784548 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11029509558611607 = (14.517384211326426 - 12.916187932077806) / 14.517384211326426 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.65913848902013 < 37.50053925426425.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 15.033875399725547 (T) = (0 -34.72642395648626) / Math.Sqrt((4.199520380629903 / (299)) + (0.1283170667194989 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09354068457855076 = (38.3099642374358 - 34.72642395648626) / 38.3099642374358 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.25613760530678 < 13.832509190734259.
IsChangePoint: Marked as a change because one of 11/17/2022 5:10:53 PM, 11/21/2022 12:46:21 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 24.584512181805394 (T) = (0 -13.184418270970161) / Math.Sqrt((0.10646512873014 / (299)) + (0.009551548266204725 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09986328084448938 = (14.647128586576832 - 13.184418270970161) / 14.647128586576832 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 8, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultiplyBenchmark - Duration of single invocation 20.53 ns 17.02 ns 0.83 0.03 False
BitwiseAndOperatorBenchmark - Duration of single invocation 14.42 ns 12.93 ns 0.90 0.02 False
MinBenchmark - Duration of single invocation 20.91 ns 19.57 ns 0.94 0.02 False
EqualsAllBenchmark - Duration of single invocation 7.92 ns 3.58 ns 0.45 0.28 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 12.30 ns 10.50 ns 0.85 0.04 False
LessThanAnyBenchmark - Duration of single invocation 11.01 ns 9.53 ns 0.87 0.07 False
EqualsBenchmark - Duration of single invocation 10.70 ns 8.31 ns 0.78 0.06 False
XorBenchmark - Duration of single invocation 15.55 ns 12.89 ns 0.83 0.04 False
ConditionalSelectBenchmark - Duration of single invocation 16.87 ns 15.41 ns 0.91 0.04 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 14.34 ns 13.15 ns 0.92 0.04 False
DivisionOperatorBenchmark - Duration of single invocation 21.63 ns 19.90 ns 0.92 0.02 False
SubtractBenchmark - Duration of single invocation 20.20 ns 16.86 ns 0.83 0.03 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 12.21 ns 10.00 ns 0.82 0.06 False
LessThanOrEqualBenchmark - Duration of single invocation 20.49 ns 18.70 ns 0.91 0.02 False
MultiplyOperatorBenchmark - Duration of single invocation 18.84 ns 17.00 ns 0.90 0.01 False
GreaterThanOrEqualBenchmark - Duration of single invocation 21.02 ns 19.49 ns 0.93 0.01 False
DotBenchmark - Duration of single invocation 10.39 ns 8.40 ns 0.81 0.05 False
BitwiseOrBenchmark - Duration of single invocation 15.58 ns 13.09 ns 0.84 0.03 False
BitwiseAndBenchmark - Duration of single invocation 15.58 ns 13.13 ns 0.84 0.02 False
LessThanBenchmark - Duration of single invocation 19.71 ns 17.72 ns 0.90 0.02 False
DivideBenchmark - Duration of single invocation 23.65 ns 19.90 ns 0.84 0.01 False
GreaterThanAllBenchmark - Duration of single invocation 11.08 ns 9.49 ns 0.86 0.06 False
AddBenchmark - Duration of single invocation 20.14 ns 16.76 ns 0.83 0.02 False
EqualsStaticBenchmark - Duration of single invocation 19.68 ns 17.93 ns 0.91 0.00 False
BitwiseOrOperatorBenchmark - Duration of single invocation 14.32 ns 12.79 ns 0.89 0.03 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.MultiplyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.016127711959218 < 19.48785942102889.
IsChangePoint: Marked as a change because one of 12/4/2022 6:53:17 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 42.05803702461491 (T) = (0 -16.954369231716715) / Math.Sqrt((0.07553101862960669 / (299)) + (0.022054530625992438 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.17784633036641773 = (20.621898141344968 - 16.954369231716715) / 20.621898141344968 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.934646871384233 < 13.616516524172553.
IsChangePoint: Marked as a change because one of 12/13/2022 1:36:02 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 45.82162137606805 (T) = (0 -12.973697618012253) / Math.Sqrt((0.25094603158683393 / (299)) + (0.0014202640901112766 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1134474776752994 = (14.63387367506764 - 12.973697618012253) / 14.63387367506764 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.5676857390653 < 19.827714482054727.
IsChangePoint: Marked as a change because one of 12/5/2022 9:44:53 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 12.917196573157826 (T) = (0 -19.378731460828934) / Math.Sqrt((0.5013179057377266 / (299)) + (0.06386941704692282 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0917479339550817 = (21.3362921872732 - 19.378731460828934) / 21.3362921872732 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.578907468388801 < 7.558387941578315.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 6.6079569004385865 (T) = (0 -4.464710699822943) / Math.Sqrt((0.2966854813913855 / (299)) + (0.7829285777235853 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.43101884442065186 = (7.846851615457957 - 4.464710699822943) / 7.846851615457957 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.501032063770163 < 11.678191958936699.
IsChangePoint: Marked as a change because one of 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 104.74690354976339 (T) = (0 -10.495018153083612) / Math.Sqrt((0.06741219093601404 / (299)) + (0.00017820466441482782 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.14416620363046825 = (12.262916231637194 - 10.495018153083612) / 12.262916231637194 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.525861958208363 < 10.413816400137947.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 12/15/2022 5:36:17 AM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 14.513520988807661 (T) = (0 -9.340098633148957) / Math.Sqrt((0.49727158218196676 / (299)) + (0.04468304592096741 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.16663092876095045 = (11.207637714778805 - 9.340098633148957) / 11.207637714778805 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.313635450675617 < 10.305703903831162.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 7.612850761732069 (T) = (0 -7.759082664426766) / Math.Sqrt((1.162695604414055 / (299)) + (0.2548447368037254 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.22626838581337627 = (10.028131876947294 - 7.759082664426766) / 10.028131876947294 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.886415189387076 < 14.750230045478075.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 12.104997917634137 (T) = (0 -12.713976765746173) / Math.Sqrt((0.497921349268063 / (299)) + (0.21617800703158066 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.20541417264872447 = (16.000759550579673 - 12.713976765746173) / 16.000759550579673 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.405608218102275 < 15.913424937136778.
IsChangePoint: Marked as a change because one of 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 61.20547132628325 (T) = (0 -15.389996423496513) / Math.Sqrt((0.15171713876827872 / (299)) + (0.0001858076006451972 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08667013050082424 = (16.85042495318325 - 15.389996423496513) / 16.85042495318325 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.146594423309487 < 13.610415988100433.
IsChangePoint: Marked as a change because one of 12/12/2022 5:25:39 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 18.828251550678118 (T) = (0 -13.05917751953101) / Math.Sqrt((0.3044749310598751 / (299)) + (0.01790984173708495 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10756151785937566 = (14.633140301398651 - 13.05917751953101) / 14.633140301398651 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.90478935032995 < 20.58722724703346.
IsChangePoint: Marked as a change because one of 12/5/2022 9:44:53 PM, 1/5/2023 11:12:07 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 41.280479757618146 (T) = (0 -19.94085776541362) / Math.Sqrt((0.2822794882737133 / (299)) + (0.0046660061751987265 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09378849488118585 = (22.00463981396833 - 19.94085776541362) / 22.00463981396833 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.861100624449037 < 19.173870233759995.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 40.98395691792196 (T) = (0 -16.68561133242469) / Math.Sqrt((0.23824160560792507 / (299)) + (0.023098506744270678 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.18460887783523744 = (20.463322298783993 - 16.68561133242469) / 20.463322298783993 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.003774908357537 < 11.531733850056849.
IsChangePoint: Marked as a change because one of 12/30/2022 6:18:01 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 79.10616848186844 (T) = (0 -10.018117247668402) / Math.Sqrt((0.07476453838598496 / (299)) + (0.0012996199420533044 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.17108950617112276 = (12.085885414953585 - 10.018117247668402) / 12.085885414953585 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.70387832770039 < 19.458505358122427.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 50.100807531453974 (T) = (0 -18.672031927472947) / Math.Sqrt((0.4625184418897495 / (299)) + (0.0009558321299411091 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1038552315514666 = (20.83595484220617 - 18.672031927472947) / 20.83595484220617 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.00224137729712 < 17.722924720387677.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 31.91449220992386 (T) = (0 -16.9527285976429) / Math.Sqrt((0.19007844211729202 / (299)) + (0.009804135553224767 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10524337211649391 = (18.946748276951663 - 16.9527285976429) / 18.946748276951663 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.494857816341163 < 19.95803000384345.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 16.608753677556056 (T) = (0 -19.35671655234837) / Math.Sqrt((0.18945124155598253 / (299)) + (0.0376064698153071 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08963902700161457 = (21.262682745059525 - 19.35671655234837) / 21.262682745059525 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.40051100636939 < 9.869789574563024.
IsChangePoint: Marked as a change because one of 1/5/2023 9:23:01 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 28.75306752086394 (T) = (0 -8.356256175590273) / Math.Sqrt((0.5659798976937674 / (299)) + (0.017671220391383864 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.23287419493038236 = (10.892941053953898 - 8.356256175590273) / 10.892941053953898 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.09227027697842 < 14.783841203305204.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 34.49470082123364 (T) = (0 -13.134408815057808) / Math.Sqrt((0.5038940037142817 / (299)) + (0.015544918496159925 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.17873366230606197 = (15.992873702748327 - 13.134408815057808) / 15.992873702748327 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.128223782133436 < 14.797136915648442.
IsChangePoint: Marked as a change because one of 11/24/2022 9:42:44 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 33.96171111653249 (T) = (0 -13.13170959525219) / Math.Sqrt((0.18810609626770866 / (299)) + (0.016767984836737082 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.16939602676564688 = (15.809832385122851 - 13.13170959525219) / 15.809832385122851 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.72270788577495 < 18.708552896647863.
IsChangePoint: Marked as a change because one of 12/14/2022 12:01:27 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 16.039640245292222 (T) = (0 -17.93893901376039) / Math.Sqrt((0.4886640096068098 / (299)) + (0.050229434116348365 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10810693344158571 = (20.113329373645808 - 17.93893901376039) / 20.113329373645808 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.90373622141254 < 22.621996684934576.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 50.96292630001259 (T) = (0 -19.892408892679736) / Math.Sqrt((0.061570813138743956 / (299)) + (0.017892072743621455 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.16752476214669804 = (23.895496211966798 - 19.892408892679736) / 23.895496211966798 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.492333461313462 < 10.58410029691956.
IsChangePoint: Marked as a change because one of 11/21/2022 12:46:21 PM, 11/24/2022 8:21:24 PM, 12/14/2022 8:02:51 PM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 21.404418578499673 (T) = (0 -9.473123745426244) / Math.Sqrt((0.3754386167813039 / (299)) + (0.020280314220959782 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.16825647861344314 = (11.389477046522812 - 9.473123745426244) / 11.389477046522812 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.76067704397044 < 19.141738841840947.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 41.67831167379934 (T) = (0 -16.73264566996325) / Math.Sqrt((0.2202779030008498 / (299)) + (0.021541835477273348 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1814236820272789 = (20.44115533589241 - 16.73264566996325) / 20.44115533589241 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.93412047677392 < 18.691574459583823.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 26.741145992080416 (T) = (0 -17.872777683971425) / Math.Sqrt((0.21778807359674912 / (299)) + (0.016846430693292867 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10648025565021096 = (20.00266675357843 - 17.872777683971425) / 20.00266675357843 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.792041842615475 < 13.696950906282462.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 59.449246982790896 (T) = (0 -12.784907846688974) / Math.Sqrt((0.2844758551793466 / (299)) + (0.00010178779575871299 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.128323679625833 = (14.667035857072557 - 12.784907846688974) / 14.667035857072557 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 8, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultiplyBenchmark - Duration of single invocation 20.89 ns 18.08 ns 0.87 0.01 False
EqualsStaticBenchmark - Duration of single invocation 18.83 ns 16.52 ns 0.88 0.02 False
LessThanAnyBenchmark - Duration of single invocation 8.58 ns 7.17 ns 0.84 0.11 False
SubtractionOperatorBenchmark - Duration of single invocation 18.94 ns 17.16 ns 0.91 0.02 False
AddOperatorBenchmark - Duration of single invocation 18.81 ns 17.40 ns 0.92 0.01 False
AddBenchmark - Duration of single invocation 20.46 ns 17.11 ns 0.84 0.03 False
BitwiseAndBenchmark - Duration of single invocation 15.56 ns 12.73 ns 0.82 0.03 False
MaxBenchmark - Duration of single invocation 21.07 ns 19.09 ns 0.91 0.01 False
XorBenchmark - Duration of single invocation 15.57 ns 12.93 ns 0.83 0.02 False
LessThanOrEqualBenchmark - Duration of single invocation 19.00 ns 16.43 ns 0.86 0.02 False
MultiplyOperatorBenchmark - Duration of single invocation 19.52 ns 18.31 ns 0.94 0.02 False
BitwiseOrOperatorBenchmark - Duration of single invocation 14.27 ns 12.82 ns 0.90 0.03 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 9.97 ns 7.35 ns 0.74 0.11 False
EqualsAllBenchmark - Duration of single invocation 9.11 ns 6.53 ns 0.72 0.08 False
BitwiseOrBenchmark - Duration of single invocation 15.55 ns 12.97 ns 0.83 0.04 False
DivideBenchmark - Duration of single invocation 22.14 ns 19.58 ns 0.88 0.02 False
MinBenchmark - Duration of single invocation 21.13 ns 19.28 ns 0.91 0.03 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 14.23 ns 12.99 ns 0.91 0.02 False
LessThanBenchmark - Duration of single invocation 18.82 ns 16.31 ns 0.87 0.03 False
SubtractBenchmark - Duration of single invocation 20.38 ns 17.48 ns 0.86 0.00 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 9.26 ns 6.51 ns 0.70 0.07 False
GreaterThanBenchmark - Duration of single invocation 19.06 ns 16.49 ns 0.86 0.03 False
AndNotBenchmark - Duration of single invocation 14.47 ns 12.97 ns 0.90 0.05 False
DivisionOperatorBenchmark - Duration of single invocation 20.81 ns 19.57 ns 0.94 0.03 False
EqualsAnyBenchmark - Duration of single invocation 9.24 ns 6.51 ns 0.70 0.11 False
ConditionalSelectBenchmark - Duration of single invocation 16.66 ns 15.23 ns 0.91 0.03 False
BitwiseAndOperatorBenchmark - Duration of single invocation 14.28 ns 13.09 ns 0.92 0.02 False
GreaterThanOrEqualBenchmark - Duration of single invocation 18.97 ns 16.50 ns 0.87 0.01 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.MultiplyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.075910878604045 < 19.88367700095958.
IsChangePoint: Marked as a change because one of 12/5/2022 9:44:53 PM, 12/12/2022 5:25:39 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 21.455629441310553 (T) = (0 -18.269752402038623) / Math.Sqrt((0.04600617312084073 / (299)) + (0.04296017911038307 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1237960996199592 = (20.851028389755378 - 18.269752402038623) / 20.851028389755378 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.518808820959865 < 18.032031748286244.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 52.716641775682135 (T) = (0 -16.48847052680885) / Math.Sqrt((0.3308095316228958 / (299)) + (0.0015453418066220174 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11405926193241053 = (18.61125673346215 - 16.48847052680885) / 18.61125673346215 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.168983759532387 < 8.147376458114335.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 9.13499489446341 (T) = (0 -7.01039264769291) / Math.Sqrt((0.05736886500905332 / (299)) + (0.07306812053936768 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.16954640648291533 = (8.441642859299263 - 7.01039264769291) / 8.441642859299263 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.15800641345725 < 17.99681381782328.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 12/16/2022 5:41:28 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 14.865422045482914 (T) = (0 -17.104385119653653) / Math.Sqrt((0.04611425598127245 / (299)) + (0.04650809149063874 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09808215491177551 = (18.964460247463585 - 17.104385119653653) / 18.964460247463585 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.39680424670911 < 17.894315752385026.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 59.62230418248147 (T) = (0 -17.356325354282927) / Math.Sqrt((0.042375725592348604 / (299)) + (0.0013109618158557938 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07633062228752857 = (18.790625491197964 - 17.356325354282927) / 18.790625491197964 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.114320745142724 < 19.415373024886858.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 44.371354597832486 (T) = (0 -17.24262068904567) / Math.Sqrt((0.06404496941939014 / (299)) + (0.013601361377162172 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.15061217602780716 = (20.300056349300995 - 17.24262068904567) / 20.300056349300995 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.730838019471149 < 14.741864592892327.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 31.75058533117523 (T) = (0 -12.905584788231408) / Math.Sqrt((0.6160408285347271 / (299)) + (0.02292705686340083 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.19506556912947604 = (16.03308827810263 - 12.905584788231408) / 16.03308827810263 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.0908590141772 < 20.04320279398477.
IsChangePoint: Marked as a change because one of 11/25/2022 6:33:31 PM, 12/5/2022 3:49:45 AM, 12/14/2022 12:01:27 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 17.534956271538327 (T) = (0 -19.230894917595055) / Math.Sqrt((0.08902629301534594 / (299)) + (0.030187966380880404 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08492761906652471 = (21.015709050225524 - 19.230894917595055) / 21.015709050225524 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.927393182406817 < 14.76359975055874.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 47.14167942477667 (T) = (0 -12.98618628537885) / Math.Sqrt((0.24386293904616269 / (299)) + (0.00861758522115141 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1806355643305769 = (15.849096836585417 - 12.98618628537885) / 15.849096836585417 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.430365494327937 < 18.02712750363428.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 57.803565322948046 (T) = (0 -16.45348478599303) / Math.Sqrt((0.4092276396866683 / (299)) + (0.0004766466612375952 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.12072989991512084 = (18.71266267828818 - 16.45348478599303) / 18.71266267828818 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.306965870586783 < 18.508772410931595.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 12/5/2022 3:49:45 AM, 12/13/2022 12:26:27 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 46.71266884870477 (T) = (0 -18.276281128098322) / Math.Sqrt((0.07603740111053857 / (299)) + (0.0007941117662207124 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.05502437968950946 = (19.340478987270895 - 18.276281128098322) / 19.340478987270895 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.817339475618653 < 13.549700785141482.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 9.197711858945864 (T) = (0 -13.098963012207973) / Math.Sqrt((0.09461452529158952 / (299)) + (0.06443579822407525 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09392590593283136 = (14.456834267724828 - 13.098963012207973) / 14.456834267724828 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.345621598777366 < 8.88798144583479.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 24.644637916358064 (T) = (0 -7.279614611865977) / Math.Sqrt((0.09504776907866061 / (299)) + (0.017488949801581895 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.20976034697909174 = (9.211907532148823 - 7.279614611865977) / 9.211907532148823 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.528595026940004 < 8.456960596142183.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 36.96893613202738 (T) = (0 -6.585996873284935) / Math.Sqrt((0.1109882855584844 / (299)) + (0.007742160656287988 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.23370318635993778 = (8.594576874201186 - 6.585996873284935) / 8.594576874201186 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.96573892028534 < 14.761773439658024.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 33.89310763558205 (T) = (0 -13.013772907451305) / Math.Sqrt((0.5721395349962712 / (299)) + (0.01697657588208002 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.18476054509847162 = (15.963129396164002 - 13.013772907451305) / 15.963129396164002 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.57705032121719 < 21.084980457361272.
IsChangePoint: Marked as a change because one of 11/22/2022 11:24:47 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 22.03344583188573 (T) = (0 -19.712221111191237) / Math.Sqrt((0.12707912257174278 / (299)) + (0.03046215209097281 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10311189248564159 = (21.978461912960153 - 19.712221111191237) / 21.978461912960153 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.279368150649926 < 20.072226234190804.
IsChangePoint: Marked as a change because one of 12/14/2022 7:00:38 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 35.14131881884794 (T) = (0 -19.189045853929724) / Math.Sqrt((0.07281679148628888 / (299)) + (0.008018284263332731 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08999610540677505 = (21.08677332914855 - 19.189045853929724) / 21.08677332914855 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.992394059779999 < 13.6008092744531.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 41.282372003258665 (T) = (0 -13.046817141116307) / Math.Sqrt((0.16900193837500282 / (299)) + (0.0023933639522253584 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10459898428617656 = (14.570920640195212 - 13.046817141116307) / 14.570920640195212 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.306276501022886 < 18.002649006196105.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 23.682172710022222 (T) = (0 -16.419822065153976) / Math.Sqrt((0.20100980163696225 / (299)) + (0.026414822112408233 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.12312124436591725 = (18.725304906355706 - 16.419822065153976) / 18.725304906355706 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.479041010947377 < 19.369309810109666.
IsChangePoint: Marked as a change because one of 12/5/2022 11:02:29 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 30.533418475248265 (T) = (0 -17.433579225146882) / Math.Sqrt((0.04283745751551181 / (299)) + (0.025609033381559773 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.140280093759069 = (20.278208168255713 - 17.433579225146882) / 20.278208168255713 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.512529431848682 < 8.898371580882703.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 23.1392857386381 (T) = (0 -6.5847092684213635) / Math.Sqrt((0.08872777505618702 / (299)) + (0.03842799089895214 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2868856881136254 = (9.233735964438967 - 6.5847092684213635) / 9.233735964438967 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.48647043494465 < 18.042857200094602.
IsChangePoint: Marked as a change because one of 12/15/2022 5:36:17 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 53.23090830274735 (T) = (0 -16.51171626900765) / Math.Sqrt((0.28228854769194406 / (299)) + (0.002349139328034925 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1181496035173792 = (18.72394267198479 - 16.51171626900765) / 18.72394267198479 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.970427237709192 < 13.773607261344019.
IsChangePoint: Marked as a change because one of 11/17/2022 5:10:53 PM, 11/21/2022 12:46:21 PM, 12/5/2022 9:44:53 PM, 12/15/2022 5:36:17 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 8.007379844251737 (T) = (0 -13.173473367540566) / Math.Sqrt((0.20980877455010444 / (299)) + (0.057953801935189654 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07919300559328885 = (14.306443638635065 - 13.173473367540566) / 14.306443638635065 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.565053105144408 < 19.695521946602504.
IsChangePoint: Marked as a change because one of 12/5/2022 9:44:53 PM, 12/16/2022 10:05:39 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 60.35035850967452 (T) = (0 -19.556918602688114) / Math.Sqrt((0.12784060262883223 / (299)) + (8.025960413395072E-05 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.06171518728405165 = (20.84326457984424 - 19.556918602688114) / 20.84326457984424 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.509781028873067 < 8.419122328967914.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 10.459773432455867 (T) = (0 -6.765061445232912) / Math.Sqrt((0.06495177494965908 / (299)) + (0.07604554019797953 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1982154227235272 = (8.437505081742389 - 6.765061445232912) / 8.437505081742389 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.228390089226801 < 15.823616375011394.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 12/5/2022 9:44:53 PM, 12/15/2022 1:21:30 AM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 17.685875067143254 (T) = (0 -15.319152486616131) / Math.Sqrt((0.11096449282402299 / (299)) + (0.017696950133441983 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08376030014713373 = (16.719590396569966 - 15.319152486616131) / 16.719590396569966 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.09157437353403 < 13.726800508945546.
IsChangePoint: Marked as a change because one of 11/24/2022 9:42:44 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 21.937804072307106 (T) = (0 -13.065446741743438) / Math.Sqrt((0.06471237945947578 / (299)) + (0.011359152845850711 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09602973299335246 = (14.453403191022607 - 13.065446741743438) / 14.453403191022607 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.498297082770907 < 18.111232099118805.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 74.76905444096762 (T) = (0 -16.45167269575478) / Math.Sqrt((0.1282258710139225 / (299)) + (0.0018561515336609534 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.12823382322863772 = (18.871657485823235 - 16.45167269575478) / 18.871657485823235 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 8, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConditionalSelectBenchmark - Duration of single invocation 16.53 ns 15.37 ns 0.93 0.03 False
GreaterThanAllBenchmark - Duration of single invocation 23.39 ns 21.17 ns 0.90 0.02 False
EqualsBenchmark - Duration of single invocation 11.26 ns 8.50 ns 0.76 0.13 False
SubtractBenchmark - Duration of single invocation 39.23 ns 34.63 ns 0.88 0.00 False
DotBenchmark - Duration of single invocation 25.56 ns 23.61 ns 0.92 0.02 False
BitwiseAndBenchmark - Duration of single invocation 15.70 ns 13.01 ns 0.83 0.02 False
AddBenchmark - Duration of single invocation 37.05 ns 34.52 ns 0.93 0.00 False
DivideBenchmark - Duration of single invocation 62.83 ns 57.68 ns 0.92 0.00 False
BitwiseOrBenchmark - Duration of single invocation 15.70 ns 13.02 ns 0.83 0.01 False
XorBenchmark - Duration of single invocation 15.39 ns 13.10 ns 0.85 0.01 False
AndNotBenchmark - Duration of single invocation 14.49 ns 13.06 ns 0.90 0.04 False
MultiplyBenchmark - Duration of single invocation 38.74 ns 35.22 ns 0.91 0.01 False
BitwiseAndOperatorBenchmark - Duration of single invocation 14.27 ns 13.22 ns 0.93 0.02 False
EqualsAnyBenchmark - Duration of single invocation 23.23 ns 21.53 ns 0.93 0.01 False
DivisionOperatorBenchmark - Duration of single invocation 62.06 ns 58.39 ns 0.94 0.00 False
GreaterThanBenchmark - Duration of single invocation 40.89 ns 38.28 ns 0.94 0.00 False
LessThanBenchmark - Duration of single invocation 38.72 ns 36.58 ns 0.94 0.00 False
BitwiseOrOperatorBenchmark - Duration of single invocation 14.42 ns 13.11 ns 0.91 0.03 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 14.23 ns 12.93 ns 0.91 0.03 False
EqualsAllBenchmark - Duration of single invocation 8.59 ns 5.47 ns 0.64 0.10 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.ConditionalSelectBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.369292215197182 < 15.813755749664088.
IsChangePoint: Marked as a change because one of 11/17/2022 4:58:45 AM, 11/21/2022 12:46:21 PM, 12/5/2022 5:12:51 PM, 12/14/2022 12:01:27 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 24.12172282831141 (T) = (0 -15.408111619256777) / Math.Sqrt((0.10023432223581315 / (299)) + (0.00788004151530633 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07851177540772795 = (16.72089909350101 - 15.408111619256777) / 16.72089909350101 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.165293866969595 < 22.198448494401084.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 34.04959740064728 (T) = (0 -21.117952970043017) / Math.Sqrt((0.4037590919153576 / (299)) + (0.002279307602519199 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.06895797072699111 = (22.68206193283527 - 21.117952970043017) / 22.68206193283527 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.500805258353775 < 10.663978596637444.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 23.858678596916008 (T) = (0 -8.562707542023299) / Math.Sqrt((1.0289937806146576 / (299)) + (0.0043768556365699415 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.16321656483950936 = (10.232883661686033 - 8.562707542023299) / 10.232883661686033 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.63260424472191 < 36.47195989150955.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 124.49965930581453 (T) = (0 -34.60109582980323) / Math.Sqrt((0.156030817493846 / (299)) + (0.0007755039281658471 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0913326662811674 = (38.078947647643496 - 34.60109582980323) / 38.078947647643496 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.611587085998437 < 24.378402629429633.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 10.965070693105314 (T) = (0 -23.420541456349184) / Math.Sqrt((0.3056685913746381 / (299)) + (0.10215450916320473 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08061286819062215 = (25.47408011928223 - 23.420541456349184) / 25.47408011928223 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.008011022408722 < 14.79517221956413.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 34.3055438866215 (T) = (0 -12.961788966869754) / Math.Sqrt((0.5815024707877164 / (299)) + (0.017951741019731276 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.19072140937263657 = (16.01647333438242 - 12.961788966869754) / 16.01647333438242 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.51735017678861 < 36.24994172778882.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 68.05157692940158 (T) = (0 -34.607675637073235) / Math.Sqrt((0.1491039380997001 / (299)) + (0.006231381272826193 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09074215636079067 = (38.06145405197676 - 34.607675637073235) / 38.06145405197676 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 57.67867034547945 < 59.75773745432403.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 23.033085470386492 (T) = (0 -57.9147254823705) / Math.Sqrt((0.28364858455983905 / (299)) + (0.1276989422809096 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07660709638372383 = (62.71948295851043 - 57.9147254823705) / 62.71948295851043 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.019783563790515 < 14.717803930144326.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 83.44725963690388 (T) = (0 -12.992646588630777) / Math.Sqrt((0.23055151584747852 / (299)) + (0.001189363952031549 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.17996255607432265 = (15.843967473524712 - 12.992646588630777) / 15.843967473524712 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.09998625173285 < 14.747687977334957.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 29.796577056223033 (T) = (0 -12.942568481655028) / Math.Sqrt((0.5563903939712522 / (299)) + (0.02660151604105259 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.19254197911310375 = (16.02878186464624 - 12.942568481655028) / 16.02878186464624 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.059672543496392 < 13.781845089819935.
IsChangePoint: Marked as a change because one of 11/17/2022 5:10:53 PM, 11/21/2022 12:46:21 PM, 12/5/2022 9:44:53 PM, 12/15/2022 8:20:21 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 27.972172266082865 (T) = (0 -12.983443475783707) / Math.Sqrt((0.2003372067028891 / (299)) + (0.0047859538840504725 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09300534827971536 = (14.314796069809434 - 12.983443475783707) / 14.314796069809434 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 35.22245416000033 < 36.809705152563794.
IsChangePoint: Marked as a change because one of 12/14/2022 12:01:27 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 15.93393727557973 (T) = (0 -35.01995015450604) / Math.Sqrt((0.29004303497924855 / (299)) + (0.1431990237934766 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09124946860476091 = (38.53637378427563 - 35.01995015450604) / 38.53637378427563 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.22264566672107 < 13.562568807304848.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 12/12/2022 5:25:39 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 11.609923030268055 (T) = (0 -12.98871849989483) / Math.Sqrt((0.12268559337525753 / (299)) + (0.04968873660666997 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10430507120750772 = (14.50127502385799 - 12.98871849989483) / 14.50127502385799 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.5307382257886 < 21.984000981566854.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 8.16276371391013 (T) = (0 -21.36126312443601) / Math.Sqrt((0.4090734138403792 / (299)) + (0.059008747085308844 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.05251484112400314 = (22.5452218689915 - 21.36126312443601) / 22.5452218689915 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 58.39231716671236 < 59.13714010816948.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 14.35720323586379 (T) = (0 -58.11404406778379) / Math.Sqrt((1.6983155304420938 / (299)) + (0.1359325744303963 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.05283946834339211 = (61.3560659734637 - 58.11404406778379) / 61.3560659734637 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.277175078115796 < 38.851737598325364.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 27.817247083456177 (T) = (0 -38.33101792680818) / Math.Sqrt((0.4231796326854973 / (299)) + (0.01745761166085233 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.05813747745744183 = (40.697041244759994 - 38.33101792680818) / 40.697041244759994 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.579144269886065 < 36.83199470443617.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 53.430601523483304 (T) = (0 -36.64607848032901) / Math.Sqrt((0.07911811450612566 / (299)) + (0.0035106102846956706 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.05233759759546875 = (38.66997190913753 - 36.64607848032901) / 38.66997190913753 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.107802862646254 < 13.675416117343179.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 24.291472304134743 (T) = (0 -13.000394718042315) / Math.Sqrt((0.06773050780421314 / (299)) + (0.009837362790479334 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09961184848063347 = (14.438655924230794 - 13.000394718042315) / 14.438655924230794 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.929015818270571 < 13.517000659890355.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 5.657959973316112 (T) = (0 -12.635805319794892) / Math.Sqrt((0.2145167085555043 / (299)) + (0.36398253006706976 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1352688958624163 = (14.612409868610975 - 12.635805319794892) / 14.612409868610975 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.465917439959197 < 8.044745833873606.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/7/2023 9:28:02 PM falls between 2/27/2023 8:52:52 AM and 3/7/2023 9:28:02 PM.
IsImprovementStdDev: Marked as improvement because 10.949073540684903 (T) = (0 -5.674612843670579) / Math.Sqrt((0.14501211134107958 / (299)) + (0.08710754306051358 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.28821071948763627 = (7.972321302149775 - 5.674612843670579) / 7.972321302149775 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@fanyang-mono fanyang-mono mentioned this issue Mar 8, 2023
Closed
@fanyang-mono fanyang-mono mentioned this issue Mar 8, 2023
Merged
@kotlarmilos kotlarmilos mentioned this issue Apr 4, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ampere arch-arm64 branch-refs/heads/main kind-micro_mono mono-aot os-linux perf-improvement runtime-mono untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kotlarmilos