[mono][interp][Perf] Interpreter: missing vector instrinsics causing 117 Regressions on 6/15/2024 2:08:36 PM #109375

performanceautofiler · 2024-06-18T08:58:18Z

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare	4834a507be8698d901ff36d10284ca8703743b93
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Vector3

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
SquareRootBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	22.83 ns	116.75 ns	5.11	0.20	True
LengthBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	15.96 ns	93.12 ns	5.83	0.21	True
DistanceSquaredBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	25.43 ns	109.34 ns	4.30	0.20	True
MultiplyFunctionBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	18.68 ns	23.17 ns	1.24	0.28	False
MaxBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	19.78 ns	151.08 ns	7.64	0.25	True
MultiplyOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	16.78 ns	27.63 ns	1.65	0.26	False
DivideByScalarBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	18.00 ns	101.79 ns	5.66	0.28	True
LerpBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	47.02 ns	235.60 ns	5.01	0.20	True
CreateFromVector2WithScalarBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	12.34 ns	20.89 ns	1.69	0.23	True
NormalizeBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	30.50 ns	190.08 ns	6.23	0.11	True
TransformNormalByMatrix4x4Benchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	42.05 ns	319.84 ns	7.61	0.12	True
MultiplyByScalarOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	17.42 ns	97.88 ns	5.62	0.22	True
MinBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	18.72 ns	154.32 ns	8.24	0.23	True
EqualityOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	8.02 ns	70.05 ns	8.74	0.40	True
DotBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	14.88 ns	89.70 ns	6.03	0.34	True
MultiplyByScalarBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	21.43 ns	98.72 ns	4.61	0.26	True
SubtractFunctionBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	15.73 ns	24.97 ns	1.59	0.30	False
SubtractOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	15.79 ns	21.56 ns	1.37	0.32	False
ReflectBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	55.57 ns	318.87 ns	5.74	0.16	True
DivideByVector3Benchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	16.04 ns	23.33 ns	1.46	0.32	False
EqualsBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	15.77 ns	52.41 ns	3.32	0.20	True
LengthSquaredBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	14.44 ns	90.73 ns	6.28	0.29	True
DistanceBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	27.16 ns	111.43 ns	4.10	0.14	True
DivideByVector3OperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	16.49 ns	23.63 ns	1.43	0.27	False
DivideByScalarOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	18.14 ns	103.09 ns	5.68	0.25	True
AddOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	15.98 ns	22.50 ns	1.41	0.34	False
AbsBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	11.63 ns	101.90 ns	8.76	0.34	True
TransformByMatrix4x4Benchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	45.86 ns	319.84 ns	6.97	0.08	True
AddFunctionBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	16.84 ns	22.45 ns	1.33	0.34	False
InequalityOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	9.13 ns	69.76 ns	7.64	0.39	True
ClampBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	36.95 ns	364.80 ns	9.87	0.15	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Build files either built locally or downloaded from payload above)

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
Mono Runtime build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Vector3*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Vector3*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages

System.Numerics.Tests.Perf_Vector3.SquareRootBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.LengthBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DistanceSquaredBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyFunctionBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MaxBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DivideByScalarBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.LerpBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.CreateFromVector2WithScalarBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.NormalizeBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.TransformNormalByMatrix4x4Benchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyByScalarOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MinBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.EqualityOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DotBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyByScalarBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.SubtractFunctionBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.SubtractOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.ReflectBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DivideByVector3Benchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.EqualsBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.LengthSquaredBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DistanceBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DivideByVector3OperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DivideByScalarOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.AddOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.AbsBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.TransformByMatrix4x4Benchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.AddFunctionBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.InequalityOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.ClampBenchmark

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler · 2024-06-18T08:58:20Z

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare	4834a507be8698d901ff36d10284ca8703743b93
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
CreateScaleFromScalarWithCenterBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	60.65 ns	91.36 ns	1.51	0.18	False
CreateRotationYWithCenterBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	51.03 ns	55.79 ns	1.09	0.11	False
CreateFromYawPitchRollBenchmarkBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	150.85 ns	167.98 ns	1.11	0.12	False
DecomposeBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	489.49 ns	1.40 μs	2.85	0.05	True
CreateScaleFromScalarXYZBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	62.05 ns	89.70 ns	1.45	0.14	False
CreateWorldBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	207.97 ns	675.51 ns	3.25	0.09	True
MultiplyByMatrixBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	287.49 ns	1.73 μs	6.03	0.04	True
CreateRotationYBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	34.06 ns	41.57 ns	1.22	0.14	False
CreateReflectionBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	312.27 ns	863.37 ns	2.76	0.13	True
CreateRotationXBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	37.47 ns	44.04 ns	1.18	0.13	False
CreateScaleFromScalarXYZWithCenterBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	61.66 ns	90.14 ns	1.46	0.13	False
MultiplyByMatrixOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	281.95 ns	1.72 μs	6.11	0.05	True
CreateTranslationFromVectorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	33.79 ns	39.71 ns	1.18	0.21	False
CreateRotationZWithCenterBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	52.08 ns	57.35 ns	1.10	0.14	False
CreateConstrainedBillboardBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	296.42 ns	1.04 μs	3.52	0.07	True
CreateRotationXWithCenterBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	52.18 ns	59.16 ns	1.13	0.16	False
CreateScaleFromVectorWithCenterBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	75.11 ns	98.74 ns	1.31	0.13	False
CreateLookAtBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	247.56 ns	849.96 ns	3.43	0.09	True
CreateShadowBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	304.77 ns	889.08 ns	2.92	0.10	True
MultiplyByScalarOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	60.97 ns	420.03 ns	6.89	0.16	True
CreateRotationZBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	37.10 ns	40.46 ns	1.09	0.22	False
MultiplyByScalarBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	63.25 ns	407.06 ns	6.44	0.19	True
CreateBillboardBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	231.11 ns	686.58 ns	2.97	0.08	True
LerpBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	95.15 ns	833.03 ns	8.75	0.06	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Build files either built locally or downloaded from payload above)

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
Mono Runtime build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Matrix4x4*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Matrix4x4*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages

System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarWithCenterBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateRotationYWithCenterBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateFromYawPitchRollBenchmarkBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.DecomposeBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateWorldBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.MultiplyByMatrixBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateRotationYBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateReflectionBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateRotationXBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZWithCenterBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.MultiplyByMatrixOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateTranslationFromVectorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateRotationZWithCenterBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateConstrainedBillboardBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateRotationXWithCenterBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromVectorWithCenterBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateLookAtBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateShadowBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateRotationZBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateBillboardBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.LerpBenchmark

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2024-06-18T08:58:21Z

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare	4834a507be8698d901ff36d10284ca8703743b93
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Matrix3x2

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
MultiplyByScalarBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	89.69 ns	365.51 ns	4.08	0.09	True
SubtractOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	131.70 ns	145.71 ns	1.11	0.06	False
EqualityOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	101.21 ns	513.73 ns	5.08	0.09	True
CreateScaleFromScalarXYWithCenterBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	51.75 ns	74.11 ns	1.43	0.11	False
CreateScaleFromScalarWithCenterBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	48.72 ns	73.87 ns	1.52	0.11	True
AddBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	121.89 ns	154.07 ns	1.26	0.11	False
EqualsBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	122.70 ns	280.36 ns	2.28	0.12	True
AddOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	123.29 ns	152.86 ns	1.24	0.11	False
SubtractBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	121.68 ns	145.53 ns	1.20	0.11	False
MultiplyByScalarOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	90.45 ns	350.79 ns	3.88	0.10	True
IsIdentityBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	72.57 ns	490.09 ns	6.75	0.09	True
InequalityOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	102.51 ns	524.29 ns	5.11	0.12	True
CreateScaleFromVectorWithCenterBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	47.85 ns	77.41 ns	1.62	0.12	True
NegateBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	82.18 ns	96.33 ns	1.17	0.09	False
LerpBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	173.46 ns	750.67 ns	4.33	0.11	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Build files either built locally or downloaded from payload above)

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
Mono Runtime build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Matrix3x2*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Matrix3x2*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages

System.Numerics.Tests.Perf_Matrix3x2.MultiplyByScalarBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.SubtractOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.EqualityOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarXYWithCenterBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarWithCenterBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.AddBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.EqualsBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.AddOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.SubtractBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.MultiplyByScalarOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.IsIdentityBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.InequalityOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromVectorWithCenterBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.NegateBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.LerpBenchmark

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2024-06-18T08:58:21Z

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare	87cd2c4257bd21e84d3d6990f55465be01f63399
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Vector2

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
LengthSquaredBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	10.95 ns	108.16 ns	9.88	0.34	True
SubtractOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	12.88 ns	21.61 ns	1.68	0.30	True
LengthBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	15.44 ns	111.62 ns	7.23	0.38	True
ClampBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	28.94 ns	358.44 ns	12.38	0.17	True
MultiplyByScalarOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	14.27 ns	98.51 ns	6.90	0.31	True
NormalizeBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	24.51 ns	218.92 ns	8.93	0.22	True
TransformNormalByMatrix3x2Benchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	73.27 ns	264.63 ns	3.61	0.13	True
AddFunctionBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	12.75 ns	21.57 ns	1.69	0.33	False
MultiplyOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	13.00 ns	23.67 ns	1.82	0.36	False
MinBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	15.37 ns	156.81 ns	10.20	0.31	True
MaxBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	16.16 ns	150.09 ns	9.29	0.28	True
SubtractFunctionBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	12.30 ns	21.80 ns	1.77	0.36	False
AddOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	12.83 ns	21.89 ns	1.71	0.28	False
DivideByScalarOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	14.40 ns	101.38 ns	7.04	0.28	True
DivideByVector2OperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	13.60 ns	21.78 ns	1.60	0.34	False
DistanceBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	19.62 ns	130.10 ns	6.63	0.21	True
EqualsBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	16.74 ns	70.86 ns	4.23	0.24	True
EqualityOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	7.91 ns	86.59 ns	10.95	0.41	True
SquareRootBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	14.85 ns	109.66 ns	7.39	0.27	True
TransformByMatrix4x4Benchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	39.30 ns	223.56 ns	5.69	0.13	True
ReflectBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	44.72 ns	338.55 ns	7.57	0.15	True
LerpBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	37.98 ns	232.58 ns	6.12	0.19	True
TransformNormalByMatrix4x4Benchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	34.21 ns	218.94 ns	6.40	0.13	True
DistanceSquaredBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	17.56 ns	125.88 ns	7.17	0.30	True
DivideByVector2Benchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	13.42 ns	21.47 ns	1.60	0.31	False
AbsBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	9.18 ns	101.51 ns	11.05	0.41	True
DotBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	10.35 ns	107.28 ns	10.37	0.47	True
TransformByMatrix3x2Benchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	79.75 ns	290.97 ns	3.65	0.11	True
MultiplyFunctionBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	12.82 ns	21.95 ns	1.71	0.31	False
InequalityOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	9.00 ns	87.44 ns	9.71	0.36	True
DivideByScalarBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	15.37 ns	102.50 ns	6.67	0.37	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Build files either built locally or downloaded from payload above)

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
Mono Runtime build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector2*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Vector2*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector2*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Vector2*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages

System.Numerics.Tests.Perf_Vector2.LengthSquaredBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.SubtractOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.LengthBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.ClampBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.MultiplyByScalarOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.NormalizeBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.TransformNormalByMatrix3x2Benchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.AddFunctionBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.MultiplyOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.MinBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.MaxBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.SubtractFunctionBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.AddOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.DivideByScalarOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.DivideByVector2OperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.DistanceBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.EqualsBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.EqualityOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.SquareRootBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.TransformByMatrix4x4Benchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.ReflectBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.LerpBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.TransformNormalByMatrix4x4Benchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.DistanceSquaredBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.DivideByVector2Benchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.AbsBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.DotBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.TransformByMatrix3x2Benchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.MultiplyFunctionBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.InequalityOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.DivideByScalarBenchmark

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2024-06-18T08:58:22Z

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare	87cd2c4257bd21e84d3d6990f55465be01f63399
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Vector4

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
LerpBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	13.24 ns	192.68 ns	14.55	0.20	True
DivideByScalarOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	7.35 ns	94.10 ns	12.81	0.36	True
MultiplyByScalarBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	6.39 ns	87.00 ns	13.61	0.34	True
DivideByScalarBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	6.70 ns	101.25 ns	15.11	0.33	True
NormalizeBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	50.39 ns	135.34 ns	2.69	0.17	False
UnitXBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	0.89 ns	3.83 ns	4.32	0.92	False
TransformVector2ByMatrix4x4Benchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	35.71 ns	221.29 ns	6.20	0.15	True
MultiplyByScalarOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	6.86 ns	89.81 ns	13.09	0.33	True
TransformByMatrix4x4Benchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	43.67 ns	317.48 ns	7.27	0.11	True
TransformVector3ByMatrix4x4Benchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	42.52 ns	318.11 ns	7.48	0.14	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Build files either built locally or downloaded from payload above)

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
Mono Runtime build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector4*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Vector4*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector4*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Vector4*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages

System.Numerics.Tests.Perf_Vector4.LerpBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector4.DivideByScalarOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector4.MultiplyByScalarBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector4.DivideByScalarBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector4.NormalizeBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector4.UnitXBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector4.TransformVector2ByMatrix4x4Benchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector4.MultiplyByScalarOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector4.TransformByMatrix4x4Benchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector4.TransformVector3ByMatrix4x4Benchmark

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare	4834a507be8698d901ff36d10284ca8703743b93
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Regressions in SIMD.RayTracerBench

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Bench - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	1.85 secs	4.21 secs	2.27	0.01	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Build files either built locally or downloaded from payload above)

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
Mono Runtime build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'SIMD.RayTracerBench*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'SIMD.RayTracerBench*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'SIMD.RayTracerBench*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'SIMD.RayTracerBench*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages

SIMD.RayTracerBench.Bench

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare	4834a507be8698d901ff36d10284ca8703743b93
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Quaternion

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
CreateFromYawPitchRollBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	68.13 ns	98.22 ns	1.44	0.10	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Build files either built locally or downloaded from payload above)

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
Mono Runtime build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Quaternion*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Quaternion*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Quaternion*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Quaternion*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages

System.Numerics.Tests.Perf_Quaternion.CreateFromYawPitchRollBenchmark

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare	4834a507be8698d901ff36d10284ca8703743b93
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:false, MonoInterpreter:true, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Plane

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
CreateFromVerticesBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	112.47 ns	408.41 ns	3.63	0.15	True
NormalizeBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	26.19 ns	98.25 ns	3.75	0.25	True
DotNormalBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	25.69 ns	97.51 ns	3.80	0.26	True
DotCoordinateBenchmark - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	20.84 ns	94.50 ns	4.53	0.24	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Build files either built locally or downloaded from payload above)

Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
Mono Runtime build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
Dotnet SDK installed for dotnet commands
Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Plane*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun" 

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Plane*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Plane*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe" 

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Numerics.Tests.Perf_Plane*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages

System.Numerics.Tests.Perf_Plane.CreateFromVerticesBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Plane.NormalizeBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Plane.DotNormalBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Plane.DotCoordinateBenchmark

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

kotlarmilos · 2024-06-29T16:33:32Z

Regressions introduced in #103462.

dotnet-policy-service · 2024-10-30T16:37:57Z

Tagging subscribers to this area: @fanyang-mono, @steveisok
See info in area-owners.md if you want to be subscribed.

matouskozak · 2024-10-30T16:38:12Z

Possibly introduced in #103462, cc @tannergooding

tannergooding · 2024-10-30T16:41:06Z

Looks like these are all centered around Vector3 which Mono doesn't have accelerated on all platforms.

This probably isn't terribly difficult to handle for most of the common scenarios.

performanceautofiler bot added arch-x64 os-linux Linux OS (any supported distro) runtime-mono specific to the Mono runtime untriaged New issue has not been triaged by the area owner labels Jun 18, 2024

performanceautofiler bot mentioned this issue Jun 18, 2024

[SENTINEL] Autofile run complete at 6/18/2024 8:59:11 AM. 8 issues filed. dotnet/perf-autofiling-issues#36497

Closed

kotlarmilos removed the untriaged New issue has not been triaged by the area owner label Jun 29, 2024

kotlarmilos mentioned this issue Jun 29, 2024

.NET 9 per-preview performance overview #98209

Open

matouskozak added the arch-arm64 label Oct 30, 2024

matouskozak changed the title ~~[Perf] Linux/x64: 117 Regressions on 6/15/2024 2:08:36 PM~~ [Perf] Interpreter: missing vector instrinsics causing 117 Regressions on 6/15/2024 2:08:36 PM Oct 30, 2024

matouskozak changed the title ~~[Perf] Interpreter: missing vector instrinsics causing 117 Regressions on 6/15/2024 2:08:36 PM~~ [mono][interp][Perf] Interpreter: missing vector instrinsics causing 117 Regressions on 6/15/2024 2:08:36 PM Oct 30, 2024

matouskozak transferred this issue from dotnet/perf-autofiling-issues Oct 30, 2024

dotnet-issue-labeler bot added the area-System.Numerics label Oct 30, 2024

matouskozak removed arch-arm64 area-System.Numerics os-linux Linux OS (any supported distro) arch-x64 runtime-mono specific to the Mono runtime labels Oct 30, 2024

dotnet-policy-service bot added the untriaged New issue has not been triaged by the area owner label Oct 30, 2024

matouskozak added runtime-mono specific to the Mono runtime area-Codegen-Intrinsics-mono arch-arm64 arch-x64 labels Oct 30, 2024

kotlarmilos added this to the 10.0.0 milestone Oct 30, 2024

kotlarmilos mentioned this issue Oct 30, 2024

[Perf] Linux/x64: 3 Regressions on 1/17/2024 11:26:01 PM dotnet/perf-autofiling-issues#27704

Closed

[mono][interp][Perf] Interpreter: missing vector instrinsics causing 117 Regressions on 6/15/2024 2:08:36 PM #109375

[mono][interp][Perf] Interpreter: missing vector instrinsics causing 117 Regressions on 6/15/2024 2:08:36 PM #109375

Comments

performanceautofiler bot commented Jun 18, 2024

Run Information

Regressions in System.Numerics.Tests.Perf_Vector3

Repro

Prerequisites (Build files either built locally or downloaded from payload above)

System.Numerics.Tests.Perf_Vector3.SquareRootBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.LengthBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DistanceSquaredBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyFunctionBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MaxBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DivideByScalarBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.LerpBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.CreateFromVector2WithScalarBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.NormalizeBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.TransformNormalByMatrix4x4Benchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyByScalarOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MinBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.EqualityOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DotBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyByScalarBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.SubtractFunctionBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.SubtractOperatorBenchmark

ETL Files

Histogram

JIT Disasms