-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Perf] Linux/arm64: 14 Regressions on 5/13/2024 11:52:16 PM #34665
Labels
ampere
arch-arm64
branch-refs/heads/main
kind-micro_mono
mono-interpreter
Observation
Observe these benchmarks in the future to see if the regression is real
os-linux
perf-regression
runtime-mono
Comments
Run Information
Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md Repro StepsPrerequisites (Build files either built locally or downloaded from payload above)
Linux # Set $RunDir to the runtime directory
RunDir=`pwd`
# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir/artifacts/bin/mono/$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
cp $RunDir/artifacts/bin/runtime/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/shared/Microsoft.NETCore.App/8.0.0 -rf
cp $RunDir/artifacts/bin/testhost/net8.0-$RunOS-Release-$RunArch/* $RunDir/artifacts/dotnet-mono -r
cp $RunDir/artifacts/bin/coreclr/$RunOS.$RunArch.Release/corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun
# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance
# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"
# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun"
# Individual Commands:
# Restore
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir/artifacts/dotnet-mono/shared/Microsoft.NETCore.App/8.0.0/corerun --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages Windows # Set $RunDir to the runtime directory
$RunDir="FullPathHere"
# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'
# Create mono dotnet
mkdir -p $RunDir/artifacts/dotnet-mono
$RunDir/build.sh -subset libs.pretest -configuration release -ci -arch $RunArch -testscope innerloop /p:RuntimeArtifactsPath=$RunDir\artifacts\bin\mono\$RunOS.$RunArch.Release /p:RuntimeFlavor=mono
xcopy $RunDir\artifacts\bin\runtime\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\shared\Microsoft.NETCore.App\8.0.0\ /e /y
xcopy $RunDir\artifacts\bin\testhost\net8.0-$RunOS-Release-$RunArch\ $RunDir\artifacts\dotnet-mono\ /e /y
xcopy $RunDir\artifacts\bin\coreclr\$RunOS.$RunArch.Release\corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun /y
# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance
# Use Interpreter
export MONO_ENV_OPTIONS="--interpreter"
# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe"
# Individual Commands:
# Restore
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>*' --anyCategories Libraries Runtime " --category-exclusion-filter NoInterpreter NoMono --logBuildOutput --generateBinLog --corerun $RunDir\artifacts\dotnet-mono\shared\Microsoft.NETCore.App\8.0.0\corerun.exe --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.InequalityOperatorBenchmarkETL FilesHistogramJIT DisasmsSystem.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.EqualsAllBenchmarkETL FilesHistogramJIT DisasmsDocsProfiling workflow for dotnet/runtime repository |
kotlarmilos
added
Observation
Observe these benchmarks in the future to see if the regression is real
and removed
untriaged
labels
May 22, 2024
Could be related to dotnet/runtime#102557. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
ampere
arch-arm64
branch-refs/heads/main
kind-micro_mono
mono-interpreter
Observation
Observe these benchmarks in the future to see if the regression is real
os-linux
perf-regression
runtime-mono
Run Information
Regressions in System.Numerics.Tests.Perf_VectorConvert
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Repro Steps
Prerequisites (Build files either built locally or downloaded from payload above)
runtime/artifacts
or build instructions: Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release
, build instructions: CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
runtime/artifacts/bin/mono/$RunOS.$RunArch.Release
, build instructions: MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
Linux
Windows
System.Numerics.Tests.Perf_VectorConvert.Convert_float_uint
ETL Files
Histogram
JIT Disasms
System.Numerics.Tests.Perf_VectorConvert.Convert_double_ulong
ETL Files
Histogram
JIT Disasms
System.Numerics.Tests.Perf_VectorConvert.Convert_float_int
ETL Files
Histogram
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Text.Json.Serialization.Tests.WriteJson<Int32>
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Repro Steps
Prerequisites (Build files either built locally or downloaded from payload above)
runtime/artifacts
or build instructions: Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release
, build instructions: CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
runtime/artifacts/bin/mono/$RunOS.$RunArch.Release
, build instructions: MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
Linux
Windows
System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToStream(Mode: Reflection)
ETL Files
Histogram
JIT Disasms
System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToStream(Mode: SourceGen)
ETL Files
Histogram
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Repro Steps
Prerequisites (Build files either built locally or downloaded from payload above)
runtime/artifacts
or build instructions: Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release
, build instructions: CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
runtime/artifacts/bin/mono/$RunOS.$RunArch.Release
, build instructions: MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
Linux
Windows
System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToStream(Mode: SourceGen)
ETL Files
Histogram
JIT Disasms
System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToStream(Mode: Reflection)
ETL Files
Histogram
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Repro Steps
Prerequisites (Build files either built locally or downloaded from payload above)
runtime/artifacts
or build instructions: Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release
, build instructions: CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
runtime/artifacts/bin/mono/$RunOS.$RunArch.Release
, build instructions: MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
Linux
Windows
System.Runtime.Intrinsics.Tests.Perf_Vector128.ConvertDoubleToLongBenchmark
ETL Files
Histogram
JIT Disasms
System.Runtime.Intrinsics.Tests.Perf_Vector128.ConvertFloatToUIntBenchmark
ETL Files
Histogram
JIT Disasms
System.Runtime.Intrinsics.Tests.Perf_Vector128.ConvertDoubleToULongBenchmark
ETL Files
Histogram
JIT Disasms
System.Runtime.Intrinsics.Tests.Perf_Vector128.ConvertFloatToIntBenchmark
ETL Files
Histogram
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Repro Steps
Prerequisites (Build files either built locally or downloaded from payload above)
runtime/artifacts
or build instructions: Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release
, build instructions: CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
runtime/artifacts/bin/mono/$RunOS.$RunArch.Release
, build instructions: MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release
Linux
Windows
System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToStream(Mode: SourceGen)
ETL Files
Histogram
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
The text was updated successfully, but these errors were encountered: