[Perf] Regressions from inliner changes #56012

DrewScoggins · 2021-07-20T17:09:52Z

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Regressions in System.Memory.ReadOnlySpan

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IndexOfString - Duration of single invocation	32.94 ns	35.56 ns	1.08	0.03	False
IndexOfString - Duration of single invocation	42.75 ns	48.92 ns	1.14	0.08	True
IndexOfString - Duration of single invocation	41.68 ns	46.95 ns	1.13	0.11	True
IndexOfString - Duration of single invocation	52.62 ns	57.75 ns	1.10	0.07	True
IndexOfString - Duration of single invocation	35.99 ns	43.20 ns	1.20	0.13	True
IndexOfString - Duration of single invocation	19.85 ns	24.64 ns	1.24	0.06	False
IndexOfString - Duration of single invocation	15.02 ns	18.69 ns	1.24	0.00	True
IndexOfString - Duration of single invocation	16.06 ns	23.18 ns	1.44	0.01	True

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "X", comparisonType: Ordinal)

System.Memory.ReadOnlySpan.IndexOfString(input: "string1", value: "string2", comparisonType: InvariantCulture)

System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAA5AAAA", value: "5", comparisonType: InvariantCulture)

System.Memory.ReadOnlySpan.IndexOfString(input: "foobardzsdzs", value: "rddzs", comparisonType: InvariantCulture)

System.Memory.ReadOnlySpan.IndexOfString(input: "ABCDE", value: "c", comparisonType: InvariantCultureIgnoreCase)

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal)

System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Regressions in System.Formats.Cbor.Tests.Perf_CborReader

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SkipValue - Duration of single invocation	63.45 ns	68.50 ns	1.08	0.01	False
SkipValue - Duration of single invocation	50.81 ns	55.85 ns	1.10	0.01	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Formats.Cbor.Tests.Perf_CborReader*'

Payloads

Baseline
Compare

Histogram

System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding: (Text String, Strict))

System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding: (Byte String, Strict))

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Regressions in System.Collections.Concurrent.Count<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Queue - Duration of single invocation	5.43 ns	6.78 ns	1.25	0.06	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Collections.Concurrent.Count&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.Count<String>.Queue(Size: 512)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Regressions in System.Tests.Perf_Boolean

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse - Duration of single invocation	15.63 ns	16.90 ns	1.08	0.02	False
Parse - Duration of single invocation	15.55 ns	16.68 ns	1.07	0.01	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_Boolean*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Boolean.Parse(value: "False")

System.Tests.Perf_Boolean.Parse(value: "True")

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Regressions in System.Tests.Perf_Single

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ToStringWithFormat - Duration of single invocation	175.72 ns	188.64 ns	1.07	0.00	False
ToStringWithFormat - Duration of single invocation	172.44 ns	185.01 ns	1.07	0.01	False
ToStringWithFormat - Duration of single invocation	173.93 ns	185.70 ns	1.07	0.01	False
ToStringWithFormat - Duration of single invocation	170.30 ns	186.85 ns	1.10	0.00	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "R")

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "R")

System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G")

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G")

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Regressions in System.Tests.Perf_Int16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	21.02 ns	22.30 ns	1.06	0.17	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int16.ToString(value: -32768)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Regressions in System.Tests.Perf_Int32

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	28.26 ns	30.10 ns	1.07	0.20	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.ToString(value: -2147483648)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Regressions in System.Net.NetworkInformation.Tests.PhysicalAddressTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
PALong - Duration of single invocation	133.16 ns	143.10 ns	1.07	0.00	False
PAMedium - Duration of single invocation	22.54 ns	23.99 ns	1.06	0.04	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Net.NetworkInformation.Tests.PhysicalAddressTests*'

Payloads

Baseline
Compare

Histogram

System.Net.NetworkInformation.Tests.PhysicalAddressTests.PALong

System.Net.NetworkInformation.Tests.PhysicalAddressTests.PAMedium

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Regressions in System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CertProp - Duration of single invocation	21.50 ns	27.51 ns	1.28	0.07	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests*'

Payloads

Baseline
Compare

Histogram

System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests.CertProp

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Regressions in System.Tests.Perf_SByte

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	17.88 ns	20.38 ns	1.14	0.23	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_SByte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_SByte.ToString(value: -128)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Regressions in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryFormat - Duration of single invocation	106.68 ns	161.42 ns	1.51	0.03	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'

Payloads

Baseline
Compare

Histogram

System.Net.Primitives.Tests.IPAddressPerformanceTests.TryFormat(address: 1020:3040:5060:7080:9010:1112:1314:1516)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

category:performance
theme:benchmarks
skill-level:intermediate
cost:medium
impact:small

The text was updated successfully, but these errors were encountered:

dotnet-issue-labeler · 2021-07-20T17:09:58Z

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

DrewScoggins · 2021-07-20T17:10:23Z

Improvements from this change:

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Improvemnts in PerfLabTests.GetMember

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
GetMethod20 - Duration of single invocation	2.25 ms	1.81 ms	0.80	0.01	False
GetMethod4 - Duration of single invocation	444.89 μs	358.61 μs	0.81	0.02	False
GetMethod10 - Duration of single invocation	1.11 ms	905.43 μs	0.82	0.02	False
GetMethod5 - Duration of single invocation	553.90 μs	447.70 μs	0.81	0.03	False
GetMethod15 - Duration of single invocation	1.66 ms	1.35 ms	0.81	0.03	False
GetMethod1 - Duration of single invocation	108.19 μs	89.30 μs	0.83	0.05	False
GetMethod2 - Duration of single invocation	220.10 μs	180.76 μs	0.82	0.01	False
GetMethod3 - Duration of single invocation	331.17 μs	268.91 μs	0.81	0.03	False
GetMethod12 - Duration of single invocation	1.32 ms	1.08 ms	0.82	0.02	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'PerfLabTests.GetMember*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.GetMember.GetMethod20

PerfLabTests.GetMember.GetMethod4

PerfLabTests.GetMember.GetMethod10

PerfLabTests.GetMember.GetMethod5

PerfLabTests.GetMember.GetMethod15

PerfLabTests.GetMember.GetMethod1

PerfLabTests.GetMember.GetMethod2

PerfLabTests.GetMember.GetMethod3

PerfLabTests.GetMember.GetMethod12

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Improvemnts in System.Tests.Perf_Int64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	45.69 ns	37.45 ns	0.82	0.03	False
TryFormat - Duration of single invocation	39.43 ns	34.17 ns	0.87	0.02	True

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_Int64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int64.ToString(value: 9223372036854775807)

System.Tests.Perf_Int64.TryFormat(value: 9223372036854775807)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Improvemnts in System.Globalization.Tests.StringSearch

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IsPrefix_DifferentFirstChar - Duration of single invocation	21.69 ns	17.27 ns	0.80	0.19	True
IsPrefix_DifferentFirstChar - Duration of single invocation	9.53 ns	6.79 ns	0.71	0.09	True
IsPrefix_DifferentFirstChar - Duration of single invocation	20.61 ns	16.43 ns	0.80	0.14	True
IsPrefix_DifferentFirstChar - Duration of single invocation	23.73 ns	17.25 ns	0.73	0.14	True
IsPrefix_FirstHalf - Duration of single invocation	12.64 ns	10.55 ns	0.83	0.23	False
IsSuffix_DifferentLastChar - Duration of single invocation	22.07 ns	17.06 ns	0.77	0.15	False
IsSuffix_DifferentLastChar - Duration of single invocation	21.89 ns	17.41 ns	0.80	0.20	False
IsSuffix_SecondHalf - Duration of single invocation	13.36 ns	10.53 ns	0.79	0.27	False
IsSuffix_DifferentLastChar - Duration of single invocation	17.70 ns	14.14 ns	0.80	0.34	False
IsSuffix_DifferentLastChar - Duration of single invocation	21.95 ns	18.00 ns	0.82	0.15	False
IsPrefix_DifferentFirstChar - Duration of single invocation	10.41 ns	8.23 ns	0.79	0.05	True
IsPrefix_DifferentFirstChar - Duration of single invocation	21.16 ns	16.37 ns	0.77	0.15	True
IsPrefix_DifferentFirstChar - Duration of single invocation	21.45 ns	16.49 ns	0.77	0.15	True
IsSuffix_DifferentLastChar - Duration of single invocation	22.07 ns	17.58 ns	0.80	0.13	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreCase, False))

System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, Ordinal, False))

System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, None, False))

System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, IgnoreCase, False))

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, Ordinal, False))

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, None, False))

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (, None, False))

System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, Ordinal, False))

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, Ordinal, False))

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (, IgnoreCase, False))

System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, OrdinalIgnoreCase, False))

System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreNonSpace, False))

System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, None, False))

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, IgnoreCase, False))

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Improvemnts in System.IO.Tests.Perf_Path

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetDirectoryName - Duration of single invocation	66.89 ns	57.87 ns	0.87	0.09	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.GetDirectoryName

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Improvemnts in System.Tests.Perf_Guid

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse - Duration of single invocation	84.35 ns	74.00 ns	0.88	0.01	True
ctor_str - Duration of single invocation	87.68 ns	77.27 ns	0.88	0.00	True

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_Guid*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Guid.Parse

System.Tests.Perf_Guid.ctor_str

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Improvemnts in System.Memory.ReadOnlySpan

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IndexOfString - Duration of single invocation	35.93 ns	30.72 ns	0.86	0.10	True
IndexOfString - Duration of single invocation	28.03 ns	24.42 ns	0.87	0.15	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase)

System.Memory.ReadOnlySpan.IndexOfString(input: "StrIng", value: "string", comparisonType: OrdinalIgnoreCase)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Improvemnts in System.Numerics.Tests.Perf_Matrix4x4

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
CreateScaleFromScalarXYZWithCenterBenchmark - Duration of single invocation	11.58 ns	8.79 ns	0.76	0.01	True
CreateFromScalars - Duration of single invocation	16.25 ns	11.00 ns	0.68	0.01	True
CreateRotationXWithCenterBenchmark - Duration of single invocation	23.77 ns	8.83 ns	0.37	0.04	True
CreateRotationYWithCenterBenchmark - Duration of single invocation	23.47 ns	8.83 ns	0.38	0.08	True
CreateScaleFromScalarXYZBenchmark - Duration of single invocation	11.58 ns	8.79 ns	0.76	0.01	True
CreateRotationZWithCenterBenchmark - Duration of single invocation	25.42 ns	8.85 ns	0.35	0.04	True
CreateScaleFromScalarWithCenterBenchmark - Duration of single invocation	10.68 ns	8.83 ns	0.83	0.00	True
CreateOrthographicOffCenterBenchmark - Duration of single invocation	11.63 ns	8.48 ns	0.73	0.00	True

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZWithCenterBenchmark

System.Numerics.Tests.Perf_Matrix4x4.CreateFromScalars

System.Numerics.Tests.Perf_Matrix4x4.CreateRotationXWithCenterBenchmark

System.Numerics.Tests.Perf_Matrix4x4.CreateRotationYWithCenterBenchmark

System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZBenchmark

System.Numerics.Tests.Perf_Matrix4x4.CreateRotationZWithCenterBenchmark

System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarWithCenterBenchmark

System.Numerics.Tests.Perf_Matrix4x4.CreateOrthographicOffCenterBenchmark

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Improvemnts in System.Tests.Perf_UInt64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	45.54 ns	37.99 ns	0.83	0.03	False
TryFormat - Duration of single invocation	40.42 ns	35.47 ns	0.88	0.02	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.ToString(value: 18446744073709551615)

System.Tests.Perf_UInt64.TryFormat(value: 18446744073709551615)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Improvemnts in System.Text.Json.Tests.Perf_Base64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
WriteByteArrayAsBase64_NoEscaping - Duration of single invocation	104.11 ns	96.00 ns	0.92	0.02	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Text.Json.Tests.Perf_Base64*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_NoEscaping(NumberOfBytes: 100)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Improvemnts in System.Tests.Perf_String

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Split - Duration of single invocation	418.44 ns	377.65 ns	0.90	0.03	False
Interpolation_MultipleArgs - Duration of single invocation	253.23 ns	106.62 ns	0.42	0.03	True
TrimEnd_CharArr - Duration of single invocation	5.04 ns	3.88 ns	0.77	0.16	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Split(s: "A B C D E F G H I J K L M N O P Q R S T U V W X Y Z", arr: [' '], options: None)

System.Tests.Perf_String.Interpolation_MultipleArgs(c: '1', s: "Foo")

System.Tests.Perf_String.TrimEnd_CharArr(s: "Test", c: [' ', ' '])

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Improvemnts in System.Tests.Perf_Enum

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse - Duration of single invocation	127.46 ns	116.24 ns	0.91	0.01	False
EnumToString - Duration of single invocation	20.66 ns	18.81 ns	0.91	0.12	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.Parse(text: "Red")

System.Tests.Perf_Enum.EnumToString(value: Yellow)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Improvemnts in System.Tests.Perf_Version

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Parse3 - Duration of single invocation	42.20 ns	36.62 ns	0.87	0.03	False
Parse4 - Duration of single invocation	66.95 ns	54.08 ns	0.81	0.03	True
Parse2 - Duration of single invocation	38.59 ns	29.58 ns	0.77	0.03	True

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_Version*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Version.Parse3

System.Tests.Perf_Version.Parse4

System.Tests.Perf_Version.Parse2

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Improvemnts in System.Tests.Perf_Double

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToStringWithFormat - Duration of single invocation	196.19 ns	184.22 ns	0.94	0.00	False
ToStringWithFormat - Duration of single invocation	199.38 ns	184.03 ns	0.92	0.00	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "E")

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "E")

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Improvemnts in System.Text.Json.Serialization.Tests.ReadJson<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
DeserializeFromUtf8Bytes - Duration of single invocation	108.74 ns	100.05 ns	0.92	0.01	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromUtf8Bytes

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Improvemnts in System.Collections.CtorDefaultSize<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	5.95 ns	4.09 ns	0.69	0.37	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Collections.CtorDefaultSize&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorDefaultSize<Int32>.Dictionary

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	92547f59890b738e02f115277f272a381c9f3f3b
Compare	56d9474f530579c67b294cbbde0724e966dbb05b
Diff	Diff

Improvemnts in System.Collections.IterateForEach<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableList - Duration of single invocation	10.66 μs	6.83 μs	0.64	0.08	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<Int32>.ImmutableList(Size: 512)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

EgorBo · 2021-07-27T11:25:35Z

This issue has 11 regressions and 16 improvements. Almost all regressions have "U"-shape - it means my first PR #52708 improved it and the second one #55478 regressed it back. So overall it's a zero impact (Actually, in some benchmarks it's still an improvement after second PR). The second PR introduced a couple of "slowdown" mechanisms for the inliner like "if there are too many locals in the root method already - slow down inlining new calls" - I'd like to keep it, but it for sure has to be tuned, like changing linear dependency on number of locals to quadratic, etc.

Here are the "U" shapes:

And here is starting point:

EgorBo · 2021-07-27T11:58:50Z

The only exception is the first one ReadOnlySpan.IndexOfString
it suffers from two things:

That mechanism to slow-down if there are too many locals
if we inline a method that has a pinvoke - we have to import CORINFO_HELP_JIT_PINVOKE_BEGIN into prologue, even if the inline was invoked only under certain condition.

EgorBo · 2021-07-27T12:00:26Z

I'm going to move it to 7.0 since the overall it's an improvement (even some of the IndexOfString were actually improved).
For 7.0 I'm going to revise those two reasons ^.

DrewScoggins added os-windows tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark arch-x64 labels Jul 20, 2021

DrewScoggins assigned EgorBo Jul 20, 2021

dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Jul 20, 2021

jeffschwMSFT added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Jul 21, 2021

JulieLeeMSFT added needs-further-triage Issue has been initially triaged, but needs deeper consideration or reconsideration and removed untriaged New issue has not been triaged by the area owner labels Jul 21, 2021

JulieLeeMSFT added this to the 6.0.0 milestone Jul 21, 2021

EgorBo modified the milestones: 6.0.0, 7.0.0 Jul 27, 2021

JulieLeeMSFT removed the needs-further-triage Issue has been initially triaged, but needs deeper consideration or reconsideration label Jul 30, 2021

danmoseley mentioned this issue Apr 6, 2022

Regressions in System.Globalization.Tests.StringSearch #67595

Closed

EgorBo modified the milestones: 7.0.0, 8.0.0 May 30, 2022

EgorBo modified the milestones: 8.0.0, Future Apr 10, 2023

[Perf] Regressions from inliner changes #56012

[Perf] Regressions from inliner changes #56012

Comments

DrewScoggins commented Jul 20, 2021 • edited by BruceForstall Loading

Run Information

Regressions in System.Memory.ReadOnlySpan

Repro

Payloads

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "string1", value: "string2", comparisonType: InvariantCulture)

System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAA5AAAA", value: "5", comparisonType: InvariantCulture)

System.Memory.ReadOnlySpan.IndexOfString(input: "foobardzsdzs", value: "rddzs", comparisonType: InvariantCulture)

System.Memory.ReadOnlySpan.IndexOfString(input: "ABCDE", value: "c", comparisonType: InvariantCultureIgnoreCase)

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal)

Docs

Regressions in System.Formats.Cbor.Tests.Perf_CborReader

Repro

Payloads

Histogram

System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding: (Text String, Strict))

System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding: (Byte String, Strict))

Docs

Run Information

Regressions in System.Collections.Concurrent.Count<String>

Repro

Payloads

Histogram

System.Collections.Concurrent.Count<String>.Queue(Size: 512)

Docs

Run Information

Regressions in System.Tests.Perf_Boolean

Repro

Payloads

Histogram

System.Tests.Perf_Boolean.Parse(value: "False")

System.Tests.Perf_Boolean.Parse(value: "True")

Docs

Run Information

Regressions in System.Tests.Perf_Single

Repro

Payloads

Histogram

System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "R")

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "R")

System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G")

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G")

Docs

Run Information

Regressions in System.Tests.Perf_Int16

Repro

Payloads

Histogram

System.Tests.Perf_Int16.ToString(value: -32768)

Docs

Run Information

Regressions in System.Tests.Perf_Int32

Repro

Payloads

Histogram

System.Tests.Perf_Int32.ToString(value: -2147483648)

Docs

Run Information

Regressions in System.Net.NetworkInformation.Tests.PhysicalAddressTests

Repro

Payloads

Histogram

System.Net.NetworkInformation.Tests.PhysicalAddressTests.PALong

System.Net.NetworkInformation.Tests.PhysicalAddressTests.PAMedium

Docs

Run Information

Regressions in System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests

Repro

Payloads

Histogram

System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests.CertProp

Docs

Run Information

Regressions in System.Tests.Perf_SByte

Repro

DrewScoggins commented Jul 20, 2021 •

edited by BruceForstall

Loading