Skip to content

Adding P50, P90 and P100 percentiles to benchmarks#2411

Merged
jmprieur merged 2 commits into
devfrom
jmprieur/Percentiles
Nov 21, 2023
Merged

Adding P50, P90 and P100 percentiles to benchmarks#2411
jmprieur merged 2 commits into
devfrom
jmprieur/Percentiles

Conversation

@jmprieur

Copy link
Copy Markdown
Contributor

Adding P50, P90 and P100 percentiles to benchmarks

Adds 3 columns to the benchmark summaries to display the P50, P90 and P100 percentiles

Description

The benchmark summaries now contains the pecentiles

Runtime = ; GC =
Mean = 10.711 us, StdErr = 0.057 us (0.53%), N = 29, StdDev = 0.308 us
Min = 10.261 us, Q1 = 10.438 us, Median = 10.682 us, Q3 = 10.955 us, Max = 11.354 us
IQR = 0.517 us, LowerFence = 9.662 us, UpperFence = 11.730 us
ConfidenceInterval = [10.500 us; 10.921 us] (CI 99.9%), Margin = 0.210 us (1.96% of Mean)
Skewness = 0.36, Kurtosis = 1.97, MValue = 2
-------------------- Histogram --------------------
[10.235 us ; 10.532 us) | @@@@@@@@@
[10.532 us ; 10.795 us) | @@@@@@@@@@@
[10.795 us ; 11.203 us) | @@@@@@@@
[11.203 us ; 11.486 us) | @
---------------------------------------------------

// * Summary *

BenchmarkDotNet=v0.13.5, OS=Windows 11 (10.0.22631.2715)
Intel Core i7-1065G7 CPU 1.30GHz, 1 CPU, 8 logical and 4 physical cores
.NET SDK=8.0.100
  [Host] : .NET 6.0.25 (6.0.2523.51912), X64 RyuJIT AVX2

Job=MediumRun  Toolchain=InProcessEmitToolchain  IterationCount=15
LaunchCount=2  WarmupCount=10
Method Mean Error StdDev P50 P99 P100 Gen0 Allocated
JsonWebTokenHandler_ValidateTokenAsync 1.471 us 0.0600 us 0.0898 us 1.46 us 1.57 us 1.66 us 0.2747 1.13 KB
JwtSecurityTokenHandler_ValidateTokenAsync 10.711 us 0.2103 us 0.3083 us 10.68 us 11.15 us 11.35 us 0.3204 1.34 KB

Fixes #2410

@jennyf19 jennyf19 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome!
:shipit:

Comment thread benchmark/Microsoft.IdentityModel.Benchmarks/AntiVirusFriendlyConfig.cs Outdated
Comment thread benchmark/Microsoft.IdentityModel.Benchmarks/AntiVirusFriendlyConfig.cs Outdated
namespace Microsoft.IdentityModel.Benchmarks
{
// Define custom columns for P50 and P99 latency
public class P50Column : IColumn

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://benchmarkdotnet.org/articles/features/statistics.html

Most of these look like they already exist in StatisticColumn?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. The columns explicitly get the ones we want and not the others

@keegan-caruso keegan-caruso left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few comments are off, otherwise LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Benchmarks should display P90 and P99,9 percentiles

3 participants