Skip to content

Measure the parse, validate and format phases separately - #378

Merged
twcclegg merged 1 commit into
mainfrom
test/benchmark-phases
Aug 5, 2026
Merged

Measure the parse, validate and format phases separately#378
twcclegg merged 1 commit into
mainfrom
test/benchmark-phases

Conversation

@twcclegg

@twcclegg twcclegg commented Aug 5, 2026

Copy link
Copy Markdown
Owner

ParseValidateAndFormatPhoneNumbers measures all three phases together, so the ~580 B allocated per
number is a total with no attribution — and the two candidates want opposite fixes: Parse allocates
through StringBuilder round trips, Format through a regex replacement chain that produces a fresh
string per step.

Adds ParseOnly, ValidateOnly and FormatOnly over the same data, pre-parsed in setup so the last
two measure only their own work. FormatOnly uses INTERNATIONAL because E164 takes an early exit
that skips pattern formatting entirely.

The end-to-end benchmark stays — parsing warms metadata the later phases reuse, so the whole isn't the
sum. All four share the existing class, since BenchmarkDotNet builds a project per class (~20s).

Lands separately from the optimisation: the comparison runs the base commit's suite, so a benchmark
added alongside the change it measures reports nothing.

@codecov

codecov Bot commented Aug 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.01%. Comparing base (ab514e1) to head (209e4a3).

Additional details and impacted files 
@@           Coverage Diff           @@
##             main     #378   +/-   ##
=======================================
  Coverage   98.01%   98.01%           
=======================================
  Files          39       39           
  Lines       52818    52818           
  Branches     1105     1105           
=======================================
  Hits        51770    51770           
  Misses        795      795           
  Partials      253      253

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@twcclegg
twcclegg force-pushed the test/benchmark-phases branch from f342a6d to 209e4a3 Compare August 5, 2026 22:10
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown

📊 Benchmark Results

Commit: 209e4a3 · Full run · Linux ubuntu-24.04-arm

Both sides were measured on the same runner in the same job, so the numbers are
comparable. Treat sub-percent differences as noise.

PR branch 

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
Neoverse-N2, 4 physical cores
.NET SDK 10.0.302
  [Host]    : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a
  .NET 10.0 : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a

Job=.NET 10.0  Runtime=.NET 10.0
Method PhoneNumberCount Mean Error StdDev Gen0 Allocated
InputDigitPerKeystroke 1000 4.273 ms 0.0097 ms 0.0086 ms 54.6875 3.87 MB 

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
Neoverse-N2, 4 physical cores
.NET SDK 10.0.302
  [Host]     : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a
  Job-AMQORM : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a

Runtime=.NET 10.0  InvocationCount=1  IterationCount=20  
LaunchCount=1  RunStrategy=ColdStart  UnrollFactor=1  
WarmupCount=1
Method Mean Error StdDev Allocated
CreateInstance 415.6 μs 103.6 μs 119.3 μs 119.48 KB
CreateInstanceAndLoadAllRegions 6,983.0 μs 367.2 μs 422.9 μs 1619.1 KB
FirstRegionLookup 439.6 μs 135.9 μs 156.5 μs 124.54 KB 

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
Neoverse-N2, 4 physical cores
.NET SDK 10.0.302
  [Host]    : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a
  .NET 10.0 : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a

Job=.NET 10.0  Runtime=.NET 10.0
Method PhoneNumberCount Mean Error StdDev Gen0 Allocated
ExtractPossibleNumber_CleanInput 1000 20.93 μs 0.020 μs 0.019 μs - -
ExtractPossibleNumber_WithLeadingJunk 1000 38.29 μs 0.047 μs 0.040 μs 0.6714 48360 B 

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
Neoverse-N2, 4 physical cores
.NET SDK 10.0.302
  [Host]    : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a
  .NET 10.0 : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a

Job=.NET 10.0  Runtime=.NET 10.0
Method PhoneNumberCount Mean Error StdDev Gen0 Allocated
FindNumbers_Valid 100 141.6 μs 0.39 μs 0.34 μs 0.9766 69.93 KB
FindNumbers_StrictGrouping 100 303.5 μs 0.68 μs 0.64 μs 1.4648 123.2 KB 

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
Neoverse-N2, 4 physical cores
.NET SDK 10.0.302
  [Host]    : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a
  .NET 10.0 : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a

Job=.NET 10.0  Runtime=.NET 10.0
Method PhoneNumberCount Mean Error StdDev Gen0 Allocated
GetDescriptionForNumber 1000 1,455.50 μs 2.486 μs 2.204 μs 1.9531 196.63 KB
GetDisplayCountry 1000 23.36 μs 0.018 μs 0.017 μs 0.0916 7.56 KB 

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
Neoverse-N2, 4 physical cores
.NET SDK 10.0.302
  [Host]    : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a
  .NET 10.0 : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a

Job=.NET 10.0  Runtime=.NET 10.0
Method PhoneNumberCount Mean Error StdDev Gen0 Allocated
ParseValidateAndFormatPhoneNumbers 1000 2,472.2 μs 7.32 μs 6.49 μs 7.8125 580.51 KB
ParseOnly 1000 431.6 μs 0.52 μs 0.46 μs 4.8828 350.95 KB
ValidateOnly 1000 763.7 μs 1.23 μs 1.03 μs - 41.11 KB
FormatOnly 1000 1,041.1 μs 1.93 μs 1.61 μs 1.9531 187.91 KB
PR base 

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
Neoverse-N2, 4 physical cores
.NET SDK 10.0.302
  [Host]    : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a
  .NET 10.0 : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a

Job=.NET 10.0  Runtime=.NET 10.0
Method PhoneNumberCount Mean Error StdDev Gen0 Allocated
InputDigitPerKeystroke 1000 4.324 ms 0.0049 ms 0.0043 ms 54.6875 3.87 MB 

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
Neoverse-N2, 4 physical cores
.NET SDK 10.0.302
  [Host]     : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a
  Job-AMQORM : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a

Runtime=.NET 10.0  InvocationCount=1  IterationCount=20  
LaunchCount=1  RunStrategy=ColdStart  UnrollFactor=1  
WarmupCount=1
Method Mean Error StdDev Allocated
CreateInstance 398.7 μs 112.0 μs 129.0 μs 119.48 KB
CreateInstanceAndLoadAllRegions 7,000.9 μs 379.8 μs 437.4 μs 1619.1 KB
FirstRegionLookup 452.6 μs 134.5 μs 154.9 μs 124.54 KB 

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
Neoverse-N2, 4 physical cores
.NET SDK 10.0.302
  [Host]    : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a
  .NET 10.0 : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a

Job=.NET 10.0  Runtime=.NET 10.0
Method PhoneNumberCount Mean Error StdDev Gen0 Allocated
ExtractPossibleNumber_CleanInput 1000 21.56 μs 0.021 μs 0.020 μs - -
ExtractPossibleNumber_WithLeadingJunk 1000 38.04 μs 0.086 μs 0.076 μs 0.6714 48360 B 

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
Neoverse-N2, 4 physical cores
.NET SDK 10.0.302
  [Host]    : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a
  .NET 10.0 : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a

Job=.NET 10.0  Runtime=.NET 10.0
Method PhoneNumberCount Mean Error StdDev Gen0 Allocated
FindNumbers_Valid 100 140.0 μs 0.42 μs 0.37 μs 0.9766 69.93 KB
FindNumbers_StrictGrouping 100 307.2 μs 3.31 μs 3.10 μs 1.4648 123.2 KB 

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
Neoverse-N2, 4 physical cores
.NET SDK 10.0.302
  [Host]    : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a
  .NET 10.0 : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a

Job=.NET 10.0  Runtime=.NET 10.0
Method PhoneNumberCount Mean Error StdDev Gen0 Allocated
GetDescriptionForNumber 1000 1,377.73 μs 2.224 μs 1.971 μs 1.9531 196.63 KB
GetDisplayCountry 1000 17.31 μs 0.060 μs 0.053 μs 0.0916 7.56 KB 

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
Neoverse-N2, 4 physical cores
.NET SDK 10.0.302
  [Host]    : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a
  .NET 10.0 : .NET 10.0.10 (10.0.10, 10.0.1026.32716), Arm64 RyuJIT armv8.0-a

Job=.NET 10.0  Runtime=.NET 10.0
Method PhoneNumberCount Mean Error StdDev Gen0 Allocated
ParseValidateAndFormatPhoneNumbers 1000 2.495 ms 0.0048 ms 0.0040 ms 7.8125 580.51 KB

@twcclegg
twcclegg merged commit ee09ac6 into main Aug 5, 2026
10 of 11 checks passed
@twcclegg
twcclegg deleted the test/benchmark-phases branch August 5, 2026 22:30
@twcclegg twcclegg mentioned this pull request Aug 5, 2026
Closed
This was referenced Aug 14, 2026
Closed
Closed
Closed
Merged
Closed
Closed
Open
Closed
Closed
This was referenced Aug 23, 2026
Closed
Closed
Closed
Closed
Merged
Closed
Closed
Closed
Closed
Closed
Closed
Open
This was referenced Aug 31, 2026
Closed
Open
This was referenced Sep 1, 2026
Open
Open
Open
Open
Merged
Merged
Merged
Merged
Merged
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@twcclegg