perf: enable Compiled regex for dynamic metadata patterns - #325
Merged
Conversation
PhoneRegex caches the per-region/per-format patterns loaded from metadata (number formats, national-prefix-for-parsing, leading digits, IDD prefix, etc.) — these are exercised on every Parse / IsValidNumber / Format call. They were created with CultureInvariant only (interpreted) since the 2023 "reduce regex abuse" change. Switch to InternalRegexOptions.Default (Compiled | CultureInvariant). The cache is bounded (one Regex per unique pattern, ~few thousand max) and entries live for process lifetime, so the per-pattern JIT cost is amortized after first use. Patterns are still constructed lazily, so cold-start cost only materializes for patterns actually exercised. Under NativeAOT, Compiled silently falls back to interpreted, so this change is AOT-safe. Source-link InternalRegexOptions.cs into MetadataBuilder so its copy of PhoneRegex still compiles. PhoneNumberWorkflowBenchmark on net9.0 (Apple M3): PhoneNumberCount Baseline Compiled Δ 1000 1.365 ms 1.208 ms -11.5% 10000 13.796 ms 11.817 ms -14.3% 100000 137.846 ms 118.501 ms -14.0% Allocations unchanged. Full test suite (348 tests) passes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Owner
Author
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #325 +/- ##
=======================================
Coverage 75.31% 75.31%
=======================================
Files 38 38
Lines 4638 4638
Branches 1097 1097
=======================================
Hits 3493 3493
Misses 917 917
Partials 228 228 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This was referenced Aug 29, 2026
pull Bot
pushed a commit
to justinwritescode/libphonenumber-csharp
that referenced
this pull request
Aug 31, 2026
Every existing benchmark here either repeats one region (cheap after the first call) or warms its whole diverse region set in GlobalSetup before the timed run starts (PhoneNumberWorkflowBenchmark's seed-data generation calls GetExampleNumberForType/IsValidNumber/Format against every supported region). Neither shape can see the cost of a region's genuinely first use in the process, which is exactly where two real regressions lived: the 2017 RegexOptions.Compiled + undersized RegexCache issue (~115x, fixed by PR #161) and the 2026 PR twcclegg#325 Compiled-regex change (~100x cold-start cost per new region, not caught by its own benchmark because that benchmark's setup already pre-warms every region it measures). Add ColdStartBenchmark.FirstUseValidateAndFormat: fresh PhoneNumberUtil per iteration, a fixed 20-region list GlobalSetup never touches, one previously-unseen region's full Parse+IsValidNumber+Format per invocation. Locally: 19.6ms median vs FirstRegionLookup's 300us (metadata-load-only), cleanly separating the regex-compile cost from the metadata-load cost this class already measured. No wiring changes needed - run_performance_tests.yml already runs `--filter "*"` for both branch and base and diffs every case via lib/compare-benchmarks.js's Welch's-t-test + 20%-floor comparison, so the new case is covered automatically. Documented the two-regression history and the "why" for this specific benchmark shape in README.md so a future benchmark addition doesn't accidentally drop the property that makes this one work.
twcclegg
pushed a commit
that referenced
this pull request
Aug 31, 2026
PR #325 made every PhoneRegex pattern build with RegexOptions.Compiled, improving steady-state throughput ~12-14% but making a region's first touch pay the RegexOptions.Compiled IL-emit JIT cost synchronously (~30ms/pattern) -- cold start across many never-before-touched regions went from ~100ms to ~620-700ms (see ColdStartBenchmark.FirstUse*). Redesign PhoneRegex so each pattern starts out built interpreted (cheap, no JIT emit). A per-pattern Interlocked usage counter tracks reuse; once a pattern crosses PromotionThreshold=2 uses, a background Task.Run compiles the same pattern with RegexOptions.Compiled and atomically swaps it in via Volatile.Write for subsequent callers -- callers already in flight keep using whichever Regex they fetched. Exactly one background compile is ever kicked off per pattern (Interlocked compare-exchange guard). The obsolete PhoneRegex(pattern, options) constructor keeps its old fixed-options behavior and opts out of promotion entirely. Threshold of 2 was chosen empirically (see the XML doc on PromotionThreshold): both 2 and 3 give FirstUse* benchmarks the full cold-start win (a pattern touched once never promotes at either value), but 3 measurably regressed PhoneNumberWorkflowBenchmark on this codebase's specific reuse shape (GlobalSetup touches each pattern once, the timed loop only reuses it a handful more times), while 2 lands back within noise of the always-compiled baseline. Also adds PhoneNumberUtil.PrewarmRegionsAsync(regionCodes) -- an optional, off-thread warm-up hook that walks Parse/IsValidNumber/Format for each region enough times to force promotion ahead of real traffic -- and an internal PhoneRegex.PrewarmAsync()/RegexHolder.ForcePromoteAsync() building block that force-promotes a pattern synchronously off the calling thread, bypassing the usage counter. PhoneRegex.cs and InternalRegexOptions.cs are source-linked standalone into PhoneNumbers.MetadataBuilder; verified that project still builds on its own. Benchmarked on this machine (BenchmarkDotNet, net10.0, medians unless noted): ColdStartBenchmark (median, before -> after): FirstUseValidateAndFormat: 49.3ms -> 0.92ms FirstUseAsYouType: 8.7ms -> 1.48ms FirstUseFindNumbers: 19.8ms -> 0.72ms FirstUseGeocode: 15.5ms -> 1.75ms (Parse-path patterns only; geocoder itself uses a prefix map, not PhoneRegex) PhoneNumberWorkflowBenchmark, PhoneNumberCount=1000 (mean, before -> after): ParseValidateAndFormatPhoneNumbers: 4.05ms -> 3.70ms ValidateOnly: 1.00ms -> 1.01ms FormatOnly: 1.58ms -> 1.50ms (only PhoneNumberCount=1000 is defined in this benchmark; 10000/100000 were deliberately trimmed in an earlier commit, "cut benchmark iterations and the redundant 10000 count") Full test suite (net8.0 + net10.0, PhoneNumbers.slnx): 419/419 passing, including a new TestPhoneRegex.cs covering the public API surface, the obsolete options constructor, cache identity, and two concurrency tests that hammer a pattern well past PromotionThreshold from multiple threads to exercise the build/promote/swap race directly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J8DATgGMQCmTpkg9dkdxks
twcclegg
added a commit
that referenced
this pull request
Sep 3, 2026
Reverts the cold-start regression from #325. Metadata-derived patterns -- everything PhoneRegex builds -- go back to interpreted; the library's own fixed regexes stay compiled, which is where compiling actually pays. RegexOptions.Compiled costs roughly 1.5 ms of IL-emit per distinct pattern and saves about 0.044 us per match, so a pattern must be matched on the order of 30,000 times before it breaks even. There are thousands of metadata patterns, each built the first time a caller touches that region, and a workload spread across regions matches each one far fewer times than that. Measured end-to-end on net8.0 -- total wall time including startup, parse + validate + format, best of 3: regions ops compiled interpreted 1 1000 35 29 1 100000 188 194 1 1000000 970 1153 20 1000 522 39 20 100000 877 529 20 1000000 2330 2455 245 1000 2284 61 245 100000 2696 642 245 1000000 5231 2967 Interpreted wins six of nine, by up to 34x. Compiling wins three, all in the concentrated high-volume corner, by 6-20%. The downside of interpreted is bounded and small; the downside of compiling is not. That asymmetry is why this is not left configurable. An opt-in nobody enables is a config surface and a second code path for no benefit, and callers who genuinely want a compiled build of a specific pattern can already pass explicit options to the PhoneRegex constructor. This is also the third time the library has shipped this regression -- 8.8.0, 8.13.0, and 9.0.30 (#325). Each time the justification came from PhoneNumberWorkflowBenchmark, whose GlobalSetup pre-warms every pattern and whose timed loop then hammers a handful: precisely the one workload shape where compiling wins. The measurements above deliberately include startup and vary region diversity, which is what that benchmark cannot see. InternalRegexOptions.Default is now derived from Interpreted, so the two sets differ by nothing except RegexOptions.Compiled and a semantic flag cannot be added to one group of regexes but not the other.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PhoneRegex caches the per-region/per-format patterns loaded from metadata (number formats, national-prefix-for-parsing, leading digits, IDD prefix, etc.) — these are exercised on every Parse / IsValidNumber / Format call. They were created with CultureInvariant only (interpreted) since the 2023 "reduce regex abuse" change.
Switch to InternalRegexOptions.Default (Compiled | CultureInvariant). The cache is bounded (one Regex per unique pattern, ~few thousand max) and entries live for process lifetime, so the per-pattern JIT cost is amortized after first use. Patterns are still constructed lazily, so cold-start cost only materializes for patterns actually exercised. Under NativeAOT, Compiled silently falls back to interpreted, so this change is AOT-safe.
Source-link InternalRegexOptions.cs into MetadataBuilder so its copy of PhoneRegex still compiles.
PhoneNumberWorkflowBenchmark on net9.0 (Apple M3):
PhoneNumberCount Baseline Compiled Δ
1000 1.365 ms 1.208 ms -11.5%
10000 13.796 ms 11.817 ms -14.3%
100000 137.846 ms 118.501 ms -14.0%
Allocations unchanged. Full test suite (348 tests) passes.
Changes