Skip to content

Scan for line breaks in a single pass in SpanLineEnumerator - #682

Merged
SimonCropp merged 1 commit into
mainfrom
perf/span-line-enumerator
Oct 12, 2026
Merged

SimonCropp merged 1 commit into
mainfrom
perf/span-line-enumerator

Conversation

@SimonCropp

Copy link
Copy Markdown
Owner

SpanLineEnumerator.MoveNext (the EnumerateLines polyfill, active below net6) searched for the next line break with remaining.IndexOfAny(newlines) over six values. On runtimes where the span IndexOfAny(values) is not vectorized (the System.Memory package on .NET Framework / netstandard2.0) that call scans the whole remainder once per value and keeps the lowest hit. \n is the last value in the list, so on LF-only text every MoveNext rescanned the rest of the input for the five values that never occur, making enumeration quadratic.

This replaces the call with a single forward pass that tests each char against \n, \r, \f, U+0085, U+2028 and U+2029. The set of recognised line breaks, \r\n handling and the trailing-empty-line behaviour are unchanged.

Measurements

net48, Release, x64, best of 3, enumerating 8,000 lines of ~30 chars:

input before after
LF only 2,209 ms 0.91 ms
CRLF 0.63 ms 0.45 ms

Tests

New PolyfillTests_SpanLineEnumerator.cs (runs against the polyfill below net6 and the BCL from net6):

  • a table of inputs covering each of the six line breaks, \r\n as one break, \n\r as two, leading/trailing/consecutive breaks, the empty string, and neighbouring chars that must not split (\v, \t, U+0084, U+0086, U+2027, U+202A)
  • an exhaustive pass over every BMP char asserting exactly those six split a line
  • 5,000 LF-terminated lines (the previously quadratic shape)
  • MoveNext keeps returning false after the end

@SimonCropp SimonCropp added this to the 11.4.5 milestone Oct 12, 2026
@SimonCropp
SimonCropp merged commit f33f077 into main Oct 12, 2026
5 checks passed
@SimonCropp
SimonCropp deleted the perf/span-line-enumerator branch October 12, 2026 12:04
This was referenced Oct 12, 2026
This was referenced Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant