Slice source text instead of copying it line by line for outlining - #20443
Slice source text instead of copying it line by line for outlining#20443xperiandri wants to merge 11 commits into
Conversation
❗ Release notes requiredYou can open this PR in browser to add release notes: open in github.dev
|
BenchmarkBenchmarkDotNet, both versions of the changed code copied verbatim into a standalone harness (the compiler-internal helpers they use are Inputs are two real files from this repo:
"Build + scan" is what one outlining pass pays outside the AST walk, and the editor pays it per keystroke. On the large file that is 4.4 MB of garbage per pass before, 1.0 MB after, a 77% reduction; on the small file 76%. Where the allocations went:
Variance is high on the multi-modal rows (the harness reports bimodal distributions on the scan benchmarks, and the full-pass rows carry a wide StdDev), so treat the time ratios as order-of-magnitude rather than precise. The allocation numbers are exact and are the substance of the change. Two caveats worth stating plainly:
|
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
793ffee to
8e642e7
Compare
This comment has been minimized.
This comment has been minimized.
|
|
||
| /// Returns outlining ranges for given parsed input. | ||
| val getOutliningRanges: sourceLines: string[] -> parsedInput: ParsedInput -> seq<ScopeRange> | ||
| val getOutliningRanges: sourceLines: ReadOnlyMemory<char>[] -> parsedInput: ParsedInput -> seq<ScopeRange> |
There was a problem hiding this comment.
🤖🕵️ Keep the public string[] getOutliningRanges entry point. Add a separately named internal memory-based entry for the editor and share the scanner implementation.
There was a problem hiding this comment.
Done in ac31157505.
getOutliningRanges keeps its string[] parameter and maps to memory; getOutliningRangesFromLineSlices is internal and takes the slices, and BlockStructureService calls that one. Both go through the same scanner — the wrapper is one line.
Two things fall out of this, both good: the surface-area baseline and the StructureTests call sites go back to exactly what they were on main, and the release note moves from Breaking Changes to Improved. PR description updated accordingly.
Build clean, StructureTests 42/42 through the public entry point.
1e21647 to
b937173
Compare
The editor's block structure built the sourceLines array for Structure.getOutliningRanges by calling ToString() per line, allocating a fresh string for the entire file on every outlining pass, once per keystroke. getOutliningRanges now takes ReadOnlyMemory<char>[] and slices the already-materialized source text once (SourceText.GetLinesAsMemory()) instead. ReadOnlySpanCharExtensions in illib mirrors the existing Ordinal string helpers so span call sites read the same way string call sites do. A local recursive function closing over a ReadOnlySpan<char>-typed sibling cannot be compiled - the CLR disallows instantiating FSharpFunc<ReadOnlySpan<char>, _> as a closure field (FS0412) - so commentTypeOf moves to module scope, next to the CommentType it classifies. StructureTests.fs slices its own lines the same way at the call site, and FSharp.Compiler.Service.Tests needs a direct System.Memory PackageReference: FSharp.Compiler.Service's own reference to it is only transitive through the net472 ProjectReference's SetTargetFramework override, mirroring the FSharp.Core pin already in this project for the same reason. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
CommentList kept a copy of every comment line next to its line number, but the number alone identifies the line in the source array the function already holds, and only the first and last lines of a group are ever read back to compute the fold's columns. Store the numbers and index the source at the end, so grouping comments allocates no tuple per line. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
CheckCodeFormatting flagged illib.fsi for a stray space before the colon in the ReadOnlySpanCharExtensions signatures; dotnet fantomas fixes it mechanically, no signature changes. check_release_notes also requires an entry for changes under vsintegration/src. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Plain_Build_Windows and Plain_Build_Linux both failed with NU1510: on the .NET Core inner build System.Memory ships with the framework, and NuGet's package-pruning check treats an unconditional explicit PackageReference to it as an error. The pin is only needed on net472, where FSharp.Compiler.Service's own PackageReference to System.Memory doesn't flow through the netstandard2.0 SetTargetFramework override. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Wrap commentTypeOf's doc comment in <summary>, move the FS0412 rationale into <remarks>, and reference the types through <see cref> rather than inline code spans. Use the shorthand lambda for the whitespace check, per review suggestion. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Slicing before searching makes the result relative to the slice, while the String siblings these mirror return an index into the whole string. A call ported from the string path would land a column short by startIndex, and "not found" would come back as -1 from the slice rather than from the string. Nothing calls them. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Restores the startIndex overloads dropped in c9fbf5a, this time reporting the position in the span they were given rather than in the slice they searched, which is what the String siblings they mirror return. A miss still comes back as -1 rather than as startIndex - 1. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A doc comment that carries markup like <see>/<paramref> needs that text inside <summary> - otherwise it renders as raw text in the generated XML and in tooltips. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
getCommentRanges recurses once per line, threading a three-way state through every call; a reference tuple heap-allocates on each of those recursive calls, a struct tuple doesn't. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Changing the signature made every FSharp.Compiler.Service consumer of the outlining API pay for a caller the editor alone has. The scanner now sits behind two entry points: the public one keeps its string[] parameter and maps to memory, and getOutliningRangesFromLineSlices takes the slices the editor already holds. The public surface is unchanged, so the surface-area baseline and the StructureTests calls return to what they were, and the release note moves out of Breaking Changes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ac31157 to
9546828
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Reverts the copies of dotnet#20443 that came in with an earlier rebase: "Slice source text instead of copying it line by line for outlining" and "Track comment lines by number instead of storing their text". They changed the public FSharp.Compiler.Service surface (Structure.getOutliningRanges took ReadOnlyMemory<char>[]) and added a System.Memory reference that fails restore with NU1510 - neither belongs here, and dotnet#20443 carries that work on its own. The Copilot snippets go back to the string[] lines Structure.getOutliningRanges takes on main. Everything outside the Copilot files is now as on main. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
🔍 Tooling Safety Check — Affects-Design-Time
|
Description
The editor's block structure built the
sourceLinesarray forStructure.getOutliningRangesby callingToString()on every line, allocating a fresh string for the whole file on each outlining pass, that is once per keystroke.The scanner now takes
ReadOnlyMemory<char>[]and sits behind two entry points:getOutliningRangeskeeps its publicstring[]signature and maps to memory, soFSharp.Compiler.Serviceconsumers are unaffected;getOutliningRangesFromLineSlicesis internal and takes the slices the caller already holds. The editor slices the materialized source text once through a newSourceText.GetLinesAsMemory()helper and calls this one.Inside the scanner, comment detection trims and classifies lines over spans instead of building trimmed strings, and comment groups track line numbers only, since only the first and last lines of a group are ever read back to compute the fold's columns.
ReadOnlySpanCharExtensionsinillibmirrors the existing ordinalStringhelpers so span call sites read the same way.No public API change: the surface-area baseline is untouched.
Fixes # (no issue)
Checklist
StructureTests(39 cases) exercise every fold kind through the shared scanner, reached via the public entry point.docs/release-notes/.FSharp.Compiler.Service/11.0.100.md(Improved) anddocs/release-notes/.VisualStudio/18.vNext.md.🤖 Generated with Claude Code