Compile the text produced by the fixer in the test harness - #1342
Merged
Merged
Conversation
ApplyFix compiled the solution produced by the CodeAction, so it validated the syntax tree built by the fixer instead of its serialization. The text compared against the expected fixed code is only produced afterwards, by Simplifier.ReduceAsync and Formatter.Format, and was never compiled. A fixer building a structurally valid tree whose text re-parses differently (a missing set of parentheses, for instance) therefore passed, even though the text is what the user gets. ApplyFix now runs every changed or added document through GetStringFromDocument and puts that text back into the solution before compiling it, so the trees that are compiled are the ones parsed from the text of the fix. Add a self-test for the harness with a test-only analyzer and two test-only fixers: one that negates an is-expression without parenthesizing it, whose text re-parses as '(!o) is string' and must be rejected, and the correct one, which must still pass.
meziantou
enabled auto-merge (squash)
August 26, 2026 18:37
This was referenced Aug 26, 2026
Closed
Bump Meziantou.Analyzer from 3.0.103 to 3.0.182
Analogy-LogViewer/Analogy.LogViewer.OpenTelemetry#98
Closed
Closed
Closed
Closed
Bump Meziantou.Analyzer from 3.0.139 to 3.0.189
Analogy-LogViewer/Analogy.LogViewer.NLog.Targets#556
Closed
This was referenced Sep 21, 2026
Closed
Closed
Bump Meziantou.Analyzer from 3.0.139 to 3.0.270
Analogy-LogViewer/Analogy.LogViewer.NLog.Targets#581
Closed
Closed
Bump Meziantou.Analyzer from 3.0.139 to 3.0.270
Analogy-LogViewer/Analogy.AspNetCore.LogProvider#562
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
chore(deps): Bump the nuget-minor-and-patch group with 2 updates
donislawdev/BetterWindowsServices#3
Merged
Closed
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1330
Problem
ApplyFixcalledcompilation.Emit(...)on the solution produced directly by theCodeAction, so it validated the syntax tree built by the fixer. The text compared against the expected fixed code is produced only afterwards, bySimplifier.ReduceAsyncandFormatter.FormatinGetStringFromDocument— and that text was never compiled.Any fixer that builds a structurally valid tree whose serialization re-parses differently therefore passed. Operator-precedence and parenthesization bugs are the most common way a Roslyn fixer corrupts user code, and the harness was structurally blind to all of them across the 109 fixers.
Change
ApplyFixnow runs every document the code action changed or added throughGetStringFromDocumentand puts that text back into the solution withWithDocumentTextbefore compiling. The trees that get compiled are the ones parsed from the text of the fix — the text the user actually gets. The failure message prints the document text rather than the raw syntax root.A self-test for the harness is added in
ProjectBuilderValidationTests, with a test-only analyzer onisexpressions and two test-only fixers:NegateWithoutParenthesesFixerbuilds!applied to the is-expression — a valid tree whose text is!o is string, which re-parses as(!o) is stringand does not compile. The test asserts the harness now fails withThe fixed code doesn't compile.NegateWithParenthesesFixeris the correct version, and the test asserts a valid fix still passes.I checked that the first test fails when
ApplyFixis reverted to its previous behavior, so it genuinely guards the change.Notes for the reviewer
The issue expected this to surface more existing fixer bugs. It does not: all 18575 tests pass across the five Roslyn versions (4.8, 4.14, 5.0, 5.6, 5.9). The MA0073 fixer bug used as the demonstration in the issue is not covered by any existing test case — no test compares an
isexpression with a bool constant — so it stays a live bug for its own issue rather than turning red here.dotnet run --project src/DocumentationGeneratorproduced no markdown changes, as expected for a test-only change.