Fail the tests when an analyzer throws - #1349
Merged
Merged
Conversation
VerifyDiagnosticResults filters the actual diagnostics by the configured default rule id before counting them. Roslyn surfaces an analyzer exception as an AD0001 diagnostic, so that filter discarded it and a test expecting no diagnostic stayed green even though the analyzer crashed. Pass CompilationWithAnalyzersOptions with an onAnalyzerException handler that records the exception, and fail right after GetAnalyzerDiagnosticsAsync, before any filtering and on the code fix paths too. The message contains the analyzer type and the full exception, which is more actionable than the AD0001 message. Keep a check on the AD0001 diagnostics as a safety net.
This was referenced Aug 27, 2026
Closed
Closed
Bump Meziantou.Analyzer from 3.0.139 to 3.0.189
Analogy-LogViewer/Analogy.LogViewer.NLog.Targets#556
Closed
Closed
Bump Meziantou.Analyzer from 3.0.139 to 3.0.189
Analogy-LogViewer/Analogy.AspNetCore.LogProvider#542
Closed
IhateTrains
pushed a commit
to ParadoxGameConverters/ImperatorToCK3
that referenced
this pull request
Aug 27, 2026
Updated [Meziantou.Analyzer](https://github.com/meziantou/Meziantou.Analyzer) from 3.0.177 to 3.0.184. <details> <summary>Release notes</summary> _Sourced from [Meziantou.Analyzer's releases](https://github.com/meziantou/Meziantou.Analyzer/releases)._ ## 3.0.184 NuGet package: <https://www.nuget.org/packages/Meziantou.Analyzer/3.0.184> ## What's Changed * Compute the package version from the git tags instead of nuget.org by @meziantou in meziantou/Meziantou.Analyzer#1350 **Full Changelog**: meziantou/Meziantou.Analyzer@3.0.183...3.0.184 ## 3.0.183 NuGet package: <https://www.nuget.org/packages/Meziantou.Analyzer/3.0.183> ## What's Changed * Fix MA0026 crash on an unterminated block comment (#1328) by @meziantou in meziantou/Meziantou.Analyzer#1346 * Fix MA0050 code fixer producing an extension local function (CS1106) by @meziantou in meziantou/Meziantou.Analyzer#1348 * Fix MA0028 code fixer producing uncompilable code (#1327) by @meziantou in meziantou/Meziantou.Analyzer#1347 * Fail the tests when an analyzer throws by @meziantou in meziantou/Meziantou.Analyzer#1349 **Full Changelog**: meziantou/Meziantou.Analyzer@3.0.182...3.0.183 ## 3.0.182 NuGet package: <https://www.nuget.org/packages/Meziantou.Analyzer/3.0.182> ## What's Changed * Fix MA0073 code fixer producing uncompilable code for low-precedence operands by @meziantou in meziantou/Meziantou.Analyzer#1344 * Do not crash on an invalid regex in the .editorconfig (MA0003, MA0104) by @meziantou in meziantou/Meziantou.Analyzer#1345 **Full Changelog**: meziantou/Meziantou.Analyzer@3.0.181...3.0.182 ## 3.0.181 NuGet package: <https://www.nuget.org/packages/Meziantou.Analyzer/3.0.181> ## What's Changed * Add a language attribute code fix and MA0218/MA0219 for XML comments by @meziantou in meziantou/Meziantou.Analyzer#1320 * Compile the text produced by the fixer in the test harness by @meziantou in meziantou/Meziantou.Analyzer#1342 * Fix StackOverflow crash on cyclic local initializers (MA0091/MA0092/MA0093) by @meziantou in meziantou/Meziantou.Analyzer#1341 * Avoid the Sqlite data-flow analysis on every MA0042 invocation by @meziantou in meziantou/Meziantou.Analyzer#1343 **Full Changelog**: meziantou/Meziantou.Analyzer@3.0.180...3.0.181 ## 3.0.180 NuGet package: <https://www.nuget.org/packages/Meziantou.Analyzer/3.0.180> ## What's Changed * Fix MAS0004 stopping at the first unrelated CA1507 diagnostic by @meziantou in meziantou/Meziantou.Analyzer#1340 **Full Changelog**: meziantou/Meziantou.Analyzer@3.0.179...3.0.180 ## 3.0.179 NuGet package: <https://www.nuget.org/packages/Meziantou.Analyzer/3.0.179> ## What's Changed * Fix bare `#pragma warning disable` in AvoidUsingRedundantElseAnalyzer by @meziantou in meziantou/Meziantou.Analyzer#1339 **Full Changelog**: meziantou/Meziantou.Analyzer@3.0.178...3.0.179 ## 3.0.178 NuGet package: <https://www.nuget.org/packages/Meziantou.Analyzer/3.0.178> ## What's Changed * Fix MA0075 false positive on FormattableString.Invariant by @meziantou in meziantou/Meziantou.Analyzer#1322 **Full Changelog**: meziantou/Meziantou.Analyzer@3.0.177...3.0.178 Commits viewable in [compare view](meziantou/Meziantou.Analyzer@3.0.177...3.0.184). </details> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This was referenced Aug 27, 2026
Closed
Closed
This was referenced Sep 21, 2026
Closed
Closed
Bump Meziantou.Analyzer from 3.0.139 to 3.0.270
Analogy-LogViewer/Analogy.LogViewer.NLog.Targets#581
Closed
Closed
Bump Meziantou.Analyzer from 3.0.139 to 3.0.270
Analogy-LogViewer/Analogy.AspNetCore.LogProvider#562
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
chore(deps): Bump the nuget-minor-and-patch group with 2 updates
donislawdev/BetterWindowsServices#3
Merged
Closed
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1331
What changed
VerifyDiagnosticResultsfilters the actual diagnostics by the configured default rule id before counting them. Roslyn surfaces an analyzer exception as anAD0001diagnostic withLocation.None:GetSortedDiagnosticsFromDocumentsdid add it to the list, and this filter then discarded it. In a test that expects zero diagnostics, expected and actual were both0and the test was green even though the analyzer threw.The harness now passes
CompilationWithAnalyzersOptionswith anonAnalyzerExceptionhandler that records the exception, andAssertNoAnalyzerExceptionfails right afterGetAnalyzerDiagnosticsAsync— before theDefaultAnalyzerIdfilter, before sorting, and on the code fix paths too, as all of them go throughGetSortedDiagnosticsFromDocuments. The failure message contains the analyzer type and the full exception with its stack trace, which is more actionable than theAD0001message. A check on theAD0001diagnostics remains as a safety net in case Roslyn ever reports a failure without invoking the handler.VerifyDiagnostic_ReportsAnalyzerExceptioncovers it: an analyzer that throws, registered withid:so theDefaultAnalyzerIdfiltering is active, and no expected diagnostic — the exact shape that was silently green before.Verification
dotnet test --max-parallel-test-modules 2: 18789 passed, 0 failed, for the 5 Roslyn versions (4.8, 4.14, 5.0, 5.6, 5.9). TheCompilationWithAnalyzersOptionsconstructor overload works on 4.8.main(60df5ba, c6054ab).dotnet run --project src/DocumentationGeneratorexits 0 and does not change any markdown file, as this is a test-only change.Not included: the same filter hiding a second rule
The issue also mentions that the
DefaultAnalyzerIdfilter hides a wrong second rule from the multi-descriptor analyzers. Removing the filter breaks 27 tests on roslyn5.9:OptimizeLinqUsageAnalyzer(12),UseStringComparisonAnalyzer(5),UseLangwordInXmlComment(3),ConcurrentDictionaryMustPreventClosureWhenAccessingTheKey(3), and 4 others. Each one needs a decision on whether the second diagnostic is expected or a false positive, so this PR leaves it out rather than blanket-annotating them. Happy to do it in a follow-up.