Make the analysis of generated code opt-in

AGENTS.md

@@ -55,7 +55,7 @@ The analyzer must use `IOperation` or `ISymbol` to analyze the content. Only fal
 
 ### Generated code
 
-The analyzers analyze generated code, so they must call `context.ConfigureAnalysisOfGeneratedCode(...)` in `Initialize` instead of `ConfigureGeneratedCodeAnalysis`, which `BannedSymbols.txt` makes a compilation error: the flags passed to it are the minimum, and `Analyze | ReportDiagnostics` is added to them unless the `MEZIANTOU_ANALYZER_GENERATED_CODE` environment variable opts out, as analyzing generated code has a cost. Pass `GeneratedCodeAnalysisFlags.None` unless the rule needs generated code to be correct even when a user opts out (`Analyze`), or reports in generated code by default (`Analyze | ReportDiagnostics`). The diagnostics located in generated code are filtered when they are reported. The filtering is done by `GeneratedCodeReporting`, which sets the `DiagnosticReporter.CanReportDiagnostic` filter of `Meziantou.Framework.Roslyn` from a module initializer, so the rules must report their diagnostics with the `ReportDiagnostic` extension methods of the analysis contexts or with a `DiagnosticReporter`; `BannedSymbols.txt` makes reporting directly on a Roslyn context a compilation error, as it would bypass the filter. Add `customTags: [GeneratedCodeReporting.ReportInGeneratedCodeTag]` to the descriptor of the rules that must report in generated code by default, such as the rules whose subject is the generated file itself, use `GeneratedCodeAnalysisFlags.Analyze | GeneratedCodeAnalysisFlags.ReportDiagnostics` in their analyzer, and run `dotnet run --project src/DocumentationGenerator` to update the list in [`docs/generated-code.md`](/docs/generated-code.md).
+The analyzers skip generated code, so they must call `context.ConfigureAnalysisOfGeneratedCode(...)` in `Initialize` instead of `ConfigureGeneratedCodeAnalysis`, which `BannedSymbols.txt` makes a compilation error: the flags passed to it are the minimum, and `Analyze | ReportDiagnostics` is added to them only when the `MEZIANTOU_ANALYZER_GENERATED_CODE` environment variable opts in, as analyzing generated code has a cost. Pass `GeneratedCodeAnalysisFlags.None` unless the rule needs generated code to be correct even when nobody opts in (`Analyze`), or reports in generated code by default (`Analyze | ReportDiagnostics`). The diagnostics located in generated code are filtered when they are reported. The filtering is done by `GeneratedCodeReporting`, which sets the `DiagnosticReporter.CanReportDiagnostic` filter of `Meziantou.Framework.Roslyn` from a module initializer, so the rules must report their diagnostics with the `ReportDiagnostic` extension methods of the analysis contexts or with a `DiagnosticReporter`; `BannedSymbols.txt` makes reporting directly on a Roslyn context a compilation error, as it would bypass the filter. Add `customTags: [GeneratedCodeReporting.ReportInGeneratedCodeTag]` to the descriptor of the rules that must report in generated code by default, such as the rules whose subject is the generated file itself, use `GeneratedCodeAnalysisFlags.Analyze | GeneratedCodeAnalysisFlags.ReportDiagnostics` in their analyzer, and run `dotnet run --project src/DocumentationGenerator` to update the list in [`docs/generated-code.md`](/docs/generated-code.md).
 
 Code snippets in tests must use raw string literals (`"""`) and must be minimized to only include the necessary code to reproduce the issue. Avoid including unnecessary code that does not contribute to the test case.
 When reporting a diagnostic, the snippet must use the `[|code|]` syntax or `{|id:code|}` syntax. Do not explicitly indicates lines or columns.

README.md

@@ -298,8 +298,19 @@ Supported values are:
 
 ## Analyzing generated code
 
-The rules analyze generated code, but most of them do not report the diagnostics located in generated code. Set
-`report_generated_code` in the `.editorconfig` file to change it, for a single rule or for all of them at once:
+The rules skip generated code, as analyzing it costs build time and you cannot fix code you do not own. Set the
+`MEZIANTOU_ANALYZER_GENERATED_CODE` environment variable to `true` (or `1`) to opt in, which makes the rules analyze
+generated code:
+
+```bash
+MEZIANTOU_ANALYZER_GENERATED_CODE=true
+```
+
+Note that the analyzers run in a long-lived process. After changing the variable, run `dotnet build-server shutdown`
+and rebuild, or restart your IDE.
+
+Opting in does not report anything by itself: `report_generated_code` decides whether a rule reports the diagnostics
+located in the generated code it analyzes, for a single rule or for all of them at once:
 
 ```ini
 [*.cs]
@@ -310,18 +321,8 @@ MA.report_generated_code = true
 MA0051.report_generated_code = false
 ```
 
-A few rules report in generated code by default, such as the Blazor rules that work on the code generated from the
-`.razor` files.
-
-Analyzing generated code costs build time. Set the `MEZIANTOU_ANALYZER_GENERATED_CODE` environment variable to
-`false` (or `0`) to opt out, which makes the rules skip generated code entirely, except the few that need it:
-
-```bash
-MEZIANTOU_ANALYZER_GENERATED_CODE=false
-```
-
-Note that the analyzers run in a long-lived process. After changing the variable, run `dotnet build-server shutdown`
-and rebuild, or restart your IDE.
+A few rules analyze generated code without the variable, and report in it by default, such as the Blazor rules that
+work on the code generated from the `.razor` files.
 
 See [Analyzing generated code](https://github.com/meziantou/Meziantou.Analyzer/blob/main/docs/generated-code.md)
 for the list of those rules and for the detection of the generated files.

docs/generated-code.md

@@ -1,14 +1,15 @@
 # Analyzing generated code
 
-The rules analyze generated code, so they see the whole compilation, but most of them do not report the diagnostics
-located in generated code, which is the expected behavior for the vast majority of projects: you cannot fix code you
-do not own. Some rules are the exception and report in generated code by default, as the generated file is the
-subject of the rule, such as the Blazor rules that work on the code generated from the `.razor` files.
+The rules skip generated code, which is the expected behavior for the vast majority of projects: you cannot fix code
+you do not own, and analyzing code nobody looks at costs build time. Some rules are the exception and analyze it
+anyway: the ones whose subject is the generated file itself, such as the Blazor rules that work on the code generated
+from the `.razor` files, and the ones that need to see the whole compilation to be correct. Set the
+`MEZIANTOU_ANALYZER_GENERATED_CODE` environment variable to opt in to analyzing generated code with every rule.
 
 ## Configuration
 
-The `report_generated_code` option indicates whether a rule reports the diagnostics located in generated code. It can
-be set for a single rule, or for all of them at once with the `MA` prefix:
+The `report_generated_code` option indicates whether a rule reports the diagnostics located in the generated code it
+analyzes. It can be set for a single rule, or for all of them at once with the `MA` prefix:
 
 ```ini
 [*.cs]
@@ -35,24 +36,23 @@ MA0051.report_generated_code = true
 When neither is set, each rule uses its own default, which is to not report in generated code, except for the rules
 below. To turn off a rule entirely, set its severity to `none` instead.
 
-## Opting out with the environment variable
+This option only decides what a rule reports, not what it analyzes, so `report_generated_code = true` has no effect on
+a rule that does not analyze generated code: it needs the environment variable below. Setting it to `false` always
+works, as a rule can only report what it is allowed to report.
 
-Analyzing generated code costs build time, and a project where nothing reports in generated code pays it for nothing.
-Set the `MEZIANTOU_ANALYZER_GENERATED_CODE` environment variable to skip generated code entirely:
+## Opting in with the environment variable
+
+Set the `MEZIANTOU_ANALYZER_GENERATED_CODE` environment variable to analyze generated code with every rule:
 
 | Value | Behavior |
 |-------|----------|
-| not set, empty, or any other value | The rules analyze generated code, and `report_generated_code` decides what they report |
-| `false` (case-insensitive) or `0` | The rules skip generated code, except the ones that need it |
-
-The variable can only remove analysis from the rules that do not need it. Two kinds of rules are unaffected: the ones
-that report in generated code by default, listed below, and the ones that need to see the whole compilation to be
-correct, such as MA0053, which reports a class that no other class inherits from and would report a false positive if
-the deriving class were declared in a generated file.
+| `true` (case-insensitive) or `1` | The rules analyze generated code, and `report_generated_code` decides what they report |
+| not set, empty, or any other value | The rules skip generated code, except the ones that need it |
 
-The option above only decides what the rules report, not what they analyze, so `report_generated_code = true` has no
-effect on a rule that skips generated code because of the variable. Setting it to `false` always works, as a rule can
-only report what it is allowed to report.
+Two kinds of rules analyze generated code without the variable: the ones that report in generated code by default,
+listed below, and the ones that need to see the whole compilation to be correct, such as MA0053, which reports a class
+that no other class inherits from and would report a false positive if the deriving class were declared in a generated
+file. `report_generated_code` works on those rules whether the variable is set or not.
 
 ## Rules reporting in generated code by default
 
@@ -99,8 +99,9 @@ written file does not make the code generated, and a partial type declared in a
 one reports only in the hand written file. Use the `generated_code` option below for the files the detection does not
 recognize.
 
-The rules that skip generated code because of the environment variable never see it, so they follow the detection of
-Roslyn instead, which also considers the symbols marked with `[GeneratedCode]` or `[DebuggerNonUserCode]` generated.
+The rules that do not analyze generated code never see it, so they follow the detection of Roslyn instead, which also
+considers the symbols marked with `[GeneratedCode]` or `[DebuggerNonUserCode]` generated. The two detections only
+differ for a rule the variable opted in.
 
 ## Using the `generated_code` option
 
@@ -121,7 +122,7 @@ generated, and `generated_code` when you want all the analyzers to treat a speci
 An analyzer must declare how it handles generated code from `Initialize(AnalysisContext)`, and the options of the
 `.editorconfig` files are not available at that point: they can only be read from the analysis callbacks, which run
 later. An environment variable is the only configuration that can be read early enough, which is why the global
-opt-out is not an `.editorconfig` option.
+opt-in is not an `.editorconfig` option.
 
 This has consequences that are worth knowing:
 

src/Meziantou.Analyzer/BannedSymbols.txt

@@ -1,5 +1,5 @@
 M:Microsoft.CodeAnalysis.Compilation.GetTypeByMetadataName(System.String); Use GetBestTypeByMetadataName instead
-M:Microsoft.CodeAnalysis.Diagnostics.AnalysisContext.ConfigureGeneratedCodeAnalysis(Microsoft.CodeAnalysis.Diagnostics.GeneratedCodeAnalysisFlags); Use ConfigureAnalysisOfGeneratedCode instead, so users can opt out of analyzing generated code
+M:Microsoft.CodeAnalysis.Diagnostics.AnalysisContext.ConfigureGeneratedCodeAnalysis(Microsoft.CodeAnalysis.Diagnostics.GeneratedCodeAnalysisFlags); Use ConfigureAnalysisOfGeneratedCode instead, so users can opt in to analyzing generated code
 M:Microsoft.CodeAnalysis.Diagnostics.SymbolAnalysisContext.ReportDiagnostic(Microsoft.CodeAnalysis.Diagnostic); Use the ReportDiagnostic methods of ContextExtensions instead, so the diagnostics located in generated code are filtered by GeneratedCodeReporting
 M:Microsoft.CodeAnalysis.Diagnostics.OperationAnalysisContext.ReportDiagnostic(Microsoft.CodeAnalysis.Diagnostic); Use the ReportDiagnostic methods of ContextExtensions instead, so the diagnostics located in generated code are filtered by GeneratedCodeReporting
 M:Microsoft.CodeAnalysis.Diagnostics.OperationBlockAnalysisContext.ReportDiagnostic(Microsoft.CodeAnalysis.Diagnostic); Use the ReportDiagnostic methods of ContextExtensions instead, so the diagnostics located in generated code are filtered by GeneratedCodeReporting

src/Meziantou.Analyzer/Internals/AnalysisContextExtensions.cs

@@ -14,11 +14,11 @@ internal static class AnalysisContextExtensions
     private static readonly GeneratedCodeAnalysisFlags AdditionalFlags = GetAdditionalFlags(ReadEnvironmentVariable());
 
     /// <summary>
-    /// Configures how the analyzer handles generated code. The rules analyze generated code and report the
-    /// diagnostics located in it, which <see cref="GeneratedCodeReporting"/> then filters, unless the
-    /// <c>MEZIANTOU_ANALYZER_GENERATED_CODE</c> environment variable opts out, as analyzing generated code has a
-    /// cost. <paramref name="defaultFlags"/> is the minimum, which opting out cannot remove: it is what the rule
-    /// needs to be correct, or to report in generated code by default.
+    /// Configures how the analyzer handles generated code. The rules skip generated code, as analyzing it has a
+    /// cost, unless the <c>MEZIANTOU_ANALYZER_GENERATED_CODE</c> environment variable opts in, in which case they
+    /// analyze it and report the diagnostics located in it, which <see cref="GeneratedCodeReporting"/> then
+    /// filters. <paramref name="defaultFlags"/> is the minimum, which not opting in cannot remove: it is what the
+    /// rule needs to be correct, or to report in generated code by default.
     /// </summary>
     public static void ConfigureAnalysisOfGeneratedCode(this AnalysisContext context, GeneratedCodeAnalysisFlags defaultFlags)
     {
@@ -29,9 +29,9 @@ public static void ConfigureAnalysisOfGeneratedCode(this AnalysisContext context
 
     internal static GeneratedCodeAnalysisFlags GetAdditionalFlags(string? value) => value?.Trim() switch
     {
-        "0" => GeneratedCodeAnalysisFlags.None,
-        var trimmed when string.Equals(trimmed, "false", StringComparison.OrdinalIgnoreCase) => GeneratedCodeAnalysisFlags.None,
-        _ => GeneratedCodeAnalysisFlags.Analyze | GeneratedCodeAnalysisFlags.ReportDiagnostics,
+        "1" => GeneratedCodeAnalysisFlags.Analyze | GeneratedCodeAnalysisFlags.ReportDiagnostics,
+        var trimmed when string.Equals(trimmed, "true", StringComparison.OrdinalIgnoreCase) => GeneratedCodeAnalysisFlags.Analyze | GeneratedCodeAnalysisFlags.ReportDiagnostics,
+        _ => GeneratedCodeAnalysisFlags.None,
     };
 
     private static string? ReadEnvironmentVariable()

src/Meziantou.Analyzer/Internals/GeneratedCodeReporting.cs

@@ -4,12 +4,12 @@
 namespace Meziantou.Analyzer.Internals;
 
 /// <summary>
-/// Decides whether a rule reports the diagnostics located in generated code. The analyzers configure Roslyn to
-/// analyze generated code and to report the diagnostics located in it, as <c>ConfigureGeneratedCodeAnalysis</c>
+/// Decides whether a rule reports the diagnostics located in generated code. The analyzers that analyze generated
+/// code also configure Roslyn to report the diagnostics located in it, as <c>ConfigureGeneratedCodeAnalysis</c>
 /// cannot read the <c>.editorconfig</c> options, so the decision is taken when a diagnostic is reported. The
-/// options below only apply to what the analyzers actually analyze: a rule stops reporting in generated code when
-/// the <c>MEZIANTOU_ANALYZER_GENERATED_CODE</c> environment variable opts out of analyzing it, unless the rule
-/// needs it (see <see cref="AnalysisContextExtensions.ConfigureAnalysisOfGeneratedCode"/>).
+/// options below only apply to what the analyzers actually analyze: a rule only reports in generated code when the
+/// <c>MEZIANTOU_ANALYZER_GENERATED_CODE</c> environment variable opts in to analyzing it, unless the rule analyzes
+/// it anyway (see <see cref="AnalysisContextExtensions.ConfigureAnalysisOfGeneratedCode"/>).
 /// </summary>
 internal static class GeneratedCodeReporting
 {

tests/Meziantou.Analyzer.Test/Internals/AnalysisContextExtensionsTests.cs

@@ -6,28 +6,28 @@ namespace Meziantou.Analyzer.Test.Internals;
 public sealed class AnalysisContextExtensionsTests
 {
     [Theory]
-    [InlineData("0")]
-    [InlineData(" 0 ")]
-    [InlineData("false")]
-    [InlineData("False")]
-    [InlineData("FALSE")]
-    [InlineData("  false  ")]
-    public void GetAdditionalFlags_OptedOut(string value)
+    [InlineData("1")]
+    [InlineData(" 1 ")]
+    [InlineData("true")]
+    [InlineData("True")]
+    [InlineData("TRUE")]
+    [InlineData("  true  ")]
+    public void GetAdditionalFlags_OptedIn(string value)
     {
-        Assert.Equal(GeneratedCodeAnalysisFlags.None, AnalysisContextExtensions.GetAdditionalFlags(value));
+        Assert.Equal(GeneratedCodeAnalysisFlags.Analyze | GeneratedCodeAnalysisFlags.ReportDiagnostics, AnalysisContextExtensions.GetAdditionalFlags(value));
     }
 
     [Theory]
     [InlineData(null)]
     [InlineData("")]
     [InlineData("   ")]
-    [InlineData("1")]
-    [InlineData("true")]
-    [InlineData("True")]
+    [InlineData("0")]
+    [InlineData("false")]
+    [InlineData("False")]
     [InlineData("dummy")]
-    [InlineData("falsy")]
-    public void GetAdditionalFlags_NotOptedOut(string? value)
+    [InlineData("truthy")]
+    public void GetAdditionalFlags_NotOptedIn(string? value)
     {
-        Assert.Equal(GeneratedCodeAnalysisFlags.Analyze | GeneratedCodeAnalysisFlags.ReportDiagnostics, AnalysisContextExtensions.GetAdditionalFlags(value));
+        Assert.Equal(GeneratedCodeAnalysisFlags.None, AnalysisContextExtensions.GetAdditionalFlags(value));
     }
 }