Skip to content

fix: align diagnostic categories with Usage, Correctness, and Best Practice#1087

Merged
rjmurillo merged 3 commits intomainfrom
fix/944-analyzer-categories
Mar 15, 2026
Merged

fix: align diagnostic categories with Usage, Correctness, and Best Practice#1087
rjmurillo merged 3 commits intomainfrom
fix/944-analyzer-categories

Conversation

@rjmurillo
Copy link
Copy Markdown
Owner

@rjmurillo rjmurillo commented Mar 15, 2026

Summary

Closes #944

  • Added Correctness and BestPractice constants to DiagnosticCategory.cs
  • Updated 12 analyzers (Moq1100-Moq1210) from Usage to Correctness
  • Updated 3 analyzers (Moq1400, Moq1410, Moq1500) from Usage to Best Practice
  • Updated AnalyzerReleases.Unshipped.md with Changed Rules section tracking all category migrations
  • Fixed README.md: Moq1101 corrected from Usage to Correctness, Moq1420 corrected from Best Practice to Usage
  • Added 24 data-driven xUnit tests verifying every analyzer reports the correct category

Breaking Change

This is a breaking change for users who filter diagnostics by category in .editorconfig or Directory.Build.props.

Affected rules: Moq1100, Moq1101, Moq1200-Moq1210, Moq1400, Moq1410, Moq1500

These rules previously reported category Usage. They now report Correctness or Best Practice.

Migration: If you suppress or configure these rules by category, update your configuration:

# Before
dotnet_analyzer_diagnostic.category-Usage.severity = none

# After - apply the correct category for each group
dotnet_analyzer_diagnostic.category-Correctness.severity = none   # Moq1100-Moq1210
dotnet_analyzer_diagnostic.category-Best Practice.severity = none  # Moq1400, Moq1410, Moq1500

Per-rule suppression (e.g., dotnet_diagnostic.Moq1100.severity = none) is unaffected.

Category Mapping

Category Rule IDs
Usage Moq1000, Moq1001, Moq1002, Moq1003, Moq1004, Moq1300, Moq1301, Moq1302, Moq1420
Correctness Moq1100, Moq1101, Moq1200, Moq1201, Moq1202, Moq1203, Moq1204, Moq1205, Moq1206, Moq1207, Moq1208, Moq1210
Best Practice Moq1400, Moq1410, Moq1500

Test plan

  • Build passes with PedanticMode=true (0 warnings, 0 errors)
  • All 3205 tests pass (3177 existing + 24 new category tests + 4 PerfDiff)
  • New DiagnosticCategoryTests verifies each analyzer reports the correct category
  • AnalyzerReleases.Unshipped.md categories match DiagnosticDescriptor values
  • README categories match code

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Improvements
    • Reclassified multiple analyzer diagnostics: many rules moved to "Correctness" and some mock-behavior rules moved to "Best Practice"; specific rule adjustments include Moq1101 → Correctness and Moq1420 → Usage.
  • Documentation
    • Updated rules listing to reflect the new categories so reported categories match guidance.
  • Tests
    • Added tests that verify analyzers report the expected categories.

…actice (#944)

Add Correctness and BestPractice constants to DiagnosticCategory.
Update 12 analyzers from Usage to Correctness (Moq1100-1210).
Update 3 analyzers from Usage to Best Practice (Moq1400, Moq1410, Moq1500).
Update AnalyzerReleases.Unshipped.md with Changed Rules section.
Fix README.md categories for Moq1101 and Moq1420.
Add 24 data-driven tests verifying each analyzer reports the correct category.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@rjmurillo rjmurillo added documentation analyzers Change that impacts an analyzer behavior releasable labels Mar 15, 2026
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 15, 2026

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: a53042cb-776a-4656-b897-39b26e914c96

📥 Commits

Reviewing files that changed from the base of the PR and between 4585ae0 and dd6e338.

📒 Files selected for processing (1)
  • tests/Moq.Analyzers.Test/DiagnosticCategoryTests.cs

📝 Walkthrough

Walkthrough

Adds two diagnostic category constants (Correctness, Best Practice), updates many analyzers' DiagnosticDescriptor Category values, adjusts release/docs entries, and introduces tests that validate each analyzer's reported category.

Changes

Cohort / File(s) Summary
Documentation
README.md, src/Analyzers/AnalyzerReleases.Unshipped.md, docs/rules/README.md
Updated rule category entries; added a "Changed Rules" table in AnalyzerReleases.Unshipped.md; moved specific rules between Usage, Correctness, and Best Practice (e.g., Moq1101, Moq1208, Moq1420).
Category Constants
src/Common/DiagnosticCategory.cs
Added internal constants Correctness and Best Practice to the diagnostic category definitions.
Analyzers — Correctness
src/Analyzers/CallbackSignatureShouldMatchMockedMethodAnalyzer.cs, src/Analyzers/EventSetupHandlerShouldMatchEventTypeAnalyzer.cs, src/Analyzers/MethodSetupShouldSpecifyReturnValueAnalyzer.cs, src/Analyzers/NoMethodsInPropertySetupAnalyzer.cs, src/Analyzers/RaiseEventArgumentsShouldMatchEventSignatureAnalyzer.cs, src/Analyzers/RaisesEventArgumentsShouldMatchEventSignatureAnalyzer.cs, src/Analyzers/ReturnsAsyncShouldBeUsedForAsyncMethodsAnalyzer.cs, src/Analyzers/ReturnsDelegateShouldReturnTaskAnalyzer.cs, src/Analyzers/SetupSequenceShouldBeUsedOnlyForOverridableMembersAnalyzer.cs, src/Analyzers/SetupShouldBeUsedOnlyForOverridableMembersAnalyzer.cs, src/Analyzers/SetupShouldNotIncludeAsyncResultAnalyzer.cs, src/Analyzers/VerifyShouldBeUsedOnlyForOverridableMembersAnalyzer.cs
Changed each analyzer's DiagnosticDescriptor Category from Usage to Correctness; no behavioral or control-flow changes.
Analyzers — Best Practice
src/Analyzers/MockRepositoryVerifyAnalyzer.cs, src/Analyzers/SetExplicitMockBehaviorAnalyzer.cs, src/Analyzers/SetStrictMockBehaviorAnalyzer.cs
Changed DiagnosticDescriptor Category from Usage to Best Practice; no behavioral or control-flow changes.
Tests
tests/Moq.Analyzers.Test/DiagnosticCategoryTests.cs
Added tests mapping analyzers and rule IDs to expected categories and a suite-level test ensuring all SupportedDiagnostics are categorized.

Sequence Diagram(s)

(omitted)

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested reviewers

  • MattKotsenas
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: aligning diagnostic categories across Usage, Correctness, and Best Practice categories.
Description check ✅ Passed The description comprehensively covers the Summary, Breaking Change details, Category Mapping, and Test plan sections, addressing all relevant aspects of the changes.
Linked Issues check ✅ Passed The PR successfully addresses all objectives from issue #944: adds Correctness and BestPractice constants, updates 15 analyzers' category fields, updates documentation, and adds verification tests.
Out of Scope Changes check ✅ Passed All changes are directly scoped to resolving issue #944: category constant additions, analyzer descriptor updates, documentation corrections, and related test coverage.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch fix/944-analyzer-categories
📝 Coding Plan
  • Generate coding plan for human review comments

Comment @coderabbitai help to get the list of available commands and usage tips.

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the clarity and organization of Moq analyzers by introducing dedicated Correctness and BestPractice diagnostic categories. This refinement allows for more precise classification of issues, helping developers distinguish between general usage recommendations, potential errors, and optimal coding patterns. The changes involve updating the core diagnostic category definitions, reassigning categories for numerous existing analyzers, and ensuring all documentation and tests are aligned with the new structure, ultimately improving the utility and guidance provided by the analyzers.

Highlights

  • New Diagnostic Categories: New Correctness and BestPractice constants were introduced in DiagnosticCategory.cs to refine the classification of analyzer rules.
  • Analyzer Reclassification: Twelve analyzers (Moq1100-Moq1210) were reclassified from the Usage category to Correctness, and three analyzers (Moq1400, Moq1410, Moq1500) were reclassified from Usage to Best Practice.
  • Documentation Updates: The README.md and AnalyzerReleases.Unshipped.md files were updated to reflect the new diagnostic categories and rule migrations.
  • Comprehensive Test Coverage: Twenty-four new data-driven xUnit tests were added to rigorously verify that every analyzer reports its correct diagnostic category.
Changelog
  • README.md
    • Corrected the category for Moq1101 from Usage to Correctness.
    • Corrected the category for Moq1420 from Best Practice to Usage.
  • src/Analyzers/AnalyzerReleases.Unshipped.md
    • Updated Moq1208 category from Usage to Correctness.
    • Added a new 'Changed Rules' section detailing category migrations for 15 analyzers.
  • src/Analyzers/CallbackSignatureShouldMatchMockedMethodAnalyzer.cs
    • Updated diagnostic category from Usage to Correctness.
  • src/Analyzers/EventSetupHandlerShouldMatchEventTypeAnalyzer.cs
    • Updated diagnostic category from Usage to Correctness.
  • src/Analyzers/MethodSetupShouldSpecifyReturnValueAnalyzer.cs
    • Updated diagnostic category from Usage to Correctness.
  • src/Analyzers/MockRepositoryVerifyAnalyzer.cs
    • Updated diagnostic category from Usage to BestPractice.
  • src/Analyzers/NoMethodsInPropertySetupAnalyzer.cs
    • Updated diagnostic category from Usage to Correctness.
  • src/Analyzers/RaiseEventArgumentsShouldMatchEventSignatureAnalyzer.cs
    • Updated diagnostic category from Usage to Correctness.
  • src/Analyzers/RaisesEventArgumentsShouldMatchEventSignatureAnalyzer.cs
    • Updated diagnostic category from Usage to Correctness.
  • src/Analyzers/ReturnsAsyncShouldBeUsedForAsyncMethodsAnalyzer.cs
    • Updated diagnostic category from Usage to Correctness.
  • src/Analyzers/ReturnsDelegateShouldReturnTaskAnalyzer.cs
    • Updated diagnostic category from Usage to Correctness.
  • src/Analyzers/SetExplicitMockBehaviorAnalyzer.cs
    • Updated diagnostic category from Usage to BestPractice.
  • src/Analyzers/SetStrictMockBehaviorAnalyzer.cs
    • Updated diagnostic category from Usage to BestPractice.
  • src/Analyzers/SetupSequenceShouldBeUsedOnlyForOverridableMembersAnalyzer.cs
    • Updated diagnostic category from Usage to Correctness.
  • src/Analyzers/SetupShouldBeUsedOnlyForOverridableMembersAnalyzer.cs
    • Updated diagnostic category from Usage to Correctness.
  • src/Analyzers/SetupShouldNotIncludeAsyncResultAnalyzer.cs
    • Updated diagnostic category from Usage to Correctness.
  • src/Analyzers/VerifyShouldBeUsedOnlyForOverridableMembersAnalyzer.cs
    • Updated diagnostic category from Usage to Correctness.
  • src/Common/DiagnosticCategory.cs
    • Added new Correctness and BestPractice constants.
  • tests/Moq.Analyzers.Test/DiagnosticCategoryTests.cs
    • Added a new test file to verify diagnostic categories for all analyzers.
Activity
  • The build passed with PedanticMode=true, indicating no new warnings or errors.
  • All 3205 tests passed, including 24 newly added category tests and 4 PerfDiff tests.
  • New DiagnosticCategoryTests were introduced to verify that each analyzer reports its correct category.
  • AnalyzerReleases.Unshipped.md categories were confirmed to match DiagnosticDescriptor values.
  • README.md categories were confirmed to match the code.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@deepsource-io
Copy link
Copy Markdown

deepsource-io bot commented Mar 15, 2026

DeepSource Code Review

We reviewed changes in 2356a3a...dd6e338 on this pull request. Below is the summary for the review, and you can see the individual issues we found as inline review comments.

See full review on DeepSource ↗

PR Report Card

Overall Grade   Security  

Reliability  

Complexity  

Hygiene  

Code Review Summary

Analyzer Status Updated (UTC) Details
C# Mar 15, 2026 10:11p.m. Review ↗

@coderabbitai coderabbitai bot requested a review from MattKotsenas March 15, 2026 20:43
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request effectively realigns diagnostic categories to be more descriptive (Usage, Correctness, Best Practice), which improves clarity for users of the analyzers. The changes are consistently applied across all relevant analyzer definitions, documentation in README.md, and release notes. The addition of comprehensive, data-driven tests in DiagnosticCategoryTests.cs is an excellent measure to ensure these categories remain correct and prevent future regressions. I have one minor suggestion for consistency in DiagnosticCategory.cs.

@codacy-production
Copy link
Copy Markdown

codacy-production bot commented Mar 15, 2026

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
+0.00% (target: -1.00%) 100.00% (target: 95.00%)
Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (2356a3a) 2554 2295 89.86%
Head commit (dd6e338) 2554 (+0) 2295 (+0) 89.86% (+0.00%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#1087) 15 15 100.00%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

@coderabbitai coderabbitai bot added the build label Mar 15, 2026
coderabbitai[bot]
coderabbitai bot previously approved these changes Mar 15, 2026
Fix Moq1101 category from Usage to Correctness and Moq1420 category
from Best Practice to Usage to match actual DiagnosticCategory values.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
coderabbitai[bot]
coderabbitai bot previously approved these changes Mar 15, 2026
Adds AllAnalyzers_ShouldHaveCategoryTest which uses reflection to
discover every concrete DiagnosticAnalyzer in the Analyzers assembly and
asserts that each diagnostic ID is covered by one of the three existing
theory data sets. This prevents new analyzers from silently missing a
category test.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@rjmurillo rjmurillo marked this pull request as ready for review March 15, 2026 22:15
Copilot AI review requested due to automatic review settings March 15, 2026 22:15
@chatgpt-codex-connector
Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aligns analyzer DiagnosticDescriptor.Category values with the documented category groups (Usage / Correctness / Best Practice), updates release notes and docs to match, and adds a dedicated test suite to prevent category regressions.

Changes:

  • Added Correctness and BestPractice category constants and migrated affected analyzers off Usage.
  • Updated release notes + docs tables to reflect the new categories.
  • Added category verification tests for all analyzers/rules.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/Moq.Analyzers.Test/DiagnosticCategoryTests.cs Adds tests intended to validate per-rule diagnostic categories and enforce coverage for all analyzers.
src/Common/DiagnosticCategory.cs Adds Correctness and BestPractice category constants for consistent reuse.
src/Analyzers/CallbackSignatureShouldMatchMockedMethodAnalyzer.cs Migrates rule category to Correctness.
src/Analyzers/NoMethodsInPropertySetupAnalyzer.cs Migrates rule category to Correctness.
src/Analyzers/SetupShouldBeUsedOnlyForOverridableMembersAnalyzer.cs Migrates rule category to Correctness.
src/Analyzers/SetupShouldNotIncludeAsyncResultAnalyzer.cs Migrates rule category to Correctness.
src/Analyzers/RaiseEventArgumentsShouldMatchEventSignatureAnalyzer.cs Migrates rule category to Correctness.
src/Analyzers/RaisesEventArgumentsShouldMatchEventSignatureAnalyzer.cs Migrates rule category to Correctness.
src/Analyzers/MethodSetupShouldSpecifyReturnValueAnalyzer.cs Migrates rule category to Correctness.
src/Analyzers/EventSetupHandlerShouldMatchEventTypeAnalyzer.cs Migrates rule category to Correctness.
src/Analyzers/ReturnsAsyncShouldBeUsedForAsyncMethodsAnalyzer.cs Migrates rule category to Correctness.
src/Analyzers/ReturnsDelegateShouldReturnTaskAnalyzer.cs Migrates rule category to Correctness.
src/Analyzers/SetupSequenceShouldBeUsedOnlyForOverridableMembersAnalyzer.cs Migrates rule category to Correctness.
src/Analyzers/VerifyShouldBeUsedOnlyForOverridableMembersAnalyzer.cs Migrates rule category to Correctness.
src/Analyzers/SetExplicitMockBehaviorAnalyzer.cs Migrates rule category to Best Practice.
src/Analyzers/SetStrictMockBehaviorAnalyzer.cs Migrates rule category to Best Practice.
src/Analyzers/MockRepositoryVerifyAnalyzer.cs Migrates rule category to Best Practice.
src/Analyzers/AnalyzerReleases.Unshipped.md Updates category for Moq1208 and introduces a “Changed Rules” section to track migrations.
docs/rules/README.md Updates the rules index table to match the new categories.
README.md Updates the top-level rules table to match the new categories.

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines +9 to +21
public static TheoryData<DiagnosticAnalyzer, string, string> UsageAnalyzers =>
new()
{
{ new NoSealedClassMocksAnalyzer(), "Moq1000", DiagnosticCategory.Usage },
{ new ConstructorArgumentsShouldMatchAnalyzer(), "Moq1001", DiagnosticCategory.Usage },
{ new ConstructorArgumentsShouldMatchAnalyzer(), "Moq1002", DiagnosticCategory.Usage },
{ new InternalTypeMustHaveInternalsVisibleToAnalyzer(), "Moq1003", DiagnosticCategory.Usage },
{ new NoMockOfLoggerAnalyzer(), "Moq1004", DiagnosticCategory.Usage },
{ new AsShouldBeUsedOnlyForInterfaceAnalyzer(), "Moq1300", DiagnosticCategory.Usage },
{ new MockGetShouldNotTakeLiteralsAnalyzer(), "Moq1301", DiagnosticCategory.Usage },
{ new LinqToMocksExpressionShouldBeValidAnalyzer(), "Moq1302", DiagnosticCategory.Usage },
{ new RedundantTimesSpecificationAnalyzer(), "Moq1420", DiagnosticCategory.Usage },
};
Comment on lines +11 to +29

### Changed Rules

Rule ID | New Category | New Severity | Old Category | Old Severity | Notes
--------|--------------|--------------|--------------|--------------|-------
Moq1100 | Correctness | Warning | Usage | Warning | CallbackSignatureShouldMatchMockedMethodAnalyzer
Moq1101 | Correctness | Warning | Usage | Warning | NoMethodsInPropertySetupAnalyzer
Moq1200 | Correctness | Error | Usage | Error | SetupShouldBeUsedOnlyForOverridableMembersAnalyzer
Moq1201 | Correctness | Error | Usage | Error | SetupShouldNotIncludeAsyncResultAnalyzer
Moq1202 | Correctness | Warning | Usage | Warning | RaiseEventArgumentsShouldMatchEventSignatureAnalyzer
Moq1203 | Correctness | Warning | Usage | Warning | MethodSetupShouldSpecifyReturnValueAnalyzer
Moq1204 | Correctness | Warning | Usage | Warning | RaisesEventArgumentsShouldMatchEventSignatureAnalyzer
Moq1205 | Correctness | Warning | Usage | Warning | EventSetupHandlerShouldMatchEventTypeAnalyzer
Moq1206 | Correctness | Warning | Usage | Warning | ReturnsAsyncShouldBeUsedForAsyncMethodsAnalyzer
Moq1207 | Correctness | Error | Usage | Error | SetupSequenceShouldBeUsedOnlyForOverridableMembersAnalyzer
Moq1210 | Correctness | Error | Usage | Error | VerifyShouldBeUsedOnlyForOverridableMembersAnalyzer
Moq1400 | Best Practice | Warning | Usage | Warning | SetExplicitMockBehaviorAnalyzer
Moq1410 | Best Practice | Info | Usage | Info | SetStrictMockBehaviorAnalyzer
Moq1500 | Best Practice | Warning | Usage | Warning | MockRepositoryVerifyAnalyzer
@rjmurillo rjmurillo merged commit 3a0a0b4 into main Mar 15, 2026
45 of 48 checks passed
@rjmurillo rjmurillo deleted the fix/944-analyzer-categories branch March 15, 2026 22:22
@rjmurillo rjmurillo added this to the vNext milestone Mar 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

analyzers Change that impacts an analyzer behavior build documentation releasable

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs: AnalyzerReleases.Unshipped.md categories all say 'Usage', contradicting README

2 participants