Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,10 @@
"humao.rest-client",
"42Crunch.vscode-openapi",
"stoplight.spectral",
"ms-dotnettools.dotnet-interactive-vscode",
"ms-dotnettools.vscode-dotnet-runtime",
"GitHub.vscode-pull-request-github",
"ms-vscode.powershell",
"ms-azure-devops.azure-pipelines",
"GitHub.copilot-chat",
"GitHub.copilot",
"github.vscode-github-actions",
"bierner.markdown-preview-github-styles",
Expand Down
41 changes: 8 additions & 33 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ In addition to the rules enforced by `.editorconfig`, you SHOULD:
- SA1316: Tuple element names should use correct casing
- SA1518: File is required to end with a single newline character
- Any update for pattern matching `or` must not break the already-correct `text == "hello" || text == "world"` operator scenario.
- For multi-targeted projects including netstandard2.0, avoid APIs unavailable on netstandard2.0 such as `ArgumentNullException.ThrowIfNull`.

## Testing Guidelines

Expand Down Expand Up @@ -58,24 +59,17 @@ In addition to the rules enforced by `.editorconfig`, you SHOULD:
**Examples:**

❌ **INCORRECT** - Direct file system usage:

```cs
// BAD - Creates real files and directories
File.WriteAllText("report.json", content); Directory.CreateDirectory("reports");
bool exists = File.Exists("test.dll");
```

✅ **CORRECT** - Mock file system:

```cs
// GOOD - Uses mocked abstraction with simulated paths
var mockFileSystem = new Mock<IFileSystem>();
mockFileSystem.Setup(x => x.Exists("/fake/path/test.dll")).Returns(true);
mockFileSystem.Setup(x => x.Exists("/fake/reports")).Returns(true);
mockFileSystem.Setup(x => x.WriteAllText(It.IsAny<string>(), It.IsAny<string>()));
// Verify the mock was called correctly
mockFileSystem.Verify(x => x.WriteAllText(It.Is<string>(path => path.EndsWith("report.json")), It.IsAny<string>()), Times.Once);
```

### Moq Testing Rules (Critical - Prevents Runtime Errors)

Expand All @@ -93,44 +87,30 @@ Extension methods are static methods that cannot be intercepted by Moq. Using th
#### Example: Mocking ILogger

❌ **INCORRECT** - Will throw `NotSupportedException`:

```cs
// This will FAIL at runtime
_mockLogger.Verify(x => x.LogInformation(It.IsAny<string>()), Times.Once);
_mockLogger.Verify(x => x.LogInformation(It.Is<string>(s => s.Contains("json"))), Times.Once);
_mockLogger.Setup(x => x.LogWarning(It.IsAny<string>()));
```

✅ **CORRECT** - Mocks the underlying `Log` method:

```cs
// Verify LogInformation was called once
_mockLogger.Verify(x => x.Log(LogLevel.Information, It.IsAny<EventId>(), It.IsAny<It.IsAnyType>(), It.IsAny<Exception?>(), It.IsAny<Func<It.IsAnyType, Exception?, string>>()), Times.Once);
// Verify LogInformation was called with a message containing "json"
_mockLogger.Verify(x => x.Log(LogLevel.Information, It.IsAny<EventId>(), It.Is<It.IsAnyType>((v, t) => v.ToString()!.Contains("json")), It.IsAny<Exception?>(), It.IsAny<Func<It.IsAnyType, Exception?, string>>()), Times.Once);
// Setup LogWarning behavior
_mockLogger.Setup(x => x.Log(LogLevel.Warning, It.IsAny<string>()));
```

#### Example: Mocking LogDebug

❌ **INCORRECT**:

```cs
// This will FAIL at runtime
_mockLogger.Verify(x => x.LogDebug(It.IsAny<string>()), Times.Once);
```

✅ **CORRECT** - Mocks the underlying `Log` method:

```cs
// Verify LogDebug was called once
_mockLogger.Verify(x => x.Log(LogLevel.Debug, It.IsAny<EventId>(), It.IsAny<It.IsAnyType>(), It.IsAny<Exception?>(), It.IsAny<Func<It.IsAnyType, Exception?, string>>()), Times.Once);
// Verify LogDebug was called with a message containing "xml"
_mockLogger.Verify(x => x.Log(LogLevel.Debug, It.IsAny<EventId>(), It.Is<It.IsAnyType>((v, t) => v.ToString()!.Contains("xml")), It.IsAny<Exception?>(), It.IsAny<Func<It.IsAnyType, Exception?, string>>()), Times.Once);
// Setup LogError behavior
_mockLogger.Setup(x => x.Log(LogLevel.Error, It.IsAny<EventId>(), It.IsAny<It.IsAnyType>(), It.IsAny<Exception?>(), It.IsAny<Func<It.IsAnyType, Exception?, string>>()));
```

**Key Points:**
1. Always use `ILogger.Log()` with the appropriate `LogLevel` instead of extension methods.
Expand Down Expand Up @@ -168,16 +148,11 @@ This codebase uses **TWO different ILogger interfaces** with different signature
**Common Pitfall - Microsoft.Testing.Platform.Logging.ILogger:**

❌ **INCORRECT** - Assumes `EventId` parameter (which doesn't exist in MTP Logger):

```cs
// BAD - Microsoft.Testing.Platform.Logging.ILogger does NOT have EventId
_mockLogger.Verify(x => x.Log(LogLevel.Information, It.IsAny<EventId>(), // ⚠️ EventId does NOT exist in MTP LOGGER
It.IsAny<It.IsAnyType>(), It.IsAny<Exception?>(), It.IsAny<Func<It.IsAnyType, Exception?, string>>()), Times.Once);
```


✅ **CORRECT** - Uses actual MTP ILogger API signature (async methods):

```cs
// GOOD - Microsoft.Testing.Platform.Logging.ILogger uses simple async methods
_mockLogger.Verify(x => x.LogInformationAsync(It.IsAny<string>(), It.IsAny<CancellationToken>()), Times.Once);
_mockLogger.Verify(x => x.LogErrorAsync(It.IsAny<string>(), It.IsAny<CancellationToken>()), Times.Once);
Expand All @@ -187,7 +162,6 @@ _mockLogger.Verify(x => x.LogInformationAsync(It.Is<string>(s => s.Contains("exp
// For synchronous LoggerExtensions (extension methods):
// NOTE: These are extension methods and cannot be verified with Moq; verify the underlying Log(...) call instead.
_mockLogger.Verify(x => x.Log(LogLevel.Information, It.Is<string>(s => s.Contains("Coverage reports generated")), It.IsAny<Exception?>(), It.IsAny<Func<string, Exception?, string>>()), Times.Once);
```

**Verification Checklist:**
- [ ] I have searched for the interface definition using `get_symbols_by_name`.
Expand All @@ -208,8 +182,6 @@ When mocking interfaces, **reference actual adapter implementations** in the cod
- `src/coverlet.core/Abstractions/ILogger.cs` - Coverlet's internal logger interface.

**Example from CoverletLoggerAdapter.cs:**

```csharp
// Shows actual MTP ILogger usage - simple methods, no EventId
public void LogInformation(string message, bool important = false)
{
Expand All @@ -221,9 +193,7 @@ public void LogInformation(string message, bool important = false)
{
_logger.LogInformation(message);
}
}
```
### Test Generation Verification (Critical Rule)
}### Test Generation Verification (Critical Rule)

**Before generating any test, you MUST:**

Expand Down Expand Up @@ -271,6 +241,10 @@ public void LogInformation(string message, bool important = false)
6. **Verify existing tests** - Check for duplicates before adding new test methods.
7. **Use Theory for parameterized tests** - Don't create multiple test methods for different input values.

## Testing with xUnit v3

For this repo's xUnit v3 Microsoft.Testing.Platform test apps, use xUnit-specific filters such as `--filter-method`, `--filter-class`, or `--filter-query`; VSTest-style `--filter` is unsupported.

## Issue-Specific Guidelines

- For issue #1965, identify problematic assemblies before instrumentation and skip them, rather than relying on partial-restore/non-fatal restore behavior after failure.
Expand Down Expand Up @@ -374,6 +348,7 @@ The one comprehensive document MUST include:
- Build status
- Test status
- Coverage metrics
```

#### Documentation Creation (Critical Rule - Always Ask First)

Expand Down
51 changes: 28 additions & 23 deletions Directory.Packages.props
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,18 @@
<GlobalPackageReference Include="Nerdbank.GitVersioning" Version="3.10.91" />
</ItemGroup>
<PropertyGroup>
<MicrosoftBuildVersion>17.11.48</MicrosoftBuildVersion> <!-- .NET 8.0 support -->
<MicrosoftCodeAnalysisVersion>5.6.0</MicrosoftCodeAnalysisVersion> <!-- .NET 8.0 support -->
<!-- .NET 8.0 support -->
<MicrosoftBuildVersion>17.11.48</MicrosoftBuildVersion>
<MicrosoftCodeAnalysisVersion>5.9.0</MicrosoftCodeAnalysisVersion>
<!-- .NET 8.0 support -->
<NugetPackageVersion>7.6.0</NugetPackageVersion>
<!-- Test Platform, .NET Test SDK and Object Model -->
<MicrosoftNETTestSdkVersion>18.8.1</MicrosoftNETTestSdkVersion>
<MicrosoftNETTestSdkVersion>18.9.0</MicrosoftNETTestSdkVersion>
<XunitV3Version>3.2.2</XunitV3Version>
<XunitRunnerVisualstudioVersion>3.1.5</XunitRunnerVisualstudioVersion>
<MicrosoftTestingPlatformVersion>2.3.3</MicrosoftTestingPlatformVersion>
<XunitRunnerVisualstudioVersion>4.0.0</XunitRunnerVisualstudioVersion>
<MicrosoftTestingPlatformVersion>2.4.0</MicrosoftTestingPlatformVersion>
<MoqVersion>4.20.72</MoqVersion>
<MicrosoftNetRuntimeVersion>10.0.12</MicrosoftNetRuntimeVersion>
</PropertyGroup>
<ItemGroup>
<PackageVersion Include="BenchmarkDotNet" Version="0.15.8" />
Expand All @@ -24,21 +27,23 @@
<PackageVersion Include="Microsoft.VisualStudio.DiagnosticsHub.BenchmarkDotNetDiagnosers" Version="18.7.37220.1" />
<!--<PackageVersion Include="Microsoft.VisualStudio.DiagnosticsHub.BenchmarkDotNetDiagnosers" Version="17.13.35606.1" />-->
<PackageVersion Include="DotNetConfig" Version="1.2.0" />
<PackageVersion Include="Microsoft.Bcl.AsyncInterfaces" Version="10.0.10" />
<PackageVersion Include="Microsoft.Bcl.AsyncInterfaces" Version="$(MicrosoftNetRuntimeVersion)" />
<PackageVersion Include="Microsoft.Build.Utilities.Core" Version="$(MicrosoftBuildVersion)" />
<PackageVersion Include="Microsoft.Build.Framework" Version="$(MicrosoftBuildVersion)" />
<PackageVersion Include="Microsoft.Build.Locator" Version="1.11.2" />
<PackageVersion Include="Microsoft.CodeAnalysis.CSharp" Version="$(MicrosoftCodeAnalysisVersion)" />
<PackageVersion Include="Microsoft.CodeAnalysis.BannedApiAnalyzers" Version="5.0.0-1.25277.114" />
<PackageVersion Include="Microsoft.Extensions.DependencyModel" Version="10.0.10" />
<PackageVersion Include="Microsoft.Extensions.Configuration" Version="10.0.10" />
<PackageVersion Include="Microsoft.Extensions.Configuration.Json" Version="10.0.10" />
<PackageVersion Include="Microsoft.Extensions.DependencyInjection" Version="10.0.10" /> <!-- latest stable version net10.0, net9.0, net8.0 -->
<PackageVersion Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="10.0.10" /> <!-- latest stable version net10.0, net9.0, net8.0 -->
<!-- For test TestInstrument_NetstandardAwareAssemblyResolver_PreserveCompilationContext -->
<PackageVersion Include="Microsoft.Extensions.FileSystemGlobbing" Version="10.0.10" />
<PackageVersion Include="Microsoft.Extensions.DependencyModel" Version="$(MicrosoftNetRuntimeVersion)" />
<PackageVersion Include="Microsoft.Extensions.Configuration" Version="$(MicrosoftNetRuntimeVersion)" />
<PackageVersion Include="Microsoft.Extensions.Configuration.Json" Version="$(MicrosoftNetRuntimeVersion)" />
<PackageVersion Include="Microsoft.Extensions.DependencyInjection" Version="$(MicrosoftNetRuntimeVersion)" />
<!-- latest stable version net10.0, net9.0, net8.0 -->
<PackageVersion Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="$(MicrosoftNetRuntimeVersion)" />
<!-- latest stable version net10.0, net9.0, net8.0 -->
<!-- For test TestInstrument_NetstandardAwareAssemblyResolver_PreserveCompilationContext -->
<PackageVersion Include="Microsoft.Extensions.FileSystemGlobbing" Version="$(MicrosoftNetRuntimeVersion)" />
<!--For test TestInstrument_NetstandardAwareAssemblyResolver_PreserveCompilationContext-->
<PackageVersion Include="Microsoft.Extensions.Logging.Abstractions" Version="10.0.10" />
<PackageVersion Include="Microsoft.Extensions.Logging.Abstractions" Version="$(MicrosoftNetRuntimeVersion)" />
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="$(MicrosoftNETTestSdkVersion)" />
<PackageVersion Include="Microsoft.Sbom.Targets" Version="4.1.5" />
<PackageVersion Include="Microsoft.TestPlatform.ObjectModel" Version="$(MicrosoftNETTestSdkVersion)" />
Expand Down Expand Up @@ -66,20 +71,20 @@
<PackageVersion Include="xunit.v3.runner.msbuild" Version="$(XunitV3Version)" />
<PackageVersion Include="xunit.runner.visualstudio" Version="$(XunitRunnerVisualstudioVersion)" />
<PackageVersion Include="System.Buffers" Version="4.6.1" />
<PackageVersion Include="System.Collections.Immutable" Version="10.0.10" />
<PackageVersion Include="System.Configuration.ConfigurationManager" Version="10.0.10" />
<PackageVersion Include="System.Diagnostics.EventLog" Version="10.0.10" />
<PackageVersion Include="System.Formats.Asn1" Version="10.0.10" />
<PackageVersion Include="System.IO.Pipelines" Version="10.0.10" />
<PackageVersion Include="System.Collections.Immutable" Version="$(MicrosoftNetRuntimeVersion)" />
<PackageVersion Include="System.Configuration.ConfigurationManager" Version="$(MicrosoftNetRuntimeVersion)" />
<PackageVersion Include="System.Diagnostics.EventLog" Version="$(MicrosoftNetRuntimeVersion)" />
<PackageVersion Include="System.Formats.Asn1" Version="$(MicrosoftNetRuntimeVersion)" />
<PackageVersion Include="System.IO.Pipelines" Version="$(MicrosoftNetRuntimeVersion)" />
<PackageVersion Include="System.Linq.Async" Version="7.0.1" />
<PackageVersion Include="System.Memory" Version="4.6.3" />
<PackageVersion Include="System.Net.Http" Version="4.3.4" />
<PackageVersion Include="System.Reflection.Metadata" Version="10.0.10" />
<PackageVersion Include="System.Reflection.Metadata" Version="$(MicrosoftNetRuntimeVersion)" />
<PackageVersion Include="System.Runtime.CompilerServices.Unsafe" Version="6.1.2" />
<PackageVersion Include="System.Security.AccessControl" Version="6.0.1" />
<PackageVersion Include="System.Security.Cryptography.Pkcs" Version="10.0.10" />
<PackageVersion Include="System.Text.Encoding.CodePages" Version="10.0.10" />
<PackageVersion Include="System.Text.Json" Version="10.0.10" />
<PackageVersion Include="System.Security.Cryptography.Pkcs" Version="$(MicrosoftNetRuntimeVersion)" />
<PackageVersion Include="System.Text.Encoding.CodePages" Version="$(MicrosoftNetRuntimeVersion)" />
<PackageVersion Include="System.Text.Json" Version="$(MicrosoftNetRuntimeVersion)" />
<PackageVersion Include="System.Text.RegularExpressions" Version="4.3.1" />
<PackageVersion Include="System.Threading.Tasks.Extensions" Version="4.6.3" />
</ItemGroup>
Expand Down
1 change: 1 addition & 0 deletions Documentation/Changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Improvements

- Publish Microsoft.Testing.Platform coverage messages from coverlet.MTP [#2019](https://github.com/coverlet-coverage/coverlet/pull/2019)
- Implement dynamic exclusion filters for assemblies (Coverlet.MTP) [#1946](https://github.com/coverlet-coverage/coverlet/pull/1946)
- Replace legacy .sln files with modern .slnx format [#1966](https://github.com/coverlet-coverage/coverlet/pull/1966)
- coverlet.console: add trace diagnostics and actionable warnings for instrumentation/hit/empty-result failures [#2005](https://github.com/coverlet-coverage/coverlet/pull/2005)
Expand Down
36 changes: 21 additions & 15 deletions Documentation/Coverlet.MTP.Integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,9 @@ dotnet exec <test-assembly.dll> --help
| `--coverlet-skip-auto-props` | Skip auto-implemented properties. (default: `false`) |
| `--coverlet-does-not-return-attribute <attribute>` | Attributes that mark methods as not returning. Can be specified multiple times. (default: `none`) |
| `--coverlet-exclude-assemblies-without-sources <value>` | Exclude assemblies without source code. Values: `MissingAll`, `MissingAny`, `None`. (default: `None`) |
| `--coverlet-threshold <threshold>` | Exits with error if the coverage % is below value |
| `--coverlet-threshold-type <line\|branch\|method>` | Coverage type(s) to apply the threshold to. Can be specified multiple times or as a comma-separated list. (default: `line`) |
| `--coverlet-threshold-stat <Average\|Minimum\|Total>` | Coverage statistic used to enforce the threshold value. (default: `Minimum`) |

> [!TIP]
> If you encounter instrumentation failures like "The process cannot access the file ... because it is being used by another process", try setting `--coverlet-exclude-assemblies-without-sources MissingAll` (or in a config file: `"ExcludeAssembliesWithoutSources": "MissingAll"`) to skip assemblies without sources and reduce access conflicts.
Expand Down Expand Up @@ -167,21 +170,21 @@ The `testconfig.json` format is the standard configuration file for Microsoft Te

| Key | Type | Description |
| :--- | :--- | :---------- |
| `include` | string | Comma-separated include filters (e.g., `[MyApp.*]*`) |
| `includeDirectory` | string | Comma-separated additional directories for sources |
| `exclude` | string | Comma-separated exclude filters (e.g., `[*.Tests]*`) |
| `excludeByFile` | string | Comma-separated glob patterns for source file exclusion |
| `excludeByAttribute` | string | Comma-separated attributes to exclude |
| `format` | string | Comma-separated output formats (default: `cobertura`) |
| `useSourceLink` | bool | Enable SourceLink support |
| `singleHit` | bool | Limit hits to one per location |
| `includeTestAssembly` | bool | Include test assembly in coverage |
| `skipAutoProps` | bool | Skip auto-implemented properties |
| `doesNotReturnAttribute` | string | Comma-separated attributes marking non-returning methods |
| `deterministicReport` | bool | Generate deterministic reports |
| `excludeAssembliesWithoutSources` | string | Values: `MissingAll`, `MissingAny`, `None` |
| `disableManagedInstrumentationRestore` | bool | Disable managed instrumentation restore |
| `mergeWith` | string | Path to existing coverage file to merge with |
| `Include` | string | Comma-separated include filters (e.g., `[MyApp.*]*`) |
| `IncludeDirectory` | string | Comma-separated additional directories for sources |
| `Exclude` | string | Comma-separated exclude filters (e.g., `[*.Tests]*`) |
| `ExcludeByFile` | string | Comma-separated glob patterns for source file exclusion |
| `ExcludeByAttribute` | string | Comma-separated attributes to exclude |
| `Format` | string | Comma-separated output formats (default: `cobertura`) |
| `UseSourceLink` | bool | Enable SourceLink support |
| `SingleHit` | bool | Limit hits to one per location |
| `IncludeTestAssembly` | bool | Include test assembly in coverage |
| `SkipAutoProps` | bool | Skip auto-implemented properties |
| `DoesNotReturnAttribute` | string | Comma-separated attributes marking non-returning methods |
| `DeterministicReport` | bool | Generate deterministic reports |
| `ExcludeAssembliesWithoutSources` | string | Values: `MissingAll`, `MissingAny`, `None` |
| `DisableManagedInstrumentationRestore` | bool | Disable managed instrumentation restore |
| `MergeWith` | string | Path to existing coverage file to merge with |

> [!NOTE]
> Keys in `testconfig.json` use **camelCase** (e.g., `excludeByAttribute`), following the Microsoft Testing Platform convention.
Expand Down Expand Up @@ -236,6 +239,9 @@ The legacy `coverlet.mtp.appsettings.json` format is still supported for backwar
| `DoesNotReturnAttribute` | string | Comma-separated attributes marking non-returning methods |
| `DeterministicReport` | bool | Generate deterministic reports |
| `ExcludeAssembliesWithoutSources` | string | Values: `MissingAll`, `MissingAny`, `None` (default: `MissingAll`) |
| `Threshold` | int | Exits with error if the code coverage [0..100%] is below value |
| `ThresholdType` | string | Comma-separated coverage type to apply the Threshold to. [default: `line`] |
| `ThresholdStat` | string | Coverage statistic used to enforce the threshold value. Values: `Minimum`, `Average`, `Total`. (default: `Minimum`) |

**Example `coverlet.mtp.appsettings.json`:**

Expand Down
Loading
Loading