Add incrementality tracking support and more detailed analysis of the build errors reported

.github/copilot-instructions.md

@@ -11,8 +11,90 @@ New files should use nullable types but don't refactor aggressively existing cod
 
 Generate tests for new codepaths, and add tests for any bugs you fix. Use the existing test framework, which is xUnit with Shouldly assertions. Use Shouldly assertions for all assertions in modified code, even if the file is predominantly using xUnit assertions.
 
+When making changes, check if related documentation exists in the `documentation/` folder (including `documentation/specs/`) and update it to reflect your changes. Keep documentation in sync with code changes, especially for telemetry, APIs, and architectural decisions.
+
 Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here.
 
+## Performance Best Practices
+
+MSBuild is performance-critical infrastructure. Follow these patterns:
+
+### Switch Expressions for Dispatch Logic
+Use tuple switch expressions for multi-condition dispatch instead of if-else chains:
+```csharp
+// GOOD: Clean, O(1) dispatch
+return (c0, c1) switch
+{
+    ('C', 'S') => Category.CSharp,
+    ('F', 'S') => Category.FSharp,
+    ('V', 'B') when value.Length >= 3 && value[2] == 'C' => Category.VB,
+    _ => Category.Other
+};
+
+// AVOID: Verbose if-else chains
+if (c0 == 'C' && c1 == 'S') return Category.CSharp;
+else if (c0 == 'F' && c1 == 'S') return Category.FSharp;
+// ...
+```
+
+### Range Pattern Matching
+Use range patterns for numeric categorization:
+```csharp
+// GOOD: Clear and efficient
+return errorNumber switch
+{
+    >= 3001 and <= 3999 => Category.Tasks,
+    >= 4001 and <= 4099 => Category.General,
+    >= 4100 and <= 4199 => Category.Evaluation,
+    _ => Category.Other
+};
+```
+
+### String Comparisons
+- Use `StringComparer.OrdinalIgnoreCase` for case-insensitive HashSets/Dictionaries when the source data may vary in casing
+- Use `char.ToUpperInvariant()` for single-character comparisons
+- Use `ReadOnlySpan<char>` and `Slice()` to avoid string allocations when parsing substrings
+- Use `int.TryParse(span, out var result)` on .NET Core+ for allocation-free parsing
+
+### Inlining
+Mark small, hot-path methods with `[MethodImpl(MethodImplOptions.AggressiveInlining)]`:
+```csharp
+[MethodImpl(MethodImplOptions.AggressiveInlining)]
+private static bool IsCompilerPrefix(string value) => ...
+```
+
+### Conditional Compilation for Framework Differences
+Use `#if NET` for APIs that differ between .NET Framework and .NET Core:
+```csharp
+#if NET
+    return int.TryParse(span, out errorNumber);
+#else
+    return int.TryParse(span.ToString(), out errorNumber);
+#endif
+```
+
+### Immutable Collections
+Choose the right immutable collection type based on usage pattern:
+
+**Build once, read many times** (most common in MSBuild):
+- Use `ImmutableArray<T>` instead of `ImmutableList<T>` - significantly faster for read access
+- Use `FrozenDictionary<TKey, TValue>` instead of `ImmutableDictionary<TKey, TValue>` - optimized for read-heavy scenarios
+
+**Build incrementally over time** (adding items one by one):
+- Use `ImmutableList<T>` and `ImmutableDictionary<TKey, TValue>` - designed for efficient `Add` operations returning new collections
+
+```csharp
+// GOOD: Build once from LINQ, then read many times
+ImmutableArray<string> items = source.Select(x => x.Name).ToImmutableArray();
+FrozenDictionary<string, int> lookup = pairs.ToFrozenDictionary(x => x.Key, x => x.Value);
+
+// AVOID for read-heavy scenarios:
+ImmutableList<string> items = source.Select(x => x.Name).ToImmutableList();
+ImmutableDictionary<string, int> lookup = pairs.ToImmutableDictionary(x => x.Key, x => x.Value);
+```
+
+Note: `ImmutableArray<T>` is a value type. Use `IsDefault` property to check for uninitialized arrays, or use nullable `ImmutableArray<T>?` with `.Value` to unwrap.
+
 ## Working Effectively
 
 #### Bootstrap and Build the Repository

documentation/VS-Telemetry-Data.md

@@ -0,0 +1,138 @@
+# MSBuild Visual Studio Telemetry Data
+
+This document describes the telemetry data collected by MSBuild and sent to Visual Studio telemetry infrastructure. The telemetry helps the MSBuild team understand build patterns, identify issues, and improve the build experience.
+
+## Overview
+
+MSBuild collects telemetry at multiple levels:
+1. **Build-level telemetry** - Overall build metrics and outcomes
+2. **Task-level telemetry** - Information about task execution
+3. **Target-level telemetry** - Information about target execution and incrementality
+4. **Error categorization telemetry** - Classification of build failures
+
+All telemetry events use the `VS/MSBuild/` prefix as required by VS exporting/collection. Properties use the `VS.MSBuild.` prefix.
+
+---
+
+## 1. Build Telemetry (`build` event)
+
+The primary telemetry event capturing overall build information.
+
+### Properties
+
+| Property | Type | Description |
+|----------|------|-------------|
+| `BuildDurationInMilliseconds` | double | Total build duration from start to finish |
+| `InnerBuildDurationInMilliseconds` | double | Duration from when BuildManager starts (excludes server connection time) |
+| `BuildEngineHost` | string | Host environment: "VS", "VSCode", "Azure DevOps", "GitHub Action", "CLI", etc. |
+| `BuildSuccess` | bool | Whether the build succeeded |
+| `BuildTarget` | string | The target(s) being built |
+| `BuildEngineVersion` | Version | MSBuild engine version |
+| `BuildEngineDisplayVersion` | string | Display-friendly engine version |
+| `BuildEngineFrameworkName` | string | Runtime framework name |
+| `BuildCheckEnabled` | bool | Whether BuildCheck (static analysis) was enabled |
+| `MultiThreadedModeEnabled` | bool | Whether multi-threaded build mode was enabled |
+| `SACEnabled` | bool | Whether Smart Application Control was enabled |
+| `IsStandaloneExecution` | bool | True if MSBuild runs from command line |
+| `InitialMSBuildServerState` | string | Server state before build: "cold", "hot", or null |
+| `ServerFallbackReason` | string | If server was bypassed: "ServerBusy", "ConnectionError", or null |
+| `ProjectPath` | string | Path to the project file being built |
+| `FailureCategory` | string | Primary failure category when build fails (see Error Categorization) |
+| `ErrorCounts` | object | Breakdown of errors by category (see Error Categorization) |
+
+---
+
+## 2. Error Categorization
+
+When a build fails, errors are categorized to help identify the source of failures.
+
+### Error Categories
+
+| Category | Error Code Prefixes | Description |
+|----------|---------------------|-------------|
+| `Compiler` | CS, FS, VBC | C#, F#, and Visual Basic compiler errors |
+| `MsBuildGeneral` | MSB4001-MSB4099, MSB4500-MSB4999 | General MSBuild errors |
+| `MsBuildEvaluation` | MSB4100-MSB4199 | Project evaluation errors |
+| `MsBuildExecution` | MSB4300-MSB4399, MSB5xxx-MSB6xxx | Build execution errors |
+| `MsBuildGraph` | MSB4400-MSB4499 | Static graph build errors |
+| `Task` | MSB3xxx | Task-related errors |
+| `SdkResolvers` | MSB4200-MSB4299 | SDK resolution errors |
+| `NetSdk` | NETSDK | .NET SDK errors |
+| `NuGet` | NU | NuGet package errors |
+| `BuildCheck` | BC | BuildCheck rule violations |
+| `NativeToolchain` | LNK, C1xxx-C4xxx, CL | Native C/C++ toolchain errors (linker, compiler) |
+| `CodeAnalysis` | CA, IDE | Code analysis and IDE analyzer errors |
+| `Razor` | RZ | Razor compilation errors |
+| `Wpf` | XC, MC | WPF/XAML compilation errors |
+| `AspNet` | ASP, BL | ASP.NET and Blazor errors |
+| `Other` | (all others) | Uncategorized errors |
+
+## 3. Task Telemetry
+
+### Task Factory Event (`build/tasks/taskfactory`)
+
+Tracks which task factories are being used.
+
+| Property | Type | Description |
+|----------|------|-------------|
+| `AssemblyTaskFactoryTasksExecutedCount` | int | Tasks loaded via AssemblyTaskFactory |
+| `IntrinsicTaskFactoryTasksExecutedCount` | int | Built-in intrinsic tasks |
+| `CodeTaskFactoryTasksExecutedCount` | int | Tasks created via CodeTaskFactory |
+| `RoslynCodeTaskFactoryTasksExecutedCount` | int | Tasks created via RoslynCodeTaskFactory |
+| `XamlTaskFactoryTasksExecutedCount` | int | Tasks created via XamlTaskFactory |
+| `CustomTaskFactoryTasksExecutedCount` | int | Tasks from custom task factories |
+
+## 4. Build Incrementality Telemetry
+
+Classifies builds as full or incremental based on target execution patterns.
+
+### Incrementality Info (Activity Property)
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `Classification` | enum | `Full`, `Incremental`, or `Unknown` |
+| `TotalTargetsCount` | int | Total number of targets |
+| `ExecutedTargetsCount` | int | Targets that ran |
+| `SkippedTargetsCount` | int | Targets that were skipped |
+| `SkippedDueToUpToDateCount` | int | Skipped because outputs were current |
+| `SkippedDueToConditionCount` | int | Skipped due to false condition |
+| `SkippedDueToPreviouslyBuiltCount` | int | Skipped because already built |
+| `IncrementalityRatio` | double | Ratio of skipped to total (0.0-1.0) |
+
+A build is classified as **Incremental** when more than 70% of targets are skipped.
+
+---
+
+## Privacy Considerations
+
+### Data Hashing
+
+Custom and potentially sensitive data is hashed using SHA-256 before being sent:
+- **Custom task names** - Hashed to protect proprietary task names
+- **Custom target names** - Hashed to protect proprietary target names
+- **Custom task factory names** - Hashed if not in the known list
+- **Metaproj target names** - Hashed to protect solution structure
+
+### Known Task Factory Names (Not Hashed)
+
+The following Microsoft-owned task factory names are sent in plain text:
+- `AssemblyTaskFactory`
+- `TaskHostFactory`
+- `CodeTaskFactory`
+- `RoslynCodeTaskFactory`
+- `XamlTaskFactory`
+- `IntrinsicTaskFactory`
+
+## Related Files
+
+| File | Description |
+|------|-------------|
+| [BuildTelemetry.cs](../src/Framework/Telemetry/BuildTelemetry.cs) | Main build telemetry class |
+| [BuildInsights.cs](../src/Framework/Telemetry/BuildInsights.cs) | Container for detailed insights |
+| [TelemetryDataUtils.cs](../src/Framework/Telemetry/TelemetryDataUtils.cs) | Data transformation utilities |
+| [BuildErrorTelemetryTracker.cs](../src/Build/BackEnd/Components/Logging/BuildErrorTelemetryTracker.cs) | Error categorization |
+| [ProjectTelemetry.cs](../src/Build/BackEnd/Components/Logging/ProjectTelemetry.cs) | Per-project task telemetry |
+| [LoggingConfigurationTelemetry.cs](../src/Framework/Telemetry/LoggingConfigurationTelemetry.cs) | Logger configuration |
+| [BuildCheckTelemetry.cs](../src/Framework/Telemetry/BuildCheckTelemetry.cs) | BuildCheck telemetry |
+| [KnownTelemetry.cs](../src/Framework/Telemetry/KnownTelemetry.cs) | Static telemetry accessors |
+| [TelemetryConstants.cs](../src/Framework/Telemetry/TelemetryConstants.cs) | Telemetry naming constants |

src/Build.UnitTests/BackEnd/BuildTelemetryErrorCategorization_Tests.cs

@@ -9,6 +9,7 @@
 using Microsoft.Build.Shared;
 using Shouldly;
 using Xunit;
+using static Microsoft.Build.BackEnd.Logging.BuildErrorTelemetryTracker;
 
 #nullable disable
 
@@ -17,19 +18,19 @@ namespace Microsoft.Build.UnitTests.BackEnd;
 public class BuildTelemetryErrorCategorization_Tests
 {
     [Theory]
-    [InlineData("CS0103", null, "Compiler")]
-    [InlineData("CS1002", "CS", "Compiler")]
-    [InlineData("VBC30451", "VBC", "Compiler")]
-    [InlineData("FS0039", null, "Compiler")]
-    [InlineData("MSB4018", null, "MSBuildEngine")]
-    [InlineData("MSB4236", null, "SDKResolvers")]
-    [InlineData("MSB3026", null, "Tasks")]
-    [InlineData("NETSDK1045", null, "NETSDK")]
-    [InlineData("NU1101", null, "NuGet")]
-    [InlineData("BC0001", null, "BuildCheck")]
-    [InlineData("CUSTOM001", null, "Other")]
-    [InlineData(null, null, "Other")]
-    [InlineData("", null, "Other")]
+    [InlineData("CS0103", null, nameof(ErrorCategory.Compiler))]
+    [InlineData("CS1002", "CS", nameof(ErrorCategory.Compiler))]
+    [InlineData("VBC30451", "VBC", nameof(ErrorCategory.Compiler))]
+    [InlineData("FS0039", null, nameof(ErrorCategory.Compiler))]
+    [InlineData("MSB4018", null, nameof(ErrorCategory.MSBuildGeneral))]
+    [InlineData("MSB4236", null, nameof(ErrorCategory.SDKResolvers))]
+    [InlineData("MSB3026", null, nameof(ErrorCategory.Tasks))]
+    [InlineData("NETSDK1045", null, nameof(ErrorCategory.NETSDK))]
+    [InlineData("NU1101", null, nameof(ErrorCategory.NuGet))]
+    [InlineData("BC0001", null, nameof(ErrorCategory.BuildCheck))]
+    [InlineData("CUSTOM001", null, nameof(ErrorCategory.Other))]
+    [InlineData(null, null, nameof(ErrorCategory.Other))]
+    [InlineData("", null, nameof(ErrorCategory.Other))]
     public void ErrorCategorizationWorksCorrectly(string errorCode, string subcategory, string expectedCategory)
     {
         // Create a LoggingService
@@ -63,28 +64,28 @@ public void ErrorCategorizationWorksCorrectly(string errorCode, string subcatego
             // Verify the appropriate count is incremented
             switch (expectedCategory)
             {
-                case "Compiler":
+                case nameof(ErrorCategory.Compiler):
                     buildTelemetry.ErrorCounts.Compiler.ShouldBe(1);
                     break;
-                case "MSBuildEngine":
-                    buildTelemetry.ErrorCounts.MsBuildEngine.ShouldBe(1);
+                case nameof(ErrorCategory.MSBuildGeneral):
+                    buildTelemetry.ErrorCounts.MsBuildGeneral.ShouldBe(1);
                     break;
-                case "Tasks":
+                case nameof(ErrorCategory.Tasks):
                     buildTelemetry.ErrorCounts.Task.ShouldBe(1);
                     break;
-                case "SDKResolvers":
+                case nameof(ErrorCategory.SDKResolvers):
                     buildTelemetry.ErrorCounts.SdkResolvers.ShouldBe(1);
                     break;
-                case "NETSDK":
+                case nameof(ErrorCategory.NETSDK):
                     buildTelemetry.ErrorCounts.NetSdk.ShouldBe(1);
                     break;
-                case "NuGet":
+                case nameof(ErrorCategory.NuGet):
                     buildTelemetry.ErrorCounts.NuGet.ShouldBe(1);
                     break;
-                case "BuildCheck":
+                case nameof(ErrorCategory.BuildCheck):
                     buildTelemetry.ErrorCounts.BuildCheck.ShouldBe(1);
                     break;
-                case "Other":
+                case nameof(ErrorCategory.Other):
                     buildTelemetry.ErrorCounts.Other.ShouldBe(1);
                     break;
             }
@@ -125,13 +126,13 @@ public void MultipleErrorsAreCountedByCategory()
 
             // Verify counts
             buildTelemetry.ErrorCounts.Compiler.ShouldBe(2);
-            buildTelemetry.ErrorCounts.MsBuildEngine.ShouldBe(1);
+            buildTelemetry.ErrorCounts.MsBuildGeneral.ShouldBe(1);
             buildTelemetry.ErrorCounts.Task.ShouldBe(1);
             buildTelemetry.ErrorCounts.NuGet.ShouldBe(1);
             buildTelemetry.ErrorCounts.Other.ShouldBe(1);
 
             // Primary category should be Compiler (highest count)
-            buildTelemetry.FailureCategory.ShouldBe("Compiler");
+            buildTelemetry.FailureCategory.ShouldBe(nameof(ErrorCategory.Compiler));
         }
         finally
         {
@@ -166,7 +167,7 @@ public void PrimaryCategoryIsSetToHighestErrorCount()
             loggingService.PopulateBuildTelemetryWithErrors(buildTelemetry);
 
             // Primary category should be Tasks (3 errors vs 1 compiler error)
-            buildTelemetry.FailureCategory.ShouldBe("Tasks");
+            buildTelemetry.FailureCategory.ShouldBe(nameof(ErrorCategory.Tasks));
             buildTelemetry.ErrorCounts.Task.ShouldBe(3);
             buildTelemetry.ErrorCounts.Compiler.ShouldBe(1);
         }
@@ -204,7 +205,7 @@ public void SubcategoryIsUsedForCompilerErrors()
             loggingService.PopulateBuildTelemetryWithErrors(buildTelemetry);
 
             // Should be categorized as Compiler based on subcategory
-            buildTelemetry.FailureCategory.ShouldBe("Compiler");
+            buildTelemetry.FailureCategory.ShouldBe(nameof(ErrorCategory.Compiler));
             buildTelemetry.ErrorCounts.Compiler.ShouldBe(1);
         }
         finally