Skip to content

[release/13.2][CI] Unified test splitting infrastructure for CI parallelization#14858

Merged
radical merged 15 commits intodotnet:release/13.2from
radical:add-test-splitting-infrastructure
Mar 3, 2026
Merged

[release/13.2][CI] Unified test splitting infrastructure for CI parallelization#14858
radical merged 15 commits intodotnet:release/13.2from
radical:add-test-splitting-infrastructure

Conversation

@radical
Copy link
Member

@radical radical commented Mar 2, 2026

Description

Replace the per-category, per-OS test enumeration system with a unified, metadata-driven test matrix infrastructure. This consolidates multiple bespoke runsheet builders and workflow jobs into a single pipeline that automatically handles test splitting, OS expansion, and dependency-based job grouping.

Key changes

New tooling (tools/ and eng/scripts/)

  • ExtractTestPartitions — .NET tool that scans test assemblies for [Trait("Partition", "N")] to discover test splits
  • split-test-projects-for-ci.ps1 — discovers partitions from built assemblies and generates per-project JSON
  • build-test-matrix.ps1 — builds a canonical, platform-agnostic test matrix from per-project metadata
  • expand-test-matrix-github.ps1 — expands the canonical matrix into GitHub Actions format (per-OS entries)
  • split-test-matrix-by-deps.ps1 — splits the expanded matrix by dependency type (no-nugets, requires-nugets, requires-cli-archive)

MSBuild infrastructure (eng/, tests/Directory.Build.targets)

  • New TestEnumerationRunsheetBuilder replaces the old TestRunsheetBuilder
  • Each test project emits a .tests-metadata.json file during build with its properties (supported OSes, timeouts, dependencies, split config)
  • Projects with SplitTestsOnCI=true also produce a .tests-partitions.json with partition/class entries
  • Removed GetTestProjects.proj and TestRunsheetBuilder.targets

CI workflows (.github/)

  • enumerate-tests action: one build + expand instead of N separate per-category builds
  • tests.yml: collapsed from 3 per-OS setup jobs + 9 category×OS job groups into 1 setup + 4 dependency-based groups
  • deployment-tests.yml: uses the unified action with OnlyDeploymentTests=true
  • Removed tests-runner.yml (superseded by run-tests.yml)

Test project configuration

  • 6 test projects migrated to use SplitTestsOnCI, RequiresNugets, RequiresCliArchive, etc.
  • 118 test classes in Aspire.Hosting.Tests annotated with [Trait("Partition", "1-6")] for balanced CI splitting
  • Helix targets updated to consume the new .tests-partitions.json format

Testing & docs

  • New Infrastructure.Tests project validating all PowerShell scripts and the ExtractTestPartitions tool
  • docs/ci/TestingOnCI.md — comprehensive architecture documentation
  • rebalance-tests agent skill for analyzing CI timings and rebalancing partitions

** Testing done **

  • Github CI passes, including non-ci workflows
  • AzDo CI doesn't have any new issues
  • Runs the same number of tests as before

CI jobs layout

Before

ci-before-graph

After

ci-after-graph

Top slowest test jobs:

Before

Screenshot 2026-02-27 at 14 37 17

After

Screenshot 2026-02-27 at 14 37 27

radical and others added 13 commits March 2, 2026 17:10
@radical
Add ExtractTestPartitions tool and CI test matrix scripts
04378a3 
Add tooling for automated CI test splitting and matrix generation:

- ExtractTestPartitions: .NET tool that scans test assemblies for
  [Trait("Partition", "N")] attributes to discover test partitions
- split-test-projects-for-ci.ps1: discovers partitions from assemblies
  and generates per-project partition JSON files
- build-test-matrix.ps1: builds a canonical test matrix JSON from
  per-project metadata files
- expand-test-matrix-github.ps1: expands the canonical matrix into
  GitHub Actions format with per-OS job entries
- split-test-matrix-by-deps.ps1: splits the expanded matrix by
  dependency type (no-nugets, requires-nugets, requires-cli-archive)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@radical
Replace TestRunsheetBuilder with TestEnumerationRunsheetBuilder
e74e0ee 
Replace the old per-OS runsheet generation with a new metadata-driven
approach:

- Add TestEnumerationRunsheetBuilder.targets: generates per-project
  .tests-metadata.json files during build, containing project metadata
  (supported OSes, timeouts, dependencies, split configuration)
- Update AfterSolutionBuild.targets: add _GenerateCanonicalMatrixJson
  target that invokes build-test-matrix.ps1 to produce a canonical
  test matrix from the metadata files
- Update Testing.props: add default timeout values and _ShortName
  computation for CI display names
- Update tests/Directory.Build.targets: replace ExtractTestClassNames
  with GenerateTestPartitionsForCI target, add _ShouldArchiveTests
  logic after Testing.targets import
- Remove TestRunsheetBuilder.targets and GetTestProjects.proj

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@radical
Configure test projects for unified CI splitting infrastructure
6ffa6de 
Update test project files to use the new TestEnumerationRunsheetBuilder:

- Aspire.Hosting.Tests: enable SplitTestsOnCI with partition-based
  splitting via TestClassNamePrefixForCI
- Aspire.Cli.EndToEnd.Tests: migrate from ExtractTestClassNamesForHelix
  to SplitTestsOnCI, add RequiresCliArchive property
- Aspire.Deployment.EndToEnd.Tests: migrate to SplitTestsOnCI, enable
  RunOnGithubActionsLinux (controlled by OnlyDeploymentTests filter)
- Aspire.EndToEnd.Tests: add RequiresNugets property
- Aspire.Templates.Tests: migrate to SplitTestsOnCI, add RequiresNugets,
  RequiresTestSdk, EnablePlaywrightInstall, and timeout properties
- Aspire.Playground.Tests: remove obsolete TestRunnerPreCommand

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@radical
Simplify CI workflows to use unified test matrix
5429bb2 
Rework the CI test orchestration to use a single, unified test matrix:

- enumerate-tests action: replace per-category build steps with a
  single build using TestEnumerationRunsheetBuilder, then expand via
  expand-test-matrix-github.ps1. Output a single all_tests matrix.
- tests.yml: collapse 3 per-OS setup jobs into 1, replace 9
  category×OS job groups with 4 dependency-based groups (no-nugets,
  no-nugets-overflow, requires-nugets, requires-cli-archive)
- deployment-tests.yml: use the unified enumerate-tests action with
  OnlyDeploymentTests=true, consume all_tests output
- run-tests.yml: pass through testProjectPath, timeouts, and
  extraTestArgs from matrix entries
- Remove tests-runner.yml (superseded by run-tests.yml)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@radical
Update Helix targets to consume new partitions JSON format
b3dff04 
Update send-to-helix-templatestests.targets to read test class names
from the new .tests-partitions.json format (produced by
GenerateTestPartitionsForCI) instead of the old .tests.list format.
Strip the 'class:' prefix from partition entries for compatibility
with the existing Helix work item generation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@radical
Add Infrastructure.Tests for CI tooling validation
5195096 
Add a new test project that validates the CI test splitting
infrastructure:

- ExtractTestPartitionsTests: tests the ExtractTestPartitions tool
  using dynamically-generated mock assemblies with various trait
  and attribute configurations
- BuildTestMatrixTests: validates build-test-matrix.ps1 canonical
  matrix generation from metadata files
- ExpandTestMatrixGitHubTests: validates expand-test-matrix-github.ps1
  GitHub Actions matrix expansion
- SplitTestMatrixByDepsTests: validates split-test-matrix-by-deps.ps1
  dependency-based matrix splitting
- SplitTestProjectsTests: validates split-test-projects-for-ci.ps1
  partition discovery
- Shared TestDataBuilder and PowerShellCommand helpers

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@radical
Add CI testing documentation and rebalance-tests skill
b44c928 
- docs/ci/TestingOnCI.md: comprehensive documentation covering the
  test splitting architecture, MSBuild properties, matrix generation
  pipeline, and how to add/rebalance test partitions
- .github/skills/rebalance-tests/SKILL.md: agent skill for analyzing
  CI build timings, downloading TRX files, computing optimal partition
  assignments via greedy bin-packing, and applying changes
- Update CLI E2E testing SKILL.md and README.md to reference the
  unified TestEnumerationRunsheetBuilder infrastructure

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@radical
Add partition traits to Aspire.Hosting.Tests for CI parallelization
08a6141 
Add [Trait("Partition", "N")] attributes to 118 test classes in
Aspire.Hosting.Tests, distributing them across 6 partitions. This
enables the CI infrastructure to split this large test project into
parallel jobs, reducing wall-clock test time.

Partition assignments are balanced by estimated test duration using
greedy bin-packing across the partitions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@radical
Remove rebalance-tests skill
0b1f639 
Will be developed separately.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@radical
Fix ExtractTestPartitions race condition and remove silent fallback
899b0e2 
- Make build/run failures of ExtractTestPartitions hard errors instead of
  silently falling back to class-based splitting
- Pre-build the tool in enumerate-tests action before parallel test matrix
  generation to eliminate file locking race condition

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@radical
Fix CLI E2E tests: add missing PR env vars to nuget-dependent test steps
f153709 
The GITHUB_PR_NUMBER, GITHUB_PR_HEAD_SHA, and GH_TOKEN environment
variables were only set on the non-nuget test steps, but CLI E2E tests
have requiresNugets=true and always run via the nuget-dependent path.
This caused 'aspire: command not found' because the install script
could not authenticate or identify the PR to download artifacts from.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@radical
Fix GenerateTestPartitionsForCI running during GitHub Actions test ex…
7338139 
…ecution

The GenerateTestPartitionsForCI target was firing during test execution jobs
in run-tests.yml because PrepareForHelix=true (passed for archiving) combined
with RunOnAzdoHelix=true (default) satisfied the condition. This caused
split-test-projects-for-ci.ps1 to run and fail since ExtractTestPartitions
wasn't built in that job.

Add IsGitHubActionsRunner!=true to the PrepareForHelix branch so partition
generation only runs via GenerateCIPartitions=true on GitHub Actions
(set by the enumerate-tests action) and via PrepareForHelix on AzDo Helix.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@radical
Fix RepoRoot path quoting for Windows in GenerateTestPartitionsForCI
b0e60dd 
On Windows, RepoRoot ends with backslash (e.g. D:\a\_work\1\s\). When
wrapped in quotes for the PowerShell command, the trailing backslash
escapes the closing quote, embedding a literal quote character in the
path. TrimEnd the backslash before quoting to fix this.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 2, 2026 22:13
@radical radical added the area-engineering-systems infrastructure helix infra engineering repo stuff label Mar 2, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Mar 2, 2026 •
edited
Loading

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/dotnet/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 14858

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/dotnet/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 14858"

@radical radical changed the title Add test splitting infrastructure [release/13.2][CI] Unified test splitting infrastructure for CI parallelization Mar 2, 2026
Copilot AI reviewed Mar 2, 2026
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Replaces the old per-category/per-OS test enumeration approach with a unified, metadata-driven test matrix pipeline. The new infrastructure emits per-test-project metadata + optional split definitions during build, then generates a canonical matrix that’s expanded and dependency-split for CI consumption.

Changes:

  • Added ExtractTestPartitions tool + PowerShell scripts to generate a canonical test matrix, expand it for GitHub Actions, and split it by dependency type.
  • Updated MSBuild test infrastructure to emit .tests-metadata.json and (when enabled) .tests-partitions.json, and to drive test splitting via SplitTestsOnCI.
  • Updated GitHub Actions workflows and the enumerate-tests composite action to consume the unified matrix, collapsing many bespoke jobs into a smaller set of dependency-based job groups.

Reviewed changes

Copilot reviewed 158 out of 158 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tools/ExtractTestPartitions/Program.cs New tool that scans assemblies for [Trait("Partition", ...)] to discover partition names.
tools/ExtractTestPartitions/ExtractTestPartitions.csproj Adds project for the partition extraction tool.
tests/helix/send-to-helix-templatestests.targets Switches Helix template test discovery to consume .tests-partitions.json.
tests/Shared/GetTestProjects.proj Removes old test project enumeration mechanism.
tests/Infrastructure.Tests/Shared/TestDataBuilder.cs Adds helpers to generate metadata/matrix JSON inputs for script tests.
tests/Infrastructure.Tests/PowerShellScripts/SplitTestProjectsTests.cs Adds tests for split-test-projects-for-ci.ps1.
tests/Infrastructure.Tests/PowerShellScripts/SplitTestMatrixByDepsTests.cs Adds tests for split-test-matrix-by-deps.ps1.
tests/Infrastructure.Tests/PowerShellScripts/PowerShellCommand.cs Adds a test helper for invoking PowerShell scripts from xUnit.
tests/Infrastructure.Tests/PowerShellScripts/BuildTestMatrixTests.cs Adds tests for build-test-matrix.ps1.
tests/Infrastructure.Tests/Infrastructure.Tests.csproj Introduces a new test project covering infra scripts/tooling.
tests/Infrastructure.Tests/ExtractTestPartitions/MockAssemblyBuilder.cs Adds Roslyn-based mock assembly generator for partition extraction tests.
tests/Infrastructure.Tests/ExtractTestPartitions/ExtractTestPartitionsTests.cs Adds unit tests for the ExtractTestPartitions tool.
tests/Infrastructure.Tests/ExtractTestPartitions/ExtractTestPartitionsFixture.cs Adds fixture to build the tool once for test suite performance.
tests/Directory.Build.targets Adds GenerateTestPartitionsForCI, refactors archiving gating via _ShouldArchiveTests.
tests/Aspire.Templates.Tests/Aspire.Templates.Tests.csproj Enables SplitTestsOnCI + marks NuGet/SDK requirements for templates tests.
tests/Aspire.Playground.Tests/Aspire.Playground.Tests.csproj Removes legacy TestRunnerPreCommand hook used by old runsheet builder.
tests/Aspire.Hosting.Tests/WithUrlsTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/WithReferenceTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/WithOtlpExporterTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/WithMcpServerTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/WithIconNameTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/WithHttpCommandTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/WithEnvironmentTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/WithEndpointTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/WithCertificateAuthorityCollection.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/WaitForTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/VersionChecking/VersionCheckServiceTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/VersionChecking/PackageUpdateHelpersTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ValueSnapshotTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Utils/WithAnnotationTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Utils/VolumeNameGeneratorTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Utils/UseCultureAttributeTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Utils/StringComparersTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Utils/PeriodicRestartAsyncEnumerableTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Utils/PasswordGeneratorTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Utils/EnvironmentVariableNameEncoderTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Utils/ContainerReferenceParserTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Utils/CommandLineArgsParserTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/UserSecretsParameterDefaultTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/StableConnectionStringBuilderTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/SlimTestProgramTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/SecretsStoreTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Schema/SchemaTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ResourceWithProbeTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ResourceNotificationTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ResourceLoggerServiceTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ResourceExtensionsTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ResourceDependencyTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ResourceCommandServiceTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ResourceCommandAnnotationTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/RequiredCommandAnnotationTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ReferenceExpressionTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Publishing/ResourceContainerImageManagerTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Publishing/PublishingExtensionsTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Publishing/PipelineActivityReporterTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Publishing/DeploymentStateManagerTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/PublishAsDockerfileTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/PublishAsConnectionStringTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ProjectResourceTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/PortAllocatorTest.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Pipelines/PipelineSummaryTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Pipelines/PipelineLoggerProviderTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Pipelines/DistributedApplicationPipelineTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/PathLookupHelperTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Orchestrator/RelationshipEvaluatorTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Orchestrator/ParameterProcessorTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Orchestrator/ApplicationOrchestratorTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/OperationModesTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ModelNameTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ManifestGenerationTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/MSBuildTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/LaunchSettingsSerializerContextTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/KestrelConfigTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/JsonFlattenerTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/InteractionServiceTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/HealthCheckTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Health/ResourceHealthCheckServiceTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Health/HealthStatusTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/FileSystemServiceTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ExternalServiceTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ExpressionResolverTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ExecutionConfigurationGathererTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ExecutableResourceTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ExecutableResourceBuilderExtensionTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Eventing/DistributedApplicationBuilderEventingTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/EndpointReferenceTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/DistributedApplicationTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/DistributedApplicationOptionsTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/DistributedApplicationModelExtensionsTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/DistributedApplicationBuilderTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/DistributedApplicationBuilderExtensionsTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Devcontainers/SshRemoteOptionsTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/DcpVisibilityTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Dcp/DcpLogParserTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Dcp/DcpHostNotificationTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Dcp/DcpExecutorTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Dcp/DcpCliArgsTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Dashboard/TransportOptionsValidatorTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Dashboard/ResourcePublisherTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Dashboard/DashboardServiceTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Dashboard/DashboardResourceTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Dashboard/DashboardOptionsTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Dashboard/DashboardLifecycleHookTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ContainerTunnelTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ContainerRegistryResourceTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ContainerImagePushOptionsTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ConnectionPropertiesExtensionsTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Codespaces/CodespacesUrlRewriterTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Cli/CliOrphanDetectorTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Backchannel/JsonElementConversionTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Backchannel/Exec/ProjectResourceExecTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Backchannel/Exec/ContainerResourceExecTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Backchannel/BackchannelContractTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Backchannel/AuxiliaryBackchannelTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Backchannel/AuxiliaryBackchannelRpcTargetTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Backchannel/AppHostBackchannelTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Ats/AtsCapabilityScannerTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/AspireStoreTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/Aspire.Hosting.Tests.csproj Enables SplitTestsOnCI and sets TestClassNamePrefixForCI.
tests/Aspire.Hosting.Tests/AsHttp2ServiceTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ApplicationModel/McpServerEndpointAnnotationTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ApplicationModel/GenerateParameterDefaultTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ApplicationModel/EndpointHostHelpersTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ApplicationModel/Docker/WithDockerfileBuilderTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ApplicationModel/Docker/DockerfileStatementsTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ApplicationModel/Docker/DockerfileStageTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ApplicationModel/Docker/DockerfileBuilderTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ApplicationModel/Docker/DockerfileBuilderIntegrationTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ApplicationModel/Docker/DockerfileBuilderEdgeCasesTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ApplicationModel/Docker/DockerfileBuilderCallbackContextTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ApplicationModel/Docker/DockerfileBuilderCallbackAnnotationTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/ApplicationModel/Docker/DockerfileBuildAnnotationTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/AppHostSmokeTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/AddParameterTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.Hosting.Tests/AddConnectionStringTests.cs Adds Partition trait annotation for CI splitting.
tests/Aspire.EndToEnd.Tests/Aspire.EndToEnd.Tests.csproj Marks EndToEnd tests as RequiresNugets.
tests/Aspire.Deployment.EndToEnd.Tests/Aspire.Deployment.EndToEnd.Tests.csproj Enables SplitTestsOnCI, Linux GHA enablement, and sets a shorter display name.
tests/Aspire.Cli.EndToEnd.Tests/README.md Updates documentation to the unified enumeration infrastructure.
tests/Aspire.Cli.EndToEnd.Tests/Aspire.Cli.EndToEnd.Tests.csproj Enables splitting, sets dependency flags, and configures CI/Helix behavior.
eng/scripts/split-test-projects-for-ci.ps1 Adds script to generate .tests-partitions.json via traits or --list-tests fallback.
eng/scripts/split-test-matrix-by-deps.ps1 Adds script to split OS-expanded matrix by dependency type + overflow.
eng/scripts/expand-test-matrix-github.ps1 Adds script to expand canonical matrix to GHA runs-on entries.
eng/scripts/build-test-matrix.ps1 Adds canonical matrix builder from per-project metadata and partitions.
eng/pipelines/templates/BuildAndTest.yml Ensures /p:ArchiveTests is set when running Helix tests.
eng/Testing.targets Adds default SplitTestsOnCI=false.
eng/Testing.props Adds default timeout properties and derives _ShortName for CI display.
eng/TestRunsheetBuilder/TestRunsheetBuilder.targets Removes legacy runsheet builder targets.
eng/TestEnumerationRunsheetBuilder/TestEnumerationRunsheetBuilder.targets New MSBuild “runsheet builder” that emits metadata and triggers partition generation.
eng/AfterSolutionBuild.targets Adds canonical matrix generation step; removes old _GenerateTestMatrix trigger for TestRunsheetBuilder.
docs/ci/TestingOnCI.md Adds documentation for the unified CI test matrix architecture.
.github/workflows/tests.yml Collapses many OS/category jobs into dependency-based matrix groups using the new enumerator + splitter.
.github/workflows/tests-runner.yml Removes superseded workflow.
.github/workflows/specialized-test-runner.yml Updates job naming to use matrix project name.
.github/workflows/run-tests.yml Adds requiresCliArchive input + gates PR metadata env vars; always archives when requested.
.github/workflows/deployment-tests.yml Switches to unified enumeration output and uses matrix-provided extraTestArgs.
.github/skills/cli-e2e-testing/SKILL.md Updates documentation to reflect unified enumeration infrastructure.
.github/actions/enumerate-tests/action.yml Replaces bespoke enumeration with canonical matrix generation + GitHub expansion output (all_tests).

tests/Infrastructure.Tests/Infrastructure.Tests.csproj
Comment on lines +13 to +16
<!-- Run only on Linux in Azdo Helix -->
<RunOnAzdoHelixWindows>false</RunOnAzdoHelixWindows>
<RunOnAzdoHelixLinux>false</RunOnAzdoHelixLinux>
</PropertyGroup>
Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment says “Run only on Linux in Azdo Helix”, but both RunOnAzdoHelixWindows and RunOnAzdoHelixLinux are set to false, which disables Helix entirely. Either the comment or the properties should be updated so they agree.

Copilot uses AI. Check for mistakes.
@github-actions github-actions bot mentioned this pull request Mar 3, 2026
Open
radical and others added 2 commits March 2, 2026 20:28
@radical
Update eng/TestEnumerationRunsheetBuilder/TestEnumerationRunsheetBuil…
7ad3683 
…der.targets

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@radical
Update eng/TestEnumerationRunsheetBuilder/TestEnumerationRunsheetBuil…
0c6a021 
…der.targets

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
mitchdenny
mitchdenny approved these changes Mar 3, 2026
View reviewed changes
Copy link
Member

@mitchdenny mitchdenny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI infrastructure only — zero product code changes. 118 files are single-line Partition trait additions, rest is new tooling/scripts/docs. Low merge conflict risk. CI is green and runs the same number of tests as before.

@radical radical merged commit 788b194 into dotnet:release/13.2 Mar 3, 2026
383 checks passed
@radical radical deleted the add-test-splitting-infrastructure branch March 3, 2026 04:36
@dotnet-policy-service dotnet-policy-service bot added this to the 13.2 milestone Mar 3, 2026
@github-actions github-actions bot mentioned this pull request Mar 3, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@mitchdenny mitchdenny mitchdenny approved these changes

@eerhardt eerhardt Awaiting requested review from eerhardt eerhardt is a code owner

@davidfowl davidfowl Awaiting requested review from davidfowl

@JamesNK JamesNK Awaiting requested review from JamesNK

@sebastienros sebastienros Awaiting requested review from sebastienros

Assignees

No one assigned

Labels

area-engineering-systems infrastructure helix infra engineering repo stuff

Projects

None yet

Milestone

13.2

Development

Successfully merging this pull request may close these issues.

3 participants

@radical @mitchdenny