Skip to content

Increase Integration Test Parallelism #4241

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
westey-m merged 1 commit into from
Feb 25, 2026
Merged

Increase Integration Test Parallelism #4241

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/dotnet-build-and-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,9 @@ jobs:
--no-build -v Normal `
--report-xunit-trx `
--ignore-exit-code 8 `
--filter-not-trait "Category=IntegrationDisabled"
--filter-not-trait "Category=IntegrationDisabled" `
--parallel-algorithm aggressive `
--max-threads 2.0x
Comment on lines +217 to +219
Copy link

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The aggressive parallel execution flags may conflict with XUnit's collection-based parallelization strategy that is deliberately used by integration tests to prevent resource conflicts.

Several integration test classes use XUnit collections to control parallelization:

  • CosmosDB tests use [CollectionDefinition("CosmosDB", DisableParallelization = true)] to prevent conflicts with the Cosmos DB Emulator
  • DurableTask tests use [Collection("Sequential")] to avoid resource contention

These collections exist because the integration tests share limited resources:

  • A single Cosmos DB Emulator instance (COSMOSDB_ENDPOINT: https://localhost:8081)
  • Durable Task and Azure Functions emulators
  • External Azure resources

Microsoft Testing Platform's parallel execution may not honor XUnit's collection-level parallelization controls, potentially causing test failures due to resource contention. Consider:

  1. Testing these changes thoroughly with the integration test suite to verify no test flakiness is introduced
  2. Using less aggressive parallelization (e.g., --max-threads 1.0x or no parallelization flags)
  3. Only applying parallelization to unit tests, which don't have these resource constraints
Suggested change
--filter-not-trait "Category=IntegrationDisabled" `
--parallel-algorithm aggressive `
--max-threads 2.0x
--filter-not-trait "Category=IntegrationDisabled"

Copilot uses AI. Check for mistakes.
Comment on lines +217 to +219
Copy link

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parallelization flags are applied to integration tests but not to unit tests (line 174-180). This is counterintuitive, as unit tests typically have fewer resource constraints and are better suited for aggressive parallelization than integration tests.

Consider applying these flags to unit tests instead, where:

  • Tests don't share emulators or external resources
  • Test isolation is typically better
  • Execution time benefits would be more reliable without risking test flakiness
Suggested change
--filter-not-trait "Category=IntegrationDisabled" `
--parallel-algorithm aggressive `
--max-threads 2.0x
--filter-not-trait "Category=IntegrationDisabled"

Copilot uses AI. Check for mistakes.
env:
# Cosmos DB Emulator connection settings
COSMOSDB_ENDPOINT: https://localhost:8081
Expand Down