Skip to content

Conversation

@vbreuss
Copy link
Contributor

@vbreuss vbreuss commented Aug 12, 2025

This PR refactors the mutation testing pipeline to split execution into separate actions for better parallelization and maintainability. The mutation tests are now divided into two independent jobs that can run concurrently, followed by a dashboard aggregation step.

Key changes:

  • Split the monolithic mutation test execution into separate targets for main library (aweXpect) and core library (aweXpect.Core)
  • Introduced parallel GitHub Actions workflows for each mutation test component

@vbreuss vbreuss self-assigned this Aug 12, 2025
Copilot AI review requested due to automatic review settings August 12, 2025 05:18
@vbreuss vbreuss added the refactor A change or improvement without functional impact label Aug 12, 2025
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the mutation testing pipeline to split execution into separate actions for better parallelization and maintainability. The mutation tests are now divided into two independent jobs that can run concurrently, followed by a dashboard aggregation step.

Key changes:

  • Split the monolithic mutation test execution into separate targets for main library (aweXpect) and core library (aweXpect.Core)
  • Introduced parallel GitHub Actions workflows for each mutation test component
  • Added proper comment management for PR updates with project-specific mutation results

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File Description
Pipeline/Build.MutationTests.cs Refactored mutation testing logic to support parallel execution with separate targets and improved PR comment handling
.github/workflows/ci.yml Split mutation-tests job into two parallel jobs for main and core libraries
.github/workflows/build.yml Added three-stage mutation testing workflow with parallel execution and centralized dashboard reporting

@vbreuss vbreuss force-pushed the topic/split-mutation-tests branch from cf913e7 to fbea44f Compare August 12, 2025 05:20
@github-actions
Copy link
Contributor

github-actions bot commented Aug 12, 2025

Test Results

    14 files   - 24      14 suites   - 24   2m 52s ⏱️ +2s
14 358 tests  - 18  14 356 ✅  - 18  2 💤 ±0  0 ❌ ±0 
39 756 runs   - 72  39 754 ✅  - 72  2 💤 ±0  0 ❌ ±0 

Results for commit 0447d93. ± Comparison against base commit 02df387.

This pull request removes 1546 and adds 1528 tests. Note that renamed tests count towards both.
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "message3d18911f-85c7-4522-80fc-b0194945305d")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "message4a3d6422-7bab-453d-a5cf-8d02336b59a1")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "messagefcc9e3e6-9856-4b1b-8024-9d963c5f2f6c")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "message0072004b-4df5-43b0-b0cb-b81db73f822c")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "messagec106a9da-fd79-42c0-a585-a2f05f053b45")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "messagec7f5f5f1-3b11-4605-92e5-6030568948ea")
aweXpect.Core.Tests.Equivalency.EquivalencyOptionsExtensionsTests ‑ Generic_For_IgnoringMember_ShouldSetOptionForType(memberToIgnore: "memberToIgnore3b7bcdd4-416b-4702-bfb4-91d644e41727")
aweXpect.Core.Tests.Equivalency.EquivalencyOptionsExtensionsTests ‑ Generic_For_IgnoringMember_ShouldSetOptionForType(memberToIgnore: "memberToIgnore7fffe52e-a598-40df-8ffe-6ca347a7e4d5")
aweXpect.Core.Tests.Equivalency.EquivalencyOptionsExtensionsTests ‑ Generic_For_IgnoringMember_ShouldSetOptionForType(memberToIgnore: "memberToIgnore9b23e4d6-51cd-4a21-a99c-cd2329e3b731")
aweXpect.Core.Tests.FailTests ‑ Test_ShouldThrowException(reason: "reason30a94c9f-4d82-40b2-a0ac-edba5deee9b6")
…
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "message716ef2cc-aebc-474b-b703-e49c24e96be2")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "message9da4fee0-c6a5-473e-b5d8-3b09f0f4ad5f")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "messageb85fce33-3672-4b07-9f8c-f2373f4dea4c")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "message727ba742-3f69-4067-b6ed-bbafe10a711f")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "message7714825c-546d-4e48-ae29-66405f4d47fa")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "messagebb1278f5-5b47-4747-a604-b729a01ee4f4")
aweXpect.Core.Tests.Equivalency.EquivalencyOptionsExtensionsTests ‑ Generic_For_IgnoringMember_ShouldSetOptionForType(memberToIgnore: "memberToIgnore044b8050-5bbe-4fbc-913b-7b5c1a8436fe")
aweXpect.Core.Tests.Equivalency.EquivalencyOptionsExtensionsTests ‑ Generic_For_IgnoringMember_ShouldSetOptionForType(memberToIgnore: "memberToIgnore1123e68b-6522-47db-a861-da304f0dc02c")
aweXpect.Core.Tests.Equivalency.EquivalencyOptionsExtensionsTests ‑ Generic_For_IgnoringMember_ShouldSetOptionForType(memberToIgnore: "memberToIgnored1d42eb4-a7c2-4bee-80a2-08ca82b7aada")
aweXpect.Core.Tests.FailTests ‑ Test_ShouldThrowException(reason: "reason8e481ae5-91cf-48ea-a5c4-d56d85a5024e")
…

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 12, 2025

🚀 Benchmark Results

Details

BenchmarkDotNet v0.14.0, Ubuntu 24.04.2 LTS (Noble Numbat)
AMD EPYC 7763, 1 CPU, 4 logical and 2 physical cores
.NET SDK 8.0.413
[Host] : .NET 8.0.19 (8.0.1925.36514), X64 RyuJIT AVX2

Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=1 WarmupCount=10

Method Mean Error StdDev Gen0 Gen1 Allocated
Bool_aweXpect 201.0 ns 1.53 ns 1.28 ns 0.0281 - 472 B
Bool_FluentAssertions 246.4 ns 1.48 ns 1.39 ns 0.0567 - 952 B
Bool_TUnit 892.5 ns 6.13 ns 5.73 ns 0.1431 - 2408 B
Equivalency_aweXpect 300,456.3 ns 1,424.07 ns 1,189.17 ns 16.6016 0.4883 284940 B
Equivalency_FluentAssertions 2,327,448.9 ns 33,025.62 ns 30,892.19 ns 273.4375 46.8750 4584416 B
Equivalency_TUnit 693,927.9 ns 3,403.36 ns 3,016.99 ns 51.7578 2.9297 880545 B
Int_GreaterThan_aweXpect 218.3 ns 5.08 ns 4.50 ns 0.0467 - 784 B
Int_GreaterThan_FluentAssertions 254.5 ns 9.87 ns 9.23 ns 0.0730 - 1224 B
Int_GreaterThan_TUnit 1,215.9 ns 14.81 ns 13.85 ns 0.1774 - 2984 B
ItemsCount_AtLeast_aweXpect 472.9 ns 7.00 ns 6.21 ns 0.0854 - 1432 B
ItemsCount_AtLeast_FluentAssertions 497.0 ns 7.97 ns 7.45 ns 0.1192 - 2008 B
ItemsCount_AtLeast_TUnit 15,089.6 ns 198.67 ns 176.11 ns 1.6327 - 27480 B
String_aweXpect 363.7 ns 10.22 ns 9.56 ns 0.0672 - 1128 B
String_FluentAssertions 465.3 ns 5.06 ns 3.95 ns 0.1292 - 2168 B
String_TUnit 1,235.2 ns 16.35 ns 15.29 ns 0.1831 - 3072 B
StringArray_aweXpect 1,331.0 ns 4.59 ns 4.30 ns 0.1640 - 2744 B
StringArray_FluentAssertions 1,321.8 ns 20.10 ns 18.80 ns 0.2480 - 4152 B
StringArray_TUnit 3,328.5 ns 33.37 ns 31.21 ns 0.3128 - 5296 B
StringArrayInAnyOrder_aweXpect 1,546.8 ns 18.75 ns 17.54 ns 0.1736 - 2920 B
StringArrayInAnyOrder_FluentAssertions 153,482.2 ns 2,118.95 ns 1,982.06 ns 3.4180 - 63788 B
StringArrayInAnyOrder_TUnit 3,923.0 ns 43.33 ns 40.53 ns 0.3853 - 6488 B

@vbreuss vbreuss force-pushed the topic/split-mutation-tests branch from a2e8d64 to 0abf7f4 Compare August 13, 2025 13:11
@sonarqubecloud
Copy link

@vbreuss vbreuss merged commit 6c33916 into main Aug 13, 2025
14 checks passed
@vbreuss vbreuss deleted the topic/split-mutation-tests branch August 13, 2025 13:38
github-actions bot added a commit that referenced this pull request Aug 13, 2025
github-actions bot added a commit that referenced this pull request Aug 13, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Sep 2, 2025

This is addressed in release v2.22.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

refactor A change or improvement without functional impact state: released The issue is released

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants