Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
235 changes: 122 additions & 113 deletions requirements.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,102 +7,8 @@
sections:
- title: TestResults Library Requirements
sections:
- title: Functional Requirements
- title: Model
requirements:
- id: REQ-FUNC-001
title: The library shall support serializing test results to TRX (Visual Studio Test Results) format.
justification: |
TRX is the native test result format for Visual Studio and Azure DevOps. Supporting TRX
serialization enables the library to integrate seamlessly with Microsoft's testing ecosystem,
allowing users to generate test results that are compatible with Visual Studio Test Explorer,
Azure Pipelines, and other Microsoft tools.
tests:
- TrxSerializer_Serialize_BasicTestResults_ProducesValidTrxXml
- TrxSerializer_Serialize_MultipleTestResults_ProducesValidTrxXml

- id: REQ-FUNC-002
title: The library shall support deserializing TRX test result files.
justification: |
Deserializing TRX files allows users to read and analyze existing test results programmatically.
This is essential for test result aggregation, reporting, conversion to other formats, and
integration with custom analysis tools.
tests:
- TrxSerializer_Deserialize_BasicTrxXml_ReturnsTestResults
- TrxSerializer_Deserialize_ComplexTrxXml_ReturnsTestResults
- TrxExampleTests_Deserialize_Example1Trx_ReturnsAllTestResults
- TrxExampleTests_Deserialize_Example2Trx_ReturnsAllTestResults

- id: REQ-FUNC-003
title: The library shall support serializing test results to JUnit XML format.
justification: |
JUnit XML is a widely-adopted, cross-platform test result format supported by many CI/CD
systems including Jenkins, GitLab CI, GitHub Actions, and CircleCI. Supporting JUnit
serialization ensures the library can be used in diverse development environments beyond
the Microsoft ecosystem.
tests:
- JUnitSerializer_Serialize_PassedTest_ProducesValidJUnitXml
- JUnitSerializer_Serialize_FailedTest_IncludesFailureElement
- JUnitSerializer_Serialize_ErrorTest_IncludesErrorElement
- JUnitSerializer_Serialize_SkippedTest_IncludesSkippedElement
- JUnitSerializer_Serialize_TestWithOutput_IncludesSystemOutAndErr
- JUnitSerializer_Serialize_MultipleTestsInClasses_GroupsByClassName

- id: REQ-FUNC-004
title: The library shall support deserializing JUnit XML test result files.
justification: |
Deserializing JUnit XML enables reading test results from a wide variety of testing
frameworks and CI/CD systems. This capability is crucial for aggregating results from
different sources, converting between formats, and building unified reporting solutions.
tests:
- JUnitSerializer_Deserialize_BasicJUnitXml_ReturnsTestResults
- JUnitSerializer_Deserialize_FailedTest_ReturnsFailureDetails
- JUnitSerializer_Deserialize_ErrorTest_ReturnsErrorDetails
- JUnitSerializer_Deserialize_SkippedTest_ReturnsSkippedStatus
- JUnitSerializer_Deserialize_TestWithOutput_ReturnsSystemOutput
- JUnitSerializer_Deserialize_MultipleTestSuites_ReturnsAllTests

- id: REQ-FUNC-005
title: The library shall automatically identify test result file format.
justification: |
Automatic format detection simplifies usage by eliminating the need for users to
explicitly specify the format. This reduces potential for errors and improves the
user experience, especially when processing test results from multiple sources.
tests:
- Serializer_Identify_TrxContent_ReturnsTrx
- Serializer_Identify_JUnitTestsuitesContent_ReturnsJUnit
- Serializer_Identify_JUnitTestsuiteContent_ReturnsJUnit

- id: REQ-FUNC-006
title: The library shall support .NET 8 runtime.
justification: |
.NET 8 is a Long-Term Support (LTS) release with support through November 2026. Supporting
this framework ensures the library can be used in stable, enterprise environments that
require long-term maintenance guarantees.
tests:
- net8.0@TrxSerializer_Serialize_BasicTestResults_ProducesValidTrxXml
- net8.0@JUnitSerializer_Serialize_PassedTest_ProducesValidJUnitXml

- id: REQ-FUNC-007
title: The library shall support .NET 9 runtime.
justification: |
.NET 9 is a Standard-Term Support (STS) release that provides access to the latest
framework features and performance improvements. Supporting this version allows users
to benefit from the newest capabilities while maintaining compatibility with current
development practices.
tests:
- net9.0@TrxSerializer_Serialize_BasicTestResults_ProducesValidTrxXml
- net9.0@JUnitSerializer_Serialize_PassedTest_ProducesValidJUnitXml

- id: REQ-FUNC-008
title: The library shall support .NET 10 runtime.
justification: |
.NET 10 is the upcoming Long-Term Support (LTS) release scheduled for November 2025.
Supporting this framework ensures the library remains current and provides a clear
upgrade path for users as they migrate to future .NET versions.
tests:
- net10.0@TrxSerializer_Serialize_BasicTestResults_ProducesValidTrxXml
- net10.0@JUnitSerializer_Serialize_PassedTest_ProducesValidJUnitXml

- id: REQ-FUNC-009
title: The library shall support the Passed test outcome.
justification: |
Expand Down Expand Up @@ -192,6 +98,43 @@ sections:
tests:
- TestOutcome_IsPassed_PassedOutcome_ReturnsTrue

- id: REQ-FUNC-020
title: The library shall support capturing test output streams.
justification: |
Capturing standard output and error streams from tests is essential for debugging
test failures and understanding test behavior. This capability enables developers to
review console output, diagnostic messages, and error logs directly within test results.
tests:
- JUnitSerializer_Serialize_TestWithOutput_IncludesSystemOutAndErr
- JUnitSerializer_Deserialize_TestWithOutput_ReturnsSystemOutput
- TrxSerializer_Serialize_BasicTestResults_ProducesValidTrxXml
- Serializer_Deserialize_JUnitWithSystemOutput_ParsesCorrectly

- id: REQ-FUNC-021
title: The library shall support capturing error messages and stack traces.
justification: |
Error messages and stack traces are critical for diagnosing test failures. They provide
developers with immediate context about what went wrong and where, enabling faster
debugging and resolution of issues.
tests:
- JUnitSerializer_Serialize_FailedTest_IncludesFailureElement
- TrxSerializer_Serialize_StackTraceWithoutMessage_IncludesStackTraceElement
- JUnitSerializer_Deserialize_FailedTest_ReturnsFailureDetails
- TrxSerializer_Deserialize_ComplexTrxXml_ReturnsTestResults

- title: Serialization
requirements:
- id: REQ-FUNC-005
title: The library shall automatically identify test result file format.
justification: |
Automatic format detection simplifies usage by eliminating the need for users to
explicitly specify the format. This reduces potential for errors and improves the
user experience, especially when processing test results from multiple sources.
tests:
- Serializer_Identify_TrxContent_ReturnsTrx
- Serializer_Identify_JUnitTestsuitesContent_ReturnsJUnit
- Serializer_Identify_JUnitTestsuiteContent_ReturnsJUnit

- id: REQ-FUNC-018
title: The library shall support conversion between TRX and JUnit formats.
justification: |
Expand All @@ -216,26 +159,92 @@ sections:
- TrxSerializer_Deserialize_BasicTrxXml_ReturnsTestResults
- JUnitSerializer_Deserialize_BasicJUnitXml_ReturnsTestResults

- id: REQ-FUNC-020
title: The library shall support capturing test output streams.
sections:
- title: TRX Serialization
requirements:
- id: REQ-FUNC-001
title: The library shall support serializing test results to TRX (Visual Studio Test Results) format.
justification: |
TRX is the native test result format for Visual Studio and Azure DevOps. Supporting TRX
serialization enables the library to integrate seamlessly with Microsoft's testing ecosystem,
allowing users to generate test results that are compatible with Visual Studio Test Explorer,
Azure Pipelines, and other Microsoft tools.
tests:
- TrxSerializer_Serialize_BasicTestResults_ProducesValidTrxXml
- TrxSerializer_Serialize_MultipleTestResults_ProducesValidTrxXml

- id: REQ-FUNC-002
title: The library shall support deserializing TRX test result files.
justification: |
Deserializing TRX files allows users to read and analyze existing test results programmatically.
This is essential for test result aggregation, reporting, conversion to other formats, and
integration with custom analysis tools.
tests:
- TrxSerializer_Deserialize_BasicTrxXml_ReturnsTestResults
- TrxSerializer_Deserialize_ComplexTrxXml_ReturnsTestResults
- TrxExampleTests_Deserialize_Example1Trx_ReturnsAllTestResults
- TrxExampleTests_Deserialize_Example2Trx_ReturnsAllTestResults

- title: JUnit Serialization
requirements:
- id: REQ-FUNC-003
title: The library shall support serializing test results to JUnit XML format.
justification: |
JUnit XML is a widely-adopted, cross-platform test result format supported by many CI/CD
systems including Jenkins, GitLab CI, GitHub Actions, and CircleCI. Supporting JUnit
serialization ensures the library can be used in diverse development environments beyond
the Microsoft ecosystem.
tests:
- JUnitSerializer_Serialize_PassedTest_ProducesValidJUnitXml
- JUnitSerializer_Serialize_FailedTest_IncludesFailureElement
- JUnitSerializer_Serialize_ErrorTest_IncludesErrorElement
- JUnitSerializer_Serialize_SkippedTest_IncludesSkippedElement
- JUnitSerializer_Serialize_TestWithOutput_IncludesSystemOutAndErr
- JUnitSerializer_Serialize_MultipleTestsInClasses_GroupsByClassName

- id: REQ-FUNC-004
title: The library shall support deserializing JUnit XML test result files.
justification: |
Deserializing JUnit XML enables reading test results from a wide variety of testing
frameworks and CI/CD systems. This capability is crucial for aggregating results from
different sources, converting between formats, and building unified reporting solutions.
tests:
- JUnitSerializer_Deserialize_BasicJUnitXml_ReturnsTestResults
- JUnitSerializer_Deserialize_FailedTest_ReturnsFailureDetails
- JUnitSerializer_Deserialize_ErrorTest_ReturnsErrorDetails
- JUnitSerializer_Deserialize_SkippedTest_ReturnsSkippedStatus
- JUnitSerializer_Deserialize_TestWithOutput_ReturnsSystemOutput
- JUnitSerializer_Deserialize_MultipleTestSuites_ReturnsAllTests

- title: Runtime
requirements:
- id: REQ-FUNC-006
title: The library shall support .NET 8 runtime.
justification: |
Capturing standard output and error streams from tests is essential for debugging
test failures and understanding test behavior. This capability enables developers to
review console output, diagnostic messages, and error logs directly within test results.
.NET 8 is a Long-Term Support (LTS) release with support through November 2026. Supporting
this framework ensures the library can be used in stable, enterprise environments that
require long-term maintenance guarantees.
tests:
- JUnitSerializer_Serialize_TestWithOutput_IncludesSystemOutAndErr
- JUnitSerializer_Deserialize_TestWithOutput_ReturnsSystemOutput
- TrxSerializer_Serialize_BasicTestResults_ProducesValidTrxXml
- Serializer_Deserialize_JUnitWithSystemOutput_ParsesCorrectly
- net8.0@TrxSerializer_Serialize_BasicTestResults_ProducesValidTrxXml
- net8.0@JUnitSerializer_Serialize_PassedTest_ProducesValidJUnitXml

- id: REQ-FUNC-021
title: The library shall support capturing error messages and stack traces.
- id: REQ-FUNC-007
title: The library shall support .NET 9 runtime.
justification: |
Error messages and stack traces are critical for diagnosing test failures. They provide
developers with immediate context about what went wrong and where, enabling faster
debugging and resolution of issues.
.NET 9 is a Standard-Term Support (STS) release that provides access to the latest
framework features and performance improvements. Supporting this version allows users
to benefit from the newest capabilities while maintaining compatibility with current
development practices.
tests:
- JUnitSerializer_Serialize_FailedTest_IncludesFailureElement
- TrxSerializer_Serialize_StackTraceWithoutMessage_IncludesStackTraceElement
- JUnitSerializer_Deserialize_FailedTest_ReturnsFailureDetails
- TrxSerializer_Deserialize_ComplexTrxXml_ReturnsTestResults
- net9.0@TrxSerializer_Serialize_BasicTestResults_ProducesValidTrxXml
- net9.0@JUnitSerializer_Serialize_PassedTest_ProducesValidJUnitXml

- id: REQ-FUNC-008
title: The library shall support .NET 10 runtime.
justification: |
.NET 10 is the upcoming Long-Term Support (LTS) release scheduled for November 2025.
Supporting this framework ensures the library remains current and provides a clear
upgrade path for users as they migrate to future .NET versions.
tests:
- net10.0@TrxSerializer_Serialize_BasicTestResults_ProducesValidTrxXml
- net10.0@JUnitSerializer_Serialize_PassedTest_ProducesValidJUnitXml