Feature request: XML Report Generation for ansible-test

[ykim-1]

Proposal: XML Report Generation for ansible-test

Author: Youjung Kim @ykim-1

Date: 2024-01-11

• Status: New
• Proposal type: Feature
• Targeted release: N/A
• Associated PR: N/A
• Estimated time to implement: N/A

Motivation

The motivation behind this proposal is to enhance the reporting capabilities of ansible-test by introducing the option to generate XML reports. This will provide users with a standardized and other readable format for test results, making it easier to integrate with external tools and systems.

Problems

The current output format of ansible-test is limited to human-readable formats, such as console output or log files. This can be inconvenient when trying to automate the testing process or integrate it into continuous integration pipelines.

Solution proposal

I propose introducing a new option in ansible-test that allows users to generate XML reports. This can be achieved by extending the existing functionality and incorporating a suitable XML format for reporting. Here's an example of how the command might look:

ansible-test --xml-report

Additionally, the generated XML format could include relevant information such as test results, execution time, and any other details. This will enable users to easily parse and process the test results easier in test pipelines and repositories.

Dependencies (optional)
This feature does not have any direct dependencies on external libraries or tools. However, it may rely on existing ansible-test infrastructure.

Testing (optional)
Testing for this feature would involve running ansible-test with the new --xml-report option and verifying that the generated XML report is accurate and contains the expected information. Integration tests should be added to ensure compatibility with various scenarios.

[mattclay]

The current output format of ansible-test is limited to human-readable formats, such as console output or log files. This can be inconvenient when trying to automate the testing process or integrate it into continuous integration pipelines.

Unit and integration tests create JUnit output files by default. Sanity tests also do so when the --junit option is used.

@ykim-1 What output are you looking for that's not in the JUnit XML files?

[ykim-1]

Hey @mattclay , thanks for the quick reply. The solution you provided in your comment might actually solve my problem when exporting the test report after GHA executions. I will give it a go and let you know

[ykim-1]

Hello @mattclay , the integration tests I am currently running in the GH repository requires the tests to be run with integration option. E.g. ansible-test integration -v api_request_basic
Can I request --junit option as a feature for integration tests as well?

[mattclay]

Hello @mattclay , the integration tests I am currently running in the GH repository requires the tests to be run with integration option. E.g. ansible-test integration -v api_request_basic Can I request --junit option as a feature for integration tests as well?

@ykim-1 Integration tests always generate JUnit output. That's why there is no --junit option.

[ykim-1]

@mattclay Ah ok, I was expecting some sort of xml format to be the output but after doing some research, Junit output can come in many different forms. I will play around with the test output and see what I can do. Thanks for the quick replies!