Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FR: upload XML file with aggregate results separate from results.tar.gz #42

Open
lrumancik opened this issue Jan 27, 2023 · 1 comment

Comments

@lrumancik
Copy link
Contributor

For LTM runs, create a top level results.xml file which combines the per-config results.xml files. Upload this separately form the results.tar.gz file similarly to how we handle the summary file. Then, convert results processing scripts to download just this XML file and use it for processing.

@lrumancik
Copy link
Contributor Author

lrumancik commented Feb 6, 2023

There is already a file I use for post processing results: ./test-appliance/files/usr/lib/python3/dist-packages/get_stats.py

This stores the results in a condensed XML file.

Instead of an entry for each time a test runs
(ex.
generic/001 -> pass
generic/001 -> pass
generic/002 -> pass
generic/002 -> pass
...)

it instead creates a stats version
(ex.
generic/001 -> skips=0, fail=0, error=0, total=2
generic/002 -> skips=0, fail=0, error=0, total=2
...
)

My other post-processing scripts (merge_stats.py, diff_stats.py) operate on this format XML file to merging results form same kernel or compare results across different kernels. get_stats.py walks through each results directory to find the results similarly to what generate_report does. Perhaps we could combine the two and generate/upload this as the separate XML results file.

The XML file already has creates sections to distinguish between configs <config=XX/YY>.

File's current structure:

<configs>
        <config name="xfs/logdev">
                <test name="generic/001" failed="0" skipped="0" error="0" total="25"/>
                <test name="generic/002" failed="0" skipped="0" error="0" total="25"/>
                <test name="generic/003" failed="0" skipped="0" error="0" total="25"/>
                <test name="generic/004" failed="0" skipped="0" error="0" total="25"/>
                ...
	</config>
	...
</configs>

The generated file does not contain test run info though so we would want to copy over the properties of each test config / test VM setup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant