add junit output and summaries for benchmarks

[altendky]

Purpose:

Current Behavior:

New Behavior:

https://github.com/Chia-Network/chia-blockchain/actions/runs/6616802037?pr=16660#summary-17971786470

Testing Notes:

[coveralls-official]

Pull Request Test Coverage Report for Build 6633372508

• 24 of 149 (16.11%) changed or added relevant lines in 6 files are covered.
• 14 unchanged lines in 6 files lost coverage.
• Overall coverage decreased (-0.1%) to 90.006%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
tests/core/test_cost_calculation.py	0	1	0.0%
tests/wallet/test_offer_parsing_performance.py	0	2	0.0%
tests/core/full_node/test_performance.py	0	3	0.0%
tests/conftest.py	12	17	70.59%
tests/util/misc.py	12	23	52.17%
tests/process_benchmarks.py	0	103	0.0%

Files with Coverage Reduction	New Missed Lines	%
chia/server/node_discovery.py	1	78.01%
tests/util/misc.py	1	61.75%
chia/farmer/farmer_api.py	2	87.14%
chia/full_node/full_node_api.py	2	77.24%
chia/timelord/timelord.py	2	73.64%
chia/wallet/wallet_node.py	6	87.51%

Totals
Change from base Build 6632571877:	-0.1%
Covered Lines:	93152
Relevant Lines:	103435

---

💛 - Coveralls

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[aqk]

--benchmark-repeats is a tiny bit confusing (eg. does it mean num_benchmark_repeats, or do_benchmark_repeats)
https://github.com/Chia-Network/chia-blockchain/pull/16660/files#diff-e52e4ddd58b7ef887ab03c04116e676f6280b824ab7469d5d3080e5cba4f2128R378

[aqk]

You could be a little less fancy here - keep the regex match, then just str.replace() [ and ]
https://github.com/Chia-Network/chia-blockchain/pull/16660/files#diff-4d8b94eb59960fd8e48cf3187b1c63427acef34c56b07ee835ec665288a66096R122

[altendky]

--benchmark-repeats is a tiny bit confusing (eg. does it mean num_benchmark_repeats, or do_benchmark_repeats) #16660 (files)

Oy, I did not add help text for the --benchmark-repeats option. Would that be good? Or do you want a different name.

[altendky]

You could be a little less fancy here - keep the regex match, then just str.replace() [ and ] #16660 (files)

I don't quite follow, but also this code is... not exactly clear about the purpose. It is meant to take the parametrization of the form [a-b-c] for the three parameter ids a, b, and c and remove a parametrization id of the form benchmark_repeat000 regardless of it being the only item, first item, middle item, or last item. Basically to get back to the parametrization as it would be presented if there were no benchmark repeat fixture parametrization in use. All without ending up with -- or [- or -]. But maybe that's what you're saying. Let those happen and then just do three replaces to catch those cases?

I probably ought to just do this on the test run side of things and then record that as another property in the results. I expect I would be able to do it there without any string manipulation.

[aqk]

Ya, I was just saying you didn't have to do the match and sub at the same time, and that might help sidestep the issue you were commenting on.

[aqk]

Adding a docstring --benchmark-repeats should be plenty!

[github-actions]

Conflicts have been resolved. A maintainer will review the pull request shortly.

[github-actions]

File	Coverage	Missing Lines
tests/conftest.py	70.6%	lines 107, 388, 392-393, 401
tests/core/full_node/test_performance.py	0.0%	lines 118, 170, 175
tests/core/test_cost_calculation.py	0.0%	lines 306
tests/process_benchmarks.py	0.0%	lines 1, 3-10, 12-13, 16-24, 26-27, 39-40, 43-44, 46-47, 49-50, 52-53, 55, 58-59, 66, 72, 78, 86, 91, 98, 103, 112-114, 118-121, 123, 127, 129-134, 136-137, 139, 145, 148, 153, 158, 163, 174-178, 180-183, 185-189, 191-194, 196-197, 199, 201-202, 204-206, 208, 211-212, 214, 216, 218, 221-222, 224, 226, 238, 240
tests/util/misc.py	52.2%	lines 68, 176, 246, 262, 286-287, 289, 293-295, 314
tests/wallet/test_offer_parsing_performance.py	0.0%	lines 31, 41

Total	Missing	Coverage
149 lines	125 lines	16%