Skip to content

Improve test report formatting#5018

Merged
boegel merged 5 commits intoeasybuilders:developfrom
Flamefire:20251007122536_new_pr_XPJPAxSjWb
Dec 7, 2025
Merged

Improve test report formatting#5018
boegel merged 5 commits intoeasybuilders:developfrom
Flamefire:20251007122536_new_pr_XPJPAxSjWb

Conversation

@Flamefire
Copy link
Copy Markdown
Contributor

@Flamefire Flamefire commented Oct 7, 2025

(created using eb --new-pr)

When an exception aborted the build the error reporting in the test report is badly formatted, especially as the traceback is printed as normal text which breaks markdown formatting

This fixes that by putting the traceback into code tags and adjusting the list items:

  • FAIL cuDNN-9.5.1.17-CUDA-12.6.0.eb (unhandled exception: name 'doesnt_exist' is not defined)
traceback

Prior: https://gist.github.com/Flamefire/a1e367edcdf25758c5fbc4be39dc710d
After: https://gist.github.com/Flamefire/24526ae1becbae76e1b31b9347d48ec7

Related: Should we improve the stdout for test report runs? Currently it is totally missing the cause for the failure:

== processing EasyBuild easyconfig /tmp/easybuild-tmp/eb-8uru6x75/files_pr23923/c/cuDNN/cuDNN-9.5.1.17-CUDA-12.6.0.eb
== building and installing Core/cuDNN/9.5.1.17-CUDA-12.6.0...
== fetching files and verifying checksums...
[snip]
== restore after iterating...
== postprocessing...
== sanity checking...
== ... (took 8 secs)
== processing EasyBuild easyconfig /tmp/easybuild-tmp/eb-8uru6x75/files_pr23923/c/cuSPARSELt/cuSPARSELt-0.6.3.2-CUDA-12.6.0.eb
== building and installing Core/cuSPARSELt/0.6.3.2-CUDA-12.6.0...
== fetching files and verifying checksums...
[snip]
== sanity checking...
== ... (took 5 secs)


Adding comment [snip]
**FAILED**
Build succeeded for 0 out of 7 (7 easyconfigs in total)
c144 - [snip machine...]
See https://gist.github.com/[...] for a full test report.'
== Test report uploaded [...]
== Build succeeded for 0 out of 7
== Summary:
   * [FAILED]  Core/cuDNN/9.5.1.17-CUDA-12.6.0
   * [FAILED]  Core/cuSPARSELt/0.6.3.2-CUDA-12.6.0
   * [FAILED]  Compiler/GCCcore/13.3.0/NCCL/2.26.2-CUDA-12.6.0
   * [FAILED]  Compiler/GCC/13.3.0/nvidia-cutlass/3.8.0.0-CUDA-12.6.0
   * [FAILED]  Compiler/GCCcore/13.3.0/pyzstd/0.16.2
   * [FAILED]  MPI/GCC/13.3.0/OpenMPI/5.0.3/Triton/3.3.1-CUDA-12.6.0
   * [FAILED]  MPI/GCC/13.3.0/OpenMPI/5.0.3/PyTorch/2.7.1-CUDA-12.6.0

@boegel boegel added this to the next release (5.2.0?) milestone Oct 8, 2025
@boegel boegel added the change label Oct 8, 2025
Copy link
Copy Markdown
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@boegel boegel removed the change label Dec 7, 2025
@boegel boegel merged commit 633cc4b into easybuilders:develop Dec 7, 2025
40 checks passed
@Flamefire Flamefire deleted the 20251007122536_new_pr_XPJPAxSjWb branch December 8, 2025 10:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants