Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --detailed option for buildtest report #1567

Merged
merged 42 commits into from
Aug 1, 2023

Conversation

szuananwar
Copy link
Collaborator

This new feature will let the user to run
buildtest rt --format name,id,user,state,returncode,runtime,outfile,errfile,buildspec --count=1
By running
buildtest rt --detailed

@codecov
Copy link

codecov bot commented Jul 19, 2023

Codecov Report

Patch coverage: 62.75% and project coverage change: +0.51% 🎉

Comparison is base (06888a8) 77.86% compared to head (eedb821) 78.36%.
Report is 46 commits behind head on devel.

Additional details and impacted files
@@            Coverage Diff             @@
##            devel    #1567      +/-   ##
==========================================
+ Coverage   77.86%   78.36%   +0.51%     
==========================================
  Files          57       56       -1     
  Lines        6621     6577      -44     
==========================================
- Hits         5155     5154       -1     
+ Misses       1466     1423      -43     
Files Changed Coverage Δ
buildtest/cli/config.py 86.76% <ø> (ø)
buildtest/executors/cobalt.py 21.51% <0.00%> (-0.60%) ⬇️
buildtest/executors/lsf.py 19.28% <0.00%> (-0.72%) ⬇️
buildtest/executors/pbs.py 21.43% <0.00%> (-0.49%) ⬇️
buildtest/executors/slurm.py 20.51% <0.00%> (-0.74%) ⬇️
buildtest/cli/report.py 96.08% <90.91%> (+6.18%) ⬆️
buildtest/builders/base.py 89.46% <100.00%> (+0.13%) ⬆️
buildtest/cli/__init__.py 98.20% <100.00%> (-1.80%) ⬇️
buildtest/executors/local.py 98.00% <100.00%> (+3.77%) ⬆️

... and 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@pull-request-size pull-request-size bot added size/M and removed size/S labels Jul 19, 2023
@szuananwar szuananwar linked an issue Jul 19, 2023 that may be closed by this pull request
7 tasks
@szuananwar szuananwar self-assigned this Jul 19, 2023
@szuananwar
Copy link
Collaborator Author

@shahzebsiddiqui Here is the buildtest report --detailed command result
68C214F5-A899-4EB6-A8CD-FB1F15C06663

@shahzebsiddiqui
Copy link
Member

@szuananwar okay taking a step back and looking at the whole code base i think the way you implemented this method is not going to work with all the other options. We really need to pass detailed as an argument to Report class

Simply running the following command it shows 1 entry which is not correct it should show entries specified by --count or one specified in configuration file

(buildtest)  ~/Documents/github/buildtest/ [add_detailed_option] buildtest rt -d
                                                                                Report File: /Users/siddiq90/Documents/github/buildtest/var/report.json                                                                                 
┏━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ name       ┃ id       ┃ user     ┃ state ┃ returncode ┃ runtime ┃ outfile                                              ┃ errfile                                              ┃ buildspec                                            ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ exit1_fail │ 47c2296e │ siddiq90 │ FAIL  │ 1          │ 0.32758 │ /Users/siddiq90/Documents/github/buildtest/var/tests │ /Users/siddiq90/Documents/github/buildtest/var/tests │ /Users/siddiq90/Documents/github/buildtest/tutorials │
│            │          │          │       │            │         │ /generic.local.bash/pass_returncode/exit1_fail/47c22 │ /generic.local.bash/pass_returncode/exit1_fail/47c22 │ /test_status/pass_returncode.yml                     │
│            │          │          │       │            │         │ 96e/exit1_fail.out                                   │ 96e/exit1_fail.err                                   │                                                      │
└────────────┴──────────┴──────────┴───────┴────────────┴─────────┴──────────────────────────────────────────────────────┴──────────────────────────────────────────────────────┴──────────────────────────────────────────────────────┘

Also we should make --detailed and --format be mutually exclusive options if that makes sense. In example below we should not have this happen because -d will expose pre-selected columns but when --format is specified then we want to specify our own columns

(buildtest)  ~/Documents/github/buildtest/ [add_detailed_option] buildtest rt -d --format name
                                                                                Report File: /Users/siddiq90/Documents/github/buildtest/var/report.json                                                                                 
┏━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ name       ┃ id       ┃ user     ┃ state ┃ returncode ┃ runtime ┃ outfile                                              ┃ errfile                                              ┃ buildspec                                            ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ exit1_fail │ 47c2296e │ siddiq90 │ FAIL  │ 1          │ 0.32758 │ /Users/siddiq90/Documents/github/buildtest/var/tests │ /Users/siddiq90/Documents/github/buildtest/var/tests │ /Users/siddiq90/Documents/github/buildtest/tutorials │
│            │          │          │       │            │         │ /generic.local.bash/pass_returncode/exit1_fail/47c22 │ /generic.local.bash/pass_returncode/exit1_fail/47c22 │ /test_status/pass_returncode.yml                     │
│            │          │          │       │            │         │ 96e/exit1_fail.out                                   │ 96e/exit1_fail.err                                   │                                                      │
└────────────┴──────────┴──────────┴───────┴────────────┴─────────┴──────────────────────────────────────────────────────┴──────────────────────────────────────────────────────┴───────────────────────

@pull-request-size pull-request-size bot added size/L and removed size/M labels Jul 31, 2023
@pull-request-size pull-request-size bot added size/M and removed size/L labels Jul 31, 2023
@shahzebsiddiqui shahzebsiddiqui merged commit 6ee4a47 into buildtesters:devel Aug 1, 2023
19 of 31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEATURE]: buildtest report --detailed
2 participants