Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --detailed option for buildtest report #1567

Merged
merged 42 commits into from
Aug 1, 2023
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
afcdce3
Add --detailed and -d options
szuananwar Jul 19, 2023
06e3ad7
Add --detailed and -d to local options under report/rt command
szuananwar Jul 19, 2023
73402c4
Merge branch 'buildtesters:devel' into add_detailed_option
szuananwar Jul 19, 2023
2c45a21
Added "Detailed Report (buildtest report --detailed" paragraph to que…
szuananwar Jul 19, 2023
25080ce
Add implementation for --detailed option
szuananwar Jul 19, 2023
b127342
Add regression test for --detailed option
szuananwar Jul 19, 2023
c55e844
Update the report_detailed method
szuananwar Jul 19, 2023
03bb642
Update the test_report_detailed method
szuananwar Jul 19, 2023
0e6d2ff
Fix style issue
szuananwar Jul 19, 2023
bf9002c
Fix style issue
szuananwar Jul 19, 2023
5c981f0
Add --detailed under optional arguments.
szuananwar Jul 19, 2023
de99b8f
Add create format_group and filter_group
szuananwar Jul 19, 2023
ba21963
Group --detailed and --format as mutually exclusive
szuananwar Jul 20, 2023
15ce5ab
Updated the doc for --detailed option
szuananwar Jul 20, 2023
0e0cbd0
Add implementation of --detailed option
szuananwar Jul 20, 2023
094252b
Add test regression of --detailed option
szuananwar Jul 20, 2023
922590d
Add ['py311'] version to the list of versions under black tool
szuananwar Jul 20, 2023
0847e10
Add regression test
szuananwar Jul 24, 2023
95c9a20
Deleting some comments lines
szuananwar Jul 24, 2023
62337ae
Documenting format_detailed variable in Report class docstring
szuananwar Jul 24, 2023
0228eab
fix style check
szuananwar Jul 24, 2023
88f263a
Work on source code reading and viewing
szuananwar Jul 24, 2023
06645ed
Add note using a sphinx note.
szuananwar Jul 24, 2023
71b85c0
Fix docstring for --detailed.
szuananwar Jul 25, 2023
294b8c5
Add note about --detailed and --format options when user use them as …
szuananwar Jul 25, 2023
7dafa0f
Raise an exception when invoke
szuananwar Jul 25, 2023
05aaea1
Fix indentation and double ticks
szuananwar Jul 28, 2023
bb018c3
Raise an exception if both format and detailed options are specified …
szuananwar Jul 28, 2023
7f7a030
Add regression test for buildtest rt -d/--detailed
szuananwar Jul 28, 2023
62467b3
Add regression test for buildtest rt -d/--detailed --format "name"
szuananwar Jul 31, 2023
7ac315e
Fix doc rendering issue
szuananwar Jul 31, 2023
8cab76f
Fix doc rendering issue
szuananwar Jul 31, 2023
ac93439
Create a class variable format_fields_detailed
szuananwar Jul 31, 2023
91dbb51
raise an exception to capture specifying both --detailed and --format
szuananwar Jul 31, 2023
931e000
Fix style check error
szuananwar Jul 31, 2023
619b9ce
Fix style check error
szuananwar Jul 31, 2023
5ad52a1
Fix style check error by removing ()
szuananwar Jul 31, 2023
f00a896
Remove ['py311'] under tool.black
szuananwar Jul 31, 2023
3cedce2
Add comments to test --detailed --format in test_report_detailed method
szuananwar Jul 31, 2023
012f879
Add 'py311' under tool.black
szuananwar Jul 31, 2023
28bf2c1
Add detailed args to report path method
szuananwar Aug 1, 2023
eedb821
fix issue with documentation
shahzebsiddiqui Aug 1, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion bash_completion.sh
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,7 @@ _buildtest ()
;;

report|rt)
local opts="--end --fail --filter --filterfields --format --formatfields --help --helpfilter --helpformat --latest --no-header --oldest --pager --pass --row-count --start --terse -e -f -h -n -p -s -t c clear l list p path sm summary"
local opts="--detailed --end --fail --filter --filterfields --format --formatfields --help --helpfilter --helpformat --latest --no-header --oldest --pager --pass --row-count --start --terse -d -e -f -h -n -p -s -t c clear l list p path sm summary"
COMPREPLY=( $( compgen -W "${opts}" -- $cur ) )
case "${prev}" in --filter)
COMPREPLY=( $( compgen -W "$(_avail_report_filterfields)" -- $cur ) )
Expand Down
34 changes: 25 additions & 9 deletions buildtest/cli/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -604,6 +604,7 @@ def build_menu(subparsers):
action="store_true",
help="Rerun last successful buildtest build command.",
)

filter_group.add_argument(
"-f",
"--filter",
Expand Down Expand Up @@ -1100,7 +1101,7 @@ def report_menu(subparsers, parent_parser):
help="Summarize test report",
parents=[parent_parser["pager"]],
)
filter_group = parser_report.add_argument_group("filter", "Filter and Format table")
filter_group = parser_report.add_argument_group("filter", "Filter options")

# buildtest report
filter_group.add_argument(
Expand All @@ -1109,28 +1110,43 @@ def report_menu(subparsers, parent_parser):
help="Filter report by filter fields. The filter fields must be a key=value pair and multiple fields can be comma separated in the following format: --filter key1=val1,key2=val2 . For list of filter fields run: --helpfilter.",
)

filter_group.add_argument(
"--format",
help="format field for printing purposes. For more details see --helpformat for list of available fields. Fields must be separated by comma (usage: --format <field1>,<field2>,...)",
)
filter_group.add_argument(
"--helpfilter",
action="store_true",
help="List available filter fields to be used with --filter option",
)
filter_group.add_argument(
"--helpformat", action="store_true", help="List of available format fields"
)

filter_group.add_argument(
"--filterfields",
action="store_true",
help="Print raw filter fields for --filter option to filter the report",
)
filter_group.add_argument(

format_group = parser_report.add_argument_group("format", "Format options")

format_group.add_argument(
"--helpformat", action="store_true", help="List of available format fields"
)

format_group.add_argument(
"--formatfields",
action="store_true",
help="Print raw format fields for --format option to format the report",
)
shahzebsiddiqui marked this conversation as resolved.
Show resolved Hide resolved

format_detailed_group = parser_report.add_mutually_exclusive_group()
format_detailed_group.add_argument(
"--format",
help="format field for printing purposes. For more details see --helpformat for list of available fields. Fields must be separated by comma (usage: --format <field1>,<field2>,...)",
)

format_detailed_group.add_argument(
"-d",
"--detailed",
help="Print a detailed summary of the test results",
action="store_true",
)

shahzebsiddiqui marked this conversation as resolved.
Show resolved Hide resolved
pass_fail = parser_report.add_mutually_exclusive_group()

pass_fail.add_argument(
Expand Down
15 changes: 15 additions & 0 deletions buildtest/cli/report.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import argparse
import datetime
import logging
import os
Expand Down Expand Up @@ -91,6 +92,7 @@ def __init__(
oldest=None,
count=None,
pager=None,
format_detailed=None,
shahzebsiddiqui marked this conversation as resolved.
Show resolved Hide resolved
detailed=None,
color=None,
):
Expand All @@ -108,6 +110,7 @@ def __init__(
oldest (bool, optional): Fetch oldest run for all tests discovered. This is specified via ``buildtest report --oldest``
count (int, optional): Fetch limited number of rows get printed for all tests discovered. This is specified via ``buildtest report --count``
pager (bool, optional): Enabling PAGING output for ``buildtest report``. This can be specified via ``buildtest report --pager``
format_detailed(bool, optional): Print a detailed summary of the test results. This can be specified via ``buildtest report --detailed``
color (str, optional): An instance of a string class that tells print_report what color the output should be printed in.

"""
Expand All @@ -126,6 +129,17 @@ def __init__(
self.color = color
self.input_report = report_file

try:
report=Report(configuration=configuration, format_detailed=True, format_args="name,id,user")
except argparse.ArgumentError:
console.print_exception()
return True
shahzebsiddiqui marked this conversation as resolved.
Show resolved Hide resolved

if format_detailed:
self.format = (
"name,id,user,state,returncode,runtime,outfile,errfile,buildspec"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Create a class variable format_fields_detailed and declare this. Then assign self.format to this variable

)

# if no report specified use default report
if not self.input_report:
self._reportfile = BUILD_REPORT
Expand Down Expand Up @@ -828,6 +842,7 @@ def report_cmd(args, configuration, report_file=None):
oldest=args.oldest,
report_file=report_file,
count=args.count,
format_detailed=args.detailed,
)

if args.report_subcommand in ["path", "p"]:
Expand Down
20 changes: 20 additions & 0 deletions docs/gettingstarted/query_test_report.rst
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,26 @@ You may find it useful to fetch all failed records and determine pass/fail crite

.. _report_format_fields:

Detailed Reports (``buildtest report --detailed``)
---------------------------------------------------

The **buildtest report --detailed** option can be used to show a detailed test report that may be of interest when
examining a test. This option is synonymous to running ``buildtest report --format name,id,user,state,returncode,runtime,outfile,errfile,buildspec``.
Shown below is an example output.

.. dropdown:: ``buildtest report --detailed``
.. command-output:: buildtest report --detailed
shahzebsiddiqui marked this conversation as resolved.
Show resolved Hide resolved

.. note::
The --detailed and --format options are mutually exclusive options because both options will alter the format
shahzebsiddiqui marked this conversation as resolved.
Show resolved Hide resolved
fields when displaying test results.

You will get the following error if you specify both options as shown below
.. dropdown::``buildtest report --detailed --format name``
shahzebsiddiqui marked this conversation as resolved.
Show resolved Hide resolved

.. command-output:: buildtest report --detailed --format name
:returncode: 1

shahzebsiddiqui marked this conversation as resolved.
Show resolved Hide resolved
Format Reports (``buildtest report --format``)
-----------------------------------------------

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ docs = [
]
[tool.black]
required-version = '23.3.0'
target-version = ['py38', 'py39', 'py310']
target-version = ['py38', 'py39', 'py310', 'py311']
skip_magic_trailing_comma = true
line-length = 88
verbose = true
Expand Down
3 changes: 3 additions & 0 deletions tests/cli/test_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,9 @@ def test_report_format():
# buildtest report --formatfields
report.print_raw_format_fields()

# buildtest report --detailed
report = Report(configuration=configuration, format_detailed=True)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a good test. I think we should update the __init__ method of Report class to also raise an exception if we invoke the following

Report(configuration=configuration,format_detailed=True,format="name,id")

Right now we have the exception raised from argparse but there is nothing stop it from happening when invoking the class method.

I think you should have this as a separate method call it test_report_detailed where you this negative assertion and also move the code

# buildtest report --detailed
report = Report(configuration=configuration, format_detailed=True)

Copy link
Collaborator Author

@szuananwar szuananwar Jul 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shahzebsiddiqui This is not clear to me, I am trying to fix it since yesterday, I used this code:
try:
report=Report(configuration=configuration, format_detailed=True, format_args="name,id,user")
except argparse.ArgumentError:
console.print_exception()
return True
The result when I run buildtest report --detailed --format name,id:
(anwar)anwar@DOE6991925 buildtest % buildtest report --detailed --format name,id
usage: buildtest [options] [COMMANDS] report [-h] [--pager] [--row-count] [-t]
[-n] [-c COUNT] [--filter FILTER]
[--helpfilter] [--filterfields]
[--helpformat] [--formatfields]
[--format FORMAT | -d] [-f | -p]
[-s START] [-e END] [--latest]
[--oldest]
...
buildtest [options] [COMMANDS] report: error: argument --format: not allowed with argument -d/--detailed
But when I run buildtest report --detailed
132 │ │ try:
│ ❱ 133 │ │ │ report=Report(configuration=configuration, format_detailed │
RecursionError: maximum recursion depth exceeded
Any advise?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix your message above it's not readable.

You need to format your message properly

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand what you are doing updating the Report class in init was meant to catch exception as part of regression test but your output above is sharing an invocation to buildtest. Either I am not clear in what I am asking or you have trouble conveying your message. Note you don't need to share the code you want to try just commit it so I can see the changes.

It's better you just share the screenshot for error and please delete your message.

# buildtest report --format XYZ is invalid format field
with pytest.raises(BuildTestError):
Report(configuration, format_args="XYZ")
Expand Down
Loading