post_run: add report data to 'Results' object/class

[umarcor]

Close #445
Close #478
Close #574

In this PR, parameters output_path and report are passed from main to the Results object built and passed to post_run callbacks. Instead of exposing the TestReport object, as suggested in #445, a subset of info is extracted for each test: name, status, time and path. The path is relative to join(output_path, 'test_output').

Usage example:

def post_func(results):
    print(results.output_path)
    for key, val in results.get_report().items():
        print(key , val)

vu.main(post_run=post_func)

@felixn, you can build the path to each test log as follows: join(results.output_path, 'test_output', val.path, 'output.txt'). Alternatively: join(results.output_path, 'test_output', results.get_report().items()[<test_name>], 'output.txt').

/cc @richjyoung

[kraigher]

Add a method to_dict to test instead of using private access

[umarcor]

I added it now. However, private access is required for _test_results_in_order(). Should I add to_dict to TestReport too?

[kraigher]

Also there are no new tests added for this.

[umarcor]

Also there are no new tests added for this.

I didn't find tests in vunit/test/unit for merge_coverage. Should I create a new vunit/test/unit/test_results.py file? Or can I add them to vunit/test/unit/test_test_report.py?

EDIT

I now added a test named test_dict_report_with_all_passed_tests to test_test_report.py. Let me know what you think.

[kraigher]

I think we should not expose dicts for individual test results on the public api since it is harder to change without breaking in the future. Also see my other line comments.

[umarcor]

I think we should not expose dicts for individual test results on the public api since it is harder to change without breaking in the future.

I added and used class ReportResult now. Let me know if that's what you meant.

Also see my other line comments.

I don't see any other comments. Maybe you have some unfinished review?