Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test printing refinement #18739

Closed
tkelman opened this issue Sep 30, 2016 · 6 comments
Closed

Test printing refinement #18739

tkelman opened this issue Sep 30, 2016 · 6 comments
Assignees
Labels
testsystem The unit testing framework and Test stdlib

Comments

@tkelman
Copy link
Contributor

tkelman commented Sep 30, 2016

followup to #17165 / #18738, cc @kshyatt

right now the base tests are printing both old as-you-go info, in addition to a new table so it looks like

     * dsp                   maxrss 3501.18 MB
     * examples              maxrss 3501.18 MB
     * compile               maxrss 3501.18 MB

Test Summary: |     Pass  Broken     Total
  Overall     | 22171317      28  22171345
Test:                 | Total time (s): | GC time (s): | Percent in gc: | Allocated (MB): | RSS (MB):
linalg/triangular     |  546.77         |   11.92      |    2.18        | 8420.77         |  680.38
linalg/qr             |   89.66         |    1.85      |    2.06        | 2312.61         |  751.55

I propose we get rid of the old style and attempt to make the table just print new rows as they come in, if that's possible. The maxrss numbers are the same so it's duplicated information.

Additional printing-related followup from #17165 (comment), printing of @test true or @test 1==1 should be adjusted to not bother showing Expression: nothing now that passing tests don't save the input expressions any more.

@tkelman tkelman added the testsystem The unit testing framework and Test stdlib label Sep 30, 2016
@tkelman
Copy link
Contributor Author

tkelman commented Sep 30, 2016

The table's also overly wide for default console widths so we should try to squeeze it down, maybe make the header wrapped to 2 rows with each title narrower, or drop the percent column?

#17165 (comment) name_align is too short if you only run short-named tests like env or int or pkg.

@kshyatt
Copy link
Contributor

kshyatt commented Sep 30, 2016

It should be possible to print row-by-row as each @testset returns.

The widest columns are Allocated, Percent in gc, and the test names (depending on what they are). One obvious fix is Percent -> %. Allocated -> Alloc.?

@tkelman
Copy link
Contributor Author

tkelman commented Sep 30, 2016

Yeah good abbreviations. Which worker the tests ran on when they execute in parallel is useful info to show, we may want to make room for it in the table.

@kshyatt
Copy link
Contributor

kshyatt commented Sep 30, 2016

test_name (worker) ?

@tkelman
Copy link
Contributor Author

tkelman commented Nov 7, 2016

This remains incomplete. @kshyatt any intention of revisiting this? Cases like https://travis-ci.org/JuliaLang/julia/jobs/173755550 give no indication of what line number within the spawn test actually failed.

@kshyatt
Copy link
Contributor

kshyatt commented Nov 11, 2016

@tkelman sorry I have been totally swamped with work. I will see if I can take a look today or tomorrow.

@kshyatt kshyatt self-assigned this Dec 7, 2016
kshyatt added a commit that referenced this issue Dec 7, 2016
Now results come in one by one and print their memory/timing data
as they come in. We will no longer print rss results twice. Also,
each test now prints which worker it ran on.
kshyatt added a commit that referenced this issue Dec 9, 2016
Now results come in one by one and print their memory/timing data
as they come in. We will no longer print rss results twice. Also,
each test now prints which worker it ran on.
kshyatt added a commit that referenced this issue Dec 11, 2016
Fix #18739, Make testset results printing better
musm pushed a commit to musm/julia that referenced this issue Dec 11, 2016
Now results come in one by one and print their memory/timing data
as they come in. We will no longer print rss results twice. Also,
each test now prints which worker it ran on.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
testsystem The unit testing framework and Test stdlib
Projects
None yet
Development

No branches or pull requests

2 participants