Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Show total progress in terminal during testing #2657

Closed
nicoddemus opened this issue Aug 4, 2017 · 14 comments
Closed

Show total progress in terminal during testing #2657

nicoddemus opened this issue Aug 4, 2017 · 14 comments
Assignees
Labels
topic: reporting related to terminal output and user-facing messages and errors type: enhancement new feature or API change, should be merged into features branch

Comments

@nicoddemus
Copy link
Member

This is a proposal to change the terminal output while running tests to include the total tests ran so far and the total tests still to run, or percentage of the total tests executed so far.

Currently here's the normal pytest output (minus header):

============================= test session starts =============================
collected 1909 items

testing\acceptance_test.py ...............s................................x.............
testing\deprecated_test.py ......
testing\test_argcomplete.py ss
testing\test_assertion.py ........................................................................s.......
<snip>

What I propose is:

============================= test session starts ===============================
collected 1909 items

testing\acceptance_test.py ...............s............................ [44/1909]
                           ....x.............                           [62/1909]
testing\deprecated_test.py ......                                       [68/1909]
testing\test_argcomplete.py ss                                          [70/1909]
testing\test_assertion.py ............................................ [114/1909]
                          ............................s.......         [150/1909] 
<snip>

This would be a better view of the overall progress and how much tests are still left on the run.

Alternatively instead of showing test counts we could just show a percentage of the total run so far:

============================= test session starts ===============================
collected 1909 items

testing\acceptance_test.py ...............s................................ [ 2%]
                           ....x.........                                   [ 3%]
testing\deprecated_test.py ......                                           [ 3%]
testing\test_argcomplete.py ss                                              [ 3%]
testing\test_assertion.py ................................................. [ 5%]
                          ............................s.                    [ 7%] 
<snip>

Thoughts? cc @RonnyPfannschmidt @hpk42 @The-Compiler @hackebrot

Side note

This idea came to be because it Jenkins (it seems) only updates its console view when the process outputs a new line. When running with xdist, pytest only prints a . per test, only printing a new line at the end of the session, so the output windows doesn't update until the end of the run.

Changing the output to the one proposed here would fix that and also provide a nice user experience (IMO).

@nicoddemus nicoddemus added type: proposal proposal for a new feature, often to gather opinions or design the API around the new feature topic: reporting related to terminal output and user-facing messages and errors labels Aug 4, 2017
@nicoddemus
Copy link
Member Author

Note: this would work with xdist outside of the box as well. 😉

@RonnyPfannschmidt
Copy link
Member

doesnt pytest-sugar already implement a much nicer version of this

@nicoddemus
Copy link
Member Author

Indeed, but pytest-sugar only works on Linux. 😬

@The-Compiler
Copy link
Member

I like it! pytest-sugar also has a lot of other problems (it being very fancy is also causing issues), and good defaults matter.

@RonnyPfannschmidt
Copy link
Member

according to Teemu/pytest-sugar#49 (comment) it only a codepage issue and solvable by a single command

@RonnyPfannschmidt
Copy link
Member

well, im fine with enhancing terminal output, but i do wonder how much harder it will make the removal of the current terminalwriter mess

also note that currently the terminal width determination of pylib is completely broken for output capture situation

@nicoddemus
Copy link
Member Author

according to Teemu/pytest-sugar#49 (comment) it only a codepage issue and solvable by a single command

Didn't know that, thanks for the link!

well, im fine with enhancing terminal output, but i do wonder how much harder it will make the removal of the current terminalwriter mess

AFAIK it won't make things worse (it will depend on tw.write as does all the other code which depends on terminalwriter) but I'm of course willing to abort the idea if the implementation gets too messy.

@nicoddemus nicoddemus self-assigned this Aug 11, 2017
@nicoddemus nicoddemus added type: enhancement new feature or API change, should be merged into features branch and removed type: proposal proposal for a new feature, often to gather opinions or design the API around the new feature labels Aug 31, 2017
@ApaDoctor
Copy link
Contributor

does anybody works on this issue?
i want to take it

@RonnyPfannschmidt
Copy link
Member

@nicoddemus do you have any plans/attempts already started, if thats not the case, i propose we let @ApaDoctor try

@nicoddemus
Copy link
Member Author

I have a work in progress at home actually, I don't think it is pushed to me fork though. @ApaDoctor let me push that branch when I get home so you can take a look. This is a little tricky than thought at first because we have to "hook" into terminal's writer to ensure we are tracking the correct number of chars written to the current line so far.

@nicoddemus
Copy link
Member Author

@Teemu
Copy link

Teemu commented Dec 12, 2017

This is pretty cool 👍

I think if this existed 4 years ago I would have never started pytest-sugar project.

@nicoddemus
Copy link
Member Author

Which would be a shame because pytest-sugar terminal is awesome 😁

@kevinpohlmeier
Copy link

When I added a pytest.ini file with the following contents, I started seeing progress on my xdist runs:

[pytest]
log_cli = 1
log_cli_level = WARN

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: reporting related to terminal output and user-facing messages and errors type: enhancement new feature or API change, should be merged into features branch
Projects
None yet
Development

No branches or pull requests

6 participants