unittest tests stop working or are not found by the extension over time

[hanpari]

Issue Type: Bug

The "python run test" works fine from the start. Then it seems extension loses location to tests as it is not able to find them. Only way to amend is to restart vscode. It helps just for a while, then the issue repeats.

Test framework is unittest.

I have tried to use completely new installation with the same result.
I experienced this with Python 3.6 and 3.7

This is from Develop console

12console.ts:136 [Extension Host] Python Extension: Error: read ECONNRESET
t.log @ console.ts:136
t._logExtensionHostMessage @ extensionHost.ts:393
(anonymous) @ extensionHost.ts:210
emitTwo @ events.js:106
emit @ events.js:194
process.nextTick @ internal/child_process.js:766
_combinedTickCallback @ internal/process/next_tick.js:73
_tickCallback @ internal/process/next_tick.js:104

Extension version: 2018.7.1
VS Code version: Code 1.25.1 (1dfc5e557209371715f655691b1235b6b26a06be, 2018-07-11T15:43:53.668Z)
OS version: Windows_NT x64 10.0.17134

System Info

Item	Value
CPUs	Intel(R) Pentium(R) CPU 2020M @ 2.40GHz (2 x 2395)
GPU Status	2d_canvas: unavailable_software flash_3d: unavailable_software flash_stage3d: unavailable_software flash_stage3d_baseline: unavailable_software gpu_compositing: unavailable_software multiple_raster_threads: disabled_off native_gpu_memory_buffers: disabled_software rasterization: unavailable_software video_decode: unavailable_software video_encode: unavailable_software vpx_decode: unavailable_software webgl: unavailable_off webgl2: unavailable_off
Memory (System)	7.88GB (3.72GB free)
Process Argv	C:\Program Files\Microsoft VS Code\Code.exe
Screen Reader	no
VM	0%

[d3r3kk]

Thanks for the report.

However, to better to diagnose the issue a bit more information would be most helpful.

What steps did you take to get into this state?

What code were you working in? (attaching a code sample, or a link to a Github repo can help here).

What other modules are being used? Version information of these as well.

[hanpari]

Ok,

steps to reproduce are quite easy, at least in my case. Just run Run all unit tests. On status bar there is number of passed tests. In my case it is 6 after start. Aftre rerun of command, it is 12, after rerun it is 18, etc.
If I intentionally put an not passing test, the result is 5+1 tests, after rerun 10+2, etc. When it is losing tests, it is hard to say. It varies. In one case it was after 8 rerun, in another 12 or more, so it seems it is rather related to passed time than nuber of run tests.

Furthermore, in developer console I saw:

INFO standard startup Object
/C:/Program Files/Microsoft VS Code/resources/app/out/vs/workbench/workbench.main.js:270 [Extension Host] Python Extension: Failed to get conda environment list from conda Error: spawn conda ENOENT
at exports._errnoException (util.js:1050:11)
at Process.ChildProcess._handle.onexit (internal/child_process.js:193:32)
at onErrorNT (internal/child_process.js:367:16)
at _combinedTickCallback (internal/process/next_tick.js:80:11)
at process._tickCallback (internal/process/next_tick.js:104:9)
9/C:/Program Files/Microsoft VS Code/resources/app/out/vs/workbench/workbench.main.js:270 [Extension Host] Python Extension: Error: read ECONNRESET
t.log @ /C:/Program Files/Microsoft VS Code/resources/app/out/vs/workbench/workbench.main.js:270

But I have unistalled anaconda recently. It almost seems that VSCode believes it is presented. Nevertheless, I tried to install Anacoda with no better result.

The modules
I am using only modules from standard library + mypy, yapf and ipython for developing.

The repo code is here:
https://github.com/hanpari/properus

But I dont believe it is related to code as I remember I saw something similar before.

[hanpari]

I tried to repeat on completely different machine with Windows 10 and Python 3.6.6 and I can confirm exactly same behavior for simple dummy test file, not related to previous one.

See pictures:

1. run all test - see exactly one test run which is correct
   

2. run - value has doubled, which is not correct as the test is only one.
   

3. run
   

4. run not performed as tests are lost
   

Hope it helps.
If you need more info, let me know.

[d3r3kk]

Thanks so much! That clears things up a lot.

I've been working on the test counter in the status bar recently, and I believe I have it solved. See issue #2143.

This is likely a related problem caused by multiple registration of event listeners. I've rectified the issue locally and will be putting forth a PR shortly to remedy it.

[d3r3kk]

But since your issue is slightly off the topic of my fix (tests stop being reported, are they even being run?) I want to keep this one open until we show it is fixed as well.

[hanpari]

If tests are found, then tests are run. Just they numbers doubles every time. I didnt encounter any incorrect results in my tests when compared to tests through command line.

Thanks for your hardwork.

[hanpari]

Just one note
I went through #2143

and I believe it is the same as mine. IMHO, this issue can be closed.

[d3r3kk]

Just being cautious! I'll be sure to add unit tests to ensure that we can run many iterations of the unittests and not lose any along the way. Seems to be a pretty decent test to have anyways 😃

[brettcannon]

Duplicate of #2143