Timeout tests when they hang.

[Korijn]

Hopefully this will snuff out the test that times out on AppVeyor.

• Added pytest-timeout
• Configured test timeout to 10s
• Added .vscode to gitignore

Continuation of #74

[Korijn]

Looks like the test suite at least completes now, instead of timing out the AppVeyor build... now to figure out which one it is!

[Korijn]

It's test_can_execute_subprocess_primitive and a few others...

[Korijn]

Oddly enough there seem to be race conditions at work:

One passed, two failed, and one is hanging indefinitely, even with pytest-timeout...

[Korijn]

I'm not seeing any failing tests anymore. But AppVeyor still returns with a nonzero exit code. What could be the root cause?

Also, in one of the passing builds, we still see this in the console output:

Exception ignored in: <bound method _ProactorSocketTransport.__del__ of <_ProactorSocketTransport closing fd=-1>>
Traceback (most recent call last):
  File "c:\python34\lib\asyncio\proactor_events.py", line 90, in __del__
    self.close()
  File "c:\python34\lib\asyncio\proactor_events.py", line 78, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "C:\projects\quamash\quamash\__init__.py", line 353, in call_soon
    return self.call_later(0, callback, *args)
  File "C:\projects\quamash\quamash\__init__.py", line 330, in call_later
    return self._add_callback(asyncio.Handle(callback, args, self), delay)
  File "C:\projects\quamash\quamash\__init__.py", line 347, in _add_callback
    self.__timers.append(timer)
AttributeError: 'NoneType' object has no attribute 'append'
Build success

[Korijn]

Perhaps we need to use pytest-xdist's option --boxed...

[Korijn]

I've configured pytest-xdist to box all tests in isolated subprocesses, and I've removed the skip markers. Let's see if that will give us a decent overview of test failures...

Ugh, it's not supported on Windows...

[harvimt]

Yeah I came accross this before, boxed still isn't supported on windows: https://bitbucket.org/pytest-dev/pytest/issues/193/xdist-support-for-boxed-on-windows

…

On Thu, Aug 10, 2017 at 11:41 PM, Korijn van Golen ***@***.*** > wrote: I've configured pytest-xdist to box all tests in isolated subprocesses, and I've removed the skip markers. Let's see if that will give us a decent overview of test failures... — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#75 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAHRQufLLcfPSzY6jrJacAbkExLvy_0pks5sW_eggaJpZM4OzBfS> .

[Korijn]

Alright, so, what I've learned in this process is that:

• All tests randomly fail on Windows, so test skip decorators don't make any sense. Therefore I've removed the decorator I added in a previous PR.
• Boxing tests to subprocesses with pytest-xdist is not supported in Windows, so that doesn't help either.
• I added pytest-timeout to make sure AppVeyor doesn't hang indefinitely.

This PR won't help getting AppVeyor builds to pass. Someone really needs to dig into the internals of the library and the tests to figure out what is going on.

I've updated #73 to reflect the findings here.

This PR can be merged as is.

[Korijn]

@harvimt Can this be merged?