Get rid of time.sleep() usage in the test suite

[giampaolo]

From [email protected] on February 23, 2009 15:01:03

Almost all test cases implemented so far use a time.sleep(0.1) call after
subprocess.Popen() as a provisional workaround to let the sub process
initialize properly.
This should be avoided by using some kind of signal that let us know when
the process is properly initialized.

Original issue: http://code.google.com/p/psutil/issues/detail?id=19

[giampaolo]

From [email protected] on February 26, 2009 14:13:04

In r184 I added a convenience "wait_for_pid(pid)" function replacing all 
time.sleep() calls.
Called right after subprocess creation, it waits for pid to show up in the 
list then return.


As consequence of this change test suite runs about 4x faster and works fine on 
but it causes different test failures on Windows where process information are 
with "null" values (e.g. name = "unknown", cmdarg = [], and so on...).

Perhaps this is a symptom of a problem with the Windows C code extension for 
suggest to open a separate issue.

Status: Started

[giampaolo]

From [email protected] on February 27, 2009 05:04:33

Fixed this in r185 - it's a race condition where the process shows up in the 
(get_process_info() already checks for the PID in the pid list anyway) but
EnumProcessModules() fails with ERROR_PARTIAL_COPY, so we can't get the process 
get_arg_list() fails to get the command line, but by the time get_ppid() is 
read the ppid, it works. 

To avoid this I'm now looping in get_process_info() instead; if 
fails with ERROR_PARTIAL_COPY then we try again until we either get a different 
or it succeeds.

However, I'm now getting errors on test_kill() and test_zombie_process() in the 
suite on this revision:

======================================================================
ERROR: test_kill (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test\test_psutil.py", line 65, in test_kill
    self.assertFalse(psutil.pid_exists(self.proc.pid) and
psutil.Process(self.proc.pid).name == PYTHON)
  File "C:\Python25\Lib\site-packages\psutil\psutil.py", line 129, in name
    self.deproxy()
  File "C:\Python25\Lib\site-packages\psutil\psutil.py", line 104, in deproxy
    self._procinfo = _platform_impl.get_process_info(self._procinfo.pid)
  File "C:\Python25\Lib\site-packages\psutil\_psmswindows.py", line 24, in 
    return callable(*args, **kwargs)
  File "C:\Python25\Lib\site-packages\psutil\_psmswindows.py", line 41, in
get_process_info
    infoTuple = _psutil_mswindows.get_process_info(pid)
WindowsError: [Error 5] Access is denied

======================================================================
ERROR: test_kill (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test\test_psutil.py", line 45, in tearDown
    psutil.Process(self.proc.pid).kill()
  File "C:\Python25\Lib\site-packages\psutil\psutil.py", line 168, in kill
    _platform_impl.kill_process(self.pid, sig)
  File "C:\Python25\Lib\site-packages\psutil\_psmswindows.py", line 24, in 
    return callable(*args, **kwargs)
  File "C:\Python25\Lib\site-packages\psutil\_psmswindows.py", line 50, in 
    return _psutil_mswindows.kill_process(pid)
WindowsError: [Error 5] Access is denied

======================================================================
ERROR: test_zombie_process (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test\test_psutil.py", line 203, in test_zombie_process
    self.assertRaises(psutil.NoSuchProcess, getattr, p, "ppid")
  File "C:\Python25\lib\unittest.py", line 320, in failUnlessRaises
    callableObj(*args, **kwargs)
  File "C:\Python25\Lib\site-packages\psutil\psutil.py", line 115, in ppid
    self.deproxy()
  File "C:\Python25\Lib\site-packages\psutil\psutil.py", line 104, in deproxy
    self._procinfo = _platform_impl.get_process_info(self._procinfo.pid)
  File "C:\Python25\Lib\site-packages\psutil\_psmswindows.py", line 24, in 
    return callable(*args, **kwargs)
  File "C:\Python25\Lib\site-packages\psutil\_psmswindows.py", line 41, in
get_process_info
    infoTuple = _psutil_mswindows.get_process_info(pid)
WindowsError: [Error 5] Access is denied

----------------------------------------------------------------------

[giampaolo]

From [email protected] on February 27, 2009 05:10:43

Note: the Access Denied exceptions above go away if I replace wait_for_pid() in 
test_kill() function with "time.sleep(0.1)" and if I add a "time.sleep(0.1)" in
test_zombie_process()

Evidently we're denied access to OpenProcess() until the process is fully
initialized. This could be normal behavior for any process that has not 
initializing, or it might be related to the subprocess module specific 
since that's what we're using to launch the process. 

kill() on Windows is already testing to see if the PID exists in the process 
before trying to call TerminateProcess() on the PID, so wait_for_pid() won't 
difference here unless it just happens to slow everything down enough for the 
condition to pass ;)

[giampaolo]

From [email protected] on February 27, 2009 06:35:01

I don't get any failure if I run the test suite as Administrator.
Only test_fetch_all fails if I run it as limited user.
IMHO time.sleep() shouldn't be used as workaround to avoid this problem anyway.

[giampaolo]

From [email protected] on February 27, 2009 06:47:07

I'm not running it as a limited user, I'm running it as Administrator user 
SP2). And the fact that it works with time.sleep() in there shows it's still a 
condition. I'm not suggesting we use it as a workaround, I'm just pointing out 
problem is there and it's caused by timing issues is all.

I can't very well loop in the kill() function waiting for access denied not to 
thrown, or we'll have an infinite loop if you try to kill a process that's
legitimately access denied. Either we need a way for subprocess to tell us when 
process is ready, we need to sleep for a while and try again, or we need to 
out a way to tell the difference between process still initializing and a real 
denied.

[giampaolo]

From [email protected] on February 27, 2009 08:15:20

After r187 the failing tests should now raise psutil.AccessDenied instead of
WindowsError: [Error 5] Access is denied.

[giampaolo]

From [email protected] on February 27, 2009 08:19:48

test_fetch_all is the only one failing for me on Windows XP SP3 due to
psutil.AccessDenied exception raised when run as user.
Can you tell me which tests are failing on Windows 2003 now?

[giampaolo]

From [email protected] on February 27, 2009 09:32:07

======================================================================
ERROR: test_zombie_process (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test\test_psutil.py", line 186, in test_zombie_process
    self.assertRaises(psutil.NoSuchProcess, getattr, p, "ppid")
  File "C:\Python25\lib\unittest.py", line 320, in failUnlessRaises
    callableObj(*args, **kwargs)
  File "C:\Python25\Lib\site-packages\psutil\psutil.py", line 107, in ppid
    self.deproxy()
  File "C:\Python25\Lib\site-packages\psutil\psutil.py", line 96, in deproxy
    self._procinfo = _platform_impl.get_process_info(self._procinfo.pid)
  File "C:\Python25\Lib\site-packages\psutil\_psmswindows.py", line 24, in 
    raise psutil.AccessDenied
AccessDenied

----------------------------------------------------------------------

[giampaolo]

From [email protected] on February 27, 2009 17:11:37

Fixed in r191 - get_process_info() was raising AccessDenied from a failure of
EnumProcessModules which was getting an access error for a dead process. 
returns a valid handle even for a dead process so we can't rely on it. Instead 
checking GetExitCodeProcess() now if we get an access denied error. If the exit 
indicates the process is closed, we raise a NoSuchProcess. Otherwise we raise 
AccessDenied error back up.

Status: Verified

[giampaolo]

From [email protected] on March 17, 2009 08:31:20

Status: Fixed

[giampaolo]

From g.rodola on March 02, 2013 03:42:43

Updated csets after the SVN -> Mercurial migration: r184 == revision 
3f81590eeea7 r185 == revision eb220774de36 r187 == revision 4a5cd6fb93bc r191