A memory leaks test script is needed

[giampaolo]

From [email protected] on March 12, 2009 13:25:51

As far as I know Jay should have wrote one.
Before adding further C code, especially on Windows, a new test script
should be added in the current code base to allow us testing that any
change applied to C extensions does not introduce memory leaks.

Original issue: http://code.google.com/p/psutil/issues/detail?id=37

[giampaolo]

From [email protected] on March 12, 2009 05:26:48

Owner: jloden

[giampaolo]

From [email protected] on March 12, 2009 09:34:43

Actually for memory leak testing I used process_listing.py: 


import psutil
import sys
import os

h = open("process_listing.pid", 'w')
h.write(str(os.getpid()))
h.close()

try: 
    run_count = 0
    while (run_count < 20000):
        run_count = run_count + 1
        sys.stdout.write("\r\r%s" % run_count)
        sys.stdout.flush()

        try:
            cmdline = psutil.Process(0).cmdline
        except Exception, e:
            print "\n"
            print "Exception for PID 0: %s" % e
            continue

        PID = -1
        for i in psutil.get_process_list():
            PID = i.pid
            try:
                cmdline = i.cmdline
            except psutil.NoSuchProcess:
                continue
            except Exception, e:
                print "\n"
                print "Exception for PID %s: %s" % (PID, e)
                continue

except:
    os.remove("process_listing.pid")


I used that to run a loop iterating over as much of the code as possible in the
background by calling str() on a Process object, then used top or ps on UNIX
platforms, or taskmgr on Windows, to monitor the process memory usage. Some shifting
of memory usage is normal, as long as it doesn't climb steadily and remains somewhere
around a consistent amount it means there are most likely no memory leaks. 

Ideally psutil will support memory statistics in the next release and we can use
psutil itself to monitor for memory leaks in a cross-platform way without manually
checking the memory usage. Then we can create a self-contained script called
something like test/memory_usage.py that runs a bunch of iterations (I'd guess that a
few thousand is enough to verify a leak) and looks for memory usage climbing beyond a
pre-set threshold.

Status: Accepted

Attachment: process_listing.py

[giampaolo]

From [email protected] on March 12, 2009 09:38:10

Putting this as blocked on Issue #38 (per-process memory statistics support) for now. https://code.google.com/p/psutil/issues/detail?id=38

Blockedon: 38

[giampaolo]

From [email protected] on March 25, 2009 05:18:49

Added as r308 .
For me it works fine on every platform except Windows where test__str__ and
test_get_pid_list fail (not always but quite often).

Status: Started

[giampaolo]

From [email protected] on March 25, 2009 06:05:45

Windows wasn't the only platform on which such test was failing since Jay told me it
was failing also on OS X every now and then.
We noticed that every time there was a difference in the memory usage it was always
of 4096 bytes, no matter how many times the loop was repeated. r315 modifies the script so that if we don't go over 4096 bytes of difference between
step 2 and step 1 the test is considered valid.

As of now I'm not aware of any failure on any platform so I guess we can consider
this one fixed.

Status: Fixed

[giampaolo]

From g.rodola on March 02, 2013 03:47:16

Updated csets after the SVN -> Mercurial migration: r308 == revision 126d2d3a5674 r315 == revision dcd82bc6d9ff