cpu_percent() examining only one cpu(?)

[giampaolo]

From [email protected] on October 30, 2010 19:49:47

What steps will reproduce the problem?  
1. import psutil,time
2. p = []
3. for i in range(100); p.append(psutil.cpu_percent); time.sleep(0.1)
4. print sum(p)/100 

What is the expected output?  


What do you see instead?  
On my idling eight cpu machine, I expect to see an average cpu_percent < 0.15, 
relatively unaffected by larger sleep times.
Instead I see an average ~ 30, and significantly less as time.sleep increases 
(eg average 6 percent with sleep 1.0 second). 

What version of psutil are you using? What Python version?  
psutil 1.3, python 2.6.5 

On what operating system? Is it 32bit or 64bit version?  
linux 64 bit 

Please provide any additional information below.  
It looks like cpu_percent is only returning percentage of cpu running the 
python process, but haven't checked source.

Original issue: http://code.google.com/p/psutil/issues/detail?id=127

[giampaolo]

From g.rodola on October 30, 2010 11:06:50

I'm not sure to understand the problem you're describing and neither I'm able 
to run that code. It fails with:

*** TypeError: unsupported operand type(s) for +: 'int' and 'function'

[giampaolo]

From g.rodola on October 30, 2010 11:14:39

Didn't realize the code had some typos.
Here's what I get on Ubuntu 64 bit, 2 CPUs:
5.14361871132

Are you sure there are no other processes which are consuming the CPU?
Have you tried to look at top/htop/ghome-system-monitor while you do this?
What's the result of psutil.NUM_CPUS?

[giampaolo]

From [email protected] on October 30, 2010 12:44:57

Sorry about the typo.  Thanks for the quick reply.

psutil.NUM_CPUS is 8.

I have been monitoring all cpus with top, rather arbitrarily, but never see a 
single cpu less than 97% idle, while 5-6 out of 8 are 100% idle, while 
psutil.cpu_percent returns between 10 and 40%, depending on the sleep time (1 
sec to 0.1 sec).  Can't find any rogue background processes.
Above is running on a remote machine with putty.  With gnome system monitor, I 
get slightly higher cpu percentages, but maybe function of using remote desktop 
(nomachine).  I never, however, see any cpu with more than 10% and most idle, 
while cpu_percent() continues to return between 10 and 40%.

By eye, it looks like cpu_percent is the sum of the cpus rather than average.

When I replace above test with Process.get_cpu_percent(), with process obj 
initialized with os.getpid(), the result is less than one percent.  So clearly 
my speculation that it examines only current process cpu is wrong.

[giampaolo]

From [email protected] on October 30, 2010 13:37:15

> By eye, it looks like cpu_percent is the sum of the cpus rather than average.

cpu_percent is the percent of CPU utilization across all CPUs since the module 
initialization or the last time the function was called, whichever is more recent.

The calculation is performed by using getting the raw CPU idle time (as 
reported by the kernel), and the clock time, and figuring out what percentage 
of time was spent in a non-idle state. It's therefore a total utilization % 
across all CPUs. If you wanted average utilization per CPU you could divide by 
psutil.NUM_CPUS. 

Note also that support for per-process CPU statistics is being tracked as Issue #125

Status: Accepted

[giampaolo]

From [email protected] on October 30, 2010 13:52:20

Ok, thanks for the clarification.  Is it possible then for cpu_percent() to 
return > 100?

[giampaolo]

From [email protected] on October 30, 2010 14:16:42

> Ok, thanks for the clarification.  Is it possible then for cpu_percent() to 
return > 100?

I should clarify we are actually dividing by NUM_CPUS in the psutil code, 
because the amount of idle time returned by the kernel is total across all 
CPUs. For example, if 5 seconds have gone by, and I have two CPUs, I'll get 10 
seconds of CPU time for that period. What we do in cpu_percent is the 
following: 

    # get the percentage of the time that was spent in idle state 
    # this value can be up to NUM_CPUS * 100
    idle_percent = (idle_delta / time_delta) * 100.0

    # calculate the total possible amount of CPU % (NUM_CPUS * 100)
    # subtract the amount of idle percentage above and divide by the NUM_CPUS
    # this gives the CPU util percent during the time frame for all CPUs
    util_percent = ((100 * NUM_CPUS) - idle_percent) / NUM_CPUS

This number should match what you see in top/taskmgr/Activity Monitor etc. as 
your overall utilization for that period. It should be possible to come out 
greater than 100%.

Status: New

[giampaolo]

From [email protected] on October 30, 2010 14:23:14

Sorry, I meant should NOT be possible to come out > 100%

[giampaolo]

From [email protected] on November 01, 2010 08:51:18

Ok, I'll look into the code if I need to now.  Anyhow, psutil has been a 
helpful utility -- I just may need to adjust my expectations of cpu_percent but 
for my purposes (determining whether a new process should be started to work on 
a subproblem) that should be fine.  Thanks.

[giampaolo]

From [email protected] on November 10, 2010 10:50:12

Closing as invalid, since no change was needed here, and we are redoing all the 
CPU percent code in the next release.

Status: Invalid