Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

We have observerd high CPU usage. Is there any way to pass a parameter and set a limit on how much cpu the tool can consume? #137

Open
fernandomt08 opened this issue Dec 17, 2021 · 6 comments
Assignees
Labels
discussion question or suggestion enhancement New feature or request patch released

Comments

@fernandomt08
Copy link

No description provided.

@xeraph xeraph self-assigned this Dec 17, 2021
@xeraph xeraph added the discussion question or suggestion label Dec 17, 2021
@xeraph
Copy link
Contributor

xeraph commented Dec 17, 2021

It may depends on what operating system and version you are using.

@jlellis
Copy link

jlellis commented Dec 17, 2021

I'd use the OS to do that:
On Linux/BSD/Etc:

$ nice java -jar scanner.jar

On windows (untested):

$ start /belownormal java -jar scanner.jar

Reference: https://stackoverflow.com/questions/4208/windows-equivalent-of-nice

@jgstew
Copy link
Contributor

jgstew commented Dec 17, 2021

It would be useful if there was some way to have the scan utility use like 10% of a single CPU core or similar to be self limiting in terms of both CPU and diskIO. This is especially important when running the scan on many virtual machines on the same host at once since they can all end up hitting the same CPU and Disks at the same time.

We do recommend staggering the scan when deploying it (or really any similar thing) which helps with this problem, but having a way to cause it to be more self limiting would help minimize the chances of problems.

That said, I have no clue how hard a feature like this would be to implement. One potential approach is to have the scan work for approximately 1 second, then sleep for 3 seconds, then work for 1 second, etc... which can approximate a maximum 25% CPU usage by doing this, assuming that it uses 100% of a cpu core when working and calculating the percentage as: 1s-work / (1s-work + 3s-sleep)

Things like "start /belownormal" and "nice" have the thread run at lower priority, but it will still run at 100% if it can. The lower priority is great for not harming things running on the system, but it does not help in the case of a shared VM host having all of it's VMs running it for 100% all at the same time.

@rprengel2
Copy link

Hallo,
we have simmilar problems here.
It would be helpful to reduce the cpu load using a config-option.

@rprengel2
Copy link

Hallo,
load is caused here by tons of files that must be checked (developer-systems).
A working wildcard feature should help but less consuming ressources would be nice too.
Thanks for your great tool.
Ralf

@xeraph
Copy link
Contributor

xeraph commented Jan 27, 2022

@fernandomt08 @rprengel2
Would you test v2.8.1 release? You can use --throttle option to limit CPU usage. For example, --throttle 1000 means that scanner is allowed to scan 1000 files per second. I also improved overall ZIP scan performance.

@xeraph xeraph added enhancement New feature or request patch released labels Jan 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion question or suggestion enhancement New feature or request patch released
Projects
None yet
Development

No branches or pull requests

5 participants