Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cpu usage 100% issue #57

Open
sampatsharma143 opened this issue Oct 27, 2024 · 3 comments
Open

Cpu usage 100% issue #57

sampatsharma143 opened this issue Oct 27, 2024 · 3 comments

Comments

@sampatsharma143
Copy link

image
Current i am running HLS-Proxy on a production app , there are 300-200 concurrent users and i have a vps with 90gb ram and 32 threads. it runs okay for some time and than after few hours( 3-4 ) cpu usages reaches 100% and it makes the video player unresponsive.
can you tell me how can i identify the core issue and fix it. it will be very helpful.
thank you

@warren-bank
Copy link
Owner

warren-bank commented Oct 27, 2024

Off-hand, I can't imagine what would cause this. I've never run into any CPU or RAM issues, but I've also never used the proxy in a production environment. I'm going to cross-reference this issue with another, which also has similar observations.

Either myself, or someone else, will need to run some kind of software to profile usage.. that can identify exactly which block of code within hlsd.js is responsible for the excessive usage.

My first question would be: what command-line options do you use when you start the proxy?

update: I've added some scripts to the repo that simplify the task of running the Node.js built-in profiler, applying some load on the proxy server, and then processing the raw log file to produce a log file that provided good insight. Unfortunately, the trivial load applied by this script doesn't produce any performance issues; nearly all of the CPU cycles are spent performing regex operations.. as would be expected. But.. it's a starting point…

@sampatsharma143
Copy link
Author

This is the pm2 command that i use to start the proxy -
pm2 start hls-proxy/bin/hlsd.js -i max --name hls-proxy -- --host "domain:443" --req-insecure --port "5001" -v 3

@sampatsharma143
Copy link
Author

sampatsharma143 commented Oct 29, 2024

I still not able to find the real cause of this so i made a workaround for some time.
i made a bash script to check the cpu usages every minute and if it's >=90% it will restart the pm2 process

Bash Script
`#!/bin/bash

Load NVM directly

export NVM_DIR="/root/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" # Load nvm
[ -s "$NVM_DIR/bash_completion" ] && . "$NVM_DIR/bash_completion" # Load nvm bash_completion (optional)

Use the specific Node.js version

nvm use 20.18.0

IDLE=$(top -bn1 | grep "Cpu(s)" | sed "s/., ([0-9.])% id.*/\1/" | awk '{print $1}')

Check if idle is 10% or less

if (( $(echo "$IDLE <= 10" | bc -l) )); then
echo "$(date -u) Overall CPU idle is at or below 10%. Restarting PM2 process hls-proxy"
pm2 restart hls-proxy
else
echo "$(date -u) Overall CPU usage is normal."
fi`

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants