Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

单节点启动,出现too many open files问题 #171

Closed
mxyue opened this issue Aug 2, 2022 · 7 comments
Closed

单节点启动,出现too many open files问题 #171

mxyue opened this issue Aug 2, 2022 · 7 comments

Comments

@mxyue
Copy link

mxyue commented Aug 2, 2022

客户端走websocket连接。

[yucuix@jgjapp-dev ~]$ sudo lsof -a -itcp -p 14723
COMMAND   PID USER   FD   TYPE    DEVICE SIZE/OFF NODE NAME
bin     14723 root    3u  IPv4 448738745      0t0  TCP localhost:ibm-mqisdp (LISTEN)
bin     14723 root    7u  IPv6 448739940      0t0  TCP *:secure-mqtt (LISTEN)
bin     14723 root    8u  IPv6 448739942      0t0  TCP *:us-cli (LISTEN)
bin     14723 root   10u  IPv4 448740762      0t0  TCP localhost:8084 (LISTEN)
bin     14723 root   11u  IPv4 448740763      0t0  TCP localhost:us-srv (LISTEN)
bin     14723 root   12u  IPv4 448740342      0t0  TCP localhost:56224->localhost:14003 (ESTABLISHED)
bin     14723 root   16u  IPv6 448782688      0t0  TCP localhost:secure-mqtt->localhost:50428 (ESTABLISHED)
bin     14723 root   20u  IPv6 448783484      0t0  TCP localhost:secure-mqtt->localhost:50324 (ESTABLISHED)
bin     14723 root   21u  IPv6 448781930      0t0  TCP localhost:secure-mqtt->localhost:50360 (ESTABLISHED)
bin     14723 root   30u  IPv6 448780932      0t0  TCP localhost:secure-mqtt->localhost:50126 (ESTABLISHED)
bin     14723 root   31u  IPv6 448778284      0t0  TCP localhost:secure-mqtt->localhost:49816 (CLOSE_WAIT)
bin     14723 root   32u  IPv6 448779220      0t0  TCP localhost:secure-mqtt->localhost:50232 (ESTABLISHED)
bin     14723 root   33u  IPv6 448782040      0t0  TCP localhost:secure-mqtt->localhost:50434 (ESTABLISHED)

端口连接看不出什么。开发环境下就几个连接,但是socket连接一直没释放断开的连接。

[root@jgjapp-dev fd]# ls /proc/31816/fd -l
总用量 0
lr-x------ 1 root root 64 8月   2 14:24 0 -> /dev/null
lrwx------ 1 root root 64 8月   2 14:24 1 -> socket:[448738742]
lrwx------ 1 root root 64 8月   2 14:24 10 -> socket:[448740762]
lrwx------ 1 root root 64 8月   2 14:24 11 -> socket:[448740763]
lrwx------ 1 root root 64 8月   2 14:24 12 -> socket:[448740342]
lrwx------ 1 root root 64 8月   2 14:24 13 -> socket:[448768168]
lrwx------ 1 root root 64 8月   2 14:24 14 -> socket:[448742405]
lrwx------ 1 root root 64 8月   2 14:24 15 -> socket:[448750807]
lrwx------ 1 root root 64 8月   2 14:24 16 -> socket:[448800654]
lrwx------ 1 root root 64 8月   2 14:24 17 -> socket:[448743237]
lrwx------ 1 root root 64 8月   2 14:24 18 -> socket:[448743240]
lrwx------ 1 root root 64 8月   2 14:24 19 -> socket:[448743301]
lrwx------ 1 root root 64 8月   2 14:24 2 -> socket:[448738742]
lrwx------ 1 root root 64 8月   2 14:24 20 -> socket:[448817234]
lrwx------ 1 root root 64 8月   2 14:24 21 -> socket:[448816826]
lrwx------ 1 root root 64 8月   2 14:24 22 -> socket:[448754806]
lrwx------ 1 root root 64 8月   2 14:24 23 -> socket:[448753159]
lrwx------ 1 root root 64 8月   2 14:24 24 -> socket:[448756526]
lrwx------ 1 root root 64 8月   2 14:24 25 -> socket:[448761346]
lrwx------ 1 root root 64 8月   2 14:24 26 -> socket:[448761448]
lrwx------ 1 root root 64 8月   2 14:24 27 -> socket:[448768765]
lrwx------ 1 root root 64 8月   2 14:24 28 -> socket:[448769909]
lrwx------ 1 root root 64 8月   2 14:24 29 -> socket:[448775662]
.........
lrwx------ 1 root root 64 8月   3 10:22 99 -> socket:[449476728]
lrwx------ 1 root root 64 8月   3 10:22 990 -> socket:[451674604]
lrwx------ 1 root root 64 8月   3 10:22 991 -> socket:[451679321]
lrwx------ 1 root root 64 8月   3 10:22 992 -> socket:[451676686]
lrwx------ 1 root root 64 8月   3 10:22 993 -> socket:[451682537]
lrwx------ 1 root root 64 8月   3 10:22 994 -> socket:[451680602]
lrwx------ 1 root root 64 8月   3 10:22 995 -> socket:[451682682]
lrwx------ 1 root root 64 8月   3 10:22 996 -> socket:[451684764]
lrwx------ 1 root root 64 8月   3 10:22 997 -> socket:[451695183]
lrwx------ 1 root root 64 8月   3 10:22 998 -> socket:[451686259]
lrwx------ 1 root root 64 8月   3 10:22 999 -> socket:[451686262]

@mxyue mxyue changed the title 但节点启动,也出现too many open files问题 单节点启动,客户端无故断开连接,同时出现too many open files问题 Aug 2, 2022
@mxyue mxyue changed the title 单节点启动,客户端无故断开连接,同时出现too many open files问题 单节点启动,同时出现too many open files问题 Aug 2, 2022
@mxyue mxyue closed this as completed Aug 2, 2022
@mxyue mxyue reopened this Aug 3, 2022
@mxyue mxyue changed the title 单节点启动,同时出现too many open files问题 单节点启动,出现too many open files问题 Aug 3, 2022
@DrmagicE
Copy link
Owner

DrmagicE commented Aug 3, 2022

可能是有bug导致连接没有释放,可以观察一下监控数据看看有没有什么端倪。(要打开Prometheus插件,默认是开的)

curl 127.0.0.1:8082/metrics

@mxyue
Copy link
Author

mxyue commented Aug 5, 2022

数量连接过多后,就没法创建新的连接了。打不开Prometheus连接
我后面再找时间看下。

@mxyue
Copy link
Author

mxyue commented Aug 9, 2022

我用go 1.18去编译的。这个有没有影响呢

@wind-c
Copy link

wind-c commented Sep 10, 2022

comqtt support the distributed cluster.If you are interested, please refer to https://github.com/wind-c/comqtt. Sorry, gmqtt is also great.

1 similar comment
@wind-c
Copy link

wind-c commented Sep 10, 2022

comqtt support the distributed cluster.If you are interested, please refer to https://github.com/wind-c/comqtt. Sorry, gmqtt is also great.

@DrmagicE
Copy link
Owner

@mxyue 有可能是鉴权失败,客户端不断开连接。导致连接开太多了。可以试一下按照这个comment先加一行能不能解决
#177 (comment)

@mxyue mxyue closed this as completed Feb 16, 2023
@mxyue
Copy link
Author

mxyue commented Feb 16, 2023

更新代码后,内存泄漏的问题没有了。可能是那个问题造成的

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants