Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Latest ntopng segfaulting (again....) #1815

Closed
nate-byrnes opened this issue Jun 11, 2018 · 5 comments
Closed

Latest ntopng segfaulting (again....) #1815

nate-byrnes opened this issue Jun 11, 2018 · 5 comments

Comments

@nate-byrnes
Copy link

nate-byrnes commented Jun 11, 2018 •
edited
Loading

Related to #1725, I've pulled in the latest from the repository, and it too is crashing. Here is the backtrace:

#0 0x00007ffff43f2c37 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007ffff43f6028 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007ffff442f2a4 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#3 0x00007ffff443b82e in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#4 0x00000000004175d5 in Checkpointable::checkpoint (this=0x8988e0, vm=0x40100378, iface=0x8988e0, checkpoint_id=0 '\000', details_level=details_high)
at src/Checkpointable.cpp:77
#5 0x000000000047476e in NetworkInterface::checkPointInterfaceCounters (this=0x8988e0, vm=0x40100378, checkpoint_id=0 '\000', details_level=details_high)
at /home/deri/ntopng/include/NetworkInterface.h:329
#6 0x00000000004656a8 in ntop_checkpoint_network_interface (vm=0x40100378) at src/Lua.cpp:2938
#7 0x0000000000541707 in lj_BC_FUNCC ()
#8 0x0000000000530bcd in lua_pcall ()
#9 0x00000000004e2ae3 in LuaHandler::luaL_dofileM (this=0x7fff78000b40, execute=true) at pro/LuaHandler.cpp:34
#10 0x0000000000471a4d in __ntop_lua_handlefile (L=0x40100378, script_path=0x7fff74000a60 "/usr/share/ntopng/scripts/callbacks/interface/5min.lua", ex=true)
at src/Lua.cpp:6940
#11 0x0000000000472950 in Lua::run_script (this=0x7fff78000b80, script_path=0x7fff74000a60 "/usr/share/ntopng/scripts/callbacks/interface/5min.lua", iface=0x8988e0)
at src/Lua.cpp:7634
#12 0x00000000004c43e4 in ThreadedActivity::runScript (this=0xa399a00, script_path=0x7fff74000a60 "/usr/share/ntopng/scripts/callbacks/interface/5min.lua", iface=0x8988e0)
at src/ThreadedActivity.cpp:170
#13 0x00000000004c4db2 in ThreadPool::run (this=0xa399aa0) at src/ThreadPool.cpp:101
#14 0x00000000004c4a7e in doRun (ptr=0xa399aa0) at src/ThreadPool.cpp:37
#15 0x00007ffff6b2e184 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#16 0x00007ffff44b9ffd in clone () from /lib/x86_64-linux-gnu/libc.so.6
@emanuele-f
Copy link
Contributor

emanuele-f commented Jun 12, 2018 •
edited
Loading

Can you confirm the ntopng version used with ntopng --version?

Can you post your ntopng log?

@nate-byrnes
Copy link
Author

v.3.5.180607 [Enterprise/Professional build]

The log is presently 409MB.... I'm not sure how to isolate the relevant portions of it.

@emanuele-f
Copy link
Contributor

Please remove the -v option from the configuration and run journalctl -u ntopng, then scroll the log to the bottom

@nate-byrnes
Copy link
Author

No systemd on this box, its ubuntu 14.04 .... I've been running in GDB to try and catch backtraces.... I've just killed it, removed the old log file, and restarted with service ntopng start. Here is the log since starting:

12/Jun/2018 16:41:21 [Prefs.cpp:968] ntopng will use redis localhost@0
12/Jun/2018 16:41:21 [Ntop.cpp:1499] Setting local networks to 173.225.53.0/24,208.83.121.0/24,208.69.85.0/24,208.68.84.137/29,10.88.0.0/16,10.254.0.0/16
12/Jun/2018 16:41:21 [Redis.cpp:127] Successfully connected to redis localhost:6379@0
12/Jun/2018 16:41:21 [Redis.cpp:127] Successfully connected to redis localhost:6379@0
12/Jun/2018 16:41:21 [NtopPro.cpp:280] [LICENSE] Reading license from Redis
12/Jun/2018 16:41:21 [NtopPro.cpp:188] [LICENSE] C75E888ABC5E5765B37E4B29ECC974B815303668851E9BBC9D: unable to validate Enterprise license [License mismatch error]
12/Jun/2018 16:41:21 [NtopPro.cpp:191] [LICENSE] C75E888ABC5E5765B37E4B29ECC974B815303668851E9BBC9D: found valid Professional license
12/Jun/2018 16:41:24 [PF_RINGInterface.cpp:53] Reading packets from PF_RING v.7.1.0 interface eth1...
12/Jun/2018 16:41:24 [PF_RINGInterface.cpp:190] Packet capture filter set to "not host 10.88.29.18"
12/Jun/2018 16:41:24 [Ntop.cpp:1626] Registered interface eth1 [id: 4]
12/Jun/2018 16:41:24 [PF_RINGInterface.cpp:53] Reading packets from PF_RING v.7.1.0 interface eth2...
12/Jun/2018 16:41:24 [PF_RINGInterface.cpp:190] Packet capture filter set to "not host 10.88.29.18"
12/Jun/2018 16:41:24 [Ntop.cpp:1626] Registered interface eth2 [id: 5]
12/Jun/2018 16:41:25 [PF_RINGInterface.cpp:53] Reading packets from PF_RING v.7.1.0 interface eth4...
12/Jun/2018 16:41:25 [PF_RINGInterface.cpp:190] Packet capture filter set to "not host 10.88.29.18"
12/Jun/2018 16:41:25 [Ntop.cpp:1626] Registered interface eth4 [id: 7]
12/Jun/2018 16:41:25 [PF_RINGInterface.cpp:53] Reading packets from PF_RING v.7.1.0 interface eth5...
12/Jun/2018 16:41:25 [PF_RINGInterface.cpp:190] Packet capture filter set to "not host 10.88.29.18"
12/Jun/2018 16:41:25 [Ntop.cpp:1626] Registered interface eth5 [id: 11]
12/Jun/2018 16:41:25 [main.cpp:316] PID stored in file /var/run/ntopng.pid
12/Jun/2018 16:41:25 [HTTPserver.cpp:917] Please read https://github.com/ntop/ntopng/blob/dev/doc/README.SSL if you want to enable SSL.
12/Jun/2018 16:41:25 [Utils.cpp:473] User changed to nobody
12/Jun/2018 16:41:25 [HTTPserver.cpp:989] Web server dirs [/usr/share/ntopng/httpdocs][/usr/share/ntopng/scripts]
12/Jun/2018 16:41:25 [HTTPserver.cpp:992] HTTP server listening on port(s) 3000
12/Jun/2018 16:41:25 [main.cpp:398] Working directory: /var/tmp/ntopng
12/Jun/2018 16:41:25 [main.cpp:400] Scripts/HTML pages directory: /usr/share/ntopng
12/Jun/2018 16:41:25 [Ntop.cpp:391] Welcome to ntopng x86_64 v.3.5.180607 - (C) 1998-18 ntop.org
12/Jun/2018 16:41:25 [Ntop.cpp:401] Built on Ubuntu 14.04.5 LTS
12/Jun/2018 16:41:25 [NtopPro.cpp:531] [LICENSE] System Id: CBCB98A709B0B922
12/Jun/2018 16:41:25 [NtopPro.cpp:532] [LICENSE] Edition: Professional
12/Jun/2018 16:41:25 [NtopPro.cpp:533] [LICENSE] License Type: Permanent License
12/Jun/2018 16:41:25 [NtopPro.cpp:536] [LICENSE] License: C75E888ABC5E5765B37E4B29ECC974B815303668851E9BBC9D
12/Jun/2018 16:41:25 [NtopPro.cpp:546] [LICENSE] Maintenance: Until Sat Jun 30 09:54:45 2018 [17 days left]
12/Jun/2018 16:41:25 [Ntop.cpp:721] Adding fe80::215:17ff:fe9c:1135/128 as IPv6 interface address for eth1
12/Jun/2018 16:41:25 [Ntop.cpp:730] Adding fe80::215:17ff:fe9c:1135/64 as IPv6 local network for eth1
12/Jun/2018 16:41:25 [Ntop.cpp:721] Adding fe80::215:17ff:fe9c:1134/128 as IPv6 interface address for eth2
12/Jun/2018 16:41:25 [Ntop.cpp:730] Adding fe80::215:17ff:fe9c:1134/64 as IPv6 local network for eth2
12/Jun/2018 16:41:25 [Ntop.cpp:721] Adding fe80::215:17ff:fe9c:1137/128 as IPv6 interface address for eth4
12/Jun/2018 16:41:25 [Ntop.cpp:730] Adding fe80::215:17ff:fe9c:1137/64 as IPv6 local network for eth4
12/Jun/2018 16:41:25 [Ntop.cpp:721] Adding fe80::215:17ff:fe9c:1136/128 as IPv6 interface address for eth5
12/Jun/2018 16:41:25 [Ntop.cpp:730] Adding fe80::215:17ff:fe9c:1136/64 as IPv6 local network for eth5
12/Jun/2018 16:41:25 [PeriodicActivities.cpp:59] Started periodic activities loop...
12/Jun/2018 16:41:27 [PeriodicActivities.cpp:100] Each periodic activity script will use 4 threads
12/Jun/2018 16:41:27 [NetworkInterface.cpp:2470] Started packet polling on interface eth1 [id: 4]...
12/Jun/2018 16:41:27 [NetworkInterface.cpp:2470] Started packet polling on interface eth2 [id: 5]...
12/Jun/2018 16:41:27 [NetworkInterface.cpp:2470] Started packet polling on interface eth4 [id: 7]...
12/Jun/2018 16:41:27 [NetworkInterface.cpp:2470] Started packet polling on interface eth5 [id: 11]...

@emanuele-f
Copy link
Contributor

Thank you, your backtrace was very useful.

emanuele-f pushed a commit that referenced this issue Jun 13, 2018
Fix bad checkpoint_id leading to concurrency issues and possibly crash
931c9c1 
Fixes #1815
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@emanuele-f @nate-byrnes