Memory profiling #831
Description
Relates to: #825
Recently I've been increasing the server where we host the live demo because the server was running out of memory.
I've also been discussing whether it makes sense to limit the memory consumed by the application.
In parallel, it would be good to know which parts of the applications are the greatest memory consumers in the applications.
Likely the torrent repository is the data structure that consumes a great percentage of the application's total memory consumption, but there could also be other parts like:
- Threads (especially if we have deadlocks that keep threads around for too long)
- Responses
I don't know how to do memory profiling, but I would start with something simple like calculating the size and number of the structs we think are the most consuming and comparing it with the total memory used by the application.
@mickvandijke was doing memory consumption calculations in this PR for the torrent repository.
@Power2All suggested using pmap. This is the current output in the live demo:
pmap 3550131
3550131: /usr/bin/torrust-tracker
000056238845c000 568K r---- torrust-tracker
00005623884ea000 8284K r-x-- torrust-tracker
0000562388d01000 2712K r---- torrust-tracker
0000562388fa7000 380K r---- torrust-tracker
0000562389006000 16K rw--- torrust-tracker
000056238900a000 4K rw--- [ anon ]
0000562389a36000 132K rw--- [ anon ]
00007f18c0000000 10976K rw--- [ anon ]
00007f18c0ab8000 54560K ----- [ anon ]
00007f18c8000000 65232K rw--- [ anon ]
00007f18cbfb4000 304K ----- [ anon ]
00007f18d0000000 65532K rw--- [ anon ]
00007f18d3fff000 4K ----- [ anon ]
00007f18d4000000 65536K rw--- [ anon ]
00007f18d8000000 65536K rw--- [ anon ]
00007f18dc000000 65536K rw--- [ anon ]
00007f18e0000000 65532K rw--- [ anon ]
00007f18e3fff000 4K ----- [ anon ]
00007f18e4000000 65532K rw--- [ anon ]
00007f18e7fff000 4K ----- [ anon ]
00007f18e8000000 65536K rw--- [ anon ]
00007f18f0000000 65536K rw--- [ anon ]
00007f18f8000000 65536K rw--- [ anon ]
00007f18fc000000 65536K rw--- [ anon ]
00007f1900000000 65536K rw--- [ anon ]
00007f1904000000 65532K rw--- [ anon ]
00007f1907fff000 4K ----- [ anon ]
00007f1908000000 65536K rw--- [ anon ]
00007f190c000000 65536K rw--- [ anon ]
00007f1910000000 65536K rw--- [ anon ]
00007f1914000000 65536K rw--- [ anon ]
00007f1918000000 65536K rw--- [ anon ]
00007f191c000000 65536K rw--- [ anon ]
00007f1920000000 65536K rw--- [ anon ]
00007f1928000000 65536K rw--- [ anon ]
00007f192c000000 65536K rw--- [ anon ]
00007f1930000000 65536K rw--- [ anon ]
00007f1934000000 65536K rw--- [ anon ]
00007f1938000000 496K rw--- [ anon ]
00007f193807c000 65040K ----- [ anon ]
00007f193c000000 524K rw--- [ anon ]
00007f193c083000 65012K ----- [ anon ]
00007f1940000000 65532K rw--- [ anon ]
00007f1943fff000 4K ----- [ anon ]
00007f1944000000 496K rw--- [ anon ]
00007f194407c000 65040K ----- [ anon ]
00007f1948000000 65536K rw--- [ anon ]
00007f194d216000 4K ----- [ anon ]
00007f194d217000 8K rw--- [ anon ]
00007f194d219000 4K ----- [ anon ]
00007f194d21a000 8K rw--- [ anon ]
00007f194d21c000 4K ----- [ anon ]
00007f194d21d000 2048K rw--- [ anon ]
00007f194d41d000 4K ----- [ anon ]
00007f194d41e000 8K rw--- [ anon ]
00007f194d420000 4K ----- [ anon ]
00007f194d421000 2048K rw--- [ anon ]
00007f194d621000 4K ----- [ anon ]
00007f194d622000 2048K rw--- [ anon ]
00007f194d822000 4K ----- [ anon ]
00007f194d823000 8K rw--- [ anon ]
00007f194d825000 4K ----- [ anon ]
00007f194d826000 8K rw--- [ anon ]
00007f194d828000 4K ----- [ anon ]
00007f194d829000 2048K rw--- [ anon ]
00007f194da29000 4K ----- [ anon ]
00007f194da2a000 2048K rw--- [ anon ]
00007f194dc2a000 4K ----- [ anon ]
00007f194dc2b000 20K rw--- [ anon ]
00007f194dc30000 152K r---- libc.so.6
00007f194dc56000 1364K r-x-- libc.so.6
00007f194ddab000 332K r---- libc.so.6
00007f194ddfe000 16K r---- libc.so.6
00007f194de02000 8K rw--- libc.so.6
00007f194de04000 60K rw--- [ anon ]
00007f194de13000 64K r---- libm.so.6
00007f194de23000 460K r-x-- libm.so.6
00007f194de96000 360K r---- libm.so.6
00007f194def0000 4K r---- libm.so.6
00007f194def1000 4K rw--- libm.so.6
00007f194def2000 12K r---- libgcc_s.so.1
00007f194def5000 92K r-x-- libgcc_s.so.1
00007f194df0c000 16K r---- libgcc_s.so.1
00007f194df10000 4K r---- libgcc_s.so.1
00007f194df11000 4K rw--- libgcc_s.so.1
00007f194df12000 788K r---- libcrypto.so.3
00007f194dfd7000 2532K r-x-- libcrypto.so.3
00007f194e250000 884K r---- libcrypto.so.3
00007f194e32d000 388K r---- libcrypto.so.3
00007f194e38e000 12K rw--- libcrypto.so.3
00007f194e391000 12K rw--- [ anon ]
00007f194e394000 124K r---- libssl.so.3
00007f194e3b3000 376K r-x-- libssl.so.3
00007f194e411000 124K r---- libssl.so.3
00007f194e430000 40K r---- libssl.so.3
00007f194e43a000 16K rw--- libssl.so.3
00007f194e43e000 12K r---- libz.so.1
00007f194e441000 76K r-x-- libz.so.1
00007f194e454000 28K r---- libz.so.1
00007f194e45b000 4K r---- libz.so.1
00007f194e45c000 4K rw--- libz.so.1
00007f194e45d000 8K rw--- [ anon ]
00007f194e45f000 4K r---- ld-linux-x86-64.so.2
00007f194e460000 148K r-x-- ld-linux-x86-64.so.2
00007f194e485000 40K r---- ld-linux-x86-64.so.2
00007f194e48f000 8K r---- ld-linux-x86-64.so.2
00007f194e491000 8K rw--- ld-linux-x86-64.so.2
00007ffdbfef9000 132K rw--- [ stack ]
00007ffdbff62000 16K r---- [ anon ]
00007ffdbff66000 8K r-x-- [ anon ]
ffffffffff600000 4K --x-- [ anon ]
total 1997268K
And for top command:
top - 16:15:12 up 4 days, 9:13, 1 user, load average: 4.32, 3.34, 3.26
Tasks: 238 total, 1 running, 131 sleeping, 0 stopped, 106 zombie
%Cpu(s): 13.3 us, 13.2 sy, 0.0 ni, 28.1 id, 18.2 wa, 0.0 hi, 3.9 si, 23.3 st
MiB Mem : 3911.9 total, 129.3 free, 2302.8 used, 1479.9 buff/cache
MiB Swap: 0.0 total, 0.0 free, 0.0 used. 1355.2 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3550131 torrust 20 0 2062800 1.6g 10032 S 21.9 43.1 446:22.78 torrust-tracker