Memory leak in ahocorasick #2258

IvanNardi · 2024-01-12T11:11:42Z

Oss-fuzz keeps reporting a memory leak in ahocorasick code, via fuzz_filecfg_protocols fuzzer.
Some examples (these reports should be public):
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64150&q=ndpi&can=1&sort=-id
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=62269&q=ndpi&can=1&sort=-id
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=61934&q=ndpi&can=1&sort=-id

It seems that the leak is about inserting duplicated patterns.

The stack reported is something like:

            #6 0x67f7c9 in ac_automata_add [ndpi/src/lib/third_party/src/ahocorasick.c:255](https://github.com/ntop/nDPI/blob/7b2bbb2309264766697507365231fd6ee5717e31/src/lib/third_party/src/ahocorasick.c#L255):19
	    #7 0x58df28 in ndpi_add_host_risk_mask [ndpi/src/lib/ndpi_main.c:4262](https://github.com/ntop/nDPI/blob/7b2bbb2309264766697507365231fd6ee5717e31/src/lib/ndpi_main.c#L4262):8
	    #8 0x592605 in ndpi_handle_rule [ndpi/src/lib/ndpi_main.c:4344](https://github.com/ntop/nDPI/blob/7b2bbb2309264766697507365231fd6ee5717e31/src/lib/ndpi_main.c#L4344):11
	    #9 0x592605 in load_protocols_file_fd [ndpi/src/lib/ndpi_main.c:5036](https://github.com/ntop/nDPI/blob/7b2bbb2309264766697507365231fd6ee5717e31/src/lib/ndpi_main.c#L5036):8

The text was updated successfully, but these errors were encountered:

mmanoj · 2024-07-20T18:19:43Z

@IvanNardi

Is this issue still valid?

IvanNardi · 2024-07-21T15:26:11Z

Yes, according to oss-fuzz. If you are interested I can provide an up-to-date artifact to reproduce the error

mmanoj · 2024-07-22T11:11:45Z

@IvanNardi

Thanks for the feedback, lets workout this issue as well together as we done same way of FPC feature.This way we can achieve more good results.

mmanoj · 2024-07-26T10:11:15Z

@IvanNardi
Can we work on this?

IvanNardi · 2024-07-29T07:22:09Z

I don't have any expertise on ahocorasick code, so I can't guide/suggest you how to fix it; I can gladly review your changes, tough.
To reproduce the error, with latest code:

ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ ./autogen.sh --enable-debug-build --enable-fuzztargets --with-sanitizer && make -s -j
[...]
ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ ./fuzz/fuzz_filecfg_protocols ~/Downloads/clusterfuzz-testcase-minimized-fuzz_filecfg_protocols-4513089035239424 
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3967398786
INFO: Loaded 1 modules   (17 inline 8-bit counters): 17 [0x55849c114e7c, 0x55849c114e8d), 
INFO: Loaded 1 PC tables (17 PCs): 17 [0x55849c114e90,0x55849c114fa0), 
./fuzz/fuzz_filecfg_protocols: Running 1 inputs 1 time(s) each.
Running: /home/ivan/Downloads/clusterfuzz-testcase-minimized-fuzz_filecfg_protocols-4513089035239424

=================================================================
==43734==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 40 byte(s) in 1 object(s) allocated from:
    #0 0x55849b7651bf in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x6f91bf) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #1 0x55849b88a986 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25
    #2 0x55849b88a9dd in ndpi_calloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:67:13
    #3 0x55849bb7181a in node_create /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ahocorasick.c:802:25
    #4 0x55849bb7307a in node_create_next /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ahocorasick.c:1007:10
    #5 0x55849bb72223 in ac_automata_add /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ahocorasick.c:255:19
    #6 0x55849b7d5f79 in ndpi_add_host_risk_mask /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4573:8
    #7 0x55849b7dfb08 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4655:11
    #8 0x55849b7df078 in load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5446:8
    #9 0x55849b7a7c1e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:21:3
    #10 0x55849b6add16 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x641d16) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #11 0x55849b697e98 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x62be98) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #12 0x55849b69d96a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x63196a) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #13 0x55849b6c72d2 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x65b2d2) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #14 0x7fcae91b9082 in __libc_start_main /build/glibc-LcI20x/glibc-2.31/csu/../csu/libc-start.c:308:16

Indirect leak of 328 byte(s) in 1 object(s) allocated from:
    #0 0x55849b7651bf in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x6f91bf) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #1 0x55849b88a986 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25
    #2 0x55849b88a9dd in ndpi_calloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:67:13
    #3 0x55849bb84148 in node_resize_mp /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ahocorasick.c:1027:13
    #4 0x55849bb733db in node_register_matchstr /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ahocorasick.c:1055:30
    #5 0x55849bb72a2d in ac_automata_add /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ahocorasick.c:280:6
    #6 0x55849b7d5f79 in ndpi_add_host_risk_mask /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4573:8
    #7 0x55849b7dfb08 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4655:11
    #8 0x55849b7df078 in load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5446:8
    #9 0x55849b7a7c1e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:21:3
    #10 0x55849b6add16 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x641d16) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #11 0x55849b697e98 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x62be98) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #12 0x55849b69d96a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x63196a) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #13 0x55849b6c72d2 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x65b2d2) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #14 0x7fcae91b9082 in __libc_start_main /build/glibc-LcI20x/glibc-2.31/csu/../csu/libc-start.c:308:16

Indirect leak of 257 byte(s) in 1 object(s) allocated from:
    #0 0x55849b7651bf in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x6f91bf) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #1 0x55849b88a986 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25
    #2 0x55849b88ac14 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:113:13
    #3 0x55849b7d5d5c in ndpi_add_host_risk_mask /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4558:14
    #4 0x55849b7dfb08 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4655:11
    #5 0x55849b7df078 in load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5446:8
    #6 0x55849b7a7c1e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:21:3
    #7 0x55849b6add16 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x641d16) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #8 0x55849b697e98 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x62be98) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #9 0x55849b69d96a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x63196a) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #10 0x55849b6c72d2 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x65b2d2) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #11 0x7fcae91b9082 in __libc_start_main /build/glibc-LcI20x/glibc-2.31/csu/../csu/libc-start.c:308:16

SUMMARY: AddressSanitizer: 625 byte(s) leaked in 3 allocation(s).

INFO: a leak has been found in the initial corpus.

INFO: to ignore leaks on libFuzzer side use -detect_leaks=0

clusterfuzz-testcase-minimized-fuzz_filecfg_protocols-4513089035239424.zip

mmanoj · 2024-08-01T08:08:05Z

@IvanNardi

Thanks for the details and sorry for the late reply.I have some experience with same type of memory issue fixing.Let me analysis the code and update my finding within early next week.So then we can plan together the fixing approach.

This reverts commit e482e28.

skipping node at depth = AC_PATTRN_MAX_LENGTH inside caused this leak this change avoid this Fix: ==3162838==ERROR: LeakSanitizer: detected memory leaks Direct leak of 40 byte(s) in 1 object(s) allocated from: 0 0x50f072 in malloc (/media/veracrypt1/work-space/ntop/nDPI/fuzz/fuzz_filecfg_protocols+0x50f072) (BuildId: 36867732af52eb22433f5e34fe5422a8af281aa6) 1 0x552f20 in malloc_wrapper /root/workspace/ntop/nDPI/fuzz/fuzz_common_code.c:16:31 2 0x68a0a5 in ndpi_malloc /root/workspace/ntop/nDPI/src/lib/ndpi_memory.c:60:25 3 0x68a0fd in ndpi_calloc /root/workspace/ntop/nDPI/src/lib/ndpi_memory.c:67:13 4 0x9c5b3a in node_create /root/workspace/ntop/nDPI/src/lib/third_party/src/ahocorasick.c:802:25 5 0x9c760e in node_create_next /root/workspace/ntop/nDPI/src/lib/third_party/src/ahocorasick.c:1007:10 6 0x9c6653 in ac_automata_add /root/workspace/ntop/nDPI/src/lib/third_party/src/ahocorasick.c:255:19 7 0x5861d0 in ndpi_add_host_risk_mask /root/workspace/ntop/nDPI/src/lib/ndpi_main.c:4573:8 8 0x590c0e in ndpi_handle_rule /root/workspace/ntop/nDPI/src/lib/ndpi_main.c:4655:11 9 0x58ffd5 in load_protocols_file_fd /root/workspace/ntop/nDPI/src/lib/ndpi_main.c:5446:8 10 0x552c60 in LLVMFuzzerTestOneInput /root/workspace/ntop/nDPI/fuzz/fuzz_filecfg_protocols.c:21:3 11 0x44be94 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/media/veracrypt1/work-space/ntop/nDPI/fuzz/fuzz_filecfg_protocols+0x44be94) (Build Id: 36867732af52eb22433f5e34fe5422a8af281aa6) 12 0x4334f7 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/media/veracrypt1/work-space/ntop/nDPI/fuzz/fuzz_filecfg_protocols+0x4334f7) (BuildId: 3 6867732af52eb22433f5e34fe5422a8af281aa6) 13 0x4391e1 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/media/veracrypt1/work-space/ntop/nDPI/fuzz/fuzz_filecfg_protocols+0x4 391e1) (BuildId: 36867732af52eb22433f5e34fe5422a8af281aa6) 14 0x467323 in main (/media/veracrypt1/work-space/ntop/nDPI/fuzz/fuzz_filecfg_protocols+0x467323) (BuildId: 36867732af52eb22433f5e34fe5422a8af281aa6) 15 0x7fce386e97e4 in __libc_start_main (/lib64/libc.so.6+0x3a7e4) (BuildId: b81415c1738806b536fb1599d7af2d15bf6a86b7) Indirect leak of 328 byte(s) in 1 object(s) allocated from: 0 0x50f072 in malloc (/media/veracrypt1/work-space/ntop/nDPI/fuzz/fuzz_filecfg_protocols+0x50f072) (BuildId: 36867732af52eb22433f5e34fe5422a8af281aa6) 1 0x552f20 in malloc_wrapper /root/workspace/ntop/nDPI/fuzz/fuzz_common_code.c:16:31 2 0x68a0a5 in ndpi_malloc /root/workspace/ntop/nDPI/src/lib/ndpi_memory.c:60:25 3 0x68a0fd in ndpi_calloc /root/workspace/ntop/nDPI/src/lib/ndpi_memory.c:67:13 4 0x9da348 in node_resize_mp /root/workspace/ntop/nDPI/src/lib/third_party/src/ahocorasick.c:1027:13 5 0x9c79d7 in node_register_matchstr /root/workspace/ntop/nDPI/src/lib/third_party/src/ahocorasick.c:1055:30 6 0x9c6f21 in ac_automata_add /root/workspace/ntop/nDPI/src/lib/third_party/src/ahocorasick.c:280:6 7 0x5861d0 in ndpi_add_host_risk_mask /root/workspace/ntop/nDPI/src/lib/ndpi_main.c:4573:8 8 0x590c0e in ndpi_handle_rule /root/workspace/ntop/nDPI/src/lib/ndpi_main.c:4655:11 9 0x58ffd5 in load_protocols_file_fd /root/workspace/ntop/nDPI/src/lib/ndpi_main.c:5446:8 10 0x552c60 in LLVMFuzzerTestOneInput /root/workspace/ntop/nDPI/fuzz/fuzz_filecfg_protocols.c:21:3 11 0x44be94 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/media/veracrypt1/work-space/ntop/nDPI/fuzz/fuzz_filecfg_protocols+0x44be94) (Build Id: 36867732af52eb22433f5e34fe5422a8af281aa6) 12 0x4334f7 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/media/veracrypt1/work-space/ntop/nDPI/fuzz/fuzz_filecfg_protocols+0x4334f7) (BuildId: 3 6867732af52eb22433f5e34fe5422a8af281aa6) 13 0x4391e1 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/media/veracrypt1/work-space/ntop/nDPI/fuzz/fuzz_filecfg_protocols+0x4 391e1) (BuildId: 36867732af52eb22433f5e34fe5422a8af281aa6) 14 0x467323 in main (/media/veracrypt1/work-space/ntop/nDPI/fuzz/fuzz_filecfg_protocols+0x467323) (BuildId: 36867732af52eb22433f5e34fe5422a8af281aa6) 15 0x7fce386e97e4 in __libc_start_main (/lib64/libc.so.6+0x3a7e4) (BuildId: b81415c1738806b536fb1599d7af2d15bf6a86b7) Indirect leak of 257 byte(s) in 1 object(s) allocated from: 0 0x50f072 in malloc (/media/veracrypt1/work-space/ntop/nDPI/fuzz/fuzz_filecfg_protocols+0x50f072) (BuildId: 36867732af52eb22433f5e34fe5422a8af281aa6) 1 0x552f20 in malloc_wrapper /root/workspace/ntop/nDPI/fuzz/fuzz_common_code.c:16:31 2 0x68a0a5 in ndpi_malloc /root/workspace/ntop/nDPI/src/lib/ndpi_memory.c:60:25 3 0x68a390 in ndpi_strdup /root/workspace/ntop/nDPI/src/lib/ndpi_memory.c:113:13 4 0x585e2b in ndpi_add_host_risk_mask /root/workspace/ntop/nDPI/src/lib/ndpi_main.c:4558:14 5 0x590c0e in ndpi_handle_rule /root/workspace/ntop/nDPI/src/lib/ndpi_main.c:4655:11 6 0x58ffd5 in load_protocols_file_fd /root/workspace/ntop/nDPI/src/lib/ndpi_main.c:5446:8 7 0x552c60 in LLVMFuzzerTestOneInput /root/workspace/ntop/nDPI/fuzz/fuzz_filecfg_protocols.c:21:3 8 0x44be94 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/media/veracrypt1/work-space/ntop/nDPI/fuzz/fuzz_filecfg_protocols+0x44be94) (BuildI d: 36867732af52eb22433f5e34fe5422a8af281aa6) 9 0x4334f7 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/media/veracrypt1/work-space/ntop/nDPI/fuzz/fuzz_filecfg_protocols+0x4334f7) (BuildId: 36 867732af52eb22433f5e34fe5422a8af281aa6) 10 0x4391e1 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/media/veracrypt1/work-space/ntop/nDPI/fuzz/fuzz_filecfg_protocols+0x4 391e1) (BuildId: 36867732af52eb22433f5e34fe5422a8af281aa6) 11 0x467323 in main (/media/veracrypt1/work-space/ntop/nDPI/fuzz/fuzz_filecfg_protocols+0x467323) (BuildId: 36867732af52eb22433f5e34fe5422a8af281aa6) ntop#12 0x7fce386e97e4 in __libc_start_main (/lib64/libc.so.6+0x3a7e4) (BuildId: b81415c1738806b536fb1599d7af2d15bf6a86b7) SUMMARY: AddressSanitizer: 625 byte(s) leaked in 3 allocation(s). Signed-off-by: mmaatuq <[email protected]>

IvanNardi added question bug help wanted and removed question labels Jan 12, 2024

IvanNardi mentioned this issue Mar 4, 2024

ahocorasick: improve matching with subdomains #2331

Merged

0xA50C1A1 mentioned this issue Mar 6, 2024

Add Path of Exile protocol dissector #2337

Merged

3 tasks

This was referenced Apr 10, 2024

Add The Elder Scrolls Online support #2376

Merged

STUN: fix boundary checks on attribute list parsing #2387

Merged

pavlinux pushed a commit to pavlinux/nDPI that referenced this issue Aug 5, 2024

Fix Memory leak in ahocorasick ntop#2258

e482e28

pavlinux mentioned this issue Aug 5, 2024

Fix Memory leak in ahocorasick #2258 #2517

Closed

3 tasks

pavlinux pushed a commit to pavlinux/nDPI that referenced this issue Aug 5, 2024

Revert "Fix Memory leak in ahocorasick ntop#2258"

77a79c2

This reverts commit e482e28.

pavlinux mentioned this issue Aug 5, 2024

Fix Memory leak in ahocorasick #2258 #2518

Closed

3 tasks

mmaatuq mentioned this issue Aug 19, 2024

ahocorasick: fix mem leaked AC_NODE_T object (#2258) #2522

Merged

3 tasks

IvanNardi closed this as completed in f03938a Aug 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Memory leak in ahocorasick #2258

Memory leak in ahocorasick #2258

IvanNardi commented Jan 12, 2024

mmanoj commented Jul 20, 2024

IvanNardi commented Jul 21, 2024

mmanoj commented Jul 22, 2024

mmanoj commented Jul 26, 2024

IvanNardi commented Jul 29, 2024

mmanoj commented Aug 1, 2024

Memory leak in ahocorasick #2258

Memory leak in ahocorasick #2258

Comments

IvanNardi commented Jan 12, 2024

mmanoj commented Jul 20, 2024

IvanNardi commented Jul 21, 2024

mmanoj commented Jul 22, 2024

mmanoj commented Jul 26, 2024

IvanNardi commented Jul 29, 2024

mmanoj commented Aug 1, 2024