Docstore related crash

[MantiRex]

Manticore Search version: Manticore 3.2.0 e526a01@191017 release
OS version: Debian 10
Description of the issue: searchd randomly dies when using docstore.
Steps to reproduce: it happens randomly. this is my searchd.log for one of the deaths

[Thu Nov 7 10:10:33.019 2019] [16663] FATAL: accept() failed: Too many open files (on linux see /etc/security/limits.conf, 'ulimit -n', max_open_files config option)
[Thu Nov 7 10:10:36.151 2019] [16663] FATAL: pthread_mutex_destroy() failed No such file or directory
------- FATAL: CRASH DUMP -------
[Thu Nov 7 10:10:33.014 2019] [16663]

--- crashed SphinxQL request dump ---
SELECT * FROM TL WHERE MATCH('@name matrix')
LIMIT 0, 75
--- request dump end ---
--- local index:
Manticore 3.2.0 e526a01@191017 release
Handling signal 11
-------------- backtrace begins here ---------------
Program compiled with 6.3.0
Configured with flags: Configured by CMake with these definitions: -DCMAKE_BUILD_TYPE=RelWithDebInfo -DDISTR_BUILD=stretch -DUSE_SSL=ON -DDL_UNIXODBC=1 -DUNIXODBC_LIB=libodbc.so.2 -DDL_EXPAT=1 -DEXPAT_LIB=libexpat.so.1 -DUSE_LIBICONV=1 -DDL_MYSQL=1 -DMYSQL_LIB=libmariadbclient.so.18 -DDL_PGSQL=1 -DPGSQL_LIB=libpq.so.5 -DLOCALDATADIR=/var/data -DFULL_SHARE_DIR=/usr/share/manticore -DICU_IS_SHARED=1 -DDL_ICU=1 -DICU_LIB=libicuuc.so.57 -DUSE_BISON=ON -DUSE_FLEX=ON -DUSE_SYSLOG=1 -DWITH_EXPAT=1 -DWITH_ICONV=ON -DWITH_MYSQL=1 -DWITH_ODBC=ON -DWITH_PGSQL=1 -DWITH_RE2=1 -DWITH_STEMMER=1 -DWITH_ZLIB=ON -DGALERA_SOVERSION=31 -DSYSCONFDIR=/etc/sphinxsearch
Host OS is Linux runner-fa6cab46-project-3858465-concurrent-0 4.19.23-coreos-r1 #1 SMP Mon Feb 25 23:40:01 -00 2019 x86_64 GNU/Linux
Stack bottom = 0x7fb1a6e95edf, thread stack size = 0x100000
Trying manual backtrace:
Something wrong with thread stack, manual backtrace may be incorrect (fp=0x55de2703f780)
Wrong stack limit or frame pointer, manual backtrace failed (fp=0x55de2703f780, stack=0x7fb1a6e90000, stacksize=0x100000)
Trying system backtrace:
begin of system symbols:
searchd(_Z12sphBacktraceib 0x9b)[0x55de271d60db]
searchd(_ZN16SphCrashLogger_c11HandleCrashEi 0x223)[0x55de27040da3]
/lib/x86_64-linux-gnu/libpthread.so.0( 0x12730)[0x7fb1f39ef730]
searchd(_ZN17DocstoreReaders_c17DeleteBySessionIdEl 0x45)[0x55de2738c905]
searchd( 0x1f9f06)[0x55de27082f06]
searchd(_Z18MinimizeAggrResultR12AggrResult_tRK9CSphQuerybRKN3sph9StringSetEP16CSphQueryProfilePK18CSphFilterSettingsb 0x36e)[0x55de2709ce7e]
[Thu Nov 7 10:10:36.322 2019] [16662] watchdog: main process 16663 exited cleanly (exit code 1), shutting down
[Thu Nov 7 10:10:36.324 2019] [16662] watchdog: got error 10, No child processes

[githubmanticore]

➤ Ilya Kuznetsov commented:

Can you try to raise open file limits and see if this crash still occurs?

[manticoresearch]

@MantiRex hi, we're waiting for your reply

[manticoresearch]

Closing as not actual as there was no reply for more than a month. Feel free to re-open