Multi-threaded activating indices restarts server process

[aschegolskiy]

I did backup and resored database without inices. After that I used a program for multithreaded activation of indices - plume (https://github.com/NeoZX/plume).
Server hangs with error:
Fatal lock manager error: invalid lock id (173352), errno: 0
It was about 5-10 times with the same results. I've tested it on two different machines, HQBird 3.0.11.33622, Firebird 3.0.6.33328. Both x64 Win versions.
Shoud the activation of indices be used only using only one thread?
server_log.txt

[hvlad]

Could you provide reproducible test case ?
Or stack trace\full memory dump of hung process.

[aschegolskiy]

Of course. Could you please provide the instructions, how can I do it?

[hvlad]

How to create reproducible test case:

• provide application
• provide database
• provide steps to reproduce the issue

How to get full memory dump on Windows for given process:

• procdump -ma <process name or ID>

[hvlad]

For memory dumps, please, use Firebird 3.0.10 release or latest snapshot build.

[aschegolskiy]

I am not shure, I can predict the moment when FB restart occures, to exectute procdump. Is there a way to catch this?

[hvlad]

You told that process is hung, isn't it ?

If process crashed actually, you could read carefully description of procdump above or this article
Collecting User-Mode Dumps

[hvlad]

Any new info ?

[aschegolskiy]

I installed Firebird-3.0.10.33601_0_x64.exe, with pdb. Then run plume and the procdump64 (in another window) and got the 10Gb dump file.
E:>procdump64.exe -ma -e -t 10772
If the command is correct, where can I store the dump?

[aschegolskiy]

Compressed dump is 28 MB

[hvlad]

Dump 10GB - looks like you uses big page cache. Anyway, 28MB compressed is not big for download.

Is google drive ok for you ?

[aschegolskiy]

Sorry, the previous dump was get on a broken database. I'm not shure it can be useful.
I prepared two another dumps, one with db page cache = 625000 pages, other with default page cache but big temporary space (TempCacheLimit = 30311M).
Both dumps have 10Gb size, but 7z archives not as small as previous: 924Mb and 1,28Gb respectively. Google Drive is OK for me. Are the file sizes OK for you?
Restore indices with both options crashes server process, with such messages in firebird log:
Fatal lock manager error: invalid lock id (300832), errno: 0
After each crash, gfix finds a lot of orphan pages in the database:
Warning: Page 14319679 is an orphan

[aschegolskiy]

I've got the smaller dump. Archive only 42MB. How can I pass the link to Google Drive?

[hvlad]

All mentioned above file sizes are not a problem for me.
You may put link here or send it to me privately (hvlad at users sf net) if required.

[aschegolskiy]

Great! I sent the email with links.

[aschegolskiy]

Actually even default server configuration with default db cache pages gives mentioned above behaviour. I can provide dump if necessary.

[aschegolskiy]

@hvlad Test build looks working. There were no server crushes after 5-7 restores of indexes with different number of threads.

[aschegolskiy]

@hvlad Thanks a lot!