segfult on Centos

[HassanHashemi]

We have changed hosting platform to Linux for a while, And one the ASP.NET apps gets killed after a few hours of running in production.

dmesg log contains:
[14721.297546] dotnet[3529]: segfault at 0 ip 00007f777d1abac5 sp 00007f722d7e94a0 error 4 in libcoreclr.so[7f777cf74000+288000]

The app is started by a typical dotnet myapp.dll command.

Repro is hard because it happens randomly on production server, So if there is any tip on how to collect more information i would appreciate that.

Note that it might be similar to dotnet/core#1245 but the issue is closed and comments are not tracked.

[jkotas]

Could you please find the stack trace of the crash from coredump? Crash like this can have many different root causes.

dotnet/coreclr#19803 is fixing a crash with same symptoms, but it is hard to tell for sure whether it will fix the crash you are hitting.

[jespersh]

Is https://github.com/aspnet/KestrelHttpServer/issues/2840 related to your issue?

[janvorli]

@jespersh the aspnet issue is unrelated.

[HassanHashemi]

Could you please find the stack trace of the crash from coredump?

The process is dead when it hits the fault, What's the best way to diagnose this one?

[jkotas]

ulimit -c unlimited will turn on core dump generation on process crashes. The core dump is generated in the current directory by default. man core has a ton of details about core dumps.

Once you have the core dump, you can open it in the debugger: lldb -c <core dump file name> or gdb -c <core dump file name> and get the stack trace using bt command to get the stacktrace. This stacktrace won't have symbols for managed methods.

SOS debugger extension can give you symbols for managed methods. It is available for lldb debugger only. A good tutorial on how to use it is at https://www.raydbg.com/2018/Debugging-Net-Core-on-Linux-with-LLDB/

[HassanHashemi]

Closing as i could not reproduce it under .net core 2.1 or 2.2 .