-
Notifications
You must be signed in to change notification settings - Fork 593
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bug(debug profile): segfault/EXC_BAD_ACCESS during backtrace capture #6205
Comments
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
maybe related: rust-lang/rust#47551 |
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
It seems that this issue happens randomly after some random changes, and will be fixed after some other random changes. |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
I have no idea how to fix it.. Maybe we can wait until it occurs more often and becomes a huge problem... 😇 |
Now it is no longer restricted to single binary ... It also happens with It feels like as long as we attempt to capture the backtrace, we have a chance to segfault. But it is deterministic given a certain commit and input query. commit: 484b9ab |
Sounds like a bug with LLVM… Also the bug sounds not like randomly happening. Something wrong with the rewind header. I’ll find an MCVE and report the bug to upstream. |
Tracked in rust-lang/rust#104388 |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
Don't use `RwError`; Use `ok_or_else` to avoid unnecesary error creation. To some extent mitigate #6205... Approved-By: BugenZhao
It seems using another linker (specifically, a newer |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
Summarize recent investigation:
TLDR: I still don't know why and how to fix it! 🤡 |
Probably not! I've compiled rustc from source and use |
Report a similar problem on a70e5b5 M1 Pro macOS. $ ./risedev p
...
libunwind: malformed DW_CFA_same_value DWARF unwind, reg too big
Segmentation fault: 11 RUST_BACKTRACE=1 RW_NODE=playground cargo run --bin risingwave --profile "${RISINGWAVE_BUILD_PROFILE}" The backtrace is similar to the context of this issue.
$ cc --version
Apple clang version 14.0.3 (clang-1403.0.22.14.1)
Target: arm64-apple-darwin22.4.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin |
I cannot reproduce it on my mac so I'm trying to upgrade XCode/macOS. Some noticeable minor changes:
|
BTW, capturing backtrace on mac release mode takes
|
|
|
On main branch (f0f96a8, likely earlier):
same segfault in |
I meet a new error message on e977dc9 🤔. It segfaults immediately when
Oh, the message is mentioned already #6205 (comment) |
I also met this issue for risedev in my local setup macOS, but it can't be reproduced if restart frontend manually as well. And the issue is gone when I rebased the main branch. 🥵 |
2342e8b SEGFAULTs immediately for me. It's caused by Really hope to investigate this problem ... if I have enough time What's worse is that adding some random dummy code doesn't fix it. 🥶 |
There's a update in the upstream issue rust-lang/rust#104388 (comment). Looks promising |
tikv/tikv#9957 Is this the same? 🤔 |
Describe the bug
When running playground on macOS using latest main (first bad commit db6691b), the following sql commands leads to a server crash with segfault/EXC_BAD_ACCESS.
It works as expected in cluster mode (
./risedev d
) rather than playground.To Reproduce
Expected behavior
Before that commit we were able to see the expected error:
Additional context
console warnings before segfault (also there on last good commit, may unrelated):
backtrace from lldb:
The text was updated successfully, but these errors were encountered: