3087 malloc.c: No such file or directory. when using memray attach.
#309
Comments
|
Also, I tried download lldb and tried. When I used that, it hangs instead of failing immediately like gdb |
|
oh also, I think this environment (it is our CI) may use dind (docker in docker). |
|
Hm. It's not a problem that it couldn't find But, there should be more output after that. When I follow your reproduction steps above, sometimes everything works fine, and the output shows: Other times it fails, with output exactly like you show. It looks like the problem is in gdb. The gdb script that we're running executes this command: Which the help built in to the gdb in Focal says is supported syntax! Despite what the GNU gdb (Ubuntu 9.2-0ubuntu1~20.04.1) 9.2 manual says, the space-separated list is not actually working, and the commands are only getting associated with the first breakpoint, and not the other 7. If I change that line to If you want to try that change out yourself, the relevant file is |
|
The lldb one is even weirder. The debugger is exiting without resuming the process that it was attached to. If you manually resume the process by sending it a SIGCONT signal, everything works just fine. That workaround seems horrific, and this feels like it must be an lldb bug (possibly unique to |
|
I've entered llvm/llvm-project#60408 for what appears to be an lldb bug explaining the hang you get with lldb. |
|
@godlygeek thanks for looking into the issue! Oddly, I found the issue never (or it probably rarely) occurs when I tested on my Mac OS (not sure if this can help your debugging though). Maybe GDB
Sounds good! Let me know when the fix is merged! I'd love to try it out! We are experimenting to introduce memray as a runtime memory profiling of ray (https://github.com/ray-project/ray/), and this issue seems to be the biggest blocker because I cannot run tests in the CI at all now because of this issue :( |
|
I've confirmed that the gdb behavior is a bug - and it's one that was fixed upstream a few years back, by da1df1d - but of course, Focal is a few years old now, too, so it doesn't have that fix. It seems that the workaround of using a range instead of a list of breakpoint numbers happens to work mostly by chance, but it does work, because of the different way in which ranges of breakpoint numbers are processed. |
|
Awesome! Will this fix be included in the next release? When do you guys plan to have one? |
|
I will also try verifying it with my environment. Can you guide me how I can download the latest master wheel? (or should I wait for next release?) |
|
Hey @rkooo567 - this will be included in the next release, which I hope will happen within the next few weeks. We don't have wheels of master, but since this change is in a text file, you can just patch it into the latest Memray wheel yourself: That asks If you feel like testing that, I'd love confirmation that it works for you, but if not, I anticipate a release with the fix should be coming pretty soon, once we finish a few other features that we're working on finalizing. |
|
(Or instead of modifying the wheel, you can of course just install the wheel and then use |
|
I will probably try this out next week! Looking forward to the next release |
|
The 1.7.0 release from last week includes this fix. Let us know if you're still seeing any issues with that version, @rkooo567! |
Is there an existing issue for this?
Current Behavior
When I ran
$(which memray) attach -o /tmp/pytest-of-root/pytest-5/test_memory_profiler0/profile/3009_memory_profiling.bin --verbose 3803inside a ubuntu container, it fails with the following error.Expected Behavior
It should attach successfully (It worked on my local Mac).
Steps To Reproduce
python 3.7.9
pip install rayRun
ray start --headpython test.pyray list actorsand get the pid. Note that ray actor is just a python process.$(which memray) attach -o /tmp/pytest-of-root/pytest-5/test_memory_profiler0/profile/3009_memory_profiling.bin --verbose
We use the container based off of ubuntu:focal
Some deps we downloaded;
Memray Version
1.6.0. But I also tried 1.5.0
Python Version
3.7
Operative System
Linux
Anything else?
No response
The text was updated successfully, but these errors were encountered: