Zombie processes cause rg --files to hang in /proc #916
Labels
bug
A bug.
Comments
|
Fascinating bug! In particular, with the invalid argument error appearing to be coming from a loop. If I look at the strace logs in Reading the man page for Following the strace logs, the problematic directory appears to be I've scoured the man pages of Given the lack of specification, the next best thing is to go out and look at how tools like |
|
Thanks for the excellent reproducible bug report!! |
|
Thank you for your detailed response. I can actually reproduce this by using just use std::env;
use std::io;
use std::fs;
use std::path::Path;
fn visit_dirs(dir: &Path) -> io::Result<()> {
for entry in fs::read_dir(dir)? {
println!("{:?}", entry);
if let Ok(entry) = entry {
let path = entry.path();
if path.is_dir() {
let _ = visit_dirs(&path);
}
}
}
Ok(())
}
fn main() {
let root = env::args().nth(1).unwrap();
let _ = visit_dirs(Path::new(&root));
}Which hangs in the same loop: |
|
Ah yeah thanks for doing that. That was going to be my next test, but ran out of time. Probably the next step is to scrutinize |
|
Oh, I just found this: BurntSushi/walkdir#98 |
|
Nice! Forgot about that. :-) |
|
So I did some more debugging and I believe I found the cause of this. When called on int readdir_r(DIR *dirp, struct dirent *entry, struct dirent **result);returns error code loop {
if readdir64_r(self.dirp.0, &mut ret.entry, &mut entry_ptr) != 0 {
return Some(Err(Error::last_os_error()))
}
if entry_ptr.is_null() {
return None
}
if ret.name_bytes() != b"." && ret.name_bytes() != b".." {
return Some(Ok(ret))
}
}(Code from the To handle this properly (without looping forever), one would probably have to check for What do you think? Is this something that should be reported upstream? |
|
@sharkdp Nice find! I think I do indeed agree with your conclusion that this should be fixed upstream. |
|
Reported here: rust-lang/rust#50619 |
|
This should be fixed in the next stable release (1.29.0). I've confirmed that it is fixed in the current beta of 1.29.0, but is still present in Rust 1.28.0. |
sharkdp
added a commit
to sharkdp/ripgrep
that referenced
this issue
Sep 17, 2018
This upgrades the minimum required version of Rust to 1.29 in order to fix BurntSushi#916 (Zombie processes cause `rg --files` to hang in `/proc`). See also: - Rust compiler bug ticket: rust-lang/rust#50619 - Rust compiler PR with the fix: rust-lang/rust#50630 closes BurntSushi#916
tavianator
added a commit
to tavianator/ripgrep
that referenced
this issue
Dec 28, 2022
The underlying Rust issue[1] was recently re-introduced and re-fixed. Rather than wait for a new Rust release, apply effectively the same fix as a workaround in the ignore crate itself. [1]: rust-lang/rust#50619
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What version of ripgrep are you using?
What operating system are you using ripgrep on?
Describe the bug.
Even if it does not seem like a good idea in general to search the whole file system without excluding special folders like
/proc(see also #311: "TL;DR is: don't search /proc"), I think that users will attempt to do so.We recently tracked down one specific reproducible state which causes the
ignorecrate to hang while traversing/proc(see sharkdp/fd#288 for more details). The same bug can be reproduced withrg --files.If this is a bug, what are the steps to reproduce the behavior?
Make sure that there are no zombie processes on your system (
ps -ef | rg '<defunct>')Run
rg --files > /dev/null 2>&1inside/proc. Everything should work fine (process exits after a few seconds).Create a zombie process on purpose
a. Copy the code from https://stackoverflow.com/a/25228579/704831 into a file called
zombie.cb. Compile it
gcc -o zombie zombie.cc. Run it:
./zombieRun
rg --files > /dev/null 2>&1inside/procagain. This seems to hang indefinitely.Get the PID of the zombie process via
ps -ef | rg '<defunct>'. Call it$PID.Run
rg --glob "\!$PID" --files > /dev/null 2>&1. This should work fine, indicating that the directory traversal hangs in/proc/$PID.If this is a bug, what is the actual behavior?
rg --fileshangs indefinitely inside the/procfolder.If this is a bug, what is the expected behavior?
rg --filestraverses the/procfolder in finite time.Note:
find, for example, does not seem to have this problem and finishes in finite time even in the presence of zombie processes.The text was updated successfully, but these errors were encountered: