Cargo gets lost in the root of a Linux filesystem

[ehuss]

Problem
cargo doc will mysteriously hang when run from the root of a Linux filesystem. The problem is that the fingerprinting code tries to find the file with the newest mtime by walking the entire filesystem. Unfortunately there are some paths that are infinite black holes.

An example is /sys/block. This tree has a complex set of cyclical symbolic links. Cargo follows a cycle until it reaches ELOOP (too many symbolic links), and then starts following another cycle. There is a permutation of cycles, and so cargo essentially gets lost forever.

I believe this also affects "old" style build scripts which also use LocalFingerprint::Precalculated.

Steps

1. Start a bash prompt with the rust:latest docker image.
2. In the root directory, run cargo init --name foo
3. Run cargo doc. This step will never finish.

Possible Solution(s)
Uncertain. A few ideas:

• Use walkdir for walking, which has symbolic link cycle detection.
• Check for special devices and ignore them.
• Detect running in root, and return an error.
• Set an upper bound on the number of files the walker will visit before it gives up.

As a side note, another problem with the implementation is that it collects the path of every file it visits in memory. This is done with the assumption that PathSource::list_files will return a reasonable number of files. However, the mtime computation code doesn't need a full list, but instead just needs to know which file has the newest mtime. Perhaps that code could be changed to take a callback instead of accumulating all the files in memory.

Notes

Output of cargo version:
cargo 1.54.0-nightly (e931e47 2021-05-24)

Linux 4.19.121

[jyn514]

@ehuss it seems strange to me that cargo looks at all files in the first place - I can look into adding --emit=dep-info to rustdoc if that would help.

[weihanglo]

Perhaps that code could be changed to take a callback instead of accumulating all the files in memory.

I prototyped an iterative version, but I don't feel comfortable about its complexity. 😞
Is there anything can do to make it simpler?

[LizardWizzard]

Faced the same issue during cargo build when I copy sources into root in docker. I see endless wall of warnings with the original repro.

Workaround for me was to create directory and copy sources there instead of root.

Will it be better to terminate with an error instead of being stuck ~forever with warnings?