Regression when ~/.rustup/downloads and/or tmp are symlinks

[ijackson]

Problem

rustcargo@zealot:/volatile/rustcargo/Rustup/Arti/arti$ ls -ald ~/.rustup/{tmp,downloads}{/,}
lrwxrwxrwx 1 rustcargo rustcargo      37 Jun  8  2021 /home/rustcargo/.rustup/downloads -> /volatile/rustcargo/rustup/downloads/
drwxrwxr-x 2 rustcargo rustcargo    4096 May  3 12:43 /home/rustcargo/.rustup/downloads/
lrwxrwxrwx 1 rustcargo rustcargo      31 Jun  8  2021 /home/rustcargo/.rustup/tmp -> /volatile/rustcargo/rustup/tmp/
drwxrwxr-x 2 rustcargo rustcargo 1249280 May  3 12:45 /home/rustcargo/.rustup/tmp/
rustcargo@zealot:/volatile/rustcargo/Rustup/Arti/arti$ rustup update
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: syncing channel updates for 'beta-x86_64-unknown-linux-gnu'
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
info: syncing channel updates for '1.31-x86_64-unknown-linux-gnu'
info: syncing channel updates for '1.37-x86_64-unknown-linux-gnu'
info: syncing channel updates for '1.40-x86_64-unknown-linux-gnu'
info: syncing channel updates for '1.51-x86_64-unknown-linux-gnu'
info: syncing channel updates for '1.53-x86_64-unknown-linux-gnu'
info: syncing channel updates for '1.54-x86_64-unknown-linux-gnu'
info: syncing channel updates for '1.63-x86_64-unknown-linux-gnu'
info: checking for self-update

   stable-x86_64-unknown-linux-gnu unchanged - rustc 1.69.0 (84c898d65 2023-04-16)
     beta-x86_64-unknown-linux-gnu unchanged - rustc 1.70.0-beta.2 (071f14baa 2023-04-30)
  nightly-x86_64-unknown-linux-gnu unchanged - rustc 1.71.0-nightly (a368898de 2023-05-02)
     1.31-x86_64-unknown-linux-gnu unchanged - rustc 1.31.1 (b6c32da9b 2018-12-18)
     1.37-x86_64-unknown-linux-gnu unchanged - rustc 1.37.0 (eae3437df 2019-08-13)
     1.40-x86_64-unknown-linux-gnu unchanged - rustc 1.40.0 (73528e339 2019-12-16)
     1.51-x86_64-unknown-linux-gnu unchanged - rustc 1.51.0 (2fd73fabe 2021-03-23)
     1.53-x86_64-unknown-linux-gnu unchanged - rustc 1.53.0 (53cb7b09b 2021-06-17)
     1.54-x86_64-unknown-linux-gnu unchanged - rustc 1.54.0 (a178d0322 2021-07-26)
     1.63-x86_64-unknown-linux-gnu unchanged - rustc 1.63.0 (4b91a6ea7 2022-08-08)

info: cleaning up downloads & tmp directories
thread 'main' panicked at 'Unable to clean up /home/rustcargo/.rustup/downloads: Os { code: 40, kind: FilesystemLoop, message: "Too many levels of symbolic links" }', src/utils/utils.rs:650:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Steps

1. Make ~/.rustup/tmp and ~/.rustup/downloads symlinks to like-named directories elsewhere
2. Run rustup

Possible Solution(s)

No response

Notes

I assume something has changed in rustup's handling of these directories. This used to work. I could probably bisect it if it's not obvious.

Rustup version

rustcargo@zealot:/volatile/rustcargo/Rustup/Arti/arti$ rustup --version
rustup 1.26.0 (5af9b9484 2023-04-05)
info: This is the version for the rustup toolchain manager, not the rustc compiler.
info: The currently active `rustc` version is `rustc 1.70.0-beta.2 (071f14baa 2023-04-30)`
rustcargo@zealot:/volatile/rustcargo/Rustup/Arti/arti$ 

Installed toolchains

rustcargo@zealot:/volatile/rustcargo/Rustup/Arti/arti$ rustup show
Default host: x86_64-unknown-linux-gnu
rustup home:  /home/rustcargo/.rustup

installed toolchains
--------------------

stable-x86_64-unknown-linux-gnu
beta-x86_64-unknown-linux-gnu (default)
nightly-2018-12-19-x86_64-unknown-linux-gnu
nightly-2022-06-24-x86_64-unknown-linux-gnu
nightly-2022-11-12-x86_64-unknown-linux-gnu
nightly-x86_64-unknown-linux-gnu
1.31-x86_64-unknown-linux-gnu
1.37-x86_64-unknown-linux-gnu
1.40-x86_64-unknown-linux-gnu
1.51-x86_64-unknown-linux-gnu
1.53-x86_64-unknown-linux-gnu
1.54-x86_64-unknown-linux-gnu
1.63-x86_64-unknown-linux-gnu
local
stage1
1.63.0-x86_64-unknown-linux-gnu

installed targets for active toolchain
--------------------------------------

wasm32-unknown-unknown
x86_64-unknown-linux-gnu
x86_64-unknown-linux-musl

active toolchain
----------------

beta-x86_64-unknown-linux-gnu (default)
rustc 1.70.0-beta.2 (071f14baa 2023-04-30)

rustcargo@zealot:/volatile/rustcargo/Rustup/Arti/arti$ 

[rbtcollins]

I think this might be fixed?

[ijackson]

Is there an easy workaround for updating to the fixed version, despite the bug, so I can test it?

[rbtcollins]

use git to clone rustup, then cargo build and finally cargo run -- self update.

Or download a prebuilt rustup and do likewise.

[ijackson]

I git cloned https://github.com/rust-lang/rustup (and got bd08d35) but:

error: error: Found argument 'self' which wasn't expected, or isn't valid in this context

USAGE:
    rustup-init [OPTIONS]

For more information try --help

I tried cargo run -- --help but the output doesn't look anything like the output of rustup --help. I think this is a different program? The rest of the cargo workspace wasn't very illuminating...

[rbtcollins]

oh sorry,
RUSTUP_FORCE_ARG0=rustup cargo run -- self update should do it.

[ijackson]

That didn't work because cargo on my path is, of course, rustup. So setting RUSTUP_FORCE_ARG0=rustup sabotages cargo run. So I did this:

rustcargo@zealot:/volatile/rustcargo/d/rustup$ RUSTUP_FORCE_ARG0=rustup target/debug/rustup-init self update
info: checking for self-update
  rustup unchanged - 1.26.0

rustcargo@zealot:/volatile/rustcargo/d/rustup$

I ran it with various --help options to see if there was a way to force it to run but wihtout success.

Note that my initial bug report shows a repro with rustup 1.26.0, which seems to be the latest tag...

[ijackson]

This is still happening to me.

[rami3l]

Looks like https://stackoverflow.com/a/58492502 which points to incorrect self-referential symlinks being created...

This is in fact a failure in handling the error from our dependency remove_dir_all:

For the reference, remove_dir_all seems to consistently drop an error when the dir is a symlink (which might not be the expected behavior on our side anyway). However on Windows it generates a custom error, whereas on Unix it returns ELOOP:

    fn open_dir(p: &Path) -> Result<File> {
        let mut options = OpenOptions::new();
        options.read(true);
        options.custom_flags(FILE_FLAG_BACKUP_SEMANTICS | FILE_FLAG_OPEN_REPARSE_POINT);
        let maybe_dir = options.open(p)?;
        if maybe_dir.metadata()?.is_symlink() {
            return Err(io::Error::new(
                io::ErrorKind::Other,
                "Path is a directory link, not directory",
            ));
        }
        Ok(maybe_dir)
    }

https://github.com/XAMPPRocky/remove_dir_all/blob/c3927b9d5c0194a0b5115bcff4a3fffbc8b97f1f/src/_impl/win.rs#L55-L67

Originally posted by @rami3l in #3737 (comment)

@rbtcollins I wonder whether this behavior should be improved on our side or from the upstream...

[rami3l]

I expect to address this issue right after #3737. To complete the context, I'll repost @rbtcollins' reply on the expected semantics below:

• when making the directory, if it exists already and is a symlink to a dir, or is a dir, use it, otherwise error
• when deleting the directory, if it is a symlink, unlink it, or if a directory, recursively delete with link-following-safety, otherwise error.

I think remove_dir_all + the fs_at crates should permit solving this quite nicely.

#3737 (comment)

[rami3l]

@ijackson A new beta release is available, would you mind trying it out?