Cargo stuck at "Updating crates.io"

[rv178]

OS: Arch Linux
Kernel: 5.18.16-arch1-1
Docker version: Docker version 20.10.17, build 100c70180f

I have been trying to get docker working for an app I was working on, but when I install packages, cargo gets stuck at "Updating crates.io" I tried some solutions, but none of them proved effective. Here's my Dockerfile:

FROM alpine:3.16

WORKDIR /opt/app
COPY . .

RUN apk add --no-cache \
		ca-certificates \
		gcc \
		curl

ENV RUSTUP_HOME=/usr/local/rustup \
	CARGO_HOME=/usr/local/cargo \
	PATH=/usr/local/cargo/bin:$PATH \
	RUST_VERSION=1.62.1

RUN set -eux; \
	apkArch="$(apk --print-arch)"; \
	case "$apkArch" in \
		x86_64) rustArch='x86_64-unknown-linux-musl'; rustupSha256='bdf022eb7cba403d0285bb62cbc47211f610caec24589a72af70e1e900663be9' ;; \
		aarch64) rustArch='aarch64-unknown-linux-musl'; rustupSha256='89ce657fe41e83186f5a6cdca4e0fd40edab4fd41b0f9161ac6241d49fbdbbbe' ;; \
		*) echo >&2 "unsupported architecture: $apkArch"; exit 1 ;; \
	esac; \
	url="https://static.rust-lang.org/rustup/archive/1.24.3/${rustArch}/rustup-init"; \
	wget "$url"; \
	echo "${rustupSha256} *rustup-init" | sha256sum -c -; \
	chmod +x rustup-init; \
	./rustup-init -y --no-modify-path --profile minimal --default-toolchain $RUST_VERSION --default-host ${rustArch}; \
	rm rustup-init; \
	chmod -R a+w $RUSTUP_HOME $CARGO_HOME; \
	rustup --version; \
	cargo --version; \
	rustc --version;

RUN	rustup target add wasm32-unknown-unknown; \
	cargo install wasm-bindgen-cli;

RUN	curl -LO https://github.com/tailwindlabs/tailwindcss/releases/latest/download/tailwindcss-linux-x64; \
	chmod +x tailwindcss-linux-x64; \
	mv tailwindcss-linux-x64 /usr/local/bin/tailwindcss; \
	curl -L https://github.com/thedodd/trunk/releases/download/v0.16.0/trunk-x86_64-unknown-linux-gnu.tar.gz | tar -xzf-; \
	mv trunk /usr/local/bin/trunk; \
	cd frontend;

cmd ["trunk", "serve"]

[SET001]

had the same issue.
It stuck on

  Installing api v0.1.0 (/usr/src/app)
    Updating crates.io index

for some time. And then throws errors

warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: static.crates.io)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: static.crates.io)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: static.crates.io)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: static.crates.io)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: static.crates.io)
error: failed to compile `api v0.1.0 (/usr/src/app)`, intermediate artifacts can be found at `/usr/src/app/target`

Caused by:
  failed to download from `https://crates.io/api/v1/crates/futures-core/0.3.21/download`

Caused by:
  [6] Couldn't resolve host name (Could not resolve host: static.crates.io)

Finally, I was managed to fix it by setting up DSN before run cargo install/build

RUN echo "nameserver 8.8.8.8" > /etc/resolv.conf && cargo install --path .

however, this looks like a temporary workaround and I think it should be better solved in some other way.

[SET001]

Also, according to this to solve this you can add this in your /etc/docker/daemon.json:

{
  "dns": ["8.8.8.8", "8.8.4.4"]
}

[rojvv]

I have the same problem.

[rojvv]

@SET001 your fixes aren't working.

[rojvv]

I'm on Arch Linux, too.

[rv178]

Yep, here too, not working.

[Luchanso]

Hey, guys, the same problem, it can be reproduce with this docker file:

FROM rust:latest

WORKDIR /usr/src/myapp
COPY . .

RUN cargo install --path .

CMD ["myapp"]

On a new project cargo new --bin myapp with any dependencies:
Cargo.toml:

[package]
name = "myapp"
version = "0.1.0"
edition = "2021"

[dependencies]
rand_core = "0.6.4"

without dependencies all working

[rojvv]

@Luchanso how much time did you give to it? I gave it like 30 minutes and then it worked.

[Luchanso]

@roj1512 about 15 minutes, but I'm not ready wait 30 minutes every time when I need new build
On my Mac (outside of docker) building take 1 - 2 min

I'll try later some older versions of base image for building (I'll write here if they work)

[rojvv]

@Luchanso The long wait was only once for me.

[Luchanso]

@roj1512 yes, because Docker have cache, but if you add new dependency, cache will invalidate and new build take again 30 min

[rojvv]

Exactly...

[Luchanso]

@roj1512 yes, I think so this happening because cargo trying download large index (117 mb) very slow

Workaround solution in single command

FROM rust:latest

# install crates.io index and cache it in the docker layer
RUN cargo new --bin build-index \
  && cd build-index \
  && cargo add rand_core \
  && cd .. \
  && rm -rf build-index

# ... other stuff ...

Explain:

1. Create empty fake project
2. Install any dependencies in this project
3. Cargo will build index
4. Remove fake project
5. Docker will cache this layer permanently, until you rewrite this command or adding some stuff before

Why is this a tradeoff? You need sometime reset cache layer, because index will outdate in time

Workaround solution 2

Or you can use rust nightly build and skip building index, but I don't verified this way and don't know pitfalls

And I'll reopen issue rust-lang/cargo#5721, because progress indicator not working in docker build

[Luchanso]

Interesting RFC about this problem https://rust-lang.github.io/rfcs/2789-sparse-index.html

[myssynglynx]

Having the same issue. I am able to add the dependencies using @Luchanso's workaround either in the Dockerfile or via terminal, but it throws the same error during the cargo build --release step. Does this have to do with internet connection? Or is it specific to particular processors? I am making sure the platform is adhering to the amd64 architecture, and I'm not on an M1 so it's not the arm7 issue.

Also, it builds perfectly fine when I leave out certain dependencies in my Cargo.toml. For the server I'm trying to containerize, it builds without an issue if I leave out the actix-web & sqlx dependencies-- including either throws these warning: spurious network errors, despite building & running perfectly well on my local machine outside of docker.

Note: I am using MacOs with a 2.3 GHz 8-Core Intel Core i9 processor, and 32 GB 2400 MHz DDR4 memory, and my internet connection is stable & fast (440 Mbps download, 20 Mpbs upload)

Since sqlx & actix-web are rather popular dependencies, I'm surprised this issue isn't more common-- unless it's a hardware or connection-related issue rather than a configuration-related issue. but looking through the documentation and solutions to similar issues, it looks like amd64 machines & a fast, stable internet connection shouldn't elicit this sort of issue.

[Luchanso]

@myssynglynx no, internet connection not affected with it, this index (~117 mb) storing on github and github limiting download speed with 32 - 128 kb/sec, so that 117 mb / 32 kb/sec ≈ 60 min for downloading index.

Rust team can solve this problem with two ways:

1. Add sparse index
2. Buy hosting and run self-hosted index without speed limit, but this can be cost is really much

But I think any $6 hosting with unlimited traffic and bandwidth 200 mb/sec handle with this task

[Luchanso]

@myssynglynx I found second workaround solution, you can copy cargo cache index from your computer into docker container

For MacOS:

cd my-project
cp -r ~/.cargo/registry ./registry

For Linux:

cd my-project
cp -r /usr/local/cargo/registry ./registry

Dockerfile:

FROM rust:alpine3.17

COPY registry /usr/local/cargo/registry
# .... other stuff ....

[myssynglynx]

@Luchanso This works. Thank you so much. It would be nice to not have to depend on anything from the developer's machine in order to get this running, but for now this will do.

[ghost]

Using sparse-registry seems to solve the problem.

https://blog.rust-lang.org/2022/06/22/sparse-registry-testing.html