Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cargo stuck at "Updating crates.io" #111

Open
rv178 opened this issue Aug 11, 2022 · 19 comments
Open

Cargo stuck at "Updating crates.io" #111

rv178 opened this issue Aug 11, 2022 · 19 comments

Comments

@rv178
Copy link

rv178 commented Aug 11, 2022

OS: Arch Linux
Kernel: 5.18.16-arch1-1
Docker version: Docker version 20.10.17, build 100c70180f

I have been trying to get docker working for an app I was working on, but when I install packages, cargo gets stuck at "Updating crates.io" I tried some solutions, but none of them proved effective. Here's my Dockerfile:

FROM alpine:3.16

WORKDIR /opt/app
COPY . .

RUN apk add --no-cache \
		ca-certificates \
		gcc \
		curl

ENV RUSTUP_HOME=/usr/local/rustup \
	CARGO_HOME=/usr/local/cargo \
	PATH=/usr/local/cargo/bin:$PATH \
	RUST_VERSION=1.62.1

RUN set -eux; \
	apkArch="$(apk --print-arch)"; \
	case "$apkArch" in \
		x86_64) rustArch='x86_64-unknown-linux-musl'; rustupSha256='bdf022eb7cba403d0285bb62cbc47211f610caec24589a72af70e1e900663be9' ;; \
		aarch64) rustArch='aarch64-unknown-linux-musl'; rustupSha256='89ce657fe41e83186f5a6cdca4e0fd40edab4fd41b0f9161ac6241d49fbdbbbe' ;; \
		*) echo >&2 "unsupported architecture: $apkArch"; exit 1 ;; \
	esac; \
	url="https://static.rust-lang.org/rustup/archive/1.24.3/${rustArch}/rustup-init"; \
	wget "$url"; \
	echo "${rustupSha256} *rustup-init" | sha256sum -c -; \
	chmod +x rustup-init; \
	./rustup-init -y --no-modify-path --profile minimal --default-toolchain $RUST_VERSION --default-host ${rustArch}; \
	rm rustup-init; \
	chmod -R a+w $RUSTUP_HOME $CARGO_HOME; \
	rustup --version; \
	cargo --version; \
	rustc --version;

RUN	rustup target add wasm32-unknown-unknown; \
	cargo install wasm-bindgen-cli;

RUN	curl -LO https://github.com/tailwindlabs/tailwindcss/releases/latest/download/tailwindcss-linux-x64; \
	chmod +x tailwindcss-linux-x64; \
	mv tailwindcss-linux-x64 /usr/local/bin/tailwindcss; \
	curl -L https://github.com/thedodd/trunk/releases/download/v0.16.0/trunk-x86_64-unknown-linux-gnu.tar.gz | tar -xzf-; \
	mv trunk /usr/local/bin/trunk; \
	cd frontend;

cmd ["trunk", "serve"]
@SET001
Copy link

SET001 commented Aug 14, 2022

had the same issue.
It stuck on

  Installing api v0.1.0 (/usr/src/app)
    Updating crates.io index

for some time. And then throws errors

warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: static.crates.io)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: static.crates.io)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: static.crates.io)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: static.crates.io)
warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: static.crates.io)
error: failed to compile `api v0.1.0 (/usr/src/app)`, intermediate artifacts can be found at `/usr/src/app/target`

Caused by:
  failed to download from `https://crates.io/api/v1/crates/futures-core/0.3.21/download`

Caused by:
  [6] Couldn't resolve host name (Could not resolve host: static.crates.io)

Finally, I was managed to fix it by setting up DSN before run cargo install/build

RUN echo "nameserver 8.8.8.8" > /etc/resolv.conf && cargo install --path .

however, this looks like a temporary workaround and I think it should be better solved in some other way.

@SET001
Copy link

SET001 commented Aug 29, 2022

Also, according to this to solve this you can add this in your /etc/docker/daemon.json:

{
  "dns": ["8.8.8.8", "8.8.4.4"]
}

@rojvv
Copy link

rojvv commented Sep 1, 2022

I have the same problem.

@rojvv
Copy link

rojvv commented Sep 1, 2022

@SET001 your fixes aren't working.

@rojvv
Copy link

rojvv commented Sep 1, 2022

I'm on Arch Linux, too.

@rv178
Copy link
Author

rv178 commented Sep 1, 2022

Yep, here too, not working.

@Luchanso
Copy link

Luchanso commented Sep 25, 2022

Hey, guys, the same problem, it can be reproduce with this docker file:

FROM rust:latest

WORKDIR /usr/src/myapp
COPY . .

RUN cargo install --path .

CMD ["myapp"]

On a new project cargo new --bin myapp with any dependencies:
Cargo.toml:

[package]
name = "myapp"
version = "0.1.0"
edition = "2021"

[dependencies]
rand_core = "0.6.4"

without dependencies all working

@rojvv
Copy link

rojvv commented Sep 26, 2022

@Luchanso how much time did you give to it? I gave it like 30 minutes and then it worked.

@Luchanso
Copy link

Luchanso commented Sep 26, 2022

@roj1512 about 15 minutes, but I'm not ready wait 30 minutes every time when I need new build
On my Mac (outside of docker) building take 1 - 2 min

I'll try later some older versions of base image for building (I'll write here if they work)

@rojvv
Copy link

rojvv commented Sep 26, 2022

@Luchanso The long wait was only once for me.

@Luchanso
Copy link

@roj1512 yes, because Docker have cache, but if you add new dependency, cache will invalidate and new build take again 30 min

@rojvv
Copy link

rojvv commented Sep 26, 2022

Exactly...

@Luchanso
Copy link

Luchanso commented Sep 26, 2022

@roj1512 yes, I think so this happening because cargo trying download large index (117 mb) very slow

image

Workaround solution in single command

FROM rust:latest

# install crates.io index and cache it in the docker layer
RUN cargo new --bin build-index \
  && cd build-index \
  && cargo add rand_core \
  && cd .. \
  && rm -rf build-index

# ... other stuff ...

Explain:

  1. Create empty fake project
  2. Install any dependencies in this project
  3. Cargo will build index
  4. Remove fake project
  5. Docker will cache this layer permanently, until you rewrite this command or adding some stuff before

Why is this a tradeoff? You need sometime reset cache layer, because index will outdate in time

Workaround solution 2

Or you can use rust nightly build and skip building index, but I don't verified this way and don't know pitfalls

And I'll reopen issue rust-lang/cargo#5721, because progress indicator not working in docker build

@Luchanso
Copy link

Interesting RFC about this problem https://rust-lang.github.io/rfcs/2789-sparse-index.html

@myssynglynx
Copy link

myssynglynx commented Dec 26, 2022

Having the same issue. I am able to add the dependencies using @Luchanso's workaround either in the Dockerfile or via terminal, but it throws the same error during the cargo build --release step. Does this have to do with internet connection? Or is it specific to particular processors? I am making sure the platform is adhering to the amd64 architecture, and I'm not on an M1 so it's not the arm7 issue.

Also, it builds perfectly fine when I leave out certain dependencies in my Cargo.toml. For the server I'm trying to containerize, it builds without an issue if I leave out the actix-web & sqlx dependencies-- including either throws these warning: spurious network errors, despite building & running perfectly well on my local machine outside of docker.

Note: I am using MacOs with a 2.3 GHz 8-Core Intel Core i9 processor, and 32 GB 2400 MHz DDR4 memory, and my internet connection is stable & fast (440 Mbps download, 20 Mpbs upload)

Since sqlx & actix-web are rather popular dependencies, I'm surprised this issue isn't more common-- unless it's a hardware or connection-related issue rather than a configuration-related issue. but looking through the documentation and solutions to similar issues, it looks like amd64 machines & a fast, stable internet connection shouldn't elicit this sort of issue.

@Luchanso
Copy link

@myssynglynx no, internet connection not affected with it, this index (~117 mb) storing on github and github limiting download speed with 32 - 128 kb/sec, so that 117 mb / 32 kb/sec ≈ 60 min for downloading index.

Rust team can solve this problem with two ways:

  1. Add sparse index
  2. Buy hosting and run self-hosted index without speed limit, but this can be cost is really much

But I think any $6 hosting with unlimited traffic and bandwidth 200 mb/sec handle with this task

@Luchanso
Copy link

Luchanso commented Dec 27, 2022

@myssynglynx I found second workaround solution, you can copy cargo cache index from your computer into docker container

For MacOS:

cd my-project
cp -r ~/.cargo/registry ./registry

For Linux:

cd my-project
cp -r /usr/local/cargo/registry ./registry

Dockerfile:

FROM rust:alpine3.17

COPY registry /usr/local/cargo/registry
# .... other stuff ....

@myssynglynx
Copy link

@Luchanso This works. Thank you so much. It would be nice to not have to depend on anything from the developer's machine in order to get this running, but for now this will do.

@ghost
Copy link

ghost commented May 24, 2023

Using sparse-registry seems to solve the problem.

https://blog.rust-lang.org/2022/06/22/sparse-registry-testing.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants