Fetching from a s3 bucket without credentials is slower than with credentials

[expipiplus1]

Describe the bug

• Have a s3 bucket serving a store
• Have suitable credentials in ~/.aws/credentials
• Run nix copy --from 's3://blah' /nix/store/foo
• Observe that it succeeds/fails promptly
• Remove the credentials file
• Run a nix copy command again
• Observe that it takes several seconds to complete

Looking at the tcpdump trace for my minio bucket, I can see that minio itself responds promptly to the http request, hence it's nix which takes a long time to even query the bucket.

The first thing I thought of was that nix is taking a long time looking for credentials past ~/.aws/credentials in the chain, but nothing there looks too expensive...

strace reports that nix is spending quite some time trying to talk to 169.254.169.254. After a quick google

169.254.169.254 is used in Amazon EC2 and other cloud computing platforms to distribute metadata to cloud instances.

So I suppose this is quite fast on AWS, but elsewhere it's waiting for it to time out.

Expected behavior

Nix is speedy in both authenticated and anonymous cases.

nix-env --version output

nix-env (Nix) 2.3.11

[expipiplus1]

Perhaps when the s3 url contains a non aws endpoint it shouldn't look for credentials at 169.254.169.254.

[edolstra]

I'm surprised this works at all. Apparently aws-sdk-cpp has some fallback for making unauthenticated S3 requests (which it will only use after trying to get credentials). If you don't have credentials, you probably should use the HTTP binary cache interface (e.g. --store https://nix-cache.s3.us-east-1.amazonaws.com).

Probably we should just make this a fatal error.

[expipiplus1]

ah! that makes a lot of sense. Thanks for explaining @edolstra!

Or in the case of a minio bucket: https://minio.example.com/nix-cache

[stale]

I marked this as stale due to inactivity. → More info

[lovesegfault]

I think we can fix this now by adding a public setting to S3BinaryCacheStore which causes auth to not even be attempted.

[lovesegfault]

I have a fix for this here lovesegfault@1112add but I need #14330 to land first.