Retry Instance Profile Credential Retrieval

[mitchellhenke]

We infrequently see failures early in the provisioning process when fetching S3 data with the exception Aws::Sigv4::Errors::MissingCredentialsError. This PR adds a few retries to hopefully avoid the exception.

Stacktrace

Aws::Errors::MissingCredentialsError: unable to sign request without credentials set (Aws::Errors::MissingCredentialsError)
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/sign.rb:115:in `rescue in initialize'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/sign.rb:104:in `initialize'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/sign.rb:30:in `new'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/sign.rb:30:in `signer_for'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/sign.rb:42:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/transfer_encoding.rb:26:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/helpful_socket_errors.rb:12:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/s3_signer.rb:48:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/redirects.rb:20:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/retry_errors.rb:360:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/user_agent.rb:37:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/http_checksum.rb:19:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/endpoint_pattern.rb:30:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/checksum_algorithm.rb:136:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/request_compression.rb:94:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/expect_100_continue.rb:23:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/bucket_name_restrictions.rb:21:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/rest/handler.rb:10:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/recursion_detection.rb:18:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/endpoints.rb:41:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/endpoint_discovery.rb:84:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/seahorse/client/plugins/endpoint.rb:47:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/param_validator.rb:26:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/seahorse/client/plugins/raise_response_errors.rb:16:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/skip_whole_multipart_get_checksums.rb:18:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/sse_cpk.rb:24:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/dualstack.rb:21:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/plugins/accelerate.rb:43:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/checksum_algorithm.rb:111:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:16:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/idempotency_token.rb:19:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/param_converter.rb:26:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/seahorse/client/plugins/request_callback.rb:71:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/aws-sdk-core/plugins/response_paging.rb:12:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/seahorse/client/plugins/response_target.rb:24:in `call'
/bundle/ruby/3.3.0/gems/aws-sdk-core-3.179.0/lib/seahorse/client/request.rb:72:in `send_request'
/bundle/ruby/3.3.0/gems/aws-sdk-s3-1.132.0/lib/aws-sdk-s3/client.rb:6081:in `get_object'
/bundle/ruby/3.3.0/bundler/gems/identity-hostdata-9e2e0441cd93/lib/identity/hostdata/s3.rb:45:in `make_s3_get_object_request'
/bundle/ruby/3.3.0/bundler/gems/identity-hostdata-9e2e0441cd93/lib/identity/hostdata/s3.rb:25:in `download_file'
/lib/deploy/activate.rb:124:in `download_from_secrets_s3_unless_exists'
/lib/deploy/activate.rb:29:in `run'
/lib/tasks/activate.rake:6:in `block (2 levels) in <main>'