Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot find module '@aws-sdk/credential-provider-process' for S3 and SNS clients. #4264

Closed
3 tasks done
dszczepaniak-cksource opened this issue Dec 8, 2022 · 12 comments
Closed
3 tasks done
Assignees
Labels
bug This issue is a bug.

Comments

@dszczepaniak-cksource
Copy link

Checkboxes for prior research

Describe the bug

I try to upgrade @AWS-SDK packages from 3.188.0 to 3.226.0 and as up to 3.224.0 I encountered issue from #4075, running clients from 3.226.0 causes that a different message is emitted:

Error: Cannot find module '@aws-sdk/credential-provider-process'
Require stack:
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/credential-provider-ini/dist-cjs/resolveProcessCredentials.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/credential-provider-ini/dist-cjs/resolveProfileData.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/credential-provider-ini/dist-cjs/fromIni.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/credential-provider-ini/dist-cjs/index.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/credential-provider-node/dist-cjs/defaultProvider.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/credential-provider-node/dist-cjs/index.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/client-sts/dist-cjs/runtimeConfig.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/client-sts/dist-cjs/STSClient.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/client-sts/dist-cjs/STS.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/client-sts/dist-cjs/index.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/client-s3/dist-cjs/runtimeConfig.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/client-s3/dist-cjs/S3Client.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/client-s3/dist-cjs/S3.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/client-s3/dist-cjs/index.js

I've tried to install @aws-sdk/credential-providers but it did not resolve the issue. The issue can be resolved by installing @aws-sdk/credential-provider-process but the docs says clearly:

## Usage
You probably shouldn't, at least directly. Please use [@aws-sdk/credential-providers](https://www.npmjs.com/package/@aws-sdk/credential-providers) instead.

So I believe it is not an intentional flow.

SDK version number

@aws-sdk/package-name@version, ...

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

16.13.1 but the same happens on Node 18

Reproduction Steps

Just initialize a SQS client

const { SQS } = require( '@aws-sdk/client-sqs' );

this._sqs = new SQS( config );

Observed Behavior

Error: Cannot find module '@aws-sdk/credential-provider-process'
Require stack:
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/credential-provider-ini/dist-cjs/resolveProcessCredentials.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/credential-provider-ini/dist-cjs/resolveProfileData.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/credential-provider-ini/dist-cjs/fromIni.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/credential-provider-ini/dist-cjs/index.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/credential-provider-node/dist-cjs/defaultProvider.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/credential-provider-node/dist-cjs/index.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/client-sts/dist-cjs/runtimeConfig.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/client-sts/dist-cjs/STSClient.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/client-sts/dist-cjs/STS.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/client-sts/dist-cjs/index.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/client-s3/dist-cjs/runtimeConfig.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/client-s3/dist-cjs/S3Client.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/client-s3/dist-cjs/S3.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/client-s3/dist-cjs/index.js

Expected Behavior

As before 3.188.0 it was possible to use clients without additional credential packages I believe that including required libs is expected. If the lib cannot be included in the client packages, then adding @aws-sdk/credential-providers should resolve the issue.

Possible Solution

No response

Additional Information/Context

No response

@dszczepaniak-cksource dszczepaniak-cksource added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Dec 8, 2022
@dszczepaniak-cksource dszczepaniak-cksource changed the title Cannot find module '@aws-sdk/credential-provider-process' for S3 and SNS clients. Cannot find module '@aws-sdk/credential-provider-process' for S3 and SNS clients. Dec 8, 2022
@RanVaknin
Copy link
Contributor

RanVaknin commented Dec 8, 2022

Hi @dszczepaniak-cksource ,

I'm not sure why you are running into this. Your repro code is missing your config, so maybe you have an issue there?
With 3.226.0 Im able to create a client and call the SQS API without an issue:

const { SQS, ListQueuesCommand } = require( '@aws-sdk/client-sqs' );


const client = new SQS( {region: "us-east-1"} );

async function run(){
    try {
        const data = await client.send(new ListQueuesCommand({}))
        console.log(data)
    } catch (error) {
        console.log(error)
    }
}
run()

Output:

{
  '$metadata': {
    httpStatusCode: 200,
    requestId: 'REDACTED',
    extendedRequestId: undefined,
    cfId: undefined,
    attempts: 1,
    totalRetryDelay: 0
  },
  NextToken: undefined,
  QueueUrls: [ 'https://sqs.us-east-1.amazonaws.com/REDACTED/test-queue' ]
}

That cognito error does not seem related to what you are describing.
I would do the following:

  1. Delete your package-lock.json and run npm install again
  2. Run npm list or npm list -a and check if you have there some unused dependencies that you might want to get rid of.

Let me know that helps at all,

Thanks,
Ran~

@RanVaknin RanVaknin removed the needs-triage This issue or PR still needs to be triaged. label Dec 8, 2022
@RanVaknin RanVaknin self-assigned this Dec 8, 2022
@RanVaknin RanVaknin added the response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days. label Dec 8, 2022
@douddle
Copy link

douddle commented Dec 12, 2022

Hello,
I have similar issue with 3 different AWS SDK libs.
I'm using PNPM with no hoisting.
Some of dependencies are set as dev dependencies even if they used in the code.

../../node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/util-dynamodb/dist-types/convertToNative.d.ts:1:32 - error TS2307: Cannot find module '@aws-sdk/client-dynamodb' or its corresponding type declarations.
1 import { AttributeValue } from "@aws-sdk/client-dynamodb";
../../node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/middleware-endpoint-discovery/dist-types/configurations.d.ts:1:39 - error TS2307: Cannot find module '@aws-sdk/node-config-provider' or its corresponding type declarations.
1 import { LoadedConfigSelectors } from "@aws-sdk/node-config-provider";
../../node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/middleware-retry/dist-types/configurations.d.ts:1:39 - error TS2307: Cannot find module '@aws-sdk/node-config-provider' or its corresponding type declarations.
1 import { LoadedConfigSelectors } from "@aws-sdk/node-config-provider";
../../node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/config-resolver/dist-types/endpointsConfig/NodeUseFipsEndpointConfigOptions.d.ts:1:39 - error TS2307: Cannot find module '@aws-sdk/node-config-provider' or its corresponding type declarations.
1 import { LoadedConfigSelectors } from "@aws-sdk/node-config-provider";

@o-alexandrov
Copy link

Same issue with npm v9 (latest).
The only aws-sdk v3 package I'm using in the project is @aws-sdk/credential-provider-ini.

Referring to the package above, triggers an error:

Error: Cannot find module '@aws-sdk/credential-provider-process'
Require stack:
- .../node_modules/@aws-sdk/credential-provider-ini/dist-cjs/resolveProcessCredentials.js
- .../node_modules/@aws-sdk/credential-provider-ini/dist-cjs/resolveProfileData.js
- .../node_modules/@aws-sdk/credential-provider-ini/dist-cjs/fromIni.js

...and unrelated stack afterwards

@RanVaknin
Copy link
Contributor

@o-alexandrov , @douddle

Can you please write some repro steps? before I can reproduce what you are experiencing I can't do anything on my end.

Thanks,
Ran~

@RanVaknin RanVaknin added response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days. and removed response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days. labels Dec 12, 2022
@dszczepaniak-cksource
Copy link
Author

dszczepaniak-cksource commented Dec 13, 2022

I would do the following:
Delete your package-lock.json and run npm install again
Run npm list or npm list -a and check if you have there some unused dependencies that you might want to get rid of.

I got the described error both on the local machine as well as on the CI pipeline. I use pnpm 6.10.3 for installing packages. The package I develop uses the following prod dependencies:

"@aws-sdk/client-sns": "3.186.0",
"@aws-sdk/client-sqs": "3.186.0",
"@aws-sdk/client-s3": "3.186.0",
"@aws-sdk/node-http-handler": "3.186.0",

I have updated packages to:

"@aws-sdk/client-s3": "3.229.0",
    "@aws-sdk/client-sns": "3.229.0",
    "@aws-sdk/client-sqs": "3.229.0",
    "@aws-sdk/node-http-handler": "3.226.0",

and the error message has been changed:

Error: Cannot find module '@aws-sdk/util-retry'
Require stack:
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/middleware-retry/dist-cjs/AdaptiveRetryStrategy.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/middleware-retry/dist-cjs/index.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/client-s3/dist-cjs/S3Client.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/client-s3/dist-cjs/S3.js
- /project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/client-s3/dist-cjs/index.js
- /project/tests/integration/IntegrationTests.ts
- /project/node_modules/.pnpm/[email protected]/node_modules/mocha/lib/mocha.js
- /project/node_modules/.pnpm/[email protected]/node_modules/mocha/lib/cli/one-and-dones.js
- /project/node_modules/.pnpm/[email protected]/node_modules/mocha/lib/cli/options.js
- /project/node_modules/.pnpm/[email protected]/node_modules/mocha/bin/mocha
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/project/node_modules/.pnpm/@[email protected]/node_modules/@aws-sdk/middleware-retry/dist-cjs/AdaptiveRetryStrategy.js:4:22)
    at Module._compile (node:internal/modules/cjs/loader:1101:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
    at Object.require.extensions.<computed> [as .js] (/project/node_modules/.pnpm/[email protected][email protected]/node_modules/ts-node/src/index.ts:807:44)
    at Module.load (node:internal/modules/cjs/loader:

In the affected file, clients are initialized as follows:

import { S3 } from '@aws-sdk/client-s3';
import { SQS, CreateQueueResult } from '@aws-sdk/client-sqs';

// ...

console.log('test1')

const s3 = new S3(
			{
				endpoint: 'http://localhost:4570',
				region: 'eu-west-1',
				credentials: {
					accessKeyId: 'accesskey',
					secretAccessKey: 'secretkey'
				},
				forcePathStyle: true
			}
		);

console.log('test2')

const sqs = new SQS(
			{
				endpoint: 'http://localhost:4576',
				region: 'eu-west-1',
				credentials: {
					accessKeyId: 'accesskey',
					secretAccessKey: 'secretkey'
				}
			}
		);

console.log('test3')

When I put a few console logs, none of the placed console logs were shown so it seems it fails before initializing clients.

Changing imports to const ... = require() syntax made no effect. Generally, I believe there's a broader problem with linking dependencies or installing required dependencies of dependencies 🤔

@douddle
Copy link

douddle commented Dec 13, 2022

PNPM is proposing a workaround by the way: PNPM/packageExtensions
It should be done in the root package.json, not in a submodule.

@github-actions github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days. label Dec 14, 2022
@RanVaknin
Copy link
Contributor

I still can't seem to reproduce. I've tried both with pnpm and npm, require and imports.
In both I'm able to construct a client and even make API calls.

Not sure why you are running into this.. any other reproduction info you could provide? node version you are using? Vanilla JS or TS? any other info?

Thanks,
Ran

@RanVaknin RanVaknin added the response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days. label Dec 20, 2022
@dszczepaniak-cksource
Copy link
Author

The issue was reproducible both on Node 18 and 16, I use TS and CommonJS approach.

@RanVaknin
Copy link
Contributor

Hi @dszczepaniak-cksource ,

Can you please create a minimal repo for reproduction?

Thanks,
Ran

@RanVaknin RanVaknin added response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days. and removed response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days. labels Dec 20, 2022
@mhassan1
Copy link
Contributor

This should be resolved by #4302.

@github-actions github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days. label Dec 21, 2022
@dszczepaniak-cksource
Copy link
Author

dszczepaniak-cksource commented Dec 21, 2022

@mhassan1 I can confirm #4302 resolved the issue, thank you for your help! I close the issue as completed. Also @RanVaknin thanks for your involvement.

@github-actions
Copy link

github-actions bot commented Jan 5, 2023

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jan 5, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug This issue is a bug.
Projects
None yet
Development

No branches or pull requests

5 participants