Optional dependency of MongoDB cannot be resolved when running in AWS using serverless component

[huksley]

Describe the bug

When using MongoDB storage adapter, the way right now it is loaded by using require_optional are not compatible with serverless-nextjs packaging (and I believe not really compatible with AWS Lambda ways of working).

When it packaged as Lambda@Edge there is no package.json anymore.

It manifests as following error in login form: Try signing with a different account.

Also error generated in AWS CloudWatch (see below)

Steps to reproduce
Clone https://github.com/huksley/next-auth-example/tree/aws-serverless (notice aws-serverless branch)

Configure .env.local with at least following defined:

SECRET=changeme
GOOGLE_ID=changeme
GOOGLE_SECRET=changeme
DATABASE_URL=changeme

You can get MongoDB instance from MongoDB Cloud Atlas for free.

Run once with npx serverless to do initial deploy and get CloudFront domain.

Add NEXTAUTH_URL=https://xxxxx.cloudfront.net to .env.local afterwards and deploy again.

  app: 
    appUrl:         https://xxxxxx.cloudfront.net
    bucketName:     xxxx-xxxxx
    distributionId: XXXXX

  68s › app › done

Navigate to https://xxxxx.cloudfront.net and try to login with Google.

Expected behavior

Google Login works with MongoDB storage adapter.

Screenshots or error logs

2020-11-27T09:49:13.920Z	bafa8d7e-8a85-4636-99e4-572cf3de89fd	ERROR	[next-auth][error][oauth_callback_handler_error] TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
    at validateString (internal/validators.js:120:11)
    at Object.dirname (path.js:1128:5)
    at find_package_json (/var/task/pages/api/auth/[...nextauth].js:153496:23)
    at find_package_json_with_name (/var/task/pages/api/auth/[...nextauth].js:153513:20)
    at require_optional (/var/task/pages/api/auth/[...nextauth].js:153548:13)
    at Object.<anonymous> (/var/task/pages/api/auth/[...nextauth].js:177924:53)
    at Generator.next (<anonymous>)
    at asyncGeneratorStep (/var/task/pages/api/auth/[...nextauth].js:177836:103)
    at _next (/var/task/pages/api/auth/[...nextauth].js:177838:194)
    at processTicksAndRejections (internal/process/task_queues.js:97:5) {
  code: 'ERR_INVALID_ARG_TYPE'
} 
https://next-auth.js.org/errors#oauth_callback_handler_error

Additional context

• It is working fine locally by using npm run dev but NOT WITH npm run build && npm run start

• It is connected with the issue discussed here: Error: no optional dependency [mongodb] defined in peerOptionalDependencies in any package.json #552

• Not sure why next-auth needs to load mongodb as an optional dependency when it will clearly fail if config.type == "mongodb"

Hot fixing the bug
Patching next-auth fixes this problem:

diff -rwu node_modules/next-auth.orig/dist/adapters/typeorm/index.js node_modules/next-auth/dist/adapters/typeorm/index.js
--- node_modules/next-auth.orig/dist/adapters/typeorm/index.js	2020-11-27 12:10:53.000000000 +0200
+++ node_modules/next-auth/dist/adapters/typeorm/index.js	2020-11-27 12:12:19.000000000 +0200
@@ -115,7 +115,7 @@
 
       if (config.type === 'mongodb') {
         idKey = '_id';
-        var mongodb = (0, _require_optional.default)('mongodb');
+        var mongodb = require('mongodb');
         ObjectId = mongodb.ObjectId;
       }

Feedback
Documentation refers to searching through online documentation, code comments and issue history. The example project refers to next-auth-example.

• Found the documentation helpful
• Found documentation but was incomplete
• Could not find relevant documentation
• Found the example project helpful
• Did not find the example project helpful

[iaincollins]

This is a problem with way the app is bundled, if you include package.json in the list of files that get bundled and deployed and that should resolve the issue - I think there are some old issues (possibly closed now) that cover this.

I think not using optional require has the impact of introducing problems for other users, which IIRC is why we don't do it - dynamic imports - which are the "right" way to solve this " - are still not well supported by some bundlers (not really NextAuth.js problem exactly, except that people would then log bug reports about that). I think there is more info about that in the old tickets.

The error isn't great and it would be nice to have a better way of doing this that is lower friction for everyone.

Splitting the SQL Adaptor from the MongoDB Adaptor (but sharing common code, like the models) is also an option and might be the best move.

[iaincollins]

PS: Thanks for recording this and for the detail. Happy to leave this open until we have a more satisfactory resolution - either code change or documentation improvement, etc.

[huksley]

As far as I understand, this also affects production builds for NextJS application, so would this mean that we can not use MongoDB in production?

Would be great to have documentation updated for this.

I see there is an alternative for this, is to move this piece of code into the userland, as a callback? i.e.

if (ObjectId && !(id instanceof ObjectId))
  id = ObjectId(id);
}

I created a more sane patch for webpack, see this PR for more information.
https://github.com/huksley/next-auth-example/pull/2/files

[iaincollins]

As far as I understand, this also affects production builds for NextJS application, so would this mean that we can not use MongoDB in production?

No it's working fine for most people.

[stale]

Hi there! It looks like this issue hasn't had any activity for a while. It will be closed if no further activity occurs. If you think your issue is still relevant, feel free to comment on it to keep it open. (Read more at #912) Thanks!

[stale]

Hi there! It looks like this issue hasn't had any activity for a while. To keep things tidy, I am going to close this issue for now. If you think your issue is still relevant, just leave a comment and I will reopen it. (Read more at #912) Thanks!

[IgorDePaula]

This is not work for me, I tring bild on aws amplify. I'm not using mongodb, but this erro r persists. I'm using cognito for auth. I updated already, I'm using nextjs 11, and nxt-auth 3.27.3, I tried using this dependency installed on project, typeorm, but only got fails. On build locally I not got this error.

Error: Command failed with exit code 1: node_modules/.bin/next build
                                  warn  - No ESLint configuration detected. Run next lint to begin setup
                                  warn - Tailwind is not purging unused styles because no template paths have been provided.
                                  warn - If you have manually configured PurgeCSS outside of Tailwind or are deliberately not removing unused styles, set `purge: false` in your Tailwind config file to silence this warning.
                                  warn - https://tailwindcss.com/docs/controlling-file-size/#removing-unused-css
                                  Failed to compile.
                                  ModuleNotFoundError: Module not found: Error: Can't resolve 'mongodb' in '/codebuild/output/src095445125/src/bbfront/node_modules/typeorm/platform'
                                  > Build error occurred
                                  Error: > Build failed because of webpack errors
                                  at /codebuild/output/src095445125/src/bbfront/node_modules/next/dist/build/index.js:15:924
                                  at async Span.traceAsyncFn (/codebuild/output/src095445125/src/bbfront/node_modules/next/dist/telemetry/trace/trace.js:6:584)
                                  info  - Using webpack 5. Reason: Enabled by default https://nextjs.org/docs/messages/webpack5
                                  info  - Checking validity of types...
                                  info  - Creating an optimized production build...
                                  at makeError (/root/.//node_modules/execa/lib/error.js:60:11)
                                  at handlePromise (/root/.//node_modules/execa/index.js:118:26)
                                  at processTicksAndRejections (internal/process/task_queues.js:97:5)
                                  at async Builder.build (/root/.//node_modules/@sls-next/lambda-at-edge/dist/build.js:348:13) {
                                  shortMessage: 'Command failed with exit code 1: node_modules/.bin/next build',
                                  command: 'node_modules/.bin/next build',
                                  escapedCommand: '"node_modules/.bin/next" build',
                                  exitCode: 1,
                                  signal: undefined,
                                  signalDescription: undefined,
                                  stdout: 'info  - Using webpack 5. Reason: Enabled by default https://nextjs.org/docs/messages/webpack5\n' +
                                  'info  - Checking validity of types...\n' +
                                  'info  - Creating an optimized production build...',
                                  stderr: 'warn  - No ESLint configuration detected. Run next lint to begin setup\n' +
                                  '\n' +
                                  'warn - Tailwind is not purging unused styles because no template paths have been provided.\n' +
                                  'warn - If you have manually configured PurgeCSS outside of Tailwind or are deliberately not removing unused styles, set `purge: false` in your Tailwind config file to silence this warning.\n' +
                                  'warn - https://tailwindcss.com/docs/controlling-file-size/#removing-unused-css\n' +
                                  'Failed to compile.\n' +
                                  '\n' +
                                  "ModuleNotFoundError: Module not found: Error: Can't resolve 'mongodb' in '/codebuild/output/src095445125/src/bbfront/node_modules/typeorm/platform'\n" +
                                  '\n' +
                                  '\n' +
                                  '> Build error occurred\n' +
                                  'Error: > Build failed because of webpack errors\n' +
                                  '    at /codebuild/output/src095445125/src/bbfront/node_modules/next/dist/build/index.js:15:924\n' +
                                  '    at async Span.traceAsyncFn (/codebuild/output/src095445125/src/bbfront/node_modules/next/dist/telemetry/trace/trace.js:6:584)',
                                  failed: true,
                                  timedOut: false,
                                  isCanceled: false,
                                  killed: false
                                  }