Can't import Firebase SDK

[rick-stevens]

Related

https://stackoverflow.com/questions/51522280/use-firebase-sdk-with-netlify-lambda-functions

Dependencies

{
  "firebase": "^5.8.2",
  "firebase-admin": "^7.0.0",
  "netlify-lambda": "^1.3.1",
}

The issue

I'd love to use Firebase (Firestore/Storage) in my Netlify function, though it won't even let me import any related SDK using netlify-lambda serve. I've tried both firebase and firebase-admin which respectively result in these delayed (5 - 10 seconds) errors:

Using firebase:

import firebase from 'firebase/app'
import 'firebase/firestore'

exports.handler = (event, context, callback) => callback(null, {
  statusCode: 200,
  body: 'OK'
})

Output:

netlify-lambda: Starting server
Lambda server is listening on 9000
Hash: e3817619e6c76f23fc78
Version: webpack 4.29.0
Time: 3210ms
Built at: 02/02/2019 11:55:21 PM
      Asset      Size  Chunks             Chunk Names
firebase.js  2.72 MiB       0  [emitted]  firebase
Entrypoint firebase = firebase.js
 [10] external "fs" 42 bytes {0} [built]
 [12] ../node_modules/firebase-admin/lib/utils/error.js 34.8 KiB {0} [built]
 [14] ../node_modules/firebase-admin/lib/utils/validator.js 7.25 KiB {0} [built]
 [56] ../node_modules/firebase-admin/lib/utils/deep-copy.js 2.6 KiB {0} [built]
[115] ../node_modules/firebase-admin/lib/auth/credential.js 15 KiB {0} [built]
[235] ../node_modules/firebase-admin/lib/index.js 1.36 KiB {0} [built]
[261] ../node_modules/firebase-admin/lib/auth/auth.js 20.5 KiB {0} [built]
[262] ../node_modules/@firebase/database/dist/index.esm.js 592 KiB {0} [built]
[264] ../node_modules/firebase-admin/lib/messaging/messaging.js 48.8 KiB {0} [built]
[265] ../node_modules/firebase-admin/lib/storage/storage.js 4.98 KiB {0} [built]
[354] ../node_modules/@google-cloud/firestore/build/src/index.js 52.9 KiB {0} [built]
[422] ./components/serviceAccountKey.json 2.32 KiB {0} [built]
[423] ./firebase.js 1.17 KiB {0} [built]
[424] ../node_modules/firebase-admin/lib/default-namespace.js 1.16 KiB {0} [built]
[425] ../node_modules/firebase-admin/lib/firebase-namespace.js 18.4 KiB {0} [built]
    + 780 hidden modules

WARNING in ../node_modules/grpc/src/grpc_extension.js 32:12-33
Critical dependency: the request of a dependency is an expression
 @ ../node_modules/grpc/index.js
 @ ../node_modules/google-gax/build/src/grpc.js
 @ ../node_modules/google-gax/build/src/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/firestore_client.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/index.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

WARNING in ../node_modules/grpc/node_modules/node-pre-gyp/lib/pre-binding.js 20:22-48
Critical dependency: the request of a dependency is an expression
 @ ../node_modules/grpc/src/grpc_extension.js
 @ ../node_modules/grpc/index.js
 @ ../node_modules/google-gax/build/src/grpc.js
 @ ../node_modules/google-gax/build/src/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/firestore_client.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/index.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

WARNING in ../node_modules/grpc/node_modules/node-pre-gyp/lib/util/versioning.js 17:20-67
Critical dependency: the request of a dependency is an expression
 @ ../node_modules/grpc/node_modules/node-pre-gyp/lib/pre-binding.js
 @ ../node_modules/grpc/src/grpc_extension.js
 @ ../node_modules/grpc/index.js
 @ ../node_modules/google-gax/build/src/grpc.js
 @ ../node_modules/google-gax/build/src/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/firestore_client.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/index.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

WARNING in ../node_modules/hash-stream-validation/index.js
Module not found: Error: Can't resolve 'fast-crc32c' in '/repos/test/node_modules/hash-stream-validation'
 @ ../node_modules/hash-stream-validation/index.js
 @ ../node_modules/@google-cloud/storage/build/src/file.js
 @ ../node_modules/@google-cloud/storage/build/src/index.js
 @ ../node_modules/firebase-admin/lib/storage/storage.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

WARNING in ../node_modules/bytebuffer/dist/bytebuffer-node.js
Module not found: Error: Can't resolve 'memcpy' in '/repos/test/node_modules/bytebuffer/dist'
 @ ../node_modules/bytebuffer/dist/bytebuffer-node.js
 @ ../node_modules/protobufjs/dist/protobuf.js
 @ ../node_modules/grpc/index.js
 @ ../node_modules/google-gax/build/src/grpc.js
 @ ../node_modules/google-gax/build/src/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/firestore_client.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/index.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

WARNING in ../node_modules/write-file-atomic/index.js
Module not found: Error: Can't resolve 'worker_threads' in '/repos/test/node_modules/write-file-atomic'
 @ ../node_modules/write-file-atomic/index.js
 @ ../node_modules/gcs-resumable-upload/node_modules/configstore/index.js
 @ ../node_modules/gcs-resumable-upload/build/src/index.js
 @ ../node_modules/@google-cloud/storage/build/src/file.js
 @ ../node_modules/@google-cloud/storage/build/src/index.js
 @ ../node_modules/firebase-admin/lib/storage/storage.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

Or using firebase-admin:

import * as admin from 'firebase-admin'

exports.handler = (event, context, callback) => callback(null, {
  statusCode: 200,
  body: 'OK'
})

Output:

netlify-lambda: Starting server
Lambda server is listening on 9000
Hash: e3817619e6c76f23fc78
Version: webpack 4.29.0
Time: 3102ms
Built at: 02/02/2019 11:53:21 PM
      Asset      Size  Chunks             Chunk Names
firebase.js  2.72 MiB       0  [emitted]  firebase
Entrypoint firebase = firebase.js
 [10] external "fs" 42 bytes {0} [built]
 [12] ../node_modules/firebase-admin/lib/utils/error.js 34.8 KiB {0} [built]
 [14] ../node_modules/firebase-admin/lib/utils/validator.js 7.25 KiB {0} [built]
 [56] ../node_modules/firebase-admin/lib/utils/deep-copy.js 2.6 KiB {0} [built]
[115] ../node_modules/firebase-admin/lib/auth/credential.js 15 KiB {0} [built]
[235] ../node_modules/firebase-admin/lib/index.js 1.36 KiB {0} [built]
[261] ../node_modules/firebase-admin/lib/auth/auth.js 20.5 KiB {0} [built]
[262] ../node_modules/@firebase/database/dist/index.esm.js 592 KiB {0} [built]
[264] ../node_modules/firebase-admin/lib/messaging/messaging.js 48.8 KiB {0} [built]
[265] ../node_modules/firebase-admin/lib/storage/storage.js 4.98 KiB {0} [built]
[354] ../node_modules/@google-cloud/firestore/build/src/index.js 52.9 KiB {0} [built]
[422] ./components/serviceAccountKey.json 2.32 KiB {0} [built]
[423] ./firebase.js 1.17 KiB {0} [built]
[424] ../node_modules/firebase-admin/lib/default-namespace.js 1.16 KiB {0} [built]
[425] ../node_modules/firebase-admin/lib/firebase-namespace.js 18.4 KiB {0} [built]
    + 780 hidden modules

WARNING in ../node_modules/grpc/src/grpc_extension.js 32:12-33
Critical dependency: the request of a dependency is an expression
 @ ../node_modules/grpc/index.js
 @ ../node_modules/google-gax/build/src/grpc.js
 @ ../node_modules/google-gax/build/src/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/firestore_client.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/index.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

WARNING in ../node_modules/grpc/node_modules/node-pre-gyp/lib/pre-binding.js 20:22-48
Critical dependency: the request of a dependency is an expression
 @ ../node_modules/grpc/src/grpc_extension.js
 @ ../node_modules/grpc/index.js
 @ ../node_modules/google-gax/build/src/grpc.js
 @ ../node_modules/google-gax/build/src/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/firestore_client.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/index.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

WARNING in ../node_modules/grpc/node_modules/node-pre-gyp/lib/util/versioning.js 17:20-67
Critical dependency: the request of a dependency is an expression
 @ ../node_modules/grpc/node_modules/node-pre-gyp/lib/pre-binding.js
 @ ../node_modules/grpc/src/grpc_extension.js
 @ ../node_modules/grpc/index.js
 @ ../node_modules/google-gax/build/src/grpc.js
 @ ../node_modules/google-gax/build/src/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/firestore_client.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/index.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

WARNING in ../node_modules/hash-stream-validation/index.js
Module not found: Error: Can't resolve 'fast-crc32c' in '/repos/test/node_modules/hash-stream-validation'
 @ ../node_modules/hash-stream-validation/index.js
 @ ../node_modules/@google-cloud/storage/build/src/file.js
 @ ../node_modules/@google-cloud/storage/build/src/index.js
 @ ../node_modules/firebase-admin/lib/storage/storage.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

WARNING in ../node_modules/bytebuffer/dist/bytebuffer-node.js
Module not found: Error: Can't resolve 'memcpy' in '/repos/test/node_modules/bytebuffer/dist'
 @ ../node_modules/bytebuffer/dist/bytebuffer-node.js
 @ ../node_modules/protobufjs/dist/protobuf.js
 @ ../node_modules/grpc/index.js
 @ ../node_modules/google-gax/build/src/grpc.js
 @ ../node_modules/google-gax/build/src/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/firestore_client.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/index.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

WARNING in ../node_modules/write-file-atomic/index.js
Module not found: Error: Can't resolve 'worker_threads' in '/repos/test/node_modules/write-file-atomic'
 @ ../node_modules/write-file-atomic/index.js
 @ ../node_modules/gcs-resumable-upload/node_modules/configstore/index.js
 @ ../node_modules/gcs-resumable-upload/build/src/index.js
 @ ../node_modules/@google-cloud/storage/build/src/file.js
 @ ../node_modules/@google-cloud/storage/build/src/index.js
 @ ../node_modules/firebase-admin/lib/storage/storage.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

[danew]

I believe this is because that package does't like being bundled with netlify-lambda's default webpack config.
I'm currently also having the same issue, I'm hoping to have some time to investigate it further tomorrow.

[danew]

It turns out that this issue is because the packages are being bundled with Webpack. Since the developer never intended this, since it's a "backend" module and wasn't intended to be bundled.

A fix for this would be to package your functions as a zip file like @DavidWells has done here https://github.com/DavidWells/function-zips/

[swyxio]

i dont have experience with firebase sdk but @danew makes sense. zip it up. i'd also welcome a PR for instructions on zipping.

we will be releasing more native support for this kind of thing in a couple months - please stay tuned.

[swyxio]

added instructions to README https://github.com/netlify/netlify-lambda/blob/master/README.md#alternative-to-netlify-lambda

[rick-stevens]

Thanks @danew / @sw-yx! This workaround does, however, cut out netlify-lambda, which was the reason for this issue. There's no need to zip the Firebase SDK if I'm not using the transpiler in the first place (as also shown in the blog post you linked from the README).

It makes developing harder as I'll have to write my own file watcher and proxy server to integrate with my app's framework. Looking forward to the native support!

[swyxio]

ok, i see. i love using firebase too and am keen to provide better DX for this. let me know if you figure out a way.

[danew]

@rick-stevens as an alternative you can still use netlify-lambda serve for local development by using a custom webpack config which includes webpack-node-externals. This will stop webpack from bundling your bundles these packages together and require them from your node_modules instead.

Then you just need to create a simple build script similar to the one I linked above for build time.

[chinchang]

Even "zip n ship" is not a solution. node_modules installed on user's machine won't necessarily work on Netlify function server. grpc has the same issues. I am guessing it gets compiled according to the host machine architecture.

[swyxio]

hmm. if you can make a small repro i can take it to our platform devs. i would v much like to support this usecase in production (even without netlify-lambda)

[swyxio]

@rick-stevens just noting a similar stackoverflow answer https://stackoverflow.com/questions/51522280/use-firebase-sdk-with-netlify-lambda-functions/51656271 it seems that instead of using the SDK you can just use their REST endpoint?

[rick-stevens]

Hi @sw-yx, I think this is the same Stack Overflow issue I noted originally. I already tried looking into their REST API, but while this is okay for some of their services, it doesn't provide one for Firebase Storage for instance (one I need in my case).

I've already switched to a full Firebase stack for my project (including Hosting and Functions), so I don't have a quick example anymore. But including firebase-admin (or other libraries like grpc as @chinchang stated) into an empty netlify-lambda project function, should be sufficient to reproduce!

[swyxio]

my bad, i skipped right past that SO answer. i see, its firebase storage thats needed. good enough for us to work with, thank you!

[chinchang]

I too switched to Firebase functions. And wanted to highlight that their functions deployment experience is much better than netlify as there is no need of packaging nor uploading node_modules. It just works. Maybe we can get a similar experience in Netlify also :)

hey @sw-yx , yea also the REST solution does not work for Cloud Firestore, only for the Realtime Database. An integrated netlify-lambda solution would be much appreciated.