Module does not self-register lmdb-store.node #67

rizrmd · 2021-06-30T10:26:08Z

I am using lmdb-store with chokidar and restart node proces when my source file changes. At first lmdb-store runs fine, and then I got this error when live-reload occured:

Error: Module did not self-register: '/Users/riz/Documents/base/node_modules/lmdb/build/Release/lmdb-store.node'.
 at Object.Module._extensions..node (node:internal/modules/cjs/loader:1154:18)
    at Module.load (node:internal/modules/cjs/loader:975:32)
    at Function.Module._load (node:internal/modules/cjs/loader:816:12)
    at Module.require (node:internal/modules/cjs/loader:999:19)
    at require (node:internal/modules/cjs/helpers:93:18)
    at load (/Users/riz/Documents/base/node_modules/node-gyp-build/index.js:21:10)
    at Object.<anonymous> (/Users/riz/Documents/base/node_modules/lmdb/index.js:7:49)
    at Module._compile (node:internal/modules/cjs/loader:1095:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1124:10)
    at Module.load (node:internal/modules/cjs/loader:975:32)
Emitted 'error' event on Thread instance at:
    at Thread.[kOnErrorMessage] (node:internal/worker:298:10)
    at Thread.[kOnMessage] (node:internal/worker:309:37)
    at MessagePort.<anonymous> (node:internal/worker:210:57)
    at MessagePort.[nodejs.internal.kHybridDispatch] (node:internal/event_target:461:20)
    at MessagePort.exports.emitMessage (node:internal/per_context/messageport:23:28) {
  code: 'ERR_DLOPEN_FAILED'
}

I am using macbook m1 fwiw, is there anything I can do to solve this ?

The text was updated successfully, but these errors were encountered:

kriszyp · 2021-06-30T19:22:38Z

Do you know how you are restarting the node process? Is it completely killing the process and restarting it? I think chokidar is just responsible for file watching, not restarting.

I did update the code to use the latest recommended node add-on API and published as part of v1.6.0-beta.1 if you want to try that and see if it makes a difference?

rizrmd · 2021-07-01T03:02:11Z

I am using worker_thread.terminate() method to kill the process. Ok, will try tomorrow maybe. I just switched using sqlite for meantime...

rizrmd · 2021-07-01T03:05:27Z

So the way I run lmdb-store is like this:

Main process
- db (worker-thread)
  
  lmdb-store running inside db worker thread

Main process will restart db worker thread on file change

kriszyp · 2021-07-01T04:22:13Z

I have not been able to reproduce this while terminating and starting new threads. Does it make any difference if the main process has loaded/required lmdb-store (I think there can be a different cleanup procedure if the module is still in use)? Also what version of node is this?

rizrmd · 2021-07-02T06:47:11Z

This is latest node v.16.4.0, Yeah now it's intermitently terminated. On a bad day, almost every reload is terminated.

rizrmd · 2021-07-02T06:49:17Z

FWIW, I use esbuild to build my ts source file, and then run resulting build file using worker_threads. So I execute different source code for each run (esbuild generate new file whenever file changes)

rizrmd · 2021-07-02T06:52:24Z

I have not been able to reproduce this while terminating and starting new threads. Does it make any difference if the main process has loaded/required lmdb-store (I think there can be a different cleanup procedure if the module is still in use)? Also what version of node is this?

When I use node-lmdb on main, it's running smooth. But, I really need to run node-lmdb on worker_thread because we have several thread to process different database, and can kill it whenever we need without interupting main process. Also we use worker_thread because we can just postMessage when we thread needs to coordinate each other.

kriszyp · 2021-07-02T14:07:30Z

When I use node-lmdb on main, it's running smooth. But, I really need to run node-lmdb on worker_thread

What I meant was if you could just load lmdb-store in the main thread and still use lmdb-store in the worker threads as you are currently doing. It sounds like you have probably already tried that though.

I have tried reproducing this on multiple machines now, including on Windows, Mac, and Linux without any issues or success in reproducing this. I suppose it is possible it is related to M1 (I don't have access to an M1 Mac right now). I have previously observed an issue (#48 ) with M1's apparently having different timing with threads resuming after a mutex unlocks, and node does use mutexes for add-on registration (and threads and non-determinism also are at play). However, that seems like it would affect (all?) other native add-ons running on Node + M1, which seems unlikely to me (surely someone else would have noticed). Is it possible for you to test if this does affect any other native add-ons? lmdb-store should transitively depend-on/install msgpackr-extract, maybe you see if this error occurs if your worker threads load/require the msgpackr-extract package (or other native add-on packages)?

Or if you have any test case that I can run, that would certainly be welcome also.

Anyway, your setup is very legitimate way of running lmdb-store (we basically do the same exact thing in our medical software, except we use processes instead of threads), so I definitely would like to get it fixed.

rizrmd · 2021-07-03T00:13:14Z

When I use node-lmdb on main, it's running smooth. But, I really need to run node-lmdb on worker_thread

What I meant was if you could just load lmdb-store in the main thread and still use lmdb-store in the worker threads as you are currently doing. It sounds like you have probably already tried that though.

I have tried reproducing this on multiple machines now, including on Windows, Mac, and Linux without any issues or success in reproducing this. I suppose it is possible it is related to M1 (I don't have access to an M1 Mac right now). I have previously observed an issue (#48 ) with M1's apparently having different timing with threads resuming after a mutex unlocks, and node does use mutexes for add-on registration (and threads and non-determinism also are at play). However, that seems like it would affect (all?) other native add-ons running on Node + M1, which seems unlikely to me (surely someone else would have noticed). Is it possible for you to test if this does affect any other native add-ons? lmdb-store should transitively depend-on/install msgpackr-extract, maybe you see if this error occurs if your worker threads load/require the msgpackr-extract package (or other native add-on packages)?

Or if you have any test case that I can run, that would certainly be welcome also.

Anyway, your setup is very legitimate way of running lmdb-store (we basically do the same exact thing in our medical software, except we use processes instead of threads), so I definitely would like to get it fixed.

Well. yeah, M1 support is definitely flaky on npm, so my choices is limited and the best I can do is report like this. It would take me a while to set up a test case. My hand is currently full, my team already past the deadline -- sorry!

I will report back to you when I can touch this issue again.

hgeist2 · 2021-10-27T15:57:58Z

I have this problem, too, when using parcel 2.0.0+

OS: 5.4.0-89-generic #100-Ubuntu SMP, Ubuntu 20.04
Node: 14.15.4
Npm: 6.14.11

It was not resolved by:

npm-force-resolutions
npm i --force [email protected] + npm shrinkwrap
updating to [email protected] and [email protected]

~~This issue could be related: parcel-bundler/parcel#6597~~ -> Other issue with other exception

kriszyp · 2021-10-27T16:03:11Z

@hgeist2 Are you getting a "symbol not found" error or a "Module does not self-register" error?

hgeist2 · 2021-10-27T16:09:05Z

@kriszyp: Its the "Module does not self-regitser" one

Error: Module did not self-register: '/home/.../node_modules/lmdb-store/build/Release/lmdb-store.node'.

kriszyp · 2021-10-27T16:16:43Z

Do you have any ideas if there is any other parcel configurations that are likely to help reproduce this? I have a ubuntu box here I can install parcel on and try, but I'm guessing that parcel folks at least tested the basic parcel+ubuntu combination before releasing. Anyway, I would love to be able to track down this mysterious bug though.

hgeist2 · 2021-10-27T16:25:14Z

I will share with you as much as i can.

I read somewhere that parcel-cache uses lmdb-store.

The repo also uses git submodules.

Full Stack trace:

Error: Module did not self-register: '.../node_modules/lmdb-store/build/Release/lmdb-store.node'.
at Object.Module._extensions..node (internal/modules/cjs/loader.js:1122:18)
at Module.load (internal/modules/cjs/loader.js:928:32)
at Function.Module._load (internal/modules/cjs/loader.js:769:14)
at Module.require (internal/modules/cjs/loader.js:952:19)
at require (.../node_modules/v8-compile-cache/v8-compile-cache.js:159:20)
at load (.../node_modules/node-gyp-build/index.js:21:10)
at Object. (.../node_modules/lmdb-store/index.js:7:49)
at Module._compile (.../node_modules/v8-compile-cache/v8-compile-cache.js:192:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
at Module.load (internal/modules/cjs/loader.js:928:32)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] start: npm run clean && parcel src/index.pug --no-autoinstall
npm ERR! Exit status 1

package.json

{
  "name": "xyz",
  "version": "1.0.0",
  "description": "",
  "scripts": {
    "debug_build": "NODE_ENV=development parcel build src/index.pug --no-optimize",
    "build": "parcel build src/index.pug --no-source-maps",
    "clean": "rimraf .parcel-cache dist",
    "deploy": "deploy-aws-s3-cloudfront --non-interactive --delete --source ./dist --bucket $npm_package_config_bucket_name --distribution $npm_package_config_distribution_id --destination /$npm_package_config_public_url --invalidation-path /$npm_package_config_public_url/* --cache-control index.html:no-cache",
    "format": "prettier \"src/**/*.+(js|jsx|json|scss|md)\"",
    "format:write": "npm run format -- --write",
    "lint": "npm run lint_js && npm run lint_scss",
    "lint:fix": "npm run lint_js:fix && npm run lint_scss:fix",
    "lint_js": "eslint --ext .js,.jsx,.ts,.tsx src",
    "lint_js:fix": "npm run lint_js -- --fix",
    "lint_scss": "stylelint 'src/**/*.scss'",
    "lint_scss:fix": "npm run lint_scss -- --fix",
    "predeploy": "npm run build",
    "start": "npm run clean && parcel src/index.pug --no-autoinstall",
    "start_debug": "serve dist/",
    "test": "jest --roots src",
    "test_u": "npm run test -- -u",
    "prepare": "husky install"
  },
  "config": {
    "bucket_name": "XYZ",
    "distribution_id": "XYZ",
    "public_url": "example.com"
  },
  "jest": {
    "setupFilesAfterEnv": [
      "./contrib/jest.setup.js"
    ],
    "transform": {
      "^.+\\.jsx?$": "babel-jest"
    },
    "moduleNameMapper": {
      "~/(.*)": "<rootDir>/src/$1",
      "(url)\\:": "jest-transform-stub",
      "\\.(s?css)$": "identity-obj-proxy",
      "(contrib)/?(.*)$": "<rootDir>/$1/$2"
    }
  },
  "alias": {
    "contrib": "./contrib"
  },
  "author": "",
  "license": "",
  "dependencies": {
    "@aws-amplify/auth": "^4.3.9",
    "@google/model-viewer": "1.6.0",
    "@jest/globals": "^26.6.2",
    "@redux-saga/core": "^1.1.3",
    "@testing-library/jest-dom": "^5.12.0",
    "@testing-library/react": "^11.2.7",
    "@tweenjs/tween.js": "^18.6.4",
    "autoprefixer": "^10.2.5",
    "axios": "0.21.1",
    "browserify": "^17.0.0",
    "classnames": "2.3.1",
    "deploy-aws-s3-cloudfront": "3.3.0",
    "hammerjs": "^2.0.8",
    "html2canvas": "^1.1.2",
    "is-ios": "^2.0.1",
    "jest-snapshot": "^26.6.2",
    "lmdb-store": "^1.6.3",
    "material-icons": "^0.6.1",
    "parcel-plugin-asset-copier": "^1.1.0",
    "parse-color": "^1.0.0",
    "react": "17.0.2",
    "react-device-detect": "^2.0.0",
    "react-dom": "17.0.2",
    "react-draggable": "^4.4.3",
    "react-iframe": "1.8.0",
    "react-indexed-db": "^1.0.12",
    "react-nl2br": "1.0.2",
    "react-phone-number-input": "^3.1.21",
    "react-redux": "7.2.3",
    "react-router-dom": "^5.2.0",
    "react-svg": "13.0.2",
    "react-swipeable-views": "0.13.9",
    "redux": "4.1.0",
    "redux-persist": "^6.0.0",
    "redux-saga": "1.1.3",
    "redux-saga-thunk": "0.7.3",
    "serve": "^12.0.1",
    "three": "^0.133.1",
    "uuid": "8.3.2",
    "video-react": "0.14.1"
  },
  "devDependencies": {
    "@babel/eslint-parser": "^7.15.8",
    "@babel/plugin-proposal-decorators": "^7.15.8",
    "@babel/plugin-transform-runtime": "7.13.15",
    "@babel/preset-env": "^7.15.8",
    "@babel/preset-react": "^7.14.5",
    "@parcel/babel-preset-env": "^2.0.0",
    "@parcel/transformer-image": "^2.0.0",
    "@parcel/transformer-pug": "^2.0.0",
    "@parcel/transformer-sass": "^2.0.0",
    "@wojtekmaj/enzyme-adapter-react-17": "0.6.1",
    "babel-jest": "26.6.3",
    "babel-plugin-transform-remove-console": "^6.9.4",
    "enzyme": "3.11.0",
    "eslint": "7.24.0",
    "eslint-config-prettier": "8.2.0",
    "eslint-config-semistandard": "15.0.1",
    "eslint-config-standard": "16.0.2",
    "eslint-import-resolver-typescript": "2.4.0",
    "eslint-plugin-import": "2.22.1",
    "eslint-plugin-node": "11.1.0",
    "eslint-plugin-prettier": "3.3.1",
    "eslint-plugin-promise": "5.1.0",
    "eslint-plugin-react": "7.23.2",
    "eslint-plugin-react-hooks": "4.2.0",
    "history": "4.6.3",
    "husky": "6.0.0",
    "identity-obj-proxy": "3.0.0",
    "jest": "26.6.3",
    "jest-transform-stub": "2.0.0",
    "parcel": "^2.0.0-nightly.893",
    "parcel-plugin-static-files-copy": "^2.6.0",
    "prettier": "2.2.1",
    "prop-types": "^15.7.2",
    "rimraf": "3.0.2",
    "stylelint": "13.12.0",
    "stylelint-config-sass-guidelines": "8.0.0",
    "stylelint-scss": "3.19.0"
  }
}

babel.config.json

{
  "env": {
    "test": {
      "presets": ["@parcel/babel-preset-env", "@babel/preset-react"]
    },
    "development": {
      "presets": ["@parcel/babel-preset-env", "@babel/preset-react"],
      "plugins": [
        "@babel/plugin-transform-runtime"
      ]
    },
    "production": {
      "presets": ["@parcel/babel-preset-env"],
      "plugins": [
        "transform-remove-console"
      ]
    },
  }
}

kriszyp · 2021-10-27T16:26:30Z

Thank you for the details! I'll see if I can reproduce.

kriszyp · 2021-10-27T19:35:06Z

@hgeist2 I was able to start this right up with your config files and just adding a index.pug, and it all ran perfectly fine for me on my linux mint computer. Based on previous comments, it sounded like this had something to do with workers/threads, is it possible that there is something that triggers multiple threads to start in parcel in your setup? And this a consistent error, or intermittent?

mischnic · 2021-10-27T19:52:21Z

By default, Parcel uses worker threads. You can override the automatic calculation with PARCEL_WORKERS=2 parcel build .... With 0, everything runs on the main thread - so if you still get the error in that case, it shouldn't be related to workers.

hgeist2 · 2021-10-27T21:30:32Z

Turns out updating to node 15.x/15.4 solves the problem.

kriszyp · 2021-10-27T21:37:33Z

Hmm, that's interesting. Node 15.x isn't really supported (by NodeJS, only 12.x, 14.x, 16.x, and 17.x are the maintained/supported versions). And the OP was using Node 16.x. I wonder if this goes away on a newer 16.x as well?

And my attempts to reproduce this were with Node 14.17.3. And I did actually try PARCEL_WORKERS=20 parcel... but that didn't cause an error either.

hgeist2 · 2021-10-27T21:41:57Z

If i switch to 16.13.0 it stops working and if i switch back to 15.4.0 it starts working again.

Im not sure PARCEL_WORKERS=x really does anything in parcel 2.0.0, i could not find anything in the docs about it.

hgeist2 · 2021-10-27T21:49:53Z

The project leader downloaded and tried to compile lmdb-store locally. That went fine, but the tests failed.

Maybe both stacktraces stem from a problem with type casting? Could be confirmed if the test below starts working on 15.4.0

Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:24 PDT 2021; root:xnu-8019.41.5~1/RELEASE_ARM64_T8101 arm64

% npm test
> [email protected] test ExternalSource/lmdb-store
> mocha test/**.test.js --recursive && npm run test:types
(node:43728) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
  lmdb-store
    Basic use
FATAL ERROR: v8::Function::Cast Value is not a Function
 1: 0x1022300a4 node::Abort() [.nvm/versions/node/v14.17.4/bin/node]
 2: 0x102230224 node::errors::TryCatchScope::~TryCatchScope() [.nvm/versions/node/v14.17.4/bin/node]
 3: 0x10235aab8 v8::Function::CheckCast(v8::Value*) [.nvm/versions/node/v14.17.4/bin/node]
 4: 0x10fb988c8 v8::Function::Cast(v8::Value*) [ExternalSource/lmdb-store/build/Debug/lmdb-store.node]
 5: 0x10fb8f984 v8::Local<v8::Function> v8::Local<v8::Function>::Cast<v8::Value>(v8::Local<v8::Value>) [ExternalSource/lmdb-store/build/Debug/lmdb-store.node]
 6: 0x10fb8eca0 EnvWrap::open(Nan::FunctionCallbackInfo<v8::Value> const&) [ExternalSource/lmdb-store/build/Debug/lmdb-store.node]
 7: 0x10fb973f4 Nan::imp::FunctionCallbackWrapper(v8::FunctionCallbackInfo<v8::Value> const&) [ExternalSource/lmdb-store/build/Debug/lmdb-store.node]
 8: 0x1023b2108 v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo) [.nvm/versions/node/v14.17.4/bin/node]
 9: 0x1023b1714 v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) [.nvm/versions/node/v14.17.4/bin/node]
10: 0x1023b0fac v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) [.nvm/versions/node/v14.17.4/bin/node]
11: 0x102acc3ec Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit [.nvm/versions/node/v14.17.4/bin/node]
12: 0x102a64ff4 Builtins_InterpreterEntryTrampoline [.nvm/versions/node/v14.17.4/bin/node]
13: 0x102a64ff4 Builtins_InterpreterEntryTrampoline [.nvm/versions/node/v14.17.4/bin/node]
14: 0x102a64ff4 Builtins_InterpreterEntryTrampoline [.nvm/versions/node/v14.17.4/bin/node]
15: 0x102a64ff4 Builtins_InterpreterEntryTrampoline [.nvm/versions/node/v14.17.4/bin/node]
16: 0x102a64ff4 Builtins_InterpreterEntryTrampoline [.nvm/versions/node/v14.17.4/bin/node]
17: 0x102a64ff4 Builtins_InterpreterEntryTrampoline [.nvm/versions/node/v14.17.4/bin/node]
18: 0x102a64ff4 Builtins_InterpreterEntryTrampoline [.nvm/versions/node/v14.17.4/bin/node]
19: 0x102a628a4 Builtins_JSEntryTrampoline [.nvm/versions/node/v14.17.4/bin/node]
20: 0x102a62548 Builtins_JSEntry [.nvm/versions/node/v14.17.4/bin/node]
21: 0x148008000 
sh: line 1: 43728 Abort trap: 6           mocha test/**.test.js --recursive
npm ERR! Test failed.  See above for more details.

hgeist2 · 2021-10-27T22:02:49Z

Sorry for the spam.

I downloaded lmdb-store 1.6.11. and ran npm test

14.17.4: all good
15.4.0: tests fail with the same stacktrace as above
16.3.0: all good

That is directly inverted to the self-register stacktrace and node relationship.

kriszyp · 2021-10-27T22:07:46Z

Sorry for the spam.

I appreciate it, would love to track this down!

So I believe the type casting stacktrace is specific to running the debug builds (in combination with some specific node version and OS, I assume, since I haven't seen it before), which are only run by default with npm run recompile, and not with the prebuilds or default fallback build. I can easily fix this, and will, but doesn't seem likely to be the cause of the self-register (as you noted with the inverted relationship).

I tried your app/config specifically with node v14.15.4 and v16.13.0, and still didn't see any self-register error. I noticed in the other stack trace that it is ARM64; are you on ARM64 too? That might be an interesting bit of commonality with the OP.

hgeist2 · 2021-10-27T22:09:48Z

I am on x64 / Ryzen. The other stacktrace with ARM64 is a new M1 Mac

hgeist2 · 2021-10-27T22:34:53Z

Self-register stacktrace prevails on 5.4.0-89-generic #100-Ubuntu SMP, Ubuntu 20.04, x64 Ryzen as above with the following node versions:

15.0.0
15.1.0
15.2.0
15.3.0
15.5.0
16.3.0
16.13.0
17.0.1

Only 15.4.0 works and i can reproduce it. Installations via nvm.

https://github.com/nodejs/node/blob/master/doc/changelogs/CHANGELOG_V15.md#15.4.0

kriszyp · 2021-10-27T23:11:20Z

Are you recompiling with each of these nvm changes?

I actually can reproduce this self-register error message by deleting/renaming my prebuilds, compiling with one node version, and then switching to a different node version (with nvm as well). This is because the compiled code is for the wrong node version. This doesn't happen if your app is using the prebuilds, since switching node versions just ends up selecting a different/appropriate prebuild.

I am guessing what happened is that lmdb-store got compiled/built on a node version that doesn't have a prebuild, specifically a v15.x version (I don't include prebuilds for the v15.x line because they aren't supported/maintained by NodeJS). And once lmdb-store gets compiled/built, it uses the locally compiled version over the prebuilds, and so even when you switch node versions, the locally compiled version gets used instead of the appropriate prebuild.

What I don't understand is why v15.x versions would be incompatible with each other, since all 15.x versions are supposed to be on the same ABI.

So I guess the questions is if this error still occurs if lmdb-store is re-installed/compiled with the node version that is currently being used? Or if it occurs with one of the prebuilds (using node 14.x or 16.x with the build/Release directory deleted/renamed)?

hgeist2 · 2021-10-27T23:46:09Z

That doesnt seem to be it. Shuffling the node js versions between 14.x, 15.x and 16.x and always deleting node_modules/ + npm i after each version switch still leads to same results.

kriszyp · 2021-10-28T01:34:10Z

Did you happen to notice if it is compiling/building for all the different node js versions (versus using the prebuild, which shouldn't involve any compiling or any files created in build/Release)? I am just curious if there are possibly any differences between locally compiled and prebuilds in regards to this.
And does your coworker get the self-register error? Do you have any idea if this is specific to certain OS?

hgeist2 · 2021-10-28T08:02:44Z

My coworker said they didnt have problem with other parcel 2.0.0 projects and any kind of machine, be it mac m1, windows 10 or mint linux. It seems to be specific to my system.

Some recompiling was done compiling cpp files for lmdb, but i did not watch closely. Will retry again tonight / next days

hgeist2 · 2021-10-28T08:16:35Z

Maybe this could be relevant to you? https://stackoverflow.com/a/41283828

I searched lmdb-store repo for NODE_SET_METHOD and NODE_MODULE without results.

Maybe it solves automagically if you provide this initialization method additionally to NODE_MODULE_INITIALIZER in node-lmdb.cpp in case some packer like parcel does something weird or use NODE_MODULE_INIT instead of NODE_MODULE_INITIALIZER like they show in their example.

https://nodejs.org/api/addons.html#addons_context_aware_addons

If you provide me with the code i could try a build like that on my machine.

…actically acceptable target name, and some crude logging, #67

kriszyp · 2021-10-28T15:19:55Z

Maybe it solves automagically if you provide this initialization method additionally to NODE_MODULE_INITIALIZER in node-lmdb.cpp in case some packer like parcel does something weird or use NODE_MODULE_INIT instead

You can't use NODE_MODULE and NODE_MODULE_INIT, since they both are defining the same function (_module), so they conflict. And using NODE_MODULE instead of the context-aware initializer doesn't work with worker threads (it will throw the self-register error the first time a worker thread tries to load it).

However, in the latest commit, I did try a few things: I switched from NODE_MODULE_INITIALIZER to NODE_MODULE_INIT. I had tried this before and it didn't work, causing compilation errors. I realized this is because the build target name had a dash in it, and the macro was directly copying the target name into the source. I fixed this by change the build target name (to just "lmdb"). With this, I was able to switch to the more standard NODE_MODULE_INIT.

I added some crude logging that can be enabled with LMDB_STORE_LOGGING=true env variable. This should log the steps of starting the module and initializing/registering.

I also saw a note in the SO post about having the registration module be the first in the list of sources. I have never heard that before, but figured I would try that as well.

I did also include a commented-out sample code that use NODE_MODULE in node-ldmb.cpp, if you want to test that (uncomment it, and comment out NODE_MODULE_INIT), but again, that isn't really a viable solution.

This update is in master. Would you be willing to try that? Have you previously been able to reproduce the error just by running the lmdb-store tests? Or do you want me to publish this in NPM?

…crude logging, #67

hgeist2 · 2021-11-01T14:17:07Z

Thank you for the fast solution, i will try to build it from master and then give you feedback.

vozeldr · 2021-11-03T17:23:00Z

I have this same issue with node v17.0.1 and npm 8.1.0 running on Pop!_OS 21.04.

My project only has these dependencies and the error is originating from v8-compile-cache

  "dependencies": {
    "tslib": "^2.3.1"
  },
  "devDependencies": {
    "@parcel/config-default": "^2.0.0",
    "@parcel/transformer-typescript-tsc": "^2.0.0",
    "@tsconfig/recommended": "^1.0.1",
    "@types/jest": "^27.0.2",
    "@types/webextension-polyfill": "^0.8.0",
    "copyfiles": "^2.4.1",
    "cross-zip-cli": "^1.0.0",
    "jest": "^27.2.4",
    "jest-webextension-mock": "^3.7.19",
    "npm-run-all": "^4.1.5",
    "parcel": "^2.0.0",
    "rename-files": "^0.0.2",
    "rimraf": "^3.0.2",
    "ts-jest": "^27.0.5",
    "typescript": "^4.4.3",
    "webextension-polyfill": "^0.8.0"
  }

kriszyp · 2021-11-03T18:34:17Z

@vozeldr Do you happen to know if lmdb-store compiled a build (created a node_modules/lmdb-store/build/Release, otherwise it is using the prebuilds)?
Would you be able to try building lmdb-store from master, in case the changes I made for @hgeist2 help? Or I could publish a release from the latest master (I just don't know if it actually fixes the issue yet).

vozeldr · 2021-11-03T19:07:06Z

Yes, sorry for not providing better initial info... That folder does exist. I only encounter this problem on subsequent builds and only "occasionally". When I get this error, I delete node_modules and reinstall and it works for a random time (it could be 1 or 2 builds, it could be a day's worth of builds). I only started encountering this problem after updating to the "new parcel", but it's been a consistent issue since then.

kriszyp · 2021-11-03T20:33:03Z

So parcel/lmdb-store will work for a while, and then start throwing this error, and that point it will just continue to throw this error until you reinstall all the packages? Any chance you could see if it is resolved with the latest updates on master?

hgeist2 · 2021-11-04T17:50:59Z

I can confirm that the latest master solves the issue.

Moved project to a new VM which was based on Xbuntu 20.04, too. Got the same error with install from the repo and it was fixed by installing the current master.

kriszyp · 2021-11-04T19:36:16Z

Thank you so much for checking this out @hgeist2! I have published v1.6.12 with this fix. If anyone else is able to try that out, would love to know if fixes the other situations where this occurs.

kriszyp added a commit that referenced this issue Jun 30, 2021

Update to using latest recommended node-add on API, #67

1bd33f0

kriszyp added a commit that referenced this issue Oct 27, 2021

Remove unnecessary onReadTxnRenew cast, #67

de03f6e

kriszyp added a commit that referenced this issue Oct 28, 2021

Try to take more steps to ensure proper module registration, use synt…

04aaf9d

…actically acceptable target name, and some crude logging, #67

kriszyp added a commit that referenced this issue Oct 29, 2021

Try to take more steps to ensure proper module registration and some …

72b3c0b

…crude logging, #67

Biosphakus mentioned this issue Nov 4, 2021

Parcel 2: parcel build fails with error related to lmdb-store parcel-bundler/parcel#7237

Closed

kriszyp added a commit that referenced this issue Nov 5, 2021

Expand builds, #92, #67

11ec0f2

kriszyp closed this as completed Apr 21, 2022

Module does not self-register lmdb-store.node #67

Module does not self-register lmdb-store.node #67

Comments

rizrmd commented Jun 30, 2021 • edited Loading

kriszyp commented Jun 30, 2021

rizrmd commented Jul 1, 2021

rizrmd commented Jul 1, 2021

kriszyp commented Jul 1, 2021 • edited Loading

rizrmd commented Jul 2, 2021

rizrmd commented Jul 2, 2021 • edited Loading

rizrmd commented Jul 2, 2021

kriszyp commented Jul 2, 2021 • edited Loading

rizrmd commented Jul 3, 2021

hgeist2 commented Oct 27, 2021 • edited Loading

kriszyp commented Oct 27, 2021

hgeist2 commented Oct 27, 2021 • edited Loading

kriszyp commented Oct 27, 2021

hgeist2 commented Oct 27, 2021 • edited Loading

kriszyp commented Oct 27, 2021

kriszyp commented Oct 27, 2021 • edited Loading

mischnic commented Oct 27, 2021

hgeist2 commented Oct 27, 2021

kriszyp commented Oct 27, 2021

hgeist2 commented Oct 27, 2021

hgeist2 commented Oct 27, 2021

hgeist2 commented Oct 27, 2021

kriszyp commented Oct 27, 2021 • edited Loading

hgeist2 commented Oct 27, 2021

hgeist2 commented Oct 27, 2021 • edited Loading

kriszyp commented Oct 27, 2021

hgeist2 commented Oct 27, 2021

kriszyp commented Oct 28, 2021 • edited Loading

hgeist2 commented Oct 28, 2021 • edited Loading

hgeist2 commented Oct 28, 2021

kriszyp commented Oct 28, 2021 • edited Loading

hgeist2 commented Nov 1, 2021

vozeldr commented Nov 3, 2021

kriszyp commented Nov 3, 2021

vozeldr commented Nov 3, 2021

kriszyp commented Nov 3, 2021

hgeist2 commented Nov 4, 2021

kriszyp commented Nov 4, 2021

rizrmd commented Jun 30, 2021 •

edited

Loading

kriszyp commented Jul 1, 2021 •

edited

Loading

rizrmd commented Jul 2, 2021 •

edited

Loading

kriszyp commented Jul 2, 2021 •

edited

Loading

hgeist2 commented Oct 27, 2021 •

edited

Loading

hgeist2 commented Oct 27, 2021 •

edited

Loading

hgeist2 commented Oct 27, 2021 •

edited

Loading

kriszyp commented Oct 27, 2021 •

edited

Loading

kriszyp commented Oct 27, 2021 •

edited

Loading

hgeist2 commented Oct 27, 2021 •

edited

Loading

kriszyp commented Oct 28, 2021 •

edited

Loading

hgeist2 commented Oct 28, 2021 •

edited

Loading

kriszyp commented Oct 28, 2021 •

edited

Loading