Bundle files accessed via fs.open on nodejs

[CodeWithOz]

i'm using a library that reads a file (using fs.open) with its own repo, and it's causing problems when I try to bundle my app. Specifically, it's this line in the geoip-liter nehahsn. The error I get is

Error: ENOENT: no such file or directory, open '/home/user/<my app's parent folder>/data/geoip-country.dat'

If you check the definition of the dataFiles variable in that file, you can see that it uses a relative path that resolves to the outer directory of that file, but that relative path is being interpreted as starting from the root of my repo. I've tried using this plugin suggestion to convert the __dirname to the actual path, but that's not working correctly. I've also tried forking the repo and adding some require statements to include the files present at those paths in the dataFiles variable (you can see here). This didn't work either. I don't want to exclude the library from the build, so using --external isn't really an option.

Looking through the code of the geoip-lite package, it seems that real problem is that the source code of geoip-lite uses fs.open to access some files, but esbuild doesn't appear to bundle those files.

What's the correct way to handle this?

[evanw]

This bundler does not do file system emulation (it’s out of scope). If you need that, you’ll have to use another bundler.

[CodeWithOz]

Okay thank you. I understand that there's no official way to do that using this bundler, but does that also mean there's no way to make it happen at all using custom plugins or other custom code? Like can you suggest an avenue that could make it work if I'm willing to explore it? Thanks.

[evanw]

Well it's not strictly impossible because in theory you could write your own fs implementation as well as an esbuild plugin that injects the entire contents of the top-level directory being bundled and all files inside it (or at least, the subset of the top-level directory that you intend to be accessible at run-time) into the bundle. Then your custom fs implementation could emulate the file system at run-time using the file system tree stored inside the bundle. That's a lot of work to build all of that, however.

[asafamr]

I needed something similar, ended up with a plugin similar to this :

const VFSPlugin = {
name: 'VirtualFs',
 setup(build) {
build.onResolve({ filter: /^fs$/ }, args => { // i'm not sure this and the namespace are needed
      return { path: args.path, namespace: VFSPlugin.name }
    })
  build.onLoad({ filter:/^fs$/), namespace: VFSPlugin.name }, args => {
    return { contents: `
      const _fs = {}; 
      _fs._init = function(obj){
      Object.assign(_fs, obj);
      Object.setPrototypeOf(_fs, Object.getPrototypeOf(obj))
    };
    module.exports = _fs
` }
  })
}
}

then inside the bundle and before any use of the fs module I do this:

import fs from 'fs'
import LocalStorageFileSystem from "browserfs/src/backend/LocalStorage"
fs._init (new LocalStorageFileSystem())

this uses https://github.com/jvilk/BrowserFS to emulate node's fs api (and also persist to local storage)
another option is https://github.com/streamich/memfs
the code can probably be cleaned up a bit but it worked for me

[nettybun]

Can you just set the global geodatadir to avoid the __dirname and parent path altogether? It's in their code:

if (typeof global.geodatadir === 'undefined'){
  geodatadir = path.join(__dirname, '/../data/');
} else {
  geodatadir = global.geodatadir;
}

[evanw]

I'm closing this issue because I consider bundling files accessed via fs.open to be out of scope. Emulating node's APIs is not esbuild's responsibility. Feel free to continue to discuss workarounds here though.