Sending class instances to worker threads

[wilk]

• Node.js Version: v10.11.0
• OS: Mac OS X High Sierra 10.13.6
• Scope (install, code, runtime, meta, other?): runtime and code
• Module (and version) (if relevant): worker threads

I'm trying to figure out how can I pass complex structures, like class instances, to worker threads.
I tried using strings and just postMessage but well there's no an easy-to-use way to serialize/deserialize class instances.
I read about SharedArrayBuffer but I don't know how to convert a class instance in pure bytes.

How can I do that?

Example:

// main thread
class Person {
  constructor(name, surname) {
    this.name = name
    this.surname = surname
  }

  fullname() {
    return `${this.name} ${this.surname}`
  }

  static whoAmI() {
    return 'a Person'
  }
}

const foo = new Person('foo', 'bar')
worker.postMessage(foo) // foo should be serialized
worker2.postMessage(Person) // Person should be serialized

// worker thread
parentPort.on('message', foo => {
  const fullname = foo.fullname()

  parentPort.postMessage(fullname)
})

// worker2 thread
parentPort.on('message', Person => {
  const whoIs = Person.whoAmI()

  parentPort.postMessage(whoIs)
})

[gireeshpunathil]

$ cat 1558.js

const {
  Worker, isMainThread, parentPort, workerData
} = require('worker_threads');

class Person {
  constructor(name, surname) {
    this.name = name
    this.surname = surname
  }

  fullname() {
    return `${this.name} ${this.surname}`
  }

  static whoAmI() {
    return 'a Person'
  }
}

if (isMainThread) {

  const foo = new Person('foo', 'bar')
  const worker = new Worker(__filename, {});

  worker.on('message', (m) => {
    console.log(`worker message: ${JSON.stringify(m)}`)
  })
  worker.postMessage(foo) // foo is serialized
} else {
  parentPort.on('message', foo => {
    console.log(`in worker, foo is like: ${JSON.stringify(foo)}`)
    parentPort.postMessage(foo)
  })
}

$ node --experimental-worker 1558

in worker, foo is like: {"name":"foo","surname":"bar"}
worker message: {"name":"foo","surname":"bar"}

^C
$

I believe the member functions are omitted upon serialization because the serializer defines a data-only specification. However, copying @addaleax to confirm.

[wilk]

@gireeshpunathil That example is different.
You're using the same sources for both the main thread and the worker.
In my case, workers and main thread don't share the same code and thus the instance of class Person must be sent via postMessage.
So, this doesn't solve the issue but thanks anyway.

[gireeshpunathil]

just shift the
if (isMainThread) {
line upwards to cover the class definition only in main thread no? Also please note that the instance foo is created only in main thread, and is made accessible to the worker thread through message only. So I don't think we are talking about different things.

Serialization exists, and is possible through the way I described. If you doubt, please split the main and worker logic into 2 files and check; it works. The only caveat is with member methods, which is inhibited by the JSON spec, not by any worker limitation.

Hope this helps.

[wilk]

Sorry, I didn't explain myself well.
In my case you don't have access to the source code.
Class Person is defined by the user and just the instance is given to the worker and I don't have any usage of the isMainThread boolean.
Basically, users invoke postMessage just to send the instance of something that they already have defined somewhere and the worker has no access nor visibility of it.

Finally, yes, you're just considering the props of the instance (and they do work with JSON.stringify), not the methods.
I need to make the instance working as if it has been created inside the worker.
So, the deserialization process is more like a rehydration process: the user instantiates something and passes it to the worker, expecting to be used as if it belongs to the same thread.

To better understand what I'm saying, take a look to GoLang coroutines: they share the scope and that's basically the same idea I'm trying to reproduce.

[addaleax]

I think the issue here is that postMessage() uses the Structured Clone Algorithm – which means, all user-defined objects are basically treated like plain JS objects, whether they come from a class or not.

If you have that class available in both the sender and the receiver environment, and you know how the data is structured, you can probably address this through some post-processing – the main difference between a class instance and a “normal” object are their prototypes, so by setting the prototype of the received object to Person.prototype, it would become an instance of Person again.

[gireeshpunathil]

ping @wilk

[gireeshpunathil]

inactive, closing

[wilk]

@gireeshpunathil Sorry, I forgot this thread.
However, there's no solution available for now: in my case, the child thread has no information about the incoming classes, so there's no way to post-process anything.
Thanks anyway 💪

[gireeshpunathil]

ok, so - is your requirement something like defineClass in Java? that takes raw bytes as input and defines a Class from those?

[gireeshpunathil]

ping @wilk

[0x7061]

Coming from Go, I'm also missing an elegant way of using complex objects/instances in shared memory.

I'm working on a prepress engine, that uses multiple functional 'nodes' for image processing etc. where the nodes (executed inside a worker thread) need access to more complex global objects (web servers, etc.) from the parent application.

[wilk]

@gireeshpunathil I'm not sure.
Instead of defining a class, in my case I need to define it and then to rehydrate its state.

Consider the following example:

const date = new Date('01-01-2000');

What I need is to pass the instance date.
This means the class Date needs to be redefined and then reinitialised with 01-01-2000.
Moreover, if the instance is modified using the class methods, like:

date.setYear(2019);

Then also the passed instance must have the same internal state.

defineClass sounds like something static but if it takes in account the run-time state, then it's ok.

[devsnek]

js only has one thread safe data structure, SharedArrayBuffer. everything else must be copied from scratch.

[addaleax]

So … to expand on what I’ve said above:

• I know there’s @Bnaya’s https://github.com/Bnaya/objectbuffer library, which might still be a bit WIP, but allows representing objects through SharedArrayBuffers
• If we’d want to support a way for e.g. classes to define their own custom serialization/deserialization mechanism out of the box, that work would likely have to be done on the HTML WebMessaging spec in order to be accepted into V8 and then Node.js. I’d personally be a fan of that, but I probably don’t have the time or resources to push such an effort forward.

[PoojaDurgad]

@wilk - were you able to follow @addaleax's suggestions ? is the issue resolved?

[gireeshpunathil]

inactive, and / or resolved, closing. pls feel free to reopen if required