FFI documentation

[hiperiondev]

First of all thank you very much for the excellent project!
I have created a simple port for esp32 in a couple of hours (https://github.com/hiperiondev/esp32-microvium) and the minimum test works perfectly. To facilitate the use it has littlefs and a small TFTP server.
I've seen quickly: https://github.com/coder-mike/microvium-ffi-example
The method is not very clear (especially because it is done in C++ and not in C)
I think that a small documentation on how the FFI process is carried out would be necessary

[coder-mike]

Hi. Great to hear you like it and thanks for linking your project!

To clarify, are you asking for how to use the low level FFI methods in microvium.h and vmImport/vmExport, or are you asking for how to create an FFI wrapper library like microvium-ffi-example, or documentation on how to use the microvium-ffi-example library (which is a C++ library)?

[hiperiondev]

Exactly. Microvium seems to me a good alternative to Lua as a "glue" language.
I would need a concrete example of how to access a function in C and how to use it with Javascript.

[hiperiondev]

microvium-ffi-example is an example of "autobuilder" code. Not a concrete example of an arbitrary function imported and used within the javascript program

[hiperiondev]

After read https://coder-mike.com/blog/2022/10/16/ffi-generator-library/ I understood a little how it would be implemented. It's a bit confusing though.
I'm going to try some tests, but a good C example would be much more beneficial and self-explanatory than an ffi "autogenerator".
On the other hand, you assume that C++ is the language used by default for microcontrollers, we could discuss it but it is much more common to use C, as well as any example being simpler and working for both C and C++.

[coder-mike]

Ok, I've now written a guide to how to interface between C and JavaScript. See doc/ffi-guide.md. Let me know if that helps!

I think perhaps you misunderstood the intention of the microvium-ffi-example. As you may have figured out, it's an example that demonstrates how to write an FFI generator library, and in particular it's an example that shows how the snapshotting and build-time execution model of Microvium makes it possible to write very powerful libraries that encapsulate a lot of the awkward detail and hazards of FFI in general. At some point, such a library may be baked into Microvium so that it's easier to use, but at the moment you either need to just the raw Microvium API or create your own library (which may be based on the one I threw together as an example or it may be completely different), or use someone else's library (of which I don't think there are any at the moment).

Hopefully the new FFI guide is more in line with what you're trying to achieve at the moment.

[hiperiondev]

THANKS A LOT!!!!
I have read the document and it is really very clear.
I will do some tests to implement some basic microcontroller functions.
Some kind of HAL would be useful to easily port to other uCs.
I'm going to approach the project that way.

[coder-mike]

A HAL would be great. Are you doing this as open source work?

[hiperiondev]

yes, MIT license.

[coder-mike]

Nice! Can you link it here?

[hiperiondev]

Of course!!
I found a generic BSD-licensed RTOS HAL library that looks interesting to implement.
When I have news I'll let you know.

[hiperiondev]

Of course!!
I found a generic BSD-licensed RTOS HAL library that looks interesting to implement (or maybe Driver Subsystem of FreeRTOS HAL)
When I have news I'll let you know.

[hiperiondev]

I'll start with: http://www.wsn.agh.edu.pl/download/public/halfred/0.2.0/

And then I'll expand on it as needed.

[hiperiondev]

I have added HAL structure and minimal functionality for wifi and filesystem. Not microvium integration for now.
You can see that at: https://github.com/hiperiondev/esp32-microvium/tree/main/components/uc-hal

[hiperiondev]

Ok...my first FFI :-)
I have added simple wifi connect to HAL.
Can you see the project ? Please destroy my horrible code!

[hiperiondev]

I have a problem with wifi scan.
APs list is a complex array ( hal_wifi_ap_record_t in: https://github.com/hiperiondev/esp32-microvium/blob/main/components/uc-hal/hal/include/hal_wifi.h )
How I can export this structure?

[coder-mike]

Can you see the project ?

Yes, I can see it :-)

How I can export this structure?

To pass a structure over the boundary, there are two ways that come to mind:

1. You build the corresponding object in JavaScript.
2. You keep the struct in C and have JavaScript functions that access each property.

To build the object in JavaScript, you need JavaScript to export some functions for building objects.

vmExport(1, () => ({})); // new object
vmExport(2, (o, k, v) => o[k] = v); // set property
vmExport(3, (o, k) => o[k]); // get property

You can call these methods in C to build up a corresponding object in JS-land.

Going the other route (option 2), you might do something like this:

// hal_wifi_ap_record_t.js

const getField = vmImport(1); // To get a field of hal_wifi_ap_record_t

export class hal_wifi_ap_record_t {
  constructor(cPointer) { this.cPointer = cPointer; };

  getBssid() { return getField(this.cPointer, 1) }
  getSsid() { return getField(this.cPointer, 2) }
  getPrimary() { return getField(this.cPointer, 3) }
};

Here, cPointer can just be an integer in JS which C interprets as a pointer.

[hiperiondev]

mmmm I think the second option is the simplest...

[hiperiondev]

Could you please give me a simple complete example of method 1?

[hiperiondev]

please?

[coder-mike]

Sure.

Does this help you get started:

// copy-struct.c/h

typedef enum FieldType {
  FT_INT,
  FT_STRING,
} FieldType;

typedef struct FieldDef {
  const char* name;
  size_t offset;
  FieldType type;
} FieldDef;

// Assuming that these imports have already been resolved
extern mvm_Value newObject;
extern mvm_Value setProp;

mvm_Value copyStructToVm(mvm_VM* vm, void* structPtr, FieldDef* fields, int fieldCount) {
  mvm_Handle obj;
  mvm_Handle key;
  mvm_TeError err;
  mvm_Value args[3];

  mvm_initializeHandle(vm, &obj);
  mvm_initializeHandle(vm, &key);

  // Create object
  err = mvm_call(vm, newObject, mvm_handleAt(&obj), NULL, 0);
  if (err) abort();

  for (int i = 0; i < fieldCount; i++) {
    FieldDef* field = &fields[i];
    mvm_handleSet(&key, mvm_newStringUtf8(vm, field->name, strlen(field->name)));
    args[0] = mvm_handleGet(&obj);
    args[1] = mvm_handleGet(&key);
    args[2] = copyFieldToVm(vm, structPtr, field);
    err = mvm_call(vm, setProp, NULL, args, 3);
    if (err) abort();
  }

  mvm_Value result = mvm_handleGet(&obj);
  mvm_releaseHandle(vm, &key);
  mvm_releaseHandle(vm, &obj);

  return result;
}

mvm_Value copyFieldToVm(mvm_VM* vm, void* structPtr, FieldDef* field) {
  void* fieldPtr = (uint8_t*)structPtr + field->offset;
  switch (field->type) {
    case FT_INT: {
      return mvm_newInt32(vm, *(int32_t*)fieldPtr);
    }
    case FT_STRING: {
      const char* s = *(char**)fieldPtr;
      return mvm_newStringUtf8(vm, s, strlen(s));
    }
    default: {
      // TODO: Other types
      abort();
    }
  }
}

// Point.h

#include "copy-struct.h"
#include <stddef.h>

typedef struct Point {
  int x;
  int y;
} Point;

static const FieldDef Point[] = {
  { "x", offsetof(struct Point, x), FT_INT },
  { "y", offsetof(struct Point, y), FT_INT },
};

// These need to be exported to the host
const newObject = () => ({});
const setProp = (o, k, v) => o[k] = v;

If you give me some time, I'm in the process of writing a full example that deals with this more elegantly.

[hiperiondev]

Thanks a lot!

[hiperiondev]

Now I am rewriting some part of the HAL and change the TFTP server for a more comfortable and standard FTP server

[hiperiondev]

AWS FreeRTOS HAL has a lot of things figured out and MIT licensed