Struct support on Linux

[ring-c]

Operating System

• Windows
• macOS
• Linux
• FreeBSD
• Android
• iOS

What feature would you like to be added?

this

Can we please have support of sending structs as arguments on linux?
I am willing to test code if required.

Why is this needed?

No response

[JupiterRider]

Maybe it is possible to write a wrapper using libffi.
libffi.so is available on most linux systems, because it is a dependency of python and ruby.

[TotallyGamerJet]

My biggest concern is maintaining the code for each platform. Structs support on Darwin is more complicated than handling all other return and argument types for all platforms combined. Even if most of the code is the same I do not have easy access to these platforms to test and validate the accuracy of the implementations. Just see the stalling of #234

The current goal right now is supporting Ebitengine and as this is not needed right now for it this feature will have to wait. Of course this is not a complete write off and it may happen in the future.

[JupiterRider]

I finally have a working example:

bar/
├── ffi
│   └── ffi.go
├── go.mod
├── go.sum
└── main.go

ffi/ffi.go

//go:build linux && amd64
// +build linux,amd64

package ffi

import (
	"unsafe"

	"github.com/ebitengine/purego"
)

func init() {
	handle, err := purego.Dlopen("libffi.so", purego.RTLD_LAZY)
	if err != nil {
		panic(err)
	}
	purego.RegisterLibFunc(&PrepCif, handle, "ffi_prep_cif")
	purego.RegisterLibFunc(&Call, handle, "ffi_call")
}

type Abi uint32

const (
	DefaultAbi Abi = 2
)

type Status uint32

const (
	OK Status = 0
)

const (
	Void    = 0
	Double  = 3
	Float   = 2
	Sint32  = 10
	Pointer = 14
)

type Typ struct {
	Size      uint64
	Alignment uint16
	Typ       uint16
	Elements  **Typ
}

var (
	TypDouble  = Typ{8, 8, Double, nil}
	TypVoid    = Typ{1, 1, Void, nil}
	TypPointer = Typ{8, 8, Pointer, nil}
	TypSint32  = Typ{4, 4, Sint32, nil}
	TypFloat   = Typ{4, 4, Float, nil}
)

type Cif struct {
	Abi      uint32
	Nargs    uint32
	ArgTypes **Typ
	RTyp     *Typ
	Bytes    uint32
	Flags    uint32
}

var PrepCif func(cif *Cif, abi Abi, nargs uint32, rtyp *Typ, atypes []*Typ) Status

var Call func(cif *Cif, fn uintptr, rvalue unsafe.Pointer, avalue []unsafe.Pointer)

main.go

//go:build linux && amd64
// +build linux,amd64

package main

import (
	"bar/ffi"
	"fmt"
	"unsafe"

	"github.com/ebitengine/purego"
)

func main() {
	// load the the library we want to use
	raylib, err := purego.Dlopen("libraylib.so", purego.RTLD_LAZY)
	if err != nil {
		panic(err)
	}
	defer purego.Dlclose(raylib)

	// look for the symbol (function)
	vector3Normalize, err := purego.Dlsym(raylib, "Vector3Normalize") // RMAPI Vector3 Vector3Normalize(Vector3 v)
	if err != nil {
		panic(err)
	}

	// define the related golang type
	type Vector3 struct {
		X, Y, Z float32
	}

	// create the blueprint (description) of how the return value and arguments look like
	elements := []*ffi.Typ{&ffi.TypFloat, &ffi.TypFloat, &ffi.TypFloat, nil}
	vec3Typ := ffi.Typ{Size: 0, Alignment: 0, Typ: 13, Elements: &elements[0]}
	args := []*ffi.Typ{&vec3Typ}

	// prepare the function signature
	var cif ffi.Cif
	if ok := ffi.PrepCif(&cif, ffi.DefaultAbi, 1, &vec3Typ, args); ok != ffi.OK {
		panic("cif prep is not OK")
	}

	// call the function
	var rv Vector3
	v := Vector3{1, 1, 1}
	av := []unsafe.Pointer{unsafe.Pointer(&v)}
	ffi.Call(&cif, vector3Normalize, unsafe.Pointer(&rv), av)

	fmt.Printf("%+v\n", rv)
}

Prints {X:0.57735026 Y:0.57735026 Z:0.57735026}

@TotallyGamerJet
Do you think it would make sense to create a different repository for such a ffi library?

[TotallyGamerJet]

Do you think it would make sense to create a different repository for such a ffi library?

Sure if you want to open source it.

[JupiterRider]

Repo is here. Feedback is welcome:
https://github.com/JupiterRider/ffi

I also added examples.

[ring-c]

As far as I understand, we need to describe C-type on Go side, like in this example we have TypeTexture.
If so, there is a need for additional types in ffi package, like char or bool.

I am trying to use it for this struct:

Struct

typedef struct {
  const char *model_path;
  const char *vae_path;
  const char *taesd_path;
  const char *control_net_path;
  const char *lora_model_dir;
  const char *embed_dir;
  const char *id_embed_dir;
  bool vae_decode_only;
  bool vae_tiling;
  bool free_params_immediately;
  int n_threads;
  enum sd_type_t wType;
  enum rng_type_t rng_type;
  enum schedule_t schedule;
  bool keep_clip_on_cpu;
  bool keep_control_net_cpu;
  bool keep_vae_on_cpu;
} new_sd_ctx_go_params;

[JupiterRider]

@ring-c
For const char * you can use the ffi.TypePointer. The go related type would be *byte then.
You can then use unix.BytePtrToString to convert the byte pointer into a go string.

For a bool u can try to use ffi.TypeUint32. Then you can check its value (0 = false and 1 = true).

[ring-c]

@JupiterRider

For const char * you can use the ffi.TypePointer. The go related type would be *byte then.

It looks like it must be *[]byte

panic: libffi.so: cannot open shared object file: No such file or directory

goroutine 1 [running]:
github.com/jupiterrider/ffi.init.0()
	[...]/github.com/jupiterrider/[email protected]/ffi.go:19 +0x13c
exit status 2

Seems like you need to export variable so i can pass path to libffi.so, for debian 12:

# find /usr/lib -name "libffi.so*"
/usr/lib/i386-linux-gnu/libffi.so.8
/usr/lib/i386-linux-gnu/libffi.so.8.1.2
/usr/lib/x86_64-linux-gnu/libffi.so.8
/usr/lib/x86_64-linux-gnu/libffi.so.8.1.2

Tested with /usr/lib/x86_64-linux-gnu/libffi.so.8, seems like it works.

How exactly go-struct maps to ffi.Type according to fields positions/order?
I am getting SIGSEGV: segmentation violation on try to pass multiply string fields.

[JupiterRider]

@ring-c
I released v0.1.0-beta.5, which looks for libffi.so.8. So you shouldn't see this error anymore.

The order of the go-type and the c-type should be the same.
When it comes to string, it really is a *byte in golang. You can use the unix.BytePtrToString function to convert them into a go-string.

If you have any further questions, feel free to open an issue in the ffi repository, so we don't spam this one too much :D

[TotallyGamerJet]

I think this should stay open so that others can see the workaround. And maybe purego will someday support this feature.