Support for generating asynchronous bindings using Lwt jobs

[yallop]

This pull request adds a new option for binding C functions, namely support for the Lwt jobs framework, allowing bound functions to execute asynchronously.

The changes to the interface are fairly minor, to the extent that existing bindings descriptions can be used unchanged.

The existing interface

For example, here is a functor which binds two functions, puts and rmdir:

module Bindings(F: Cstubs.FOREIGN) =
struct
  open F
  let puts = foreign "puts" (string @-> returning int)
  let rmdir = foreign "rmdir" (string @-> returning int)
end

With the existing interface you can generate C and ML by passing Bindings to the write_c and write_ml functions, like this:

write_c fmt ~prefix (module Bindings)
write_ml fmt ~prefix (module Bindings)

These calls generate a set of C functions and an OCaml module (say "Generated_module") containing code that binds puts and rmdir. Applying Bindings to Generated_module builds a module containing bindings to those functions, ready for use in an OCaml program:

(*
sig
  val puts : string -> int
  val rmdir : string -> int
end
*)
module C = Bindings(Generated_module)

The new interface

To create asynchronous bindings to puts and rmdir, most of the pieces above, including the Bindings functor, can be reused. The only change needed is an additional argument to write_c and write_ml:

write_c fmt ~concurrency:lwt_jobs ~prefix (module Bindings)
write_ml fmt ~concurrency:lwt_jobs ~prefix (module Bindings)

The type of the generated module ("Generated_lwt_module", say) is slightly different, reflecting the fact that the bindings are Lwt jobs rather than synchronous functions:

(*
sig
  val puts : string -> int Generated_lwt_module.return
  val rmdir : string -> int Generated_lwt_module.return
end
*)
module C_lwt = Bindings(Generated_module)

The type Generated_lwt_module.return is a simple wrapper around Lwt.t:

type 'a return = { lwt: 'a Lwt.t }

After projecting out the lwt field the results of the functions can be used with the Lwt library in the usual way:

(C_lwt.puts "Hello, world").lwt >>= fun () ->
...

[coveralls]

Coverage decreased (-0.07%) to 88.091% when pulling af89504 on yallop:lwt-jobs into d271413 on ocamllabs:master.

[samoht]

I am curious why you used type 'a return = { lwt: 'a Lwt.t } instead of type 'a return = 'a Lwt.t. Do you expect to add more fields in there?

(btw, that PR is great!)

[yallop]

I am curious why you used type 'a return = { lwt: 'a Lwt.t } instead
of type 'a return = 'a Lwt.t. Do you expect to add more fields in
there?

It's a workaround for a restriction in the type system. If you have an abstract type with a parameter, like this

   type 'a foo

then you're not allowed to use it to define a GADT that uses 'foo' in the return type of one of the constructors, like this

   type 'a t = T : 'a -> 'a foo t

Allowing that definition would cause problems if foo was defined as follows:

   type 'a foo = int 

The restriction was introduced in OCaml 4.01.0. Here's an example using OCaml 4.00.1 showing what goes wrong:

$ cat inj.ml
module F (X: sig type 'a foo end) =
struct
  type 'a t = T : 'a -> 'a X.foo t
  let unT : type a. a X.foo t -> a = function (T x) -> x
end

include F(struct type 'a foo = int end)

let () = print_endline (unT (T 0))
$ ocaml inj.ml 
Segmentation fault

The module generated by Cstubs includes a type like foo for defining function type representations:

   type 'a fn =
     Returns : 'a typ -> 'a return fn
   | Function : 'a typ * 'b fn -> ('a -> 'b) fn

This definition of fn would not be allowed if we had type 'a return = 'a Lwt.t, since Lwt.t is an abstract type. The one-field record definition is a way to work around the restriction.

[yallop]

(btw, that PR is great!)

Thanks!