Polling mode - Native FSevents

[rgrinberg]

This is still WIP. I only committed the C bindings since those are fairly stable and ready for review. I'm now working on the other tasks:

• Create c bindings for the fsevents api we intend to use (just needs some error handling tweaks).
• Tests for the bindings
• Integrate fsevents with dune
• Remove inotify-wait and fswatch support
• Fix the bootstrap

[nojb]

Given that the UInt32 and UInt64 values are not really manipulated from the OCaml side, can't you use plain Int32 and Int64 values instead? That would save you from vendoring ocaml-integers.

[rgrinberg]

Good point. In the beginning I intended to do the flag manipulation on the OCaml side but decided against it because it would require a lot more bindings to constants without much clear benefits. I'll see if I can simplify.

[rgrinberg]

@jeremiedimino looks bootstrap has to be slightly altered to support a couple of new requirements. The build script for fsevents looks like:

(rule
 (deps (:script gen_flags.ml))
 (action (with-stdout-to library_flags.sexp (run ocaml %{script} %{system}))))

(library
 (name fsevents)
 (synopsis "minimal bindings for fsvents on macos")
 (library_flags (:include library_flags.sexp))
 (foreign_stubs
  (language c)
  (names fsevents_stubs))
 (libraries dyn stdune))

So we need both of the following:

1. Support conditional flags. We need to pass certain flags to link apple's sdk only on macos.
2. Pass flags to the linker.

[ghost]

Agreed. It's fine to hardcode these flags in the bootstrap program for now BTW. If things become more complicated, we can always revisit but for now hardcoding will be good enough.

[rgrinberg]

We decided to keep fswatch for now after all. It's the only thing that is usable on bsd. We'll get rid of it once we introduce a polling backend

[ghost]

I'm seeing spurious output when I do dune runtest:

dune build @runtest
inline_test_runner_csexp_rpc_tests alias test/expect-tests/csexp_rpc/runtest
()
inline_test_runner_dune_config_test alias test/expect-tests/dune_config/runtest
()
inline_test_runner_dune_file_watcher_tests_linux alias test/expect-tests/dune_file_watcher/runtest
()
inline_test_runner_dune_rpc_tests alias test/expect-tests/dune_rpc/runtest
()
inline_test_runner_dune_unit_tests alias test/expect-tests/runtest
()
inline_test_runner_dune_rpc_lwt_tests alias otherlibs/dune-rpc-lwt/test/runtest
()
inline_test_runner_vcs_tests alias test/expect-tests/runtest
()
inline_test_runner_fsevents_tests alias test/expect-tests/fsevents/runtest
()

Maybe a leftover debug printing somewhere?

I haven't read the fsevent code in detail. If there is something you'd like me to look at more in detail, please let me know.

[rgrinberg]

I haven't read the fsevent code in detail. If there is something you'd like me to look at more in detail, please let me know.

I would have a liked a glance over how I interface with the GC thankfully @nojb gave good feedback here already.

[nojb]

I haven't read the fsevent code in detail. If there is something you'd like me to look at more in detail, please let me know.

I would have a liked a glance over how I interface with the GC thankfully @nojb gave good feedback here already.

I don't mind reviewing the PR (or at least parts of it) before it is merged. I am planning to do the analogous thing for Windows so it would help me get a better feel for it. I should be able to do it over the weekend (maybe earlier, but no promises :))

[nojb]

LGTM. I left a suggestion for a small simplification of the Dune_file_watcher API and an alternative way to handle the registration of the "exception" global root in the C stubs. Feel free to ignore them.

[rgrinberg]

Maybe a leftover debug printing somewhere?

It's gone now. A subtle little issue.

One annoying thing is that we can't use the fsevents exclusion mechanism to ignore the build path and still get sync events which occur in the build dir. @aalekseyev can we move sync events outside of the build directory perhaps? They're only used for tests anyway.