update to recent paf and irmin

[hannesm]

No description provided.

[hb9cwp]

I have been able to build your branch updat for both targets unix and hvt on OpenBSD-6.9 amd64 now *), though not functionally tested them yet.
Do you plan to merge PR#4 over at unipi into a similar feature branch?
Thanks.

*) But not yet for --target=hvt on OpenBSD-7.0 amd64 due to a problem with the assembler (whereas --target=unix builds OK):

[rs@obsdBlue:dns-primary-git]$ mirage configure -t hvt
[rs@obsdBlue:dns-primary-git]$ make depend
...
[x509.0.15.1] found in cache
[zarith-freestanding.1.12] found in cache

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[ERROR] The compilation of ocaml-freestanding failed at "/usr/local/bin/gmake".

#=== ERROR while compiling ocaml-freestanding.0.6.5 ===========================#
# context     2.0.8 | openbsd/x86_64 | ocaml-system.4.10.0 | https://opam.ocaml.org#78f7e61d
# path        ~/.opam/default/.opam-switch/build/ocaml-freestanding.0.6.5
# command     /usr/local/bin/gmake
# exit-code   2
# env-file    ~/.opam/log/ocaml-freestanding-65167-63c3c6.env
# output-file ~/.opam/log/ocaml-freestanding-65167-63c3c6.out
### output ###
# [...]
# amd64.S:812:9: error: changed section flags for .rodata.cst8, expected: 0x12
#         .section .rodata.cst8,"a",@progbits
#         ^
# amd64.S:812:9: error: changed section entsize for .rodata.cst8, expected: 8
#         .section .rodata.cst8,"a",@progbits
#         ^
# If your assembler produced syntax errors, it is probably
# unhappy with the preprocessor. Check your assembler, or
# try producing amd64.o by hand.
# gmake[1]: *** [Makefile:352: amd64.o] Error 2
# gmake[1]: Leaving directory '/home/rs/.opam/default/.opam-switch/build/ocaml-freestanding.0.6.5/ocaml/runtime'
# gmake: *** [Makefile:62: ocaml/runtime/libasmrun.a] Error 2

<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
+- The following actions failed
| - build ocaml-freestanding 0.6.5
+- 
- No changes have been performed
*** Error 31 in /home/rs/unikernel/roburio/dns-primary-git (Makefile:15 'depend')
$

[hannesm]

Thanks for testing. Yes, the robur-coop/unipi#4 is similar to this PR and is in my re-review pipeline and will be merged soon.

About the issue:

# amd64.S:812:9: error: changed section flags for .rodata.cst8, expected: 0x12
#         .section .rodata.cst8,"a",@progbits
#         ^

This has been discussed and fixed upstream (discussion in ocaml/ocaml#9981). The fix is included in 4.10.2 (that should be straightforward for OpenBSD to update to), also 4.11.2, and since 4.12.0.

I suspect the OpenBSD port is still at 4.10.0, eventually with the patch applied manually, but ocaml-freestanding takes the exact same version as the installed OCaml compiler (4.10.0 in your case) without additional patches.

A workaround is to create an opam switch with a different OCaml compiler (opam switch create 4.13.1) and compile & install the dns-primary-git there.

OpenBSD 6.9 seems to use an older clang that does not complain about the assembly error.

[hb9cwp]

@hannesm Thanks for your hints. Managed to build a package with 4.10.2 derived from 4.10.0 and then upgrade ocaml . Currently waiting until underlying base package requirements are upgraded too, following the recipe found in ocaml/opam#3708 after opam upgrade --fixup had failed with opam 2.0.8.
Once hvt builds as well, I will start functional tests. If successful, I intend to upstream the rather trivial patch for 4.10.2 via Anil who is maintainer of the OpenBSD port of ocaml.

[hannesm]

so this leads to successful compilation of dns-primary-git, but what is still missing are positive functional tests -- and what should be unrelated to let the git implementation use ipv4/ipv6 (by using happy-eyeballs within mimic/paf).

[hb9cwp]

@hannesm Yes correct. For the functional tests, I intended to get Unipi hvt target working first, as a Web server might presumably "simpler" than an authoritative DNS server. But maybe I should try with dns-primary-git in the mean-time.
Also, so far, I was only testing against a public Github repo, because I could not figure out yet how to make Irmin (cohttp, etc.) accept my self-signed certificate which my internal Gitea instance uses.

[hb9cwp]

@hannesm After unipi, your updat branch of dns-primary-git builds and runs for both unix and hvt targets on OpenBSD 7.0 amd64 in an IPv4/IPv6 dual-stack setup with ocaml 4.10.2 and mirage v3.10.6.
So far, I have only tested pulling the zone file from Github using HTTPS, as well as DNS queries to the unikernels' IPv4/6 addresses of their tap interfaces.
Next, I will try to get them working against my local Gitea, and setup post-commit hooks to send signed NOTIFIES with onotify :-)

[hannesm]

this was merged manually into the main branch.

[hannesm]

@hb9cwp how I have it set up is indeed a self-hosted gitea instance (git.robur.io), and I use git over ssh for the dns-primary-git, i.e.:

• execute "awa_gen_key" outputs (a) a seed (b) a ssh public key
• add a user to gitea with access to the right repository, and the public key from above as authentication mechanism
• run ssh-keygen -lf <(ssh-keyscan -t rsa git.robur.io 2>/dev/null)
• start the unikernel with --seed=(a) (from awa_gen_key) and --authenticator=SHA256:<FP> (from ssh-keyscan) and --remote=git@git.robur.io:<my-repo.git>

I used to suffer from some issues in the git-http transport (that may be fixed by now), and lack of knowledge how to setup http user authentication (before I started using gitea).

[hb9cwp]

@hannesm Thank you for guidance, which confirms what I tried to gather from your Deploying authoritative OCaml-DNS servers as MirageOS unikernels earlier, and encourages me to have another go at using SSH with both public Github and Gitea today.

I used to suffer from some issues in the git-http transport (that may be fixed by now)

Yesterday, I spent a quite a few hours to setup self-hosted Gitea for HTTPS correctly, e.g. providing it with fresh self-signed cert that matches its FQDN instead of server IP address, etc. With the result that the unikernels with debug log level fails now with
... 147d?\252\222\255)o\133\211z\019\229\128.\246<\197\199MY\220\163\020\247t]Rl\205\000\000")) irmin: [DEBUG] (fail-alert-out (FATAL BAD_CERTIFICATE)) irmin: [DEBUG] (failure (Error (AuthenticationFailure "invalid certificate chain"))) ^C
while current Chrome browser does not complain (though it shows warning in red "Not secure", of course).

More irritating is the default log output on the side of the Gitea server which kept me from trying to resolve the problem above (10.0.0.7 is the unix unikernel that connects to Gitea on another OpenBSD host in the same local subnet):
2021/11/09 10:05:13 ...c/net/http/server.go:3139:logf() [I] http: TLS handshake error from 10.0.0.7:35616: local error: tls: bad record MAC

According to a few reports, the Go library tends to log such errors if there are problems with checksum offloading to NICs, in my case a Realtek re(4), whereas the unikernel's host has an Intel em(4). However, none of them shows checksum errors with netstat -s.With OpenBSD, there is apparently no (quick) way to disable checksum offloading other than rebuiling the NIC drivers from source (or add another non-re(4) NIC to the Gitea host). Let's see if the SSH route is easier than debugging HTTPS further...

[hannesm]

The log output of gitea is indeed irritating. On the OCaml side, this ocaml-git code plays a role, noteworthy:

  module Nss = Ca_certs_nss.Make (Pclock) (* https://github.com/mirage/ca-certs-nss package *)
  [...]
  let authenticator = Rresult.R.failwith_error_msg (Nss.authenticator ())
  let default_tls_cfg = Tls.Config.client ~authenticator ()
  [...]
     dft tls default_tls_cfg

Basically this means that the default authenticator is chain of trust validation of the server certificate, using the nss trust anchors (also used by mozilla firefox, ca-certs-nss extracts them from nss into OCaml). This also means that your self-signed certificate won't be authenticated by git-paf. The bad record mac may need some investigation, my expectation was a bad certificate sent by the OCaml client to the go server.

And (/cc @dinosaure) git-paf should offer other authenticators via boot parameters, so that you can specify your own CA certificate, a key or certificate fingerprint, etc. -- similar to what git_mirage_ssh does with the --authenticator argument (by using Awa.Keys.authenticator_of_string). Maybe worth to report an issue to the mirage/ocaml-git repository so that this won't be forgotten.