Skip to content

Conversation

@patricoferris
Copy link
Owner

@patricoferris patricoferris commented Jul 20, 2021

This PR starts to implement the "static home directory" idea where instead of setting the home directory to the build directory we instead have one fixed home directory per build and mount the build directory to this one. This implementation uses ZFS to mount the build directory as I'm not sure there's any way around that.

The main pain-point that this PR does a not-so-great job of fixing is the log file. It is opened in the build directory (i.e. the ZFS dataset directory) and so when that dataset directory is remounted to the fixed home directory... all sorts of errors start to happen. The workaround here is to record the log in /tmp/<id>-log and then copy it over after the build... not ideal... but the code seemed to be working for small examples. The next step will be to try the failing opam-repo-ci ocamlfind jobs.

(note: the "always mounting, never unmounting" ZFS datasets problem is not resolved (yet) in this PR)

@patricoferris
Copy link
Owner Author

Successfully built the following spec :)) (I think this was failing before)

((from macos-homebrew-ocaml-4.12)
 (user (uid 1000) (gid 1000))
 (run (shell "ln -f ~/local/bin/opam-2.1 ~/local/bin/opam"))
 (run (shell "opam init --reinit -ni"))
 (env OPAMDOWNLOADJOBS 1)
 (env OPAMERRLOGLEN 0)
 (env OPAMSOLVERTIMEOUT 500)
 (env OPAMPRECISETRACKING 1)
 (run (shell "rm -rf opam-repository/"))
 (copy (src .) (dst opam-repository/))
 (run (shell "opam repository set-url --strict default \"file://$HOME/opam-repository\""))
 (run (shell "opam pin add -k version -yn ppx_deriving_cmdliner.0.6.0 0.6.0"))
 (run (network host)
      (shell "opam update --depexts"))
 (run (cache (opam-archives (target /home/opam/.opam/download-cache)))
      (network host)
      (shell  "opam remove ppx_deriving_cmdliner.0.6.0 && opam install --deps-only ppx_deriving_cmdliner.0.6.0 && opam install -v ppx_deriving_cmdliner.0.6.0;\
             \n        res=$?;\
             \n        test \"$res\" != 31 && exit \"$res\";\
             \n        export OPAMCLI=2.0;\
             \n        build_dir=$(opam var prefix)/.opam-switch/build;\
             \n        failed=$(ls \"$build_dir\");\
             \n        for pkg in $failed; do\
             \n          if opam show -f x-ci-accept-failures: \"$pkg\" | grep -qF \"\\\"macos-homebrew\\\"\"; then\
             \n            echo \"A package failed and has been disabled for CI using the 'x-ci-accept-failures' field.\";\
             \n          fi;\
             \n        done;\
             \n        exit 1"))
 (run (network host)
      (shell "opam update --depexts"))
 (run (cache (opam-archives (target /home/opam/.opam/download-cache)))
      (network host)
      (shell  "opam remove ppx_deriving_cmdliner.0.6.0 && opam install --deps-only --with-test ppx_deriving_cmdliner.0.6.0 && opam install -v --with-test ppx_deriving_cmdliner.0.6.0;\
             \n        res=$?;\
             \n        test \"$res\" != 31 && exit \"$res\";\
             \n        export OPAMCLI=2.0;\
             \n        build_dir=$(opam var prefix)/.opam-switch/build;\
             \n        failed=$(ls \"$build_dir\");\
             \n        for pkg in $failed; do\
             \n          if opam show -f x-ci-accept-failures: \"$pkg\" | grep -qF \"\\\"macos-homebrew\\\"\"; then\
             \n            echo \"A package failed and has been disabled for CI using the 'x-ci-accept-failures' field.\";\
             \n          fi;\
             \n        done;\
             \n        exit 1"))
)

@patricoferris
Copy link
Owner Author

cc: @kit-ty-kate I've added the ZFS mounting alternative to #4 here, testing it out on various specs copied from opam-repo-ci. If it all works at some point I'll update the workers and it should mean we have caches and can renable the tests for macos too... in theory...

@kit-ty-kate
Copy link

\o/ :tada:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants