Skip to content

ocamlPackages: init a bunch of libraries for the MirageOS xen target#118066

Merged
Ericson2314 merged 4 commits into
NixOS:masterfrom
sternenseemann:mirage-xen
Apr 25, 2021
Merged

ocamlPackages: init a bunch of libraries for the MirageOS xen target#118066
Ericson2314 merged 4 commits into
NixOS:masterfrom
sternenseemann:mirage-xen

Conversation

@sternenseemann
Copy link
Copy Markdown
Member

@sternenseemann sternenseemann commented Mar 30, 2021

Motivation for this change

#23955

cc @Ericson2314 as well, I'd love to hear your thoughts on the pkg-config specific changes to the findlib setup hook.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@github-actions github-actions Bot added the 6.topic: ocaml OCaml is a general-purpose, high-level, multi-paradigm programming language. label Mar 30, 2021
@sternenseemann sternenseemann force-pushed the mirage-xen branch 2 times, most recently from 518d02f to c2344a4 Compare March 30, 2021 13:34
@ofborg ofborg Bot added 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 101-500 This PR causes between 101 and 500 packages to rebuild on Linux. labels Mar 30, 2021
@sternenseemann
Copy link
Copy Markdown
Member Author

sternenseemann commented Mar 30, 2021

@ofborg build ocamlPackages.mirage-net-xen ocamlPackages.mirage-bootvar-xen

@r-rmcgibbo
Copy link
Copy Markdown

r-rmcgibbo commented Mar 30, 2021 •
edited
Loading

Result of nixpkgs-review pr 118066 at c2344a44 run on x86_64-linux 1

1 package marked as broken and skipped:
  • hhvm
53 packages skipped due to time constraints:
  • abella
  • beluga
  • coccinelle
  • compcert
  • coqPackages.contribs.zorns-lemma
  • coq_8_10
  • coq_8_12
  • coq_8_13
  • coq_8_5
  • coq_8_6
  • ...
36 packages built successfully:
  • acgtk
  • alt-ergo
  • coq_8_11
  • dot-merlin-reader
  • dune_1
  • dune_2
  • eff
  • google-drive-ocamlfuse
  • haxe (haxe_4_2)
  • hxcpp (haxePackages.hxcpp)
  • haxePackages.hxcs
  • haxePackages.hxjava
  • haxePackages.hxnodejs_4
  • hevea
  • libguestfs
  • liquidsoap
  • obelisk
  • ocaml-crunch
  • ocaml-top
  • ocamlformat (ocamlformat_0_17_0)
  • ocamlformat_0_11_0
  • ocamlformat_0_12
  • ocamlformat_0_13_0
  • ocamlformat_0_14_0
  • ocamlformat_0_14_1
  • ocamlformat_0_14_2
  • ocamlformat_0_14_3
  • ocamlformat_0_15_0
  • ocamlformat_0_15_1
  • ocamlformat_0_16_0
  • ocsigen-i18n
  • opaline
  • opam-installer
  • reason
  • virt-top
  • why3

Result of nixpkgs-review pr 118066 at c2344a44 run on aarch64-linux 1

18 packages marked as broken and skipped:
  • coccinelle
  • coq_8_5
  • coq_8_6
  • cubicle
  • haxe_3_2
  • haxe_3_4
  • libvmi
  • mldonkey
  • monotoneViz
  • obliv-c
  • ...
7 packages failed to build:
26 packages skipped due to time constraints:
  • abella
  • coqPackages.contribs.zorns-lemma
  • coq_8_10
  • coq_8_11
  • coq_8_12
  • coq_8_13
  • coq_8_7
  • coq_8_8
  • coq_8_9
  • flow
  • ...
33 packages built successfully:
  • acgtk
  • alt-ergo
  • beluga
  • dot-merlin-reader
  • dune-release
  • dune_1
  • dune_2
  • eff
  • fstar
  • google-drive-ocamlfuse
  • haxe (haxe_4_2)
  • hxcpp (haxePackages.hxcpp)
  • haxePackages.hxcs
  • haxePackages.hxjava
  • haxePackages.hxnodejs_4
  • hevea
  • libguestfs
  • liquidsoap
  • obelisk
  • ocaml-crunch
  • ocaml-top
  • ocamlformat (ocamlformat_0_17_0)
  • ocamlformat_0_14_3
  • ocamlformat_0_15_0
  • ocamlformat_0_15_1
  • ocamlformat_0_16_0
  • opaline
  • opam-installer
  • orpie
  • patdiff
  • reason
  • satysfi
  • virt-top

Note that build failures may predate this PR, and could be nondeterministic or hardware dependent.
Please exercise your independent judgement.

Ericson2314
Ericson2314 reviewed Mar 30, 2021
View reviewed changes
Comment thread pkgs/development/ocaml-modules/mirage-xen/pkg-config.patch Outdated
Copy link
Copy Markdown
Member

@Ericson2314 Ericson2314 Mar 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm normally we keep prefix and friends? And prefix is set to $out by default as a best effort.

I went into the rabbit hole a bit, looking at ocaml/dune#1253 and linked issues like:

It looks like perhaps the pieces are there and those issues can be resolved?

Copy link
Copy Markdown
Member Author

@sternenseemann sternenseemann Mar 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh right, I completely forgot about that. I quickfixed this on a local branch and was meaning to follow up. I'll see what this is all about :)

Copy link
Copy Markdown
Member Author

@sternenseemann sternenseemann Apr 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue boils down to the following: The pkg config file in the repository has been manually created and assumes the default relation between dune's prefix and libdir (which would be "${prefix}/lib" for the latter). However for ocamlPackages we install libraries into a version specific site-lib, so that doesn't work out.

This is probably fixable in mirage-xen's build system, but they probably have little interest in fixing that before the upcoming rework.

Copy link
Copy Markdown
Member Author

@sternenseemann sternenseemann Apr 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be better now!

@Ericson2314
Copy link
Copy Markdown
Member

Ericson2314 commented Mar 30, 2021

mirage/mirage#969 mirage/mirage#1195 also make me begin to understand the pkg-config might duplicate functionality for the OCaml -> OCaml case (vs OCaml -> C or C -> OCaml case), and why they thus might want to get rid of it.

@sternenseemann
Copy link
Copy Markdown
Member Author

sternenseemann commented Mar 30, 2021

mirage/mirage#969 mirage/mirage#1195 also make me begin to understand the pkg-config might duplicate functionality for the OCaml -> OCaml case (vs OCaml -> C or C -> OCaml case), and why they thus might want to get rid of it.

Yep, I talked to them on IRC and basically they want to use dune for everything in MirageOS 4 including stub libs for the freestanding stuff. That should be no problem as dune is pretty well-behaved in my experience. Apparently the use of pkg-config was only introduced because dune used to automatically pass some “default” CFLAGS including -fPIC, so they pkg-config was introduced as an alternative way of passing CFLAGS around

@sternenseemann sternenseemann requested a review from vbgl April 20, 2021 11:46
@vbgl
Copy link
Copy Markdown
Contributor

vbgl commented Apr 21, 2021

I don’t understand why the PKG_CONFIG_PATH environment variable is set within the findlib hook: the two things look unrelated.

@sternenseemann
Copy link
Copy Markdown
Member Author

sternenseemann commented Apr 21, 2021

I don’t understand why the PKG_CONFIG_PATH environment variable is set within the findlib hook: the two things look unrelated.

The location of the .pc files is OCaml-specific unfortunately, but I don't think we'll need this in the long term (since MirageOS 4 promises to get rid of pkg-config), so I think I'll move it into an ocamlPkgConfigHook package, so we don't have to have it everywhere (and removing it doesn't trigger a huge rebuild).

@sternenseemann
ocamlPackages.shared-memory-ring(-lwt): init at 3.1.0
b5f7ed7
@sternenseemann
ocamlPackages.mirage-xen: init at 6.0.0
980a2be 
Unfortunately this requires a bit of trickery with pkg-config to get to
work. The root issue is that the mirage-xen assumes that we use the
default libdir of dune ($out/lib) whereas we install to an OCaml
version-specific site-lib directory. Thus the manually created
pkg-config file makes wrong assumptions (which warrants a patch) and the
.pc file is installed to the wrong location (which is fixed with a mv
invocation).
@sternenseemann
ocamlPackages.{mirage-net-xen,netchannel}: init at 2.0.0
2fe8239
@sternenseemann
ocamlPackages.mirage-bootvar-xen: init at 0.8.0
4f7a661
@sternenseemann
Copy link
Copy Markdown
Member Author

sternenseemann commented Apr 25, 2021

I believe, I've cleaned this up now: We don't need a new setup hook or the likes, we can just install the .pc file to its standard location in $out/lib/pkgconfig. This is probably the intention of mirage-xen anyways: From the .pc file I inferred that it assumes that dune's default libdir is used (which is $prefix/lib).

@sternenseemann sternenseemann requested a review from a user April 25, 2021 11:18
@ofborg ofborg Bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. and removed 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 101-500 This PR causes between 101 and 500 packages to rebuild on Linux. labels Apr 25, 2021
@Ericson2314
Copy link
Copy Markdown
Member

Ericson2314 commented Apr 25, 2021

Was waiting for you to say you think it's ready. LGTM!

@Ericson2314 Ericson2314 merged commit a86418a into NixOS:master Apr 25, 2021
@sternenseemann sternenseemann deleted the mirage-xen branch April 25, 2021 14:50
@sternenseemann sternenseemann restored the mirage-xen branch July 24, 2021 13:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Ericson2314 Ericson2314 Ericson2314 left review comments

@vbgl vbgl Awaiting requested review from vbgl

Assignees

No one assigned

Labels

6.topic: exotic Exotic hardware or software platform 6.topic: ocaml OCaml is a general-purpose, high-level, multi-paradigm programming language. 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@sternenseemann @r-rmcgibbo @Ericson2314 @vbgl