diff --git a/nixos/doc/manual/configuration/configuration.md b/nixos/doc/manual/configuration/configuration.md index 4c966f3325b99..c54b5a18a3ad1 100644 --- a/nixos/doc/manual/configuration/configuration.md +++ b/nixos/doc/manual/configuration/configuration.md @@ -10,6 +10,7 @@ file-systems.chapter.md x-windows.chapter.md wayland.chapter.md gpu-accel.chapter.md +fhs.chapter.md xfce.chapter.md networking.chapter.md linux-kernel.chapter.md diff --git a/nixos/doc/manual/configuration/fhs.chapter.md b/nixos/doc/manual/configuration/fhs.chapter.md new file mode 100644 index 0000000000000..3d3eae9dc3e6a --- /dev/null +++ b/nixos/doc/manual/configuration/fhs.chapter.md @@ -0,0 +1,201 @@ +# Binaries from other distributions {#sec-fhs} + +Software packaged in Nixpkgs is built to be platform-agnostic in the sense that +it doesn't expect any dependencies to be implicitly provided by the host +environment. If a Nix-built program needs to load dynamic shared libraries, the +concrete revisions of these libraries are always deployed with the program in +predictable locations. This way, a program built by Nix can be run on any other +machine with the same architecture, regardless of the environment. For example, +such a program doesn't need global paths like `/bin` or `/usr/lib` to exist, +and it always loads the same versions of its dependencies. +This determinism has known limits, cf. [](#sec-fhs-drivers). + +Programs distributed in the binary form outside Nixpkgs will often expect to +find their dependencies in some of the several common global locations. +Specifically they would expect to be run in the so-called "merged FHS +environment" (for [Filesystem Hierarchy +Standard](https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard)). This +concerns software downloaded using `pip` or `conda install`, toolchains fetched +by build systems such as `gradle`, shell scripts with hard-coded paths, or +anything downloaded from the web. + +In Nixpkgs the "FHS" hierarchy is splayed, rather than merged: every revision +of a program is installed into its own prefix. NixOS does not materialize this +hierarchy into any global paths, meaning that the user wishing to run a program +expecting the "FHS" environment needs to explicitly point the program at its +dependencies. NixOS provides several tools that automate or otherwise +accommodate this task. + +## Dynamic linkage issues {#sec-fhs-dynamic-linkage} + +Dynamically linked native programs built for "FHS" environments expect to find +the so-called "dynamic linker" in a fixed location, e.g. +`/lib64/ld-linux-x86-64.so.2` depending on the platform. +In nixpkgs on the other hand the same binary will have the linker hard-coded +to a specific glibc package in the nix store +(i.e. `/nix/store/00000000000000000000000000000000-glibc-2.xx-xx/lib/ld-linux-x86-64.so.2`). +They might also expect a number of "system" libraries to be provided by the environment. +In "FHS" distributions these are deployed in `/usr/lib`, and paths to them are +discovered by the dynamic linker through the `/etc/ld.so.conf` file. These +files are not available in NixOS by default. + +NixOS provides the [](#opt-programs.nix-ld.enable) option which deploys a proxy where +"FHS" distributions deploy the dynamic linker (e.g. +`/lib64/ld-linux-x86-64.so.2`). This proxy allows the user to redirect "FHS" +programs to the "system dependencies" they expect using the `NIX_LD` and +`NIX_LD_LIBRARY_PATH` variables. These variables are safe in that they do not +affect software from Nixpkgs. The `nix-ld` NixOS module also sets up default +values for these variables. + +For example, with the following configuration: + +```nix + programs.nix-ld.enable = true; +``` + +...packages from conda can be used as follows: + +```bash +$ nix-shell -p micromamba +$ micromamba env create -n some-env python pytorch +$ eval "$(micromamba shell hook --shell=bash)" +$ micromamba activate some-env +$ python +>>> import torch +``` + +Without `nix-ld` it wouldn't be possible to run the conda-installed python +interpreter, because it wouldn't know where to find the dynamic linker: + +```bash +$ micromamba activate some-env +$ python +bash: /home/nixos/micromamba/envs/some-env/bin/python: cannot execute: required file not found +$ patchelf --print-interpreter $(which python) +/lib64/ld-linux-x86-64.so.2 +``` + +Similarly, native libraries installed from binaries may be used with `nix-ld`: + +```bash +$ python -m venv ~/some-venv +$ source ~/some-venv/bin/activate +$ pip install zmq +$ python +>>> import zmq +``` + +Without `nix-ld` the above would have failed: + +```bash +$ source ~/some-venv/bin/activate +$ python +>>> import zmq +... +ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory +``` + +A more complex scenario may be handled using ad hoc Nix shells: + +```nix +# shell.nix + +with import { }; + +mkShell { + packages = [ micromamba ]; + NIX_LD_LIBRARY_PATH = lib.makeSearchPath "lib" [ + addOpenGLRunpath.driverLink # "/run/opengl-driver" + glib.out + libglvnd + stdenv.cc.cc.lib + ]; +} +``` + +```bash +$ nix-shell shell.nix +$ micromamba env create -n omnimotion python=3.8 +$ eval "$(micromamba shell hook --shell=bash)" +$ micromamba activate omnimotion +$ micromamba install ... +$ pip install ... +``` + +## General missing paths issues {#sec-fhs-missing-paths} + +Software built for "FHS" environments might sometimes also access other +executables by their global "FHS" paths, instead of relying on the `PATH` +variable and accessing them by name. E.g. it is common for shell scripts to +start with a shebang like `#!/bin/bash` instead of `#!/usr/bin/env bash`. NixOS +does not, by default, deploy `/bin/bash`, but it does deploy `/usr/bin/env` +which can find `bash` using the `PATH` environment variable. + +NixOS provides the `services.envfs.enable` option for mounting a virtual +filesystem in `/bin` and `/usr/bin`, which would resolve attempts at opening +files in global locations (like `/bin/bash`) based on the `PATH` variable of +the calling process: + +```nix + services.envfs.enable = true; +``` + +## Simulating an "FHS" environment {#sec-fhs-containers} + +Nixpkgs also provide tools for composing packages into a merged file tree. It +is also possible to present this merged tree to "FHS" programs as overlaid on +top of the real file system using the kernel feature called "user namespaces". +This can be used to run "FHS" programs on NixOS Cf. [the Nixpkgs +manual](https://nixos.org/manual/nixpkgs/unstable/#sec-fhs-environments) + +## Graphics driver libraries {#sec-fhs-graphics-drivers} + +For certain libraries it isn't practical to link them directly like Nixpkgs do, +because of their relation to hardware and the kernel. This includes graphics +(OpenGL, Vulkan) and GPGPU compute (CUDA, ROCm) userspace drivers. These +userspace drivers are special dynamic shared libraries that are deployed and +updated with the respective kernel modules. Instead of linking concrete +versions of kernel-specific drivers, Nixpkgs software expects to find the +host-specific drivers in a fixed global location (currently +`/run/opengl-driver/lib`), or to be communicated a different location using the +`LD_LIBRARY_PATH` variable. + +NixOS deploys the GPU drivers in the location expected by Nixpkgs. Software +from other distributions, such as installed by `pip`, may not be aware of this +NixOS-specific path. In this case the user may communicate the location of the +drivers using the `NIX_LD_LIBRARY_PATH` environment variable (`LD_LIBRARY_PATH` +if not using `nix-ld`, cf [](#sec-fhs-dynamic-linkage)): + +```nix +# /etc/nixos/configuration.nix + hardware.opengl.enable = true; + services.xserver.videoDrivers = [ "nvidia" ]; + programs.nix-ld.enable = true; +``` + +```nix +# shell.nix + +with import { }; + +mkShell { + packages = [ micromamba ]; + NIX_LD_LIBRARY_PATH = lib.makeSearchPath "lib" [ + addOpenGLRunpath.driverLink # "/run/opengl-driver" + stdenv.cc.cc.lib + ]; +} +``` + +```bash +$ nix-shell +$ micromamba env create -n some-env python pytorch +$ eval "$(micromamba shell hook --shell=bash)" +$ micromamba activate some-env +$ python +>>> import torch +>>> torch.cuda.is_available() +True +``` + +to [](#sec-gpu-accel).