Expand manual on derivation outputs

Note, this includes some text adapted from from Eelco's dissertation

Author: Ericson2314

.gitignore

@@ -14,7 +14,7 @@
 /tests/functional/lang/*.err
 /tests/functional/lang/*.ast
 
-outputs/
+/outputs
 
 *~
 

doc/manual/source/SUMMARY.md.in

@@ -22,7 +22,10 @@
   - [Store Object](store/store-object.md)
     - [Content-Addressing Store Objects](store/store-object/content-address.md)
   - [Store Path](store/store-path.md)
-  - [Store Derivation and Deriving Path](store/drv.md)
+  - [Store Derivation and Deriving Path](store/derivation/index.md)
+    - [Derivation Outputs and Types of Derivations](store/derivation/outputs/index.md)
+      - [Content-addressing derivation outputs](store/derivation/outputs/content-address.md)
+      - [Input-addressing derivation outputs](store/derivation/outputs/input-address.md)
   - [Building](store/building.md)
   - [Store Types](store/types/index.md)
 {{#include ./store/types/SUMMARY.md}}

doc/manual/source/glossary.md

@@ -22,15 +22,15 @@
 - [store derivation]{#gloss-store-derivation}
 
   A single build task.
-  See [Store Derivation](@docroot@/store/drv.md#store-derivation) for details.
+  See [Store Derivation](@docroot@/store/derivation/index.md#store-derivation) for details.
 
   [store derivation]: #gloss-store-derivation
 
 - [derivation path]{#gloss-derivation-path}
 
   A [store path] which uniquely identifies a [store derivation].
 
-  See [Referencing Store Derivations](@docroot@/store/drv.md#derivation-path) for details.
+  See [Referencing Store Derivations](@docroot@/store/derivation/index.md#derivation-path) for details.
 
   Not to be confused with [deriving path].
 
@@ -252,7 +252,7 @@
 
   Deriving paths are a way to refer to [store objects][store object] that might not yet be [realised][realise].
 
-  See [Deriving Path](./store/drv.md#deriving-path) for details.
+  See [Deriving Path](./store/derivation/index.md#deriving-path) for details.
 
   Not to be confused with [derivation path].
 

doc/manual/source/language/advanced-attributes.md

@@ -99,8 +99,8 @@ Derivations can declare some infrequently used optional attributes.
     to make it use the proxy server configuration specified by the user
     in the environment variables `http_proxy` and friends.
 
-    This attribute is only allowed in *fixed-output derivations* (see
-    below), where impurities such as these are okay since (the hash
+    This attribute is only allowed in [fixed-output derivations][fixed-output derivation],
+    where impurities such as these are okay since (the hash
     of) the output is known in advance. It is ignored for all other
     derivations.
 
@@ -119,135 +119,6 @@ Derivations can declare some infrequently used optional attributes.
     [`impure-env`](@docroot@/command-ref/conf-file.md#conf-impure-env)
     configuration setting.
 
-  - [`outputHash`]{#adv-attr-outputHash}; [`outputHashAlgo`]{#adv-attr-outputHashAlgo}; [`outputHashMode`]{#adv-attr-outputHashMode}\
-    These attributes declare that the derivation is a so-called *fixed-output derivation* (FOD), which means that a cryptographic hash of the output is already known in advance.
-
-    As opposed to regular derivations, the [`builder`] executable of a fixed-output derivation has access to the network.
-    Nix computes a cryptographic hash of its output and compares that to the hash declared with these attributes.
-    If there is a mismatch, the derivation fails.
-
-    The rationale for fixed-output derivations is derivations such as
-    those produced by the `fetchurl` function. This function downloads a
-    file from a given URL. To ensure that the downloaded file has not
-    been modified, the caller must also specify a cryptographic hash of
-    the file. For example,
-
-    ```nix
-    fetchurl {
-      url = "http://ftp.gnu.org/pub/gnu/hello/hello-2.1.1.tar.gz";
-      sha256 = "1md7jsfd8pa45z73bz1kszpp01yw6x5ljkjk2hx7wl800any6465";
-    }
-    ```
-
-    It sometimes happens that the URL of the file changes, e.g., because
-    servers are reorganised or no longer available. We then must update
-    the call to `fetchurl`, e.g.,
-
-    ```nix
-    fetchurl {
-      url = "ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz";
-      sha256 = "1md7jsfd8pa45z73bz1kszpp01yw6x5ljkjk2hx7wl800any6465";
-    }
-    ```
-
-    If a `fetchurl` derivation was treated like a normal derivation, the
-    output paths of the derivation and *all derivations depending on it*
-    would change. For instance, if we were to change the URL of the
-    Glibc source distribution in Nixpkgs (a package on which almost all
-    other packages depend) massive rebuilds would be needed. This is
-    unfortunate for a change which we know cannot have a real effect as
-    it propagates upwards through the dependency graph.
-
-    For fixed-output derivations, on the other hand, the name of the
-    output path only depends on the `outputHash*` and `name` attributes,
-    while all other attributes are ignored for the purpose of computing
-    the output path. (The `name` attribute is included because it is
-    part of the path.)
-
-    As an example, here is the (simplified) Nix expression for
-    `fetchurl`:
-
-    ```nix
-    { stdenv, curl }: # The curl program is used for downloading.
-
-    { url, sha256 }:
-
-    stdenv.mkDerivation {
-      name = baseNameOf (toString url);
-      builder = ./builder.sh;
-      buildInputs = [ curl ];
-
-      # This is a fixed-output derivation; the output must be a regular
-      # file with SHA256 hash sha256.
-      outputHashMode = "flat";
-      outputHashAlgo = "sha256";
-      outputHash = sha256;
-
-      inherit url;
-    }
-    ```
-
-    The `outputHash` attribute must be a string containing the hash in either hexadecimal or "nix32" encoding, or following the format for integrity metadata as defined by [SRI](https://www.w3.org/TR/SRI/).
-    The "nix32" encoding is an adaptation of base-32 encoding.
-    The [`convertHash`](@docroot@/language/builtins.md#builtins-convertHash) function shows how to convert between different encodings, and the [`nix-hash` command](../command-ref/nix-hash.md) has information about obtaining the hash for some contents, as well as converting to and from encodings.
-
-    The `outputHashAlgo` attribute specifies the hash algorithm used to compute the hash.
-    It can currently be `"blake3", "sha1"`, `"sha256"`, `"sha512"`, or `null`.
-    `outputHashAlgo` can only be `null` when `outputHash` follows the SRI format.
-
-    The `outputHashMode` attribute determines how the hash is computed.
-    It must be one of the following values:
-
-      - [`"flat"`](@docroot@/store/store-object/content-address.md#method-flat)
-
-        This is the default.
-
-      - [`"recursive"` or `"nar"`](@docroot@/store/store-object/content-address.md#method-nix-archive)
-
-        > **Compatibility**
-        >
-        > `"recursive"` is the traditional way of indicating this,
-        > and is supported since 2005 (virtually the entire history of Nix).
-        > `"nar"` is more clear, and consistent with other parts of Nix (such as the CLI),
-        > however support for it is only added in Nix version 2.21.
-
-      - [`"text"`](@docroot@/store/store-object/content-address.md#method-text)
-
-        > **Warning**
-        >
-        > The use of this method for derivation outputs is part of the [`dynamic-derivations`][xp-feature-dynamic-derivations] experimental feature.
-
-      - [`"git"`](@docroot@/store/store-object/content-address.md#method-git)
-
-        > **Warning**
-        >
-        > This method is part of the [`git-hashing`][xp-feature-git-hashing] experimental feature.
-
-  - [`__contentAddressed`]{#adv-attr-__contentAddressed}
-
-    > **Warning**
-    > This attribute is part of an [experimental feature](@docroot@/development/experimental-features.md).
-    >
-    > To use this attribute, you must enable the
-    > [`ca-derivations`][xp-feature-ca-derivations] experimental feature.
-    > For example, in [nix.conf](../command-ref/conf-file.md) you could add:
-    >
-    > ```
-    > extra-experimental-features = ca-derivations
-    > ```
-
-    If this attribute is set to `true`, then the derivation
-    outputs will be stored in a content-addressed location rather than the
-    traditional input-addressed one.
-
-    Setting this attribute also requires setting
-    [`outputHashMode`](#adv-attr-outputHashMode)
-    and
-    [`outputHashAlgo`](#adv-attr-outputHashAlgo)
-    like for *fixed-output derivations* (see above).
-
-    It also implicitly requires that the machine to build the derivation must have the `ca-derivations` [system feature](@docroot@/command-ref/conf-file.md#conf-system-features).
-
   - [`passAsFile`]{#adv-attr-passAsFile}\
     A list of names of attributes that should be passed via files rather
     than environment variables. For example, if you have
@@ -370,6 +241,134 @@ Derivations can declare some infrequently used optional attributes.
 
   ensures that the derivation can only be built on a machine with the `kvm` feature.
 
-[xp-feature-ca-derivations]: @docroot@/development/experimental-features.md#xp-feature-ca-derivations
+## Setting the derivation type
+
+As discussed in [Derivation Outputs and Types of Derivations](@docroot@/store/derivation/outputs/index.md), there are multiples kinds of derivations / kinds of derivation outputs.
+The choice of the following attributes determines which kind of derivation we are making.
+
+- [`__contentAddressed`]
+
+- [`outputHash`]
+
+- [`outputHashAlgo`]
+
+- [`outputHashMode`]
+
+The three types of derivations are chosen based on the following combinations of these attributes.
+All other combinations are invalid.
+
+- [Input-addressing derivations](@docroot@/store/derivation/outputs/input-address.md)
+
+  This is the default for `builtins.derivation`.
+  Nix only currently supports one kind of input-addressing, so no other information is needed.
+
+  `__contentAddressed = false;` may also be included, but is not needed, and will trigger the experimental feature check.
+
+- [Fixed-output derivations][fixed-output derivation]
+
+  All of [`outputHash`], [`outputHashAlgo`], and [`outputHashMode`].
+
+  <!--
+
+  `__contentAddressed` is ignored, becaused fixed-output derivations always content-address their outputs, by definition.
+
+  **TODO CHECK**
+
+  -->
+
+- [(Floating) content-addressing derivations](@docroot@/store/derivation/outputs/content-address.md)
+
+  Both [`outputHashAlgo`] and [`outputHashMode`], `__contentAddressed = true;`, and *not* `outputHash`.
+
+  If an output hash was given, then the derivation output would be "fixed" not "floating".
+
+Here is more information on the `output*` attributes, and what values they may be set to:
+
+  - [`outputHashMode`]{#adv-attr-outputHashMode}
+
+    This specifies how the files of a content-addressing derivation output are digested to produce a content address.
+
+    This works in conjunction with [`outputHashAlgo`](#adv-attr-outputHashAlgo).
+    Specifying one without the other is an error (unless [`outputHash` is also specified and includes its own hash algorithm as described below).
+
+    The `outputHashMode` attribute determines how the hash is computed.
+    It must be one of the following values:
+
+      - [`"flat"`](@docroot@/store/store-object/content-address.md#method-flat)
+
+        This is the default.
+
+      - [`"recursive"` or `"nar"`](@docroot@/store/store-object/content-address.md#method-nix-archive)
+
+        > **Compatibility**
+        >
+        > `"recursive"` is the traditional way of indicating this,
+        > and is supported since 2005 (virtually the entire history of Nix).
+        > `"nar"` is more clear, and consistent with other parts of Nix (such as the CLI),
+        > however support for it is only added in Nix version 2.21.
+
+      - [`"text"`](@docroot@/store/store-object/content-address.md#method-text)
+
+        > **Warning**
+        >
+        > The use of this method for derivation outputs is part of the [`dynamic-derivations`][xp-feature-dynamic-derivations] experimental feature.
+
+      - [`"git"`](@docroot@/store/store-object/content-address.md#method-git)
+
+        > **Warning**
+        >
+        > This method is part of the [`git-hashing`][xp-feature-git-hashing] experimental feature.
+
+    See [content-addressing store objects](@docroot@/store/store-object/content-address.md) for more information about the process this flag controls.
+
+  - [`outputHashAlgo`]{#adv-attr-outputHashAlgo}
+
+    This specifies the hash alorithm used to digest the [file system object] data of a content-addressing derivation output.
+
+    This works in conjunction with [`outputHashMode`](#adv-attr-outputHashAlgo).
+    Specifying one without the other is an error (unless [`outputHash` is also specified and includes its own hash algorithm as described below).
+
+    The `outputHashAlgo` attribute specifies the hash algorithm used to compute the hash.
+    It can currently be `"blake3"`, "sha1"`, `"sha256"`, `"sha512"`, or `null`.
+
+    `outputHashAlgo` can only be `null` when `outputHash` follows the SRI format, because in that case the choice of hash algorithm is determined by `outputHash`.
+
+  - [`outputHash`]{#adv-attr-outputHashAlgo}; [`outputHash`]{#adv-attr-outputHashMode}\
+
+    This will specify the output hash of the single output of a [fixed-output derivation].
+
+    The `outputHash` attribute must be a string containing the hash in either hexadecimal or "nix32" encoding, or following the format for integrity metadata as defined by [SRI](https://www.w3.org/TR/SRI/).
+    The "nix32" encoding is an adaptation of base-32 encoding.
+
+    > **Note**
+    >
+    > The [`convertHash`](@docroot@/language/builtins.md#builtins-convertHash) function shows how to convert between different encodings.
+    > The [`nix-hash` command](../command-ref/nix-hash.md) has information about obtaining the hash for some contents, as well as converting to and from encodings.
+
+  - [`__contentAddressed`]{#adv-attr-__contentAddressed}
+
+    > **Warning**
+    >
+    > This attribute is part of an [experimental feature](@docroot@/development/experimental-features.md).
+    >
+    > To use this attribute, you must enable the
+    > [`ca-derivations`][xp-feature-ca-derivations] experimental feature.
+    > For example, in [nix.conf](../command-ref/conf-file.md) you could add:
+    >
+    > ```
+    > extra-experimental-features = ca-derivations
+    > ```
+
+    This is a boolean with a default of `false`.
+    It determines whether the derivation is floating content-addressing.
+
+[`__contentAddressed`]: #adv-attr-__contentAddressed
+[`outputHash`]: #adv-attr-outputHash
+[`outputHashAlgo`]: #adv-attr-outputHashAlgo
+[`outputHashMode`]: #adv-attr-outputHashMode
+
+[fixed-output derivation]: @docroot@/glossary.md#gloss-fixed-output-derivation
+[file system object]: @docroot@/store/file-system-object.md
+[store object]: @docroot@/store/store-object.md
 [xp-feature-dynamic-derivations]: @docroot@/development/experimental-features.md#xp-feature-dynamic-derivations
 [xp-feature-git-hashing]: @docroot@/development/experimental-features.md#xp-feature-git-hashing

doc/manual/source/language/derivations.md

@@ -1,7 +1,7 @@
 # Derivations
 
 The most important built-in function is `derivation`, which is used to describe a single store-layer [store derivation].
-Consult the [store chapter](@docroot@/store/drv.md) for what a store derivation is;
+Consult the [store chapter](@docroot@/store/derivation/index.md) for what a store derivation is;
 this section just concerns how to create one from the Nix language.
 
 This builtin function takes as input an attribute set, the attributes of which specify the inputs to the process.
@@ -16,7 +16,7 @@ It outputs an attribute set, and produces a [store derivation] as a side effect
 - [`name`]{#attr-name} ([String](@docroot@/language/types.md#type-string))
 
   A symbolic name for the derivation.
-  See [derivation outputs](@docroot@/store/drv.md#outputs) for what this is affects.
+  See [derivation outputs](@docroot@/store/derivation/index.md#outputs) for what this is affects.
 
   [store path]: @docroot@/store/store-path.md
 
@@ -34,7 +34,7 @@ It outputs an attribute set, and produces a [store derivation] as a side effect
 
 - [`system`]{#attr-system} ([String](@docroot@/language/types.md#type-string))
 
-  See [system](@docroot@/store/drv.md#system).
+  See [system](@docroot@/store/derivation/index.md#system).
 
   > **Example**
   >
@@ -64,7 +64,7 @@ It outputs an attribute set, and produces a [store derivation] as a side effect
 
 - [`builder`]{#attr-builder} ([Path](@docroot@/language/types.md#type-path) | [String](@docroot@/language/types.md#type-string))
 
-  See [builder](@docroot@/store/drv.md#builder).
+  See [builder](@docroot@/store/derivation/index.md#builder).
 
   > **Example**
   >
@@ -113,7 +113,7 @@ It outputs an attribute set, and produces a [store derivation] as a side effect
 
   Default: `[ ]`
 
-  See [args](@docroot@/store/drv.md#args).
+  See [args](@docroot@/store/derivation/index.md#args).
 
   > **Example**
   >

doc/manual/source/store/building.md

@@ -10,7 +10,7 @@
 
 ## Builder Execution
 
-The [`builder`](./drv.md#builder) is executed as follows:
+The [`builder`](./derivation/index.md#builder) is executed as follows:
 
 - A temporary directory is created under the directory specified by
   `TMPDIR` (default `/tmp`) where the build will take place. The