add maintainer scripts for haskell package generation #121391
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| #! /usr/bin/env nix-shell | ||
| #! nix-shell -i bash -p coreutils haskellPackages.cabal2nix-latest -I nixpkgs=. | ||
|
|
||
| # This script is used to regenerate nixpkgs' Haskell package set, using a tool | ||
| # called hackage2nix. hackage2nix looks at | ||
| # pkgs/development/haskell-modules/configuration-hackage2nix.yaml and generates | ||
| # a Nix expression for package version specified there, using the Cabal files | ||
| # from the Hackage database (available under all-cabal-hashes) and its | ||
| # companion tool cabal2nix. | ||
| # | ||
| # Related scripts are update-hackage.sh, for updating the snapshot of the | ||
| # Hackage database used by hackage2nix, and update-cabal2nix-latest.sh, | ||
| # for updating the version of hackage2nix used to perform this task. | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| extractionDerivation='with import ./. {}; runCommand "unpacked-cabal-hashes" { } "tar xf ${all-cabal-hashes} --strip-components=1 --one-top-level=$out"' | ||
| unpacked_hackage="$(nix-build -E "$extractionDerivation" --no-out-link)" | ||
|
|
||
| hackage2nix --hackage "$unpacked_hackage" --preferred-versions <(for n in "$unpacked_hackage"/*/preferred-versions; do cat "$n"; echo; done) --nixpkgs "$PWD" --config pkgs/development/haskell-modules/configuration-hackage2nix.yaml |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| #! /usr/bin/env nix-shell | ||
| #! nix-shell -i bash -p coreutils curl jq gnused haskellPackages.cabal2nix-latest -I nixpkgs=. | ||
|
|
||
| # Updates cabal2nix-latest to the latest master of the nixos/cabal2nix repository. | ||
| # See regenerate-hackage-packages.sh for details on the purpose of this script. | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| # fetch current master HEAD from Github | ||
| head_info="$(curl -H "Accept: application/vnd.github.v3+json" https://api.github.com/repos/NixOS/cabal2nix/branches/master)" | ||
| # extract commit hash | ||
| commit="$(jq -r .commit.sha <<< "$head_info")" | ||
| # extract commit timestamp and convert to date | ||
| date="$(date "--date=$(jq -r .commit.commit.committer.date <<< "$head_info")" +%F)" | ||
| # generate nix expression from cabal file, replacing the version with the commit date | ||
| echo '# This file defines cabal2nix-latest, used by maintainers/scripts/haskell/regenerate-hackage-packages.sh.' > pkgs/development/haskell-modules/cabal2nix-latest.nix | ||
| cabal2nix "https://github.com/NixOS/cabal2nix/archive/$commit.tar.gz" | sed -e 's/version = ".*"/version = "'"$date"'"/' >> pkgs/development/haskell-modules/cabal2nix-latest.nix |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| #! /usr/bin/env nix-shell | ||
| #! nix-shell -i bash -p nix curl jq nix-prefetch-github -I nixpkgs=. | ||
|
|
||
| # See regenerate-hackage-packages.sh for details on the purpose of this script. | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| pin_file=pkgs/data/misc/hackage/pin.json | ||
| current_commit="$(jq -r .commit $pin_file)" | ||
| head_commit="$(curl -H "Accept: application/vnd.github.v3+json" https://api.github.com/repos/commercialhaskell/all-cabal-hashes/branches/hackage | jq -r .commit.sha)" | ||
|
|
||
| if [ "$current_commit" != "$head_commit" ]; then | ||
| url="https://github.com/commercialhaskell/all-cabal-hashes/archive/$head_commit.tar.gz" | ||
| hash="$(nix-prefetch-url "$url")" | ||
| jq -n \ | ||
| --arg commit "$head_commit" \ | ||
| --arg hash "$hash" \ | ||
| --arg url "$url" \ | ||
| '{commit: $commit, url: $url, sha256: $hash}' \ | ||
| > $pin_file | ||
| fi |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| #! /usr/bin/env bash | ||
|
|
||
| set -eu -o pipefail | ||
|
|
||
| tmpfile=$(mktemp "update-stackage.XXXXXXX") | ||
| # shellcheck disable=SC2064 | ||
| trap "rm ${tmpfile} ${tmpfile}.new" 0 | ||
|
|
||
| curl -L -s "https://stackage.org/nightly/cabal.config" >"$tmpfile" | ||
| version=$(sed -rn "s/^--.*http:..(www.)?stackage.org.snapshot.nightly-//p" "$tmpfile") | ||
|
|
||
| # Create a simple yaml version of the file. | ||
| sed -r \ | ||
| -e '/^--/d' \ | ||
| -e 's|^constraints:||' \ | ||
| -e 's|^ +| - |' \ | ||
| -e 's|,$||' \ | ||
| -e '/installed$/d' \ | ||
| -e '/^$/d' \ | ||
| < "${tmpfile}" | sort --ignore-case >"${tmpfile}.new" | ||
|
|
||
| # Drop restrictions on some tools where we always want the latest version. | ||
| sed -r \ | ||
| -e '/ cabal-install /d' \ | ||
| -e '/ cabal2nix /d' \ | ||
| -e '/ cabal2spec /d' \ | ||
| -e '/ distribution-nixpkgs /d' \ | ||
| -e '/ git-annex /d' \ | ||
| -e '/ hindent /d' \ | ||
| -e '/ hledger/d' \ | ||
| -e '/ hlint /d' \ | ||
| -e '/ hoogle /d' \ | ||
| -e '/ hopenssl /d' \ | ||
| -e '/ jailbreak-cabal /d' \ | ||
| -e '/ json-autotype/d' \ | ||
| -e '/ language-nix /d' \ | ||
| -e '/ shake /d' \ | ||
| -e '/ ShellCheck /d' \ | ||
| -e '/ stack /d' \ | ||
| -e '/ weeder /d' \ | ||
| < "${tmpfile}.new" > "${tmpfile}" | ||
|
|
||
| # Drop the previous configuration ... | ||
| # shellcheck disable=SC1004 | ||
| sed -e '/ # Stackage Nightly/,/^$/c \TODO\ | ||
| ' -i pkgs/development/haskell-modules/configuration-hackage2nix.yaml | ||
|
|
||
| # ... and replace it with the new one. | ||
| sed -e "/TODO/r $tmpfile" \ | ||
| -e "s/TODO/ # Stackage Nightly $version/" \ | ||
| -i pkgs/development/haskell-modules/configuration-hackage2nix.yaml | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,6 +1,10 @@ | ||
| # Hackage database snapshot, used by maintainers/scripts/regenerate-hackage-packages.sh | ||
| # and callHackage | ||
| { fetchurl }: | ||
| let | ||
| pin = builtins.fromJSON (builtins.readFile ./pin.json); | ||
| in | ||
| fetchurl { | ||
| inherit (pin) url sha256; | ||
| passthru.updateScript = ../../../../maintainers/scripts/haskell/update-hackage.sh; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| { | ||
| "commit": "f04ff9179e228b67aee1241fe8c6508f91e92fb5", | ||
| "url": "https://github.com/commercialhaskell/all-cabal-hashes/archive/f04ff9179e228b67aee1241fe8c6508f91e92fb5.tar.gz", | ||
| "sha256": "11k786zcj21hgjxd2fvfshllxdl1z252x01wvfs16wp66n720wly" | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| # This file defines cabal2nix-latest, used by maintainers/scripts/haskell/regenerate-hackage-packages.sh. | ||
| { mkDerivation, aeson, ansi-wl-pprint, base, bytestring, Cabal | ||
| , containers, deepseq, directory, distribution-nixpkgs, fetchzip | ||
| , filepath, hackage-db, hopenssl, hpack, language-nix, lens, lib | ||
| , monad-par, monad-par-extras, mtl, optparse-applicative, pretty | ||
| , process, split, tasty, tasty-golden, text, time, transformers | ||
| , yaml | ||
| }: | ||
| mkDerivation { | ||
| pname = "cabal2nix"; | ||
| version = "2021-05-01"; | ||
| src = fetchzip { | ||
| url = "https://github.com/NixOS/cabal2nix/archive/849a3507c849e3e2331efbc5ebe391b70a215ddc.tar.gz"; | ||
| sha256 = "0g91d2bd72l3kkykc47a2raymvgw6427n7cg9ayzvrpldkd0silc"; | ||
| }; | ||
| isLibrary = true; | ||
| isExecutable = true; | ||
| libraryHaskellDepends = [ | ||
| aeson ansi-wl-pprint base bytestring Cabal containers deepseq | ||
| directory distribution-nixpkgs filepath hackage-db hopenssl hpack | ||
| language-nix lens optparse-applicative pretty process split text | ||
| time transformers yaml | ||
| ]; | ||
| executableHaskellDepends = [ | ||
| aeson base bytestring Cabal containers directory | ||
| distribution-nixpkgs filepath hopenssl language-nix lens monad-par | ||
| monad-par-extras mtl optparse-applicative pretty | ||
| ]; | ||
| testHaskellDepends = [ | ||
| base Cabal containers directory filepath language-nix lens pretty | ||
| process tasty tasty-golden | ||
| ]; | ||
| preCheck = '' | ||
| export PATH="$PWD/dist/build/cabal2nix:$PATH" | ||
| export HOME="$TMPDIR/home" | ||
| ''; | ||
| homepage = "https://github.com/nixos/cabal2nix#readme"; | ||
| description = "Convert Cabal files into Nix build instructions"; | ||
| license = lib.licenses.bsd3; | ||
| } | ||
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now that we're on Stackage Nightly, I imagine we can drop quite a few of these lines here, since these packages will generally also be the latest version on Hackage. That is to say, the version of the package in Stackage Nightly should also be the latest version of the package on Hackage.
If the packages in Stackage Nightly are not the latest version on Hackage, then the Stackage maintainers have probably done that for a good reason. For instance, the packages aren't compatible and don't compile together.
Although this is not something that should block this PR. We can do this at some point in the future.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should drop every one of these except the ones that are nixpkgs-related
jailbreak-cabal,cabal2nix, ...There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, let’s clean that up. But that requires a bit of careful deliberation to see if we break anything right now, so I would like this to happen in an independent PR.