typst: add initial support for typst packages

[cherrypiejam]

This PR adds initial support to build Typst documents with Typst packages. In particular, it adds a Typst package builder and exposes it at the top level to ease the process of defining new Typst packages. It further adds a new function to the Typst derivation for creating a new Typst environment in which the Typst compiler understands a specific set of packages, namely typst.withPackages. Moreover, three Typst packages are defined and collected as a single attribution set as an initial demonstration.

Only a few Typst packages are included in this PR for a reason. The Typst Universe (the official site for Typst packages) maintains a single git repo that contains all Typst packages. Each package entry does not necessarily map to the same version specified in their respective git repo published by their authors (if there is). This is because the package authors need to manually move their package and copy the source, instead of a pointer, to the Typst Universe repo. Breaking the Typst Universe repo down to per package requires separate places to store each of them, and due to the aforementioned reason, the exact source may disagree with the original git repo ¹. This is why, in my opinion, we might want to collect our own set of Typst packages instead of directly pulling from the Typst Universe repo. For gradual adaption, typst.withPackages also provides a method to override the predefined set of Typst packages, namely typst.withPackages.override, making it easier for end users overriding the existing set of Typst packages and integrating new packages into their project.

All packages from the Typst Universe are included in this PR, along with a maintenance script. The script simply generates a set of nix expressions for all packages from the Typst Universe. Each individual package is now fetched as a tarball from the Typst Universe instead of their individual repository.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

Footnotes

1. Assuming the author(s) also publish their package in another repo outside of the Typst Universe repo. ↩

[SigmaSquadron]

Welcome to Nixpkgs! This is truly amazing; I've wanted to build Typst packages for a very long time but never got around to it.

Note that the commits should be split as much as possible. Each new typstPackage should be added in a separate commit. The changes to the base typst package to wrap it should also be a separate commit.

[SigmaSquadron]

Regarding the actual package set, even though we can't import from Universe directly, it might still be a good idea to scrape it for information and build packages automatically. typst.toml gives us all of the necessary information to build each package.

[drupol]

Hello,

Thanks for tackling this, I have a few comments, let me know if you have questions.

@jtojnar : I know you're busy these days, but if you could have a look at this, it would be great.

Thanks!

[cherrypiejam]

Regarding the actual package set, even though we can't import from Universe directly, it might still be a good idea to scrape it for information and build packages automatically. typst.toml gives us all of the necessary information to build each package.

Ah that is great. Didn't know they already have that information included. I'll work on the script when I got time, but it will have to be waited for another week or so.

[SigmaSquadron]

Looks good to me, just one final open question for anyone who knows about packagesFromDirectoryRecursive.

[drupol]

Would it be a good idea to have some documentation for this ?

[cherrypiejam]

Would it be a good idea to have some documentation for this ?

I can write it, but where should I put it?

(Module updates) Added a release notes entry if the change is significant

Not sure if this change should be considered as significant

[jtojnar]

Looks like the archive content still is not immutable, though: typst/packages#2043 (comment)

[cherrypiejam]

Looks like the archive content still is not immutable, though: typst/packages#2043 (comment)

They are not. Packages are allowed to have minor fixes. But for now I suspect we no longer suffer from hash instability using fetchzip because I updated the Typst Universe packages today and there is no hash changes (only new packages)

[philiptaron]

Given the number of packages added, should this target staging? cc @mweinelt and @vcunat.

I see eight builds failing from typstPackages:

• typstPackages.glossarium_0_2_3
• typstPackages.glossarium_0_2_4
• typstPackages.numty_0_0_5
• typstPackages.glossarium_0_2_0
• typstPackages.glossarium_0_2_5
• typstPackages.glossarium_0_1_0
• typstPackages.ncku-later_0_1_0
• typstPackages.songb_0_1_0

I was surprised to see so many previous releases in this package set.

[philiptaron]

Does this too want to be lib.extendMkDerivation, this time with buildEnv as the build helper?

[mweinelt]

The builder seems to be mostly copying stuff around, so it is probably fine to take into master.

Please make sure to fix or disable the broken packages before going for the merge.

[cherrypiejam]

Please make sure to fix or disable the broken packages before going for the merge.

Here is the thing. Since the broken packages is due to the upstream changing their content (and thus their hash), if we can mark the package is broken, we can update its hash to unbroken it. So we can do nothing more about it --- we can always make sure the package is latest before merge but they are going to be broken, soon or later, when the upstream updates the package content

[philiptaron]

After doing a build on x86_64-linux, assuming it goes well, I intend to merge.

[philiptaron]

I think gaining access to the typst universe is worth more than whatever nits I might make. All packages save for a few old ones build successfully.

[nim65s]

fwiw I've been using this a bit and I'm super happy with it. Thanks ! :)

[cherrypiejam]

Is there a CI to keep packages up-to-date for nixpkgs? Also, lmk if there's anything else I need to fix if not merging it

[mweinelt]

Yes, if an individual package has a passthru.updateScript configured it may receive automatic update PRs.

There is nothing like this for whole package sets at once.

[RossSmyth]

Oh I did not think to look here, I made my own little library for building Typst packages a little bit ago
https://github.com/RossSmyth/press

integrating into nixpkgs is much nicer though. The main difference is that my library allows to put packages into custom namespaces.

[cherrypiejam]

Oh I did not think to look here, I made my own little library for building Typst packages a little bit ago https://github.com/RossSmyth/press

integrating into nixpkgs is much nicer though. The main difference is that my library allows to put packages into custom namespaces.

This is very nice! It would complete the support to build Typst documents (and can perhaps reuse infrastructures provided by this PR) in Nix :) This PR aims to provide a way to instantiate a Typst environment that contains packages from Typst Universe --- meaning that to build a Typst document, you would still need to write the primitive mkDerivation and handle all the local Typst packages/fonts in buildPhase with Typst compiler you own.

[drupol]

I merged this PR since it has 3 approvals. Congratulations on all the work done in here!

[NotAShelf]

Nice to finally see this live 🎉

[RossComputerGuy]

"Build Nixpkgs Manual v2" action is failing since this PR was merged.

 Error: Code block mismatch detected in: /nix/store/q7hw0h0ymg8gbdmihmqr292l2njxncsv-doc/languages-frameworks/typst.section.md (preset: nixfmt, language: nix)
Validate, execute and optionally rewrite code blocks in Markdown files based on arbitrary commands

Usage: mdcr [OPTIONS] --config <CONFIG> <PATH>

Arguments:
  <PATH>  Path to the Markdown file or directory

Options:
      --config <CONFIG>  Path to the config JSON file
      --check            Run in check mode (do not write changes)
      --dry-run          Simulate changes, but don't write files
      --log <LOG>        Log level (error, warn, info, debug, trace) [default: warn] [possible values: error, warn, info, debug, trace]
      --verbose          Verbose mode (set the log level to `trace`)
  -h, --help             Print help
  -V, --version          Print version
Error:  command failed. Please make sure the Nix code snippets in Markdown files are correctly formatted.

Run this command from the Nixpkgs repository root for automatic formatting:

    mdcr --log debug --config /home/runner/work/nixpkgs/nixpkgs/doc/tests/mdcr-config.toml /home/runner/work/nixpkgs/nixpkgs/doc

[drupol]

@cherrypiejam The file pkgs/by-name/ty/typst/typst-packages-from-universe.toml is 3 month old in NixOS/nixpkgs. Is there an automated procedure to update it ?

[cherrypiejam]

@cherrypiejam The file pkgs/by-name/ty/typst/typst-packages-from-universe.toml is 3 month old in NixOS/nixpkgs. Is there an automated procedure to update it ?

There's an script ./maintainers/scripts/update-typst-packages.py for that. It will create typst-packages-from-universe.toml under the current working directory.

It wouldn't be 3 month old tho cuz I think I updated it one a month ago. I am happy to integrate it if there's a CI for automatic nixpkgs updating

[drupol]

@cherrypiejam The file pkgs/by-name/ty/typst/typst-packages-from-universe.toml is 3 month old in NixOS/nixpkgs. Is there an automated procedure to update it ?

There's an script ./maintainers/scripts/update-typst-packages.py for that. It will create typst-packages-from-universe.toml under the current working directory.

It wouldn't be 3 month old tho cuz I think I updated it one a month ago. I am happy to integrate it if there's a CI for automatic nixpkgs updating

Yeah we should automate the updates. Perhaps @emaryn could help ?

[cherrypiejam]

Run this in typst's update script? Currently, most update scripts are for single packages.

What's the constraint here tho? This script could be run less frequently (bidaily or weekly?) since it fetches all packages from the Typst universe

[cherrypiejam]

This works locally

    updateScript = {
      command = [
        (writeShellScript "update-typst.sh" ''
          currentVersion=$(nix-instantiate --eval -E "with import ./. {}; typst.version or (lib.getVersion typst)" | tr -d '"')
          ${lib.getExe nix-update} typst > /dev/null
          latestVersion=$(nix-instantiate --eval -E "with import ./. {}; typst.version or (lib.getVersion typst)" | tr -d '"')
          changes=()
          if [[ "$currentVersion" != "$latestVersion" ]]; then
            changes+=("{\"attrPath\":\"typst\",\"oldVersion\":\"$currentVersion\",\"newVersion\":\"$latestVersion\",\"files\":[\"pkgs/by-name/ty/typst/package.nix\"]}")
          fi
          maintainers/scripts/update-typst-packages.py --output pkgs/by-name/ty/typst/typst-packages-from-universe.toml > /dev/null
          ${lib.getExe gitMinimal} diff --quiet HEAD -- pkgs/by-name/ty/typst/typst-packages-from-universe.toml || changes+=("{\"attrPath\":\"typstPackages\",\"oldVersion\":\"0\",\"newVersion\":\"1\",\"files\":[\"pkgs/by-name/ty/typst/typst-packages-from-universe.toml\"]}")
          echo -n "["
            IFS=,; echo -n "''${changes[*]}"
          echo "]"
        '')
      ];
      supportedFeatures = [ "commit" ];
    };

Oh nice. Is this attribute defined under the typst package? If it works, maybe we should just merge it. Alternatively I could do a sanity check on my side