pkgs(bonfire): init at 1.0.1-beta.11

[ju1m]

This PR packages Bonfire. It is ready for reviewing.
For the service, see stacked PR #1871

Meta
I am editing the intro message to match with the current state of this PR.

State
Build with:

nix -L build -f . bonfire.social

RoadMap

• Fix more dependencies.
• Rename to bonfire-app?
• Enable WITH_IMAGE_VIX to use evision/opencv.
• Add a passthru.updateScript to run deps_nix.
• Get bonfire to not crash at stratup for obvious reasons due to its packaging
• Ping relevant people.
• Test env.FLAVOUR="open_science"
• Fix buildMix/mixRelease, and test them in NGIpkgs before trying to push them into Nixpkgs.

Pings

• @Chickensoupwithrice, who has written a previous packaging of Bonfire
• @ivanminutillo, who has written upstream's flake.nix
• @fishinthecalculator, who has written bonfire-nix and bonfire-guix

Related

• Closes bonfire: Package bonfire for NGIpkgs #1856
• Closes bonfire: Implement an update script for bonfire #1857

Upstream

• module Bonfire.RuntimeConfig is not available bonfire-networks/bonfire-app#1651
• https://github.com/bonfire-networks/bonfire-app/issues/1651 bonfire-networks/bonfire-app#1652
• security/confidentiality: secrets are stored in environment variables bonfire-networks/bonfire-app#1663
• security/confidentiality: pinned ranch is too old to support Unix socket permissions bonfire-networks/bonfire-app#1667
• MacOS Dev Setup Incompatibility bonfire-networks/bonfire-app#1670 (comment)
• typo: cdlr.ex => cldr.ex bonfire-networks/bonfire-app#1676
• ERROR! the application :mime has a different value set for key :extensions during runtime compared to compile time. bonfire-networks/bonfire-app#1696
• Cannot setup the community flavour in non-interactive mode bonfire-networks/bonfire-app#1737

[ju1m]

• Drop deps/mjml.nix leftover from before using deps_nix; deps_nix manages the Rust part correctly alone.

[ju1m]

• Remove projects/bonfire/ to do that in another PR
• Move appConfigPath in another location
• Fix dependency bonfire_federate_activitypub
• Fix dependency bonfire_ui_me
• Fix typo in comments
• Fix/workaround bonfire crash at startup when failing to load some .beam modules, due to bonfire pinning git repositories directly, and deps_nix not setting version for them. Yes…

[ju1m]

• Remove detailed explanation and report Git dependencies' version= is set to rev, causing the app to crash at startup code-supply/deps_nix#35
• Workaround and report rustler's load_from defaults to priv/native/#{crate}, but deps_nix installs into priv/native/lib#{crate} code-supply/deps_nix#36
• Report bonfire-app always loads appsignal's closed-source agent bonfire-networks/bonfire-app#1637
• Contribute the lazy_html fix upstream Workarounds for lazy_html code-supply/deps_nix#33 (comment)

[fishinthecalculator]

awesome stuff @ju1m , are you also working on a NixOS module for Bonfire? I think a lot of code from https://github.com/bonfire-networks/bonfire-nix could be reused, in fact if you are not I can adapt the Bonfire flake to use your packages.

[ju1m]

• Add a passthru.updateScript to run deps_nix. To be used with update bonfire. Note that gitUpdater downgrades -unstable versions. So if you want to test, comment-out gitUpdater until there's a new release of bonfire.
• Enable WITH_IMAGE_VIX to use evision/opencv.
  • Fix (or at least workaround) missing flann symbols errors. And report upstream in undefined flann symbols cocoa-xu/evision#293
• Update to bonfire-1.0.0-unstable-2025-11-17 to avoid packaging its closed-source appsignal dependency.
• Expose passthru.appConfigPath.
• Remove version workaround now that deps_nix has a proper fix.

[ju1m]

@fishinthecalculator, thanks for your support, yes I'm slowly coming to the service part that I'll do in another PR, but first I'd like to finish this PR ie. get to the point where bonfire does not crash at startup for obvious reasons due to its packaging. Big thanks for your work on the service, I cannot tell you what I'll reuse or not yet, but I'll keep you posted!

[ju1m]

Changes

• Clarify updateScript by using a dedicated derivation, renaming deps-update.nix to simply update.nix.
• Fix (partially) module Bonfire.RuntimeConfig is not available bonfire-networks/bonfire-app#1651 and use WITH_FORKS=0 to get ember and social into mix.lock.
• Run update bonfire to get ember and social in deps.nix.
• Fix dependency ember
• Fix dependency social
• Continue to fill meta.

State

result/bin/bonfire start still crashes at startup, but on a new error:

ERROR! the application :mime has a different value set for key :extensions during runtime compared to compile time. Since this application environment entry was marked as compile time, this difference can lead to different behavior than expected:

  * Compile time value was set to: %{"zip" => "application/zip", "livemd" => "text/markdown", "mov" => "video/quicktime", "webm" => "audio/webm", "3g2" => "video/3gpp2", "csv" => "text/csv", "ogv" => "video/ogg", "avi" => "video/x-msvideo", "text" => "text/plain", "mobi" => "application/x-mobipocket-ebook", "m4a" => "audio/mp4", "activity+json" => "application/activity+json", "bzip2" => "application/x-bzip2", "jetpack" => "text/jetpack", "webp" => "image/webp", "m1v" => "video/mpeg", "asc" => "text/plain", "mkd" => "text/markdown", "png" => "image/png", "markdown" => "text/markdown", "mpeg" => "video/mpeg", "txt" => "text/plain", "json" => "application/json", "wav" => "audio/wav", "7z" => "application/x-7z-compressed", "mpg" => "video/mpeg", "3gp" => "video/3gpp", "gz" => "application/gzip", "ld+json" => "application/ld+json", "oga" => "audio/ogg", "rar" => "application/vnd.rar", "vcs" => "application/ics", "mpg4" => "video/mp4", "jpeg" => "image/jpeg", "prc" => "application/x-mobipocket-ebook", "vcf" => "text/x-vcard", "apng" => "image/apng", "mkv" => "application/x-matroska", "flac" => "audio/flac", "rtf" => "application/rtf", "swiftui" => "text/swiftui", "jpg" => "image/jpeg", "styles" => "text/styles", "mp2" => "audio/mpeg", "md" => "text/markdown", "ics" => "application/ics", "gif" => "image/gif", "aac" => "audio/aac", "mpe" => "video/mpeg", "log" => "text/plain", ...}
  * Runtime value was set to: %{"livemd" => "text/markdown", "mov" => "video/quicktime", "webm" => "audio/webm", "3g2" => "video/3gpp2", "csv" => "text/csv", "ogv" => "video/ogg", "avi" => "video/x-msvideo", "text" => "text/plain", "mobi" => "application/x-mobipocket-ebook", "m4a" => "audio/mp4", "activity+json" => "application/activity+json", "webp" => "image/webp", "m1v" => "video/mpeg", "asc" => "text/plain", "mkd" => "text/markdown", "png" => "image/png", "markdown" => "text/markdown", "mpeg" => "video/mpeg", "txt" => "text/plain", "json" => "application/json", "wav" => "audio/wav", "rss+xml" => "application/rss+xml", "mpg" => "video/mpeg", "3gp" => "video/3gpp", "ld+json" => "application/ld+json", "oga" => "audio/ogg", "vcs" => "application/ics", "mpg4" => "video/mp4", "jpeg" => "image/jpeg", "prc" => "application/x-mobipocket-ebook", "vcf" => "text/x-vcard", "apng" => "image/apng", "mkv" => "application/x-matroska", "flac" => "audio/flac", "rtf" => "application/rtf", "jpg" => "image/jpeg", "mp2" => "audio/mpeg", "md" => "text/markdown", "ics" => "application/ics", "gif" => "image/gif", "aac" => "audio/aac", "mpe" => "video/mpeg", "log" => "text/plain", "atom+xml" => "application/atom+xml", "epub" => "application/epub+zip", "mpa" => "video/mpeg", "svg" => "image/svg+xml", "ogg" => "audio/ogg", "jrd+json" => "application/jrd+json", "m2v" => "video/mpeg", ...}

[ju1m]

Changes

• Cleanup bonfire_common by dropping a previous fix no longer needed with the global src workaround in buildMix.
• Package bonfire_editor_milkdown's yarn assets.
• Package bonfire_geolocate's yarn assets.
• Package bonfire_ui_common's yarn assets.
• Cleanup bonfire_ui_common by dropping a previous fix no longer needed with the global src workaround in buildMix.
• Fix ex_cldr by not dropping previousAttrs.postInstall.
• Fix iconify_ex.
• Package iconify_ex's yarn assets.
• Make lazy_html depend on fine.src instead of "${fine}/src", it was required when I was not able to replace $out/src, but now it just pulls fine.src for nothing so I'll likely revert.
• Include appConfigPath inside a broader bonfireSetup that includes configs from the flavour.
  • Tip: bonfire has fake libraries, eg. attempting to compile bonfire.mixNixDeps.bonfire_ui_common is meaningless, it has to be built inside bonfire-app due to paths like ../../../../.
• Workaround mix compile.surface (a compiler generating *.hooks.js from Elixir files) depending on Elixir dependencies being built in $out instead of ephemeral $src (sic). See detailed comments inlined. Related upstream issue is: Fix compiler when using ERL_COMPILER_OPTIONS=deterministic surface-ui/surface#762
  • Tip: to troubleshoot this problem requiring to dive into mix internals, learning a bit of Elixir and using the interactive prompt iex (eg. iex --erl "-kernel shell_history enabled" -S mix compile --no-deps-check) helped a lot.
  • Tip: when blocked, being able to build the package successfully out of nix is a must to keep hope and get a base from which to investigate what had to be done and when, most of the ~1300LoC justfile is bringing no added-value to the nix package, I've only kept just flavour_make_symlinks so far, and it may be replaced entirely at some point in the future if bonfire's way to install a flavour proves to be stable.
• Add a pkgs/by-name/bonfire/test.sh to test the package. This is only to help reviewers, I'll remove it when introducing the service.
• Make updateScript able to update yarnOfflineCaches too, using nurl --expr to compute the hash of those fixed-output derivations. I chose to put the hash in a separated file out of simplicity (avoid the need to call substituteInPlace to replace the old hash).
• Replace mix bonfire.install.copy_migrations --force with a simple for loop, because I was unable to make it work correctly (it was not installing all the SQL migrations). Yet another subtle bug is lurking here, and mix makes it too hard for me to troubleshoot what it is without spending sleepless nights on it again.

State

With a postgresql server configured correctly, pkgs/by-name/bonfire/test.sh builds and run bonfire in its social flavour, accessible in a Web browser at http://localhost:4000. Yurray!

ToDo

• Support other flavours. But probably in another PR, because I don't want to be drained by yet another fight with mix.

[ju1m]

Changes

• Fix fetchYarnDeps's name instead of pname, because it overrides it.
• Fix lazy_html to use ${fine}/src instead of ${fine.src}.
• Remove commented debug code.
• Use the existence of yarnOfflineCache to select yarn dependencies.
• Minor cleanups

[ju1m]

Changes

• Squash all changes into a single commit, to maintain buildability at the commit level, plus it's shorter in the log and remove useless tries when git blame/credit.
• Update to latest alpha version. This version brought some improvements for reverse-proxying to Bonfire via a Unix socket, but another problem had to be reported to upstream in Reverse-proxying with a Unix socket raises exception: (Protocol.UndefinedError) protocol String.Chars not implemented for type Tuple. bonfire-networks/bonfire-app#1698, it may be fixed in a later update.
• Backport fix for ffmpeg-8 into evision's opencv-4.11
• Autoupdate ex_cldr FOD.
• Remove explanations on buildMix fix that will go in the Contributor's Manual, or later Nixpkgs' source core
• Bisect and report to upstream the recent increase in build time of bonfire_social from ~30s to ~30min Mastodon-compatible API bonfire-networks/bonfire-app#916 (comment)
• Remove TZDATA_DIR, that can now be done with BONFIRE_RUNTIME_CONFIG.
• Introduce BONFIRE_RUNTIME_CONFIG to modify config/runtime.exs at runtime (from the NixOS service).
• Keep validate_compile_env: false and report the problem to upstream in ERROR! the application :mime has a different value set for key :extensions during runtime compared to compile time. bonfire-networks/bonfire-app#1696 that may be fixed in a later update.
• Drop pkgs/by-name/bonfire/test.sh now that there is a proper NixOS test in the relevant stacked PR.
• No longer use bonfire.overrideAttrs but bonfire.passthru.beamPackages.mixRelease instead, to avoid loading previous mixNixDeps when updating it. This is a direct consequence of mixRelease not supporting lib.extendMkDerivation, hence not offering the possibility to empty mixNixDeps.
• Ensure the requested mixEnv is used when running mix deps.get and / mix deps.nix.
• Document the update process in the relevant section of the Contributor's Manual.

RoadMap

• Fix buildMix/mixRelease, and test them in NGIpkgs before trying to push them in Nixpkgs.

Remark

• I'm sorry I rebased upon latest main before pushing instead of doing it in a separate force-push, this makes GitHub's force-pushed/Compare buttons even more painful to read than they already notoriously are. If reviewing locally, you may use git range-diff --no-dual-color 815c607e8cd4677b8782d8c9a1818654749da92a b763b6e 8dbdd30 for a workaround.

[ju1m]

Changes

• Support for other flavours (only open_science builds at the moment, see Cannot setup the community flavour in non-interactive mode bonfire-networks/bonfire-app#1737)
• Improve resilience and clarify update scripts, moving the reusable part into profiles/pkgs/development/beam-modules/mix-update.nix.
• Document update process (see manuals/Contributor/How_to/update/pkgs/bonfire.flavour.md in doc: init #1891)
• Remove import-from-derivation by vendoring-in Cargo.lock files, hence deps_nix's support for Rust is not longer used.
• Vendor-in Nixpkgs' pkgs/development/beam-modules/{build-mix,mix-release}.nix in profiles/ and make them use lib.extendMkDerivation to provide finalAttrs, rewrite everything to use finalAttrs as an open-recursive-let scoped to a package, to be overrided with overrideAttrs (much more resilient than override which does not survive callPackage, see manuals/Contributor/How_to/reuse/a_package/with_different_arguments/when_lib_makeOverridable_is_nested_into_another.md).
• Init internal lib that does not require users to do anything for modules to use it (ie. modules use direct import:

lib = import ../../../../lib/default.nix { inherit (args) lib; };

instead of the user having to overlay lib has previously tried in #1927)

• Add lib.types.credential
• Add lib.nixos.systemd.serviceConfig.loadCredential to properly support LoadCredential= or LoadCredentialEncrypted=
• Move pkgs/by-name/bonfire/package.nix to pkgs/by-name/bonfire/generic.nix
  and write a new pkgs/by-name/bonfire/package.nix using lib.recurseIntoAttrs to support multiple flavours of bonfire.

State

nom build -f. bonfire.social

builds bonfire.ember, bonfire.social and bonfire.open_science.

RoadMap

• Support Unix socket whenever upstream will. Code is already there in comments to do so.
• See what to do about shell in profiles/pkgs/development/beam-modules/build-mix.nix.
• Review and merge this PR

[imincik]

@ju1m , I can't see project metadata in this PR - this PR shouldn't close #1765 . Metadata is in #1871 .

[imincik]

Julien, great thanks for your work ! I am not very competent to review this PR, but I went through all files and I can't see any obvious blocker.

[imincik]

@eljamm , are you OK with merging this PR now ?