Skip to content

WIP: bun: init configHook and fetchDeps; bun: switch to finalAttrs from rec#376299

Open
Eveeifyeve wants to merge 4 commits intoNixOS:masterfrom
Eveeifyeve:bun-hooks
Open

WIP: bun: init configHook and fetchDeps; bun: switch to finalAttrs from rec#376299
Eveeifyeve wants to merge 4 commits intoNixOS:masterfrom
Eveeifyeve:bun-hooks

Conversation

@Eveeifyeve
Copy link
Member

@Eveeifyeve Eveeifyeve commented Jan 24, 2025

Adds bun.fetchDeps & bun.configHook to bun and updates bun to 1.2.
Blocked by Mic92/nix-update#557 for nix-update-script's.
This pr will lead up to the building bun from source.

NOTE: If you have something to say in this pr please think about if it's spam, off-topic, has already been said (which may have a comment about why I have resolved it) or would be preferred to be a message in matrix, before commenting as this pr. As this pr is becoming big with a lot of people subscribed and I am trying to keep it organized, maintained and not ping heavy.

All related issues that it closes:

closes #335534
closes #255890
closes #414837
closes #424038

This work is semi sponsored by DigitalBrewStudios.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added 8.has: documentation This PR adds or changes documentation 6.topic: nodejs Node.js is a free, open-source, cross-platform JavaScript runtime environment labels Jan 24, 2025
@nix-owners nix-owners bot requested a review from winterqt January 24, 2025 04:54
@Eveeifyeve Eveeifyeve marked this pull request as draft January 24, 2025 04:57
@Eveeifyeve

This comment was marked as spam.

@Eveeifyeve

This comment was marked as outdated.

@github-actions github-actions bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Jan 24, 2025
@Eveeifyeve Eveeifyeve force-pushed the bun-hooks branch 2 times, most recently from 5b658cf to fac0e2a Compare January 24, 2025 05:12
Copy link
Member

@winterqt winterqt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To even begin reviewing this, this needs to be properly split up into multiple commits; ideally, the Bun upgrade would be done in a separate PR entirely.

@Eveeifyeve

This comment was marked as outdated.

@Eveeifyeve

This comment was marked as outdated.

@Eveeifyeve

This comment was marked as outdated.

@Eveeifyeve Eveeifyeve changed the title bun: init configHook and fetchDeps; bun 1.1.43 -> 1.2.0 bun: init configHook and fetchDeps; bun 1.1.43 -> 1.2.0; bun: switch to finalAttrs from rec Jan 24, 2025
@Eveeifyeve Eveeifyeve force-pushed the bun-hooks branch 3 times, most recently from a72a721 to 30e4175 Compare January 25, 2025 11:31
@Eveeifyeve

This comment was marked as outdated.

@getchoo
Copy link
Member

getchoo commented Jan 27, 2025

I am still keeping this draft for now, because of one waiting for #376054 to merged

Merged

@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. and removed 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Feb 3, 2026
@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. and removed 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Feb 5, 2026
@nyabinary nyabinary mentioned this pull request Feb 19, 2026
13 tasks
@Eveeifyeve
Copy link
Member Author

Eveeifyeve commented Feb 21, 2026

Just re-requesting for reviews that are outdated from the following reviewers or have been made a topic already about and have been marked resolve because it's a duplicate.

I am also cleaning up this pr for comments resolved, outdated or should have been asked in the matrix. UPDATE: cleaned up and made a message nice an clear in pr description.

@kuflierl
Copy link
Member

kuflierl commented Feb 21, 2026

You do realise the code isn't evaluating right?

@Eveeifyeve

This comment was marked as resolved.

Comment on lines +25 to +33
bunWorkspaces = [ "./packages/opencode" ];
bunDeps = bun.fetchDeps {
inherit (finalAttrs)
pname
version
src
bunWorkspaces
;
hash = "sha256-HGoeaGrrWYIuaI9yEQJNadCUmmsgFYiQW56ginUa1Bk=";
Copy link
Contributor

@delafthi delafthi Feb 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change will probably break opencode-desktop since it inherits node_modules from opencode and depends on more than just ./packages/opencode. However, I'm currently not a fan of how opencode and opencode-desktop are two separate packages but depend so heavily on each other (both need to be updated at the same time).

Copy link
Member Author

@Eveeifyeve Eveeifyeve Feb 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only thing that is breaking is opencode dependency out of this.
I will probably look into a separate hook for the bun deps for desktop still dependent on src and version from opencode. However probably best in a separate issue to separate the dependent of opencode version src and stuff and make it separate.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only thing is left to do is fix opencode, so I do the hash for desktop.

Copy link
Contributor

@delafthi delafthi Feb 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, I just tested the build and got

specified: sha256-HGoeaGrrWYIuaI9yEQJNadCUmmsgFYiQW56ginUa1Bk=
   got:    sha256-bxDOFMH70AivlhQpxbyDE+hI4372GveEGxjkEQGOA9w=

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even when it unpacks through that phase still broken as the tarball is not unpacking before it gets installed.

Copy link
Contributor

@delafthi delafthi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opencode LGTM. Just added a comment about opencode-desktops dependency to opencode.

@DuskyElf
Copy link
Member

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 376299 --package opencode --package opencode-desktop
Commit: 7d44e4fcff590a2d7b05abc14fbb370b8ea54e05


x86_64-linux

❌ 2 packages failed to build:
  • opencode
  • opencode-desktop
error: hash mismatch in fixed-output derivation '/nix/store/zhx1rfqs5gaigv5h28fwbrm5whhwbgqb-opencode-desktop-1.2.9-bun-deps.tar.gz.drv':
         specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
            got:    sha256-NluoZOsWpt2gdUY69cOnCbmEdve6NoQOTglDDm6GsCM=
error: hash mismatch in fixed-output derivation '/nix/store/86w41zn9b5fc2riak7r5d2d03ar2z9h7-opencode-1.2.9-bun-deps.tar.gz.drv':
         specified: sha256-HGoeaGrrWYIuaI9yEQJNadCUmmsgFYiQW56ginUa1Bk=
            got:    sha256-e1Zio0/ifSbkODcFTGgrueopizTTBWXn5n3vg/Arcw4=

Copy link
Member

@pyrox0 pyrox0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found some grammar fixes and one small change I'd like to be made, or at least considered.


#### Dealing with CPU/OS specific dependencies {#javascript-bun-oscpu}

Some bun dependencies may require OS or CPU architecture specific dependency variations. With the `os` and `cpu` attributes in `fetchDeps`, you can specify which to fetch. By default the fetcher will get all CPU architectures for linux, darwin and freebsd.
Copy link
Member

@pyrox0 pyrox0 Feb 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All other fetchers fetch all dependencies regardless of platform. Is this an option for bun? Making this non-configurable leads to fewer code paths and thus less room for error and bugs to creep in. Also, fetching only specified platforms means if we change some default "blessed" set of platforms, we'd have to regenerate all hashes that depend on that set, whereas fetching all platforms means that hashes won't change in those cases(because they would not exist due to there being no blessed set of platforms we fetch for by default)

Copy link
Member

@kuflierl kuflierl Feb 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The thing is that there is no such default. Bun will by default only pull dependencies for the current system. That makes the output non-deterministic. Also pulling in windows dependencies wouldn't make much sense as these take up a lot of space for nothing. But we also have nixpkgs pkgs that only support one arch or platform. It wouldn't make much sense to also pull in unused code.

These are in essence fetcher parameters and it is expected that the output will change when the input parameters change.

Other fetchers generally only fetch source code and not binaries and are plotform/os agnostic.

@RossSmyth
Copy link
Contributor

This should really use lib.extendMkDerivation and not makeOverridable and such.

@RossSmyth
Copy link
Contributor

RossSmyth commented Feb 22, 2026

The FODs produced are not reproducible. The build fails if you do not have the FOD as it reports a different hash.

Copy link
Contributor

@RossSmyth RossSmyth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think very carefully about the public API. What should be able to be overriden? What shouldn't be? Making the public API as small as possible for the MVP is good because expanding it later is not a breaking change, while reducing it is.

Also giving this a try, the FODs are not reproducible as noted above.

dontConfigure = args'.dontConfigure or true;
dontFixup = args'.dontFixup or true;

inherit outputHash outputHashAlgo;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please actually think about this review instead of saying that is is complicated. It is making it more future-proof.

@Eveeifyeve
Copy link
Member Author

Eveeifyeve commented Feb 22, 2026

The FODs produced are not reproducible. The build fails if you do not have the FOD as it reports a different hash.

For this I have the check with other apps that use bun and see if it's globally producing .npm files. Personally someone had a solution here, but I don't like the use of the arbitrary & seems malicious dd command personally, unless it's okay to use it for this case so I have asked @winterqt to review it, before I go ahead and add it.

There is also some known issues I am trying to fix before this like in the configHook it's not unpacking the tar correctly.

@kuflierl
Copy link
Member

kuflierl commented Feb 23, 2026

For this I have the check with other apps that use bun and see if it's globally producing .npm files. Personally someone had a solution here, but I don't like the use of the arbitrary & seems malicious dd command personally, unless it's okay to use it for this case so I have asked @winterqt to review it, before I go ahead and add it.

The dd command is just writing 4 bytes at offset 96 in the file. Which does fit very nicely with my diff where I found the difference at 0x60. This would be valid as long as we assume that the position of the timestamp is always at that static offset and that we are dealing with a 4 byte little endian encoded unix timestamp. Considering that there were 4 trailing 0 bytes, we might be dealing with an 8 byte value.

We definitely need to look into the binary format.

Also I don't expect the second command restoring the date to really work since it seems to try to read the current date via the date command.

Comment on lines +15 to +26
# `bunDeps` should be a tarball
unpackFile "$bunDeps"
bunDepsCopy="$(realpath "$(stripHash "$bunDeps")")"

echo "Configuring bun store"

HOME=$(mktemp -d)
BUN_INSTALL_CACHE_DIR=$(mktemp -d)
export HOME BUN_INSTALL_CACHE_DIR

cp -Tr "$bunDepsCopy" "$BUN_INSTALL_CACHE_DIR"
chmod -R +w "$BUN_INSTALL_CACHE_DIR"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why exactly are we creating 2 copies of the unpacked FOD? Couldn't we just unpack into a temp dir and point BUN_INSTALL_CACHE_DIR at it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It needs to first unpack the deps to $bunDeps and then next line stripes the hash before it can be copied to the BUN_INSTALL_CACHE_DIR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hold on, aren't we unpacking from bunDeps to the current working directory? The location of the folder is then stored in bunDepsCopy where the content is then copied to Bun_install_cache_Dir.

This seems a bit unnecessarily complicated.

Why not something like this:

HOME=$(mktemp -d)
pushd $(mktemp - d)
unpackFile "$bunDeps"
export BUN_INSTALL_CACHE_DIR="$(realpath "$(stripHash "$bunDeps")")"
popd

Copy link
Member Author

@Eveeifyeve Eveeifyeve Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It supposed to be unpacking to $bunDeps. The reason as to why it's not is because it doesn't have a root dir. Please have a look at @getchoo review if your unsure.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which do you mean?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fi

# `bunDeps` should be a tarball
unpackFile "$bunDeps"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Known problem: This is not unpacking to $bunDeps which is not correct in fetcher.

Eveeifyeve and others added 4 commits February 23, 2026 18:53
Co-authored-by: Winter <winter@winter.cafe>
Co-authored-by: Seth Flynn <getchoo@tuta.io>
Co-authored-by: kuflierl <41301536+kuflierl@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

2.status: merge conflict This PR has merge conflicts with the target branch 6.topic: nodejs Node.js is a free, open-source, cross-platform JavaScript runtime environment 8.has: documentation This PR adds or changes documentation 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 12.approvals: 1 This PR was reviewed and approved by one person. 12.approved-by: package-maintainer This PR was reviewed and approved by a maintainer listed in any of the changed packages.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bun.fetchDeps & bun.ConfigHook Package request: buildBunModule