Skip to content

treewide: use writableTmpDirAsHomeHook #378110

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
GaetanLepage merged 14 commits into from
Jan 31, 2025
Merged

treewide: use writableTmpDirAsHomeHook #378110

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
62d4ca6
build-support: update `addBinToPathHook` hook
drupol Jan 30, 2025
ce1e95c
awscli2: use `addBinToPathHook` and `writableTmpDirAsHomeHook`
drupol Jan 30, 2025
e28bd60
meli: use `writableTmpDirAsHomeHook`
drupol Jan 30, 2025
6e7356c
pylyzer: use `writableTmpDirAsHomeHook`
drupol Jan 30, 2025
72cf3eb
shopware-cli: minor cleanup
drupol Jan 30, 2025
f75606e
snapcraft: use `writableTmpDirAsHomeHook`
drupol Jan 30, 2025
e3d1fe0
menulibre: use `writableTmpDirAsHomeHook`
drupol Jan 30, 2025
57e5e28
hedgedoc: use `writableTmpDirAsHomeHook`
drupol Jan 30, 2025
3f1c6f0
hatch: use `writableTmpDirAsHomeHook`
drupol Jan 30, 2025
15fa6be
aws-shell: use `writableTmpDirAsHomeHook`
drupol Jan 30, 2025
9322094
amp: use `writableTmpDirAsHomeHook`
drupol Jan 30, 2025
6f6f573
apx: use `writableTmpDirAsHomeHook`
drupol Jan 30, 2025
a74cba7
podman-tui: reformat + cleanup
drupol Jan 30, 2025
71cfe3b
python312Packages.accelerate: use `addBinToPathHook`, use `writableTm…
drupol Jan 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions pkgs/build-support/setup-hooks/add-bin-to-path.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,8 @@ export PATH

addBinToPath () {
# shellcheck disable=SC2154
if [ -d "$out/bin" ]; then
PATH="$out/bin:$PATH"
export PATH
fi
PATH="$out/bin:$PATH"
export PATH
}

# shellcheck disable=SC2154
Expand Down
8 changes: 5 additions & 3 deletions pkgs/by-name/am/amp/package.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
libgit2,
darwin,
curl,
writableTmpDirAsHomeHook,
}:

rustPlatform.buildRustPackage rec {
Expand All @@ -20,7 +21,7 @@ rustPlatform.buildRustPackage rec {
src = fetchFromGitHub {
owner = "jmacdonald";
repo = "amp";
rev = version;
tag = version;
hash = "sha256-xNadwz2agPbxvgUqrUf1+KsWTmeNh8hJIWcNwTzzM/M=";
};

Expand Down Expand Up @@ -48,8 +49,9 @@ rustPlatform.buildRustPackage rec {
]
);

# Tests need to write to the theme directory in HOME.
preCheck = "export HOME=`mktemp -d`";
nativeCheckInputs = [
writableTmpDirAsHomeHook
];

meta = {
description = "Modern text editor inspired by Vim";
Expand Down
15 changes: 9 additions & 6 deletions pkgs/by-name/ap/apx/package.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
installShellFiles,
distrobox,
podman,
writableTmpDirAsHomeHook,
}:

buildGoModule rec {
Expand All @@ -14,7 +15,7 @@ buildGoModule rec {
src = fetchFromGitHub {
owner = "Vanilla-OS";
repo = "apx";
rev = "v${version}";
tag = "v${version}";
hash = "sha256-60z6wbbXQp7MA5l7LP/mToZftX+nbcs2Mewg5jCFwFk=";
};

Expand All @@ -26,6 +27,10 @@ buildGoModule rec {
podman
];

nativeCheckInputs = [
writableTmpDirAsHomeHook
];
Comment on lines +30 to +32
Copy link
Contributor

@jian-lin jian-lin Feb 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find that this pattern does not work for go packages. The writableTmpDirAsHome bash function is never called and $HOME remains to be HOME=/homeless-shelter. Any ideas? cc @ShamrockLee

Copy link
Contributor

@GaetanLepage GaetanLepage Feb 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tried recently ?
The hook was initially broken but got fixed afterwards.

Copy link
Contributor

@jian-lin jian-lin Feb 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My test is on the master branch 8b564bb.

Copy link
Contributor

@GaetanLepage GaetanLepage Feb 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the right commit ?

Copy link
Contributor

@jian-lin jian-lin Feb 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works if I do one of these things:

  • also add writableTmpDirAsHomeHook to checkInputs
  • disable strictDeps: nix-repl> :b apx.overrideAttrs { strictDeps = false; }

Copy link
Contributor Author

@drupol drupol Feb 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, I have no clue who to contact. Can you ping the relevant nicknames here perhaps?

Copy link
Contributor

@ShamrockLee ShamrockLee Feb 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NixOS/stdenv @NixOS/golang @infinisil @roberth Sorry for bothering, but does anyone know why setup hooks using addEnvHooks "$targetOffset" don't take effect when passing to buildGoModule?

Copy link
Contributor

@jian-lin jian-lin Feb 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NickCao told me how setup hooks work and why this issue happens only for go packages here. Thank you, @NickCao .

The following patch fixes it:

-addEnvHooks "$targetOffset" writableTmpDirAsHome
+addEnvHooks "$hostOffset" writableTmpDirAsHome

Here is the explanation. Assume writableTmpDirAsHomeHook is added to nativeBuildInputs (or nativeCheckInputs). If hostOffset is used, the bash function writableTmpDirAsHome is run against each dependency in depsBuildBuild depsBuildHost(nativeBuildInputs) depsBuildTarget. If targetOffset is used, it is run against each dependency in depsHostHost depsHostTarget(buildInputs). Go packages usually have no dependencies in depsHostHost depsHostTarget(buildInputs), so the bash function writableTmpDirAsHome is never run and we observe this issue. Since we assume writableTmpDirAsHomeHook is added to nativeBuildInputs, there is at least one dependency (the hook itself) in depsBuildBuild depsBuildHost(nativeBuildInputs) depsBuildTarget. As a result, the bash function writableTmpDirAsHome will always run when hostOffset is used.

The implementation of addEnvHooks may be helpful.

Copy link
Contributor Author

@drupol drupol Feb 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in: #378927

Copy link
Contributor Author

@drupol drupol Feb 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for making the investigation, very interesting !!!


ldflags = [
"-s"
"-w"
Expand All @@ -46,8 +51,6 @@ buildGoModule rec {
install -Dm444 README.md -t $out/share/docs/apx
install -Dm444 COPYING.md $out/share/licenses/apx/LICENSE

# Create a temp writable home-dir so apx outputs completions without error
export HOME=$(mktemp -d)
# apx command now works (for completions)
# though complains "Error: no such file or directory"
installShellCompletion --cmd apx \
Expand All @@ -56,12 +59,12 @@ buildGoModule rec {
--zsh <($out/bin/apx completion zsh)
'';

meta = with lib; {
meta = {
description = "Vanilla OS package manager";
homepage = "https://github.com/Vanilla-OS/apx";
changelog = "https://github.com/Vanilla-OS/apx/releases/tag/v${version}";
license = licenses.gpl3Only;
maintainers = with maintainers; [
license = lib.licenses.gpl3Only;
maintainers = with lib.maintainers; [
dit7ya
chewblacka
];
Expand Down
6 changes: 2 additions & 4 deletions pkgs/by-name/aw/aws-shell/package.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
lib,
fetchFromGitHub,
awscli,
writableTmpDirAsHomeHook,
}:

python3Packages.buildPythonApplication rec {
Expand Down Expand Up @@ -38,12 +39,9 @@ python3Packages.buildPythonApplication rec {

nativeCheckInputs = with python3Packages; [
pytestCheckHook
writableTmpDirAsHomeHook
];

preCheck = ''
export HOME=$(mktemp -d)
'';

meta = {
homepage = "https://github.com/awslabs/aws-shell";
description = "Integrated shell for working with the AWS CLI";
Expand Down
9 changes: 4 additions & 5 deletions pkgs/by-name/aw/awscli2/package.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
nix-update-script,
testers,
awscli2,
addBinToPathHook,
writableTmpDirAsHomeHook,
}:

let
Expand Down Expand Up @@ -121,9 +123,11 @@ py.pkgs.buildPythonApplication rec {
];

nativeCheckInputs = with py.pkgs; [
addBinToPathHook
jsonschema
mock
pytestCheckHook
writableTmpDirAsHomeHook
];

postInstall =
Expand All @@ -136,11 +140,6 @@ py.pkgs.buildPythonApplication rec {
rm $out/bin/aws.cmd
'';

preCheck = ''
export PATH=$PATH:$out/bin
export HOME=$(mktemp -d)
'';

# Propagating dependencies leaks them through $PYTHONPATH which causes issues
# when used in nix-shell.
postFixup = ''
Expand Down
6 changes: 2 additions & 4 deletions pkgs/by-name/ha/hatch/package.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
git,
cargo,
versionCheckHook,
writableTmpDirAsHomeHook,
darwin,
nix-update-script,
}:
Expand Down Expand Up @@ -63,17 +64,14 @@ python3Packages.buildPythonApplication rec {
++ [
cargo
versionCheckHook
writableTmpDirAsHomeHook
]
++ lib.optionals stdenv.hostPlatform.isDarwin [
darwin.ps
];

versionCheckProgramArg = [ "--version" ];

preCheck = ''
export HOME=$(mktemp -d);
'';

pytestFlagsArray =
[
# AssertionError on the version metadata
Expand Down
5 changes: 3 additions & 2 deletions pkgs/by-name/he/hedgedoc/package.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
nodejs,
python3,
nixosTests,
writableTmpDirAsHomeHook,
}:

let
Expand All @@ -17,7 +18,7 @@ let
src = fetchFromGitHub {
owner = "hedgedoc";
repo = "hedgedoc";
rev = version;
tag = version;
hash = "sha256-cRIpcoD9WzLYxKYpkvhRxUmeyJR5z2QyqApzWvQND+s=";
};

Expand All @@ -31,10 +32,10 @@ let
gitMinimal # needed to download git dependencies
nodejs # needed for npm to download git dependencies
yarn
writableTmpDirAsHomeHook
];

buildPhase = ''
export HOME=$(mktemp -d)
yarn config set enableTelemetry 0
yarn config set cacheFolder $out
yarn config set --json supportedArchitectures.os '[ "linux" ]'
Expand Down
6 changes: 2 additions & 4 deletions pkgs/by-name/me/meli/package.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
mandoc,
rustfmt,
file,
writableTmpDirAsHomeHook,

# build inputs
openssl,
Expand Down Expand Up @@ -58,6 +59,7 @@ rustPlatform.buildRustPackage rec {
nativeCheckInputs = [
file
gnum4
writableTmpDirAsHomeHook
];

postInstall = ''
Expand All @@ -68,10 +70,6 @@ rustPlatform.buildRustPackage rec {
--prefix PATH : ${lib.makeBinPath [ gnum4 ]}
'';

preCheck = ''
export HOME=$(mktemp -d)
'';

checkFlags = [
"--skip=test_cli_subcommands" # panicking due to sandbox
];
Expand Down
8 changes: 3 additions & 5 deletions pkgs/by-name/me/menulibre/package.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
nix-update-script,
testers,
menulibre,
writableTmpDirAsHomeHook,
}:

python3Packages.buildPythonApplication rec {
Expand All @@ -19,7 +20,7 @@ python3Packages.buildPythonApplication rec {
src = fetchFromGitHub {
owner = "bluesabre";
repo = "menulibre";
rev = "menulibre-${version}";
tag = "menulibre-${version}";
hash = "sha256-IfsuOYP/H3r1GDWMVVSBfYvQS+01VJaAlZu+c05geWg=";
};

Expand All @@ -35,6 +36,7 @@ python3Packages.buildPythonApplication rec {
intltool
gobject-introspection
wrapGAppsHook3
writableTmpDirAsHomeHook
];

postPatch = ''
Expand All @@ -43,10 +45,6 @@ python3Packages.buildPythonApplication rec {
--replace-fail 'update_desktop_file(desktop_file, script_path)' ""
'';

preBuild = ''
export HOME=$TMPDIR
'';

passthru = {
updateScript = nix-update-script { };
tests.version = testers.testVersion {
Expand Down
25 changes: 16 additions & 9 deletions pkgs/by-name/po/podman-tui/package.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,11 @@
{ lib, stdenv, fetchFromGitHub, buildGoModule, testers, podman-tui }:
{
lib,
stdenv,
fetchFromGitHub,
buildGoModule,
testers,
podman-tui,
}:

buildGoModule rec {
pname = "podman-tui";
Expand All @@ -15,15 +22,15 @@ buildGoModule rec {

env.CGO_ENABLED = 0;

tags = [ "containers_image_openpgp" "remote" ]
++ lib.optional stdenv.hostPlatform.isDarwin "darwin";
tags = [
"containers_image_openpgp"
"remote"
] ++ lib.optional stdenv.hostPlatform.isDarwin "darwin";

ldflags = [ "-s" "-w" ];

preCheck = ''
export USER="$(whoami)"
export HOME="$(mktemp -d)"
'';
ldflags = [
"-s"
"-w"
];

checkFlags =
let
Expand Down
10 changes: 4 additions & 6 deletions pkgs/by-name/py/pylyzer/package.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,13 @@
stdenv,
rustPlatform,
fetchFromGitHub,
git,
gitMinimal,
python3,
makeWrapper,
writeScriptBin,
versionCheckHook,
nix-update-script,
writableTmpDirAsHomeHook,
}:

rustPlatform.buildRustPackage rec {
Expand All @@ -26,19 +27,16 @@ rustPlatform.buildRustPackage rec {
cargoHash = "sha256-mi4pCYA0dQnv3MIpZxVVY0qLdIts/qvxS4og0Tyxk3w=";

nativeBuildInputs = [
git
gitMinimal
python3
makeWrapper
writableTmpDirAsHomeHook
] ++ lib.optionals stdenv.hostPlatform.isDarwin [ (writeScriptBin "diskutil" "") ];

buildInputs = [
python3
];

preBuild = ''
export HOME=$(mktemp -d)
'';

postInstall = ''
mkdir -p $out/lib
cp -r $HOME/.erg/ $out/lib/erg
Expand Down
3 changes: 1 addition & 2 deletions pkgs/by-name/sh/shopware-cli/package.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ buildGoModule rec {
src = fetchFromGitHub {
repo = "shopware-cli";
owner = "FriendsOfShopware";
rev = version;
tag = version;
hash = "sha256-nFjm1z7QXuUkZ4sVRY0KdFpo0SXBTmJNA28YJpAyG2w=";
};

Expand All @@ -30,7 +30,6 @@ buildGoModule rec {
vendorHash = "sha256-7DVC68tKoEwaTbF6Lkv1Ib1imZojTPW3G/QS3W6N8ys=";

postInstall = ''
export HOME="$(mktemp -d)"
installShellCompletion --cmd shopware-cli \
--bash <($out/bin/shopware-cli completion bash) \
--zsh <($out/bin/shopware-cli completion zsh) \
Expand Down
11 changes: 4 additions & 7 deletions pkgs/by-name/sn/snapcraft/package.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
{
fetchFromGitHub,
git,
gitMinimal,
glibc,
lib,
makeWrapper,
nix-update-script,
python3Packages,
squashfsTools,
stdenv,
writableTmpDirAsHomeHook,
}:

python3Packages.buildPythonApplication rec {
Expand Down Expand Up @@ -138,17 +139,13 @@ python3Packages.buildPythonApplication rec {
pytestCheckHook
responses
setuptools
writableTmpDirAsHomeHook
]
++ [
git
gitMinimal
squashfsTools
];

preCheck = ''
mkdir -p check-phase
export HOME="$(pwd)/check-phase"
'';

pytestFlagsArray = [ "tests/unit" ];

disabledTests = [
Expand Down
Loading