{nix,lix}: add withNixDebug overrideable#331794
Closed
philiptaron wants to merge 3 commits intoNixOS:masterfrom
Closed
{nix,lix}: add withNixDebug overrideable#331794philiptaron wants to merge 3 commits intoNixOS:masterfrom
philiptaron wants to merge 3 commits intoNixOS:masterfrom
Conversation
The patch was produced by running the following script in the nixpkgs root:
```nix
let
pkgs = import ./. { };
inherit (pkgs) stdenv lib git lixVersions;
derivationsOnly =
name: value:
let
result = builtins.tryEval value;
in
result.success && lib.isDerivation result.value;
derivations = lib.unique (lib.attrValues (lib.filterAttrs derivationsOnly lixVersions));
in
map (
package:
stdenv.mkDerivation {
name = "set-NIX_DEBUG-on-${package.name}.patch";
inherit (package) src;
nativeBuildInputs = [ git ];
buildPhase = ''
export HOME=$(mktemp -d)
git init .
git add -A .
git config --global user.email "nixpkgs"
git config --global user.name "nixpkgs"
git commit -m "initial"
grep -r -l -Z -e 'env."NIX_LOG_FD".*"2"' src | \
xargs -0 sed -i '/NIX_LOG_FD.*2/a \ env["NIX_DEBUG"] = "@NIX_DEBUG@";'
'';
installPhase = ''
git diff | sed '/index/d' > $out
'';
}
) derivations
```
The patch was produced by running the following script in the nixpkgs root:
```nix
let
pkgs = import ./. { };
inherit (pkgs) stdenv lib git nixVersions;
derivationsOnly =
name: value:
let
result = builtins.tryEval value;
in
result.success && lib.isDerivation result.value;
derivations = lib.unique (lib.attrValues (lib.filterAttrs derivationsOnly nixVersions));
in
map (
package:
stdenv.mkDerivation {
name = "set-NIX_DEBUG-on-${package.name}.patch";
inherit (package) src;
nativeBuildInputs = [ git ];
buildPhase = ''
export HOME=$(mktemp -d)
git init .
git add -A .
git config --global user.email "nixpkgs"
git config --global user.name "nixpkgs"
git commit -m "initial"
grep -r -l -Z -e 'env."NIX_LOG_FD".*"2"' src | \
xargs -0 sed -i '/NIX_LOG_FD.*2/a \ env["NIX_DEBUG"] = "@NIX_DEBUG@";'
'';
installPhase = ''
git diff | sed '/index/d' > $out
'';
}
) derivations
```
github-actions
bot
added
the
8.has: documentation
Closed
13 tasks
ofborg
bot
added
10.rebuild-darwin: 0
philiptaron
requested review from
Atemu,
Qyriad,
emilazy,
fricklerhandwerk and
lf-
13 tasks
wegank
added
the
2.status: merge conflict
13 tasks
Member
|
What happened to this? Did you upstream this instead? |
Contributor
Author
|
Nothing happened. I just don't like holding onto PRs that won't be merged. Eventually, we'll have to try out structured logging and performing the filtering in Nix itself. When next I need to do stdenv development, I'll do a similar process to this PR to rebuild Nix or Lix with the |
Member
|
Fair enough, thanks :) |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/community-team-updates/56458/11 |
13 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation for change
The Nixpkgs stdenv documents an environment variable called
NIX_DEBUGbut provides no mechanism to set it. After a thorough search through Nix and Nixpkgs history, it must be set inside Nix when the derivation build environment is concocted.What's the value of this environment variable? It allows the nixpkgs stdenv to produce more verbose logging output, allowing curious users such as myself deeper insight into how the stdenv works and what the impact of my changes might be.
Why does Nix itself have to be recompiled? Any other solution changes the input hash of the derivation, and changing the stdenv's hash means recompiling a lot.
The upstream Nix and Lix projects provide no such functionality to edit the environment variables while building. Nor really should they. This is a fast path to unsoundness in the general case. (We'll leave FODs and impureEnvVars alone.)
What would better look like?
Both projects support structured output logging (through the
@nix { "action": "msg", "level": "[0-7]", "msg": "some message" }special messages output to the file descriptor that$NIX_LOG_FDpoints to.I haven't deeply explored how these log messages show up in the
nix-build,nix build, and family of Nix tools, nor how they get persisted to disk or propagated over the network for remote builders.My intuition, and those of everyone I've spoken to on Matrix about this, is that better looks like using this structured logging functionality to always output diagnostic logging from the nixpkgs stdenv, then using the log level filtering behavior to control how and where it's displayed.
Description of changes
This is a complete hack: a set of patches to each Nix and Lix that allows a variant version that sets the
NIX_DEBUGenvironment value to the user's choice.I (@philiptaron) do not honestly believe this PR will or even ought to be merged. It exists in part to have something that people can search for if they also wonder how the heck to set
NIX_DEBUGto get this diagnostic logging.Things done
nix.conf? (See Nix manual)sandbox = relaxedsandbox = truenix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)