Skip to content

Restore cross-patch-shebangs branch #49608

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
matthewbauer merged 3 commits into from
Nov 7, 2018
Merged

Restore cross-patch-shebangs branch #49608

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
bde9909
Revert "Revert "patch-shebangs: respect cross compilation""
matthewbauer Oct 31, 2018
eb7c50a
patch-shebangs: use --build for auto patch shebangs
matthewbauer Oct 31, 2018
2f2e635
darwin/stdenv: bash is a build input
matthewbauer Oct 31, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 52 additions & 3 deletions pkgs/build-support/setup-hooks/patch-shebangs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,32 @@
# rewritten to /nix/store/<hash>/bin/python. Interpreters that are
# already in the store are left untouched.

fixupOutputHooks+=('if [ -z "$dontPatchShebangs" -a -e "$prefix" ]; then patchShebangs "$prefix"; fi')
fixupOutputHooks+=(patchShebangsAuto)

# Run patch shebangs on a directory.
# patchShebangs [--build | --host] directory

# Flags:
# --build : Lookup commands available at build-time
# --host : Lookup commands available at runtime

# Example use cases,
# $ patchShebangs --host /nix/store/...-hello-1.0/bin
# $ patchShebangs --build configure

patchShebangs() {
local pathName

if [ "$1" = "--host" ]; then
pathName=HOST_PATH
shift
elif [ "$1" = "--build" ]; then
pathName=PATH
shift
fi

local dir="$1"

header "patching script interpreter paths in $dir"
local f
local oldPath
Expand All @@ -27,6 +49,14 @@ patchShebangs() {
oldInterpreterLine=$(head -1 "$f" | tail -c+3)
read -r oldPath arg0 args <<< "$oldInterpreterLine"

if [ -z "$pathName" ]; then
if [ -n "$strictDeps" ] && [[ "$f" = "$NIX_STORE"* ]]; then
pathName=HOST_PATH
else
pathName=PATH
fi
fi

if $(echo "$oldPath" | grep -q "/bin/env$"); then
# Check for unsupported 'env' functionality:
# - options: something starting with a '-'
Expand All @@ -35,14 +65,17 @@ patchShebangs() {
echo "$f: unsupported interpreter directive \"$oldInterpreterLine\" (set dontPatchShebangs=1 and handle shebang patching yourself)"
exit 1
fi
newPath="$(command -v "$arg0" || true)"

newPath="$(PATH="${!pathName}" command -v "$arg0" || true)"
else
if [ "$oldPath" = "" ]; then
# If no interpreter is specified linux will use /bin/sh. Set
# oldpath="/bin/sh" so that we get /nix/store/.../sh.
oldPath="/bin/sh"
fi
newPath="$(command -v "$(basename "$oldPath")" || true)"

newPath="$(PATH="${!pathName}" command -v "$(basename "$oldPath")" || true)"

args="$arg0 $args"
fi

Expand All @@ -65,3 +98,19 @@ patchShebangs() {

stopNest
}

patchShebangsAuto () {
if [ -z "$dontPatchShebangs" -a -e "$prefix" ]; then

if [ -z "${strictDeps-}"]; then
patchShebangs --build "$prefix"
# Dev output will end up being run on the build platform. An
# example case of this is sdl2-config. Otherwise, we can just
# use the runtime path (--host).
elif [ "$output" != out ] && [ "$output" = "${!outputDev}" ]; then
patchShebangs --build "$prefix"
else
patchShebangs --host "$prefix"
fi
fi
}
8 changes: 4 additions & 4 deletions pkgs/stdenv/darwin/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -278,8 +278,8 @@ in rec {
# enables patchShebangs above. Unfortunately, patchShebangs ignores our $SHELL setting
# and instead goes by $PATH, which happens to contain bootstrapTools. So it goes and
# patches our shebangs back to point at bootstrapTools. This makes sure bash comes first.
Copy link
Member

@Ericson2314 Ericson2314 Nov 1, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Edit this comment to match the new reality, too? I also feel like if we can use your disallowRequisites patch with this, we can skip pkgs.bash in either list, which would be best. I'd be very down to collaborate getting all the stdenv deps to use strictDeps :).

Copy link
Member Author

@matthewbauer matthewbauer Nov 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that sounds good! It might have to go in another PR for now - I'm more worried about fixing cross compilation right now.

One thing these are all going to need though is bash in the buildInputs. I think there's quite a few of those that don't have it right now.

I think we could also do an override here that does something like:

stdenv = super.stdenv // { mkDerivation = args: super.stdenv.mkDerivation (args // { strictDeps = true; }); };

To avoid individually setting all of these.

Copy link
Member

@Ericson2314 Ericson2314 Nov 2, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well I do want to audit those packages anyways. I think I've decided this PR and the disallowRequites one are good as is, and then all the per-package cleanup can happen in a 3rd PR.

extraNativeBuildInputs = with pkgs; [ xz pkgs.bash ];
extraBuildInputs = [ pkgs.darwin.CF ];
extraNativeBuildInputs = with pkgs; [ xz ];
extraBuildInputs = [ pkgs.darwin.CF pkgs.bash ];
libcxx = pkgs.libcxx;

extraPreHook = ''
Expand Down Expand Up @@ -335,8 +335,8 @@ in rec {
};
in with prevStage; stageFun 4 prevStage {
shell = "${pkgs.bash}/bin/bash";
extraNativeBuildInputs = with pkgs; [ xz pkgs.bash ];
extraBuildInputs = [ pkgs.darwin.CF ];
extraNativeBuildInputs = with pkgs; [ xz ];
extraBuildInputs = [ pkgs.darwin.CF pkgs.bash ];
libcxx = pkgs.libcxx;

extraPreHook = ''
Expand Down
8 changes: 0 additions & 8 deletions pkgs/stdenv/generic/setup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -257,17 +257,9 @@ shopt -s nullglob

# Set up the initial path.
PATH=
HOST_PATH=
for i in $initialPath; do
if [ "$i" = / ]; then i=; fi
addToSearchPath PATH "$i/bin"

# For backward compatibility, we add initial path to HOST_PATH so
# it can be used in auto patch-shebangs. Unfortunately this will
# not work with cross compilation.
if [ -z "${strictDeps-}" ]; then
addToSearchPath HOST_PATH "$i/bin"
fi
done

if (( "${NIX_DEBUG:-0}" >= 1 )); then
Expand Down