Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 37 additions & 13 deletions doc/stdenv.xml
Original file line number Diff line number Diff line change
Expand Up @@ -846,8 +846,9 @@ following:
subdirectories of <envar>$out</envar> to
<filename>share/</filename>.</para></listitem>

<listitem><para>It strips libraries and executables of debug
information.</para></listitem>
<listitem><para>It strips libraries and executables of debug information.
This step is crucial to remove references to build inputs. It also reduces
the size of binaries.</para></listitem>

<listitem><para>On Linux, it applies the <command>patchelf</command>
command to ELF executables and libraries to remove unused
Expand All @@ -869,8 +870,27 @@ following:

<varlistentry>
<term><varname>dontStrip</varname></term>
<listitem><para>If set, libraries and executables are not
stripped. By default, they are.</para></listitem>
<listitem><warning><para>Stripping is required to avoid propagating build
inputs as run-time dependencies. Please ensure you understand the
implications before disabling stripping. To disable stripping on selected
files of folders, see <varname>dontStripList</varname> or
<varname>strip{Debug,All}List</varname>.</para></warning><para>If set,
libraries and executables are not stripped. By default, they are.
</para></listitem>
</varlistentry>

<varlistentry>
<term><varname>dontStripList</varname></term>
<listitem><para>List of paths that should not be stripped. Bash patterns
are accepted, and are relative to the package's prefix. For example,
<literal>["share" "modules/*.i" "bin/libannoying.so"]</literal> is an
acceptable value. By default it's empty. Avoid trailing slashes as the
underlying <command>find</command> ignores such paths.</para>
<para>Two good reasons to add paths to this list are when
<command>strip</command> fails on the file, breaking the build, and when
the package requires the debug info. In the latter case, you should ensure
that the unstripped executable does not maintain reference to build inputs.
</para></listitem>
</varlistentry>

<varlistentry>
Expand All @@ -881,11 +901,12 @@ following:

<varlistentry>
<term><varname>stripAllList</varname></term>
<listitem><para>List of directories to search for libraries and
executables from which <emphasis>all</emphasis> symbols should be
stripped. By default, it’s empty. Stripping all symbols is
risky, since it may remove not just debug symbols but also ELF
information necessary for normal execution.</para></listitem>
<listitem><para>List of paths to search for libraries and executables from
which <emphasis>all</emphasis> symbols should be stripped. See
<varname>dontStripList</varname> for detailed information about accepted
values. By default, it’s empty. Stripping all symbols is risky, since it
may remove not just debug symbols but also ELF information necessary for
normal execution.</para></listitem>
</varlistentry>

<varlistentry>
Expand All @@ -898,10 +919,13 @@ following:

<varlistentry>
<term><varname>stripDebugList</varname></term>
<listitem><para>List of directories to search for libraries and
executables from which only debugging-related symbols should be
stripped. It defaults to <literal>lib bin
sbin</literal>.</para></listitem>
<listitem><para>List of paths to search for libraries and executables from
which only debugging-related symbols should be stripped. See
<varname>dontStripList</varname> for detailed information about accepted
values. By default, everything is stripped. The same warning as for
<varname>dontStrip</varname> applies here. For most purposes,
<varname>dontStripList</varname> should be used in place of this option.
</para></listitem>
</varlistentry>

<varlistentry>
Expand Down
81 changes: 63 additions & 18 deletions pkgs/build-support/setup-hooks/strip.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,33 +4,78 @@ fixupOutputHooks+=(_doStrip)

_doStrip() {
if [ -z "$dontStrip" ]; then
stripDebugList=${stripDebugList:-lib lib32 lib64 libexec bin sbin}
if [ -n "$stripDebugList" ]; then
stripDirs "$stripDebugList" "${stripDebugFlags:--S}"
fi

stripAllList=${stripAllList:-}
if [ -n "$stripAllList" ]; then
stripDirs "$stripAllList" "${stripAllFlags:--s}"
stripDirs "${stripDebugList:-*}" "${stripDebugFlags:--S}"

if [ -n "${stripAllList:-}" ]; then
stripDirs "${stripAllList:-}" "${stripAllFlags:--s}"
fi
fi
}
Copy link
Member

@vcunat vcunat Jun 9, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: in _doStrip I would change all :- to -, so that people can explicitly define those variables as empty without being overridden back to the default values. (Yes, one could define them as " ", but it seems nicer to allow empty string as well.)

Note that otherwise e.g. the first if will always fire, as * is used whenever the expansion could be empty.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does an empty string mean ? Strip everything or nothing ?
For now, passing an empty string to stripDirs strips nothing because it is interpreted as a gob pattern. This is why "" is replaced by "*".

If you want to disable stripping, why not use dontStrip ?

My fear is that if the variable is defined for some reason, or remains (say, from another output), the result may not be expected by the user.

I may however remove the useless if. Would that suit you ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the if in bda09f6.


stripDirs() {
local dirs="$1"
local stripFlags="$2"
local dirsNew=

for d in ${dirs}; do
if [ -d "$prefix/$d" ]; then
dirsNew="${dirsNew} $prefix/$d "
# Ensure $prefix is a directory
if [ -f "$prefix" ]; then
header "Stripping (with flags $stripFlags) the file ${prefix}."
echo -ne $prefix'\0' | stripAll "$stripFlags"
stopNest
fi
[ -d "$prefix" ] || return 0

# Ensure expansion occurs within $prefix/.
pushd "$prefix" >/dev/null
shopt -s dotglob
local stripDirs=( $1 )
local ignoreDirs=( $dontStripList )
shopt -u dotglob
popd >/dev/null
local debugIncluded= debugExcluded= p=

# `find` fails on missing paths. Remove them.
local roots=()
for p in "${stripDirs[@]}"; do
if [ -e "$prefix/$p" ]; then
roots+=( "$prefix/$p" )
debugIncluded="$debugIncluded${debugIncluded:+,}$p"
fi
done
dirs=${dirsNew}
[ "${#roots[@]}" -eq 0 ] && return 0
Copy link
Member

@vcunat vcunat Jun 9, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line will fail if the length is nonzero... which clearly isn't good (IIRC we run in mode where failing line aborts everything). Let me suggest [ "${#roots[@]}" -gt 0 ] || return 0

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I see the pattern is used a couple times in stripDir, too. It's possible our setup currently doesn't abort after a line fails (which would be considered a bug by me), but I did re-verify that such lines do fail.

Copy link
Member Author

@layus layus Jun 27, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More explanations on this. There are two concepts at play here.
The AND list (A && B) does indeed return the return value of the last executed command, and therefore has a non-zero return value if the arg list is empty.
BUT, the option that makes bash fail (-e) does not fail on non-zero return values from commands within an AND list, except if it is the last one.
This means that this line is correct in this context.
The former issue was that I used return without argument, which means that return returns the return value of the last command executed, which was the previous one in the AND list.

From the bash man page:

set -e
[...]
The shell does not exit if the command that fails is [...], part of any command executed in a && or || list except the command following the final && or ||, [...]

See bash lists[1] and the set builtin[2]
[1] https://www.gnu.org/software/bash/manual/bashref.html#Lists
[2] https://www.gnu.org/software/bash/manual/bashref.html#The-Set-Builtin


if [ -n "${dirs}" ]; then
header "stripping (with flags $stripFlags) in$dirs"
find $dirs -type f -print0 | xargs -0 ${xargsFlags:--r} strip $commonStripFlags $stripFlags || true
stopNest
fi
# Add ignored paths to find command.
local findArgs=( "${roots[@]}" )
for p in "${ignoreDirs[@]}"; do
if [ -e "$prefix/$p" ]; then
findArgs+=( "-path" "$prefix/$p" "-prune" "-o" )
debugExcluded="$debugExcluded $p"
fi
done

header "Stripping (with flags $stripFlags) in $prefix${debugIncluded:+"/{$debugIncluded}"}${debugExcluded:+", skipping$debugExcluded"}."
find "${findArgs[@]}" -type f -print0 | stripAll "$stripFlags"
stopNest
}

stripAll() {
local stripFlags="$1"

local _ls f stripOutput
while IFS= read -r -d $'\0' f; do
# Skip very small files, they make strip choke.
_ls=( $(ls -Ln "$f") )
[ "${_ls[4]}" -lt 4 ] && continue;

if stripOutput=$(strip $commonStripFlags $stripFlags "$f" 2>&1); then
echo "Stripped $f"
else
# Ignore failures on files that cannot be stripped.
[ "$stripOutput" = "strip:$f: File format not recognized" ] && continue
[ "$stripOutput" = "strip:$f: File truncated" ] && continue

[ -n "$stripOutput" ] && echo "$stripOutput"
echo "Strip failed on file $f"
false # fail !
fi
done
}
2 changes: 2 additions & 0 deletions pkgs/development/compilers/go/1.4.nix
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,8 @@ stdenv.mkDerivation rec {

setupHook = ./setup-hook.sh;

dontStripList = [ "share/go/src/debug/elf/testdata" ];

meta = with stdenv.lib; {
branch = "1.4";
homepage = http://golang.org/;
Expand Down
2 changes: 2 additions & 0 deletions pkgs/development/compilers/go/1.6.nix
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,8 @@ stdenv.mkDerivation rec {

setupHook = ./setup-hook.sh;

dontStripList = [ "share/go/src/debug/elf/testdata" ];

disallowedReferences = [ go_bootstrap ];

meta = with stdenv.lib; {
Expand Down
2 changes: 2 additions & 0 deletions pkgs/development/compilers/go/1.7.nix
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,8 @@ stdenv.mkDerivation rec {

setupHook = ./setup-hook.sh;

dontStripList = [ "share/go/src/debug/elf/testdata" ];

disallowedReferences = [ go_bootstrap ];

meta = with stdenv.lib; {
Expand Down
12 changes: 10 additions & 2 deletions pkgs/development/misc/avr-gcc-with-avr-libc/default.nix
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,10 @@ stdenv.mkDerivation {

hardeningDisable = [ "format" ];

# Make sure we don't strip the libraries in lib/gcc/avr.
stripDebugList= [ "bin" "avr/bin" "libexec" ];
dontStripList = [
"lib/gcc/avr" # Stripping here makes the whole toolchain unstable
"avr/lib" # Avr binaries need to be stripped with avr-strip; see below.
];

installPhase = ''
# important, without this gcc won't find the binutils executables
Expand Down Expand Up @@ -67,6 +69,12 @@ stdenv.mkDerivation {
popd
'';

postFixup = ''
# Strip avr/lib with avr-strip
alias strip=$out/bin/avr-strip
dontStripList= stripDirs "avr/strip" "''${stripDebugFlags:--S}"
'';

meta = with stdenv.lib; {
description = "AVR development environment including binutils, avr-gcc and avr-libc";
# I've tried compiling the packages separately.. too much hassle. This just works. Fine.
Expand Down
3 changes: 3 additions & 0 deletions pkgs/development/tools/misc/swig/2.x.nix
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ stdenv.mkDerivation rec {
./autogen.sh
'';

# Some files in share look like malformed executables.
dontStripList = [ "share/swig" ];

meta = {
description = "SWIG, an interface compiler that connects C/C++ code to higher-level languages";

Expand Down
3 changes: 3 additions & 0 deletions pkgs/development/tools/misc/swig/3.x.nix
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ stdenv.mkDerivation rec {
./autogen.sh
'';

# Some files in share look like malformed executables.
dontStripList = [ "share/swig" ];

meta = with stdenv.lib; {
description = "SWIG, an interface compiler that connects C/C++ code to higher-level languages";
homepage = http://swig.org/;
Expand Down
3 changes: 3 additions & 0 deletions pkgs/development/tools/misc/swig/default.nix
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ stdenv.mkDerivation rec {

configureFlags = "--disable-ccache";

# Some files in share look like malformed executables.
dontStripList = [ "share/swig" ];

meta = {
description = "Interface compiler that connects C/C++ code to higher-level languages";

Expand Down
2 changes: 0 additions & 2 deletions pkgs/os-specific/linux/syslinux/default.nix
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,6 @@ stdenv.mkDerivation rec {
substituteInPlace utils/ppmtolss16 gpxe/src/Makefile --replace /usr/bin/perl $(type -P perl)
'';

stripDebugList = "bin sbin share/syslinux/com32";

makeFlags = [
"BINDIR=$(out)/bin"
"SBINDIR=$(out)/sbin"
Expand Down
2 changes: 0 additions & 2 deletions pkgs/servers/http/apache-httpd/2.2.nix
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,6 @@ stdenv.mkDerivation rec {

enableParallelBuilding = true;

stripDebugList = "lib modules bin";

postInstall = ''
mkdir -p $doc/share/doc/httpd
mv $out/manual $doc/share/doc/httpd
Expand Down