Replace netcat-openbsd with libressl by matthewbauer · Pull Request #39634 · NixOS/nixpkgs

matthewbauer · 2018-04-28T00:36:34Z

Motivation for this change

Libressl comes with the netcat program. This is much more portable and from the exact same source as Debian's version.

GrahamcOfBorg · 2018-04-28T00:50:57Z

Success on x86_64-darwin (full log)

Attempted: libressl, netcat-gnu

Partial log (click to expand)

stripping (with command strip and flags -S) in /nix/store/54manzdzmz76b478pjqw6brvs5jn4m7y-libressl-2.5.5-bin/bin
patching script interpreter paths in /nix/store/54manzdzmz76b478pjqw6brvs5jn4m7y-libressl-2.5.5-bin
strip is /nix/store/kdff2gim6417493yha769kh00n63lnrw-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/fgs67frjn3qz4fnkzqc12k1fmpfnim8c-libressl-2.5.5-dev/lib
patching script interpreter paths in /nix/store/fgs67frjn3qz4fnkzqc12k1fmpfnim8c-libressl-2.5.5-dev
strip is /nix/store/kdff2gim6417493yha769kh00n63lnrw-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/8v9sxmabj4xbwhxnpd2haqjl6ksi57aq-libressl-2.5.5/lib
patching script interpreter paths in /nix/store/8v9sxmabj4xbwhxnpd2haqjl6ksi57aq-libressl-2.5.5
strip is /nix/store/kdff2gim6417493yha769kh00n63lnrw-cctools-binutils-darwin/bin/strip
patching script interpreter paths in /nix/store/s01404lk42kdhxkaxqqiafhh5rnl4j2l-libressl-2.5.5-man

GrahamcOfBorg · 2018-04-28T00:52:21Z

Success on x86_64-linux (full log)

Attempted: libressl, netcat-gnu

Partial log (click to expand)

stripping (with command strip and flags -S) in /nix/store/fsp9c5fazv7k8nxsvcqm74hgxbn3k4rj-libressl-2.5.5/lib
patching script interpreter paths in /nix/store/fsp9c5fazv7k8nxsvcqm74hgxbn3k4rj-libressl-2.5.5
checking for references to /build in /nix/store/fsp9c5fazv7k8nxsvcqm74hgxbn3k4rj-libressl-2.5.5...
shrinking RPATHs of ELF executables and libraries in /nix/store/xfw7f4y869q0c3frzr4chbccm4w8hph0-libressl-2.5.5-man
gzipping man pages under /nix/store/xfw7f4y869q0c3frzr4chbccm4w8hph0-libressl-2.5.5-man/share/man/
strip is /nix/store/j75dgadrff2d1fyc4fczmcgqkid2imdx-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/xfw7f4y869q0c3frzr4chbccm4w8hph0-libressl-2.5.5-man
checking for references to /build in /nix/store/xfw7f4y869q0c3frzr4chbccm4w8hph0-libressl-2.5.5-man...
/nix/store/cnnm8az8knygq84z1swvgirq7ngx6lk3-libressl-2.5.5-bin
/nix/store/v8xvjqcj8mavgnv1ir79gix09xwwv91n-netcat-gnu-0.7.1

GrahamcOfBorg · 2018-04-28T00:52:29Z

Success on aarch64-linux (full log)

Attempted: libressl, netcat-gnu

Partial log (click to expand)

stripping (with command strip and flags -S) in /nix/store/dvr9kshjpk52v5lc0pc46d4mfg08vfhr-libressl-2.5.5/lib
patching script interpreter paths in /nix/store/dvr9kshjpk52v5lc0pc46d4mfg08vfhr-libressl-2.5.5
checking for references to /build in /nix/store/dvr9kshjpk52v5lc0pc46d4mfg08vfhr-libressl-2.5.5...
shrinking RPATHs of ELF executables and libraries in /nix/store/bdspxs778gqj3zwd897kmcasb5hfhbrq-libressl-2.5.5-man
gzipping man pages under /nix/store/bdspxs778gqj3zwd897kmcasb5hfhbrq-libressl-2.5.5-man/share/man/
strip is /nix/store/j7d4mr0ikv974ig7yzhknpsq288js4bs-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/bdspxs778gqj3zwd897kmcasb5hfhbrq-libressl-2.5.5-man
checking for references to /build in /nix/store/bdspxs778gqj3zwd897kmcasb5hfhbrq-libressl-2.5.5-man...
/nix/store/w502vgdw0xvvlaqx7a7829a9znjhwx25-libressl-2.5.5-bin
/nix/store/4688g72pxm23drwza5paqikyqihkvfff-netcat-gnu-0.7.1

dtzWill · 2018-04-28T06:02:26Z

Woohoo! LGTM! Thank you! Looks good w/musl!

Mic92 · 2018-04-28T08:36:05Z

pkgs/development/libraries/libressl/default.nix

+
    enableParallelBuilding = true;

    outputs = [ "bin" "dev" "out" "man" ];


Maybe a separate output for netcat? It feels weird to me that if I install netcat I also get a ssl library.

Yes i can look into it. My hope was that it would be small enough to not matter though.

The problem with somethig like this is that it’s not obvious what is in the netcat output. For instance should nc.1 go in man or netcat?

Everything that belongs to netcat should also go there.

@Mic92

I still feel weird about doing this because it seems a little hacky but this was requested by @Mic92 and seems understandable to not want to mix up libressl outputs with netcat stuff.

aszlig · 2018-04-29T21:36:39Z

@matthewbauer: Be sure to note this in the release notes, because this is incompatible with openbsd-netcat. Most options are compatible, but for example there is no -q option in the version from libressl. The latter already breaks our hibernate test, so this should have a release note entry so that people are aware of this.

In the meantime, I'm going to fix the hibernate test...

matthewbauer · 2018-04-29T21:40:05Z

Ok will do! They looked identical when I tried them but thanks for catching that.

matthewbauer · 2018-04-29T21:41:34Z

Actually if it does really need it, we can probably still apply this patch (with minor modifications):

https://sources.debian.org/patches/netcat-openbsd/1.105-7/0006-quit-timer.patch/

Which enables the -q flag. We would definitely want to upstream it to OpenBSD. I strongly dislike Debian's tendency of "patch hording".

@matthewbauer

I'm not sure why 024b501 used -q 0 because even netcat-openbsd has the -N flag which IMO is the better way to shutdown the socket on EOF. Our default netcat implementation has changed once again[1] in 3c3b822 and we're now using LibreSSL's implementation, which doesn't have a -q flag. See #39634 for the pull request introducing the switch. [1]: #19982 Signed-off-by: aszlig <aszlig@nix.build> Cc: @matthewbauer, @dtzWill, @Mic92

matthewbauer · 2018-04-29T22:53:09Z

@aszlig Are the Nixpkgs release notes no longer used? I can definitely put the release note in nixos/doc/manual/rl-1809.xml just want to make sure that is the right place.

aszlig · 2018-04-29T23:12:40Z

@matthewbauer: This is mainly a NixOS issue anyway, because netcat-openbsd is part of NixOS by default so scripts and services that depend on it might break due to the change. As for the release notes in general... we don't have a real nixpkgs release, only NixOS releases which ship a certain snapshot of nixpkgs with it.

@matthewbauer

I'm not sure why 024b501 used -q 0 because even netcat-openbsd has the -N flag which IMO is the better way to shutdown the socket on EOF. Our default netcat implementation has changed once again[1] in 3c3b822 and we're now using LibreSSL's implementation, which doesn't have a -q flag. See NixOS#39634 for the pull request introducing the switch. [1]: NixOS#19982 Signed-off-by: aszlig <aszlig@nix.build> Cc: @matthewbauer, @dtzWill, @Mic92

linas · 2019-05-09T21:46:26Z

FYI, we tripped over the missing -q flag in netcat here: opencog/opencog-nix#28

dudebout · 2020-01-01T19:21:14Z

Note that this makes netcat hard to install from nix-env because it is provided in the nc output which is not part of outputsToInstall. A workaround is to use an overlay like:

self: super:
{
    netcat = super.libressl.overrideAttrs (oldAttrs: {
    meta = oldAttrs.meta // { outputsToInstall = [ "nc" ]; };
  });
}

dudebout · 2020-01-02T01:30:41Z

Alternatively define a separate derivation for netcat-libressl as follows:

  # Hack around nix-env's inability to install outputs that are not in
  # outputsToInstall: nc(1) is in the "nc" output of the libressl package, but
  # "nc" is not in the outputsToInstall
  stdenv.mkDerivation {
    pname = "netcat-libressl";
    version = libressl.version;
    unpackPhase = "true";
    installPhase = ''
      mkdir $out
      cp -r ${libressl.nc}/* $out
    '';
  }

This makes it clearer which version of netcat it actually is, and where netcat is coming from when running nix-env -q.

sebastianblunt · 2022-04-16T07:07:22Z

FWIW #101601 - the one from debian appears to be more portable.

matthewbauer added 3 commits April 27, 2018 19:32

netcat-gnu: move to pkgs/tools/networking/netcat

99b8d10

libressl: build netcat

949bb98

netcat-openbsd: remove

3c3b822

GrahamcOfBorg added 8.has: clean-up This PR removes packages or removes other cruft 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. labels Apr 28, 2018

Mic92 reviewed Apr 28, 2018

View reviewed changes

libressl: move netcat stuff to '.nc' output

c214848

I still feel weird about doing this because it seems a little hacky but this was requested by @Mic92 and seems understandable to not want to mix up libressl outputs with netcat stuff.

matthewbauer merged commit 6583ec5 into NixOS:master Apr 29, 2018

Radivarig mentioned this pull request May 5, 2019

[opencog] ShellUTest test fails (nc: invalid option -- 'q') opencog/opencog-nix#28

Closed

dudebout mentioned this pull request Jan 1, 2020

document nix-env bug relating to multiple output installation #76794

Merged

Artturin mentioned this pull request May 8, 2022

netcat-openbsd: init at 1.218-5 #171953

Merged

13 tasks


		enableParallelBuilding = true;

		outputs = [ "bin" "dev" "out" "man" ];

Uh oh!

Comments

Conversation

matthewbauer commented Apr 28, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation for this change

Uh oh!

GrahamcOfBorg commented Apr 28, 2018

Uh oh!

GrahamcOfBorg commented Apr 28, 2018

Uh oh!

GrahamcOfBorg commented Apr 28, 2018

Uh oh!

dtzWill commented Apr 28, 2018

Uh oh!

Mic92 Apr 28, 2018

Choose a reason for hiding this comment

Uh oh!

matthewbauer Apr 28, 2018

Choose a reason for hiding this comment

Uh oh!

matthewbauer Apr 28, 2018

Choose a reason for hiding this comment

Uh oh!

Mic92 Apr 28, 2018

Choose a reason for hiding this comment

Uh oh!

aszlig commented Apr 29, 2018

Uh oh!

matthewbauer commented Apr 29, 2018

Uh oh!

matthewbauer commented Apr 29, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

matthewbauer commented Apr 29, 2018

Uh oh!

aszlig commented Apr 29, 2018

Uh oh!

linas commented May 9, 2019

Uh oh!

dudebout commented Jan 1, 2020

Uh oh!

dudebout commented Jan 2, 2020

Uh oh!

sebastianblunt commented Apr 16, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

matthewbauer commented Apr 28, 2018 •

edited

Loading

matthewbauer commented Apr 29, 2018 •

edited

Loading