Introduce USE_GPL_LIBS Makefile flag to build Julia without GPL libraries

[ViralBShah]

Disable Rmath, FFTW, and SuiteSparse builds if USE_GPL_LIBS=0
Remove inclusion of julia bindings in sysimg if USE_GPL_LIBS=0

[ViralBShah]

This is now ready and passes tests, even with USE_GPL_LIBS=0. I'll merge this in a couple of days.

[garrison]

I recall a conversation where it was suggested to use USE_GPL_LIBS instead of NO_GPL, but I'm having a hard time finding this now. I think ENABLE_GPL_BUILD would be an even better name for this. After all, even if USE_GPL_LIBS is set to zero, it is still possible to use GPL-licensed external packages.

[tkelman]

little weird to prepend all the linalg tests except this one...

[tkelman]

This is now ready and passes tests

No it doesn't.

This fails on CI due to the .jl suffix. And there's an old-style concatenation warning here.

[tkelman]

And we bundle GPL command-line Git in both the Windows and Mac binaries, and Busybox on Windows. Plus GCC runtime libraries on all platforms.

[vtjnash]

i'm pretty sure that gcc has a linking exception that explicitly permits redistributing the runtime libraries (busybox and git are certainly another matter altogether however)

[tkelman]

Yeah I think you're right about the linking exception on the gcc runtime libs. Should be able to disable busybox and git in make win-extras, make dist, and contrib/mac/app/Makefile.

[ViralBShah]

So, until we move to libgit2, does this effectively mean that we can't have a GPL free distribution? I am sure libgcc would have the exception, but we need to verify that.

What should we do about git and busybox for now? Should we just disable them for now in the GPL version - and you just have to live with that functionality missing?

[tkelman]

So, until we move to libgit2, does this effectively mean that we can't have a GPL free distribution?

Not one that has a working package manager, without relying on the user manually installing git somewhere else on the path. We don't use busybox for much right now, mostly just to pass a few spawn tests, but it will be handy after we do get rid of git to still allow shelling out to coreutils functions on Windows.

[ViralBShah]

I guess this PR is good as it is, because you can at least now build from source and get a Julia distribution that does not include any GPL components. Only if you build the mac and windows installers, git and busybox get bundled.

[ViralBShah]

I don't understand the comment about arnoldi. I am trying to disable it, because it has a depedency on suitesparse being present.

[ViralBShah]

@garrison I think the NO_GPL line comments went away with my forced push.

[garrison]

Not one that has a working package manager, without relying on the user manually installing git somewhere else on the path.

Whether julia is a "derived work" of git when git is called by julia as a command is probably independent of whether git is distributed alongside julia (or whether the user installed it manually). I could probably find this in the GPL FAQ, but IANAL.

[vtjnash]

getting rid of busybox is pretty painful since window's doesn't actually provide anything nearly as functional by default, it's hard to fallback to anything else. see the discussion on the original PR. there's perhaps toybox (https://github.com/landley/toybox), but it would take a lot of work to make that reasonably feature complete with unix and then also port it to windows.

[ViralBShah]

We have had such discussions before too - but I just find it easier to keep it clean in such cases. For people who want to build Julia without GPL dependencies, I am sure they can figure out how to install git, at least for now.

[aviks]

re: git

IANAL .. etc..

There is an argument to be had that shelling out to a program does not make a derivative work. And if Julia is not a derivative work of Git, then Git's distribution restrictions do not apply to Juila. We just distribute Git according to its own license.

http://www.ifross.org/en/program-forks-gpl-licensed-program-system-or-vice-versa-call-derivative-work

[tkelman]

I don't understand the comment about arnoldi. I am trying to disable it, because it has a depedency on suitesparse being present.

It's fine to disable it, but note the order in which the test list gets assembled:

julia> tests = [
               "linalg", "core", "keywordargs", "numbers", "strings",
               "dates", "dict", "hashing", "remote", "iobuffer", "staged",
               "arrayops", "tuple", "subarray", "reduce", "reducedim", "random",
               "intfuncs", "simdloop", "blas", "sparse",
               "bitarray", "copy", "math", "fastmath", "functional",
               "operators", "path", "ccall",
               "bigint", "sorting", "statistics", "spawn", "backtrace",
               "priorityqueue", "file", "version", "resolve",
               "pollfd", "mpfr", "broadcast", "complex", "socket",
               "floatapprox", "readdlm", "reflection", "regex", "float16",
               "combinatorics", "sysinfo", "rounding", "ranges", "mod2pi",
               "euler", "show", "lineedit", "replcompletions", "repl",
               "replutil", "sets", "test", "goto", "llvmcall", "grisu",
               "nullable", "meta", "profile", "libgit2", "docs", "markdown",
               "base64", "parser", "serialize", "functors", "char", "misc",
               "enums", "cmdlineargs", "i18n"
           ];
julia> if "linalg" in tests
               # specifically selected case
               filter!(x -> x != "linalg", tests)
               prepend!(tests, ["linalg1", "linalg2", "linalg3", "linalg4",
                                "linalg/lapack", "linalg/triangular", "linalg/tridiag",
                                "linalg/bidiag", "linalg/diagonal",
                                "linalg/pinv", "linalg/givens", "linalg/cholesky", "linalg/lu",
                                "linalg/symmetric"])
               if true # Base.USE_GPL_LIBS
                   tests = [tests, "linalg/arnoldi";]
               end
           end
93-element Array{ASCIIString,1}:
 "linalg1"
 "linalg2"
 "linalg3"
 "linalg4"
 "linalg/lapack"
 "linalg/triangular"
 "linalg/tridiag"
 "linalg/bidiag"
 "linalg/diagonal"
 "linalg/pinv"
 "linalg/givens"
 "linalg/cholesky"
 "linalg/lu"
 "linalg/symmetric"
 "core"
 "keywordargs"
 "numbers"
 "strings"
 ⋮
 "grisu"
 "nullable"
 "meta"
 "profile"
 "libgit2"
 "docs"
 "markdown"
 "base64"
 "parser"
 "serialize"
 "functors"
 "char"
 "misc"
 "enums"
 "cmdlineargs"
 "i18n"
 "linalg/arnoldi"

See how linalg/arnoldi ends up at the end? That's all I'm nitpicking about. Can fix it via something like

    if "linalg" in tests
        # specifically selected case
        filter!(x -> x != "linalg", tests)
        linalgtests = ["linalg1", "linalg2", "linalg3", "linalg4",
                         "linalg/lapack", "linalg/triangular", "linalg/tridiag",
                         "linalg/bidiag", "linalg/diagonal",
                         "linalg/pinv", "linalg/givens", "linalg/cholesky", "linalg/lu",
                         "linalg/symmetric"]
        if Base.USE_GPL_LIBS
            push!(linalgtests, "linalg/arnoldi")
        end
        prepend!(tests, linalgtests)
    end

[tkelman]

getting rid of busybox is pretty painful since window's doesn't actually provide anything nearly as functional by default, it's hard to fallback to anything else.

Actually it does. It's called powershell. It just happens to not be syntax-compatible with coreutils.

[ViralBShah]

I am merging this, because it does not change the default behaviour.

Should we add a note about this flag to the README and the LICENSE files?

[tkelman]

readme sure, I think not in license though since this is a non-default option for now

[ViralBShah]

I feel like it is worth discussing the cases under which julia builds end up being GPL vs. non-GPL. I thought the LICENSE file is the right place to clarify this, since the README is just too long, and perhaps could use some splitting.

[tkelman]

The readme could stand splitting into a few pieces, build troubleshooting vs other stuff.

You could maybe move FFTW and SuiteSparse from the "Julia's standard library uses the following external libraries, which have their own licenses:" section into a separate section with the same title plus "unless the make variable USE_GPL_LIBS=0 is set", and add the same caveat to the Rmath section.

[tknopp]

@ViralBShah: I don't understand this PR. In order to make Julia GPL free one just would have to remove the dll/so/dynlib. Why does the Julia wrapper has to be removed its MIT!

One example: There is a the Intel MKL providing an ABI compatible interface of the fftw. Using the existing julia wrapper of the fftw it should be possible to use it in a GPL-free manner.

[ViralBShah]

I will enable that with future commits. Note that Intel fftw wrapper is not complete, and does not support the full functionality we need.

I don't like the idea of shipping a bunch of code with missing libraries. Perhaps if we can figure our a way whereby we ship a stub for those missing libraries, which print out a warning, that would be useful.

[tkelman]

This is the hacky simple way of disabling chunks of code. It's not a great substitute for real modularity in being able to selectively enable/disable different components without completely breaking things, but it's better than nothing.

[tknopp]

I would argue that we do not need those hacks unless someone of the Julia team wants to use it. Someone who really wants to use Julia in a commercial product will need to think about deployment and making Julia GPL free is absolutely trivial in that case (thanks to the readline removal).

[tkelman]

There was a mailing list post where someone asked about this.

[ViralBShah]

@tknopp We can't link MKL for example. I personally want to be able to do such things and at least have the ability to distribute Julia with MKL.

[tknopp]

Why should we link against MKL? ccall is not linking. You can distribute a Julia version that automatically detects if MKL is available and calls it and otherwise calls a fallback. The only problem is distributing with MKL/GPL libs.

[tkelman]

The following files in Julia's repository are still GPL:

• contrib/mac/app/script
• test/perf/kernel/ziggurat.c

plus a few LGPL.

[ViralBShah]

Also, git and busybox. I don't know the license of the Windows installers either.

[ViralBShah]

ziggurat.c is just there for performance comparisons, and I don't believe is distributed. We can remove it at a moment's notice.

LGPL is fine.

[ihnorton]

Our Windows installer? NSIS is zlib, as is the MUI2 script.

(anyway, the NSIS license is irrelevant to the generated installer license)

[tknopp]

It is not relevant if there are parts in the repository that are GPL. The only thing that counts is what is statically and dynamically linked into the julia executable. If git is GPL this is not an issue because a process is spawned

[tkelman]

Also, git and busybox.

Those are dependencies, not files checked into the repo. I think you're right about not distributing the perf test files but would have to double-check.

The mac app script is really short, if we ever want to distribute non-GPL OSX binaries we probably need to spec it out and have someone clean-room recreate it.