Add notes for package maintainers #25912
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -294,15 +294,15 @@ Building Julia requires that the following software be installed: | |
|
|
||
| Julia uses the following external libraries, which are automatically downloaded (or in a few cases, included in the Julia source repository) and then compiled from source the first time you run `make`: | ||
|
|
||
| - **[LLVM]** (3.9 + patches) — compiler infrastructure (see [note below](#llvm)). | ||
| - **[FemtoLisp]** — packaged with Julia source, and used to implement the compiler front-end. | ||
| - **[libuv]** (custom fork) — portable, high-performance event-based I/O library. | ||
| - **[OpenLibm]** — portable libm library containing elementary math functions. | ||
| - **[DSFMT]** — fast Mersenne Twister pseudorandom number generator library. | ||
| - **[OpenBLAS]** — fast, open, and maintained [basic linear algebra subprograms (BLAS)](https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms) library, based on [Kazushige Goto's](https://en.wikipedia.org/wiki/Kazushige_Goto) famous [GotoBLAS](https://www.tacc.utexas.edu/research-development/tacc-software/gotoblas2) (see [note below](#blas-and-lapack)). | ||
| - **[LAPACK]** (>= 3.5) — library of linear algebra routines for solving systems of simultaneous linear equations, least-squares solutions of linear systems of equations, eigenvalue problems, and singular value problems. | ||
| - **[MKL]** (optional) – OpenBLAS and LAPACK may be replaced by Intel's MKL library. | ||
| - **[SuiteSparse]** (>= 4.1) — library of linear algebra routines for sparse matrices (see [note below](#suitesparse)). | ||
| - **[ARPACK]** — collection of subroutines designed to solve large, sparse eigenvalue problems. | ||
| - **[PCRE]** (>= 10.00) — Perl-compatible regular expressions library. | ||
| - **[GMP]** (>= 5.0) — GNU multiple precision arithmetic library, needed for `BigInt` support. | ||
|
|
@@ -347,12 +347,39 @@ Julia uses the following external libraries, which are automatically downloaded | |
| [mbedtls]: https://tls.mbed.org/ | ||
| [pkg-config]: https://www.freedesktop.org/wiki/Software/pkg-config/ | ||
|
|
||
| ### Notes for distribution package maintainers | ||
|
|
||
| Package maintaners will typically want to make use of system libraries where possible. Please refer to the above version requirements and notes below. | ||
|
|
||
| Currently community maintained packages are: | ||
| - Fedora: [official](https://src.fedoraproject.org/rpms/julia), [Copr (for backports)](https://copr.fedorainfracloud.org/coprs/nalimilan/julia/) | ||
| - [Arch](https://www.archlinux.org/packages/community/x86_64/julia/) | ||
| - [Gentoo](https://packages.gentoo.org/packages/dev-lang/julia) | ||
| - [Raspbian](https://github.com/JuliaBerry/julia-raspbian) | ||
|
|
||
| Additionally, the following unmaintained packages may still be useful: | ||
| - [Homebrew](https://github.com/staticfloat/homebrew-julia) | ||
| - [Ubuntu PPA](https://launchpad.net/~staticfloat/+archive/ubuntu/juliareleases) (0.5) | ||
| - [Debian](https://packages.debian.org/sid/julia) (0.4) | ||
|
|
||
| ### System Provided Libraries | ||
|
|
||
| If you already have one or more of these packages installed on your system, you can prevent Julia from compiling duplicates of these libraries by passing `USE_SYSTEM_...=1` to `make` or adding the line to `Make.user`. The complete list of possible flags can be found in `Make.inc`. | ||
|
|
||
| Please be aware that this procedure is not officially supported, as it introduces additional variability into the installation and versioning of the dependencies, and is recommended only for system package maintainers. Unexpected compile errors may result, as the build system will do no further checking to ensure the proper packages are installed. | ||
|
|
||
| ### LLVM | ||
|
|
||
| The most complicated dependency is LLVM, for which we require version 3.9 with some additional patches (LLVM is not backward compatible). We recommend either: | ||
|
There was a problem hiding this comment. Would be good to mention that these are upstream patches? Packagers really appreciate this kind of information. |
||
| - adding the patches to the LLVM 3.9 package of the distribution (note: these are backported upstream bug fixes, _not_ Julia-specific patches: all have been contributed into upstream LLVM), or | ||
|
There was a problem hiding this comment. Maybe add a note the the list of patches is available at Maybe Lines 517 to 519 in 251a501 llvm is modified so that it can be used safely with other versions of llvm (4.0 or a system 3.9). This is only true for Julia 0.7+, before that collisions between the Julia version and the system version are likely.
There was a problem hiding this comment. Maybe that note would be better if it appeared in the Makefile itself? (it does appear apart from the others) There was a problem hiding this comment. @vchuravy Would you mind adding something to that effect (since I don't quite understand these issues)? There was a problem hiding this comment. What Valentine just said seems to conflict with the text; that sure seems like a julia-specific patch. (But does it matter if we're using distribution-packaged LLVM? We don't care about conflicts then ight?) There was a problem hiding this comment. I'm not sure what this patch does exactly, but we should leave packagers deal with the issue of parallel installation of multiple LLVM versions. It's kind of complex, with some hacking and lots of options to set, and it's far beyond the scope of Julia-specific patches. If that's useful, you can point at how Fedora handles this: https://src.fedoraproject.org/rpms/llvm3.9/blob/master/f/llvm3.9.spec |
||
| - bundling a Julia-only LLVM library inside the Julia package. | ||
|
|
||
| Using an unpatched or different version of LLVM will result in errors and/or poor performance. | ||
|
|
||
| ### BLAS and LAPACK | ||
|
|
||
| As a high-performance numerical language, Julia should be linked to a multi-threaded BLAS and LAPACK, such as OpenBLAS, and _not_ the reference implementations which are the default on some systems. | ||
|
There was a problem hiding this comment. Including an example here would be nice. Something like "For example, packages such as |
||
|
|
||
| ### SuiteSparse | ||
|
|
||
| SuiteSparse is a special case, since it is typically only installed as a static library, while `USE_SYSTEM_SUITESPARSE=1` requires that it is a shared library. Running the script `contrib/repackage_system_suitesparse4.make` will copy your static system SuiteSparse installation into the shared library format required by Julia. `make USE_SYSTEM_SUITESPARSE=1` will then use the SuiteSparse that has been copied into Julia's directory, but will not build a new SuiteSparse library from scratch. | ||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
these were all removed as unsupported in #23484 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I see.
My argument for having this is so that maintainers can look to see what others have done. Perhaps a permalink to somewhere julialang.org or a discourse page would be more useful.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about describing them as no-longer-maintained and still mentioning them here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They are maintained, just not by us. OTC a different bullet below mentions packages which are unmaintained.
We could move instructions for packagers to a separate file so that casual users don't find them. But honestly we're talking about README.md, not the main download page, so I'm not too concerned. Anyway people are going to use these packages, so refusing to mention them is a bit silly IMHO. It's even useful to distinguish actively maintained packages from totally outdated ones to prevent surprises.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the stance of the upstream project, that people should be using them or that how they're built can't be vouched for and may introduce problems that are strictly due to packaging? Maintaining a list of what's maintained is bound to get out of date and doesn't sound so useful. https://julialang.org/downloads/platform.html says "It is strongly recommended that the official generic linux binaries from the downloads page be used to install Julia" and "The following distribution-specific packages are community contributed. They may not use the right versions of Julia dependencies or include important patches that the official binaries ship with. In general, bug reports will only be accepted if they are reproducible on the generic linux binaries on the downloads page." so why should the readme have redundant or contradictory statements about them?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree we should probably move them, but for different reasons: the README is a snapshot: it isn't that useful to know which packages were maintained with 0.6 when looking at the 0.7 README. I think better to move them to a maintainable place, such as a discourse wiki page, or even a subpage of https://julialang.org/downloads/, and mention that in the README.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Which of these would people prefer?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah I didn't see https://julialang.org/downloads/platform.html, we can just use that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, let's put that information on that page, with all warnings we may want to add, especially regarding the fact that distribution packages generally lag behind upstream since they only provide the lastest version which was available when the distro was released. Better acknowledge that these packages exist and inform users than ignore them and leave users on their own discovering the potential issues.