[release-0.4] backports for RC2

[tkelman]

Will run PackageEvaluator on this, at the time of writing this is everything that looks ready / necessary for RC2.

[tkelman]

My current packageevaluator run of dcd7ffc is at V, so almost done. If the results look good, would people rather merge this then tag RC2, or pull in a few more of the pending backports from last night? My turnaround time on packageevaluator runs is limited by the really old hardware I have access to. If someone has a fast machine with a working vagrant (which julia.mit.edu doesn't have right now) that they'd like to donate ssh access to, shoot me an email.

[StefanKarpinski]

How is it possible that we still don't have those machines that were ordered at MIT?

[tkelman]

Better shape now, only Augur (due to reload ref #13107) and Playground (ArgumentError: Lint not found in path\n in require at ./loading.jl:252) are failing now but weren't at b23b11a.

--- b23b11a370/summary  2015-09-14 09:08:16.348503305 -0700
+++ dcd7ffc119/summary  2015-09-17 12:12:22.962570777 -0700
@@ -31,7 +31,7 @@
 nightly/AudioIO.json
 "tests_fail"
 nightly/Augur.json
-"tests_pass"
+"tests_fail"
 nightly/AutoHashEquals.json
 "tests_pass"
 nightly/Autoreload.json
@@ -81,7 +81,7 @@
 nightly/Boltzmann.json
 "tests_pass"
 nightly/Bootstrap.json
-"tests_fail"
+"tests_pass"
 nightly/BoundingBoxes.json
 "tests_pass"
 nightly/Brim.json
@@ -179,7 +179,7 @@
 nightly/ContinuedFractions.json
 "no_tests"
 nightly/Contour.json
-"tests_fail"
+"tests_pass"
 nightly/Convex.json
 "tests_fail"
 nightly/CoreNLP.json
@@ -187,7 +187,7 @@
 nightly/Cosmology.json
 "tests_pass"
 nightly/CovarianceMatrices.json
-"tests_fail"
+"tests_pass"
 nightly/CoverageBase.json
 "tests_pass"
 nightly/Coverage.json
@@ -522,6 +522,8 @@
 "tests_pass"
 nightly/Humanize.json
 "tests_pass"
+nightly/Hwloc.json
+"tests_pass"
 nightly/HyperDualNumbers.json
 "tests_pass"
 nightly/HyperLogLog.json
@@ -544,6 +546,8 @@
 "tests_pass"
 nightly/ImageView.json
 "tests_fail"
+nightly/Immerse.json
+"tests_fail"
 nightly/ImmutableArrays.json
 "tests_pass"
 nightly/ImplicitEquations.json
@@ -686,6 +690,8 @@
 "tests_pass"
 nightly/LibGit2.json
 "tests_fail"
+nightly/LibHealpix.json
+"tests_fail"
 nightly/LIBSVM.json
 "tests_pass"
 nightly/LibTrading.json
@@ -749,9 +755,9 @@
 nightly/MarketData.json
 "tests_pass"
 nightly/MarketTechnicals.json
-"tests_fail"
+"tests_pass"
 nightly/Match.json
-"tests_fail"
+"tests_pass"
 nightly/Mathematica.json
 "not_possible"
 nightly/MathProgBase.json
@@ -809,7 +815,7 @@
 nightly/MinimalPerfectHashes.json
 "no_tests"
 nightly/MixedModels.json
-"tests_fail"
+"tests_pass"
 nightly/MixtureModels.json
 "no_tests"
 nightly/MLBase.json
@@ -944,6 +950,8 @@
 "tests_fail"
 nightly/Pandas.json
 "not_possible"
+nightly/ParallelSparseMatMul.json
+"no_tests"
 nightly/Parameters.json
 "tests_pass"
 nightly/Pardiso.json
@@ -981,9 +989,11 @@
 nightly/Pipe.json
 "tests_pass"
 nightly/Playground.json
-"tests_pass"
+"tests_fail"
 nightly/Plotly.json
 "tests_fail"
+nightly/Plots.json
+"tests_pass"
 nightly/PLX.json
 "tests_fail"
 nightly/PolarFact.json
@@ -1079,7 +1089,7 @@
 nightly/ReverseDiffOverload.json
 "no_tests"
 nightly/ReverseDiffSource.json
-"tests_fail"
+"tests_pass"
 nightly/ReverseDiffSparse.json
 "tests_pass"
 nightly/Rif.json
@@ -1097,7 +1107,7 @@
 nightly/Roots.json
 "tests_pass"
 nightly/RouletteWheels.json
-"tests_fail"
+"tests_pass"
 nightly/RudeOil.json
 "not_possible"
 nightly/RunTests.json
@@ -1133,7 +1143,7 @@
 nightly/Shannon.json
 "tests_fail"
 nightly/Shapefile.json
-"no_tests"
+"tests_pass"
 nightly/ShapeModels.json
 "tests_pass"
 nightly/Showoff.json
@@ -1277,7 +1287,7 @@
 nightly/TimeModels.json
 "tests_fail"
 nightly/TimeSeries.json
-"tests_fail"
+"tests_pass"
 nightly/Timestamps.json
 "tests_fail"
 nightly/TimeZones.json

Do we need a Jeff decision on whether to do RC2 now or after one more round?

[tkelman]

@JeffBezanson ping, opinion please?

edit: imo we're overdue for rc2 and though we know we'll need more backports, I'll want to test them before tagging and if anything causes problems it'll delay by another day

[JeffBezanson]

Maybe 64d0cc9 should be added, since it adds a deprecation warning for Union(). Sorry I forgot about that for so long.

I'd also like to include #13060 assuming the PR is ready to go.

[tkelman]

I'll take that as "do another round." A few people have volunteered machines so hopefully it'll be faster this time.

[tkelman]

Hm, the eigs and fft doc commits here result in warnings from genstdlib. Not sure how big of a deal that is.

[tkelman]

Full results for the failure cases: https://gist.github.com/cec1045b5aea925c7a1a

[tkelman]

Docile and Lexicon are having trouble at UndefVarError: doc_str not defined in exec at /home/vagrant/.julia/v0.4/Docile/src/Runner/lookup.jl:137 cc @MichaelHatherly

Gadfly and ReverseDiffSource are timing out probably due to too many deprecation warnings. I hope the new binding deprecations aren't leaking memory like regular depwarns were at one point.

Mux and WebSockets have some messed-up JuliaWeb dependencies and aren't installing MbedTLS properly. New tags of a few JuliaWeb packages would hopefully fix that cc @malmaud

ReverseDiffSparse is missing an ArrayViews dependency cc @mlubin

SVM is missing a StatsBase dependency, but is apparently soon to be deprecated cc @johnmyleswhite

I think we can go ahead with tagging RC2, the Docile issue is the most worrying.

[tkelman]

And GeoIP (cc @sbromberger) is giving

ERROR: LoadError: failed to find end of centeral directory record
 in _find_enddiroffset at /home/vagrant/.julia/v0.4/ZipFile/src/ZipFile.jl:251
 in call at /home/vagrant/.julia/v0.4/ZipFile/src/ZipFile.jl:104
 in dldata at /home/vagrant/.julia/v0.4/GeoIP/src/geoip-module.jl:52
 in update at /home/vagrant/.julia/v0.4/GeoIP/src/geoip-module.jl:91
 in geolocate at /home/vagrant/.julia/v0.4/GeoIP/src/geoip-module.jl:129

[MichaelHatherly]

Thanks @tkelman, I'll look into the Docile issue a little later tonight. It shouldn't really be a problem since packages should be conditionally importing it only in 0.3 and should be using Base@doc in 0.4, or at least that's what's meant to happen.

[sbromberger]

@tkelman let me check. GeoIP is a bear to test because it requires a download of a very large database in order to do what it does. I've been toying with the idea of creating a test database that doesn't have anything significant in it, and I think you may have just given me the reason to do so.

[tkelman]

GeoIP seems to be the only failure that isn't either already fixed or a dependency issue:

WARNING: Module Zlib uuid did not match cache file
ERROR: LoadError: LoadError: __precompile__(true) but require failed to create a precompiled cache file
 in require at ./loading.jl:252
 in include at ./boot.jl:261
 in include_from_node1 at ./loading.jl:304
 in require at ./loading.jl:243
 in include at ./boot.jl:261
 in include_from_node1 at ./loading.jl:304
 in process_options at ./client.jl:308
 in _start at ./client.jl:411
while loading /home/tkelman/.julia/v0.4/GeoIP/src/GeoIP.jl, in expression starting on line 7

We can try to work that out but in the meantime I'm going to tag rc2.

edit: closing and restarting julia makes that error go away, but something's fishy in precompile land

[yuyichao]

Ref #13200

[yuyichao]

@tkelman Is this a Mac?

[tkelman]

no, that was julia.mit.edu

[yuyichao]

Hmm, interesting. So #13200 might not be OSX specific....

[tkelman]

Something I backported since rc2 may have broken Distributions and/or WorldBankData?

--- 1c584eafb9/summary  2015-09-21 14:28:59.048895924 -0400
+++ d8930a01ac/summary  2015-09-21 14:30:19.461019843 -0400
@@ -210,6 +210,8 @@
 "tests_pass"
 nightly/CRF.json
 "tests_pass"
+nightly/CRlibm.json
+"tests_pass"
 nightly/CrossDecomposition.json
 "tests_pass"
 nightly/Crypto.json
@@ -287,11 +289,11 @@
 nightly/DistributedArrays.json
 "tests_pass"
 nightly/Distributions.json
-"tests_pass"
+"tests_fail"
 nightly/Diversity.json
 "tests_fail"
 nightly/Docile.json
-"tests_fail"
+"tests_pass"
 nightly/Docker.json
 "no_tests"
 nightly/DocOpt.json
@@ -386,6 +388,8 @@
 "no_tests"
 nightly/FLANN.json
 "tests_pass"
+nightly/FMIndexes.json
+"tests_pass"
 nightly/Fontconfig.json
 "tests_pass"
 nightly/Formatting.json
@@ -405,7 +409,7 @@
 nightly/FunctionalUtils.json
 "no_tests"
 nightly/Gadfly.json
-"tests_fail"
+"tests_pass"
 nightly/GARCH.json
 "tests_fail"
 nightly/Gaston.json
@@ -421,7 +425,7 @@
 nightly/GeoInterface.json
 "tests_pass"
 nightly/GeoIP.json
-"tests_fail"
+"tests_pass"
 nightly/GeoJSON.json
 "tests_pass"
 nightly/GeometricalPredicates.json
@@ -685,7 +689,7 @@
 nightly/Levenshtein.json
 "tests_pass"
 nightly/Lexicon.json
-"tests_fail"
+"tests_pass"
 nightly/LibBSON.json
 "tests_pass"
 nightly/LibCURL.json
@@ -815,7 +819,7 @@
 nightly/Metis.json
 "tests_pass"
 nightly/MFCC.json
-"no_tests"
+"tests_pass"
 nightly/Millboard.json
 "tests_pass"
 nightly/MinimalPerfectHashes.json
@@ -869,7 +873,7 @@
 nightly/MutableStrings.json
 "no_tests"
 nightly/Mux.json
-"tests_fail"
+"tests_pass"
 nightly/NaiveBayes.json
 "tests_pass"
 nightly/NamedArrays.json
@@ -910,6 +914,8 @@
 "no_tests"
 nightly/NPZ.json
 "no_tests"
+nightly/NullableArrays.json
+"tests_pass"
 nightly/NumericExtensions.json
 "no_tests"
 nightly/NumericFuns.json
@@ -1097,7 +1103,7 @@
 nightly/ReverseDiffSource.json
 "tests_fail"
 nightly/ReverseDiffSparse.json
-"tests_fail"
+"tests_pass"
 nightly/Rif.json
 "tests_pass"
 nightly/RLEVectors.json
@@ -1108,6 +1114,8 @@
 "not_possible"
 nightly/RobustShortestPath.json
 "tests_fail"
+nightly/ROCAnalysis.json
+"tests_pass"
 nightly/RomanNumerals.json
 "tests_pass"
 nightly/Roots.json
@@ -1119,7 +1127,7 @@
 nightly/RunTests.json
 "tests_fail"
 nightly/SALSA.json
-"tests_fail"
+"tests_pass"
 nightly/Sampling.json
 "tests_fail"
 nightly/SaveREPL.json
@@ -1241,7 +1249,7 @@
 nightly/SunlightAPIs.json
 "tests_pass"
 nightly/SVM.json
-"tests_fail"
+"tests_pass"
 nightly/SVMLightLoader.json
 "tests_pass"
 nightly/Switch.json
@@ -1367,7 +1375,7 @@
 nightly/Weave.json
 "tests_fail"
 nightly/WebSockets.json
-"tests_fail"
+"tests_pass"
 nightly/WinRPM.json
 "no_tests"
 nightly/Winston.json
@@ -1375,7 +1383,7 @@
 nightly/WoodburyMatrices.json
 "tests_pass"
 nightly/WorldBankData.json
-"tests_pass"
+"tests_fail"
 nightly/WORLD.json
 "tests_pass"
 nightly/WriteVTK.json

Anyone know what's up with those? https://gist.github.com/100e3fec30e28ebca3d2
cc @4gh @simonbyrne