Can't load a JLD file saved with HDF5.jl ≤ v"0.2.26" in later versions if it contains Vector{Vector{T}} objects

[carlobaldassi]

I have files which I cannot load unless I downgrade HDF5 to v"0.2.26".
Steps to reproduce:

• downgrade HDF5 to v"0.2.26": Pkg.rm("HDF5"); Pkg.add("HDF5", v"0.2.26", v"0.2.27")
• save a variable of Vector{Vector{T}} type: a=Vector{Int}[[1,2,3]]; @save "tst.jld" a
• upgrade HDF5 to v"0.2.27" or later: Pkg.rm("HDF5"); Pkg.add("HDF5")
• load the previous file: @load "tst.jld"

ouput:

HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 140731934467968:
  #000: ../../../src/H5Dio.c line 182 in H5Dread(): can't read data
    major: Dataset
    minor: Read failed
  #001: ../../../src/H5Dio.c line 438 in H5D__read(): unable to set up type info
    major: Dataset
    minor: Unable to initialize object
  #002: ../../../src/H5Dio.c line 939 in H5D__typeinfo_init(): unable to convert between src and dest datatype
    major: Dataset
    minor: Feature is unsupported
  #003: ../../../src/H5T.c line 4525 in H5T_path_find(): no appropriate function for conversion path
    major: Datatype
    minor: Unable to initialize object
ERROR: Error reading dataset /a
 in error at error.jl:22
 in readarray at /home/carlo/.julia/v0.3/HDF5/src/plain.jl:1596
 in read at /home/carlo/.julia/v0.3/HDF5/src/plain.jl:1293
 in read at /home/carlo/.julia/v0.3/HDF5/src/jld.jl:350
 in read at /home/carlo/.julia/v0.3/HDF5/src/jld.jl:286
 in read at /home/carlo/.julia/v0.3/HDF5/src/jld.jl:241
 in anonymous at no file

Everything works if keeping the same version of HDF5. Looks like some backwards incompatible change has been introduced in v"0.2.27".

[carlobaldassi]

The commit which introduced the incompatibility is of course 3a95493: "Better JLD support for arrays-of-arrays of basic types "

[timholy]

Argh, I was worried about that but I assumed the old version would still work. How about I bump the version and introduce a compatibility function?

It will probably be tomorrow before I can get to this.

[carlobaldassi]

I'll see if I'm able to do that.

[carlobaldassi]

At least the offending behaviour should be fixed, but probably you'd still like to adjust things.

[timholy]

Looks good to me. I am now planning a JLDArchives package that will simply contain archives of old JLD files. Someday we will want to streamline this package, in which event maybe backwards-compatibility code can move there.

But this is a masterpiece of brevity! Nice.