string interpolation in @load

[cbecker]

I may be doing something wrong, but

using HDF5,JLD
using PyPlot

for testNo=1:3
    @load "x-$testNo"
end

throws

julia> include("test.jl")
ERROR: testNo not defined
 in include at boot.jl:244
while loading test.jl, in expression starting on line 4

while the following works fine

using HDF5,JLD
using PyPlot

for testNo=1:3
    @save "x-$testNo" fName
end

[simonster]

This is basically the same issue as #83. The file name needs to be known at compile time to put the variables into the calling scope.

[cbecker]

I see, but this brings me to another doubt: is it possible to read all variables and place them in a Dict ?

From what I've been using so far, that would be great, so one could do

vars = load(fName)  
@show vars["myvar"]  # should print the variable "myvar" loaded from fName

or maybe this functionality exists and I totally missed it.

[timholy]

Yes, we need this. I swear someone implemented what seemed to be a nice package on top of JLD that had this and more, but I am having trouble finding it. But presumably at least something like load should go into JLD.

[rened]

I have an implementation of something in that direction here: https://github.com/rened/DictFiles.jl

[timholy]

Thanks, @rened!

[cbecker]

Great. I guess something like this should work:

function dictLoad( filename::String )

    D = Dict{String,Any}()

    readexprs = Array(Expr, 0)
    vars = Array(Expr, 0)
    f = jldopen(filename)
    nms = names(f)

    for n in nms
        obj = f[n]
        if isa(obj, JLD.JldDataset)
            D[n] = read(f,n)
        end
    end

    close(f)

    return D
end

But to create a meaningful PR:

• there exists a load() function in JLD, though that is not exported because it doesn't seem to be directly useful (returns a tuple, and requires var names as inputs). Shall it be renamed to something else so that dictLoad above becomes load ? I know this could break compatibility now, but this is what most users would expect, at least coming from MATLAB.
• I don't know which is the common practice in Julia to avoid leaving files opened (in this case f above), if there are exceptions, I suppose try..finally blocks?

EDIT:

• Maybe we should have a keyword option as well, to allow to use symbols for indexing, rather than strings, so we can have as well

data = dictLoad("test.jld")
@show data[:varName]

this assumes no spaces in var names, of course.

[timholy]

Looks good to me. As you noticed, this will only work for top-level datasets.
I suppose an alternative would be to descend into groups and name them by full path.

Regarding your questions:

• It's fine to rename the current load. Since it's not exported, I'm not concerned about breakage.
• Right, either try...finally or the do block syntax (which internally is implemented in terms of try...finally) is the way to go.
• I'd be OK with a keyword syntax to index by symbols, although that will be limiting if we decide to descend into groups: :a/b is parsed as :(a)/b, whereas "a/b" is of course fine. Is it for the performance advantage that you're interested in this?

We may find that the default display of dictionaries is a bit inconvenient. If desired, I have a disp function in my own private repository that I could contribute. It prints one entry per line (similar to Matlab, and also how properties are displayed in Images).

[cbecker]

About descending into groups, I am not familiar with HDF5 but I think I get the idea. I suppose that we can first target the 'most common usage' of top-level datasets.

About symbols, it is just a shortcut. I would add a keyword argument for using symbols, which by default would be false, so that by default we don't get in trouble with symbols.

The display function would be great, I agree that the default implementation would be messy to interpret. I will get back to this soon and create a PR.

[timholy]

Implemented by @mbauman in #101.