Add save and load functions that operate on dictionaries

[mbauman]

These functions are similar to their macro counterparts, except that they operate on and return dictionaries instead of variables directly in the global namespace.

save takes a filename and a dictionary with bytestring keys. The dictionary key-value pairs are saved in the file, with the keys as the variable names and the values as their contents.

load takes a filename and (optionally) a list of variable names to read. If no variable names are given, all names in the file are read. It reads the variables into a dictionary that is returned.

Furthermore, the @load macro now returns a list of variables that it loaded.

Would fix #98. Still needs tests and documentation. And perhaps a bike-shed: Should save and load be exported?

I needed this for myself; it's very simple but I figured I may as well push it upstream, too.

[timholy]

Love it, thanks for doing this! Definitely these should be exported---I'd really rather encourage people towards the functions, because the macros are very fragile. The internal functions that support the macros can be renamed to something private so they are not exported.

I don't have any reservations about any aspect of your implementation---nice work!

[mbauman]

Glad you like it. After playing with it a bit more, I reverted the "r+" mode in save as it requires that the file already exist. And the semantics there are tricky. We can just clobber and document that it clobbers. This is also the default mode for Matlab's save unless append is explicitly passed, so it shouldn't be too jarring to users.

[mbauman]

Ok, updated with documentation and tests. But I come back with two issues:

1. There is already a load function! I missed it on my first pass. It's semantics are a little different: it takes an array of variable names and returns a tuple of values. This is admittedly very nice: it allows you to be explicit about which variables are named what in your local namespace (I actually have a custom Matlab load function that does exactly this, too). But then there's a disconnect between save and load — with the PR as is, there's the nice property that what goes in is exactly what comes out. Perhaps they could co-exist, but it's not obvious which should be used for any given signature. Thoughts?

2. Upon running the tests:

   Warning: using DataFrames.save in module Main conflicts with an existing identifier.

   Save is a very generic function name. I'm sure there are other packages that would have a similar conflict.

[timholy]

1. Good catch; it's been long enough since I've worked on this code base that I've forgotten what's there. I'm unsure whether co-existence is necessary, but it occurs to me that there's a certain sense in the following API:

   vardict = load(filename)
   var1, var2 = load(filename, "var1", "var2")
   save(filename, vardict)
   save(filename, "var1", var1, "var2", var2)
   

   The main idea is that when you don't know what variable names to request, you get name/value pairs back; when you do know enough about the file to request particular variables, there is no need to return the names again. The fourth one is basically there for logical consistency with the others. Thoughts?

2. Ugh. My suspicion is that the tests should be run with import rather than using. We could try to test for the existence of a save function in Main, and then import it if present, but in my experiments along such lines I've never been too happy with the result---it makes it impossible to specify that you want to run a particular version. As Julia grows these kinds of name conflicts are inevitable, and I suspect we'll all just have to get used to module-scoping more of our calls. Unless you have a better idea, that is.

[mbauman]

1. Yup, that looks wonderful. Best of both worlds. Shouldn't be too difficult to implement.
2. Really, these calls are akin to jldopen — they cannot extend a base function because they simply take a filename input. But calling them jldsave or JLD.load would really stink. DataFrames calls them save and loaddf, which doesn't really make too much sense, either (and should really be using JLD, not serialize!).

Well, the way operating systems do this is by the extension. Are there other top-level functions that are like this? Is it too feature-y to have Base Julia register filetypes, define save and load, and do the "dispatch" to non-exported functions? Or we could define a new string type that's parametric with its extension! Single-quote 'strings' are available, no? 😈

[timholy]

Your idea about the file extension is interesting. In Images there is a mechanism for on-demand loading of support-code for particular file formats. This makes me wonder if that should become more generic.

[mbauman]

Updated and good-to-go. We can punt on the name clashes for now, I think. But this could be very powerful with some base language support. read and write are similarly not defined for strings; were this in place you wouldn't need the im prefix to imread.

[timholy]

Discussion started at JuliaLang/julia#7299

[mbauman]

Very cool.

I forgot to mention that I deleted the load(filename, varnames::Array) signature. It was never exported, so I think we should be good, but we could very easily deprecate it if you want to.

[timholy]

Thanks so much for tackling this; I think it was the biggest API wart in JLD.