Loading models without pre-defining them

[mirestrepo]

Loading models requires predefining the values and functions that construct them. Ideally the loader should do that. Serialization may not be possible in some cases but we should look into this

[DilumAluthge]

Yeah I would really like this to work. Actually it's how I originally envisioned it working. I think the only thing we have to do is figure out how to serialize functions to/from file, since we need that for Knet (predict function and loss function).

[DilumAluthge]

The other idea I once had was this - instead of having the user define the predict and loss functions as functions, they actually define strings that contain the source code of the function. Then we can serialize those strings really easily. The tricky part is parsing and eval-ing those strings without getting weird errors.

[DilumAluthge]

We would need eval() to evaluate the predict and loss functions in the context of, e.g., the fit!() function... but I think eval() always evaluates in the top-level module. So that's where I got stuck, and I didn't pursue the idea further.

[mirestrepo]

As an intermediate step could just reload all data-structures and require setting the predict and loss functions.

Or have the predict and loss be loaded from a Julia file and only the path be save....

Buts let's keep exploring, none seem like and elegant solution

[DilumAluthge]

@mirestrepo Take a look at the discussion here - denizyuret/Knet.jl#144

[DilumAluthge]

Also take a look here - https://github.com/denizyuret/Knet.jl/blob/master/examples/dl-tutorial/mlp.ipynb (see input cell 30). They use JLD to save the models to file, but they only save the weights (as well as the current training loss, testing loss, etc). They don't serialize the predict function ( multilinear(w,x) in this example).

[DilumAluthge]

One last thought: I've been playing around with BSON.jl (https://github.com/MikeInnes/BSON.jl), and it seems that BSON.jl supports saving functions to file. So this may be an easy way to solve this issue.

[DilumAluthge]

Fixed in develop. You can load models without predefining them. The one exception is Knet models - you have to define predict() and loss() before loading the model.