Better gradient interface

[yebai]

It would be nice if we can have a user-friendly interface for automatic differentiation, e.g.

@model gdemo(x) = begin   
  s ~ InverseGamma(2,3)
  m ~ Normal(0,sqrt(s))
  for i=1:length(x)
    x[i] ~ Normal(m, sqrt(s)) 
  end
  return(s, m, x)
end

g  = @gradient(gdemo([2.0, 3.0]), varInfo = nothing)

By default, the user does not need to pass in varInfo (i.e. differentiate through all parameters).

This would be very helpful for users who want to contribute new gradient-based inference methods.

[xukai92]

I think this interface is actually done by the closure way of creating model. Like what we did in test file for AD, we can now use

∇E = gradient(vi, ad_test_f)

to get the gradient dictionary.

I guess user have to pass an VarInfo to it otherwise we don't know the derivative at which points we want to compute?

[yebai]

That's true. We can probably close this issue now.

[xukai92]

Cool.