Re-using models without recompiling / CompiledModel API

[hottwaj]

Hi there pymc team, as always thanks for all your work on this package :)

Is there an approach to follow to re-use an existing model with new data, but without recompiling the model?

A couple of use cases:

• unit tests to check that a model works as desired on lots of different sets of randomly generated data
• sampling from posterior predictive with different sets of data e.g. for cross validation

From looking at code for e.g. pymc.find_MAP, I noticed that the model is compiled on every call to pymc.find_MAP, even if only mutable data has changed via pymc.set_data. I wrote a slightly tweaked version of find_MAP to cache compiled functions on a model so that they can be reused on subsequent calls, and got a nice speedup for my unittests. I have not looked at how pymc.sample handles compiling, but assume it might also recompile the model each time it is called on the same model? Please correct me if I'm wrong.

Would I be right in guessing that models currently have no metadata stored on them that indicates when they are "clean" (no changes made since last compile) or "dirty"?

Assuming that is the case, would it make sense for the model API to be enhanced to allow a process such as the following to be followed:

1. define model "description" - as is done currently
2. compile model - returns an instance of "CompiledModel" or "FrozenModel" or similar - new API needed for this
3. set inputs on CompiledModel/FrozenModel
4. run sample/find_MAP/sample_posterior_predictive etc on CompiledModel

Steps 3&4 can then be repeated as much as user wants without needing to recompile model each time, and the original model "description" could be modified without affecting the Compiled/FrozenModel

I think this could be implemented without interfering with the current pymc.sample and pymc.find_MAP API - though it might make sense for the implementation of those functions to be changed to compile the current model to a Compiled/FrozenModel and then run steps3/4 in the background

Would be interested to know your thoughts - probably these are things already addressed/thought about. Thanks!

[michaelosthege]

We're generally in favor of avoiding unnecessary recompilation, but there are a few things to consider:

• find_MAP/sample/prior/posterior_predictive/initial_point all need different sub-graphs from the model, so there is little room for improvement
• Needing to track whether or not a recompile is needed

You might want to use mode="FAST_COMPILE" to speed up compilation in unit tests by skipping graph optimizations.

[ricardoV94]

Closing this discussion in favor of #7177 thanks for bringing it up to our attention @hottwaj