Add Differential Equation API

[Dpananos]

This work is a part of Google Summer of Code 2019.

This PR adds:

• ode submodule, which contains DifferentialEquation, the main tool for doing inference with ODEs.

• An ODEGradop for computation of the gradients required for HMC.

• A function for computing an augmented system of ODEs (that is, the original ODE + ODEs for the sensitivity equations) called augment_system.

• Unit tests for ode

• A Notebook showcasing how to use DifferentialEquation in a couple of examples regarding a free falling object and a spread of an infection.

• Rerun existing ODE notebook for comparison

---

Sorry, I was having some issues with the last PR so I thought I would just start again.Here is a link to the old PR should you need to reference it.

[review-notebook-app]

Check out this pull request on ReviewNB: https://app.reviewnb.com/pymc-devs/pymc3/pull/3590

You'll be able to see notebook diffs and discuss changes. Powered by ReviewNB.

[michaelosthege]

Could the test failure be related to this issue? #1640

[Dpananos]

FYI: Test pass now because I have included the pytest.xfail decorator on previously failing tests. See pymc3/tests/test_ode.py

[ColCarroll]

I think that's a good choice. For reference, this means it might not work on GPUs -- I am not sure if that is an OK tradeoff for ODEs, but I do think such a failure should not block this PR. We should remember to open an issue once it is merged, though.

[ColCarroll]

Left a few nitpicky things, and one musing suggestion. None of them are dealbreakers, and I will be delighted when this gets merged, even if none of the changes get made.

As another musing suggestion, running

python -m pylint --enable=all pymc3/step_methods/hmc/nuts.py

on each file you changed -- many suggestions will be silly, or ones we do not enforce, but some are helpful.

If you want to be really fancy

git diff --name-only HEAD master | xargs python -m pylint --enable=all

will pipe just the changed files through pylint

[Dpananos]

Thanks @ColCarroll, I think I added everything you suggested.

[michaelosthege]

Your module is currently not imported by default. Please add this after line 10 of pymc3\__init__.py:

from . import ode

(just like gp is imported in line 7)

[twiecki]

As this is new code, can you just run black on the new files?

[Dpananos]

@twiecki Ran black on all the new files.

[michaelosthege]

@Dpananos running black gave some nice results there! Can you also run it on test_ode.py?

@twiecki @ColCarroll Are we ready to merge?

There are some improvements regarding the returned shape & the grad that will follow in the near future, but the functionality is already there with this PR.

[twiecki]

It seems like black was run on just ode.py, can you run it on all new files here?

[Dpananos]

I ran black on the pymc3/ode directory. It left the majority of the files unchanged. I made the change in the docstring, and also ran black on the test_ode.py file.

[canyon289]

Just some minor comments on some code style. The notebook could also use spaces on the inline comments. I know its picky, but for me its jarring after seeing a bunch of PEP8 style comments.

On the notebook the PyMC3 version is shown, just to double check will that be the version of PyMC3 that includes the ODE solver, or will we increment the PyMC3 version to 3.8 once this ODE solver get merged to master?

[twiecki]

Need to add to release-notes and link the new NBs to the docs.

[Dpananos]

@twiecki Not sure how to update the docs, but I did find a reference to the other ODE notebook in docs/source/notebooks/table_of_contents_examples.js. I just swapped out my notebook for that one.

@canyon289 Changes made and made the notebook more pep-8 friendly wrt spaces (I will never get used to one space after assignment, no space after arguments).

If that's all, are we ready to merge?

[twiecki]

These still have the old doc-format.

[twiecki]

Merged and fixed on master.

[twiecki]

Congrats again @Dpananos, this was a beast!