Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
43c7778
Plots of continuous distributions (#2463)
aasensio Aug 1, 2017
259613f
Multivariate floating point precision fix (#2470)
ericmjl Aug 3, 2017
41d70b8
Fix bug in nuts stats (#2467)
aseyboldt Aug 3, 2017
f845575
Implement random method for LKJCorr (#2443)
Aug 3, 2017
3232e16
Fix adapt init (#2465)
aseyboldt Aug 3, 2017
47c334a
implement random method for wishart (#2473)
Aug 3, 2017
1085aa4
Typo in _repr_latex_ for Flat distribution
ColCarroll Aug 4, 2017
18108f9
Merge pull request #2477 from pymc-devs/flat_latex
fonnesbeck Aug 4, 2017
6e04322
Draft intro page (#2393)
fonnesbeck Aug 5, 2017
ed288ed
use Bayesian bootstrapping to compute weights (#2479)
aloctavodia Aug 5, 2017
4bd6f7e
Merge remote-tracking branch 'bwengals/find_MAP-float32' into mletest
kyleabeauchamp Aug 7, 2017
4647c3e
Add failing test case for 2482
kyleabeauchamp Aug 8, 2017
1b67b26
Escape latex, fix Flat distribution (#2485)
ColCarroll Aug 8, 2017
30262d7
NF notebook cleanup (#2481)
fonnesbeck Aug 9, 2017
0ec182b
Speed up to DIC and BPIC (#2488)
denadai2 Aug 9, 2017
25bd58a
DOCS - Adding docs explaining the behaviour of logp (#2490)
springcoil Aug 9, 2017
f581520
fix AR(1) log likelihood (#2285)
eph Aug 10, 2017
ed2aa49
Change nuts init default to jitter+adapt_diag (#2475)
aseyboldt Aug 10, 2017
945fd71
Fix nuts initialization docstring (#2492)
aseyboldt Aug 10, 2017
0a22388
fix MvNormal error (#2464)
ferrine Aug 10, 2017
d5a011f
Changed docstring to correctly specify the support for Categorical (c…
Daniel-Nichol Aug 11, 2017
d852ba5
Change init for test_linear() in timeseries (#2494)
Aug 11, 2017
0e14c11
Run lint before tests, so travis fails faster (#2461)
ColCarroll Aug 11, 2017
278c107
Deprecated Constant class (#2452)
fonnesbeck Aug 11, 2017
2793db3
Fix typo in theano intro (#2497)
aseyboldt Aug 12, 2017
5b87e36
Merge remote-tracking branch 'upstream/master' into mletest
kyleabeauchamp Aug 12, 2017
f06ab8f
Fix some test failures
kyleabeauchamp Aug 12, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions docs/source/api/distributions/continuous.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,23 +7,28 @@ Continuous

Uniform
Flat
HalfFlat
Normal
HalfNormal
SkewNormal
Beta
Exponential
Laplace
StudentT
StudentTpos
Cauchy
HalfCauchy
Gamma
Weibull
StudentTpos
Lognormal
ChiSquared
HalfNormal
Wald
Pareto
InverseGamma
ExGaussian
VonMises
Triangular
Gumbel

.. automodule:: pymc3.distributions.continuous
:members:
5 changes: 4 additions & 1 deletion docs/source/api/distributions/discrete.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,18 @@ Discrete
.. autosummary::

Binomial
ZeroInflatedBinomial
BetaBinomial
Bernoulli
Poisson
ZeroInflatedPoisson
NegativeBinomial
ConstantDist
ZeroInflatedPoisson
DiscreteUniform
Geometric
Categorical
DiscreteWeibull
Constant

.. automodule:: pymc3.distributions.discrete
:members:
2 changes: 2 additions & 0 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'matplotlib.sphinxext.only_directives',
'matplotlib.sphinxext.plot_directive',
'sphinx.ext.autodoc',
'sphinx.ext.autosummary',
'sphinx.ext.doctest',
Expand Down
Binary file added docs/source/images/forestplot.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ Contents:
.. toctree::
:maxdepth: 3

intro
getting_started
prob_dists
examples
Expand Down
145 changes: 145 additions & 0 deletions docs/source/intro.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
.. _intro:

************
Introduction
************


Purpose
=======

PyMC3 is a probabilistic programming module for Python that allows users to fit Bayesian models using a variety of numerical methods, most notably Markov chain Monte Carlo (MCMC) and variational inference (VI). Its flexibility and extensibility make it applicable to a large suite of problems. Along with core model specification and fitting functionality, PyMC3 includes functionality for summarizing output and for model diagnostics.



Features
========

PyMC3 strives to make Bayesian modeling as simple and painless as possible, allowing users to focus on their scientific problem, rather than on the methods used to solve it. Here is a partial list of its features:

* Modern methods for fitting Bayesian models, including MCMC and VI.

* Includes a large suite of well-documented statistical distributions.

* Uses Theano as the computational backend, allowing for fast expression evaluation, automatic gradient calculation, and GPU computing.

* Built-in support for Gaussian process modeling.

* Model summarization and plotting.

* Model checking and convergence detection.

* Extensible: easily incorporates custom step methods and unusual probability
distributions.

* Bayesian models can be embedded in larger programs, and results can be analyzed
with the full power of Python.


What's new in version 3
=======================

The third major version of PyMC has benefitted from being re-written from scratch. Substantial improvements in the user interface and performance have resulted from this. While PyMC2 relied on Fortran extensions (via f2py) for most of the computational heavy-lifting, PyMC3 leverages Theano, a library from the Montréal Institute for Learning Algorithms (MILA), for array-based expression evaluation, to perform its computation. What this provides, above all else, is fast automatic differentiation, which is at the heart of the gradient-based sampling and optimization methods currently providing inference for probabilistic programming.

Major changes from previous versions:

* New flexible object model and syntax (not backward-compatible with PyMC2).

* Gradient-based MCMC methods, including Hamiltonian Monte Carlo (HMC), the No U-turn Sampler (NUTS), and Stein Variational Gradient Descent.

* Variational inference methods, including automatic differentiation variational inference (ADVI) and operator variational inference (OPVI).

* An interface for easy formula-based specification of generalized linear models (GLM).

* Elliptical slice sampling.

* Specialized distributions for representing time series.

* A library of Jupyter notebooks that provide case studies and fully developed usage examples.

* Much more!

While the addition of Theano adds a level of complexity to the development of PyMC, fundamentally altering how the underlying computation is performed, we have worked hard to maintain the elegant simplicity of the original PyMC model specification syntax.


History
=======

PyMC began development in 2003, as an effort to generalize the process of
building Metropolis-Hastings samplers, with an aim to making Markov chain Monte
Carlo (MCMC) more accessible to applied scientists.
The choice to develop PyMC as a python module, rather than a standalone
application, allowed the use MCMC methods in a larger modeling framework. By
2005, PyMC was reliable enough for version 1.0 to be released to the public. A
small group of regular users, most associated with the University of Georgia,
provided much of the feedback necessary for the refinement of PyMC to a usable
state.

In 2006, David Huard and Anand Patil joined Chris Fonnesbeck on the development
team for PyMC 2.0. This iteration of the software strives for more flexibility,
better performance and a better end-user experience than any previous version
of PyMC. PyMC 2.2 was released in April 2012. It contained numerous bugfixes and
optimizations, as well as a few new features, including improved output
plotting, csv table output, improved imputation syntax, and posterior
predictive check plots. PyMC 2.3 was released on October 31, 2013. It included
Python 3 compatibility, improved summary plots, and some important bug fixes.

In 2011, John Salvatier began thinking about implementing gradient-based MCMC samplers, and developed the ``mcex`` package to experiment with his ideas. The following year, John was invited by the team to re-engineer PyMC to accomodate Hamiltonian Monte Carlo sampling. This led to the adoption of Theano as the computational back end, and marked the beginning of PyMC3's development. The first alpha version of PyMC3 was released in June 2015. Over the following 2 years, the core development team grew to 12 members, and the first release, PyMC3 3.0, was launched in January 2017.


Usage Overview
==============

For a detailed overview of building models in PyMC3, please read the appropriate sections in the rest of the documentation. For a flavor of what PyMC3 models look like, here is a quick example.

First, import the PyMC3 functions and classes you will need for building your model. You can import the entire module via ``import pymc3 as pm``, or just bring in what you need::

from pymc3 import Model, Normal, invlogit, Binomial, sample, forestplot
import numpy as np

Models are defined using a context manager (``with`` statement). The model is specified declaratively inside the context manager, instantiating model variables and transforming them as necessary. Here is an example of a model for a bioassay experiment.

::

# Data
n = np.ones(4)*5
y = np.array([0, 1, 3, 5])
dose = np.array([-.86,-.3,-.05,.73])

with Model() as bioassay_model:

# Prior distributions for latent variables
alpha = Normal('alpha', 0, sd=100)
beta = Normal('beta', 0, sd=100)

# Linear combinations of parameters
theta = invlogit(alpha + beta*dose)

# Model likelihood
deaths = Binomial('deaths', n=n, p=theta, observed=y)

Save this file, then from a python shell (or another file in the same directory), call.

::

with bioassay_model:

# Draw wamples
trace = sample(1000, njobs=2)
# Plot two parameters
forestplot(trace, varnames=['alpha', 'beta'])

This example will generate 1000 posterior samples on each of two cores using the NUTS algorithm, preceded by 500 tuning samples (the default number). The sampler is also initialized using variational inference.

::

Auto-assigning NUTS sampler...
Initializing NUTS using ADVI...
Average Loss = 12.562: 6%|▌ | 11412/200000 [00:00<00:14, 12815.82it/s]
Convergence archived at 11900
Interrupted at 11,900 [5%]: Average Loss = 15.168
100%|██████████████████████████████████████| 1500/1500 [00:01<00:00, 787.56it/s]

The sample is returned as arrays inside of a ``MultiTrace`` object, which is then passed to a plotting function. The resulting graphic shows a forest plot of the random variables in the model, along with a convergence diagnostic (R-hat) that indicates our model has converged.

.. image:: ./images/forestplot.png
371 changes: 371 additions & 0 deletions docs/source/notebooks/AR.ipynb

Large diffs are not rendered by default.

Loading