[documentation: some suggestions for MLE & MAP guide]

[nipunbatra]

Hi,
I was going through the MLE and MAP estimation page. It is very well written. Thank you!

I had a few suggestions:

• Add analytical MLE expression and show learnt MLE is same as analytical MLE
• Add analytical MAP expression and show learnt MAP is same as analytical MAP
• render_model() showing the graphical models and how they differ for MLE and MAP model
• use uniform prior additionally for MAP model and show MAP = MLE
• Show neg loglikelihood(optimum) = SVI loss for MLE (optimum)

I can make a PR on the above.

Also, it may be useful to additionally show:

• how the auto guide is similar to the manually defined guide -- similar params and samples? I am not sure how best to show this.

[fritzo]

Hi @nipunbatra, those all sound like good improvements. Feel free to submit a PR.

[nipunbatra]

Thanks @fritzo

I am making the above changes.

I was calculating the analytical MAP and comparing with learnt MAP estimate. The learnt MAP estimate is 0.536 whereas the analytical one is 0.533. I was wondering if this is expected or if I'm doing something wrong. Can you please clarify?

The analytical calculation for MAP below.

Our Beta prior is parameterised by $\alpha_{Heads}$ (= 10 in our example) and $\alpha_{Tails}$ (= 10 in our example). The closed form expression for MAP estimate is:
$\frac{\alpha_{Heads} + #Heads}{\alpha_{Heads} + #Heads + \alpha_{Tails} + #Tails}$ = $\frac{10 + 6}{10 + 6 + 10 + 4}$ = $\frac{16}{30} = 0.533$

NB -- I am also planning to add the following figure to explain existing text comparing the MAP estimate to the MLE estimate.

[nipunbatra]

I'd also seek your inputs on how to show the Autoguide to be similar/same as the custom guide in the tutorial.

list(autoguide_map.named_parameters()), list(autoguide_map.named_pyro_params())

Output

([('latent_fairness_unconstrained',
   Parameter containing:
   tensor(0.1431, requires_grad=True))],
 [('latent_fairness', tensor(0.5357, grad_fn=<ClampBackward1>))])

The custom guide was:

def guide_map(data):
    f_map = pyro.param("f_map", torch.tensor(0.5),
                       constraint=constraints.unit_interval)
    pyro.sample("latent_fairness", dist.Delta(f_map))

How should we show the equivalence?

1. Need to show Delta() distribution used in Autoguide.
2. Need to show the param corresponding to f_map in the auto guide
3. Need to show the constraints on the unit interval in the Autoguide.

Would appreciate your inputs on this @fritzo

[fritzo]

How should we show the equivalence?

I'm not sure how to show code equivalence, but you might show parameter equivalence by printing the param store after each guide is trained:

def print_param_store():
    store = pyro.get_param_store()
    for name, value in sorted(store.items()):
        constraint = store._constraints.get(name, "real")
        print(f"{name} constrained to {constraint} has value\n{value}")