computational issues in runCrossValidate

[paciorek]

For predictive loss, we calculate the loss using the uncompiled model.
Also, in line 46 of crossValidation.R we need to save the result of compileNimble in order for it to be used later to do the loss calculation.

@danielturek I'm assigning this to you in light of the bug discussed in PR #1068, as I suspect you'll be in there anyway.

[paciorek]

While we are on the topic of CV, I'd also like to discuss whether the predictive loss should be given on the non-logged scale (including the handling of logging in calcCrossValSD).

[danielturek]

PR #1299 saves the compiled model object, and uses it for calculations in the case of predictive loss.

It's still an open discussion regarding whether predictive loss should be given on a non-logged scale.

[paciorek]

Regarding the loss, in looking back at one of the key WAIC papers, I see that the averaging over the posterior samples averages the predictive density not log predictive density, so I think it's fine as it is. We really should use the logsumexp trick here, but I will leave that for future work.

I'll leave this open as a reminder of that, but make any other comments for now in the PR.

[danielturek]

@paciorek Noted, that sounds good. I could fix the exponentiation computational approach at some point, as well.

For now, I'll leave this open. I'm going to merge #1299, which might automatically close this issue, but if so then I'll reopen it.