Description

@lostella suggested you might be interested in having the mean absolute log error (MALE) and mean square log error (MSLE) error metrics in GluonTS.

Here's a quick summary of the MALE and MSLE:

• The metrics are both based on the “log error”, LE = log(f / y) (here f is the forecast, and y is the observed value).
• Like other “relative” metrics (MAPE, sMAPE), they only really make sense for strictly positive data, and they assume the data have a meaningful zero point.
• But in this case, they have some nice properties:
  • They produce estimates of the median and geometric mean respectively. By contrast, the MAPE and sMAPE produce biased estimates in general.
  • They can be interpreted as the average “relative” error (i.e., by what multiplicative factor you expect to be wrong).
  • They can be easily compared / aggregated across time series. (Like other relative metrics.)
  • They avoid the main problems of the MAPE and sMAPE.
• They are basically the MAE and MSE on the log scale.

I wrote a longer (~10min) article making the case for these metrics here.

References

[1] https://towardsdatascience.com/mean-absolute-log-error-male-a-better-relative-performance-metric-a8fd17bc5f75
[2] Tofallis, Chris, A Better Measure of Relative Prediction Accuracy for Model Selection and Model Estimation (July 2014). Journal of the Operational Research Society (2015) 66, 1352–1362, Available at SSRN: https://ssrn.com/abstract=2635088