End-to-end probabilistic hierarchical forecasting of large hierarchies via probabilistic top-down

Retail and supply chain operations rely on demand forecasts to drive decisions, from replenishment at the product level to capacity planning at the store level.

These forecasts should be probabilistic, to allow risk-aware decisions, and coherent across the aggregation hierarchy, so that decisions taken at different levels are not based on conflicting demand forecasts.

However, producing coherent probabilistic forecasts is computationally demanding; at retail scale, with hierarchies of thousands of time series, this cost becomes a first-order operational concern.

Existing two-step forecast-then-reconcile procedures and end-to-end neural models scale poorly, rely on restrictive assumptions, or require specialized hardware and engineering effort.

We propose e2eTD, a fast and scalable method for probabilistic coherent forecasting of large hierarchical and grouped time series. e2eTD directly forecasts only a small subset of aggregate series (about 0.3\% of the hierarchy in our experiments), which are smoother and thus more predictable than the intermittent bottom series.

The resulting forecast samples are propagated to the bottom level through a novel probabilistic top-down sampling algorithm, in which the historical disaggregation proportions are modeled as joint distributions, estimated in-sample.

Coherent forecasts for all aggregation levels are then obtained by summing the joint bottom-level samples.

On the two largest publicly available retail datasets, M5 and Favorita, e2eTD achieves the lowest weighted scaled pinball loss (the M5 competition's probabilistic score) across aggregation levels among all competing methods; it would have ranked 11th of 892 teams in the M5 Uncertainty competition.

On a standard laptop, e2eTD runs in about five minutes on M5 ($\sim$40K series) and twenty minutes on Favorita ($\sim$300K series).