Trend strength predicts when generative foundation models win: a power-controlled benchmark, a mechanism, and an actionable selection rule
Abstract
Pretrained generative foundation models cast forecasting as conditional generation from a learned predictive distribution and forecast unseen series zero-shot.
We establish three results that turn their reported success into an actionable, mechanistic understanding.
First (a positive benchmark result): on a power-controlled study of 1728 rolling-origin forecasts over 36 series from 19 datasets spanning the full range of STL trend strength (F_T in [0.17, 1.00]), a zero-shot Chronos model significantly outperforms four strong classical baselines -- drift, seasonal-naive, Theta, and additive Holt-Winters/ETS -- with the best mean MASE (1.187 vs.
Theta 1.337, ETS 1.656; Friedman chi^2 = 46.08, p = 8.75e-09; Holm-corrected Wilcoxon p <= 0.015 against every baseline; a Nemenyi critical difference separating it from the classical pack).
Second (a novel, quantified mechanism): a controlled synthetic experiment with a known trend-generating process shows why -- and reveals that the win does not come from better trend extrapolation.
When the true trend is linear, damped, or exponential, additive ETS tracks the slope (slope-tracking ratio 1.02, 1.34, 0.98) whereas Chronos systematically under-extrapolates, behaving as a trend-shrinkage estimator (ratio 0.80, 0.49, 0.36).
Third (an actionable selection rule): because the advantage is a shrinkage effect, it is predictable from trend strength alone -- the generative model wins 78% of low-trend series but only 44% of high-trend ones, and its edge over ETS is significant on the low-trend stratum (0.982 vs.
1.671, p = 0.002) yet a tie on the high-trend stratum (p = 0.18).
Trend strength, computable before forecasting from the training context alone, is therefore a practical a-priori indicator of when to deploy a foundation model.
We additionally document a calibration shortfall (80% intervals cover 0.77).
이 뉴스, 어떠셨어요?
탭 한 번으로 반응 · 로그인 불필요