Why Time Series Forecasting Remains Unreasonably Difficult for AI Models
The unreasonable difficulty of time series forecasting
I explored why sophisticated models like LLMs and transformers often fail at time series forecasting compared to simple statistical baselines. The core issue is that time series data comes from a single dependent process rather than independent samples, leading to low signal-to-noise ratios, an effective lack of data due to autocorrelation, and inevitable distribution shifts during extrapolation.
"In time series, we are observing a single realization or trajectory through time, not independent samples, which means the sequence is a joint probability distribution over the entire path rather than individual data points."
HN discussion
- Financial markets are fundamentally adversarial and non-stationary because successful prediction strategies alter market dynamics, causing the very patterns they exploit to vanish. This feedback loop makes time series forecasting in finance impossible for any single actor to sustain, unlike natural phenomena such as planetary motion.
- The difficulty in forecasting often stems from the 'Anna Karenina principle' where anomalies possess unique characteristics that prevent them from being explained by common features, making them indistinguishable from noise in standard models.
- Practitioners suggest shifting focus from precise point forecasts to detecting large deviations or forecasting probabilities of threshold breaches, which is more valuable for use cases like incident detection or traffic jam prediction.
- While pure statistical models struggle with non-stationary data, some argue that incorporating exogenous features and causal narratives via hybrid LLM and time series models could synthesize better predictions by accounting for real-world event drivers.
- Server metrics and operational data face similar distribution shift challenges as financial markets due to unpredictable external events like the World Cup or severe weather, proving that non-financial time series are also difficult to forecast accurately.