Why Time Series Forecasting Remains Unreasonably Difficult for AI Models
The unreasonable difficulty of time series forecasting
I explored why sophisticated models like LLMs and transformers often fail at time series forecasting compared to simple statistical baselines. The core issue is that time series data comes from a single dependent process rather than independent samples, leading to low signal-to-noise ratios, an effective lack of data due to autocorrelation, and inevitable distribution shifts during extrapolation.
In time series, we are observing a single realization or trajectory through time, not independent samples, which means the sequence is a joint probability distribution over the entire path rather than individual data points.
- jftuga
Is there a github repo for her time series forecasting experiments? I'd like to see her code and be able to run it against my own data sets.
- crystal_revenge
> I’ve been thinking recently about what makes time series forecasting problems so difficult compared to other sequence learning tasks
Whenever I teach people time series forecasting, I always point out that one of the biggest challenges is that you will always have values at prediction time that are out side the range of values observed during training (specifically the value of t).
In plenty of other machine learning and statistical modeling tasks this is not the case. You can train on every token you'll ever see and every pixel value you'll ever see, you can do regression analysis on every categorical value you include and an least an observation from within a range of every continuous and discrete value you'll observe. But with forecasting you will always have values you predict that are outside the range of anything you trained on.
You would run into similar problems if you tried to create a statistical model of the density of water given a temperature but your training data only included values between 0-100 C and you went out and started predicting values covering all the temperatures found on Earth.
For whatever reason, when time is a variable we somehow think it is immune from the obvious limitation of predicting on values outside of the range of values you trained on.
- klodolph
I see that a lot of these are markets.
Yes, it’s hard to predict markets. Because anybody who can successfully predict markets, does so, makes money, and changes the market so their predictions lose their edge.
Time series forecasts are a lot easier if you are forecasting, say, disk use in your servers or whatnot. (By “easy” I mean you can do a simple prediction and get useful insights.)
- c7b
Well. Lots of math that boils down to 'predicting the future is hard'. Especially when the future is one of social construction, that's what gets lost a bit here. Predicting the future is easier for planetary motions than for Bitcoin.
- ForceBru
Are there any real-world time-series that are known to be easy to forecast? Something I could use as an example of "this is what forecasts look like when it's actually possible to predict your time-series and your model is actually doing it"?