Time-series forecasting predicts future values of a sequence observed over time — demand, traffic, energy load, revenue, sensor readings — from its own history and from related covariates. Unlike ordinary regression, observations are not independent: order matters, the target autocorrelates with its own past, and the future can only be validated with data that comes after the training window.
This page covers the structure of time series (trend, seasonality, stationarity), classical models (ARIMA, exponential smoothing), the machine-learning approach of lag features with gradient boosting, deep and foundation models, how to validate correctly, and the metrics that matter.
Most series can be described as a trend (long-run level change), one or more seasonal cycles (daily, weekly, yearly), and a residual. STL decomposition (Seasonal-Trend using LOESS) separates them robustly. Decomposition is useful both for understanding the data and as a modeling strategy: forecast each component separately, or remove seasonality before modeling.
A series is stationary if its mean, variance, and autocorrelation do not change over time. Many classical models assume it. Differencing (y't = yt − yt−1), seasonal differencing (yt − yt−m), and log or Box-Cox transforms are the standard ways to get there. The Augmented Dickey-Fuller test checks for a unit root; ACF and PACF plots show which lags carry signal.
ℓt = αyt + (1−α)ℓt−1 for the level-only case.y't = c + Σφiy't−i + Σθjεt−j + εt. SARIMA adds seasonal terms; SARIMAX adds exogenous regressors.These models fit one series at a time, produce prediction intervals naturally, and work well with short histories.
Random K-fold cross-validation is wrong for time series: it trains on the future and tests on the past. Use walk-forward (rolling-origin) evaluation: train on data up to time T, forecast the next h steps, move the origin forward, repeat. scikit-learn's TimeSeriesSplit implements the expanding-window version; a sliding window drops old data when the process drifts.
Leave a gap between train and test equal to the latency of your data pipeline if features would not be available in time in production, and make sure every feature (rolling means, target encodings) is computed only from data available at forecast time.
Turning forecasting into supervised learning: for each time t, build features from the past — lags (yt−1, yt−7), rolling statistics (7-day mean, 28-day std), calendar features (day of week, month, holiday flags), and known-future covariates (planned price, promotions). Then fit gradient boosting on all series at once (a global model) with series identifiers as features. This approach dominated the M5 retail forecasting competition.
Multi-step horizons need a strategy: recursive (predict t+1, feed it back as a lag for t+2 — errors compound), direct (a separate model per horizon), or multi-output. Deep models such as LSTMs, N-BEATS, Temporal Fusion Transformer, and pretrained time-series foundation models (e.g. Chronos, TimesFM) learn the patterns directly from raw windows and shine with many related series.
import numpy as np, pandas as pd
from statsmodels.tsa.holtwinters import ExponentialSmoothing
from sklearn.metrics import mean_absolute_error
import lightgbm as lgb
# Synthetic daily series: trend + weekly season + noise
rng = np.random.default_rng(0)
idx = pd.date_range("2023-01-01", periods=730, freq="D")
t = np.arange(len(idx))
y = pd.Series(100 + 0.05 * t + 10 * np.sin(2 * np.pi * t / 7) + rng.normal(0, 3, len(t)), index=idx)
h = 28
train, test = y[:-h], y[-h:]
# 1) Seasonal-naive baseline: repeat last week
snaive = np.tile(train[-7:].values, h // 7 + 1)[:h]
# 2) Holt-Winters (additive trend + weekly seasonality)
hw = ExponentialSmoothing(train, trend="add", seasonal="add", seasonal_periods=7).fit()
hw_fc = hw.forecast(h)
# 3) LightGBM on lag features (direct strategy: only lags >= h, so no recursion needed)
df = pd.DataFrame({"y": y})
for lag in [28, 35, 42, 364]:
df[f"lag_{lag}"] = df["y"].shift(lag)
df["roll28_mean"] = df["y"].shift(28).rolling(28).mean()
df["dow"] = df.index.dayofweek
df["t"] = t
df = df.dropna()
tr, te = df[df.index < test.index[0]], df[df.index >= test.index[0]]
X_cols = [c for c in df.columns if c != "y"]
gbm = lgb.LGBMRegressor(n_estimators=400, learning_rate=0.03, num_leaves=15, verbose=-1)
gbm.fit(tr[X_cols], tr["y"])
gbm_fc = gbm.predict(te[X_cols])
def mase(actual, pred, insample, m=7):
scale = np.mean(np.abs(insample[m:] - insample[:-m]))
return np.mean(np.abs(actual - pred)) / scale
for name, fc in [("seasonal naive", snaive), ("Holt-Winters", hw_fc), ("LightGBM", gbm_fc)]:
print(f"{name:15s} MAE={mean_absolute_error(test, fc):.2f} MASE={mase(test.values, np.asarray(fc), train.values):.2f}")
Note the trees cannot extrapolate a trend beyond the range seen in training; including t helps little. Detrend first or model the residual from a linear fit when trend matters — a classic gradient-boosting forecasting trap.
| Metric | Notes |
|---|---|
| MAE / RMSE | Scale-dependent; RMSE penalizes large errors more. |
| MAPE | Undefined at zero, explodes near zero, asymmetric. Avoid for intermittent demand. |
| sMAPE / WAPE | WAPE = Σ|e| / Σ|y| is robust and easy to explain to business users. |
| MASE | MAE scaled by in-sample seasonal-naive MAE; < 1 means you beat the baseline. |
| Pinball / quantile loss, CRPS | For probabilistic forecasts (P10/P50/P90), which are what inventory and capacity decisions actually need. |
Simple exponential smoothing keeps one number, the level ℓ, and updates it with ℓt = αyt + (1 − α)ℓt−1. The forecast for every future step is the latest level. With α = 0.5, a starting level of 10 and observations 12, 9, 14:
ℓ1 = 0.5·12 + 0.5·10 = 11ℓ2 = 0.5·9 + 0.5·11 = 10ℓ3 = 0.5·14 + 0.5·10 = 12, so the forecast for all horizons is 12.Unrolling the recursion shows where the name comes from: the forecast is a weighted average of past observations with weights α, α(1−α), α(1−α)2, … — 0.5, 0.25, 0.125 here; with α = 0.2 they would be 0.2, 0.16, 0.128, a much longer memory. A high α reacts quickly to changes but chases noise; a low one is smooth but lags. Holt's method adds a trend term updated the same way, Holt-Winters adds a seasonal term, and a damped-trend variant shrinks the trend toward flat at long horizons, which is often more accurate than extrapolating a straight line indefinitely. In practice α and the other smoothing parameters are fit by maximum likelihood (statsmodels ExponentialSmoothing or ETSModel), and the ETS state-space form also yields prediction intervals.
Real forecasting problems rarely involve one series. Retail demand is forecast per product per store, then summed to regions and the total; energy load by feeder, substation and grid. Independently produced forecasts at each level will not add up, which breaks planning processes that use several levels at once.
Two other structural issues come up constantly. Intermittent demand (many zeros, occasional sales) defeats both percentage metrics and ordinary models; Croston's method and its variants forecast the size of non-zero demand and the interval between demands separately, and a Poisson or Tweedie objective helps gradient-boosting models. Cold-start series (new products) have no history at all, which is where global models that learn from similar items and their attributes earn their keep.
Most forecasting failures in production are data and process problems, not model problems:
Related: Gradient Boosting for the global-model approach, GRU and LSTM for recurrent forecasters, and Anomaly Detection for cleaning history before training.