Time Series with Machine Learning (Part 4)

Exponentially Weighted Mean Features

The rolling mean from the previous post computes a simple average over a fixed window of past values. Every observation inside the window contributes equally, a sale from 364 days ago counts just as much as a sale from 2 days ago. The triangular window improved on this slightly by weighting recent values more, but only linearly. There is a more principled way to weight history: exponential decay.

An exponentially weighted mean (EWM) assigns each past observation a weight that shrinks exponentially the further back it sits. The most recent value gets the highest weight. Every step further into the past multiplies the weight by a constant factor smaller than one. The result is a smooth, continuously updating average that naturally forgets old data, fast or slow, depending on one parameter: alpha.

Alpha — The Speed of Forgetting

Alpha (α) is a number between 0 and 1. It controls how quickly the exponential weight decays as you move into the past. The weight assigned to an observation k steps ago follows this rule:

weight at lag k  =  α · (1 − α)^k

At k=0 (the most recent past observation, after shifting), the weight is α itself. Additionally, at k=1, it is α·(1−α). At k=2, it is α·(1−α)². Each additional step multiplies by (1−α) again. The smaller (1−α) is, that is, the larger α is, the faster the weights collapse toward zero.

Think of it this way. With α = 0.99, the term (1−α) = 0.01. Two steps into the past, the weight has already shrunk to 0.99 × 0.01² = 0.000099. The model is essentially ignoring everything except the very last observation. With α = 0.1, (1−α) = 0.9, and the weights decay slowly, an observation 20 steps ago still carries about 12% of the weight of the most recent one. The model maintains a long, gradually fading memory of the past.

This is what distinguishes EWM from a rolling mean. The rolling mean treats all values inside the window as equals and ignores everything outside. EWM has no hard window; every past observation contributes, with weight fading continuously. High alpha is a short, intense memory. Low alpha is a long, gentle one.

How It Looks on a Real Series

Plotting EWM alongside the raw series and a rolling mean makes the difference concrete. A high-alpha EWM tracks the series almost like a lagged copy, it reacts immediately to every spike and drop. A low-alpha EWM looks more like a trend line, absorbing individual fluctuations into a broad average. The rolling mean sits somewhere in between, but with a hard cutoff at the window boundary rather than a smooth fade.

Notice that α = 0.99 and the raw series are nearly identical, which is exactly what you would expect given how severely it down-weights everything beyond the most recent observation. At the other extreme, α = 0.10 barely moves: it is carrying the weighted average of many months of history, so any single day’s spike barely registers.

Shift First — Same Rule as Rolling Mean

Just like the rolling mean, EWM must be computed on a shifted series. Without shifting, the EWM at row t would incorporate y(t), the value being predicted, which is data leakage. Shifting by one row first ensures the exponential average at row t is built entirely from y(t−1), y(t−2), and so on.

The table below shows what happens at each row. At t=0, there is no prior value after shifting, so the EWM is NaN. At t=1, the only available observation is y(0) = 13, so all alpha values produce 13.00 — identical, because there is only one data point to weight. From t=2 onward, the alpha values begin to diverge: high alpha immediately chases the most recent value (11), while low alpha is pulled back toward the older observation (13).

This is exactly what the notes show: at t=3, actual sales is 13 and the three values to predict from are 11 (t=2), 13 (t=1), 14 (t=0). As alpha decreases, the EWM pulls away from 11 and toward the older values, slowly converging to their weighted blend. The lower alpha, the more the model remembers.

Adding the Lag Dimension

There is a second design choice on top of alpha: how far back to start the EWM window. In the code, .shift(lag) is applied before .ewm(), not just .shift(1). This means the exponential average does not begin at the immediately preceding observation; it begins at a point lag days in the past and then accumulates backwards from there.

Why do this? Because the forecast horizon is 91 days. If the model needs to predict sales for January 1st 2018, the last day it can observe is October 1st 2017 (91 days prior). There is no point computing an EWM that starts at t−1 if t−1 is in the forecast period and therefore unavailable. The minimum safe lag is 91 days, the same constraint that governed the lag features in Part 2.

The lags chosen are 91, 98, 105, 112, 180, 270, 365, 546, and 728 days. Each one anchors the EWM at a different distance into the past and then accumulates the exponentially weighted history from that anchor point backward. Combined with 5 alpha values, this produces 45 distinct features, each capturing a different combination of how far back to look and how quickly to down-weight older observations.

The correlation heatmap reveals something important. The signal is not uniform across lags. Lag 365 and lag 728, exactly one and two years back, show consistently strong positive correlation with actual sales, regardless of alpha. This makes sense: the same month last year is a natural predictor of this month. Lag 180 and 270, six to nine months back, show near-zero or negative correlation. Looking back to the opposite season is not helpful and can even mislead the model. This is exactly the kind of structure EWM with multiple lags is designed to capture: not all history is equally useful, and the grid of alpha × lag combinations lets the model sort out which combinations carry the most predictive signal.

With EWM features added, the feature engineering stage is complete. The dataset now contains date features, lag features, rolling mean features, and exponentially weighted mean features, four complementary families of signal, each encoding a different aspect of temporal structure. Together they give the model a rich, structured view of time without any explicit time-series equations.