Time Series with Machine Learning (Part 5)

Part 5: The Final Preparations Before Modelling

The feature engineering is almost done. Four families of features now cover calendar position, lagged history, rolling averages, and exponentially weighted averages. Two housekeeping steps remain before the model can be trained: encoding the categorical columns properly and transforming the target variable.

One-Hot Encoding

The dataset still contains several columns where the values are integers that represent categories: store (1–10), item (1–50), day_of_week (0–6), and month (1–12). Leaving them as integers sends the model a false signal. An integer column implicitly says there is an ordering and a magnitude: store 10 is somehow “greater” than store 1, and the distance between them is exactly 9. Neither is true. Store IDs are labels, not measurements.

The Problem with Integer Codes

The real-world bar chart makes this concrete. Store 2 has the highest average daily sales in the dataset, while Store 7 has the lowest. They are not neighbours on any meaningful scale. A model that treats their IDs as numbers will try to fit a linear or monotonic relationship between the store integer and sales, a relationship that simply does not exist in the data.

Creative writing process for ideas development.
Figure 1. Left: treating store IDs as integers implies a numeric ordering and equal spacing between stores. Right: actual average sales by store, no monotonic pattern, so an integer encoding would mislead the model.

The Fix: Binary Columns

One-hot encoding removes this assumption entirely. Each category level gets its own binary column: store_1, store_2, …, store_10. A row belonging to Store 2 gets a 1 in the store_2 column and a 0 everywhere else. The model now has no implied distance between stores — it treats each one as a completely independent identity.

Creative writing process for idea development.
Figure 2. The store column before and after one-hot encoding. Each store ID becomes its own binary flag column. The same process is applied to item, day_of_week, and month.

The same transformation is applied to item (50 levels), day_of_week (7 levels), and month (12 levels). Together with the 10 store levels, that adds 10 + 50 + 7 + 12 = 79 binary columns. Though in practice, one column per group can be dropped to avoid perfect multicollinearity, leaving 76. Combined with the 70 features already built, the total reaches 146.

Handwriting notes on a notebook with a laptop nearby for idea development.
Figure 3. How the feature set accumulates from 4 raw columns to 146. OHE alone contributes more than half the total column count, all of them simple binary flags.A person writes ideas in a notebook, illustrating the creative process of developing thoughts for writing projects.

A note on tree models specifically: LightGBM can in principle handle categorical integers natively by treating each split differently. However, one-hot encoding makes the categoricality explicit and consistent, avoids any ambiguity about ordering, and keeps the feature representation uniform across all variable types. For a dataset of this size, the extra columns are not a performance concern.

Transforming the Target: log(1 + sales)

The second step applies to the target variable itself, not the features. Sales is replaced by its natural logarithm, specifically, log(1 + sales), also written as log1p. The +1 inside the logarithm prevents a problem when sales is zero: log(0) is undefined, but log(1 + 0) = log(1) = 0. Every sales value maps to a valid, finite number.

Why Transform the Target?

Why do this at all? The raw sales distribution is right-skewed, most days see 20–60 units sold, but a long tail stretches past 150. In gradient-boosted trees, the model iteratively fits residuals (the gap between predictions and actuals). When the target has a long tail, a handful of high-sales observations produce very large residuals that dominate the gradient updates. The model spends most of its effort chasing those outliers rather than learning the general pattern.

The log transform compresses the tail. A sale of 200 units becomes log(201) ≈ 5.30. A sale of 10 units becomes log(11) ≈ 2.40. The ratio between them shrinks from 20:1 to roughly 2.2:1. The distribution becomes more symmetric, residuals are more balanced across the range, and the model converges more reliably.

  • raw:      sales ∈ [0, 231],   skew = 0.87
  • log1p:    log(1+sales) ∈ [0, 5.44],  skew = -0.38
Auto Draft.
Figure 4. Sales distribution before (left) and after (right) the log1p transform. The right skew of 0.87 becomes a near-symmetric -0.38. Mean and median, which were 5 units apart, are now almost identical.

Inverting Predictions After Training

One practical consequence: predictions from the model will come out in log-scale. To get back to actual unit sales, which is what the competition evaluates, each prediction must be inverted with expm1: the inverse of log1p. If the model predicts 3.85, the actual sales estimate is expm1(3.85) = e^3.85 − 1 ≈ 46 units.

predicted_sales  =  expm1( model_output )  =  e^(model_output) − 1

This is an optional step, the model would still train without it. But for a gradient descent-based method working on a skewed target, the log transform typically speeds up training and produces slightly better-calibrated predictions, especially at the lower end of the sales range where errors matter proportionally more.

The feature engineering phase is now complete. Starting from four raw columns; date, store, item, sales the dataset has been expanded to 146 input features covering calendar patterns, lagged history, rolling averages, exponentially weighted means, and one-hot encoded identities. The target variable has been stabilised with a log transform.

The next post will be about modeling.

Leave a Reply

Create a website or blog at WordPress.com

Up ↑