Learning to Respect Time

Special Topics in Time Series — Part 1

This semester, I started taking a course on time series analytics, and I’d like to share some of the ideas and concepts I’m learning along the way.

---

Accuracy Is Not Enough

Modern machine learning culture often prioritizes predictive performance. But the first philosophical anchor here is different:

Build deep models on statistical foundations. Balance predictive power with interpretability. Before asking, “How do I optimize the model?”

You should ask:

• How is the data represented?
• What structure does time impose?
• What assumptions am I making?

The emphasis is clear: Modeling without structure is fragile.

---

What Makes Time Series Different?

A time series is more than “data indexed by time.”

It is a sequence of observations where order carries information.

Unlike standard supervised learning:

• You cannot randomly shuffle observations.
• You cannot let future information influence training.
• You cannot treat samples as exchangeable.

Time introduces directionality. And directionality introduces constraints.

Interestingly, many forms of data can be reinterpreted as sequences:

• Cognitive scores tracked longitudinally
• Sleep patterns over weeks
• EEG signals
• DNA sequences
• Customer behavior trajectories

Representation matters. If you can frame something as a structured sequence, you can analyze it temporally.

---

Why Interpretability Still Matters

Black-box models perform well, but performance is not always sufficient.

In health applications, for example, if a model flags cognitive decline risk, we must explain:

• Is age driving it?
• Is sleep variability contributing?
• Is there a gradual trend or sudden anomaly?

Interpretability provides:

• Decomposable structure
• Feature-level insight
• Visualizable contributions

This is where Generalized Additive Models (GAMs) become interesting.

---

From Linear Models to Interpretable Deep Learning

A linear model assumes that each variable contributes in a straight-line fashion.

A GAM relaxes that assumption.

Instead of:

$y = \beta_0 + \beta_1 x_1 + \beta_2 x_2$

we allow:

$y = f_1(x_1) + f_2(x_2)$

Each feature can have a smooth, nonlinear effect, but the model remains additive and interpretable.

This is powerful for longitudinal modeling.

For example:

$Risk = f(\text{age}) + g(\text{sleep quality}) + h(\text{physical activity})$

Each component can be visualized separately.

GAMs sit between two extremes:

• Linear models (simple but rigid)
• Neural networks (flexible but opaque)

The interesting transition is toward additive neural architectures, where each feature is processed through its own sub-network:

$y = NN_1(x_1) + NN_2(x_2)$

This preserves decomposability while increasing flexibility. Interpretability need not be sacrificed for expressiveness.

---

The Hidden Problem: Temporal Leakage

Perhaps the most critical technical lesson so far is this:

Many models appear accurate because they accidentally use the future.

Temporal leakage occurs when future information contaminates training.

Common pitfalls include:

• Normalizing using the full dataset (including test period)
• Shuffling observations before splitting
• Applying cross-validation without preserving order
• Creating rolling statistics that implicitly include future values
• Generating lag features incorrectly (e.g., negative shifts)
• Performing decomposition or dimensionality reduction before the train/test split

The result? Artificially optimistic performance.

Proper temporal evaluation requires:

• Training strictly on past data
• Testing strictly on future data
• Using time-aware cross-validation
• Ensuring all engineered features depend only on historical information

For longitudinal cognitive modeling, this is especially important. If future decline patterns leak into feature construction, early detection models become meaningless. Out-of-sample evaluation is not a technicality. It is a methodological discipline.

---

Early Takeaways

• Time has direction, and models must respect it.
• Good validation is a design choice, not an afterthought.
• Interpretability can coexist with flexibility.
• Most mistakes in time series are subtle, not obvious.

---

Estimated reading time: 3 minutes