Chapter 1 · Lesson 1.5
Model life cycle: drift and explainability
Updated: November 2026. Sources: Ribeiro, Singh and Guestrin, “Why Should I Trust You?” (LIME), KDD 2016; Lundberg and Lee, “A Unified Approach to Interpreting Model Predictions” (SHAP), NeurIPS 2017; Lazer et al., “The Parable of Google Flu”, Science, 2014. Operation and incidents: lesson 6-6.
A trained model is not software that you deliver once and for all. It was trained on a snapshot of the past; the world, for its part, keeps moving. And when it gets things wrong, you still have to understand why.
The life cycle of a model
Drift: when the world changes under the model
Data drift
The inputs change: a sensor ages and reads 3 °C too high, winter follows summer, a new product range arrives on the line. The learned rule remains true, but the model sees cases it has never encountered.
Concept drift
The relation between inputs and outcome changes: a new pump normally vibrates harder than the old one, a fraud changes its modus operandi. The inputs may look normal while the learned rule has become wrong.
A famous example: Google Flu Trends estimated flu from Google searches. In February 2013, it announced more than double the consultations for influenza-like illness measured by the US health authorities (Lazer et al., 2014). Search habits and the search engine itself had changed; the model had not.
Monitor the inputs (do their distributions move away from those of training?), the sorties (does the alert rate change for no reason?) and, as soon as ground truth arrives, the actual performance. Set alert thresholds in advance and decide what to do when they are crossed: these are risk indicators like any other.
Explainability: understanding a decision
A neural network does not give its reasons. Some methods estimate after the fact the contribution of each variable to a prediction:
- LIME (2016) locally replaces the model with a simple model around the case to be explained.
- SHAP (2017) distributes the gap between the prediction and the average among the variables, according to Shapley values from game theory.
They are useful to debug (is the model looking at the right thing?), contester a decision or detect a variable that serves as a proxy for a protected characteristic (lesson 3-7). But they are approximations : two methods can give two different explanations, and a plausible explanation is not proof that the model reasons correctly.
Hands-on: an alert model that drifts
Going further: measuring drift
The most common indicator in industry is thepopulation stability index (PSI): the variable is split into bins, and the share p of the training data and the share q of the recent data are compared in each bin: PSI = Σ (q − p) · ln(q / p). A widely used rule of thumb, with no normative value: below 0.1 stable, 0.1 to 0.25 to be watched, above 0.25 marked drift. The Kolmogorov-Smirnov test or the distance between distributions are also used.
For a linear model with independent variables, the SHAP contribution of a variable is exactly w · (x − mean): this is what the demo displays. For a neural network, SHAPestime by sampling, with some uncertainty.
A model degrades when the inputs change (data drift) or when the learned relationship changes (concept drift), without anyone having touched the code. Inputs, outputs and actual performance are monitored, with thresholds set in advance. LIME and SHAP estimate the contribution of variables to a decision: useful for debugging and challenging, but they are approximations, not proof.