Data Leakage: The 97% Test Score That Became 78% in Production

A model scores 97% accuracy in testing — the percentage of predictions it gets right on data it never trained on. You deploy it to production, and real-world accuracy drops to under 78%. Nothing crashed. Nothing was miscoded.

So what actually broke? Nothing broke. The test score was measuring something that doesn't exist in the real world.

This is data leakage — the quietest, most dangerous failure mode in machine learning, because it doesn't look like a failure at all. It looks like success.

What Data Leakage Actually Is

Data leakage is information that shouldn't be available at prediction time sneaking into training anyway. The model learns to rely on it. The test score looks great, because that same leaked information is sitting in the test set too. The moment the model is in the real world, making a prediction before that information exists, it falls apart.

Isn't more information always better for a model, though? Not if the model will never actually have that information at the moment it needs to predict. A feature that helps in testing but doesn't exist yet at prediction time isn't a helpful feature — it's a leak.

A Feature That Looks Helpful But Can't Exist Yet

Take a loan default model with two honest features: income, and loan amount. Someone adds a third column: whether the account is currently in collections.

Here's the problem: an account only shows up "in collections" after the person has already defaulted. At the moment this model would actually be used — when someone applies for the loan — that column doesn't have a value yet. It can't. The outcome hasn't happened.

So why would training on it even work at all? Because the training data was built after the fact, with the outcome already known — so the leaked column and the label it's supposed to predict end up telling the model almost the same thing.

Watching the Inflation Happen

Train the exact same model twice on the same simulated loan data — once with the leaky collections_flag column, once without it:

model_leaky = LogisticRegression()
model_leaky.fit(X_train_leak, y_train)
acc_leaky = accuracy_score(y_test, model_leaky.predict(X_test_leak))

model_clean = LogisticRegression()
model_clean.fit(X_train_clean, y_train)
acc_clean = accuracy_score(y_test, model_clean.predict(X_test_clean))
Bar chart comparing 97.0% test accuracy with the leaky feature against 77.7% without it

97.0% with the leaky feature. 77.7% without it — the honest number, verified by actually running this exact code. The leaky feature didn't make the model smarter. It let the model cheat on a test it will never get to cheat on again once it's deployed.

The Fix Is a Discipline, Not a Feature List

The test to apply to any feature: would you actually know this value at the moment you need to make the prediction — before the outcome happens? If the answer is no, it doesn't belong in training, no matter how much it helps the accuracy number.

But is it only obviously-cheating features like this one that cause leakage? Not even close — leakage also happens somewhere much easier to miss. If a mean for scaling, a category list for encoding, or a fill-in value for missing data gets calculated using the entire dataset — test data included — before the split, information from the test set has already leaked into training. The fix is the same discipline every time: split first, and calculate everything else only from the training data, after that split.

Why This Is Worse Than a Crash

Isn't a crash the worst thing that can happen to a model? Not really — a crash tells you something's wrong immediately. A scaling mismatch produces a worse model, but at least the model is still visibly trying. Leakage produces a model that looks better than it should — it gets shipped with confidence, and it fails somewhere nobody's watching for it.

Get the Code

The full simulation — leaky vs. honest logistic regression, run on the same synthetic loan data, with the exact numbers above: DataLeakage.py.

Summary

Data leakage happens when information that wouldn't exist yet at prediction time sneaks into training — a feature calculated after the fact, or a statistic computed on the full dataset before splitting. It inflates the test score (97.0% in this simulation) without making the model any better; the honest number, once the leaky feature is removed, was 77.7%. The fix is a discipline, not a checklist: split the data first, and calculate every feature and statistic only from the training side of that split. Unlike a crash or a silent bug, leakage produces a model that looks better than it should — which is exactly why it's the failure mode most likely to ship unnoticed.

Comments

Post a Comment

Popular posts from this blog

Working with Multi-Member Physical Files - IBM i

Multi-member Physical Files At times, Multi-member physical files are very helpful in IBM i. A physical file is a file that stores data in a specific format. A physical file can have multiple members, each of which is act as a separate data set within the file.  This allows us to store different types of data (format of the file remains same) or data for different purposes within the same file. Creating a Multi-member Physical File Create a physical file (PF) with single member (default) and add an addition member after PF is created. A physical file is created by using CRTPF command. CRTPF FILE(<Library>/<File>) MBR(<Member Name>) By default, Member is created with the same name as the file name. If a specific name is required, member name can be added as well.  CRTPF FILE(LIBRARY/FILE) RCDLEN(10) MBR(MEMBER1) TEXT('Temporary File') For the simplicity, I have created the flat file by directly mentioning the record length (RCDLEN) as 10. We would usually be c...
Read more

Gradient Descent: The Update Rule That Trains Every Model

Image
Every model — a neural network, a regression line, anything that learns from data — is really just a set of internal numbers called parameters, plus a single score called the loss that measures how wrong its current predictions are. Every parameter has a slope: nudge it up or down, and the loss changes. Stack all those slopes into one vector and you get the Gradient . It always points uphill — toward the steepest way to make the loss worse — whether you have 2 parameters or 175 billion. But knowing the uphill direction isn't the same as knowing what to do with it. The gradient only points up. So what good is that to a model trying to reach the bottom? Gradient Descent takes "here is uphill" and turns it into "here is your next step downhill." That's the whole idea. Every neural network, every regression model, every fine-tuning run learns using exactly this. The Update Rule Here is the entire algorithm, in one line: θ(...
Read more

Why Linear Regression Actually Works: Simulating the Central Limit Theorem

Image
Why Linear Regression Actually Works: Simulating the Central Limit Theorem In Linear Regression, we almost always use Mean Squared Error (MSE) as our loss function. When we apply Gradient Descent, we assume this function forms a nice, convex shape. We find the bottom of this bowl, minimize the error, and assume our parameters are optimal. But there is a hidden assumption here: We assume the noise (error) in our data follows a Normal Distribution. In the real world, data is messy. It rarely follows a perfect Bell Curve. It might be uniform, skewed, or exponential. This leads to a critical question: Why does minimizing squared errors work even when the original data is NOT normal? The answer lies in a statistical law so powerful it feels like a cheat code: The Central Limit Theorem (CLT) . What is the Central Limit Theorem? The Central Limit Theorem states that if you extract sufficiently large samples (ideally n ≥ 30) fr...
Read more