The ML Pipeline: From Raw Data to a Deployed Model

the seven stages, at a glance

Continuing the running example from What Is Machine Learning, Actually? — predicting whether a CI pipeline run will fail — here's what actually happens between "we have some data" and "there's a model running in production":

Stage	Question it answers	In the CI example
1. Data collection	do we have enough representative examples?	pull a year of historical run metadata: diff size, files touched, author, time of day, outcome
2. Data cleaning	is the data even trustworthy?	drop runs that were manually cancelled (no real outcome), fix a logging bug that mislabeled 3 weeks of runs as "passed"
3. Preprocessing	is it in a shape a model can consume?	encode "author" and "branch" as numbers, scale "lines changed," split into train/test
4. Model selection	what kind of function are we fitting?	start with a decision tree — cheap, interpretable, a reasonable baseline
5. Training	how do we make the model good at this?	fit the tree on the training split, watch the error go down
6. Evaluation	is it actually good, or just memorizing?	check accuracy/precision/recall on the held-out test split
7. Deployment & monitoring	does it stay good once it's live?	serve predictions to the CI dashboard, watch for accuracy drift as the codebase evolves

Every dedicated page later in this track — decision trees, gradient descent, regularization, evaluation metrics — is a deep dive into one row of this table. This page is the map; the rest of the beginner section fills in stage 3 onward one concept at a time.

stage 1 — data collection: enough, and representative

Two failure modes show up here, and they're opposites. Not enough data means the model never sees the rare-but-important cases — if only 40 runs in your history actually failed, there isn't much signal to learn "what a failure looks like" from. Unrepresentative data is sneakier: you might have a million runs, but if they're all from one team's repo that never touches the flaky integration-test suite, the model learns a pattern that doesn't hold once it's applied company-wide.

The question to ask before writing a single line of model code isn't "how much data do I have" — it's "does this data cover the situations I actually want to predict for?" A model trained only on daytime runs from one team will confidently mispredict a midnight run from a team it's never seen, because nothing in training ever told it those inputs looked different.

stage 2 — data cleaning: fixing what's broken

Cleaning is about correctness, not shape. Typical problems: missing fields (a run whose lines_changed never got logged), mislabeled outcomes (a flaky test made a run register as "failed" when the code itself was fine), duplicate records (the same run logged twice by a retry), and records that shouldn't be in the dataset at all (a manually cancelled run has no real pass/fail outcome to learn from).

This is unglamorous and it is, in nearly every real project, where most of the calendar time goes — not in tuning the model. A model trained on a small amount of clean, correctly labeled data usually beats one trained on ten times as much data with a labeling bug baked into it, because the model has no way to tell "this label is wrong" from "this label is an unusual but valid pattern."

stage 3 — preprocessing: putting data in a shape a model can use

Where cleaning fixes what's wrong, preprocessing changes what's already-correct data into a form the algorithm can actually consume. Three recurring pieces:

Step	Why it's needed
Numerical encoding	most models only understand numbers — "author: alice" has to become a number or vector before a decision tree or neural net can use it
Scaling / normalization	"lines changed" might range from 1 to 5,000 while "hour of day" ranges 0–23 — some algorithms (and nearly all neural nets) train better when every feature lives on a comparable scale, covered in depth in a later page
Train/test split	held-out data the model never trains on, so evaluation later actually means something — see the dedicated page on this

One preprocessing issue is worth flagging early because it's easy to miss: class imbalance. If 95% of historical runs passed, a model can hit 95% accuracy by predicting "pass" every single time — and learn nothing useful about failures at all. A common fix is class weighting: telling the training process to penalize a missed failure more heavily than a missed pass, so the model is actually pushed to learn the minority pattern instead of ignoring it.


# naive: the model can "win" by never predicting failure
loss = mean(error(prediction, actual) for each sample)

# class-weighted: a missed failure costs more than a missed pass,
# so the model can't shortcut its way to a good score by ignoring failures
loss = mean(class_weight[actual] * error(prediction, actual) for each sample)

stage 4 — model selection: choosing the shape of the function

This is the stage most beginners want to jump straight to, and it's usually the fastest one once the data is actually ready. The question is simply: what family of function are we asking an algorithm to fit? A decision tree, a linear model, and a neural network can all be trained on the exact same preprocessed CI data — they differ in what shapes of pattern they can represent, how interpretable the result is, and how much data they need to train well. This track builds up the individual options (linear regression, decision trees, ensembles, neural networks) one page at a time.

stage 5 — training: measure, diagnose, adjust

Once a model shape is chosen, training is a loop: measure how wrong the model's current predictions are (a loss function), and adjust the model's internal numbers to make that number smaller (an optimization algorithm like gradient descent), repeated until the error stops meaningfully improving. Both of those get their own dedicated pages later in this track — this stage is a two-line summary of what will eventually be several pages of detail.

stage 6 — evaluation: is it good, or just memorizing?

A model that scores perfectly on the data it trained on has told you nothing yet — it might have genuinely learned the pattern, or it might have just memorized the training examples (overfitting). Evaluation means checking performance on the held-out test split from stage 3, data the model never saw during training, and — for a problem like CI-failure prediction where "pass" vastly outnumbers "fail" — checking more than plain accuracy. A model that never predicts "fail" can still look 95% accurate; precision and recall on the failure class are what actually reveal whether it's useful. Both overfitting and the right choice of evaluation metric get their own pages later in this track.

stage 7 — deployment & monitoring: the pipeline doesn't end at a good test score

A model that scored well in evaluation still has to be wired into something that calls it on new, live data — in the CI example, a service the pipeline queries before running the full test suite. Deployment is an engineering problem as much as an ML one: latency, versioning, what happens if the model service is down.

Monitoring is the step beginners most often skip entirely, and it's the reason "we shipped a good model" and "the model is still good six months later" are different claims. Codebases change — a new microservice, a refactored test suite — and the patterns the model learned can quietly stop matching reality. This is drift: the model didn't get worse, the world it was trained on moved. Without monitoring live accuracy against real outcomes, a drifted model can keep serving confidently wrong predictions indefinitely.

practical note: it's a loop, not a line

The seven stages are numbered for clarity, not because real projects walk through them once in order. Evaluation routinely sends you back to preprocessing (a feature turns out to leak information it shouldn't) or even data collection (the model needs a signal that was never logged in the first place). Monitoring in production sends you back to training, with fresh data, on a recurring schedule. Treat the pipeline as a diagram with a "start over here" arrow from every later stage back to an earlier one — because in practice, that arrow gets used constantly.

where to go from here

What Is Machine Learning, Actually? — if you haven't read the taxonomy this page builds on.
Linear Regression: Cost Functions and the Line of Best Fit — stage 4 (model selection), the first concrete algorithm in this track.
ML Basics & Practical Notes — hands-on notes on exactly the kind of preprocessing and leakage pitfalls stage 3 and stage 6 touch on here.
This track continues with decision trees and train/test splitting — stage 4 and stage 6 above, in depth — as the next pages ship.

reference

Google ML Crash Course — framing an ML problem
scikit-learn — train_test_split