the problem: predicting a number

Back to the CI example from What Is Machine Learning, Actually? — but this time the target isn't pass/fail, it's a number: how many minutes will this run take? That's a regression problem, not a classification one — the output can be any value on a continuous scale, not one of a fixed set of categories. Say we have one feature to start: lines_changed. Plot historical runs as points (lines changed, minutes taken), and the pattern is roughly a straight line trending upward — bigger diffs tend to take longer to build and test. Linear regression is the algorithm that finds the specific straight line that fits that pattern best.

the model: a weighted sum

A line is y = w·x + b: a weight w that scales the input, plus a bias b that shifts the whole line up or down. In ML terms, w and b are the model's parameters — the numbers training will actually adjust. Everything else about the setup (how many features, what learning rate, how many training passes) is a hyperparameter: a choice you make before training starts, not something the algorithm learns on its own. 

predicted_minutes = w * lines_changed + b

# training doesn't touch this formula — it only searches for
# the specific numeric values of w and b that fit the data best
"Best fit" doesn't mean the line passes through every point exactly — real data is noisy (a slow CI runner on a given day, a flaky retry, a sensor-equivalent logging glitch), so a line that hit every point exactly would just be memorizing noise. Linear regression deliberately looks for the line that captures the trend, not one that interpolates every individual observation.

measuring wrong: the cost function

To find "the best line," you first need a number that says how bad a given line is. For each historical run, the error (or residual) is the gap between what actually happened and what the line predicts: error_i = actual_i - predicted_i. The standard way to turn a whole column of per-run errors into one overall score is mean squared error (MSE): 

MSE = (1/n) * sum((actual_i - predicted_i)^2 for i in 1..n)
Squaring isn't arbitrary — it does two things a plain average of the raw errors wouldn't. A run predicted 10 minutes too fast and another predicted 10 minutes too slow would cancel out to zero average error without squaring, even though both are equally wrong; squaring makes every error positive before averaging, so mistakes can't hide behind each other. Squaring also weights large errors much more heavily than small ones — an error of 20 contributes 400, not just "twice as bad" as an error of 10 — which pushes training hard to fix its worst predictions first.
Cost functionFormulaBehavior
Mean Squared Error (MSE)mean((actual - predicted)^2)penalizes large errors heavily; sensitive to outliers
Mean Absolute Error (MAE)mean(|actual - predicted|)treats every unit of error equally; more robust when a few runs have wildly wrong durations (a hung runner, a stuck queue)
MSE is the default for linear regression specifically because it has a useful mathematical property the next section depends on: it's smooth and differentiable everywhere, which is exactly what an optimization algorithm needs to work with.

finding the best line: minimizing the cost

"Fit the line" now has a precise meaning: find the w and b that make MSE as small as possible. For plain linear regression with MSE, there's actually a closed-form formula — the normal equation — that computes the optimal w and b directly from the data in one shot, no trial and error needed. It doesn't scale well to models with many features or to other model types, which is why the more general tool — gradient descent, nudging w and b a little at a time in the direction that reduces the cost — gets its own dedicated page later in this track. Both approaches are minimizing the exact same MSE cost function; they just get there differently.

multiple linear regression

Real predictions rarely depend on one feature. Add more inputs — files changed, hour of day, whether the run touches a known-slow test suite — and the model becomes a weighted sum over all of them, one weight per feature: 

predicted_minutes = (w1 * lines_changed
                    + w2 * files_changed
                    + w3 * touches_slow_suite   # 0 or 1
                    + b)
Nothing about the cost function or the "minimize MSE" goal changes — there are just more numbers to solve for. One practical wrinkle does show up here that didn't with a single feature: lines_changed might range into the thousands while touches_slow_suite is only ever 0 or 1. Left unscaled, gradient descent struggles because the cost surface is stretched much further in one direction than another — covered properly in the feature engineering page later in this track.

beyond a straight line: polynomial regression

If duration doesn't grow linearly with diff size — maybe very large diffs trigger disproportionately slower integration tests — adding a squared term lets the same linear regression machinery fit a curve instead of a line: 

predicted_minutes = w1 * lines_changed + w2 * lines_changed**2 + b
This is still "linear regression" in the technical sense — it's linear in the parameters w1, w2, even though the prediction curve itself is no longer a straight line. The obvious temptation is to keep adding higher powers until the curve threads every single training point — at which point it's stopped modeling the trend and started memorizing the noise. That failure mode, overfitting, gets a full page of its own later in this track.

categorical features aren't free

A weighted sum only understands numbers. A feature like branch_type (feature, release, hotfix) has to become a number before it can be multiplied by a weight at all — but the obvious move, assigning 0, 1, 2, is a trap: it silently tells the model that hotfix is "twice as much" of something as feature, a relationship that was never actually there. This is exactly the "other cases" pitfall for regression inputs, and it's worth flagging here even though the fix (one-hot encoding, mostly) belongs to the feature engineering page later in this track.

why not use this for classification too?

It's tempting to reuse the same line for the pass/fail problem from earlier pages: predict a number, then call anything above 0.5 "fail" and anything below "pass." It breaks in a specific, instructive way — a linear model's output is unbounded, so one unusually huge diff in the training data can drag the whole line's slope around and shift where "0.5" falls for every other point, even ones far from that outlier. Linear regression also has no natural floor or ceiling, so nothing stops it from predicting a "probability" of 1.4 or -0.2, which is meaningless. That's precisely the gap logistic regression — next in this track — is built to close.

practical notes

from sklearn.linear_model import LinearRegression| https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html | fits via the normal equation under the hood -- fine for datasets small enough to fit in memory, not what production-scale training uses |'lr_sk1'
model.fit(X_train, y_train); model.predict(X_test)| | fit() solves for w and b on the training split; predict() applies that fixed line to new inputs -- it never "keeps learning" after fit() returns |'lr_sk2'
One gotcha worth internalizing early: a fitted line is only trustworthy inside the range of the data it was fit on. A model trained on diffs of 1–2,000 lines has no real basis for predicting the duration of a 50,000-line diff — it will still return a number (linear functions are defined everywhere), just not a meaningful one. Extrapolating past the training range is a common, easy-to-miss source of confidently wrong predictions.

where to go from here

Logistic Regression — the classification counterpart to this page, reusing the same weighted-sum model.
The ML Pipeline — if you want the bigger picture this page (stage 4, model selection) fits into.
ML Basics & Practical Notes — practical scaling and leakage notes that apply directly to fitting a model like this one.

reference

scikit-learn — Ordinary Least Squares
Google ML Crash Course — Linear Regression