In Algebra 1 you wrote linear models and drew lines of fit by eye. Different people draw different lines. The least-squares regression line is the one line everyone agrees on, and it is built from the correlation and the two standard deviations.
Key idea
The least-squares line $\hat{y} = a + bx$ makes the sum of squared residuals $\sum (y - \hat{y})^2$ as small as possible. Its slope and intercept are
$$b = r\dfrac{s_y}{s_x}, \qquad a = \bar{y} - b\bar{x}.$$
The line always passes through the point of means $(\bar{x}, \bar{y})$.
Why it works
Work in z-scores first. A line through the origin $\hat{z}_y = m z_x$ has squared error $$\sum (z_y - m z_x)^2 = \sum z_y^2 - 2m\sum z_x z_y + m^2 \sum z_x^2$$ Each sum of squared z-scores is $n - 1$, and $\sum z_x z_y = (n - 1)r$, so the error is $(n - 1)(1 - 2mr + m^2)$, a parabola in $m$ with its lowest point at $m = r$. So the best line in z-units is $\hat{z}_y = r z_x$: one SD up in x predicts $r$ SDs up in y. Turning z-scores back into x and y gives $b = r s_y/s_x$, and the line goes through $(\bar{x}, \bar{y})$.
Worked example: Building the line from summaries
For students' study hours (x) and scores (y): $\bar{x} = 6$, $s_x = 2$, $\bar{y} = 75$, $s_y = 10$, $r = 0.6$. Find the least-squares line and predict the score for 8 hours.
- Slope: $b = 0.6 \cdot \dfrac{10}{2} = 3$ points per hour.
- Intercept: $a = 75 - 3(6) = 57$.
- Line: $\hat{y} = 57 + 3x$.
- Predict: $\hat{y} = 57 + 3(8) = 81$.
Answer: $\hat{y} = 57 + 3x$; about 81 points.
Worked example: A prediction too far
The students in that data studied between 2 and 10 hours. What does the line predict for 30 hours, and should you trust it?
- $\hat{y} = 57 + 3(30) = 147$ points.
- The test is out of 100, so the prediction is impossible.
- The data only show the pattern between 2 and 10 hours. Beyond that, the line has no support.
Answer: 147, which is nonsense: this is extrapolation.
Common mistake
Extrapolation: using the line for x-values far outside the range of the data. The straight-line pattern may bend, level off or stop. Check: is the x you are plugging in between the smallest and largest x in the data?
Because $b = r s_y/s_x$ and $|r| < 1$, a student 1 SD above average in study time is predicted to be only $r$ SDs above average in score. Predictions are pulled toward the mean: this is "regression to the mean." The econometrics course builds on this line in Simple Regression: Fitting a Line.