Performance Metrics

Performance Metrics#

Mean Absolute Error (MAE)#

\[ MAE = \frac{1}{n} \sum_{i=1}^{n} |y_i - \hat{y}_i| \]

  • Measures the average magnitude of errors (absolute difference between actual and predicted).

  • Pros: Simple, interpretable, robust to outliers.

  • Cons: Doesn’t penalize large errors strongly.

👉 Example: If house price is $100,000 and prediction is $90,000 → error = $10,000.

Mean Squared Error (MSE)#

\[ MSE = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2 \]

  • Measures average squared error.

  • Pros: Penalizes large errors more than small ones.

  • Cons: Not interpretable in the same unit as target (because it’s squared).

👉 Example: Errors of 10 and 100 → Squared = 100 and 10,000 → large errors dominate.

Root Mean Squared Error (RMSE)#

\[ RMSE = \sqrt{MSE} \]

  • Square root of MSE.

  • Same units as the target variable → easier to interpret.

  • Sensitive to outliers (because of squaring).

👉 If predicting house prices in dollars, RMSE also comes in dollars.

R-Squared ( \(R^2\) )#

\[ R^2 = 1 - \frac{SS_{res}}{SS_{tot}} \]

Where:

  • \(SS_{res} = \sum (y_i - \hat{y}_i)^2\) → residual sum of squares

  • \(SS_{tot} = \sum (y_i - \bar{y})^2\) → total variance

  • Represents proportion of variance explained by the model.

  • Values range from 0 to 1 (closer to 1 = better fit).

  • Can be negative if model performs worse than baseline mean prediction.

👉 Example: \(R^2 = 0.85\) → model explains 85% of the variance in the data.

Adjusted R-Squared#

\[ R^2_{adj} = 1 - (1 - R^2)\frac{n-1}{n-p-1} \]

Where:

  • \(n\) = number of data points

  • \(p\) = number of features

  • Adjusts R² for the number of predictors.

  • Prevents artificial inflation of R² when irrelevant features are added.

Summary Table#

Metric

Intuition

Penalizes Large Errors?

Units

MAE

Avg. absolute error

❌ No

Same as target

MSE

Avg. squared error

✅ Yes (strongly)

Squared units

RMSE

Square root of MSE

✅ Yes

Same as target

R²

Variance explained

❌ No

Ratio (0–1)

Adj. R²

R² adjusted for #features

❌ No

Ratio (0–1)

Rule of Thumb:

  • Use MAE when interpretability is important.

  • Use RMSE when large errors matter more.

  • Use R² / Adjusted R² to judge overall model fit.

Contents