Assumptions#
1. Nearby points have similar labels (locality assumption)#
Core assumption:
If two points are close in feature space, they are likely to have the same class (classification) or similar values (regression).
This is basically saying the data is locally smooth.
2. The chosen distance metric is meaningful#
KNN assumes the distance function (Euclidean, Manhattan, cosine, etc.) correctly reflects similarity.
If features are on different scales (e.g., height in cm vs income in lakhs), distance won’t make sense unless you normalize/standardize.
3. Features are equally important (unless weighted)#
Default KNN assumes every feature contributes equally to distance.
If one feature is irrelevant or has high variance, it can distort distances.
Sometimes solved with feature scaling, feature selection, or distance weighting.
4. Decision boundary is locally smooth#
KNN assumes the decision boundary between classes is not too irregular, so local neighborhoods give meaningful predictions.
With noisy data or highly overlapping classes, this assumption breaks.
5. No strong independence assumption (unlike Naive Bayes)#
Unlike probabilistic models, KNN does not assume feature independence.
But it does assume “closeness in feature space = similarity in outcome”.
This assumption may fail in high-dimensional spaces (curse of dimensionality).
6. Data distribution is representative & dense#
KNN works best when:
Training data covers the space well (dense enough).
Otherwise, new points may fall in empty regions, making prediction unreliable.
This means:
KNN assumes training data is large, clean, and representative of the population.
In summary
KNN assumes:
Local smoothness → neighbors have similar outcomes.
Distance metric is meaningful.
Features are scaled & equally important (unless weighted).
Decision boundary is not too complex.
Sufficient data density for reliable neighborhoods.