Learning approach Variants

Learning approach Variants#

Instance-based learning#

  • Learns by memorizing training examples.

  • No explicit model is built.

  • Prediction is made by comparing a new instance with stored instances.

  • Uses a similarity (distance) measure to find closest examples.

Examples:

  • k-Nearest Neighbors (kNN)

  • Locally Weighted Regression

Pros:

  • Simple, flexible.

  • Works well if decision boundary is irregular.

Cons:

  • Expensive at prediction time (must compare with many stored examples).

  • Sensitive to noise and irrelevant features.

Model-based learning#

  • Learns a general model from training data.

  • The model captures underlying relationships, then is used for prediction.

  • Parameters are estimated during training.

Examples:

  • Linear Regression

  • Logistic Regression

  • Neural Networks

  • Decision Trees

Pros:

  • Fast prediction once model is trained.

  • Generalizes well if model is appropriate.

Cons:

  • Training can be computationally heavy.

  • If model is too simple, it underfits; if too complex, it overfits.

Key Difference

  • Instance-based: “Remember examples, predict by similarity.”

  • Model-based: “Learn rules (parameters), predict by applying model.”

Contents