SVM

SVM#

Support Vector Machine (SVM) is a supervised machine learning algorithm used for classification and regression. Its goal is to find the optimal decision boundary (hyperplane) that separates data points of different classes with the maximum possible margin.

Key ideas:#

  1. Hyperplane: A line in 2D, a plane in 3D, or a higher-dimensional surface that divides classes.

  2. Margin: The distance between the hyperplane and the nearest data points from each class. SVM maximizes this margin.

  3. Support Vectors: The critical data points that lie closest to the hyperplane. They “support” the decision boundary.

  4. Linear vs. Nonlinear separation:

    • If classes are linearly separable, SVM finds a straight hyperplane.

    • If not, SVM uses the kernel trick (e.g., polynomial, RBF) to project data into higher dimensions where it becomes linearly separable.

  5. Soft Margin: Allows some misclassifications (controlled by parameter C) to balance margin size and error.

  6. Applications: Text classification, image recognition, bioinformatics (like gene classification), anomaly detection.

Advantages:#

  • Effective in high-dimensional spaces.

  • Works well with clear margin of separation.

  • Memory efficient since only support vectors are used.

Limitations:#

  • Training can be slow on very large datasets.

  • Choice of kernel and hyperparameters is crucial.

  • Less effective when classes overlap heavily.

Click here for Sections
Contents