SVC

SVC #

Support Vector Classifier (SVC) is the classification version of Support Vector Machines.

Core idea:

Data points are separated by a hyperplane.
The algorithm chooses the hyperplane that maximizes the margin (distance to nearest data points).
The nearest data points on each side are the support vectors.

Mathematical form: For input \(x\), prediction is:

\[ f(x) = \text{sign}(w^T x + b) \]

where \(w\) and \(b\) define the separating hyperplane.

Soft margin:

Perfect separation may not be possible.
Slack variables \(\xi_i\) allow misclassifications.
Trade-off controlled by regularization parameter \(C\):
- Large \(C\): less tolerance for errors (narrow margin).
- Small \(C\): more tolerance, wider margin, better generalization.

Non-linear boundaries:

Kernel trick replaces dot product with kernel function \(K(x_i, x)\).
Common kernels: linear, polynomial, RBF (Gaussian).

Summary:

SVC = SVM applied to classification.
Works well with high-dimensional data.
Requires scaling of features.
Sensitive to \(C\), kernel, and kernel parameters.

import numpy as np
import matplotlib.pyplot as plt
from sklearn import svm
from sklearn.datasets import make_blobs

# Generate synthetic data
X, y = make_blobs(n_samples=100, centers=2, random_state=6)

# Fit SVM classifier
clf = svm.SVC(kernel='linear', C=1)
clf.fit(X, y)

# Plot data points
plt.figure(figsize=(8,6))
plt.scatter(X[:, 0], X[:, 1], c=y, cmap=plt.cm.coolwarm, s=50, edgecolors='k')

# Plot decision boundary
ax = plt.gca()
xlim = ax.get_xlim()
ylim = ax.get_ylim()

# Create grid to evaluate model
xx = np.linspace(xlim[0], xlim[1], 30)
yy = np.linspace(ylim[0], ylim[1], 30)
YY, XX = np.meshgrid(yy, xx)
xy = np.vstack([XX.ravel(), YY.ravel()]).T
Z = clf.decision_function(xy).reshape(XX.shape)

# Plot decision boundary and margins
ax.contour(XX, YY, Z, colors='k', levels=[-1, 0, 1], alpha=0.7, linestyles=['--', '-', '--'])

# Plot support vectors
ax.scatter(clf.support_vectors_[:, 0], clf.support_vectors_[:, 1], s=120, linewidth=2, facecolors='none', edgecolors='k', label='Support Vectors')

plt.title('SVM Decision Boundary and Support Vectors')
plt.xlabel('Feature 1')
plt.ylabel('Feature 2')
plt.legend()
plt.show()

SVC

SVC#

SVC #