Supervised Learning & SVM
Massive textbook-accurate expansion of Supervised Learning fundamentals and Support Vector Machines, including image recognition and the kernel trick.
📋 2 · Supervised Learning & SVM
1 · The General Idea of Supervised Learning
Supervised learning algorithms work with training data where each instance has an input (a set of attributes) and a desired output (a target class). As the textbook clarifies: "We use this data to train a model that will predict the same target class for new unseen instances." These methods are standard tools in disciplines ranging from medical diagnosis to particle physics at the Large Hadron Collider (LHC).
2 · Image Recognition with Support Vector Machines
Imagine instances in your dataset as points in a multidimensional space. Support Vector Machines (SVM) attempt to find a hyperplane that separates instances of one class from the rest in an optimal way, selecting those that pass through the widest possible gaps between instances of different classes.
The "red surface" typically represents the hyperplane with the maximum margin; it is the most distant hyperplane from the closest instances of the two categories. This approach lowers the generalization error, making the model resistant to overfitting.
4 · The Kernel Trick
SVM can construct hyperplanes in high or infinite dimensional spaces using nonlinear surfaces, such as polynomial or radial basis functions (RBF), by using the kernel trick. This implicitly maps inputs into high-dimensional feature spaces without the computational cost of direct transformation.
The SVC implementation in Scikit-Learn defaults to the rbf kernel. The C parameter controls the trade-off between smooth decision boundaries and classifying training points correctly.
5 · Practical Example: Face Recognition
Using the fetch_olivetti_faces dataset, we treat pixel values as learning attributes. Since pixel values are already in a uniform range (0 to 1), normalization is often unnecessary.
from sklearn.svm import SVC
from sklearn.cross_validation import train_test_split
# Initialize SVC with a linear kernel
svc_1 = SVC(kernel='linear')
# Split dataset (75% training, 25% testing)
X_train, X_test, y_train, y_test = train_test_split(
faces.data, faces.target, test_size=0.25, random_state=0)
# Fit the model
svc_1.fit(X_train, y_train)
print(f"Accuracy: {svc_1.score(X_test, y_test)}")