Improving Accuracy: Tips and Tricks for Eigenfaces Models

Implementing Eigenfaces in Python: Step-by-Step TutorialFace recognition is a classic problem in computer vision. One of the earliest and most influential approaches is the Eigenfaces method, which uses Principal Component Analysis (PCA) to represent faces in a lower-dimensional subspace. This tutorial walks through the concepts, mathematics, and a complete Python implementation with practical tips and evaluation.

What you’ll learn

• What Eigenfaces are and why they work
• How to prepare and preprocess a face dataset
• How to compute Eigenfaces using PCA
• How to project faces into the Eigenface space and reconstruct them
• How to build a simple face recognition system using nearest-neighbor search
• How to evaluate performance and improve results

1. Theory: Eigenfaces and PCA (brief)

Eigenfaces model faces as linear combinations of a set of basis images — the eigenvectors of the covariance matrix computed from a set of aligned, normalized face images. PCA finds directions (principal components) that capture the largest variance in the data. For face images, the top principal components often correspond to meaningful facial features (overall face shape, eyes, mouth, lighting patterns).

Mathematically:

• Stack N flattened grayscale images of size H×W into a data matrix X of shape (N, D) where D = H·W.
• Subtract the mean face μ: Xc = X − μ.
• Compute covariance matrix C = (1/N) Xcᵀ Xc (D×D). Direct computation can be expensive when D is large.
• Use the trick of computing the smaller N×N matrix L = (1/N) Xc Xcᵀ and find its eigenvectors; eigenvectors of C can be obtained as linear combinations of Xcᵀ and eigenvectors of L.
• The eigenvectors (reshaped to H×W) are the Eigenfaces. Project images onto the top K eigenvectors to get compact representations.

2. Dataset and preprocessing

Choose a dataset of aligned, grayscale face images. Good options:

• AT&T (ORL) Faces dataset (small, well-aligned)
• Yale Face Database
• Labeled Faces in the Wild (LFW) — requires alignment for best results
• Your own collected dataset (ensure faces are aligned and cropped consistently)

Preprocessing steps:

• Convert to grayscale.
• Crop/align faces so eyes/nose roughly match positions across images.
• Resize to a manageable resolution (e.g., 64×64 or 100×100).
• Flatten each image to a 1D vector and stack into an array.
• Normalize pixel values (e.g., scale to [0,1] or mean-center).

Practical tip: Smaller image sizes reduce computation but remove detail. 64×64 is a common compromise.

3. Full Python implementation

Below is a complete implementation using NumPy and scikit-learn for PCA and example code to load the ORL dataset using scikit-learn utilities. Replace dataset loading with your own images if needed.

# eigenfaces_tutorial.py import os import numpy as np from sklearn.datasets import fetch_olivetti_faces from sklearn.model_selection import train_test_split from sklearn.decomposition import PCA from sklearn.neighbors import KNeighborsClassifier from sklearn.metrics import classification_report, accuracy_score import matplotlib.pyplot as plt # 1. Load dataset (Olivetti faces included in scikit-learn) data = fetch_olivetti_faces(shuffle=True, random_state=42) images = data.images         # shape (400, 64, 64) X = data.data                # shape (400, 4096) y = data.target              # labels 0..39 # 2. Train/test split X_train, X_test, y_train, y_test, img_train, img_test = train_test_split(     X, y, images, test_size=0.25, stratify=y, random_state=42 ) h, w = images.shape[1], images.shape[2] # 3. Compute mean face and PCA (Eigenfaces) n_components = 100  # number of eigenfaces pca = PCA(n_components=n_components, svd_solver='randomized', whiten=True, random_state=42) pca.fit(X_train) eigenfaces = pca.components_.reshape((n_components, h, w)) mean_face = pca.mean_.reshape(h, w) # 4. Project training and test images into Eigenface space X_train_p = pca.transform(X_train) X_test_p = pca.transform(X_test) # 5. Train a simple classifier in the reduced space knn = KNeighborsClassifier(n_neighbors=3) knn.fit(X_train_p, y_train) y_pred = knn.predict(X_test_p) print("Accuracy:", accuracy_score(y_test, y_pred)) print(classification_report(y_test, y_pred)) # 6. Visualization: mean face and top eigenfaces plt.figure(figsize=(12, 6)) plt.subplot(2, 6, 1) plt.imshow(mean_face, cmap='gray') plt.title("Mean face") plt.axis('off') for i in range(10):     plt.subplot(2, 6, i+2)     plt.imshow(eigenfaces[i], cmap='gray')     plt.title(f"Eigenface {i+1}")     plt.axis('off') plt.show() # 7. Reconstruct an example face from its projection idx = 5 proj = pca.transform(X_test[idx].reshape(1, -1)) recon = pca.inverse_transform(proj).reshape(h, w) plt.figure(figsize=(8,4)) plt.subplot(1,3,1); plt.imshow(img_test[idx], cmap='gray'); plt.title('Original'); plt.axis('off') plt.subplot(1,3,2); plt.imshow(recon, cmap='gray'); plt.title('Reconstruction'); plt.axis('off') plt.subplot(1,3,3); plt.imshow((img_test[idx]-recon), cmap='seismic'); plt.title('Difference'); plt.axis('off') plt.show() 

Notes:

• Olivetti (fetch_olivetti_faces) provides 400 images of 40 people (10 each), 64×64 pixels — useful for demonstration.
• We use PCA with whitening to normalize component scales; whitening can help classifiers but may amplify noise.
• K-NN in Eigenface space is a simple baseline; more advanced classifiers (SVM, logistic regression) can be used.

4. Understanding results and evaluation

• Measure classification accuracy and per-class performance (precision/recall).
• Visual inspection of mean face and eigenfaces helps interpret what PCA captured.
• Reconstruction error (MSE between original and reconstructed image) indicates how well chosen K captures information.

Common observations:

• The first few eigenfaces capture lighting and coarse face structure.
• Higher-order eigenfaces capture finer details and noise.
• Too many components can overfit; too few lose discriminative detail.

5. Improvements and practical tips

• Align faces precisely (eye coordinates) before PCA to make eigenfaces meaningful.
• Use histogram equalization or illumination normalization to reduce lighting variation.
• Try different numbers of components K and plot accuracy vs K.
• Use cross-validation to select hyperparameters for classifier and PCA.
• Consider LDA (Fisherfaces) when you have labelled classes — LDA maximizes class separation.
• For large datasets, incremental PCA or randomized SVD helps with memory and speed.

6. Limitations

• Sensitive to alignment and lighting changes.
• Less robust than modern deep learning approaches (CNNs) for unconstrained face recognition.
• Assumes linear subspace; real face variation can be nonlinear.

7. Further reading and next steps

• Try implementing from scratch using NumPy’s SVD to understand the math.
• Compare Eigenfaces with Fisherfaces and deep learning methods (e.g., FaceNet, ArcFace).
• Explore using eigenfaces for face clustering, face synthesis, or as a visualization tool.