<aside> 📊

This lecture brings together the core ideas of linear algebra by introducing eigenvalues and eigenvectors — special directions along which a linear transformation acts as simple scaling. The lecture covers motivation (PCA, solving differential equations, computing A¹⁰⁰), the geometric idea, the characteristic equation, a full worked example, geometric examples (scaling and reflection), and diagonalisation to compute matrix powers efficiently.

</aside>

📋 At a Glance

• [ ] Understand why eigenvalues/eigenvectors matter (PCA, solving linear systems, matrix powers)
• [ ] Know the definition: Av = λv (eigenvector v is only scaled by A)
• [ ] Derive the characteristic equation det(A − λI) = 0
• [ ] Solve the characteristic polynomial for eigenvalues (roots of polynomial)
• [ ] Find eigenvectors by solving (A − λI)v = 0 (underdetermined system)
• [ ] Work through the full example: A = (1,1; 6,2) → λ₁=4, λ₂=−1, v₁=(1,3)ᵀ, v₂=(1,−2)ᵀ
• [ ] Identify eigenvectors geometrically (scaling map, reflection map)
• [ ] Diagonalise a matrix: A' = M⁻¹AM where M = [v₁, v₂]
• [ ] Compute A¹⁰⁰ = M · D¹⁰⁰ · M⁻¹ using eigendecomposition
• [ ] Know when diagonalisation works (need n linearly independent eigenvectors)

1. Motivation

Dimensionality Reduction and PCA

In machine learning, you often have high-dimensional data (many features) and want to classify data points. Too many features causes problems — algorithms struggle and results are hard to visualise.

Principal Component Analysis (PCA) projects data onto the directions of greatest variation, reducing dimensionality while preserving the most important structure. PCA is essentially an eigendecomposition — the principal components are eigenvectors of the data's covariance matrix.

Example: Iris dataset
- Original: ~10-20 features per flower
- After PCA: projected onto 2 principal components
- Result: 3 species become clearly separated clusters
→ Can then apply classifiers (e.g. SVMs) to the reduced data

Breaking Down Linear Systems