📐 Concept diagram

08-08 — Eigenvalues and Eigenvectors

Phase: 8 — Linear Algebra (Rigorous) Subject: 08-08 Prerequisites: 08-07 — Determinants (Deep) Next subject: 08-09 — Diagonalisation

Learning Objectives

By the end of this subject, you will be able to:

1. Define eigenvalues and eigenvectors, and interpret them geometrically as directions preserved (up to scaling) by a linear transformation
2. Compute eigenvalues by solving the characteristic equation det(A − λI) = 0, and find corresponding eigenvectors by solving (A − λI)x = 0
3. Distinguish between algebraic multiplicity (power in characteristic polynomial) and geometric multiplicity (dimension of eigenspace)
4. Determine whether a matrix is diagonalizable based on the relationship between algebraic and geometric multiplicities
5. Apply eigendecomposition to compute powers of matrices and solve systems of linear differential equations

Core Content

1. Definitions and Geometric Meaning

Let A be an n × n matrix. A scalar λ is an eigenvalue of A if there exists a nonzero vector x such that:

$Ax = λx
$

The vector x is called an eigenvector associated with λ.

CRITICAL — Foundational: Eigenvalues reveal the 'natural coordinate system' — directions that are merely scaled. $det(A − λI) = 0$ is THE equation. Powers PCA, PageRank, quantum mechanics, and all of spectral theory.

Geometric interpretation: An eigenvector is a direction that A merely scales (by factor λ), without rotating. The transformation A acts independently on each eigen-direction.

Eigenspace: For an eigenvalue λ, the set E_λ = {x : Ax = λx} = N(A − λI) is a subspace called the eigenspace for λ. Its dimension is the geometric multiplicity of λ.

Examples of geometric meaning: - Projection matrix P: eigenvalues 1 (vectors in the subspace, unchanged) and 0 (vectors orthogonal to it, annihilated) - Rotation by 90° in R^2: No real eigenvectors (all vectors rotate), but complex eigenvalues ±i - Shear [1 k; 0 1]: λ = 1 (algebraic multiplicity 2), only one eigenvector direction (1, 0)^T

2. Computing Eigenvalues — The Characteristic Equation

Rewrite Ax = λx as (A − λI)x = 0. For a nonzero solution, A − λI must be singular:

$det(A − λI) = 0
$

This is the characteristic equation. p(λ) = det(A − λI) is the characteristic polynomial, degree n.

Steps to find eigenvalues and eigenvectors: 1. Compute det(A − λI) to get the characteristic polynomial 2. Solve p(λ) = 0 to find eigenvalues 3. For each eigenvalue λ, solve (A − λI)x = 0 to find eigenvectors

Example:

$A = [3  1]
    [0  2]
$

Step 1: A − λI = [3−λ 1; 0 2−λ]. det = (3−λ)(2−λ) − 1·0 = (3−λ)(2−λ) = λ² − 5λ + 6.

Step 2: λ² − 5λ + 6 = (λ−2)(λ−3) = 0 → λ₁ = 2, λ₂ = 3.

Step 3 — For λ₁ = 2: A − 2I = [1 1; 0 0]. Solve x₁ + x₂ = 0 → x = t(−1, 1)^T. Eigenspace: span{(−1, 1)}.

For λ₂ = 3: A − 3I = [0 1; 0 −1]. Solve x₂ = 0, −x₂ = 0 → x₂ = 0, x₁ free. x = t(1, 0)^T. Eigenspace: span{(1, 0)}.

3. Algebraic and Geometric Multiplicity

Definition: The algebraic multiplicity (AM) of an eigenvalue λ is the number of times (λ − λ₀) appears as a factor in the characteristic polynomial.

Definition: The geometric multiplicity (GM) of λ is dim(E_λ) = dim(N(A − λI)) = n − rank(A − λI).

Key theorem: 1 ≤ GM ≤ AM.

If GM = AM for every eigenvalue, the matrix is diagonalizable. If GM < AM for some eigenvalue, the matrix is defective and NOT diagonalizable.

Common Pitfall: Algebraic multiplicity (from characteristic polynomial) counts 'how many times' an eigenvalue appears. Geometric multiplicity (dimension of eigenspace) counts 'how many independent eigenvectors.' They can differ! Diagonalizable ONLY when GM = AM for ALL eigenvalues.

Example (defective matrix):

$A = [2  1]
    [0  2]
$

Characteristic polynomial: (2−λ)² = 0 → λ = 2, AM = 2. A − 2I = [0 1; 0 0], rank = 1. GM = 2 − 1 = 1. Since GM = 1 < 2 = AM, A is defective (not diagonalizable). It's a Jordan block.

4. Properties of Eigenvalues

1. Trace = sum of eigenvalues: tr(A) = Σ λ_i (counting algebraic multiplicity)
2. Determinant = product of eigenvalues: det(A) = Π λ_i (Hence det(A) = 0 iff 0 is an eigenvalue)
3. Eigenvalues of A^k: If Ax = λx, then A^k x = λ^k x (So the eigenvalues of A^k are the k-th powers of eigenvalues of A)
4. Eigenvalues of A⁻¹: If A is invertible, eigenvalues are 1/λ
5. Eigenvalues of A + cI: If Ax = λx, then (A + cI)x = (λ + c)x
6. Eigenvalues of transpose: A^T has the same eigenvalues as A
7. Eigenvectors for distinct eigenvalues are linearly independent
8. Symmetric matrices: All eigenvalues are real; eigenvectors for distinct eigenvalues are orthogonal

Example (trace and determinant): A = [3 1; 0 2]: λ₁ = 2, λ₂ = 3. Σ λ = 5 = tr(A) = 3 + 2. ✓ Π λ = 6 = det(A) = 3·2 − 1·0 = 6. ✓

5. Diagonalization (Preview)

If A has n linearly independent eigenvectors (i.e., GM = AM for all eigenvalues), then:

$A = P D P⁻¹
$

where: - Columns of P are the eigenvectors - D is diagonal with eigenvalues on the diagonal

Then A^k = P D^k P⁻¹ (easy to compute: just raise each λ to power k).

Key Terms

• 08 08 Eigenvalues And Eigenvectors
• Algebraic and Geometric Multiplicity
• Computing Eigenvalues — The Characteristic Equation
• Correct: A) 2, 2, 2
• Correct: B) 2
• Correct: B) An eigenvector of A with eigenvalue λ
• Correct: B) GM < AM for at least one eigenvalue
• Correct: B) They are linearly independent
• Correct: C) 3
• Correct: C) 7
• Correct: C) det(A) = 0
• Definitions and Geometric Meaning

Worked Examples

Example 1: Full Eigendecomposition

Problem: Find all eigenvalues and eigenvectors of A = [2 1; 1 2].

Solution:

Step 1: A − λI = [2−λ 1; 1 2−λ]. det = (2−λ)² − 1 = λ² − 4λ + 3 = (λ−1)(λ−3).

Step 2: λ₁ = 1, λ₂ = 3.

Step 3 — λ = 1: A − I = [1 1; 1 1]. x₁ + x₂ = 0 → x = t(−1, 1). Eigenvector: (−1, 1).

λ = 3: A − 3I = [−1 1; 1 −1]. −x₁ + x₂ = 0 → x = t(1, 1). Eigenvector: (1, 1).

Check: A(−1,1) = (−2+1, −1+2) = (−1, 1) = 1·(−1,1). ✓ A(1,1) = (2+1, 1+2) = (3, 3) = 3·(1,1). ✓

Eigenvectors are orthogonal (since A is symmetric). AM = GM = 1 for both → diagonalizable.

$P = [-1  1]    D = [1  0]    P⁻¹ = [-1/2  1/2]
    [ 1  1]        [0  3]          [ 1/2  1/2]

A = PDP⁻¹ ✓
$

Example 2: Complex Eigenvalues

Problem: Find eigenvalues of A = [0 −1; 1 0] (90° rotation).

Solution: A − λI = [−λ −1; 1 −λ]. det = λ² + 1 = 0 → λ = ±i.

This matrix has no real eigenvectors (a pure rotation). It's not diagonalizable over R but IS diagonalizable over C:

$P = [1  1]    D = [i   0]
    [i -i]        [0  -i]
$

Example 3: Repeated Eigenvalue (Diagonalizable vs Not)

Problem 3a: A = [1 0; 0 1] (identity). Characteristic polynomial: (1−λ)². λ = 1, AM = 2. A − I = 0, rank = 0. GM = 2 − 0 = 2 = AM. Diagonalizable (already diagonal).

Problem 3b: A = [1 1; 0 1]. Characteristic polynomial: (1−λ)². λ = 1, AM = 2. A − I = [0 1; 0 0], rank = 1. GM = 2 − 1 = 1 < AM. NOT diagonalizable (defective).

Example 4: 3 × 3 Eigenvalue Problem

Problem: Find eigenvalues of A = [2 0 0; 0 3 1; 0 0 3].

Solution: This is block diagonal / triangular. Eigenvalues are the diagonal entries: λ = 2 (AM=1), λ = 3 (AM=2).

For λ = 2: A − 2I = [0 0 0; 0 1 1; 0 0 1]. Solve: y+z=0, z=0 → y=0. x free. Eigenvector: (1, 0, 0). GM=1.

For λ = 3: A − 3I = [−1 0 0; 0 0 1; 0 0 0]. Solve: −x=0, z=0. x=0, z=0, y free. Eigenvector: (0, 1, 0). GM=1 < AM=2. Matrix is defective.

Practice Problems

(Answers are below. Try each problem before checking.)

Problem 1: Find eigenvalues and eigenvectors of A = [4 −1; 2 1].

Problem 2: For A = [3 0 0; 0 2 0; 0 0 1], verify tr(A) = Σ λ and det(A) = Π λ.

Problem 3: Find eigenvalues of A = [1 2; 2 1] and show the eigenvectors are orthogonal.

Problem 4: Determine whether A = [2 1 0; 0 2 0; 0 0 3] is diagonalizable.

Problem 5: If Ax = λx and A is invertible, show that A⁻¹x = (1/λ)x.

Problem 6: Compute A¹⁰ for A = [1 1; 0 0] by finding eigenvalues and eigenvectors.

Problem 7: Find all eigenvalues of the 3 × 3 matrix with all entries equal to 1.

Summary

1. An eigenvector is a direction preserved by A (only scaled); λ tells us the scaling factor — this decomposes A's action into simpler 1D problems
2. Eigenvalues come from det(A−λI) = 0; eigenvectors from solving (A−λI)x = 0; the trace equals the sum of eigenvalues and the determinant equals their product
3. Algebraic multiplicity counts how many times λ appears as a root; geometric multiplicity is the dimension of the eigenspace N(A−λI); 1 ≤ GM ≤ AM always holds
4. A matrix is diagonalizable iff GM = AM for every eigenvalue (i.e., there are n independent eigenvectors); defective matrices (GM < AM for some λ) cannot be diagonalized
5. Eigendecomposition A = PDP⁻¹ enables simple computation of A^k, matrix exponentials, and solutions to systems of differential equations

Quiz

Q1: If Ax = λx and x ≠ 0, what is x called?

A) A singular vector of A B) An eigenvector of A with eigenvalue λ C) A vector in the null space of A D) A row of A

Q2: The characteristic polynomial of an n × n matrix has degree:

A) n − 1 B) n C) n + 1 D) It depends on the matrix entries

Q3: If λ = 0 is an eigenvalue of A, what must be true?

A) A is the zero matrix B) A is invertible C) det(A) = 0 D) All entries of A are zero

Q4: Algebraic multiplicity refers to:

A) The dimension of the eigenspace B) The multiplicity of λ as a root of the characteristic polynomial C) The number of eigenvectors for λ D) The rank of A − λI

Q5: A defective matrix is one where:

A) All eigenvalues are zero B) GM < AM for at least one eigenvalue C) The determinant is zero D) The matrix is complex

Pitfalls

1. Confusing algebraic and geometric multiplicity. AM is the exponent of (λ − λ₀) in the characteristic polynomial; GM is dim(N(A − λI)). They satisfy 1 ≤ GM ≤ AM, but they need not be equal. When GM < AM for any eigenvalue, the matrix is defective and NOT diagonalizable.

2. Assuming repeated eigenvalue means defective. A repeated eigenvalue does NOT guarantee defectiveness. The identity matrix I has λ = 1 with AM = n and GM = n — it is perfectly diagonalizable. Always compute GM by finding the rank of (A − λI) before concluding defectiveness.

3. Expecting all real matrices to have n real eigenvalues. Real matrices can have complex eigenvalues — they come in conjugate pairs. A 2×2 rotation matrix has eigenvalues ±i with no real eigenvectors. Diagonalizability over R requires the characteristic polynomial to split over R.

4. Forgetting the zero eigenvalue means singularity. If λ = 0 is an eigenvalue, then det(A) = 0 and A is singular. Conversely, if det(A) = 0, then λ = 0 is an eigenvalue. This is a quick singularity test from the trace/determinant relationship.

5. Assuming eigenvectors for the same eigenvalue are orthogonal. Eigenvectors for DISTINCT eigenvalues of a symmetric matrix are orthogonal, but eigenvectors within the SAME eigenspace are not automatically orthogonal — they just span the eigenspace. Apply Gram-Schmidt within each eigenspace if orthogonality is needed.

Next Steps

Move on to 08-09 — Diagonalisation to deepen your understanding of when matrices can be diagonalized, how similarity transformations work, computing powers of diagonalizable matrices, and applications to differential equations and dynamical systems.