📐 Concept diagram

08-02 — Linear Independence and Basis

Phase: 8 — Linear Algebra (Rigorous) Subject: 08-02 Prerequisites: 08-01 — Vector Spaces Next subject: 08-03 — Linear Transformations

Learning Objectives

By the end of this subject, you will be able to:

1. Distinguish between linearly independent and linearly dependent sets using the definition and the determinant test (for square systems)
2. Define a basis, prove a set is a basis using the spanning + independence criteria, and compute the dimension of a vector space
3. Express vectors in coordinates relative to a given basis and relate different coordinate representations via change-of-basis matrices
4. Apply basis concepts to concrete spaces: find bases for N(A), C(A), and polynomial spaces
5. Prove fundamental facts: every linearly independent set extends to a basis, every spanning set contains a basis, and all bases of a finite-dimensional space have the same size

Core Content

1. Linear Independence and Dependence

CRITICAL — Foundational: Linear independence is the most important concept in linear algebra. The test is simple — $c₁v₁ + ... + c_n v_n = 0 ⇒ all c_i = 0$ — but applying it correctly across different spaces (polynomials, functions, matrices) takes practice.

Definition: A set of vectors {v₁, v₂, ..., v_n} in a vector space V is linearly independent if the equation

$c₁v₁ + c₂v₂ + ... + c_n v_n = 0
$

has ONLY the trivial solution c₁ = c₂ = ... = c_n = 0.

The set is linearly dependent if there exist scalars, NOT ALL ZERO, such that the linear combination equals the zero vector.

Equivalent characterizations: - Dependent iff at least one vector in the set is a linear combination of the others - A set containing the zero vector is ALWAYS dependent (take c₁ = 1 for the zero vector, all others 0) - A set of one vector {v} is independent iff v ≠ 0

Geometric intuition (R^n): - 1 vector: independent if nonzero → spans a line - 2 vectors: independent if not collinear → span a plane - 3 vectors: independent if not coplanar → span R^3 - m > n vectors in R^n: ALWAYS dependent (more on this with dimension)

Testing independence (computational approach):

Form a matrix A whose columns are the vectors. Solve Ac = 0: - If the only solution is c = 0, the columns are independent - If nontrivial solutions exist, the columns are dependent (each free variable gives a dependence relation)

Example: Test {(1, 2, 3), (4, 5, 6), (7, 8, 9)} in R^3.

$[1  4  7] [c₁]   [0]
[2  5  8] [c₂] = [0]
[3  6  9] [c₃]   [0]
$

Row reduce: R₂−2R₁, R₃−3R₁:

$[1  4  7]           [1   4   7]
[0 -3 -6]  R₃/R₂   [0   1   2]
[0 -6 -12]          [0   0   0]
$

Free variable c₃ = t → c₂ = −2t, c₁ = −4c₂ − 7c₃ = 8t − 7t = t. So (t, −2t, t) is a nontrivial solution for any t ≠ 0. The set is dependent. Indeed, v₃ = v₁ + v₂.

Theorem: In R^n, a set of n vectors is independent iff det(A) ≠ 0, where A has them as columns.

2. Basis

Definition: A set B = {b₁, b₂, ..., b_n} is a basis for a vector space V if: 1. B spans V: span(B) = V (every v ∈ V is a linear combination of the bᵢ) 2. B is linearly independent

Key property of a basis (unique representation): Every v ∈ V can be expressed UNIQUELY as v = c₁b₁ + ... + c_n b_n. The scalars (c₁, ..., c_n) are the coordinates of v relative to B, written [v]_B.

Proof of uniqueness: If v = Σcᵢbᵢ = Σdᵢbᵢ, then Σ(cᵢ−dᵢ)bᵢ = 0. By independence, all cᵢ−dᵢ = 0, so cᵢ = dᵢ. ∎

Standard bases: - R^n: e₁ = (1,0,...,0), e₂ = (0,1,0,...,0), ..., e_n = (0,...,0,1) — the standard basis - P_n: {1, x, x², ..., x^n} — the monomial basis - M_{2×2}: {[1 0; 0 0], [0 1; 0 0], [0 0; 1 0], [0 0; 0 1]}

Non-standard basis example for R^2: B = {(1, 1), (1, −1)}. - Independence: c₁(1,1) + c₂(1,−1) = (0,0) → (c₁+c₂, c₁−c₂) = (0,0) → c₁ = c₂ = 0 ✓ - Spanning: For any (a, b), solve c₁(1,1) + c₂(1,−1) = (a,b) → c₁ = (a+b)/2, c₂ = (a−b)/2 ✓

So [v]_B = ((a+b)/2, (a−b)/2) for v = (a, b).

3. Dimension

Theorem: If a vector space V has a finite basis, then ALL bases of V have the same number of vectors.

Common Pitfall: Students confuse 'a basis' (any set satisfying spanning + independence) with 'the standard basis' (the familiar one). There are infinitely many bases — the standard basis is just one choice.

Definition: The dimension of V, denoted dim(V), is the number of vectors in any basis. V is finite-dimensional if it has a finite basis.

Key facts: - dim(R^n) = n - dim(P_n) = n + 1 - dim(M_{m×n}) = mn - dim({0}) = 0 (the empty set is a basis)

Theorems about dimension: 1. If V is finite-dimensional with dim(V) = n, then any independent set has ≤ n vectors, and any spanning set has ≥ n vectors 2. Any independent set of n vectors in V is automatically a basis 3. Any spanning set of n vectors in V is automatically a basis 4. Every independent set can be extended to a basis 5. Every spanning set can be reduced to a basis

4. Bases for Fundamental Subspaces

Basis for N(A): From RREF, assign each free variable to be 1 in turn (and all others 0), solve for pivot variables. These vectors form a basis. Size = n − rank(A).

Basis for C(A): The pivot columns of the ORIGINAL matrix A (not the RREF) form a basis. Size = rank(A).

Example: A = [1 2 1; 2 4 3]

RREF: [1  2  0]
      [0  0  1]

Cols 1 and 3 are pivot columns → basis of C(A): {(1,2)^T, (1,3)^T}. Free variable x₂ = t → x₁ = −2t, x₃ = 0 → basis of N(A): {(−2, 1, 0)}.

5. Change of Basis

Given two bases B = {b₁, ..., b_n} and C = {c₁, ..., c_n} of V, the change-of-basis matrix P_{C←B} maps coordinates relative to B to coordinates relative to C:

$[v]_C = P_{C←B} [v]_B
$

Construction: The i-th column of P_{C←B} is [b_i]_C — the coordinates of the i-th basis vector of B expressed in C.

Properties: - P_{B←B} = I (the identity) - P_{B←C} = (P_{C←B})⁻¹ - P_{D←B} = P_{D←C} P_{C←B} (chain rule)

Example (R^2): B = {(1, 1), (1, −1)}, C = standard basis {e₁, e₂}.

P_{C←B}: column 1 = [(1,1)]_C = (1,1), column 2 = [(1,−1)]_C = (1,−1).

$P_{C←B} = [1   1]
           [1  -1]
$

If [v]B = (3, 1), then [v]_C = P{C←B} [v]_B = [1 1; 1 −1][3; 1] = (4, 2). Indeed, 3(1,1) + 1(1,−1) = (4,2). ✓

Common misconception: The change-of-basis matrix does NOT change the vector; it changes the coordinate REPRESENTATION of the same vector. The vector v = (4, 2) is the same point in R^2 regardless of basis.

Key Terms

• 08 02 Linear Independence And Basis
• Bases for Fundamental Subspaces
• Basis
• Change of Basis
• Correct: B) 4
• Correct: B) A basis
• Correct: B) An n × n invertible matrix
• Correct: B) [b_i]_C
• Correct: B) n + 1
• Correct: B) {(−2, 1)}
• Correct: D) 6
• Correct: D) All of the above

Worked Examples

Example 1: Testing Independence in P₂

Problem: Are p₁(x) = 1 + x, p₂(x) = 1 − x, p₃(x) = x² linearly independent in P₂?

Solution: Check c₁(1+x) + c₂(1−x) + c₃(x²) = 0 (the zero polynomial).

Expand: (c₁ + c₂) + (c₁ − c₂)x + c₃ x² = 0 + 0x + 0x².

Equate coefficients:

$c₁ + c₂ = 0     (constant term)
c₁ − c₂ = 0     (x term)
c₃ = 0          (x² term)
$

From equations 1 and 2: c₁ + c₂ = 0 and c₁ − c₂ = 0 → add: 2c₁ = 0 → c₁ = 0 → c₂ = 0. So c₁ = c₂ = c₃ = 0. The set is linearly independent.

Since dim(P₂) = 3 and we have 3 independent vectors, {1+x, 1−x, x²} is a basis for P₂!

Example 2: Extending to a Basis

Problem: Extend {(1, 2, 1)} to a basis of R^3.

Solution: dim(R^3) = 3, so we need 2 more independent vectors. We add standard basis vectors and discard any that become dependent.

Start with B = {(1, 2, 1)}.

Try e₁ = (1, 0, 0): Are (1,2,1) and (1,0,0) independent? Yes — they're not scalar multiples. Add e₁: B = {(1,2,1), (1,0,0)}.

Try e₂ = (0, 1, 0): Check if (0,1,0) is in span{(1,2,1), (1,0,0)}. Solve:

$a(1,2,1) + b(1,0,0) = (0,1,0)
→ (a+b, 2a, a) = (0,1,0)
→ a = 0, b = 0, but 2a = 0 ≠ 1. No solution. Independent!
$

So B = {(1,2,1), (1,0,0), (0,1,0)} is a basis of R^3.

The determinant check: |1 1 0; 2 0 1; 1 0 0| = 1(0−0) − 1(0−1) + 0 = 1 ≠ 0. ✓

Example 3: Coordinates and Change of Basis

Problem: In R^2, B = {(2, 1), (1, 1)} and C = {(1, 2), (2, 1)}. Find the change-of-basis matrix P_{C←B}.

Solution:

Step 1: Find [b₁]_C = coordinates of (2, 1) in C. Solve α(1,2) + β(2,1) = (2,1):

$α + 2β = 2
2α + β = 1
$

From eq2: β = 1 − 2α. Substitute into eq1: α + 2(1−2α) = 2 → α + 2 − 4α = 2 → −3α = 0 → α = 0, β = 1. So [b₁]_C = (0, 1).

Step 2: Find [b₂]_C = coordinates of (1, 1) in C. Solve α(1,2) + β(2,1) = (1,1):

$α + 2β = 1
2α + β = 1
$

From eq1: α = 1 − 2β. Substitute: 2(1−2β) + β = 1 → 2 − 4β + β = 1 → −3β = −1 → β = 1/3, α = 1 − 2/3 = 1/3. So [b₂]_C = (1/3, 1/3).

Step 3: Assemble:

$P_{C←B} = [[b₁]_C  [b₂]_C] = [0   1/3]
                              [1   1/3]
$

Verification: For v = (2,1) = 1·b₁ + 0·b₂: [v]B = (1, 0). Then [v]_C = P{C←B} (1,0)^T = (0, 1). Indeed, (2,1) = 0·(1,2) + 1·(2,1) = (2,1). ✓

Example 4: Finding a Basis for N(A) from RREF

Problem: Find a basis for N(A) where A = [1 3 0 2; 0 0 1 4; 0 0 0 0].

Solution: A is already in RREF. Pivot columns: 1 and 3. Free variables: x₂, x₄.

$x₁ + 3x₂ + 0x₃ + 2x₄ = 0  →  x₁ = −3x₂ − 2x₄
0x₁ + 0x₂ + x₃ + 4x₄ = 0  →  x₃ = −4x₄
$

Set (x₂, x₄) = (1, 0): x = (−3, 1, 0, 0) → basis vector v₁. Set (x₂, x₄) = (0, 1): x = (−2, 0, −4, 1) → basis vector v₂.

Basis for N(A): {(−3, 1, 0, 0), (−2, 0, −4, 1)}. dim(N(A)) = 2 = 4 − rank(A) = 4 − 2. ✓

Quiz

Q1: A set of vectors {v₁, ..., v_k} is linearly independent if:

A) None of the vectors is the zero vector B) The only solution to c₁v₁ + ... + c_k v_k = 0 is c₁ = ... = c_k = 0 C) The vectors are all orthogonal to each other D) The vectors span the space

Correct: B)

• If you chose B: Linear independence means the only linear combination giving zero is the trivial one. Correct!
• If you chose A: Nonzero vectors can still be dependent (e.g., {v, 2v}).
• If you chose C: Orthogonal vectors are independent, but independence doesn't require orthogonality.
• If you chose D: A spanning set can be linearly dependent.

Q2: A basis of a vector space V is a set of vectors that is:

A) Orthogonal B) Both linearly independent and spanning C) Linearly dependent but spanning D) Any collection of dim(V) vectors

Correct: B)

• If you chose B: A basis must span V and be linearly independent — this gives a unique representation for every vector. Correct!
• If you chose A: Orthogonality is extra structure (inner product), not required for a basis.
• If you chose C: A dependent set can't be a basis — representations wouldn't be unique.
• If you chose D: dim(V) vectors could be dependent — must also be independent.

Q3: The dimension of a vector space is:

A) The number of vectors in any set B) The number of vectors in any spanning set C) The number of vectors in any basis D) Always infinite

Correct: C)

• If you chose C: All bases of a vector space have the same number of vectors — that number is the dimension. Correct!
• If you chose A: A random set of vectors doesn't determine dimension.
• If you chose B: A spanning set could be larger than a basis (redundant).
• If you chose D: Only some vector spaces are infinite-dimensional.

Q4: If {v₁, v₂, v₃} is a basis of V, then any vector w ∈ V can be expressed as:

A) w = v₁ + v₂ + v₃ only B) w = c₁v₁ + c₂v₂ + c₃v₃ for unique scalars c₁, c₂, c₃ C) w = c₁v₁ only D) w cannot be expressed as a combination

Correct: B)

• If you chose B: Every vector has a unique representation as a linear combination of basis vectors. Correct!
• If you chose A: That's just one specific vector, not a general one.
• If you chose C: That only spans a 1D subspace.
• If you chose D: The definition of a basis guarantees every vector can be expressed.

Q5: The standard basis of R³ is:

A) {(1, 0, 0), (0, 1, 0), (0, 0, 1)} B) {(1, 1, 1), (0, 1, 1), (0, 0, 1)} C) {(1, 0, 0), (0, 2, 0), (0, 0, 3)} D) Any three orthogonal vectors

Correct: A)

• If you chose A: e₁ = (1,0,0), e₂ = (0,1,0), e₃ = (0,0,1) is the standard basis of R³. Correct!
• If you chose B: These are linearly independent and form a basis, but not the standard one.
• If you chose C: Also a basis but not standard.
• If you chose D: Not any three orthogonal vectors — they must span R³.

Q6: If a set of 5 vectors spans a 3-dimensional space, what must be true?

A) The vectors form a basis B) The vectors are linearly independent C) At least 2 vectors are redundant (the set is linearly dependent) D) The dimension must be at least 5

Correct: C)

• If you chose C: In a 3D space, any set with more than 3 vectors must be linearly dependent. A spanning set of 5 vectors necessarily contains redundancy. Correct!
• If you chose A: A basis of a 3D space has exactly 3 vectors.
• If you chose B: 5 vectors in a 3D space cannot all be independent.
• If you chose D: The dimension is 3, regardless of the spanning set size.

Practice Problems

(Answers are below. Try each problem before checking.)

Problem 1: Determine whether {(1, 2, 3), (4, 5, 6), (7, 8, 9)} is independent in R^3. If dependent, find the dependence relation.

Problem 2: Show that B = {1+x, 1−x, x²} is a basis for P₂ by proving it is both independent and spanning.

Problem 3: In R^3, let fan = {(1, 2, 1), (2, 1, −1)}. Is this a basis for R^3? If not, why not? Extend it to a basis.

Problem 4: Find [v]_B for v = (5, 7) and B = {(1, 1), (2, 3)}.

Problem 5: Given bases B = {e₁, e₂} (standard) and C = {(1, 1), (1, −1)} for R^2, compute P_{C←B} and P_{B←C}, then verify they are inverses.

Problem 6: Determine whether {sin x, cos x, sin(2x)} is linearly independent in C[0, 2π] (continuous functions on [0, 2π]).

Problem 7: Find a basis for the column space of A = [1 2 3 4; 2 4 6 8; 1 1 1 1].

$P_{C←B} = [1/2   1/2]
           [1/2  -1/2]
$

$P_{B←C} = [1   1]
           [1  -1]
$

$[1 2 3 4]           [1 2 3  4]           [1 2 0 -2]
[2 4 6 8]  R₂-2R₁  [0 0 0  0]  R₁-3R₃  [0 0 0  0]
[1 1 1 1]  R₃-R₁   [0 -1 -2 -3]           [0 1 2  3]
$

Summary

1. A set is linearly independent if the only linear combination yielding zero is trivial; geometrically, no vector is in the span of the others — at most n vectors in R^n can be independent
2. A basis is a minimal spanning set (equivalently, a maximal independent set); every v ∈ V has unique coordinates relative to a basis, enabling us to identify any n-dimensional vector space with R^n
3. ALL bases of a finite-dimensional vector space have the same size, which defines dim(V); this ties together all the structural properties of the space
4. RREF gives bases for N(A) and C(A) algorithmically: N(A) from free variables, C(A) from pivot columns of the original matrix
5. Change-of-basis matrices convert coordinate vectors between bases; P_{C←B} has columns [bᵢ]C, and P{B←C} = P_{C←B}⁻¹

Pitfalls

1. Confusing linear independence with "no vector is a scalar multiple of another." This test only works for sets of two vectors. For three or more vectors, independence means NO nontrivial linear combination yields zero — pairs may be independent while the whole set is dependent (e.g., (1,0,0), (0,1,0), (1,1,0)).

2. Thinking any spanning set of n vectors in an n-dimensional space is automatically a basis. This is true ONLY if the vectors are also independent. A set of n vectors that spans an n-dimensional space IS automatically a basis (and therefore independent), but you should verify spanning — not just count vectors.

3. Using pivot columns from the RREF instead of the original matrix for C(A). The column space basis comes from the pivot columns of the ORIGINAL matrix A, not the RREF. Row operations preserve the null space but change the column space. Using RREF columns for C(A) is a classic error.

4. Forgetting that change-of-basis matrices act on coordinates, not vectors. P_{C←B} converts [v]B to [v]_C — it operates on coordinate vectors relative to B, not on the vector v itself. Multiplying P{C←B} by v (instead of [v]_B) gives a meaningless result.

5. Assuming a set of more than n vectors in R^n must be a spanning set. More than n vectors in R^n are always linearly dependent, but they may or may not span R^n. For example, three vectors all lying in the same plane in R^3 are dependent but do NOT span R^3.

Next Steps

Move on to 08-03 — Linear Transformations to learn the formal definition of linear transformations, their properties (kernel, range, injectivity, surjectivity), matrix representations, and how they compose and invert — the core machinery connecting vector spaces.