📐 Concept diagram

06-07 — Optimization (Multivariable)

Phase: 6 — Calculus III: Multivariable Calculus Subject: 06-07 Prerequisites: 06-03 — Partial Derivatives, 06-06 — Directional Derivatives and Gradient, single-variable optimization Next subject: 06-08 — Lagrange Multipliers

Learning Objectives

By the end of this subject, you will be able to:

1. Find critical points of f(x, y) by solving ∇f = 0
2. Classify critical points as local maxima, local minima, or saddle points using the second derivative test (Hessian determinant)
3. Find absolute (global) extrema of a continuous function on a closed, bounded region
4. Apply multivariable optimization to real-world problems (distance minimization, volume maximization, least-squares)
5. Understand when the second derivative test is inconclusive and what to do next

Core Content

1. Local Extrema and Critical Points

⚠️ CRITICAL FOUNDATION: The second derivative test D = f_xx·f_yy - (f_xy)² classifies critical points. D > 0 with f_xx > 0 means local minimum; D > 0 with f_xx < 0 means local maximum; D < 0 means saddle point; D = 0 is inconclusive.

Definition: f(x, y) has a local maximum at (a, b) if f(a, b) ≥ f(x, y) for all (x, y) in some disk around (a, b). Similarly for a local minimum with ≤.

Fermat's Theorem (multivariable): If f has a local extremum at (a, b) and the partial derivatives exist there, then:

$f_x(a, b) = 0    and    f_y(a, b) = 0
$

Equivalently: ∇f(a, b) = ⟨0, 0⟩.

A point where ∇f = ⟨0, 0⟩ (or where a partial derivative doesn't exist) is called a critical point.

Important: ∇f = 0 is NECESSARY for a local extremum at an interior point (if partials exist), but it is NOT SUFFICIENT — a point with ∇f = 0 could be a maximum, a minimum, or a saddle point.

Example 1: Find the critical points of f(x, y) = x² + y² − 2x − 6y + 14.

$f_x = 2x − 2 = 0  →  x = 1
f_y = 2y − 6 = 0  →  y = 3

Critical point: (1, 3)
$

Complete the square: f(x, y) = (x − 1)² + (y − 3)² + 4. Since squares are always ≥ 0, f(1, 3) = 4 is the global minimum.

Example 2: Find the critical points of f(x, y) = x² − y².

$f_x = 2x = 0  →  x = 0
f_y = −2y = 0  →  y = 0

Critical point: (0, 0)
$

But f(0, 0) = 0. Along x-axis: f(x, 0) = x² > 0 for x ≠ 0. Along y-axis: f(0, y) = −y² < 0 for y ≠ 0. So (0, 0) is neither max nor min — it's a saddle point.

2. The Second Derivative Test (Hessian Determinant)

To classify critical points, we use second-order partial derivatives. Define the Hessian determinant (or discriminant) D:

$D(x, y) = f_xx · f_yy − (f_xy)²
        | f_xx  f_xy |
  = det |              |
        | f_yx  f_yy |
$

At a critical point (a, b) where ∇f = ⟨0, 0⟩:

Why this works (intuition): D comes from the quadratic terms in the Taylor expansion:

$f(x, y) ≈ f(a, b) + ½[f_xx(x−a)² + 2f_xy(x−a)(y−b) + f_yy(y−b)²]
$

The sign of this quadratic form determines the local shape. D > 0 means the form is definite (all positive → min; all negative → max). D < 0 means indefinite (saddle).

3. Applying the Second Derivative Test

Example 3: Classify the critical points of f(x, y) = x⁴ + y⁴ − 4xy + 1.

Step 1 — Critical points:

$f_x = 4x³ − 4y = 0  →  y = x³
f_y = 4y³ − 4x = 0  →  x = y³
$

Substitute y = x³ into x = y³: x = (x³)³ = x⁹ → x⁹ − x = 0 → x(x⁸ − 1) = 0. So x = 0 or x = ±1. - x = 0 → y = 0: (0, 0) - x = 1 → y = 1: (1, 1) - x = −1 → y = −1: (−1, −1)

Three critical points.

Step 2 — Second partials:

$f_xx = 12x²
f_yy = 12y²
f_xy = −4

D(x, y) = (12x²)(12y²) − (−4)² = 144x²y² − 16
$

At (0, 0): f_xx = 0, D = −16 < 0 → saddle point.

At (1, 1): f_xx = 12 > 0, D = 144 − 16 = 128 > 0 → local minimum, f(1, 1) = 1 + 1 − 4 + 1 = −1.

At (−1, −1): f_xx = 12 > 0, D = 128 > 0 → local minimum, f(−1, −1) = −1.

Example 4: f(x, y) = x³ − 3xy + y³.

$f_x = 3x² − 3y = 0  →  y = x²
f_y = 3y² − 3x = 0  →  x = y²

y = x² and x = y² → x = (x²)² = x⁴ → x(x³ − 1) = 0 → x = 0 or x = 1.
x = 0 → y = 0: (0, 0)
x = 1 → y = 1: (1, 1)
$

Second partials: f_xx = 6x, f_yy = 6y, f_xy = −3. D = 36xy − 9.

At (0, 0): D = −9 < 0 → saddle point. f_xx = 0 (expected at a saddle).

At (1, 1): f_xx = 6 > 0, D = 36 − 9 = 27 > 0 → local minimum. f(1, 1) = 1 − 3 + 1 = −1.

4. Absolute Extrema on Closed Bounded Regions

Extreme Value Theorem: A continuous function on a closed, bounded set in R² attains both an absolute maximum and absolute minimum.

Procedure (closed region method):

1. Find critical points of f in the interior of the region (∇f = 0)
2. Evaluate f at each interior critical point
3. On each boundary segment, parameterize and use single-variable optimization (or Lagrange multipliers, next subject)
4. Evaluate f at all vertices (corners)
5. The largest value among all candidates is the absolute maximum; the smallest is the absolute minimum

Example 5: Find the absolute extrema of f(x, y) = x² + y² − 2x − 4y on the triangle with vertices (0, 0), (3, 0), (0, 3).

Step 1 — Interior critical points:

$f_x = 2x − 2 = 0 → x = 1
f_y = 2y − 4 = 0 → y = 2
$

(1, 2) is inside the triangle? Check: the triangle is bounded by x ≥ 0, y ≥ 0, x + y ≤ 3. At (1, 2): 1 + 2 = 3 ≤ 3 (on the boundary!) and 1 ≥ 0, 2 ≥ 0. So (1, 2) is a boundary point, not an interior point. There are no interior critical points.

Step 2 — Boundaries (three edges):

Edge 1: y = 0, 0 ≤ x ≤ 3. g(x) = f(x, 0) = x² − 2x. g′(x) = 2x − 2 = 0 → x = 1. Point (1, 0). g(0) = 0, g(1) = −1, g(3) = 3.

Edge 2: x = 0, 0 ≤ y ≤ 3. h(y) = f(0, y) = y² − 4y. h′(y) = 2y − 4 = 0 → y = 2. Point (0, 2). h(0) = 0, h(2) = −4, h(3) = −3.

Edge 3: x + y = 3, y = 3 − x, 0 ≤ x ≤ 3. k(x) = f(x, 3−x) = x² + (3−x)² − 2x − 4(3−x) = x² + 9 − 6x + x² − 2x − 12 + 4x = 2x² − 4x − 3. k′(x) = 4x − 4 = 0 → x = 1, y = 2. Point (1, 2). k(0) = −3 (point (0,3)), k(1) = 2 − 4 − 3 = −5, k(3) = 18 − 12 − 3 = 3 (point (3,0)).

Step 3 — All candidates: (0,0): 0; (1,0): −1; (3,0): 3; (0,2): −4; (0,3): −3; (1,2): −5.

Absolute minimum: −5 at (1, 2). Absolute maximum: 3 at (3, 0).

5. Applications

Application 1 — Least squares (simple linear regression): Given data points (x₁, y₁), ..., (x_n, y_n), find the line y = mx + b that minimizes the sum of squared errors:

$S(m, b) = Σ(y_i − mx_i − b)²
$

Find ∂S/∂m = 0 and ∂S/∂b = 0 to get the normal equations. This is the theoretical foundation of linear regression.

Application 2 — Maximizing volume: Find the dimensions of a rectangular box with no top that maximizes volume, given a fixed surface area constraint. (Often solved with Lagrange multipliers — see 06-08.)

Application 3 — Minimizing distance: Find the point on the plane x + 2y + 3z = 6 closest to the origin.

Minimize d² = x² + y² + z² subject to x + 2y + 3z = 6.

Key Terms

• Hessian determinant
• Local maximum
• Local minimum
• Saddle point

Worked Examples

Worked Example 1: Classify All Critical Points

Problem: Find and classify all critical points of f(x, y) = 3x²y + y³ − 3x² − 3y² + 2.

Solution:

$f_x = 6xy − 6x = 6x(y − 1) = 0  →  x = 0 or y = 1
f_y = 3x² + 3y² − 6y = 3(x² + y² − 2y) = 0
$

Case 1: x = 0. Then f_y = 3(0 + y² − 2y) = 0 → 3y(y − 2) = 0 → y = 0 or y = 2. Points: (0, 0), (0, 2).

Case 2: y = 1. Then f_y = 3(x² + 1 − 2) = 0 → 3(x² − 1) = 0 → x = ±1. Points: (1, 1), (−1, 1).

Four critical points: (0,0), (0,2), (1,1), (−1,1).

Second partials:

$f_xx = 6y − 6
f_yy = 6y − 6
f_xy = 6x

D = (6y − 6)² − 36x² = 36[(y − 1)² − x²]
$

At (0, 0): f_xx = −6 < 0, D = 36(1 − 0) = 36 > 0 → local max, f = 2. At (0, 2): f_xx = 6 > 0, D = 36(1 − 0) = 36 > 0 → local min, f = 0 + 8 − 0 − 12 + 2 = −2. At (1, 1): D = 36(0 − 1) = −36 < 0 → saddle point, f = 3 + 1 − 3 − 3 + 2 = 0. At (−1, 1): D = 36(0 − 1) = −36 < 0 → saddle point, f = 3 + 1 − 3 − 3 + 2 = 0.

Worked Example 2: Absolute Extrema on a Region

Problem: Find the absolute extrema of f(x, y) = x² + 2y² on the disk x² + y² ≤ 1.

Solution: Interior: f_x = 2x = 0, f_y = 4y = 0 → (0, 0). f(0, 0) = 0.

Boundary x² + y² = 1: parameterize x = cos t, y = sin t. g(t) = cos²t + 2 sin²t = cos²t + 2(1 − cos²t) = 2 − cos²t.

Maximum of g(t): when cos²t = 0 → sin²t = 1 → g = 2. At (0, ±1), f = 2. Minimum of g(t): when cos²t = 1 → sin²t = 0 → g = 1. At (±1, 0), f = 1.

Candidates: (0, 0): 0; (±1, 0): 1; (0, ±1): 2. Absolute minimum: 0 at (0, 0). Absolute maximum: 2 at (0, 1) and (0, −1).

Worked Example 3: When D = 0 (Inconclusive)

Problem: Test f(x, y) = x⁴ + y⁴ for extrema at (0, 0).

Solution: f_x = 4x³ = 0, f_y = 4y³ = 0 → (0, 0) is the only critical point. f_xx = 12x², f_yy = 12y², f_xy = 0. At (0,0): f_xx = 0, f_yy = 0, D = 0 − 0 = 0. Inconclusive!

But we can see directly: f(x, y) = x⁴ + y⁴ ≥ 0, and f(0, 0) = 0. Since f > 0 everywhere else, (0, 0) is a global minimum. The test just couldn't confirm it because the Hessian matrix is the zero matrix — the function is "too flat" at the origin for second-order information to suffice.

Quiz

Q1: For f(x, y) = x² + xy + y², what condition must hold at a critical point?

A) f_xx + f_yy = 0 B) f_x = 0 and f_y = 0 C) D = f_xx·f_yy − (f_xy)² = 0 D) f(x, y) = 0

Correct: B)

• If you chose B: A critical point is where both partial derivatives vanish simultaneously — ∇f = ⟨0,0⟩. Correct!
• If you chose A: That's not the condition for a critical point; it relates to the Laplacian.
• If you chose C: D = 0 means the second derivative test is inconclusive, not that it's a critical point.
• If you chose D: That's the equation f = 0, which is unrelated to finding critical points of f.

Q2: The second derivative test uses D = f_xx·f_yy − (f_xy)². If at a critical point D > 0 and f_xx > 0, the point is a:

A) Saddle point B) Local maximum C) Local minimum D) Point where the test is inconclusive

Correct: C)

• If you chose C: D > 0 means the Hessian is definite; f_xx > 0 means it's positive definite → local minimum. Correct!
• If you chose A: A saddle point occurs when D < 0.
• If you chose B: Local maximum needs D > 0 and f_xx < 0.
• If you chose D: The test is inconclusive when D = 0.

Q3: For f(x, y) = x² − y², the critical point (0, 0) is classified as:

A) Local minimum B) Local maximum C) Saddle point D) The test is inconclusive

Correct: C)

• If you chose C: f_xx = 2, f_yy = −2, f_xy = 0 → D = (2)(−2) − 0 = −4 < 0 → saddle point. Along x-axis f > 0, along y-axis f < 0. Correct!
• If you chose A: f increases along the x-axis but decreases along the y-axis — not a minimum.
• If you chose B: Similarly, not a maximum because f can be positive.
• If you chose D: The test gives D = −4 ≠ 0, so it is conclusive.

Q4: What is the first step in finding absolute extrema of f(x, y) on a closed bounded region?

A) Evaluate f at all boundary vertices B) Compute the Hessian determinant at (0, 0) C) Find critical points in the interior by solving ∇f = 0 D) Parameterize the boundary and use single-variable calculus

Correct: C)

• If you chose C: The closed region method starts with interior critical points (∇f = 0), then examines boundaries and vertices. Correct!
• If you chose A: Vertices are checked, but only after interior points and boundary edges.
• If you chose B: The Hessian is used for classification, not for finding critical points.
• If you chose D: Boundary parameterization is step 3; interior critical points come first.

Q5: If the second derivative test gives D = 0 at a critical point, what should you do?

A) Conclude it's a saddle point B) Conclude it's a local minimum C) Conclude the function is constant D) Use other methods (graph, algebraic manipulation, or paths) to determine the behavior

Correct: D)

• If you chose D: D = 0 means the Hessian doesn't provide enough information. You must examine the function directly, e.g., completing the square or testing along curves. Correct!
• If you chose A: D < 0 indicates a saddle point, not D = 0.
• If you chose B: A local minimum requires D > 0 and f_xx > 0.
• If you chose C: D = 0 doesn't imply the function is constant (e.g., f(x,y) = x⁴ + y⁴ has D = 0 at (0,0) but isn't constant).

Q6: In least-squares linear regression, the sum of squared errors S(m, b) = Σ(y_i − mx_i − b)² is minimized by solving:

A) S(m, b) = 0 B) ∂S/∂m = 0 and ∂S/∂b = 0 simultaneously C) D(m, b) = S_mm·S_bb − (S_mb)² = 0 D) m² + b² = 1

Correct: B)

• If you chose B: Setting both partial derivatives to zero gives the normal equations — the same critical point method used throughout multivariable optimization. Correct!
• If you chose A: S(m,b) = 0 would mean zero error, which is usually impossible with noisy data.
• If you chose C: That's the Hessian test for classification, not for finding the minimum.
• If you chose D: That's a constraint, not a minimization condition.

Practice Problems

(Answers are below. Try each problem before checking.)

Problem 1: Find and classify the critical points of f(x, y) = x² + xy + y² − 3x.

Problem 2: Find and classify the critical points of f(x, y) = xy(1 − x − y).

Problem 3: Find the absolute maximum and minimum of f(x, y) = x² + y² − 2x on the closed triangular region with vertices (0, 0), (2, 0), (0, 2).

Problem 4: Find and classify the critical points of f(x, y) = x³ + y³ − 3x² − 3y² − 9x.

Problem 5: Find the point on the plane x + y + z = 1 that is closest to the origin by minimizing d² = x² + y² + z² subject to z = 1 − x − y.

Problem 6: For f(x, y) = e^(−x²−y²), find all critical points and classify them.

Problem 7: Find the dimensions of the rectangular box (with a bottom but no top) of maximum volume that can be made from 12 m² of material.

Summary

1. Critical points occur where ∇f = ⟨0, 0⟩ (both partial derivatives zero) or where a partial derivative doesn't exist — these are the only candidates for interior local extrema
2. The second derivative test uses the Hessian determinant D = f_xx f_yy − (f_xy)²: D > 0 with f_xx > 0 gives a local minimum; D > 0 with f_xx < 0 gives a local maximum; D < 0 gives a saddle point; D = 0 is inconclusive
3. For absolute extrema on a closed bounded region, check all interior critical points, optimize on each boundary segment, and evaluate at all corner points — the extreme values are among these candidates
4. Real-world optimization problems (distance, volume, cost, least-squares) reduce to finding critical points of an appropriate multivariable function
5. When D = 0, use direct comparison, inequalities, or higher-order Taylor expansion to determine the nature of the critical point

Pitfalls

• Assuming every critical point is an extremum. ∇f = 0 is necessary but NOT sufficient for a local extremum. A critical point can be a local maximum, local minimum, or saddle point. Always apply the second derivative test rather than assuming ∇f = 0 implies a max or min.
• Miscomputing the Hessian determinant. D = f_xx·f_yy − (f_xy)². A common error is writing D = f_xx·f_yy − f_xy (forgetting to square the mixed partial) or computing f_xy incorrectly. Remember that f_xy = f_yx (Clairaut), but compute both carefully.
• Forgetting to check boundaries for absolute extrema. The closed region method requires checking interior critical points AND boundary optima AND vertices. Students often find the interior critical point, classify it as a local min, and declare it the absolute min — but the absolute min could be on the boundary.
• Misinterpreting D = 0 as a saddle point. D = 0 means the second derivative test is INCONCLUSIVE, not that the point is a saddle. A saddle requires D < 0. When D = 0, you must use other methods: direct comparison, sections through the point, or higher-order Taylor terms.
• Applying the second derivative test to non-critical points. The test only classifies points where ∇f = ⟨0, 0⟩. Computing D at an arbitrary point tells you nothing about extrema — the test presupposes a critical point.

Next Steps

Move on to 06-08 — Lagrange Multipliers to learn how to optimize a function subject to one or more constraints, the geometry behind the method (parallel gradients), and applications to constrained optimization problems.