Lagrange Multipliers

Concept Explanation

Geometric Intuition: Tangency of Level Curves

Consider the level curves of the objective function $f(x, y) = c$ overlaid onto the fixed constraint curve $g(x, y) = k$.

• As you travel along the path $g(x, y) = k$, you cut across different level curves of $f$, increasing or decreasing the value of $c$.
• If the constraint path crosses a level curve of $f$ transversely (at a non-zero angle), you can continue moving along the path in one direction to increase $f$, or reverse direction to decrease $f$.
• Therefore, an extreme value along the constraint path can occur only when the constraint curve is tangent to a level curve of $f$.

                   ^ y
                   |        Level curve f(x, y) = c_3 (higher)
                   |       .-----------------.
                   |      /                   \
                   |     /    Level curve      \
                   |    /     f(x, y) = c_2     \
                   |   |     .------------.      |
                   |   |    /      P       \     |
                   |    \  |       *        |   /
      Constraint   |     \  \     /|\      /   /
      g(x, y) = k  |      \  '---' | '----'   /
      ------------>|-------'-------|---------'---------> x
                   |               |
                   |       grad f  |  grad g
                   |         (Parallel Vectors!)

Orthogonality of Gradients

Recall from The Gradient Vector that:

• The gradient $\nabla f$ is orthogonal to the level curves of $f$.
• The gradient $\nabla g$ is orthogonal to the constraint curve $g(x, y) = k$.

Because the two curves are mutually tangent at the optimal point $P$, their normal lines must coincide. This implies their gradient vectors point along the exact same line, meaning $\nabla f$ is a scalar multiple of $\nabla g$:

Physical and Economic Interpretation of $\lambda$ (Shadow Price / Sensitivity)

The Lagrange multiplier $\lambda$ is not merely an auxiliary algebraic tool; it possesses profound physical and economic significance. Let $f_{\text{opt}}(k)$ denote the optimal value of the objective function as a function of the constraint parameter $k$. It can be proven that:

• Rate of Relaxation: $\lambda$ quantifies the instantaneous rate of change in the optimal objective value per unit increase (relaxation) in the constraint threshold $k$.
• In economics and systems engineering, $\lambda$ is called the shadow price or marginal value: the maximum price a system designer should pay to relax a budget, volume, or stress limit by one unit.

Step-by-Step Problem-Solving Sequence

1. Identify the Functions: Formally define the objective function $f(x, y, z)$ to optimize and the constraint equation $g(x, y, z) = k$.
2. Compute Gradients: Compute $\nabla f = \langle f_x, f_y, f_z \rangle$ and $\nabla g = \langle g_x, g_y, g_z \rangle$.
3. Formulate the Scalar Component Equations:
   $$\hat{i}: f_x = \lambda g_x$$
   $$\hat{j}: f_y = \lambda g_y$$
   $$\hat{k}: f_z = \lambda g_z$$
   $$\text{Constraint}: g(x, y, z) = k$$
4. Eliminate $\lambda$: Express $\lambda$ in terms of coordinates or solve for one variable in terms of others.
   • Critical Warning: Never divide by a variable without proving it cannot be zero.
5. Substitute into the Constraint: Plug the variable relations into $g(x, y, z) = k$ to solve for candidate coordinates.
6. Evaluate Candidates: Compute $f$ at all candidate points to determine the absolute maximum and minimum.

Engineering Applications

• Aerospace Trajectory Optimization: Spacecraft thruster firings minimize a quadratic fuel burn metric $f(v_x, v_y) = c_1 v_x^2 + c_2 v_y^2$ subject to an orbital insertion velocity plane $g(v_x, v_y) = a v_x + b v_y = \Delta V$. The multiplier $\lambda$ gives the fuel mass sensitivity per unit increase in required mission $\Delta V$.
• Civil & Hydraulic Engineering: Open stormwater channel design minimizes wetted cross-section perimeter $P(b, y) = b + 2y$ (reducing friction and concrete lining costs) subject to a required hydraulic flow area $A(b, y) = by = A_0$.
• Robotics & Manufacturing: Sizing rectangular packaging enclosures minimizes raw sheet metal surface area $S(x, y, z) = 2xy + 2xz + 2yz$ subject to a mandatory payload volume constraint $V(x, y, z) = xyz = V_0$.

Example Problems

Example 1: Distance from Origin to a Plane in 3D

Find the point on the plane $x + 2y + 3z = 14$ that is closest to the origin, and determine the minimum distance.

Example 2: Non-Linear Circular Constraint

Find the maximum and minimum values of $f(x, y) = xy$ subject to the circular constraint $x^2 + y^2 = 8$.

Connections & References

• Prerequisites:
  • The Gradient Vector — Calculating $\nabla f$ and understanding orthogonality to level curves.
  • Level Curves and Contour Maps — Geometric foundation of tangency.
  • Extrema on Closed and Bounded Regions — Context for boundary optimization.
• Direct Successors:
  • Multiple Integrals — Setting up integration limits over optimized regions.
• Course Sessions:
  • Session 13 - Optimization with Constraints