Every section so far has optimized freely, with no restriction on where \mathbf x is allowed to sit. Almost no real problem works that way -- a production plan can't use negative materials, a portfolio has to sum to one, a design has to fit inside a budget. Calculus III's answer to "the best point subject to a constraint" was the Lagrange multiplier, and it comes back here unchanged, as the equality-constrained special case of the much larger constrained-optimization theory this course spends chapter 6 building.
Theorem
The Method of Lagrange Multipliers
also known as: the Lagrange condition
Recalled from Calculus III — Multivariable Calculus · Lagrange multipliers — first taught there.
Suppose f and g have continuous partial derivatives, and f has a local extremum at a point (a,b) on the constraint curve g(x,y) = k, with \nabla g(a,b) \neq \mathbf{0}. Then there is a scalar \lambda (the Lagrange multiplier) with \nabla f(a,b) = \lambda\, \nabla g(a,b). Every constrained extremum is therefore found among the solutions of the system \nabla f = \lambda \nabla g together with g(x,y) = k: three equations (f_x = \lambda g_x, f_y = \lambda g_y, g = k) in three unknowns x, y, \lambda.
Parametrize the constraint curve near (a,b) by arc length, \mathbf{r}(t) = (x(t), y(t)), with \mathbf{r}(0) = (a,b) — this is possible precisely because \nabla g(a,b) \neq \mathbf{0}, which keeps the level curve g=k smooth with a well-defined tangent there. Restrict f to the curve: h(t) = f(x(t), y(t)). If f has a local extremum at (a,b) subject to the constraint, then h has an ordinary single-variable local extremum at t=0, so h'(0) = 0. The chain rule expands that as h'(0) = f_x(a,b)\,x'(0) + f_y(a,b)\,y'(0) = \nabla f(a,b)\cdot \mathbf{r}'(0) = 0, meaning \nabla f(a,b) is perpendicular to the curve's tangent vector \mathbf{r}'(0) at that point. But \nabla g(a,b) is also perpendicular to the same tangent vector — that's what it means for g=k to be a level curve of g. Two vectors both perpendicular to the same tangent line in the plane must be parallel to each other, so \nabla f(a,b) = \lambda \nabla g(a,b) for some scalar \lambda.
Imagine walking along a fixed trail (the constraint) that winds across a hilly landscape (the objective f), trying to find the trail's highest point. At most spots on the trail, stepping forward either climbs or descends -- so that spot can't be the highest, since walking a little further would do better. The highest point on the trail is the one special spot where the trail runs exactly along a level line of the landscape, tangent to it rather than crossing it, so there's no forward or backward step left that still climbs. That tangency is exactly \nabla f=\lambda\nabla g: both gradients point perpendicular to their own level curve or constraint curve, and tangency forces those two perpendiculars to line up, possibly stretched or flipped by the scalar \lambda. Two constraints at once means walking along the intersection of two trails, a single narrower path, and the same tangency logic now needs both constraint directions combined to pin down where climbing stops completely.
To optimize f(x,y) subject to a single constraint g(x,y)=k, the Lagrange condition requires the gradients to be parallel at the optimum: \nabla f=\lambda\nabla g for some scalar \lambda, the Lagrange multiplier. Geometrically, the constraint curve g=k and the level curve of f passing through the optimum are tangent there -- if they crossed transversally instead, sliding along the constraint curve would still change f, so the point couldn't be optimal yet. With two constraints g(x,y,z)=k_1 and h(x,y,z)=k_2 at once, the same idea generalizes to \nabla f=\lambda\nabla g+\mu\nabla h: the objective's gradient must be some combination of both constraint gradients, since the feasible set is now a curve (the intersection of two surfaces) rather than a single surface, and \nabla f has to lie in the plane those two constraint gradients span to rule out an improving direction still open along that curve.
The objective's own gradient field
This is f(x,y)=xy, with the dial at the point (2,2). Turn the amber dial until it lines up with the thick green gradient arrow -- that heading is the direction f increases fastest from here, constraint or no constraint.
Loading interactive…
Parallel gradients at the constrained optimum
At the constrained optimum, \\nabla f=\\lambda\\nabla g for some scalar \\lambda -- here \\nabla f=(2,2) and \\nabla g=(4,4), so \\lambda=1/2. Drag either vector and notice they only ever differ by a scaling, never a direction.
Loading interactive…
Splitting a gradient between two constraints
Schematic in 2D: \\nabla f sits in the plane spanned by \\nabla g and \\nabla h. Drag either colored vector and watch their sum trace out that plane's worth of reachable combinations -- \\nabla f has to be reachable this way for a two-constraint optimum.
Loading interactive…
Worked example
Maximizing f(x,y)=x^2y subject to x+y=6
-
1
Find the maximum of f(x,y)=x^2y on the constraint g(x,y)=x+y=6.
-
2two equations, one shared unknown lambda -- eliminate it next
Gradients: \nabla f=\langle2xy,\,x^2\rangle and \nabla g=\langle1,1 \rangle. Set \nabla f=\lambda\nabla g: 2xy=\lambda and x^2=\lambda.
two equations, one shared unknown lambda -- eliminate it next
-
3
Since both equal \lambda: 2xy=x^2, so x(2y-x)=0, giving x=0 or x=2y.
-
4
If x=0: the constraint gives y=6, and f(0,6)=0. If x=2y: the constraint gives 2y+y=6, so y=2 and x=4, and f(4,2)=(16)(2)=32.
-
5
\boxed{\text{Maximum }f=32\text{ at }(x,y)=(4,2).} The candidate (0,6) satisfies the Lagrange condition too (trivially, since \nabla f= \mathbf 0 there), but comparing values settles which one actually wins.
Where this goes next
Chapter 6: KKT conditions
Everything here assumes an equality constraint -- the feasible set is a curve or surface with no inside to speak of. The moment a constraint becomes an inequality, like g(x,y)\le k instead of g(x,y)=k, the optimum might land strictly inside the feasible region where the constraint plays no role at all, or right on its boundary where this section's tangency condition still applies. Telling those two cases apart, and handling several inequality constraints of both kinds at once, is exactly what chapter 6's KKT conditions generalize this section's \nabla f=\lambda\nabla g into.
Quick review
Practice from earlier coursework, pulled in for this topic.
From Calculus III — Multivariable Calculus
For f(x,y)=x+y subject to x^2+y^2=2, the Lagrange system gives 1=2\lambda x and 1=2\lambda y, so x=y. Which point gives the MAXIMUM of f?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
This point satisfies x=y and the constraint, but f(-1,-1)=-2, the minimum, not the maximum.
This point doesn't even satisfy x=y from the Lagrange condition.
This point doesn't satisfy x=y either.
Why
x=y with x^2+y^2=2 gives x=y=1 or x=y=-1. f(1,1)=2 is larger than f(-1,-1)=-2, so (1,1) is the maximum.
Solve x=y together with the constraint x^2+y^2=2 to get the two candidate points.
Evaluate f=x+y at each candidate point and compare.
From Calculus III — Multivariable Calculus
At a constrained extremum found by the Lagrange multiplier method, which relationship holds between the gradients?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
It's the opposite -- the gradients are PARALLEL at a constrained extremum, not perpendicular to each other.
This forces lambda = -1 specifically, which is only one special case, not the general condition.
Requiring the zero vector would mean an unconstrained critical point, not a constrained one.
Why
The Lagrange condition is exactly that the two gradients are scalar multiples of each other -- geometrically, the objective's level curve is tangent to the constraint curve there.
From Calculus III — Multivariable Calculus
Maximize f(x,y)=xy subject to x+y=10. Using \nabla f = \lambda \nabla g: (y,x)=\lambda(1,1) gives x=y. Combined with the constraint, find x.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
x=y and x+y=10 give 2x=10, x=5.
Correct answer
Also accepted:
Why
Since x=y, substitute into the constraint x+y=10.
Quick review
Practice from earlier coursework, pulled in for this topic.
From Calculus III — Multivariable Calculus
With two constraints g(x,y,z)=k1 and h(x,y,z)=k2, what does the Lagrange condition become?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
That's the single-constraint version -- two constraints need a second multiplier term.
The cross product isn't the relationship here -- it's a linear combination with two independent scalars.
This isn't the right combination -- it's a sum of two scaled gradients, not a product.
Why
With two constraints, the objective's gradient must be a linear combination of both constraint gradients: grad f = lambdagrad g + mugrad h.
From Calculus III — Multivariable Calculus
Extremize f(x,y,z)=x+y+z subject to x^2+y^2+z^2=4 and z=0. Since z=0 collapses the problem to two variables on a radius-2 circle, find the maximizing x-value.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
With z=0, maximize x+y on x^2+y^2=4, which (by the single-constraint Lagrange method) gives x=y=sqrt(2).
Correct answer
Also accepted:
Why
With z fixed at 0, this reduces to the single-constraint problem: maximize x+y on x^2+y^2=4.
From Calculus III — Multivariable Calculus
Minimize f(x,y,z)=x^2+y^2+z^2 subject to x+y+z=6 and x-y=0. Since x=y, substitute into the first constraint to express z in terms of x: z=6-2x. If the minimizing x=2, find the corresponding z.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
z = 6 - 2(2) = 2.
Correct answer
Also accepted:
Why
Substitute the given x=2 directly into z=6-2x.
Practice: Setting up and reading the Lagrange condition (3 problems · doesn’t affect your progress)
At a candidate optimum, \nabla f=(6,9) and \nabla g=(2,3). What value of \lambda makes \nabla f=\lambda\nabla g hold?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
6=\lambda(2)\Rightarrow\lambda=3; checking the second component, 9=3(3) confirms it.
Correct answer
Also accepted:
Why
At a different candidate point, \nabla f=(4,5) and \nabla g=(2,3). Does this point satisfy the Lagrange condition?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Pointing in a roughly similar direction isn't enough -- the Lagrange condition demands the two vectors be EXACT scalar multiples of each other.
All-positive components say nothing about whether one vector is a scalar multiple of the other.
The condition is a direct, checkable ratio comparison -- nothing further is needed to decide it here.
Why
\nabla f=\lambda\nabla g requires a single \lambda that works for every component at once; 4/2=2 and 5/3\approx1.67 disagree, so no such \lambda exists.
No -- 4/2=2 but 5/3\ne2, so the two gradients aren't parallel
With two constraints, \nabla f=\lambda\nabla g+\mu\nabla h. If \nabla g=(1,0), \nabla h=(0,1), \lambda=5, and \mu=-2, what is \nabla f's first component?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
\nabla f=5(1,0)+(-2)(0,1)=(5,-2); the first component is 5.
Correct answer
Also accepted:
Why