Section 1 used the gradient and the second partials only long enough to set them to zero and read off a sign pattern. From here on, this course treats both of them as first-class objects in their own right, not steps on the way to somewhere else: the gradient \nabla f is the vector every descent method steps opposite to, and the Hessian H is the matrix every second-order method uses to model how the objective curves. Getting comfortable with both as objects -- not just numbers to plug into a formula -- is what this section is for.
Formula
The Gradient in Three Variables
Recalled from Calculus III — Multivariable Calculus · The gradient in three variables — first taught there.
For a differentiable function f(x,y,z), the gradient generalizes with one more component: \nabla f(x,y,z) = \langle f_x,\,f_y,\,f_z\rangle.
Every fact already established in two variables carries over unchanged. For a unit vector \mathbf{u}=\langle a,b,c\rangle, D_{\mathbf{u}}f = \nabla f\cdot\mathbf{u} = f_x a + f_y b + f_z c.
\nabla f points in the direction of steepest increase of f, with magnitude |\nabla f| equal to that maximum rate. Treating f as a level-surface function F, \nabla f is also normal to the level surface f(x,y,z)=k through any point where it is nonzero.
Think of the gradient as a compass needle that has learned to point uphill instead of north, planted at whichever spot on the landscape you're standing on -- it swings to a new heading and a new strength (how steep the climb is) every time you move. The Hessian is a different kind of instrument: not a direction, but a report on how the steepness itself is changing as you look around. A calm, bowl-shaped valley reports "curving the same amount everywhere I look." A mountain pass reports "curving upward one way and downward the other," and that report is exactly what lets you tell a bowl from a saddle without ever having to walk the whole landscape by hand. Because the Hessian is built from the same list of mixed partial derivatives no matter which one you compute first, it is always perfectly symmetric across its own diagonal -- a fact section 3 turns into real leverage.
The gradient collects every first partial derivative into one column vector, \nabla f=\langle f_x,f_y,f_z\rangle (or however many variables f has) -- a single object pointing in the direction f increases fastest, with length equal to that fastest rate of increase. The Hessian collects every second partial derivative into a square matrix, H=\begin{pmatrix}f_{xx}&f_{xy}\\f_{xy}&f_{yy}\end{pmatrix} in two variables, symmetric whenever the mixed partials are continuous (Clairaut's theorem again, now framed as "the Hessian is always a symmetric matrix," which is exactly the property section 3 needs). D from section 1's test is nothing but \det H in the 2\times2 case, restated so it never needed the word "matrix" at all -- but the moment a third variable enters, there is no single number that plays D's role anymore, and the Hessian itself, eigenvalues and all, is the only version of the test that still works.
The gradient as a compass-needle field
This is f(x,y)=4-x^2-y^2. Every arrow is \\nabla f planted at its own tail point -- drag the dial at the marked point until it lines up with the arrow there, and watch the readout hit its largest value exactly when it does.
Loading interactive…
A gradient field in three variables
This is \\nabla f for f(x,y,z)=xyz. Toggle the sphere and drag its radius slider -- the arrows are no longer purely radial the way a distance-from-origin function's gradient would be, since this f depends on the product of all three coordinates.
Loading interactive…
Where the Hessian reports mixed curvature
For f(x,y)=x^2-y^2 at (0,0): f_{xx}=2, f_{yy}=-2, f_{xy}=0, so H=\begin{pmatrix}2&0\\0&-2\end{pmatrix} -- positive curvature on one diagonal entry, negative on the other. Drag along the x-axis (rising) and then the y-axis (falling) through the marked point to see what a mixed-sign Hessian actually looks like.
Loading interactive…
Worked example
Gradient and Hessian of f(x,y,z)=x^2y+yz^2 at a point
-
1
Find \nabla f and, treating z as fixed at its given value, the Hessian in x and y, both evaluated at (1,2,3).
-
2
First partials: f_x=2xy, f_y=x^2+z^2, f_z=2yz. At (1,2,3): \nabla f(1,2,3)=\langle4,10,12\rangle.
-
3fixing z turns this into the familiar 2x2 Hessian from section 1
Second partials in x,y: f_{xx}=2y, f_{yy}=0, f_{xy}=2x. At (1,2,3): f_{xx}=4, f_{yy}=0, f_{xy}=2.
fixing z turns this into the familiar 2x2 Hessian from section 1
-
4
\boxed{\nabla f(1,2,3)=\langle4,10,12\rangle,\quad H=\begin{pmatrix}4&2\\2&0\end{pmatrix}} Two completely different objects computed from the same function: one vector telling you which way is steepest, one symmetric matrix telling you how that steepness itself curves.
Where this goes next
Every algorithm this course builds
The gradient is the one ingredient every first-order method in this course needs -- plain gradient descent, momentum, stochastic gradient descent, all of them step opposite \nabla f and differ only in how. The Hessian is the one ingredient every second-order method needs -- Newton's method models the whole objective locally as a quadratic built from H and jumps straight to that quadratic's minimum. A course-long theme starts right here: cheap methods use only \nabla f, and expensive-but-fast methods spend the extra cost computing H too.
Quick review
Practice from earlier coursework, pulled in for this topic.
From Calculus III — Multivariable Calculus
Find the gradient \nabla f(1,2,-1) for f(x,y,z)=2x^2+y^2-z^2.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Enter as components — <2, -1, 3>, (2, -1, 3), or 2i - j + 3k all count. Any algebraically equivalent component counts too.
Why
f_x(1,2,-1)=4(1)=4, f_y(1,2,-1)=2(2)=4, f_z(1,2,-1)=-2(-1)=2.
Correct answer
Also accepted:
Why
f_x=4x, f_y=2y, f_z=-2z.
From Calculus III — Multivariable Calculus
Find the steepest-ascent rate of f(x,y,z)=x^2+y^2+z^2 at (1,2,2).
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
\nabla f(1,2,2)=\langle2,4,4\rangle, so |\nabla f(1,2,2)|=\sqrt{4+16+16}=\sqrt{36}=6.
Correct answer
Also accepted:
Why
The rate is |\nabla f|.
From Calculus III — Multivariable Calculus
For f(x,y,z)=x^2+2y^2-z^2, find D_{\mathbf{u}}f(1,1,1) in the direction from (1,1,1) toward (2,3,1).
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
D_{\mathbf u}f=\dfrac{\langle2,4,-2\rangle\cdot\langle1,2,0\rangle}{\sqrt5}=\dfrac{2+8+0}{\sqrt5}=\dfrac{10}{\sqrt5}=2\sqrt5\approx4.4721.
Correct answer
Also accepted:
Why
The direction vector is \langle1,2,0\rangle, length \sqrt5.
\nabla f(1,1,1)=\langle2,4,-2\rangle.
Quick review
Practice from earlier coursework, pulled in for this topic.
From Calculus III — Multivariable Calculus
f(x,y) = x^3-3x+y^2 has critical points at (1,0) and (-1,0). Classify (-1,0).
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Check f_xx at x=-1: f_xx = 6x = -6, and check the sign of D before deciding.
D is negative at this point, which rules out a local minimum.
D is nonzero here.
Why
f_xx=6x, f_yy=2, f_xy=0. At (-1,0): D=(-6)(2)-0^2=-12<0, a saddle point.
Compute D = f_xx f_yy - f_xy^2 at x=-1, y=0.
From Calculus III — Multivariable Calculus
For f(x,y) = x^3 - 3x + y^2, find D at the critical point (1, 0).
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
f_{xx}=6x, f_{yy}=2, f_{xy}=0. At (1,0): D=(6)(2)-0^2=12.
Correct answer
Also accepted:
Why
First confirm (1,0) is a critical point: f_x = 3x^2-3 = 0 at x=1, f_y=2y=0 at y=0.
From Calculus III — Multivariable Calculus
For f(x,y) = xy - x^2 - y^2 - 2x - 2y + 4, find f_{xx} (a constant).
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
f_x = y-2x-2, so f_{xx}=-2.
Correct answer
Also accepted:
Why
Find f_x first, then differentiate that result with respect to x again.
Practice: Computing gradients and Hessians together (3 problems · doesn’t affect your progress)
For f(x,y,z)=x^2y-yz, what is the first component of \nabla f at (2,1,3)?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
f_x=2xy, so f_x(2,1,3)=2(2)(1)=4.
Correct answer
Also accepted:
Why
For f(x,y)=x^3-3xy, what is the Hessian entry f_{xy}?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
f_x=3x^2-3y, so f_{xy}=\partial f_x/\partial y=-3.
Correct answer
Also accepted:
Why
A Hessian has one positive and one negative diagonal entry after diagonalization (eigenvalues of opposite sign). What does this say about the point?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
A local minimum needs every eigenvalue positive, not just one.
A local maximum needs every eigenvalue negative, not just one.
A symmetric Hessian can absolutely have mixed-sign eigenvalues -- that's precisely the saddle-point signature, not an error.
Why
Mixed-sign eigenvalues mean the surface curves upward in one principal direction and downward in another -- the defining shape of a saddle.
It is a saddle point