Averaging first and then applying a function almost never gives the same answer as applying the function first and then averaging. Jensen's inequality says exactly which direction the gap goes, and that direction depends only on whether the function is convex or concave.
Theorem
Jensen's Inequality
also known as: the convexity inequality for expectations
Let g be a convex function and X a random variable. Then E[g(X)] \geq g(E[X]). If g is concave instead, the inequality reverses: E[g(X)] \leq g(E[X]).
Direct, for the convex case. Let m = E[X]. Because g is convex, its graph lies on or above every one of its tangent lines — in particular the tangent line at the single point x=m: g(x) \geq g(m) + g'(m)(x-m) \quad \text{for all } x (if g isn't differentiable at m, any supporting line at that point works the same way). Take the expectation of both sides over X: E[g(X)] \geq E\big[g(m) + g'(m)(X-m)\big] = g(m) + g'(m)\big(E[X]-m\big). Since m = E[X], the term E[X]-m is exactly 0, so the right side collapses to g(m) = g(E[X]): E[g(X)] \geq g(E[X]). The concave case follows by applying this result to -g, which is convex whenever g is concave, and flipping the sign back.
Picture a bowl-shaped curve (convex) and mark several points along it, scattered left and right. Their average height on the curve — go straight up from each point, read the height, then average those numbers — sits above the height of the curve at the average x-position. That's because the curve dips down toward the middle between any two of those points (it's a bowl), so the middle point on the curve is always lower than the average of the outer ones. Flip the bowl into a dome (concave) and the inequality flips with it: the average height beats the height at the average position.
The order of operations is the entire content here: "average, then apply g" versus "apply g, then average" are different numbers whenever g actually curves, and Jensen's inequality is the statement of which one is bigger.
The proof's engine is the chord/tangent-line picture from the previous section: a convex function's graph sits on or above every one of its own tangent lines. Evaluating that fact at the single point x=E[X] and taking an expectation is what produces the inequality — no calculus beyond "a convex function lies above its tangent" is needed. Two conditions matter for correct use. First, direction depends entirely on convex vs. concave — reusing the inequality with the wrong sign is the single most common misuse. Second, the inequality is about E[g(X)] versus g(E[X]) specifically; it says nothing about g(E[X]) versus any other statistic of X, such as its median or mode.
Worked example
A concrete gap, not just an inequality symbol
-
1
Let X take values 1 and 9, each with probability \frac12. Compute both E[X^2] and (E[X])^2, and confirm Jensen's inequality holds for the convex function g(x)=x^2.
-
2
E[X] = \frac12(1) + \frac12(9) = 5, so (E[X])^2 = 25. Separately, E[X^2] = \frac12(1) + \frac12(81) = 41.
-
3\boxed{E[X^2] = 41 \geq 25 = (E[X])^2}
This particular instance of Jensen's inequality is worth remembering by name: E[X^2] \geq (E[X])^2 always holds, for any random variable, and it's exactly the fact that makes variance (V[X] = E[X^2]-(E[X])^2) never negative.
Worked example
Equality only when there's nothing to average over
-
1
For which distributions of X does Jensen's inequality become an equality, E[g(X)] = g(E[X]), when g is strictly convex (like x^2)?
-
2
Recognition. The proof's key step was g(x) \geq g(m) + g'(m)(x-m) — the curve sitting on or above its tangent line at m=E[X]. That step is an equality only where the curve touches its own tangent line, which for a strictly convex function happens at exactly one point: x=m itself.
-
3\boxed{\text{Equality holds only when } X = E[X] \text{ with probability 1 — i.e. } X \text{ is constant.}}
Any genuine spread in X's distribution makes the inequality strict.
Watch the gap open as spread increases
Drag s up from 0 — as the two points spread further from the center, the chord's midpoint rises further above the curve.
Loading interactive…
Where does this distribution balance?
Guess the balance point of the X from the worked example above, then reveal E[X]=5.
Loading interactive…
Which direction does the inequality go?
Sort each setup by whether E[g(X)] is above or below g(E[X]).
Loading interactive…
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
What this task checks
Structure checked automatically. Whether your ideas are convincing isn't graded, so reread with the checklist.
The compiler says
A model answer
Correct answer
Also accepted:
Why
Why
Where this goes next
Every non-negativity proof ahead — entropy, relative entropy, mutual information — is Jensen's inequality applied to a specific convex or concave function, most often -\log p(x). Learning to recognize "this is Jensen's inequality in disguise" is worth more than memorizing any one of those individual proofs.
Check your understanding (counts toward your progress)
X takes values 2 and 8, each with probability \frac12. Compute E[X^2] - (E[X])^2.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
E[X]=5 so (E[X])^2=25. E[X^2] = \frac12(4)+\frac12(64)=34. The difference is 34-25=9 — this is exactly V[X], and Jensen's inequality guarantees it's never negative.
Correct answer
Also accepted:
Why
Compute E[X] first and square it, then separately compute E[X^2] by averaging the squared values — don't combine the two calculations into one step.
For which distribution of X does Jensen's inequality become an equality for a strictly convex g?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Symmetry alone isn't enough — a symmetric two-point distribution still has a strict gap, as the worked example showed.
Finite variance doesn't prevent a strict gap; only ZERO variance does.
Why
The proof's tangent-line step is an equality only where the curve touches its tangent line, which for a strictly convex function is a single point — so X must equal E[X] with probability 1.
Revisit the proof's tangent-line step, g(x) \geq g(m) + g'(m)(x-m) evaluated at m=E[X] — ask where a STRICTLY convex curve can actually touch its own tangent line.
A strictly convex curve touches any one of its tangent lines at exactly one point — translate that geometric fact into a statement about how much X is allowed to vary.
X is uniform on \{1,2,3,4,5\} (each value equally likely). Using g(x)=x^2, compute E[X^2] - (E[X])^2.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
E[X]=3, so (E[X])^2=9. E[X^2] = \frac15(1+4+9+16+25) = \frac{55}{5}=11. The gap is 11-9=2, positive as Jensen's inequality with g(x)=x^2 guarantees for any non-constant X.
Correct answer
Also accepted:
Why
List all five values of X^2 first (1,4,9,16,25), average those five numbers for E[X^2], then separately square the average of X itself for (E[X])^2.
For a concave function g and random variable X, Jensen's inequality says:
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
That direction is for CONVEX functions — concave flips it.
Equality only holds when X is constant, not in general.
Why
Concave functions bulge above their chords, so averaging first and applying g beats applying g first and averaging: E[g(X)] \leq g(E[X]).
Practice: Jensen's inequality (5 problems · doesn’t affect your progress)
Convex functions and concave functions give Jensen's inequality in opposite directions. Which pairing is correct?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
This pairing has both directions swapped from the correct one — check which shape (bowl or dome) bulges above its chord.
The direction genuinely flips between the two shapes; it can't be the same inequality for both.
Equality in general only holds when X is constant — that's not the typical case for either shape.
Why
A convex (bowl-shaped) function bulges below its chords, giving E[g(X)]\geq g(E[X]); a concave (dome-shaped) function bulges above its chords, flipping the inequality to E[g(X)]\leq g(E[X]).
Convex: E[g(X)]\geq g(E[X]); concave: E[g(X)]\leq g(E[X])
X takes values 1, 4, and 7 with probabilities 0.2, 0.5, and 0.3. Compute E[X^2] - (E[X])^2.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
E[X] = 0.2(1)+0.5(4)+0.3(7) = 4.3, so (E[X])^2=18.49. E[X^2] = 0.2(1)+0.5(16)+0.3(49) = 22.9. The gap is 22.9-18.49=4.41.
Correct answer
Also accepted:
Why
Compute E[X] and square it separately first, then compute E[X^2] by averaging the squared values across all three outcomes.
X is a random variable and g(x)=\sqrt{x} (concave on x>0). Which inequality must hold?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
That direction belongs to a convex function — \sqrt{x} is concave, so the inequality flips.
Equality only holds when X is constant, not for a general distribution.
The DIRECTION of the inequality is fixed by concavity alone; only the SIZE of the gap depends on the distribution.
Why
\sqrt{x} is concave on x>0, so Jensen's inequality gives E[g(X)] \leq g(E[X]), i.e. E[\sqrt X] \leq \sqrt{E[X]}.
E[\sqrt X] \leq \sqrt{E[X]}
You're told only that E[X]=10 and that g is a strictly convex function — you don't know X's full distribution. What can you conclude about E[g(X)] compared to g(10)?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Jensen's inequality gives a guaranteed bound from convexity and the mean alone — no further distribution details are needed for the inequality itself.
Equality only holds when X is constant with probability 1, which isn't given here.
Convex flips this the other way from what's being proposed — check the direction Jensen's inequality guarantees for a convex g.
Why
Jensen's inequality needs only convexity and the mean — E[g(X)]\geq g(E[X])=g(10) holds regardless of what specific distribution produced that mean.
E[g(X)] \geq g(10), even without knowing X's full distribution
The equality condition worked out earlier required X to be exactly constant for a STRICTLY convex g. If g is convex but not strict — say a function that's exactly linear on some sub-interval — can equality E[g(X)]=g(E[X]) hold even when X isn't constant?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
The strictness of g is exactly what forced that conclusion earlier — drop it, and the argument no longer applies.
Jensen's inequality still holds for a non-strict convex function; it just doesn't force the strict-inequality-except-at-one-point conclusion.
Symmetry of X's distribution isn't the relevant condition — where X's values sit relative to g's linear stretch is what matters.
Why
Strict convexity is exactly what rules out equality for non-constant X — drop that strictness (a linear stretch somewhere in the domain) and any X confined to that stretch makes the chord and curve coincide, so equality can hold without X being constant.
Yes — if X's entire distribution sits inside the region where g is linear, the chord coincides with the curve there