Skip to content
Free public beta until December 4, 2026: every course is open, no card needed. After that, the first chapter of each course stays free and full access is a one-time $39 ($19 for up to 100 founding members chosen for helpful feedback). Your progress is kept. How the beta works

Chapter 0

Readiness: what optimization leans on

A saddle-shaped surface with a marked critical point, a two-variable feasible region shaded beneath a sliding objective line, and a short iterate sequence x^(0), x^(1), x^(2), ... climbing toward a target point, all arranged around one shared coordinate frame
One critical point, one feasible region, one iterate sequence -- the three pictures this whole course keeps redrawing.

The Toolkit Optimization Leans On

Every algorithm in this course does the same three things, over and over: it checks whether a point is a plausible answer, it accounts for whatever constraints the point has to respect, and it builds a sequence of better and better guesses on the way to an answer it can certify as correct. None of those three moves is new. Checking whether a point is a plausible answer is exactly what a critical point, a gradient, and a Hessian already do in Calculus III. Accounting for constraints is exactly what a Lagrange multiplier and a two-variable feasible region already do in Calculus III and College Algebra. And the machinery behind "a sequence of better and better guesses" -- eigenvalues, symmetric matrices, quadratic forms -- comes straight out of Linear Algebra. This chapter's first five sections put all of that back on the workbench, each one reviewed at full strength and not a lighter pass, because every later chapter assumes it is sitting there ready to use without a re-derivation.

Section 1 reopens critical points and the second derivative test: where a smooth surface goes flat, and how the sign pattern of D and f_{xx} tells a bowl from a dome from a saddle. Section 2 puts the gradient and the Hessian back on the table as the two objects every optimization method actually computes -- a vector pointing uphill and a matrix of curvature information, not just intermediate steps toward a single number. Section 3 turns to eigenvalues, symmetric matrices, and quadratic forms, the exact linear-algebra machinery a Hessian's signs will lean on the moment this course needs to certify a critical point as a genuine minimum rather than a saddle in disguise. Section 4 revisits Lagrange multipliers, the gradient- parallelism condition that this course's KKT theory later generalizes to inequality constraints. Section 5 reopens linear programming exactly as College Algebra first drew it: a shaded feasible region, a sliding objective line, and an optimum sitting at a corner.

The last three sections are a different kind of readiness -- not a review of a subject taught elsewhere, but the shared notation this course is about to use on every single page and has not needed a name for until now. Section 6 lays out how a sequence and a limit describe an algorithm's progress across many steps, and how big-O notation compares how fast two quantities grow or shrink without pinning down either one exactly. Section 7 fixes this course's own convention for a decision variable, a vector of several of them collected into one design point, and a matrix of many design points stacked into rows. Section 8 closes the chapter with summation notation and the iterate-sequence superscript x^{(k)}, the bookkeeping every algorithm in every later chapter uses to talk about "the guess after k steps" without redefining it each time.

Nothing in this chapter is new mathematics. What changes is the lens: a critical point stops being the end of a calculus problem and becomes a candidate answer waiting to be certified; a feasible region stops being a shaded picture and becomes the object an entire algorithm walks around the edge of. By the end of the chapter, every tool this course leans on is back within reach, and the notation for talking about all of them at once is finally on the page.

  • No matches.