Skip to content
Free public beta until December 4, 2026: every course is open, no card needed. After that, the first chapter of each course stays free and full access is a one-time $39 ($19 for up to 100 founding members chosen for helpful feedback). Your progress is kept. How the beta works

Which way of handling inequality constraints do I use?

KKT says what an optimal point must satisfy once inequality constraints are in play; penalty, augmented Lagrangian, barrier, interior-point, and SQP are the algorithms that actually chase one down.

No feasible starting point on hand? Penalty or augmented Lagrangian methods don't need one — augmented Lagrangian trades a little extra bookkeeping (a multiplier update) for much better conditioning than plain penalty as the penalty weight grows. Need every intermediate iterate to stay feasible, not just the final answer? Barrier methods (few constraints) or interior-point methods (many constraints, large-scale) both require a feasible start but never leave the feasible region while solving. Is the problem well approximated locally by a quadratic objective and linear constraints? SQP converges fast when that local model is trustworthy, wrapped in a trust region when it isn't.

What "satisfied" means

An inequality constraint that holds with equality at the current point.
The four conditions — stationarity, primal feasibility, dual feasibility, complementary slackness — a constrained optimum must satisfy.
A convex problem has a point that strictly satisfies every inequality constraint.

Consider the convex problem \min_{x} f_0(x) \quad\text{s.t.}\quad f_i(x)\le 0\ (i=1,\dots,m), \quad Ax=b, with f_0,f_1,\dots,f_m convex and differentiable, and let p^* denote its optimal value (finite). Slater's condition holds if there is a point \tilde x with A\tilde x = b and f_i(\tilde x) < 0 for every i (strictly feasible for every inequality constraint).

If Slater's condition holds, then strong duality holds (p^*=d^*, the common value attained by some dual-feasible (\lambda^*,\nu^*)), and consequently the KKT conditions are not just sufficient for optimality of a feasible x^* (which convexity alone already guarantees) but necessary: every primal optimal x^* is paired with a dual optimal (\lambda^*,\nu^*) satisfying stationarity, dual feasibility, and complementary slackness (see kkt-conditions).

Sanity check: take the convex problem \min x^2 s.t. x\le -1 (so f_1(x)=x+1\le0). Slater's condition holds: \tilde x=-2 gives f_1(-2)=-1<0, strictly feasible. The optimum is x^*=-1 (the boundary, since x^2 is increasing in |x| away from 0 and the feasible set excludes 0). KKT stationarity: \nabla_x L = 2x+\lambda = 0 at x^*=-1 gives \lambda^*=2\ge0 (dual feasible), and complementary slackness \lambda^*f_1(x^*) = 2\cdot 0=0 holds since f_1(-1)=0 is tight — exactly as the theorem promises for a Slater-satisfying convex problem. Contrast a problem where Slater fails: \min x^2 s.t. x^2\le 0 (only x=0 is feasible, so no strictly feasible point exists) — here the KKT machinery built on Slater simply doesn't apply, which is the gap this theorem exists to name.

The regularity condition that has to hold before KKT is trusted as more than a necessary condition.

Show proof

Step 1 — weak duality (always true, no convexity needed). Define the Lagrangian L(x,\lambda,\nu) = f_0(x) + \sum_i \lambda_i f_i(x) + \nu^\top(Ax-b) and the dual function g(\lambda,\nu) = \inf_x L(x,\lambda,\nu). For any \lambda\ge0 and any primal-feasible x, every term \lambda_i f_i(x)\le0 and \nu^\top(Ax-b)=0, so L(x,\lambda,\nu)\le f_0(x); taking the infimum over x preserves this, so g(\lambda,\nu)\le f_0(x) for every feasible x, hence g(\lambda,\nu)\le p^* for every dual-feasible (\lambda,\nu) — this is weak-duality-lp's general (nonlinear) counterpart, and gives d^*=\sup g \le p^* always.

Step 2 — Slater's condition closes the gap. Consider the set in \mathbb{R}^m\times\mathbb{R} \mathcal A = \{(u,t) : \exists x,\ Ax=b,\ f_i(x)\le u_i\ \forall i,\ f_0(x)\le t\}. Convexity of each f_i makes \mathcal A convex: given (u,t),(u',t')\in \mathcal A witnessed by x,x', the point x''=\theta x+(1-\theta)x' satisfies Ax''=b and, by convexity of each f_i, f_i(x'')\le \theta f_i(x)+(1-\theta)f_i(x') \le \theta u_i+(1-\theta)u_i', so (\theta u+(1-\theta)u',\, \theta t+(1-\theta)t')\in\mathcal A.

The point (\mathbf 0, p^*) is not in the interior of \mathcal A: any (u,t) with every u_i<0 and t<p^* in \mathcal A would exhibit a feasible x (since u_i<0 forces f_i(x)\le u_i<0) with f_0(x)\le t<p^*, contradicting p^*'s definition as the infimum. So (\mathbf 0,p^*) lies on the boundary of (the closure of) \mathcal A, and by the supporting-hyperplane theorem there is a nonzero (\lambda,\mu)\in\mathbb{R}^m\times\mathbb{R} with \lambda\ge0,\mu\ge0 such that \mu\, t + \lambda^\top u \ \ge\ \mu\, p^* \qquad \text{for every } (u,t)\in\mathcal A. \qquad(\ast)

Step 3 — \mu>0, using Slater. Suppose instead \mu=0. Then (\ast) reads \lambda^\top u \ge 0 for every u with u_i \ge f_i(x) for some feasible x; in particular at the Slater point \tilde x, u=f(\tilde x) gives \lambda^\top f(\tilde x)\ge0. But f_i(\tilde x)<0 for every i and \lambda\ge0 with \lambda\ne0 (since (\lambda,\mu)\ne0 and \mu=0) forces \lambda^\top f(\tilde x) < 0 — contradiction. So \mu>0, and dividing (\ast) by \mu we may take \mu=1 without loss of generality: t+\lambda^\top u \ge p^* for every (u,t)\in\mathcal A, with \lambda\ge0.

Step 4 — this \lambda is dual optimal with no gap. Taking, for any feasible x, u=f(x) and t=f_0(x) (a valid element of \mathcal A) gives f_0(x) + \lambda^\top f(x) \ge p^* for every x satisfying Ax=b — and since this only used Ax=b, it in fact holds with some matching \nu absorbed into the affine part, i.e. \inf_{x} \{f_0(x)+\lambda^\top f(x) + \nu^\top(Ax-b)\} \ge p^* for the appropriate \nu, i.e. g(\lambda,\nu)\ge p^*. Combined with weak duality (g(\lambda,\nu)\le p^* always), this forces g(\lambda,\nu)=p^* exactly — strong duality, with (\lambda,\nu)=(\lambda^*,\nu^*) dual optimal.

Step 5 — KKT necessity follows. Let x^* be primal optimal. Since g(\lambda^*,\nu^*)=f_0(x^*) (both equal p^*) and g(\lambda^*,\nu^*)=\inf_x L(x,\lambda^*,\nu^*)\le L(x^*,\lambda^*,\nu^*) = f_0(x^*)+\sum_i\lambda_i^* f_i(x^*), and \sum_i\lambda_i^*f_i(x^*)\le0 (feasibility and \lambda^*\ge0), the whole chain f_0(x^*) = g(\lambda^*,\nu^*) \le L(x^*,\lambda^*,\nu^*) \le f_0(x^*) must be equality throughout. Equality in \sum_i\lambda_i^*f_i(x^*)\le 0 term by term (each term is \le0) forces \lambda_i^* f_i(x^*)=0 for every i — complementary slackness. And x^* minimizing L(x,\lambda^*,\nu^*) over x (equality g(\lambda^*,\nu^*)= L(x^*,\lambda^*,\nu^*) with g defined as the infimum) at a differentiable L forces \nabla_x L(x^*,\lambda^*,\nu^*)=\mathbf 0 — stationarity. Both together with \lambda^*\ge0 (dual feasibility) and x^* feasible (primal feasibility) are exactly the KKT conditions, and they hold at every primal optimum whenever Slater's condition holds — necessity, not just sufficiency.

The algorithm families

A term added to the objective that charges an increasing cost for breaking a constraint.

Simplest to set up; conditioning worsens as the penalty weight grows toward infinity.

A penalty function combined with a multiplier estimate, letting the penalty weight stay fixed instead of growing without bound.

Fixes penalty's conditioning problem by also tracking a multiplier estimate.

A term added to the objective that grows without bound as a point approaches a constraint boundary from inside.

Blows up at the feasible region's boundary, keeping every iterate strictly interior.

The curve traced by a barrier method's minimizer as its weight shrinks toward zero.
A method that follows the central path using one cheap Newton step per parameter value, instead of fully re-solving each barrier subproblem.

Follows the central path with a primal-dual update — the choice for large-scale problems with many constraints.

An algorithm that trusts a local quadratic model's advice only within a shrinking or growing radius around the current point.

Keeps a local quadratic model honest by capping how far one step is allowed to trust it.

The decision chain

Graph settings

Calculator