Recipe

Run the method of multipliers

Use when: plain penalty's growing ρ is making subproblems too stiff to solve reliably, and the constraint is (or can be turned into) an equality

Goal: a multiplier estimate that converges to the true one, at a fixed, modest ρ

Quick recipe

1
Add both a multiplier term and a quadratic penalty for each constraint: $L_A(x,\nu;\rho)=f(x)+\nu\,h(x)+\frac{\rho}2h(x)^2$ .

Example

For $h(x,y)=x+y-4$ : $L_A=x^2+y^2+\nu(x+y-4)+\frac12(x+y-4)^2$ at $\rho=1$ .

Why: the multiplier term alone would give plain Lagrange duality; the penalty term alone would give Section 2's method. Together, each covers the other's weak spot.
2
Choose $\nu_0$ (often $0$ ) and a fixed $\rho$ (often $1$ or $10$ ) that you will not need to keep raising.

Why: unlike plain penalty, ρ does not need to grow without bound here.
3
Minimize $L_A(x,\nu_k;\rho)$ over $x$ to get $x_k$ , warm-started from $x_{k-1}$ .

Why: this is an ordinary unconstrained subproblem, solvable with any Chapter 3 or 4 method.
4
Set $\nu_{k+1}=\nu_k+\rho\,h(x_k)$ .
- $h(x_k)$ is close enough to zero — stop; $x_k$ is your answer.
- it isn't — repeat the previous step with the new $\nu_{k+1}$ .
Why: this is a fixed-point step that walks the multiplier toward the true value, using how far the last subproblem's answer missed feasibility.

Common mistakes

Forgetting to warm-start $x_k$ from $x_{k-1}$ , which throws away the whole reason this converges fast.
Updating $\nu$ using the WRONG sign of $h(x_k)$ , which walks the multiplier away from $\nu^*$ instead of toward it.
Treating $\rho$ as something that still needs to grow — that is plain penalty's job, not this method's.