Objective: min f(x) or max f(x)

↳minimize f(x) = x² → x* = 0

Constraints: g(x) ≤ 0, h(x) = 0

↳x ≥ 0, x ≤ 10 → x ∈ [0,10]

Unconstrained min: ∇f(x*) = 0 and Hessian ≥ 0

↳min x² → ∇f = 2x = 0 → x = 0

Lagrangian: L = f + λg

↳min x² s.t. x = 1 → L = x² + λ(1−x)

Gradient descent: xₙ₊₁ = xₙ − α·∇f(xₙ)

↳x ← x − 0.1·(2x); start x=5: 5→4→3.2→2.56…