Nonlinear Programming

Nonlinear programming (NLP) is the subfield of optimization theory concerned with minimizing (or maximizing) an objective function subject to constraints, where either the objective or at least one constraint is a nonlinear function. Unlike linear programming, where the geometry of the feasible set is polyhedral and global optima are guaranteed by local optima, nonlinear programming admits curved, non-convex landscapes with multiple local minima, saddle points, and regions of deceptive smoothness.

The field is unified by the KKT conditions, which provide first-order necessary conditions for optimality under appropriate constraint qualifications. These conditions reduce to simpler rules in special cases: in unconstrained optimization, they become the requirement that the gradient vanish; in equality-constrained problems, they become the classical Lagrange multiplier conditions. The Fritz John conditions provide a fallback when qualifications fail.

Nonlinear programming sits at the boundary between the tractable world of convex optimization — where every local minimum is global — and the wilderness of general non-convex optimization, which is NP-hard. Much of practical NLP research is devoted to identifying problem structures that restore tractability: sparsity, low-rankness, decomposability, or local convexity near the solution. Sequential quadratic programming, trust-region methods, and interior point approaches all navigate this boundary by solving sequences of simpler subproblems that approximate the true nonlinear landscape.

The applications are everywhere: engineering design, chemical process optimization, robotics trajectory planning, and the training of neural networks all reduce to nonlinear programs. The field's persistent challenge is that practitioners often do not know whether their problem is convex until they have already invested in solving it — a diagnostic gap that costs time, money, and occasionally safety.