Particle swarm optimization

Particle swarm optimization (PSO) is a computational method for finding optimal solutions to complex problems by simulating the social behavior of organisms, most notably bird flocks and fish schools. Developed by James Kennedy and Russell Eberhart in 1995, PSO represents candidate solutions as particles moving through a multidimensional search space. Each particle adjusts its trajectory based on its own best-known position and the best-known position discovered by any particle in its neighborhood. The result is a distributed search process that balances exploration (searching new regions) with exploitation (refining promising regions).

Unlike gradient-based optimization methods, PSO requires no differentiable objective function and makes no assumptions about the problem's mathematical structure. This makes it applicable to discontinuous, noisy, and non-convex optimization problems where traditional methods fail. The algorithm's simplicity is deceptive: a typical implementation requires only a few lines of code, yet PSO has been successfully applied to function optimization, neural network training, control system design, and feature selection.

The connection to swarm intelligence is direct. PSO literalizes the biological metaphor: the swarm is not merely an analogy for the search process; it is the search process. The intelligence that finds the optimum is distributed across particles, none of which possesses a model of the global landscape. This architectural choice trades the rapid convergence of centralized methods for robustness to local optima and the ability to search spaces with irregular topology.

However, PSO has known limitations. It struggles with high-dimensional problems, where the volume of the search space grows exponentially and particle density becomes insufficient. It lacks a mechanism for maintaining diversity once the swarm converges, leading to premature convergence on suboptimal solutions. And like all metaheuristic methods, it provides no guarantee of optimality and no certificate of solution quality. PSO finds good solutions quickly; it does not find the best solution provably.

Particle swarm optimization is what happens when you take the boids model seriously as a computational paradigm rather than merely a graphics trick. The particles are not birds; they are hypotheses, moving through hypothesis space, attracted to their own past successes and the swarm's collective memory. The question is not whether this works — it demonstrably does — but whether nature arrived at the same architecture through convergent evolution, and what that convergence tells us about the fundamental structure of search problems.