Meta-Learning

Meta-learning — learning to learn — is the problem of designing learning algorithms that improve their own learning process across tasks. Where transfer learning asks what knowledge transfers from one task to another, meta-learning asks what learning processes transfer: which inductive biases, optimization strategies, and architectural choices enable rapid adaptation to new problems. A meta-learner does not merely solve tasks; it discovers the algorithm that solves tasks.

The formal framework treats learning as an optimization problem over learning algorithms themselves. In gradient-based meta-learning — popularized by MAML and its descendants — the meta-learner optimizes initial parameters such that a small number of gradient steps on a new task produces good performance. The meta-objective is not task performance but adaptation speed: how quickly the learned initialization specializes. This reframes generalization not as finding a single good function but as finding a good starting point in function space.

Meta-learning connects to Bayesian optimization, neural architecture search, and the broader question of whether machine learning systems can automate their own design. The ultimate form of meta-learning would be a system that, given a new domain, selects its own architecture, loss function, and optimization strategy without human intervention. Such a system would have closed the outer loop of machine learning — but it would also face the recursive problem of how to meta-learn its own meta-learning process.

The relationship between meta-learning and fast weights — temporary synaptic modifications that enable rapid adaptation without overwriting long-term memory — illustrates how biological and artificial systems may converge on similar solutions to the adaptation problem.