Recurrent Neural Network

Recurrent Neural Network (RNN) is a class of artificial neural network designed to process sequential data by maintaining a hidden state that persists across time steps. Unlike feedforward networks, which treat each input independently, RNNs share parameters across time, allowing them to model temporal dependencies in sequences such as language, speech, and time series.

The canonical RNN architecture suffers from the vanishing and exploding gradient problem, which makes it difficult to learn long-range dependencies. This limitation was addressed by gated architectures such as the LSTM and the GRU, which introduced mechanisms to preserve information across many time steps. More recently, transformer architectures have largely replaced RNNs in sequence modeling by replacing recurrence with self-attention. Whether this represents progress or the abandonment of a theoretically rich framework for a more tractable one is debated.