Decision tree

A decision tree is a supervised learning model that recursively partitions a dataset into subsets based on feature values, producing a tree-like structure of decision rules. Each internal node represents a test on a feature, each branch represents an outcome of that test, and each leaf node represents a predicted class or value. Decision trees are intuitive, interpretable, and form the building blocks of more powerful ensemble methods such as random forests and gradient boosting, but they are notoriously unstable: a small change in the training data can produce a radically different tree structure.

The instability of single trees is not merely an inconvenience — it is the fundamental reason ensemble methods work. Because trees are high-variance, low-bias learners, averaging many trees trained on different data subsets dramatically reduces variance without sacrificing the expressiveness of the individual model. The decision tree is therefore best understood not as a standalone classifier but as a modular component whose fragility is strategically exploited by ensemble architectures.