Metamorphic testing

Metamorphic testing is a testing methodology that sidesteps the Oracle problem by checking relationships between outputs rather than comparing each output to a ground-truth oracle. The core insight is that even when the correct output for a given input is unknown, one often knows that certain transformations of the input should produce predictably related transformations of the output.\n\nFor example, if a program sorts a list, then reversing the sorted list and sorting it again should yield the original sorted list. If a sine function is correct, then sin(x) and sin(x + 2π) should be equal. These are metamorphic relations: properties of the program that must hold across multiple executions, even though no single execution has an independently verifiable expected result.\n\nMetamorphic testing transforms the verification problem from "is this output correct?" to "do these outputs cohere?" This is a profound shift: it replaces the impossible task of knowing the right answer with the feasible task of knowing what answers must have in common. The method has proven particularly powerful in machine learning, where the oracle problem is acute: one cannot know the "correct" classification of every possible image, but one can assert that rotating an image should not change its classification.\n\nThe limitation is that metamorphic relations are themselves specifications, and they can be incomplete or wrong. A program may satisfy every metamorphic relation its testers can imagine and still contain catastrophic bugs in untested dimensions. Metamorphic testing does not solve the oracle problem; it relocates it.\n\n\n\n\nSee also: Property-based testing, Differential testing