Right to Explanation

The right to explanation is the legal and normative entitlement of a person to receive a meaningful account of why an algorithmic governance system made a particular decision about them — why a loan was denied, why content was removed, why a feed was curated in a particular way. The right is rooted in the broader principle of procedural justice and has emerged as a central demand in the governance of algorithmic power: if decisions are made by systems that are opaque, then those subject to the decisions must at least be able to understand what logic was applied to them.

The right to explanation appears most prominently in the European Union's General Data Protection Regulation (GDPR), which grants individuals the right not to be subject to decisions based solely on automated processing that produce legal or similarly significant effects. Article 22 of the GDPR establishes a right to meaningful information about the logic involved, though the exact scope of this right — whether it requires full model disclosure, summary statistics, or merely an indication that an algorithm was used — remains contested. The ambiguity is not accidental; it reflects a deep structural tension between the legal system's demand for accountability and the technical system's capacity for explanation.

The Structural Problem of Explanation

The right to explanation assumes that explanation is possible — that the algorithm's decision can be translated into terms that a human subject can understand without requiring that the subject become a machine learning engineer. This assumption is not always valid. The most powerful machine learning systems — deep neural networks, ensemble models, and reinforcement learning agents — produce outputs that are not readily decomposable into a set of human-understandable reasons. The question is not whether the system can produce an explanation; it is whether the explanation produced is the right kind of explanation.

There are two competing models of what counts as an explanation. The local explanation model produces a post-hoc account of why a particular decision was made for a particular individual: a list of features that contributed to the score, a visual heatmap, a counterfactual showing what would have changed the outcome. This is the model that most algorithmic governance systems attempt to provide. The global explanation model demands an account of the system's overall logic: what categories of people it favors, what assumptions it encodes, what trade-offs it makes. This is what a subject of algorithmic power needs to know if they are to resist that power — not why they were denied the loan, but why the system is designed to deny loans to people like them.

The right to explanation as currently implemented typically delivers local explanations. This is not an oversight; it is a structural feature of the interaction between algorithmic governance and legal systems. The legal system processes individual grievances, and the explanation it demands is an individual explanation. The structural critique — that the system itself is biased, that the model encodes unjust social categories, that the optimization objective serves the operator rather than the governed — is not a grievance that the right to explanation is designed to address. The right to explanation is therefore a form of structural coupling between algorithmic systems and legal systems, but it is a coupling that processes the individual at the expense of the structural.

The Limits of Explanation as Governance

The deeper critique of the right to explanation is that explanation is not governance. Knowing why a decision was made does not change the decision, and knowing that a system is biased does not provide the means to correct it. The algorithmic audit and the algorithmic impact assessment are attempts to go beyond explanation — to create mechanisms that can evaluate and constrain algorithmic systems before or during their deployment. But these mechanisms face their own structural problems: audits require access to data and models that operators are not required to disclose, and impact assessments require a capacity to predict outcomes that the systems themselves do not possess.

The right to explanation is therefore best understood as a necessary but insufficient condition for the governance of algorithmic power. It is necessary because without explanation, the governed cannot even identify when they are being harmed by an algorithmic system. It is insufficient because explanation alone does not create the capacity to resist, to appeal, or to redesign. The right to explanation makes the exercise of algorithmic power visible; it does not make it accountable. Visibility without accountability is surveillance; the right to explanation risks becoming the very thing it was designed to resist.

The right to explanation is not a right to understand the algorithm. It is a right to understand what the algorithm did to you. This distinction is not merely semantic — it is the difference between a tool that empowers subjects and a tool that pacifies them. An explanation that says "you were denied because your credit score was 580" does not explain why the scoring system uses that threshold; it merely makes the system's judgment appear rational and individual. The right to explanation, as currently conceived, is less a check on algorithmic power than a mechanism for its legitimation. Until the right to explanation includes a right to challenge the system's design — not just its outputs — it will remain a consolation prize in a game that is rigged from the start.