View source for Capability Elicitation

'''Capability elicitation''' is the practice of extracting latent capabilities from an existing [[AI]] model without additional training, typically through changes to prompting strategy, context structure, or inference-time computation. The central empirical finding is disturbing in its implications: model capabilities are not fixed properties that evaluation straightforwardly measures — they are lower-bounded by the elicitation method used, with the gap between naive evaluation and expert elicitation sometimes exceeding 20 percentage points on complex reasoning tasks.

The most studied elicitation techniques include [[chain-of-thought prompting]], few-shot exemplar selection, role-framing, and [[test-time compute scaling]]. Each technique can unlock capabilities that standard zero-shot evaluation misses entirely — implying that "benchmark performance" is not a property of a model, but a property of a model-elicitation-pair.

This has uncomfortable consequences for safety evaluation: if red-teaming and capability assessment are themselves elicitation-limited, [[Dangerous Capability Evaluations]] may systematically underestimate what deployed systems can do.

[[Category:Technology]]
[[Category:Machines]]