In the complex ecosystem of contemporary public management, we have developed a sort of addiction to data collection. We accumulate metrics like someone hoarding relics, deploy massive surveys, and dive into the depths of Big Data under a seductive yet incomplete premise: that information, by sheer volume, will dictate the path forward.
However, there is a critical chasm between data and decision-making. It is the ability to interpret what that data actually means, transforming it into information that can be understood in terms of value. This missing piece is Evaluative Reasoning (ER). Without it, we fall into what Michael Quinn Patton describes as a “manic-methodological obsession”: the implementation of impeccable collection methods, but with an absent logic to conclude whether something is good, fair, or valuable.
It is necessary to understand that ER is the connective tissue that transforms facts into wisdom. Based on this, let’s explore five lessons to stop merely measuring and start judging with rigor.
1. Evaluation as an “Alpha Discipline”: From models to theories
To elevate our practice, it is vital to distinguish between Evaluation Theory (with a capital T) and evaluation models (with a lowercase m). While models are metaphors or procedures, Theory implies a prescriptive logic that validates conclusions. In this sense, Michael Scriven elevates evaluation to the status of an “alpha discipline.” It is not a mere technical accessory but a full (transdisciplinary) discipline capable of examining and judging all others. ER is the engine of this challenge which, as Fournier pointed out in 1995, remains our final frontier:
“Professional evaluation has devoted much of its time and effort to methodological sophistication, and less to logical sophistication. Understanding the reasoning process used to reach the evaluative conclusions drawn in practice must be the field’s greatest unfulfilled challenge.”
2. The collapse of the wall between “Facts” and “Values”
Old tradition taught us that facts are objective and values are subjective (some retro-endemic voices even defend the contradiction that the subjective constitutes facts). ER dismantles this false dichotomy. Following Hilary Putnam, we understand that both are “entangled.”
Consider House and Howe’s Fact-Value Continuum. At one end, we find brute facts (an elephant is larger than a mouse) and at the other, values. Public policy lives in the center. For example, in Team Oracle’s victory in the 2013 America’s Cup: the “fact” is who won, but the “evaluative conclusion” as to whether it was a victory of seamanship or computer technology is a judgment where fact and value are inseparable.
- Traditional View: “Neutrality” is the dogma and values contaminate.
- ER View: Choosing what to measure is already a value judgment, and objectivity lies in making the reasoning transparent and traceable.
3. The leap of Abductive Logic: From “Is” to “Ought”
How do we solve Hume’s dilemma? How do we move from what is to what ought to be? ER achieves this through Abductive Logic. Unlike deduction (rules) or induction (generalization), abduction is the “mental leap” triggered by a “surprising” observation that the data alone cannot explain. This leads us to prima facie conclusions: defensible judgments sufficient for action in the real world. ER does not seek the sterile certainty of the laboratory, but a solid and human foundation for public decision-making.
4. Experienced Quality
It is useful to distinguish between quality as a measure of something and quality as an experience of that thing. In this sense, the evaluator acts as a connoisseur. Elliot Eisner illustrated this with a footwear expert who does not just measure the foot, but knows how to “notice” the subtlety of a stitch. In the public sector, noticing is a professional skill that involves using expert intuition to identify details that standard metrics ignore, valuing practical knowledge to make sense of what the citizen actually experiences.
For evaluation, strengthening evaluative reasoning is not an academic luxury, but an ethical obligation. We must move from “manual-based” evaluation—limited to checking compliance boxes—toward a “craft-based” evaluation, which requires judgment, wisdom, and total transparency in the chain of inferences. In this transition, ER is our shield against arbitrariness. At the end of the day, the question for your team is simple yet fundamental:
Is your organization simply measuring performance, or is it equipped to judge value?