Literature Map: The Genealogy of EBPM — Evidence-Based Policy Making and ISVD's Data-Driven Approach

This note is part of the literature map series of the Social Design Lab (ISVD-LAB-003). It traces the intellectual genealogy of Evidence-Based Policy Making (EBPM) and clarifies where ISVD's data-driven approach is positioned within it.

EBPM (Evidence-Based Policy Making) has become the dominant paradigm in 21st-century public policy. The proposition that "policy should be designed and evaluated on the basis of objective evidence, not assumptions or precedent" appears, on its face, irrefutable. Yet the questions of "what counts as evidence" and "who produces that evidence, and whose questions does it answer" have scarcely been posed by EBPM's own advocates.

ISVD places data-driven social analysis at the core of its methodology. Automated retrieval of government statistics via the e-Stat API, construction of statistical dashboards, data-informed column writing — these appear on the surface to resonate with EBPM. But ISVD's question is not "Did our policy work?" but rather "What is the true shape of this problem?" To clarify this difference, it is necessary to trace the intellectual genealogy of EBPM through the literature.

The Birth of the Evidence Hierarchy: From EBM to EBP

The intellectual source of EBPM lies in medicine, tracing back to Archie Cochrane. In his 1972 book Effectiveness and Efficiency, Cochrane argued that medical interventions should be validated through randomized controlled trials (RCTs). This argument was systematized in the 1990s as "evidence-based medicine (EBM)" by David Sackett and colleagues (Sackett et al., 1996). At the heart of EBM lies the "evidence hierarchy" — a pyramid placing meta-analyses and RCTs at the apex, with expert opinion at the base.

This medical methodology crossed over into public policy in late-1990s Britain. In 1999, the Blair government published the white paper Modernising Government, declaring "evidence-based policy making" as an official government principle. NICE (established 1999) led the institutional use of evidence in healthcare policy, while the Campbell Collaboration (established 2000) extended evidence synthesis into social policy, education, and criminal justice.

The Rise of Nudge and Behavioral Economics in Policy

The second wave of EBPM arrived as the policy application of behavioral economics. Thaler & Sunstein's Nudge (2008) proposed policy design premised on human cognitive biases — choice architecture.

In 2010, the UK established the Behavioural Insights Team (BIT) — colloquially the "Nudge Unit" — within the Cabinet Office, introducing the EAST framework (Easy, Attractive, Social, Timely) into policy design. Nudge became coupled with a cost-effectiveness narrative of "small interventions, large behavioral change," making it an attractive tool for governments under austerity.

The RCT Revolution and Development Economics

The third wave of EBPM was the "RCT revolution" in development economics. Abhijit Banerjee, Esther Duflo, and Michael Kremer established the methodology of validating poverty reduction interventions through RCTs, winning the Nobel Prize in Economics in 2019. J-PAL (the Abdul Latif Jameel Poverty Action Lab), which they co-founded at MIT, has conducted and supported over 1,000 RCTs.

The RCT revolution generated a dynamic that narrowed "evidence-based policy" into "RCT-based policy." Challenging this trend, Deaton & Cartwright (2018) argued that "RCTs alone cannot deliver understanding of causal mechanisms," calling for a reconsideration of the evidence hierarchy.

EBPM in Japan: Institutionalization and Its Challenges

The institutionalization of EBPM in Japan began in earnest in 2017. The "Final Report of the Statistical Reform Promotion Council" explicitly stated the promotion of EBPM as a government policy, and an EBPM Promotion Committee was established within the Cabinet Secretariat. RIETI opened its EBPM Center in 2022, initiating methodological research on policy evaluation and practical support.

The opening of government statistics also progressed. e-Stat expanded its API provision as the comprehensive portal for government statistics, and RESAS (Regional Economy and Society Analyzing System) was developed as a data platform to support local government policy making.

However, Japan's EBPM faces several structural problems. First, the Monthly Labour Survey falsification scandal that came to light in 2018–19 shook the very reliability of the government statistics that form the foundation of evidence. Second, the proliferation of "pseudo-EBPM" has been widely noted — the practice of selectively presenting convenient data under the banner of "EBPM" to justify budgets while ignoring inconvenient evidence.

Criticism of EBPM: The Relationship Between Politics and Knowledge

Academic criticism of EBPM goes beyond methodological debate. In The Politics of Evidence-Based Policy Making, Cairney (2016) argued that "policy making is inherently a political process, and evidence is selectively utilized within it." The linear model of "evidence → policy" that scientists assume rarely holds in real policy processes.

Deaton & Cartwright (2018) went further, pointing out that the high "internal validity" of RCTs does not guarantee "external validity" — applicability to different contexts. An intervention that proved effective in one region may not produce the same results under different institutional, cultural, and social structures.

What these critiques share is a fundamental question: "Evidence for whom, and for what purpose?" And this question connects directly to ISVD's problem formulation.

The EBPM Paradigm and How ISVD Differs

The basic question of EBPM is "Did our policy work?" Several assumptions are embedded in this question's structure.

1. The problem is already defined: EBPM presupposes that there is no dispute about what the problem is. But in reality, the definition of "what the problem is" is itself politically constituted.
2. Evidence is neutral: Data is assumed to reflect objective facts, but the choice of what to measure and what not to measure already embodies value judgments.
3. Policy exists to solve problems: Yet, as Cairney points out, policy also serves purposes beyond problem-solving — budget acquisition, political legitimation, organizational maintenance.

ISVD's question is "What is the true shape of this problem?" This question begins by interrogating the definition of the problem itself.

ISVD's data-driven approach shares tools with EBPM on the surface — statistical data retrieval, quantitative analysis, visualization. But the purposes differ. In EBPM, data is "an instrument for measuring policy effects," whereas in ISVD, data is "a clue for bringing invisible structures to the surface." This difference arises from the agnotological perspective, one of the six intellectual sources described in "The Intellectual Coordinates of Social Design".

Structural Revelation vs. Budget Justification

As EBPM becomes institutionally established, "pseudo-EBPM" frequently emerges. This is the practice of selectively using evidence to justify policy, and Japan's Monthly Labour Survey falsification represents an extreme case.

ISVD's approach occupies the opposite pole from this "pseudo-EBPM." ISVD holds no position in favor of promoting any particular policy. Its purpose is structural revelation — depicting, as accurately as possible, what shape society actually takes. Here, ISVD's data-driven approach draws closer not to EBPM's policy evaluation but to the methodological stance of post-normal science. In situations characterized by high uncertainty, conflicting interests, and urgent decision-making, the posture is not to present "the right answer" but to make "the structure" visible.

What the EBPM Genealogy Means for ISVD

There are three reasons why ISVD should study the genealogy of EBPM.

First, methodological refinement. Evidence synthesis methods such as RCTs, systematic reviews, and meta-analyses serve as methodological reference points for ISVD's analysis of social structures. The evidence hierarchy debate provides a framework for judging "which data can be trusted, and to what extent."

Second, learning from institutional failure. Cases of pseudo-EBPM and statistical falsification in Japan's EBPM promotion illustrate the traps that data-handling organizations can fall into. ISVD's emphasis on open access and independence is, in part, a response to these institutional failures.

Third, interlocutors for critical dialogue. The critiques of Cairney, Deaton & Cartwright, and others resonate with ISVD's problem formulation — "Evidence for whom?" and "What is not being measured?" The questions that EBPM's critics raise from within the policy process, ISVD re-poses from the side of civil society.