Introduction to EBPM — What Evidence-Based Policymaking Asks of Nonprofits

Introduction

Governments are placing growing emphasis on data-driven evidence as the basis for budget allocation decisions. The era when "passion and a track record of dedication" alone could secure public funding for nonprofits is fading. What is now being asked is the ability to articulate — through logic and numbers — why a given program produces its intended effects.

It is in this context that EBPM (Evidence-Based Policy Making) has come to the fore.

---

What Is EBPM?

EBPM refers to the practice of clarifying policy objectives and grounding the planning and design of policies in rational evidence. The Cabinet Secretariat (内閣官房) of Japan has adopted a similar definition, and the concept has become standard within the country's administrative reform discourse.

The origins of EBPM can be traced to EBM (Evidence-Based Medicine) in the healthcare sector. During the 1990s, a movement to shift from "medicine based on experience and intuition" to "medicine grounded in research evidence" gained momentum and gradually extended into the domain of policymaking. In the United Kingdom, the Blair government from 1997 onward actively promoted EBPM. The What Works Centres were established in 2013, creating an infrastructure for aggregating and publishing the best available evidence across fields such as education, criminal justice, and social welfare.

In the United States, the Foundations for Evidence-Based Policymaking Act was enacted in 2016, mandating the appointment of Chief Data Officers and Evaluation Officers across federal agencies. These Anglo-American precedents have driven policymaking reforms worldwide.

---

The Current State of EBPM in Japan

The turning point for EBPM adoption in Japan came with the 2017 Basic Policy on Economic and Fiscal Management and Reform (経済財政運営と改革の基本方針, commonly known as "Honebuto no Houshin"). The promotion of EBPM was formally articulated and, in coordination with the final report of the Advisory Council on Statistical Reform, a cross-ministerial framework was established.

Today, the Administrative Reform Promotion Council serves as the central body driving EBPM implementation, and each ministry has established its own "EBPM Promotion Committee." Furthermore, in December 2024, the "EBPM Action Plan 2024" was adopted by the Council on Economic and Fiscal Policy. The principle of "pursuing wise spending (ワイズスペンディング) based on objective data in order to generate high policy impact from limited resources" is now transitioning into the operational phase.

Ministry-level case studies continue to accumulate. The Ministry of Health, Labour and Welfare (MHLW) conducted an impact evaluation of a special depreciation system for medical devices using difference-in-differences analysis. The Ministry of Education, Culture, Sports, Science and Technology (MEXT) quantitatively assessed the effect of overseas dispatch experience on teachers' professional development. At the local government level, the city of Sendai has advanced the use of data to visualize challenges in emergency medical services. Practical applications are steadily expanding.

---

The Hierarchy of Evidence

In EBPM, not all evidence carries equal weight. There is a well-defined hierarchy determined by the rigor of the research design.

1Meta-analysis / Systematic Review

2Randomized Controlled Trial (RCT)

3Quasi-experiment / Cohort Study

4Pre-post Comparison / Case Study

5Expert Opinion / Activity Report

High internal validityHigh ease of implementation

The gold standard is the RCT (Randomized Controlled Trial). By randomly assigning participants to intervention and control groups, it eliminates the influence of confounding factors and measures the "pure effect" of a program. The underlying logic is straightforward and easy to communicate.

However, RCTs have their limitations. There are cases where they are ethically infeasible, situations where costs and time constraints are prohibitive, and issues of external validity — where results from one context may not generalize to another. Given these constraints, RCTs cannot be applied in every policy evaluation scenario. Quasi-experimental methods and observational studies, when employed appropriately, can also yield sufficiently robust evidence. What matters most is the awareness of where one's evidence stands within this hierarchy and the integrity to disclose that positioning transparently.

---

Why Nonprofits Should Engage with EBPM

The Terms of Government Partnership Are Shifting

As the government's commitment to EBPM intensifies, the criteria by which it selects nonprofits for commissioned projects and grants have also evolved. Beyond "having a track record," the ability to "measure and report program effectiveness" is increasingly being assessed. Whether an organization can speak the language of evidence is opening a widening gap in the opportunities it can access.

Outcomes-Based Funding Is Expanding

Outcomes-based contracts, exemplified by Social Impact Bonds (SIBs), define performance indicators in advance and determine compensation based on the degree to which those targets are met. The ability to measure program effectiveness is a prerequisite. For nonprofits to leverage such funding mechanisms, EBPM-oriented thinking — the capacity to define, measure, and report outcomes — becomes foundational.

As a Matter of Public Accountability

Nonprofits operate under a public-interest mandate. They bear a responsibility to donors, supporters, and beneficiaries to demonstrate that their activities genuinely create social value. The shift from "we have a sense that things are working" to "this data substantiates that claim" is essential. Possessing evidence strengthens the foundation of organizational trust.

---

A Practical Starting Point — Connecting to the Logic Model

EBPM is not about mastering advanced statistical techniques. The first actionable step is to organize the logic underlying one's own activities.

An effective starting point for this is the logic model. This tool visualizes the causal chain of "inputs → activities → outputs → outcomes → impact" and provides the structural backbone for answering the kinds of questions EBPM demands.

The EBPM process can be outlined in the following stages:

1. Problem definition (which social issue to address)
2. Hypothesis formulation (why this intervention is expected to produce an effect)
3. Data collection design (what to measure, when, and how)
4. Analysis and interpretation (to what extent were outcomes achieved, and what external factors were at play)
5. Feedback into policy and programs (applying findings to the next cycle)

The logic model establishes the foundation for Steps 1 and 2 and guides the measurement design in Step 3. For nonprofits already engaged in programming, the simple act of articulating a logic model will naturally surface the outcome indicators that should be measured.

Sophisticated statistical analysis comes later. The first step is to structure the logic of one's own activities. That is the shortest path to EBPM.

---

Steps for Nonprofits to Begin Practicing EBPM

The concepts are understood. The question is: where to start? Here are practical steps that even small organizations can begin today.

Map Your Program Logic on a Single Page

For your organization's primary program, create a logic model on a single page. A sheet of paper or a spreadsheet will suffice. Draw the flow from "inputs → activities → outputs → outcomes → impact" with arrows connecting each element. This exercise will naturally reveal the outcomes that need to be measured. For detailed guidance on building a logic model, see What Is a Logic Model?.

Select One or Two Outcome Indicators

From the "outcomes" identified in the previous step, choose the one or two most important and convert them into measurable indicators. Rather than "participants' behavior changed," specify something like "the proportion of participants who maintain employment three months after program completion." Be specific about timing and measurement method. For detailed guidance on indicator design, see Designing Outcome Indicators.

Establish a Baseline

Record data before your program begins. Post-program surveys alone cannot demonstrate "change." The minimum baseline requirement is simply measuring participants' initial status via a Google Form and accumulating the data in a spreadsheet. For specific low-cost implementation examples, see Introduction to Data Utilization for Nonprofits.

Test on a Small Scale and Use the Findings in Reporting

Conduct follow-up measurements after three to six months and compare them against the baseline. The ability to write a single sentence such as "X% of participants showed improvement in their scores" dramatically enhances the persuasiveness of grant reports and supporter newsletters. Even without a rigorous RCT, accumulating small pieces of evidence through pre-post comparisons constitutes the first step in EBPM practice.

---

ISVD's Perspective

The spread of EBPM is not a threat to nonprofits and social enterprises — it is an opportunity. Organizations that can articulate their impact through data command greater influence in society.

That said, not every organization can immediately build quantitative evidence. The first priority is to clarify the vision of "what kind of change are we trying to bring about?" That is the entry point to EBPM, and it is also the foundation of social design.

The Institute for Social Vision and Design (ISVD) offers the SDI Assessment, which helps individuals and organizations working on social issues to articulate and structure their vision. Visualizing where your social vision currently stands, then building evidence from that foundation — this sequence, though it may seem like a detour, is in fact the most reliable path forward.