Designing Outcome Indicators — Evaluation Thinking for Social Projects Beyond KPIs and KGIs

Introduction

"200 participants reached." "12 workshops delivered." Numbers like these fill the reports of social projects. They convey the scale of an activity—but did that activity change anyone's life?

Answering that question is the purpose of outcome indicators. In social programs run by nonprofits and government agencies, it is common for evaluations to conflate activity volume (outputs) with results (outcomes). Grant reports accumulate impressive figures, yet the actual effects of the program remain invisible. This guide lays out a thinking framework for indicator design.

---

Why KPIs and KGIs Fall Short

KPIs and KGIs were originally designed for corporate management. They assume variables an organization can directly control—revenue, profit, market share—and operate on short- to medium-term measurement cycles suited to business contexts where causal relationships are relatively clear.

Social projects do not meet these assumptions. Change among beneficiaries involves multiple interacting factors, and it is not uncommon for the effects of an intervention to take years to materialize. Setting a KGI such as "100 job placements" leaves the metric at the mercy of economic fluctuations and labor market conditions—external factors entirely beyond an organization's control.

The limitations of a KPI framework can be summarized in three points. First, it was designed for the corporate sector and presupposes "ownership" of results. Second, it carries an implicit assumption that outcomes are controllable. Third, its time horizon is too short to accommodate the lag inherent in social change. Indicator design begins with recognizing this structural mismatch.

---

The Three-Layer Model: Output / Outcome / Impact

The foundational framework for organizing the results of social programs is the three-layer structure of Output, Outcome, and Impact.

Output

Output

e.g. Trained 50 people

Short-term Outcome

Immediate

e.g. Changes in knowledge and awareness

Medium-term Outcome

Intermediate

e.g. Changes in behavior and habits

Long-term Outcome

Long-term

e.g. Changes in social conditions

Output

Output

e.g. Trained 50 people

Short-term Outcome

Immediate

e.g. Changes in knowledge and awareness

Medium-term Outcome

Intermediate

e.g. Changes in behavior and habits

Long-term Outcome

Long-term

e.g. Changes in social conditions

As one moves up the layers, the degree of organizational control decreases while societal significance increases. Outputs are easy to manage but carry less meaning; impact is profoundly meaningful but difficult to attribute to a single actor.

The layer that should occupy the center of evaluation in social projects is the outcome layer. It is the most practical arena for assessment—where organizational effort connects to beneficiary change. Impact serves as a long-term vision, while outcomes anchor the day-to-day evaluation cycle. This distinction is the key to making indicator design actionable on the ground.

---

The OECD DAC Six Criteria: An International Evaluation Framework

In the field of international development cooperation, the six evaluation criteria established by the OECD Development Assistance Committee (DAC) are widely used. They constitute a versatile framework applicable to domestic nonprofits and social programs as well.

1. Relevance — Does the program address the needs of its target population and society?
2. Coherence — Is it consistent with other policies and programs?
3. Effectiveness — Is it achieving its intended outcomes?
4. Efficiency — Are the results commensurate with the resources invested?
5. Impact — What are the broader effects on society and the environment?
6. Sustainability — Will the results endure after the support ends?

In 2019, "Coherence" was added to the previous five criteria—a shift signaling a move from evaluating individual programs in isolation to examining the coherence of entire ecosystems. In the evaluation of social projects, effectiveness and sustainability are the two axes most rigorously scrutinized. Even if short-term metrics look favorable, the program's design itself warrants reconsideration if its results vanish the moment support is withdrawn.

---

Outcome Indicator Examples by Field

Outcome indicators must be tailored to each field. Rather than abstract notions of "change," the following examples illustrate indicators translated into measurable form.

The "90-day retention rate" in employment support is a quintessential outcome indicator. It captures not job placements (an output) but post-placement retention—a change in the beneficiary's circumstances. The 90-day threshold is a practical benchmark aligned with the typical end of a probationary period.

Subjective indicators such as "community attachment" in community development are often assumed to resist quantification. However, when standardized questionnaires (e.g., community attachment scales) are employed, longitudinal comparison is entirely feasible. Qualitative indicators are not inherently unquantifiable; it is a matter of measurement design.

Under the Dormant Deposits Utilization Program (休眠預金等活用事業, JANPIA), grant application guidelines from 2021 onward require applicants to explicitly distinguish outcome indicators from output indicators. In grant-writing practice, this three-layer distinction has become a concrete operational requirement.

---

Design Checkpoints

The following questions serve as a checklist when designing outcome indicators.

Have you articulated your causal hypothesis? Put into words the logic of "why this activity will produce change in the target population." The most reliable method is to construct a logic model. An indicator without a hypothesis yields data that cannot be interpreted.

Have you established a baseline? Without pre-intervention data, "change" cannot be measured. Administer surveys and collect records at the start of the program. Realizing after the fact that "we should have measured that" is an extraordinarily common regret.

Are beneficiary voices treated as primary indicators? What service providers perceive as success and what beneficiaries experience as change are not the same thing. Position interviews and self-assessments by those directly affected not as supplementary indicators but as primary ones.

Have you distinguished attribution from contribution? Always ask: "Was this change caused solely by our intervention?" When attribution is difficult, describing the relationship in terms of contribution is the more intellectually honest approach. Stating that "this program contributed to the change" is an evaluation that acknowledges the presence of other factors.

Have you limited indicators to three to five? Too many indicators inflate measurement costs and burden frontline staff. The discipline lies in selecting the most essential outcomes. Attempting to measure everything can be tantamount to measuring nothing.

Have you paired quantitative and qualitative indicators? Numbers alone cannot explain why a result occurred. Always pair quantitative data with qualitative evidence—case studies, narratives, observational records. This paired structure strengthens the persuasiveness of evaluation reports.

Have you planned for post-program measurement? Changes observed immediately after support ends may not be sustained. Build follow-up measurements at three and six months into the design from the outset. Sustainability is, after all, one of the DAC evaluation criteria.

---

These checkpoints represent an extension of the SMART criteria (Specific, Measurable, Achievable, Relevant, Time-bound) adapted for social programs. SMART is a goal-setting framework, but in social projects the interpretation of "Achievable" must shift from "fully controllable" to "within the sphere of influence." Goal-setting that presupposes complete control over results does not function in a context where external factors loom large.

It is worth noting that Japan's Basic Policy on Economic and Fiscal Management and Reform 2025 (骨太方針2025) explicitly calls for the promotion of evidence-based policymaking (EBPM), positioning the combination of logic model verification and KPI monitoring as a standard methodology for policy evaluation. In collaborative projects with government agencies, outcome-oriented indicator design will be increasingly expected.

---

ISVD's Perspective

The Institute for Social Vision and Design (ISVD) views the evaluation of social projects not as "reporting numbers" but as a "learning cycle." Indicators exist not merely for measurement but to inform the next iteration of program design.

The very process of deliberating what to measure aligns team values and establishes a shared understanding of the intervention logic. Designing outcome indicators is, at its core, a dialogue that deepens an organization's grasp of its purpose and its beneficiaries.

Through SDI (Social Design Intelligence), you can visualize which layer of a social issue your initiative is addressing. Mapping the overall structure of the challenge before entering indicator design helps prevent drift in indicator selection.