Theories of Change: Making Programs Accountable and Making Sense of Program Accountability

Executive Summary

by Stacy R. Aronson, Sue E. Mutchler, and Diane T. Pan

Making Programs Accountable

This paper illustrates a program planning and evaluation process that combines results-based accountability with a theory-driven approach to program design, implementation, and evaluation. This two-sided approach can assist planners, policymakers, funding agencies, and the public as they develop comprehensive education initiatives and other human service programs for children and families. The process can also guide these stakeholders in developing and evaluating programs that are responsive to unique local conditions.

Using Results-Based Accountability and Theories of Change

Publicly funded programs for children and families are undergoing transformation. In recent years two major trends have contributed to this transformation: (1) a demand for evidence of program results and, (2) a trend toward comprehensive community-based programs. Program planners, in response to these trends, must find ways to shift from accountability through monitoring of inputs to accountability through outcome measures. They must also work effectively with local stakeholders as responsibility for program design moves from state and federal agencies to local communities.

Two planning and evaluation approaches, working in tandem, can help planners establish a responsive and inclusive design process as well as effective implementation and accountability systems. The first approach, results-based accountability, uses intended outcomes as the means by which programmatic decisions remain focused, coherent, and appropriate. The process helps planners document program results and also creates a system through which they can use results information to guide program decisions over time.

Linking the second approach, theories of change, to results-based accountability further refines the planning and evaluation process. All public programs—whether small, local efforts or statewide initiatives—are based on theories, often unstated and unrecognized, about what children and families need and which program strategies will enable them to meet these needs. Identification of these theories creates the foundation upon which a results-based accountability system can be built.

The following six steps show how planners might use theories of change to enhance program accountability. The process guides stakeholders’ decisions from initial planning and selection of program strategies through implementation and evaluation.

• Step 1: Setting the course

• Step 2: Designing the initiative

• Step 3: Negotiating program plans

• Step 4: Articulating actions towards results

• Step 5: Implementing the plan

• Step 6: Evaluating progress toward goals

This paper explores the application of theories of change to results-based accountability. It illustrates the process with an example in which diverse stakeholders in a community use these six steps to work collaboratively towards a common goal of improving educational achievement.

Making Sense of Accountability

Infusing theory-based decision making into a results-based accountability process offers a useful framework that can guide the design, implementation, and evaluation of complex, publicly supported initiatives—initiatives that support education, children, and families. The benefits and opportunities of this approach are multiple:

• It allows stakeholders to focus on continuous improvement of creative programs that fit the goals and needs of local communities.

• It provides the opportunity for diverse stakeholders to develop a shared vision of which goals are important and a method for achieving those goals regardless of existing political, economic, and service delivery structures.

• It helps stakeholders negotiate a coherent and feasible plan that can ensure the final initiative represents feasible and defensible uses of resources.

A theory-based approach has particular benefits for program evaluation. The framework provided by theories of change can help stakeholders structure evaluation design and guide data collection, analysis, and interpretation while revealing gaps in knowledge. Articulating theories of change and using them to guide decisions can help reinforce the broader goals of comprehensive community-based initiatives for children.

Combining theories of change with results-based accountability is not without challenges:

• The theories of change approach to results-based accountability is not a panacea. The process alone cannot ensure that stakeholders will identify needed and missing elements in the programs they plan.

• Strong demand for positive client outcomes may tempt program implementers and policymakers to avoid difficult cases or to focus on results that show measurable gains, thereby neglecting results that are equally important but harder to document.

• The collaborative effort inherent in the process can create ambiguous roles and responsibilities among stakeholders and cause conflict over shared resources.

• Policymakers, funders, and the public may expect short-term positive results even though it takes time for programs to show significant improvement. Planners must clearly communicate the timeline of program impacts during the design phase.

• The focus on results may tempt stakeholders to abandon all input and process regulations. Certain procedural protections and assistance can serve as safeguards against fraud, abuse, poor services, and inequities or discrimination.

Conclusions

Merging the theories of change approach with results-based accountability can help communities and states succeed in several ways. It can encourage people—government officials and the public, educators and parents, service providers and service recipients—to work together in a systematic manner and to base the design and implementation of their comprehensive, community-based programs on well-considered research and experiential knowledge. The approach encourages planners to make theory-based decisions at every step and to use those decisions to determine more creative and diverse solutions for addressing the needs of children and families. The use of theories to inform evaluation and future activities might improve the likelihood that comprehensive, education-related community initiatives will produce better results. And consideration of results in light of a program’s underlying theories might give policymakers a stronger basis for increasing or shifting policy and resource support to publicly funded programs.

View this item in the SEDL Catalog

This publication is based on work sponsored wholly, or in part, by the Office of Educational Research and Improvement, U.S. Department of Education, under Contract #RJ96006801. The content of this publication does not necessarily reflect the views of OERI, the Department, or any other agency of the U.S. Government. SEDL is an Equal Opportunity Affirmative Action employer.