Creating New Governance Structures
(Fall 1994 Networkshop)
Ensuring Accountability
Concluding the Networkshop presentations, Judith Chynoweth and Rachel Lodge framed their discussion of performance accountability as answers to eight key accountability questions.
- What is performance accountability? First, it is a means of
judging policies and programs by measuring outcomes against agreed-upon
standards for indicators. Beyond that, it is a tool that organizes
information for effective use by all involved in a process. Simply
stated, an accountability system tries to answer the evaluative
question, "Did we accomplish our goal?" by asking two operational
questions: "What was done?" and "What were the results?" Unlike an
evaluation system, an accountability system makes no attempt to
attribute causes to outcomes.
California's Healthy Start initiative is being monitored under a three-year long evaluation system which is tracking each site in two outcome areas: individual child and family outcomes and the collaboration process. According to Chynoweth, the two objectives of this evaluation embody some tension. One has a formative purpose in providing information to help Healthy Start site managers guide program development; the other has a summative purpose in providing results for policymakers to use in making decisions about resource allocation. Potentially, the same data that point to changes in a promising program could be used to decrease or eliminate funding for the program.
- What outcomes are measured? Solid research should be the basis for
developing a list of outcomes. Chynoweth recommended an important
resource for identifying outcomes and indicators of significance in
education, health, and human services: the Improved Outcomes for
Children Project--a collaborative effort of the Center for the Study of
Social Policy, the Harvard Project on Effective Services, the National
Alliance for Restructuring Education, and the National Center for
Education and the Economy.
- Why have an outcomes-based accountability focus? Such focus has a
number of benefits:
- It keeps state and local activity focused on a goal.
- It provides a basis for collaboration.
- It promotes "consistency of action" among service deliverers by balancing the desire to respond to clients with the need for alignment with a set of framed purposes.
- increases effectiveness.
- It raises morale through tracking success.
- It fosters commitment and encourages support from collaborative partners and funders by establishing service deliverers' credibility.
- It empowers a community by bringing external recognition and support.
- It can be "exchanged" for funding support.
- Who makes determinations about accountability measures? In
California, the legislature has taken the lead. Healthy Start
legislation established the following set of education, health, and
social outcome measures:
- School attendance
- Academic performance
- Dropout rates
- Pupil grades
- Post-secondary education/training
- Immunizations
- Birth weights
- Diagnostic screenings
- Out-of-home placement rates
- Child protective services referrals
- Family functioning
- Self-esteem
- Self-esteem
With subsequent additions, there are now approximately 20 state-established outcome measures. The process of establishing these measures has produced tension: state agencies and the Foundation Consortium have differed in their ideas about the purpose and necessary rigor of evaluations, while local Healthy Start collaborative efforts have established goals, outcomes, and measures appropriate to local needs. Lessons learned so far are these:
- The state should mandate that localities look at outcomes.
- Site personnel require information, training, and support in understanding what outcomes are and why they are regularly monitored.
- Site personnel may not always want to collect information the state seeks.
- In order for the accountability system to be meaningful, there must be significant local input into outcome selection.
- There will be an ongoing need for exchange of information between
the state and community about accountability purposes and
procedures.
- What are the criteria for choosing outcomes? Questions that must
be asked in establishing these criteria are:
- What outcomes does research reveal as important to track? How do service interventions affect those outcomes?
- What is measurable? What is hard, but nevertheless important, to measure?
- What few, key outcomes can be agreed upon?
- What outcomes relate to the agreed-upon vision and goals?
- What information is comfortable to collect at the local level?
- What is the information-gathering and reporting process? A major
reporting requirements review was in process in California in fall
1993. The Foundation Consortium is sponsoring an effort to rationalize
data collection systems through a project funded across agencies for
developing a common set of core data elements. Each element is to have
a standard definition, or translation, which will be incorporated into
new MIS systems for integrated services.
- How are outcomes interpreted and reported to the public? By the
end of its third year, a Healthy Start site must be funded through
locally-derived sources. Consequently, continued funding may depend on
effective interpretation and reporting of outcomes to potential
supporters. A promising reporting strategy is a community or county
report card which lists outcome indicators and ranks the performance of
the site in relation to desired standards.
- How are rewards and penalties assessed? Carefully! In designing a reward and penalty structure founded on performance, consideration must be given to the tension between an accountability-based system and funding equity. Furthermore, local officials, who are elected at least partly on the basis of how well they capture resources for their constituency, may be affected by performance-based rewards and penalties. As a result, difficulties can arise in obtaining agreement among politicians on whether to reduce local resource allocations or on how to do so.
