Playing it Safe with School Safety Programs
Some school safety programs require additional investigation beforeimplementation. These strategies may help educators choose wisely.
A quick survey of the recent literature reveals thatschool safety is a subject of great interest to educators. Several importantpublications on school safety have appeared, including the Harvard EducationReview's special issue on youth and violence, Phi Delta Kappan's "SpecialReport on Youth Violence," and Educational Leadership's "Contemporary Issues:Violence in School."
Educators are not only reading about school safety, they are adopting schoolsafety programs. One indication of the adoption boom is the number of schoolsimplementing conflict resolution programs with peer mediation components.According to the National Association for Mediation in Education, an estimated5,000 peer mediation programs are in schools today, compared to 100 five yearsago.
Educators bringing safety programs into their schools can profit from firstapplying strategies to assess the value of these curricula. Thoughknowledgeable in their disciplines, many educators may need additionalresources to review and choose judiciously from among the highly specializedschool safety and violence prevention programs.
The truth is, studies reveal that some violence prevention programs promiseoutcomes that are unverified by research. "In fact," writes Marc Posner in TheHarvard Education Letter, "researchers are beginning to question whether themost commonly used school-based programs for violence prevention and conflictresolution actually do what they are supposed to do."
In their review, What Works in Reducing Adolescent Violence, Patrick Tolen andNancy Guerra of the University of Illinois-Chicago go one step further. Theywrite:
Programs that train peers to serve as mediators of disputes and train youth inconflict resolution skills have become increasingly popular since themid-1980s....However, despite the soaring popularity of this type ofintervention at the elementary school, middle school, and high school levels,and a number of laudatory "testimonials" from teachers and otherparticipants,...we could not locate a well-designed empirical study thatevaluated behavioral outcomes with adolescents. Although peer mediation has anintuitive appeal, particularly in terms of reducing situational andinterpersonal violence, its efficacy has simply not been determined (p. 34).
Yet school administrators can follow guidelines based on common sense whenselecting school safety programs. "Administrators should act as smartconsumers," advises Jack Lumbley, a SEDL evaluation associate who has assessedschool programs for more than 20 years. Lumbley recommends thateducators apply some of the strategies they use when weighing other purchasesand call upon the expertise of others while assessing such programs.
Consider the source of the program, Lumbley says. Traditional publisherstypically subject their materials to a rigorous testing sequence and releasethem only after appropriate outcomes have been demonstrated reliably. Incontrast, some antiviolence programs have been created by organizationsspecializing in conflict resolution rather than in curriculum design. Theseorganizations, many of which are nonprofit, may lack sufficient resources tosubject their curricula to structured field testing by the public schoolstudents and personnel who will ultimately use them. And, Lumbley notes,programs released without proper preparation may not deliver the results onemight expect.
Dan Kmitta agrees. He is a researcher investigating how schools areimplementing conflict resolution programs and how best to measure theeffectiveness of such programs. Kmitta is completing a three-year study ofconflict resolution programs in 14 Cincinnati, Ohio schools.
If the program producer is a nonprofit organization specializing in conflictresolution strategies, program evaluation is "generally done in-house" when itis done at all, Kmitta says. Such an internal evaluation is bound to be lessstringent than tests conducted by objective third-party observers;unintentional bias may skew results.
Some developers of conflict resolution programs counteract this potentialproblem by incorporating in their program a process evaluation component whereparticipants rate the content. "That produces valuable information forpractitioners," Kmitta says. "It does not give those of us in the field who arelooking at a program much information about its efficacy."
Information about program effectiveness may also appear in training manuals andother support literature. Even so, administrators should contact the programdeveloper and ask how and where the program was evaluated, Lumbley says. Findout how extensively the program has been tested and ask for written evidencesupporting the purported outcomes.
Questions one could ask when interviewing a program developerinclude:
- Did the evaluator have sufficient expertise in the subject area? Anexpert on peer mediation techniques, for instance, might not be a suitablereviewer of a program that teaches bias identification.
- Were the tests actually conducted in a school or in a test environment thatsimulated a school?
- If the program was tested in a school, what was the school like?Were conditions comparable to those in the administrator's own school? Programoutcomes at a sub-urban school may be very different from those at arural school or those at an inner city school.
- Who presented the program? The choice of trainers and theirknowledge of program materials can affect program outcomes dramatically. "The sameprogram targeting similar audiences may have an entirely different outcomeswhen presented by different implementers," Lumbley says.
Finally, check if the product's marketing literature contains publishedevidence of its effectiveness, Lumbley says. Good marketing literature shouldclarify the purpose of the program while providing evidence that the promisedoutcomes have been achieved.
Lumbley identifies another pitfall in program selection. Administrators mightassume that a program on a "recommended curriculum list" has already beenreviewed, particularly a program recommended by a state board of education. Thesame pitfall can occur with any blanket recommendation: a program that works inone school may fail or create problems in another.
The program should be designed to serve the specific student population in theschool. Crediting Johns Hopkins University researcher Daniel Webster with theinsight, Posner writes in The Harvard Education Letter, "...many conflictresolution programs teach the kinds of negotiation skills that may be usefulfor middle-class students whose disputes stem from competing interests, but notfor poor, high-risk youth for whom violent conflict is often the result ofmacho posturing and competition for status."
"For this reason," says Lumbley, "administrators selecting programs ought toask if they have been implemented elsewhere, and then should follow up on thosereferrals. Determine the similarities and differences between the referralschools and one's own. Ask how well students in the referral schools haveaccepted the program, what the program has done for the schools, and whatdifficulties occurred during program implementation.
"And," Lumbley advises, "go beyond the schools mentioned in the referrals tothe local courts, the juvenile justice system, and social service agencies. Askthe professionals in these institutions for their perspectives on the programs.Specialists in state education agencies who track curricula may also be asource of reliable program appraisals."
School violence is systemic and stretches beyond the schoolyard. That's all themore reason to contact other institutions, Lumbley says. With such alliancesand a proven, appropriate program, educators can better assure that studentsare safe.
For more information, and to order copies of the sources cited in this article,contact the publishers.
Posner, Marc (1994, May/June). "Research Raises Troubling Questions aboutViolence Prevention Programs." The Harvard Education Letter, 12, 3, 1-5. Call617/495-3432.
Tolen, Patrick and Guerra, Nancy (1994). What Works in Reducing AdolescentViolence: An Empirical Review of the Field. Boulder, CO: Center for the Studyand Prevention of Violence, University of Colorado, 1-97. Call 303/492-1032.