EducationNews.org

Student mobility in Milwaukee could hamper the state's latest attempt to evaluate voucher schools, warns a report from the Legislative Audit Bureau released Wednesday

The report analyzed the early results of a project by University of Arkansas researchers charged with comparing the test scores of students enrolled in the Milwaukee Parental Choice Program with their counterparts in Milwaukee Public Schools.

A little more than one year into the study, 430 voucher students - nearly 16% of those in the study - had transferred to MPS schools, the audit bureau found.

By not including those students in its analysis, the audit bureau report found no statistical differences in the reading and math gains made on state tests by students in either MPS or voucher schools. The University of Arkansas study, which included the data from the former voucher students in its figures for voucher students, found a statistically significant advantage in math gains made by seventh- and eighth-graders in the voucher schools.

The University of Arkansas researchers defended including voucher students in the study even after they had moved to MPS, contending that the state tests are given near the start of the school year and therefore do not reflect much of those students' experience in public schools. The university's project aims to follow students from both programs over the course of five years, monitoring progress on the reading and math portions of the Wisconsin Knowledge and Concepts Examinations.

The audit bureau chose not to include the departed voucher students, however, and cautioned about the effect of continued attrition in future years. In addition to the voucher students who transferred to MPS schools, an additional 7.4% of choice students and 10.6% of MPS students could not be found in the second year of the study.

"Without test scores for sufficient numbers of choice and MPS pupils, it may be challenging to conclude that any differences in scores from one year to the next are statistically significant," the audit bureau report says. "In general, the more test scores included in an analysis, the more likely it is that any differences in average scores are statistically significant."

Study defended

Mobility poses problems for any long-term study, especially in urban school systems, said Patrick Wolf, an education reform professor at the University of Arkansas and the project's lead researcher.

But Wolf said his project started with large numbers of students - 2,727 each in the choice program and in MPS.

"I think the most important thing for everyone to understand is that the Legislative Audit Bureau chose a somewhat different approach to comparing the results for choice students versus MPS students than we employed in our analysis and in our reports," he said. "We also would encourage people not to read too much into these initial preliminary results."

The Legislature plans to carefully monitor the methodology used by researchers to make sure the data generated from the study is accurate, said state Rep. Peter Barca (D-Kenosha), co-chairman of the Legislature's joint audit committee.

"I think it's useful that these reviews are being done," he said, "but we still have a ways to go to really be able to draw firm conclusions."