NAGB Again Pursuing Analysis of Public Charter Schools on NAEP

For almost a decade, there’s been a heated debate over what the National Assessment of Educational Progress (NAEP), commonly referred to as the nation’s report card, tells us about student achievement in public charter schools.

A couple of items of consensus emerged from the debate:

• NAEP data should be used and interpreted with caution. NAEP data present snapshot results of different students at different points in time. Moreover, comparing a snapshot of student performance between charters and traditional public schools does not control for the potential bias of students selecting to attend charter schools. (Check out this transcript of a 2004 PBS NewsHour interview with our President and CEO Nina Rees, who was then Assistant Deputy Secretary for the Office of Innovation and Improvement at the U.S. Department of Education, commenting on the release of 2003 NAEP data.)
• NAEP data may be used for descriptive analyses that can generate hypotheses for further study.

With that background, the National Assessment Governing Board (NAGB), the governing board of NAEP, commissioned a new study of NAEP data for charter schools this past summer. At their November board meeting, exploratory results were presented. The study addresses the following questions:

• Who attends charter schools?
• Does student achievement differ between charter schools and regular public school students?
• How do student subgroups perform in charter schools?
• How do charter schools compare with regular public schools in urban school systems?

While the study authors, Naomi Chudowsky and Alan Ginsburg, use caution about comparing public charter schools and traditional public schools with NAEP data, they report the following:

• “There is a consistent pattern of higher average NAEP scores for regular public schools than for charter schools when we look at the nation as a whole. But public school advantage has diminished.”
• “However, the closer we focus in on large cities, where most charter schools are located, the more the picture changes in favor of charter schools. In all large cities combined, student achievement is roughly even overall, but the black and Hispanic subgroups show higher scores in charter schools.”
• “When we examined four urban areas specifically (DC, Atlanta, Chicago, Milwaukee), students in charter schools significantly outperformed their peers in regular public schools in almost all of the subjects/grades analyzed.”
• “Charter school class sizes are smaller, and there is some evidence that charter school students receive more instructional time in some subjects.”

Last year we wrote about the most recent 2011 NAEP data (see here and here), pointing out the rise in NAEP test scores for public charter schools, especially among several subgroups and on the Trial Urban District Assessment (TUDA).

So what can NAEP data tell us about the performance of charter schools? In the best case scenario, the uptick in NAEP scores reflects the overall upward trend in charter student performance that more rigorous studies have been identifying over the past decade (see our write-up here). NAEP data on charter schools should be used in context and with caution. The higher NAEP test scores for charter schools may suggest that charter schools are getting better in comparison with traditional public schools, but the data cannot say this definitively. Fortunately, longitudinal studies using student-level data (and a meta-analysis of these studies) are pointing in the direction of increasing student performance of charter schools.