Data Quality Matters and Data Reporting Errors Have Real Consequences

lead image

Parents, policy makers, and stakeholders rely on research and data and to help them make decisions, allocate resources, and develop new programs, laws, and policies—all with the intention of improving student outcomes and opportunities. 

While the data will never be able to tell the whole story, it can help bolster our understanding of what is happening on the ground and help us to make comparisons across states and subgroups. And groundbreaking research can fundamentally change the way that we see and understand the world. For example, the Coleman Report first introduced the concept of the achievement gap and the (then revolutionary) idea that out-of-school factors like socioeconomic status and family background are just as important as factors like funding, facility quality, and teacher effectiveness in explaining the variation in student performance and long-term outcomes.

Much of the subsequent education research over the past 50 years has been centered on better understanding the intersection of race and class and various educational outcomes and opportunities—and proposing solutions for closing the achievement gap, improving school quality, and empowering parents. And all of this research has relied on the collection of accurate, relevant, and timely public education data.

However, for both state and federal datasets, there are number of reporting layers that can complicate and sometimes undermine data quality. Public schools are typically required to report data to their school district (also known as a Local Education Agency or LEA), their state department of education (also known as a State Education Agency or SEA), and/or the U.S. Department of Education. SEAs publish data regarding student enrollment, demographics, and performance (among other variables) as required by their state law, and at the direction of policymakers, to help increase transparency and provide important information to the public. In addition, the federal government collects data from SEAs and LEAs to aggregate and publish national datasets that are both comprehensive and comparable.

Ultimately, ensuring data quality involves multiples steps and numerous stakeholders:

  1. State and federal agencies must provide clear definitions and instructions for populating and submitting data and understand the reporting burden it puts on schools.
  2. Schools must take the time to carefully review the instructions and accurately assemble and submit their data.
  3. Intermediaries (including LEAs and SEAs) must properly review and aggregate the data before passing it along.
  4. State and federal agencies must properly clean, review, and verify the data before they publish it.
  5. Schools, researchers, and others stakeholders must review the data and report any data quality issues to relevant agencies.
  6. If material errors are identified and/or reported, state and federal agencies must work to correct and update public datasets in the interest of accuracy and transparency.

However, this process can break down at different levels and the resulting data quality issues often have very real consequences. In the 2010-11 Civil Rights Data Collection (CRDC), it was not possible to compare the proportion of English Language Learners (ELLs) in district schools and charter schools, because over one-third of charter schools did not report an ELL count. Self-inflicted data entry and submission errors caused later headaches and embarrassment for a number of charter schools and management organizations when researchers pointed out that their reported rates of chronic teacher absenteeism were much higher than state and local averages. And more recently, a state reporting error in the 2015-16 Common Core of Data (CCD) impacted the free and reduced-price lunch (FRPL) counts for hundreds of charter schools in New York. More specifically, the CCD incorrectly reported FRPL counts of zero for more than 250 charter schools, despite the fact that these schools served significant proportions of low-income students.

Education data is used for numerous purposes that have real consequences for schools. Enrollment and demographic information is used to determine school funding. Performance data is used for accountability systems, school ratings and rankings, and public dissemination and consumption. And other indicators of school quality and performance, including school safety, student and teacher absenteeism, graduation rates, and student discipline (among many other important factors) are used by reporters, researchers, and the public to hold schools accountable, protect the most disenfranchised students, improve transparency, monitor trends, and provide information that grounds and informs policymaking and resource allocation. Accordingly, it is absolutely essential that schools accurately report their data and that state and federal agencies publish data of the highest possible quality.

Data collection, monitoring, and reporting have become a part of life in American public schools. And this process has led to legitimate concerns about the reporting burden, time spent away from the classroom, and student privacy. School districts and state and federal agencies must continue to ensure that data collection procedures are clear, efficient, and ultimately valuable. However, there is real power in education data. It provides parents, policymakers, and stakeholders with actionable information about important trends and differences across subgroups—and a better understanding of what is and is not working.

When data reporting and publication systems break down, it has real implications for schools, parents, and decision makers—and stakeholders from across the public education sector must work together to further improve data quality.

Kevin Hesla is the Director of Research and Evaluation at the National Alliance for Public Charter Schools.

Add new comment