Do Test Scores Determine School Quality?

Student performance on standardized tests is typically the main measure of school quality. However, is it the case that schools that improve academic test scores are the same schools that also improve important longer-run outcomes such as crime, risky behaviors, college attendance, and earnings?

Important decisions (such as school closures, the design of teacher performance pay schemes, and school accountability procedures) often rely on test-score metrics. These metrics may be simple test-score averages or more sophisticated measures of school effectiveness at improving test scores. The more sophisticated measures approximate the specific contribution of each school to the improvement of students’ outcomes and are often called value-added.

 If schools that improve academic test scores are not the same schools that also improve important longer-run outcomes, basing policy decisions solely on test-score metrics (which is common) may not improve the broader adult outcomes that policymakers and parents may value.

Can test scores be interpreted as a summary measure of school quality?

In a recently published study at the Review of Economic Studies, jointly authored by Diether W. Beuerman (Inter-American Development Bank), C. Kirabo Jackson (Northwestern University), Laia Navarro-Sola (Stockholm University), and Francisco Pardo (University of Texas at Austin), we use administrative data covering the full population of Trinidad and Tobago to answer this critical question for the first time.

Our study shows that schools provide meaningful contributions to the trajectories of many academic and non-academic outcomes, such as test scores, school dropout, youth crime, teen motherhood, and adult formal employment.

However, the relationship between the contributions of each school to the improvement of tests scores and the contributions of the same schools to the improvement of other outcomes is surprisingly low.

For example, the correlations between improvements on test scores and improvements on other important outcomes such as school completion, non-teen motherhood, or being formally employed in adulthood lie between 0.04 and 0.15. These results imply that school output is multidimensional, such that schools that improve test scores are often not those that improve outcomes related to broader adult well-being (which parents may value).

Accordingly, policymakers should be cautious (and thoughtful) regarding using test-score metrics as the sole measure in accountability systems and incentive pay schemes and should instead adopt a more holistic view of school quality.

Do parents look beyond test scores when choosing schools for their children?

Our study then seeks to better understand parental preferences for schools. We do so by linking parents’ ranked lists of schools to the estimated contributions of each school to the improvement of the different academic and non-academic outcomes.

Our results uncover several novel findings:

1. Parents of higher-achieving children assign higher rankings to schools that display relatively larger contributions to the improvement of test scores.
2. Looking at non-academic outcomes, parents also privilege schools that reduce youth arrests, reduce teen births, and increase adult formal labor market participation.

These results suggest that parents may use reasonable measures of school quality when making investment decisions for their children—a requirement for the potential benefits of school choice.

The fact that parents do not only choose schools that improve academics but also those that improve non-academic and longer-run outcomes suggests that the benefits of school choice may extend to a wide range of outcomes above and beyond test scores.

This result, therefore, implies that policy evaluations based solely on test scores may be misleading about the welfare effects of school choice.

We also document key differences in the choices between parents of low-achieving students and high-achieving students: parents of low-achievers value schools that are effective at improving non-test outcomes relatively more than schools that are effective at improving test scores, while the opposite is true for parents of high-achievers.

This suggests that market forces may drive competition more strongly to raise test scores among schools serving high-achieving populations and to raise non-academic outcomes among schools serving low-achieving populations.

If these differences reflect parents’ true preferences, this may be efficient. However, if they reflect differences in information, there may be value to aid the school choice process with the provision of information to parents regarding the effectiveness of schools at improving a wide array of both academic and non-academic outcomes (as opposed to only school averages of the outcomes).

This may improve the decisions of all parents and could increase the potential allocative efficiencies and competitive benefits of school choice.

What do you think about school quality and how should it be measured? Leave your comment!