Three starting points to understand education quality

I recently came across the question: if I was going to identify the main determinants of low quality of education in a given population, where would I start? Asking education system stakeholders involves capturing perceptions that may respond to group interests that do not necessarily provide an answer. Ideally, one would let the data lead to an answer.

After spending much time reviewing studies that look at very specific aspects of the complex education process, my recommendation is to start with three factors. First, you have to know your students. Second, you have to know your schools. Third, you have to know your system.

Let me explain what I mean.

First, you have to know who your students are.  The largest share of variation in test scores happens within the classroom in most countries. This means that a “bad” student in a good school will most probably score worse than a “good” student in a bad school.

For example, in Italy, 83% of test score variation is attributable to within classroom variation, 11% between groups within a school and 6% across schools. Much of this variation seems to be explained by socioeconomic status, i.e. parental education and household income.

In Chile, between 75 and 86% of the variation of test scores across schools is explained by parental education and household income. Interventions aiming at increasing or substituting for household investment in education seem very effective.

For example, improving nutrition (and here) and home educational resources (and here) show effects of up to 0.80 standard deviations and 0.56 standard deviations respectively. Another point to have in mind is that education is cumulative and early investments seem to have higher returns.

For example, access to pre-primary schooling, age of entry and early literacy all have positive effects on test scores later on. Also, improved nutrition at 12-24 months seems to have a stronger effect than improved nutrition later.

Therefore it is useful to get a sense on which investments have been done and are being done on students. So, you may want to start by looking at the empirical evidence on  household income and parent´s education.

Second, look at the specific unmet needs in schools. One size fits all interventions such as increasing spending per student alone or providing training, credentials or experience to teachers have been found not to have a consistent effect on the quality of education.

On the other hand, teacher and resource interventions directed to address specific unmet needs in schools do seem to work. The most successful interventions are those that specifically attempt to increase instructional time a given student receives.

Examples include increasing teacher attendance through monitoring, tutoring, increasing school hours, reducing class size and providing computer assisted instruction. Successful interventions that provide resources provide them in a way such that they adapt to the level and learning pace of students.

For example, providing textbooks mostly benefits top performers and computer assisted instruction seem to only work when the software adapts to student needs.

You may also want to estimate some proxy of instructional time to a given student.

If information is available, it should also include what resources are available to them in the school and how adequate they are to their needs. You should try to include instructional materials and infrastructure. If available, you should also include some proxy of teacher quality.

Third, know your system. You should understand the incentives and information set of key actors. Political decisions based on the power of stakeholders in the education system do not necessarily focus on improving the quality of education and may lead to suboptimal budget allocation.

Nevertheless, interventions aligning incentives seem to improve learning in students. Examples include school autonomy, vouchers and school based management. As it is the case with students, results depend on school characteristics.

For example, some researchers have found that school decentralization improved test scores overall but did not reach the poor. Some reasons may be that the poor may lack the ability to voice their preferences, local elites may capture public resources or that local governments may be less technically able than the central government to administer public services.

In another example, village education committees were strengthened by providing better information and sensitizing people about the importance of education. This study found that providing information to villagers without facilitating the use of that information was not useful.

Your list of factors should include a description of the incentives present, restrictions and information available to the subject of interest. Of special interest is to understand the context of teachers, as teachers have been suggested to be the most important school factor that explains quality of education.

Identifying the extent to which different inputs contribute to student learning is difficult. Debate continues on which inputs should be included to improve the quality of education. A starting point to find the factors that explain quality of education in a population are student characteristics, appropriate resources available to them and the context in which these resources are provided.

You can get started by knowing your students, knowing your schools and knowing your system.