Does it matter how we compare schools?

In Mexico, over 95% of elementary schools receive annual report cards. These cards provide information on the standardized average school test scores in Math, Spanish and a rotating subject.

The card also includes past performance and State and Country averages.  Starting in 2009, the state and country averages were made specific to the marginality level of the locality where the school is located.

The idea was to provide information on performance for schools under similar circumstances. Schools in localities with high marginality face more restrictions than schools located in localities with low marginality and therefore test scores tend to be lower.

In practice, some hard working schools in highly marginalized localities were performing poorly when compared to the average but now are performing well when compared to the adjusted group average. Likewise not so hard working schools in localities with low marginality were performing well when compared to the average but now are performing poorly when compared to the adjusted group average.

Do you think a change in the comparison group would change how schools perform?

To answer the question matters for two reasons. First, there are theoretical considerations to believe that information on relative performance changes behavior.

Second, the empirical evidence shows we may have some schools benefit at the cost of some other schools adversely affected.
First, there are theoretical reasons to believe that relative performance changes behavior. Standard labor models state that the decision on how much effort to put into a task will be determined up to the point where marginal benefit equals marginal cost.

There are two channels through which the information on relative performance could change your behavior: it changes the information available to you, and it changes the information available to others. Changes in information available to you may change your beliefs on benefits or costs.

The effect of this information may be completely different for different persons. Suppose you are already doing your best, and then you are told you are performing poorly with respect to others. This news may discourage you. On the other hand it may turn you are performing well above average.

This news may make you relax. The provision of information available to you may also change your behavior if you have inherent competitive preferences or show pro-social behavior (if you do not like to feel you are not acting cooperatively).

The provision of information to others may induce social pressure and peer monitoring. The effects will depend on your position relative to others. If you underperform social pressure and peer monitoring will likely make you increase effort while if you outperform then the lack of social pressure may make you relax.

Second, there is empirical evidence that information on relative performance changes behavior. A study by Azmat and Iriberri (2010) on the effects of information provided to students on relative performance finds a 5% increase in student’s grades.

They find that this effect was significant both for good and bad students. The authors argue that feedback on performance relative to others may change your beliefs on your own ability or that students may have a preference to compete. Another study related to relative performance news is that of Mas and Moretti (2009).

In this study the authors analyze the behavior of cashiers who can observe the length of lines of other cashiers, therefore obtaining information on relative performance. The authors found that when an efficient cashier arrives, it tends to improve performance among other cashiers. They argue that social pressure or pro-social behavior may be motivating them. Both of these show positive peer effects.

A contrasting paper showing negative peer effects is that of Eriksson et al. (2009). In this paper, they set up an experiment to test the effects of relative feedback on performance. They ask individuals to add four two digit numbers in 20 minutes. They pay to the winner but nothing to the looser.

In one group, individuals can see how the other person is doing. On a second group individuals cannot see how the other person is performing. They find that the underdogs make significant more mistakes in the first group. In this case, social pressure may result in negative peer effects.

We are now comparing schools that received good news to those that received bad news. We will keep you posted when we have some results. This is work in progress. We would love to hear any ideas you may have.

If you would like to learn more about the ENLACE test and see a report card, please click here.

To see why marginality would matter in education, you can read my previous blog by clicking here.