I’m happy to share with you the IDB has launched its 2010 Development Effectiveness Overview (DEO). The 2010 DEO shows that in 2010, 27% of our projects had rigorous impact evaluation plans designed at project approval.
The DEO reports on key impact evaluations, both ongoing and recently completed, of Bank’s projects in our five institutional priorities.
Projects include areas such as health in rural areas, early childhood education, neighborhood improvement, renewable energy, access to credit, citizen security, science and technology, tourism and agriculture, among others.
In this blog we will review some of these evaluations in this and future entries, to show with concrete examples how we learn from our work and how we apply such knowledge.
A fine example of the learning process is an experiment that the IDB supported in Argentina to test three innovative approaches to improve learning in mathematics and science in elementary schools that serve disadvantaged kids.
The 2008-09 DEO reported on the design of the experiment, and this year we present the results and policy implications.
The experiment involved two different pedagogical models—Science, Technology and Creativity (CTC) and the Scientific Literacy Program (PAC) — and a play-based mathematics approach called Mathematics for All.
They were tested through an experimental evaluation in which schools were randomly assigned to either one of the three treatment groups or to a control group that received the standard program in natural science and mathematics.
The pilot covered over 18,100 fourth-grade students in 675 schools in the provinces of Tucumán and Buenos Aires. These two areas were selected based on disadvantaged socioeconomic characteristics and poor educational results.
Already after one academic year, both qualitative and quantitative results are observable according to the study carried out by a team of external evaluators from the Catholic University of Uruguay (UCUDAL) and the International Institute for Education Planning (IIPE).
The results of the qualitative evaluation reveal improvements in teachers’ knowledge of curricular concepts, professional self-image, and beliefs about their students’ ability in these subjects. The results of the quantitative evaluation indicate that all beneficiary students improved in their learning more than those in the control group.
The Math for All model had the strongest effect on student learning, with the average test score increasing by 44 points (close to half a standard deviation) compared to 19 points among students who received the traditional curriculum.
The impact was particularly strong in Buenos Aires, where the difference between the two groups was 34 points, or a third of a standard deviation. This is a huge impact given that, on average, students learn 40 percent of a standard deviation during a school year.
For all three models, the effect sizes varied by module, providing important information on what dimensions need strengthening. For example, in the Math for All program the greatest progress was observed in Arithmetic, which the qualitative and process evaluation suggest was the area that was better implemented.
Based on the results of the economic analysis of the pilot, which revealed that PAC was more cost-effective than CTC, PAC will be brought to a national scale through a Bank-financed operation.
However, given the short timeframe of the pilot, the experimental application of the CTC model will be continued in 200 primary schools. Also Mathematics for All will be scaled up to benefit students throughout the country.
This example shows development effectiveness in action. Sensible solutions to longstanding development challenges were rigorously tested and policy implications followed. The result: children are learning more math and science in Argentina.
Leave a Reply