Everyone has great ideas when it comes to designing development projects or policies.
At the drawing board, and with a bit of ideological support, an abundance of ideas are born that could easily improve countless lives. However, these ideas, while sound in theory, don´t always work. If only there was a time machine to take us to the future to verify the effectiveness of our projects and policies before implementation.
In the British TV show “Doctor Who”, which has been running for over 50 years (talk about an effective intervention!), the main character travels through time and space in a device called the Tardis. With a Tardis, our projects would certainly reach 100% of their development goals and we would be the super champions of development effectiveness.
Unfortunately, economists don’t have such a device. Instead we have economic theory and empirical methods to validate some of our theories and anticipate the results of our projects. Among them are impact evaluations.
In a nutshell, impact evaluations measure the results of an intervention for a certain group of people. The methodologies frequently used identify changes in any relevant dimension in the population that benefits from an intervention and compares them to a group of people (preferably identical) that did not receive it. This is a way to differentiate ideas that work from the ones that don’t.
However, as economists love to say, there is no free lunch. Impact evaluations, like most activities that are intensive in data collection and analysis, can be expensive. That’s why it is necessary to use them strategically in activities where it is of particular relevance to enhance accountability (large and very visible projects), where there are notable knowledge gaps, and to evaluate if pilot projects are worth scaling up.
In the last five years the IDB started nearly 150 impact evaluations in different areas. Some of them have been finished and show interesting lessons that can increase the Bank’s development effectiveness. Among them are evaluations that provide evidence on the effectiveness of projects that could be used to nurture similar discussions in other countries.
The IDB´s recently published annual report, “Development Effectiveness Overview,” highlights these lessons.
For example, a recent evaluation of an employment program in Mexico shows the benefits of properly designing programs that increase formal employment and worker’s income. These evaluations are useful instruments that contribute to the policy debate in other countries, eventually providing a starting point for similar projects elsewhere.
Other evaluations allow us to verify if certain interventions are in fact, reaching their intended development goals. For instance, a frequent intervention is to provide loans to SMEs through a country’s development banks.
This is a common practice, but there is little evidence of its effectiveness. A set of evaluations carried out with Bancoldex, a Colombian development bank, has helped to close the knowledge gap by showing that these types of loans increase employment, productivity, investment, and income in the SMEs that receive them.
While we do not have a time machine like the Tardis that would allow us to be completely certain on the impact of our work, we do have increasingly reliable instruments to guide us through the world of development effectiveness.
Brilliant!
Brilliant!
This sounds interesting, but it would be useful if you could link to the evaluations you discuss instead to just stories about them. The links to the employment program in Mexico and to the Colombia work are just to glossy stories, which claim impact evaluation shows success, but there is no discussion of what impact evaluation method was used, what the time frame of results were, etc. Links to the actual impact evaluation studies would be appreciated.
Thanks for the comment David. In the story on Colombia in the webpage that is linked to the post, you will find a link to the Bancoldex impact evaluation and to other relevant information on the project. This is at the at the end of the page. I noted that we missed the link to the ones on Mexico, but they will be added shortly.
David, while the get posted in the page the links to the evaluations of Mexico are the following:
i.Impact evaluation to the subprogram “Bécate”
http://idbdocs.iadb.org/wsdocs/getdocument.aspx?docnum=38685632
ii.Impact evaluation of the “Employment Support Program (PAE)”
http://idbdocs.iadb.org/wsdocs/getdocument.aspx?docnum=38685628
This sounds interesting, but it would be useful if you could link to the evaluations you discuss instead to just stories about them. The links to the employment program in Mexico and to the Colombia work are just to glossy stories, which claim impact evaluation shows success, but there is no discussion of what impact evaluation method was used, what the time frame of results were, etc. Links to the actual impact evaluation studies would be appreciated.
Thanks for the comment David. In the story on Colombia in the webpage that is linked to the post, you will find a link to the Bancoldex impact evaluation and to other relevant information on the project. This is at the at the end of the page. I noted that we missed the link to the ones on Mexico, but they will be added shortly.
David, while the get posted in the page the links to the evaluations of Mexico are the following:
i.Impact evaluation to the subprogram “Bécate”
http://idbdocs.iadb.org/wsdocs/getdocument.aspx?docnum=38685632
ii.Impact evaluation of the “Employment Support Program (PAE)”
http://idbdocs.iadb.org/wsdocs/getdocument.aspx?docnum=38685628
Dear Mr. Galindo,
As someone interested in development projects in Latin America and a fan of Doctor Who, I never thought I would see those two things in the same article.
With that being said, I think the blog post is a really interesting perspective on impact evaluations and how they ultimately effect program policies, especially with the Mexican and Colombian cases. I would love to learn more.
Dear Mr. Galindo,
As someone interested in development projects in Latin America and a fan of Doctor Who, I never thought I would see those two things in the same article.
With that being said, I think the blog post is a really interesting perspective on impact evaluations and how they ultimately effect program policies, especially with the Mexican and Colombian cases. I would love to learn more.