Development that Works
  • About

    This blog highlights effective ideas in the fight against poverty and exclusion, and analyzes the impact of development projects in Latin America and the Caribbean.
  • Timely thoughts on impact evaluations

    18
    Mar
    2014

    By

    blog arturo tardis

    Everyone has great ideas when it comes to designing development projects or policies.

    At the drawing board, and with a bit of ideological support, an abundance of ideas are born that could easily improve countless lives. However, these ideas, while sound in theory, don´t always work.  If only there was a time machine to take us to the future to verify the effectiveness of our projects and policies before implementation.

    In the British TV show “Doctor Who”, which has been running for over 50 years (talk about an effective intervention!), the main character travels through time and space in a device called the Tardis. With a Tardis, our projects would certainly reach 100% of their development goals and we would be the super champions of development effectiveness.

    Unfortunately, economists don’t have such a device. Instead we have economic theory and empirical methods to validate some of our theories and anticipate the results of our projects. Among them are impact evaluations.

    In a nutshell, impact evaluations measure the results of an intervention for a certain group of people.  The methodologies frequently used identify changes in any relevant dimension in the population that benefits from an intervention and compares them to a group of people (preferably identical) that did not receive it. This is a way to differentiate ideas that work from the ones that don’t.

    However, as economists love to say, there is no free lunch. Impact evaluations, like most activities that are intensive in data collection and analysis, can be expensive.  That’s why it is necessary to use them strategically in activities where it is of particular relevance to enhance accountability (large and very visible projects), where there are notable knowledge gaps, and to evaluate if pilot projects are worth scaling up.

    blog arturo infografics

    In the last five years the IDB started nearly 150 impact evaluations in different areas. Some of them have been finished and show interesting lessons that can increase the Bank’s development effectiveness.  Among them are evaluations that provide evidence on the effectiveness of projects that could be used to nurture similar discussions in other countries.

    The IDB´s recently published annual report, “Development Effectiveness Overview,” highlights these lessons.

    For example, a recent evaluation of an employment program in Mexico shows the benefits of properly designing programs that increase formal employment and worker’s income.   These evaluations are useful instruments that contribute to the policy debate in other countries, eventually providing a starting point for similar projects elsewhere.

    Other evaluations allow us to verify if certain interventions are in fact, reaching their intended development goals.  For instance, a frequent intervention is to provide loans to SMEs through a country’s development banks.

    This is a common practice, but there is little evidence of its effectiveness. A set of evaluations carried out with Bancoldex, a Colombian development bank, has helped to close the knowledge gap by showing that these types of loans increase employment, productivity, investment, and income in the SMEs that receive them.

    While we do not have a time machine like the Tardis that would allow us to be completely certain on the impact of our work, we do have increasingly reliable instruments to guide us through the world of development effectiveness.

     

    5 Responses to “Timely thoughts on impact evaluations”

    • Mani :

      Brilliant!

    • This sounds interesting, but it would be useful if you could link to the evaluations you discuss instead to just stories about them. The links to the employment program in Mexico and to the Colombia work are just to glossy stories, which claim impact evaluation shows success, but there is no discussion of what impact evaluation method was used, what the time frame of results were, etc. Links to the actual impact evaluation studies would be appreciated.

    • Chelsea :

      Dear Mr. Galindo,

      As someone interested in development projects in Latin America and a fan of Doctor Who, I never thought I would see those two things in the same article.

      With that being said, I think the blog post is a really interesting perspective on impact evaluations and how they ultimately effect program policies, especially with the Mexican and Colombian cases. I would love to learn more.

    Comment Arturo Galindo

    Subscription
    Categories
    Archives