Inter-American Development Bank
facebook
twitter
youtube
linkedin
instagram
Abierto al públicoBeyond BordersCaribbean Development TrendsCiudades SosteniblesEnergía para el FuturoEnfoque EducaciónFactor TrabajoGente SaludableGestión fiscalGobernarteIdeas MatterIdeas que CuentanIdeaçãoImpactoIndustrias CreativasLa Maleta AbiertaMoviliblogMás Allá de las FronterasNegocios SosteniblesPrimeros PasosPuntos sobre la iSeguridad CiudadanaSostenibilidadVolvamos a la fuente¿Y si hablamos de igualdad?Home
Citizen Security and Justice Creative Industries Development Effectiveness Early Childhood Development Education Energy Envirnment. Climate Change and Safeguards Fiscal policy and management Gender and Diversity Health Labor and pensions Open Knowledge Public management Science, Technology and Innovation  Trade and Regional Integration Urban Development and Housing Water and Sanitation
  • Skip to main content
  • Skip to secondary menu
  • Skip to primary sidebar
  • Skip to footer

Impacto

  • HOME
  • CATEGORIES
    • Beyond development effectiveness
    • Evaluation methods and techniques
    • Measuring our performance
    • What does and doesn’t work in development
  • Authors
  • English
    • Español

3 good examples of the impact of impact evaluations

July 19, 2012 por Francisco Mejía Leave a Comment


examples of impact of impact evaluationsA few weeks ago, I published a blog post on some of the unsettling implications of this paper that suggested that some interventions lose their punch when done by public agencies.

One of the takeaways was the need to “go up the bureaucratic supply chain” as nicely put by Justin Sandefur in a tweet on the post.

In other words, the need to jump over the “challenge of implementation” hurdle as Gabriel Demombynes described in his excellent blog.

Just as Esther Duflo has argued that we need to understand that the environment in which people make decisions is very different for the poor than for the rich, we also need to understand that institutions (in a general sense) are endogenous in impact evaluations.

Just as we know that most vaccines will not be as effective (and sometimes even harmful) when they are not kept frozen or refrigerated, it is important to acknowledge that many interventions could backfire when incentives differ significantly from the experimental setting.

Understanding institutional frameworks seems critical if impact is to have any scale.

So, how do we build that capacity to evaluate impact in challenging institutional settings?

Let me suggest one direction.

There are many ridges to the question and it cannot be answered in one blog post. But a good initial glimpse at the answer can be had by looking into what governments around the world are doing to embed rigorous evidence into policy design and implementation.

Here are some examples.

1.  In the US, many Departments in the Executive Branch – such as Health and Human Services or Justice – have been conducting RCTs as a core element in their policy formulation and implementation for many years.

In 2009, the Office of Management and Budget announced a government wide initiative to fund rigorous evaluations in federal spending. In May of this year, the White House instructed all agencies to demonstrate the use of evidence throughout their Fiscal Year 2014 budget submissions.

“The American” foreign aid agency, USAID, has also pushed hard for evidence based policy in its programs worldwide.

2.  In the UK, where impact evaluations are also conducted in many individual Departments, the Cabinet Office created its Behavioural Insights Team which relies heavily on rigorous evidence.  DFID, the UKs foreign aid arm, is also pushing the evidence agenda as a core element in its work as its Head of Evaluation explains in this interview.

3.  In Latin America, the pioneer was Mexico. The first Conditional Cash Transfer in the world, Oportunidades – created as Progresa 25 years ago in 1997- was designed and expanded on the basis of the results of rigorous Impact and Cost Benefit Evaluations.

Since then, Progresa-Oportunidades has evolved and the model has been adapted by many other CCT programs in Latin America (and elsewhere). Evidence based policy making is permeating not only social policy as shown by the role that CONEVAL plays in Mexico. It also is becoming national policy.

Perú and Chile sought assistance from MIT´s Abdul Latif Jameel Poverty Lab to “generate innovative proposals and empirical evidence that governments can use to answer key policy questions and design and implement better public policies.” Colombia has had SINERGIA, an evaluation institutional evaluation framework for over 10 years that is at the core of national social policy.

The Mexican, Chilean and Colombian experiences are reviewed in this paper by Marie Gaarder and Bertha Briceño. If you are bored reading and are craving some video, the CONEVAL and SINERGIA experiences are discussed here.

Impact evaluations are like protons, neutrons, and electrons which acquire mass only when interacting with the Higgs field. Impact evaluations will really affect national policies when they show effects in imperfect institutions. Maybe understanding better the institutional underpinnings that make them effective is our elusive particle.


Filed Under: What does and doesn't work in development Tagged With: Abdul Latif Jameel Poverty Lab, Behavioural Insights Team, Bertha Briceño, Cabinet Office, Chile, Colombia, CONEVAL Mexico, Cost Benefit, DFID, esther duflo, Gabriel Demombynes, Higgs field, impact, impact evaluation, institutions, Justin Sandefur, Kenya, Marie Gaarder, MIT, Office of Management and Budget, oportunidades, Peru, Progresa, SINERGIA, USAID, vaccines

Francisco Mejía

Francisco Mejía is a Consultant at the Office of Strategic Planning and Development Effectiveness at the Inter-American Development Bank.

Reader Interactions

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Primary Sidebar

Follow Us

Subscribe

Search

About this blog

This blog highlights effective ideas in the fight against poverty and exclusion, and analyzes the impact of development projects in Latin America and the Caribbean.

Categories

Footer

Banco Interamericano de Desarrollo
facebook
twitter
youtube
youtube
youtube

    Blog posts written by Bank employees:

    Copyright © Inter-American Development Bank ("IDB"). This work is licensed under a Creative Commons IGO 3.0 Attribution-NonCommercial-NoDerivatives. (CC-IGO 3.0 BY-NC-ND) license and may be reproduced with attribution to the IDB and for any non-commercial purpose. No derivative work is allowed. Any dispute related to the use of the works of the IDB that cannot be settled amicably shall be submitted to arbitration pursuant to the UNCITRAL rules. The use of the IDB's name for any purpose other than for attribution, and the use of IDB's logo shall be subject to a separate written license agreement between the IDB and the user and is not authorized as part of this CC- IGO license. Note that link provided above includes additional terms and conditions of the license.


    For blogs written by external parties:

    For questions concerning copyright for authors that are not IADB employees please complete the contact form for this blog.

    The opinions expressed in this blog are those of the authors and do not necessarily reflect the views of the IDB, its Board of Directors, or the countries they represent.

    Attribution: in addition to giving attribution to the respective author and copyright owner, as appropriate, we would appreciate if you could include a link that remits back the IDB Blogs website.



    Privacy Policy

    Derechos de autor © 2025 · Magazine Pro en Genesis Framework · WordPress · Log in

    Banco Interamericano de Desarrollo

    Aviso Legal

    Las opiniones expresadas en estos blogs son las de los autores y no necesariamente reflejan las opiniones del Banco Interamericano de Desarrollo, sus directivas, la Asamblea de Gobernadores o sus países miembros.

    facebook
    twitter
    youtube
    This site uses cookies to optimize functionality and give you the best possible experience. If you continue to navigate this website beyond this page, cookies will be placed on your browser.
    To learn more about cookies, click here
    X
    Manage consent

    Privacy Overview

    This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
    Necessary
    Always Enabled
    Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
    Non-necessary
    Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
    SAVE & ACCEPT