Inter-American Development Bank
facebook
twitter
youtube
linkedin
instagram
Abierto al públicoBeyond BordersCaribbean Development TrendsCiudades SosteniblesEnergía para el FuturoEnfoque EducaciónFactor TrabajoGente SaludableGestión fiscalGobernarteIdeas MatterIdeas que CuentanIdeaçãoImpactoIndustrias CreativasLa Maleta AbiertaMoviliblogMás Allá de las FronterasNegocios SosteniblesPrimeros PasosPuntos sobre la iSeguridad CiudadanaSostenibilidadVolvamos a la fuente¿Y si hablamos de igualdad?Home
Citizen Security and Justice Creative Industries Development Effectiveness Early Childhood Development Education Energy Envirnment. Climate Change and Safeguards Fiscal policy and management Gender and Diversity Health Labor and pensions Open Knowledge Public management Science, Technology and Innovation  Trade and Regional Integration Urban Development and Housing Water and Sanitation
  • Skip to main content
  • Skip to secondary menu
  • Skip to primary sidebar
  • Skip to footer

Impacto

  • HOME
  • CATEGORIES
    • Beyond development effectiveness
    • Evaluation methods and techniques
    • Measuring our performance
    • What does and doesn’t work in development
  • Authors
  • English

Randomized Control Trials on a Budget

March 27, 2012 por Francisco Mejía Leave a Comment


Randomized Control Trials on a BudgetA few weeks ago, Martin Ravallion wrote a great blog on “shoe-string” evaluations, in which he suggested that there are ways of bypassing two of the most costly items in rigorous Impact Evaluations: contemporaneous credible baseline data; and second, “objectively assessed outcome indicators”, which are based on surveys.

Ravallion then proposed a “shoestring” solution that replaces costly quantitative contemporaneous baseline data, with retrospective qualitative data gathered by asking post intervention treated and untreated groups on how much their welfare has improved since the intervention began.

To test the method, Ravallion then applied it to a full blown IE in China and found it unreliable. Being this a test where n=1, he then calls for additional such tests, given that their marginal costs should be very low and have great potential.

So it seems the jury is still out on reaching credibility inexpensively.

Or is it? Maybe not, if we take look at the potential of administrative data.

The Coalition for Evidence Based Policy has just published a great resource paper on the potential embedded in administrative data to provide cheaper information for rigorous impact evaluations.

That is, instead of using customized survey information specifically designed to capture expected outcomes via predefined outcomes indicators, it is often possible to reduce costs by using existing administrative data that has already been collected for other purposes such as health records, criminal arrests and test scores, for example.

To illustrate the feasibility – and flexibility – of this approach, the paper summarizes five RCTs, in areas as diverse as criminal justice (here and here), child welfare and substance abuse, community wide interventions to improve parenting skills, and teacher incentive systems.

The paper also highlights that certain conditions must be met in order for this type of low cost RCT to be feasible. The first one is obvious, access to quality data on outcomes has to be cheap. Second, a sizeable number of individuals are available without special recruitment efforts, and third, key policy officials have to be on board to assure access over time.

In addition to the examples listed above, there are many good applications of RCTs using administrative data. Alessandro Maffioli reported here on the potential use of administrative data and that there is a real “island of misfit toys” for evaluators out there in the evaluation of innovation programs. An island that can even be used to document the impact of land distribution on political power in 16th century Buenos Aires.

Note: the Coalition for Evidence based Policy has an outstanding publication list which includes not only technical papers and reviews, but comprehensive Guides and Checklists (great for those of us who love the Cool-Aid of the Checklist Manifesto) that are comprehensive, simple and extremely useful.


Filed Under: Beyond development effectiveness Tagged With: administrative data, Coalition for Evidence Based Policy, low cost evaluation, Martin Ravallion, Randomized Control Trials

Francisco Mejía

Francisco Mejía is a Consultant at the Office of Strategic Planning and Development Effectiveness at the Inter-American Development Bank.

Reader Interactions

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Primary Sidebar

Follow Us

Subscribe

Search

About this blog

This blog highlights effective ideas in the fight against poverty and exclusion, and analyzes the impact of development projects in Latin America and the Caribbean.

Categories

Footer

Banco Interamericano de Desarrollo
facebook
twitter
youtube
youtube
youtube

    Blog posts written by Bank employees:

    Copyright © Inter-American Development Bank ("IDB"). This work is licensed under a Creative Commons IGO 3.0 Attribution-NonCommercial-NoDerivatives. (CC-IGO 3.0 BY-NC-ND) license and may be reproduced with attribution to the IDB and for any non-commercial purpose. No derivative work is allowed. Any dispute related to the use of the works of the IDB that cannot be settled amicably shall be submitted to arbitration pursuant to the UNCITRAL rules. The use of the IDB's name for any purpose other than for attribution, and the use of IDB's logo shall be subject to a separate written license agreement between the IDB and the user and is not authorized as part of this CC- IGO license. Note that link provided above includes additional terms and conditions of the license.


    For blogs written by external parties:

    For questions concerning copyright for authors that are not IADB employees please complete the contact form for this blog.

    The opinions expressed in this blog are those of the authors and do not necessarily reflect the views of the IDB, its Board of Directors, or the countries they represent.

    Attribution: in addition to giving attribution to the respective author and copyright owner, as appropriate, we would appreciate if you could include a link that remits back the IDB Blogs website.



    Privacy Policy

    Derechos de autor © 2025 · Magazine Pro en Genesis Framework · WordPress · Log in

    Banco Interamericano de Desarrollo

    Aviso Legal

    Las opiniones expresadas en estos blogs son las de los autores y no necesariamente reflejan las opiniones del Banco Interamericano de Desarrollo, sus directivas, la Asamblea de Gobernadores o sus países miembros.

    facebook
    twitter
    youtube
    This site uses cookies to optimize functionality and give you the best possible experience. If you continue to navigate this website beyond this page, cookies will be placed on your browser.
    To learn more about cookies, click here
    X
    Manage consent

    Privacy Overview

    This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
    Necessary
    Always Enabled
    Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
    Non-necessary
    Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
    SAVE & ACCEPT