Inter-American Development Bank
facebook
twitter
youtube
linkedin
instagram
Abierto al públicoBeyond BordersCaribbean Development TrendsCiudades SosteniblesEnergía para el FuturoEnfoque EducaciónFactor TrabajoGente SaludableGestión fiscalGobernarteIdeas MatterIdeas que CuentanIdeaçãoImpactoIndustrias CreativasLa Maleta AbiertaMoviliblogMás Allá de las FronterasNegocios SosteniblesPrimeros PasosPuntos sobre la iSeguridad CiudadanaSostenibilidadVolvamos a la fuente¿Y si hablamos de igualdad?Home
Citizen Security and Justice Creative Industries Development Effectiveness Early Childhood Development Education Energy Envirnment. Climate Change and Safeguards Fiscal policy and management Gender and Diversity Health Labor and pensions Open Knowledge Public management Science, Technology and Innovation  Trade and Regional Integration Urban Development and Housing Water and Sanitation
  • Skip to main content
  • Skip to secondary menu
  • Skip to primary sidebar
  • Skip to footer

Impacto

  • HOME
  • CATEGORIES
    • Beyond development effectiveness
    • Evaluation methods and techniques
    • Measuring our performance
    • What does and doesn’t work in development
  • Authors
  • English
    • Español

Can we randomize institutional reform?

May 24, 2011 por Koldo Echebarría Leave a Comment


A few days ago I attended a book presentation on experimental evaluations where one participant stated, “this is all well and good, but it does not apply to institutional reforms; one cannot ‘randomize’ the reform of a ministry.”

Similarly, a document from a bilateral aid agency stated that it can experimentally evaluate only 5 percent of the total development assistance under its management, noting in particular its inability to perform evaluations of institutional reforms.

I disagree. Institutional structures and reforms affect – at the end of the day – individual behavior in citizens and functionaries.

And we can generate rigorous evidence on this behaviour based on randomized evaluations.

And we have to be able to generalize and validate these evaluations in order to inform the larger institucional frameworks, based on hard evidence.

The fact is, I believe, that this kind of conclusions follow from the skewed approach we apply to institutional reforms, which helps to explain our limited success in their application. Historically, institutional reforms have focused more on the instrument (the formal or informal rules that we want change), than on the behavior (of citizens or public officials) that the change in rules seeks to influence in order to achieve a specific result (for example, to increase tax receipts, formalize employment, or provide more access to the judicial system).

By focusing our attention on rules or institutional arrangements, the path to change tends to become mired in ideologies that claim superiority for a certain institutional engineering approach over another (usually because it is practiced in a more advanced or more admired country).

We operate through imitation (even in his time, de Tocqueville said that institutional museums contained very few originals and many copies.)

The result is that many institutional transplants are rejected and the new rules are merely approved, and not translated into changes in behavior.  Even worse, they are twisted by the participants in ways that subvert their original intention.

This is what a colleague of mine calls the corruption of anti-corruption, when warnings are issued about anti-corruption measures and attention is diverted from substantive issues that remain unaddressed.

We frequently invest in institutional changes that are not based on empirical evidence that demonstrate causal connections between institutions, effects on behavior, and ultimate objectives. Very frequently the excuse is that these changes do not have a counter factual. The truth is that very few assessments provide not even a rigorous before and after narrative that describes the changes taking place

If we focus on changes in behavior of officials and citizens that we want to modify, and we apply experimental logic to their design and evaluation processes, we will uncover more effective and efficient approaches.

“Randomization” is one of the methods at our disposal for testing the effects of various approaches (e.g., different relationships between tax inspectors and groups of randomly selected taxpayers, and ascertaining their effects on compliance and tax revenue). Other empirical methods can be applied to collect evidence to match tables of available administrative data.

An example of this effort is contained in an article on the effects of different levels of administrative effort in implementing labor standards in Brazil.

Some will say that this cannot be done, that all citizens should be treated equally. But the fact is that random selection of a sample can, in fact, give all potentially affected actors the same options.  In addition, the different approaches would be carried out within reasonable policy spaces where citizens and public officials interact.

Sometimes, just by using a different way to frame options for citizens to relate to governmental officials can produce substantial change.

This is also the point of Nudge, the provocative text of Thaler and Sunstein where we can find many examples that show the potential for public policy of this approach.


Filed Under: Evaluation methods and techniques Tagged With: Empirical evidence, Institutional Reform, Nudge, Randomization, Sustein, Thaler

Koldo Echebarría

Koldo Echebarría is former General Manager and Chief of the Office of Strategic Planning and Development Effectiveness at the Inter-american Development Bank.

Reader Interactions

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Primary Sidebar

Follow Us

Subscribe

Search

About this blog

This blog highlights effective ideas in the fight against poverty and exclusion, and analyzes the impact of development projects in Latin America and the Caribbean.

Categories

Footer

Banco Interamericano de Desarrollo
facebook
twitter
youtube
youtube
youtube

    Blog posts written by Bank employees:

    Copyright © Inter-American Development Bank ("IDB"). This work is licensed under a Creative Commons IGO 3.0 Attribution-NonCommercial-NoDerivatives. (CC-IGO 3.0 BY-NC-ND) license and may be reproduced with attribution to the IDB and for any non-commercial purpose. No derivative work is allowed. Any dispute related to the use of the works of the IDB that cannot be settled amicably shall be submitted to arbitration pursuant to the UNCITRAL rules. The use of the IDB's name for any purpose other than for attribution, and the use of IDB's logo shall be subject to a separate written license agreement between the IDB and the user and is not authorized as part of this CC- IGO license. Note that link provided above includes additional terms and conditions of the license.


    For blogs written by external parties:

    For questions concerning copyright for authors that are not IADB employees please complete the contact form for this blog.

    The opinions expressed in this blog are those of the authors and do not necessarily reflect the views of the IDB, its Board of Directors, or the countries they represent.

    Attribution: in addition to giving attribution to the respective author and copyright owner, as appropriate, we would appreciate if you could include a link that remits back the IDB Blogs website.



    Privacy Policy

    Derechos de autor © 2025 · Magazine Pro en Genesis Framework · WordPress · Log in

    Banco Interamericano de Desarrollo

    Aviso Legal

    Las opiniones expresadas en estos blogs son las de los autores y no necesariamente reflejan las opiniones del Banco Interamericano de Desarrollo, sus directivas, la Asamblea de Gobernadores o sus países miembros.

    facebook
    twitter
    youtube
    This site uses cookies to optimize functionality and give you the best possible experience. If you continue to navigate this website beyond this page, cookies will be placed on your browser.
    To learn more about cookies, click here
    X
    Manage consent

    Privacy Overview

    This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
    Necessary
    Always Enabled
    Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
    Non-necessary
    Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
    SAVE & ACCEPT