Public policies and development cooperation have shifted toward an evidence-based approach. “Evidence” is no longer the exclusive domain of forensic and other scientists: the term is now regularly featured in the design and implementation of public policies in both developed and developing countries.
Monitoring and evaluating development projects helps make the causal link between a project’s outputs and its intermediate outcomes clear and observable. Regional cooperation projects face greater challenges than other development projects when it comes to shedding light on the attribution chain between project implementation and expected effects. Despite the complexities involved in this process, the international community has shown a growing interest in rigorously evaluating development interventions in the framework of a management for development results approach.
Since 2005, the Inter-American Development Bank has financed 154 regional cooperation projects in Latin America and the Caribbean through its Regional Public Goods (RPG) Initiative, granting a total of US$112 million in nonrefundable resources.
RPG projects are selected through public calls for proposals and must be submitted by at least three countries from Latin America and the Caribbean seeking to address a common development challenge or seize an opportunity through collective action and cooperation, creating more value-added than national-level interventions.
An RPG project can finance, for example, the development of an IT tool for public management, meetings for sharing of information and best practices among officials from different agencies, and outreach events. The expected intermediate outcome of such an intervention would be the adoption of the new public management system by a certain number of public agencies. Up to this point, the correlation between outputs and outcomes is observable and measurable. With a certain degree of confidence, we can assert that the fact that the public officials participated in the exchange of information and best practices and were exposed to the new public management tool is directly correlated with its adoption. Although variables like political will and opportunity windows for reforms could explain why only some of the participating governments adopt the tool whereas others do not, these factors do not call into question the fact that the explanatory variable (that is, the driver of change) is the RPG project.
So why is it more challenging to measure the impact of regional cooperation projects?
First, although RPG projects are regional by nature, they are implemented at the national level, which involves dealing with different national institutional factors. For example, an RPG project that seeks to implement changes in school infrastructure regulation hinges on strictly national variables such as the political leverage of the technical focal point in the participating agency, the government’s agenda, and past reforms.
Second, the time needed to observe the impact of an RPG project is greater than the one needed for a project implemented at the national level. Back to the school infrastructure example, an intermediate result of an RPG project would be the improvement of school infrastructure regulations in participating countries. One might infer that improvements such as higher square footage per student and increased comfort conditions would result in better school performance among students from participating countries. After controlling for different variables, researchers could observe whether students at the schools that complied with the new regulations performed better than those at schools built under the old regulations.
Lastly, some projects promote changes and reforms with tangible and observable benefits, such as a shared protocol for government procurement of pharmaceuticals which generates aggregate savings of more than US$40 million, thanks to the collective purchasing power that allows the health ministries of eight Central American countries to pay less for pharmaceuticals by guaranteeing larger purchase volumes for tendering companies. However, projects aiming at strengthening regional working groups on various issues, consolidating networks, or facilitating the exchange of experiences may find it harder to identify a causal link between outputs and expected outcomes, and could present an omitted variable bias.
To tackle these constraints, both timing and methodological considerations need to be factored in when evaluating regional cooperation projects.
Therefore, the evaluation should be included in the design phase of regional cooperation projects to allow data to be collected from the outset, facilitating the creation of baselines and increasing efficiency through, for example, the use of administrative data that participating agencies may already be gathering for other purposes. Furthermore, a creative approach will be required when selecting the evaluation methodology, which may involve both quantitative and qualitative methods. These could range from quasi-experimental or experimental methods, depending on the nature of the project and the possibility of identifying a control group, to the use of public perception surveys, focus groups, and other qualitative tools.
Developing rigorous assessments that consider the abovementioned factors specific to regional cooperation projects will allow the collection of evidence on both successes and challenges in their implementation. Gathering results for regional cooperation projects that can be generalized, without incurring in external validity problems, will be useful in understanding which institutional arrangements make the provision of public goods in Latin America and the Caribbean successful. The identification of patterns of association will help provide a clear outlook on how international cooperation is taking shape in the region and what the best ways of facilitating sustainable development are.[1]
[1] Estevadeordal and Goodman (2017) explore the relationship between international cooperation, regional public goods and sustainable development. Among other aspects, the authors inquire into the ideal institutional framework for providing regional public goods that will contribute to sustainability in Latin America and the Caribbean.
Leave a Reply