The puzzle of unlikely success

When I engage with clients in the field about project execution and impact, I invariably encounter a puzzle.

Our clients usually point out that things would be going much better if it weren’t for the poor technical and management capabilities of the executing agency.  And they are right.  A lack of capacity is the headwind faced by most public sector initiatives in our countries, and a predictable explanation for middling results.  

And yet, even as they discuss these shortcomings in capacity, most clients are also quick to share stories about what I call “unlikely success:”

• A fraud-detection application developed by a small team of software developers in the Brazilian government that generated half a billion dollars in savings in payroll and pension payments in less than six years.
• A water utility in Haiti that developed and implemented a strategy for distributing water reliably in the lowest-income neighborhoods.
• A police reform in Honduras that contributed to a 60% reduction in the homicides.
• A public-sector procurement system overhaul in Colombia that quickly achieved 20 percent price reductions across all product categories.

There are countless other examples. I call these successes “unlikely” because they took place in countries and organizations where capacity is widely regarded as very weak and where—in theory—achieving these outcomes should not have been possible.

So, what happened? Do these success cases have common features? And if they do, how can we, the IDB and our clients, work together to harness those features and make them a consistent part of the way we design and carry out projects?

Old and new theories of change

Of course, talking about implementing effectively, achieving results, and building capability takes us back to very old discussions. In the quest to identify the essential drivers of development and improved wellbeing, state capacity has played a fluctuating and sometimes contentious role.  

In the 1990s and the early part of this century, building state capacity was a central objective of country-level strategies by multilateral development banks. The IDB, the World Bank and other MDBs financed ambitious capacity-building programs, dispatching teams of international consultants who introduced the latest approaches to solving problems in areas ranging from tax collection and procurement to provision of basic services (water, education, health) and judicial reform.

Many of these efforts did not meet expectations. Even when the short-term project objectives were met, the boost in capacity tended to melt away after the project concluded and consultants departed, exposing the original institutional weaknesses that the operation had been intended to overcome. The critical convergence of organizational and agency skills, experience, versatility and continuity seemed to always remain out of reach.

Many development practitioners began to doubt whether such programs were worth the effort. Some specialists questioned the conceptual underpinnings of capacity-building strategies. Others lowered expectations by making capacity just one more item on the checklist of desired outcomes in a development program.

At the IDB, most of us never stopped believing that capacity is foundational in development. Research by Lant Pritchett and others at Harvard University has given even greater conceptual and empirical weight to the notion that state capability is an essential prerequisite to progress across the whole spectrum of actions that lead to improved well-being and greater opportunities. Recently, Pritchett has shown, strikingly, that around 90 percent of the cross-national variation in measures of human well-being is accounted for by just two elements of national development: labor productivity and state capacity.  But at the same time the standard on state capability and rule of law has barely improved in over two decades—and in some cases it has actually deteriorated. Latin America is part of what has been called the “big stuck” in state capability. 

If we agree that state capability—the ability of public sector organizations and agencies to find effective solutions and implement them—is essential, but at the same time acknowledge that efforts to build capacity have been disappointing, where do we go from here?

One possible roadmap can be derived from alternative “theories of change” that have more recently emerged among researchers and practitioners in the capability community. While any distinction into two approaches will be crude and miss nuances, there are two broad narratives of capability.

Theories of change: Two broad narratives of state capability

A traditional—and still popular—narrative is the “solution driven” approach.  In this narrative, better outcomes are mainly achieved by the introduction and implementation of already known and fixed solutions.  The adoption of better de jure policies and regulations, perhaps those that are regarded as global “best practice” based on evidence and experience from cutting edge countries, is seen as “the solution”—both in effective action and in creating capability. 

This inevitably leads to two ideas on building capability.  The first is that the path to capability is the de jure adoption of generic features of “ideal” public sector organizations, like open, meritocratic recruitment, transparency, and strict controls on front-line workers.  The second aspect of the “solution driven” model is that the capability to be built is the capability to implement an already known solution.  This leads to a top-down, “compliance-driven” approach to organizations and has been the dominant model in MDB-financed projects and capacity programs over the past three decades.  It is this model and its “theory of change” about building state capability that has been part and parcel of the “big stuck.” 

In my own research into the characteristics of successful leaders in various public sector organizations (in policing, in education, in industrial policy) in Brazil, I have found that this “compliance-driven” approach almost never leads to effective results. Instead, I found that successful public sector leaders were those that operated at the “fringes of formality” and set out to discover workable solutions in their own local context.

And in my own recent experience of managing a large organization in Brazil that was responsible for collecting reliable data in the midst of the COVID crisis, I learned the hard way that attempting to impose “best practices” in a top-down way would neither solve the problems we faced or build the long-term capabilities the organization needed. 

My research and experience are part of a large, broader, shift in thinking about state capability.  Faced with the limitations of the “solution driven” model that emphasizes “process compliance” in adopting fixed and known solutions, and building capability only to comply, practitioners and academics have been exploring a variety of new approaches for assessing and overcoming capacity problems.

I call this new current the “solution-search” approach, and it encompasses many variants.  These include bottom-up, “problem-driven” approaches to capacity building such as Doing Development Differently, Problem Driven Iterative Adaptation, Thinking and Working Politically, and Collaboration, Learning and Adaptation, among others. The PEFA framework and the “Country Policy and Institutional Assessment” are examples of methodologies currently being used by the World Bank, the African Development Bank and donors such as DFAT, DFID, and USAID.  

These approaches share an understanding that more effective organizations emerge when purpose-driven people work to solve pressing problems. In other words, organizations build capacity by solving problems. Whereas a compliance-driven approach builds capability in the abstract and then attempts to apply that capacity to the implementation of “known” solutions, a solution-search approach builds capacity by the organization working to find feasible and locally constructed solutions (drawing on evidence from elsewhere but not slavish to it) to pressing problems.

Four characteristics of successful solution-search approaches

Which brings us back to the puzzle of “unlikely successes.”  In talking with IDB colleagues about their own examples of “unlikely success” in the field, we often find some of the core principles that have emerged from the “solution-search” paradigms described above.

This blog briefly describes four such principles in relation to specific IDB operations in Brazil, Honduras, Ecuador and Paraguay. It argues that these principles should increasingly guide the way the IDB selects, designs and implements projects, especially in settings were capabilities present a challenge. And it proposes that we further pilot and evaluate these approaches to ensure that they fully reflect our client’s needs and expectations.

First, states, organizations and institutions that harness a solution-search approach come to understand the complexity of a given problem as a whole – not in a fragmented manner.

This means that they start by deconstructing the problem, engaging with its complexity, and fully understanding its root causes. Only after this is done do they design solutions to address the problem.

The IDB can be a critical ally for governments that want to adopt this approach. In Brazil, for example, several state governments are trying to accelerate digital transformation in public services. But in a context of fragmented digital governance, conflicting regulations and incompatible technologies, some state authorities are wary of prematurely investing in “solutions” that ultimately turn out to be incompatible with local realities.

To avoid this mistake, several state governments have sought the IDB´s assistance under the Bank’s “Brasil Mais Digital” program. The program has financed preparatory work to fully understand the existing landscape of digital skills and practices, bring order and coherence to the relevant governance and regulations, and empower a single governing body that can ensure an effective implementation of future investments.

In the state of Alagoas, these activities included a public opinion survey designed to test assumptions regarding the “digital divide” and public attitudes about digital services. The survey revealed much higher levels of broadband penetration and digital literacy among the local population than had been assumed by the government. The IDB also recommended conducting a thorough assessment of the digital skills of career civil servants who will be the main users and implementers of any digital transformation projects. That survey helped to pinpoint specific capability gaps that will need to be addressed to ensure that rank-and-file civil servants are ready and able to use new technologies from day one.

As a result of these and other IDB-financed activities, authorities in the states of Alagoas, Ceará and São Paulo have embraced an integrated and transversal strategy for digital transformation that stands in stark contrast to the fragmented, wasteful and short-lived efforts of the past. Such an approach to understanding the “problem as a whole” increases the odds that digital investments will meet their intended objectives and enjoy sustained uptake from both citizens and civil servants.

Second, the solution-search approach starts from the assumption that states build capacity “from the inside out.”

This means that development agencies don’t try to fix hollow organizations–they try to fill them. Instead of using ad-hoc reforms (such as improving human resources processes, adding transparency requirements, or introducing information technology), agencies focus on building human capacities around a concrete purpose so as to strengthen the “core” of those organizations.

The quest for law enforcement reform in Honduras offers a compelling example of this principle. In 2011 Honduras faced a political crisis because of a surge in high-profile homicides.When the government approached the IDB for help, discussions initially centered on a traditional multi-pronged approach that would include the construction of new security infrastructure, acquiring better equipment and systems for law enforcement, and a variety of social “conflict prevention” programs in high-risk communities. Honduran authorities rejected this approach, arguing that the fundamental problem was the lack of a professional and well-trained police force. They explained how the existing system for training police officers was based on foreign recommendations that did not reflect local realities and consistently failed to create a stable force capable of investigating and preventing crime.

In this context, Honduras asked the IDB to finance a pioneering program that would focus resources almost entirely on creating a comprehensive system for training and professionalizing officers. The program would establish a modern police academy and prioritize gender equality in the police force, while strengthening internal control mechanisms. Honduran officials insisted that this was the only way to simultaneously improve the capability of the police in Honduras while rebuilding institutional legitimacy and restoring public trust. This approach faced considerable skepticism at the time, but the Bank ultimately approved a program that would introduce an entirely new police curriculum, raise the minimum education requirements for new recruits with a gender approach, extend the basic training period from three to eleven months, and offer more efficient and respectful security services to citizens.

The program has since recruited and trained 14,000 officers, including more than 3,200 women. It has introduced specialized training to key actors across the criminal investigation system, established an integrated platform for data exchange and crime analysis, and boosted the capacity of municipalities to prevent crime. The national homicide rate has dropped from 91 per 100,000 in 2011 to 35.8 in 2022, and several public opinion polls have shown a sustained improvement in public confidence in the police force.

Security continues to be a critical issue for Honduras, and the reform effort is far from complete. But by insisting on building capacity from the inside out, Honduras is making steady progress on one of the region’s most intractable development challenges.

Third, a solution-search approach requires iteration and adaptation.

Even the best-designed programs can fail when confronted with fast-changing circumstances on the ground. Success requires frequent and agile interactions to address clients’ shifting needs, flexibility to quickly change implementation, and qualitative and quantitative data to monitor and track progress.

During the COVID-19 pandemic public health authorities in Ecuador were under enormous pressure to contain the spread and impact of the disease—all on a woefully insufficient budget. Following WHO guidelines, Ecuador implemented a COVID Strategic Preparedness and Response Plan with the support of IDB financing from the Contingent Credit Facility (CCF) and other investment loans. Ecuador’s Ministry of Health and Bank specialists quickly agreed on priority interventions for the most severely affected parts of the country.

But even as the first disbursement was being made, information collected on the ground showed that local conditions were changing. From one day to the next, for example, the need for COVID tests and personal protective equipment at individual hospitals became less urgent, even as the number of COVID patients requiring hospitalization and ICU support increased exponentially.

Like many countries in the region, Ecuador now has data systems that yield timely patient-level information. This gave authorities the potential to precisely target limited resources for the greatest possible impact—but only if the implementation of the COVID response effort could be adjusted “on the fly.” Leveraging this institutional capability, the IDB coordinated with the authorities and project execution teams on a near-daily basis, rapidly making adjustments to project execution plans based on the latest information. This flexibility not only allowed the efficient use of resources; it also improved health outcomes by delivering aid to the right place, at the right time.

Fourth, the solution-search approach includes working through pilots and leveraging their demonstration effects when they succeed.

Lasting solutions grow out of well-executed pilots, but development agencies have an essential role in helping governments to apply insights from these pilots and scale them effectively.

As part of its IDB-financed Public Sector Transformation Support Program, for example, Paraguay is seeking to advance ambitious reforms of its civil service and public procurement systems.

But instead of starting the process by drafting a comprehensive reform law based on best practices in OECD countries, the government and the IDB are working together on an approach that includes a series of “pilot reforms.” These pilots are deliberately designed to be modest in scope, in order to field-test specific changes in the Paraguayan context and generate the capabilities needed in the public sector. To determine the viability of a new approach to hiring senior civil servants through a competitive process, for example, one pilot is carrying out a “trial run” with a limited number of candidates through a voluntary approach in a few ministries. This pilot is also incorporating technical assistance from specialists in other Latin American countries that are implementing comparable reforms. A similar approach is being taken to pilot new approaches to job evaluation and pay policy.

The goal is to ensure that the draft reform laws that are eventually presented to Paraguay’s Congress will feature changes based on real experience, and that the lessons learned with the pilots will also ensure public servants are ready to implement the proposed reform. This is expected to increase the credibility of the proposed law in the eyes of legislators who have seen many “imported” reforms fail in the past.

In that sense, “piloting the problems” is helping to improve both the technical strength of the proposed reforms and their political viability in the legislature, thus helping to secure the votes necessary for their approval.

The road ahead

These four brief examples show that solution-search approaches are already being applied in many IDB operations, and that Bank specialists have been at the forefront of enabling these approaches in a wide spectrum of sectors and settings.  But often these successful efforts, instead of being supported and enabled by the internal processes of the IDB, have occurred because staff themselves operate on “the fringes of formality.”  The challenge is not getting people at the IDB to discover ways to work effectively: that knowledge is already here. The challenge is to see, surface, and acknowledge the principles behind that success and create a culture that makes them consistently achievable in practice.

The question, of course, is whether we can leverage these experiences to improve state capacity at a large scale. The honest answer is that even when we know what works and what doesn’t in particular settings like the ones described above, we still have much to learn about how to replicate these isolated best practices across large bureaucracies.

As the IDB drafts a new institutional strategy, I think we should acknowledge this fact and confront the challenge head on. Our experience so far shows that in addition to providing prescriptive recommendations and protocols to enhance state capacity, we need to foster “bottom-up” processes of engagement and iteration similar to the ones I have briefly summarized. In other words, we cannot have development “without the struggle” of finding, testing and refining solutions in the field.

I am convinced we can achieve such a balanced approach through four mutually-reinforcing lines of action: incentivizing the adoption of best practices that have demonstrated impact on the ground; enhancing the ability to measure those efforts; extracting lessons that can be replicated and mainstreamed in our operations; and finally, creating the space for the IDB—and our counterparts—to “formalize” these informal practices.

By showcasing the power and impact of solution-search approaches, we have an opportunity to expand the IDB’s role as a catalyst for lasting, home-grown progress that harnesses institutional capacity to enable development effectiveness.