Solving development challenges is a complex task. Rigorous research show that some programs do not work as intended and that most that do work have no huge impacts.
A sensible way to proceed when designing development projects is to be guided by three types of evidence: evidence in the identification of the problem, in the drafting of the solution, and evidence generated from the results.
Evidence-based diagnosis. It is important to identify as well as possible the development problem to be addressed, and to be based on evidence about its magnitude and determinants is necessary.
How big is the problem? What are its causes? The answers to these questions need to be supported by evidence.
Evidence-based solution. Besides being linked to the determinants of the program, there needs to be evidence about the efficacy, effectiveness or at least theoretical plausibility of the solution.
Ideally one would apply a proven solution, but that the fact that the problem exists in a given place means that a successful solution in the particular context has not been implemented.
So we need evidence that the solution has worked elsewhere, either in a similar context (second best) or that there is a mechanism that supports the intervention. If possible, the available evidence should be incorporated into guidelines on how to implement these policies. This is the process by which evidence on what works is incorporated into policymaking.
Generating evidence through monitoring and evaluation. Even if one identifies the problem with extensive evidence and designs a sensible solution based on rigorous evidence, the outcome in uncertain. Policymaking is full of good intentions with not-so-good or even bad outcomes.
Hence, it is always important to benchmark the results obtained with the evidence of how the program is working. Are the expected results and impacts being realized? Actual effectiveness needs to be gauged against what, based on evidence, one could expect.
An interesting illustration of this process is provided by Lawrence Sherman in his 1998 Police Foundation note on Evidence-Based Policing.
According to Sherman, “Evidence-based policing is the use of the best available research on the outcomes of police work to implement guidelines and evaluate agencies, units, and officers… evidence-based policing uses research to guide practice and evaluate practitioners. It uses the best evidence to shape the best practice. It is a systematic effort to parse out and codify unsystematic ‘experience” as the basis for police work, refining it by ongoing systematic testing of hypotheses.”
These principles can and have been applied to areas as diverse as policing (Sheman’s note shows the example of policies to reduce domestic violence), medicine and education (interesting examples are provided in Super Crunchers, by Ian Ayres).
The hardest point, illustrated by Sherman and Ayres, is making people change their behavior, to create a culture in which evidence dominates the expert’s opinion, the rule of thumb or the gut-feeling of what should work.
Implementing development programs in the field may be a daunting job that requires passionate and committed people to run them. The challenge is to direct that passion and commitment not with a particular intervention, but with solving the development challenges that justify working on a given intervention.
This will provide with the flexibility necessary to adapt and learn from the evidence, and ultimately enhance development effectiveness.
Leave a Reply