Let’s suppose that we wanted to evaluate the broken windows policing theory developed by Kelling and Wilson (1982) and so widely implemented in New York City in the last few decades. Broken windows policing is based on the idea that enforcement of minor crimes should lead to a decrease in the realization of more serious crimes.
The argument is that neighborhood disorder and vandalism signal a generalized indifference by the State, which in turn encourages more severe criminal behavior; therefore, by reducing minor neighborhood disorder problems, police can effectively reduce major criminality issues.
So, if we were interested in measuring the impact of broken windows policing on crime, we would probably start thinking of a way of setting an experiment in which a sample of high-crime neighborhoods were randomly assigned to treatment and control groups, with treatment neighborhoods implementing broken windows policing.
But what if instead, we thought of going the other way around and decided to set an experiment in which we parked a considerable number of used cars with broken windows in half of the neighborhoods and did nothing on the other half?While the idea might initially sound kind of unrealistic and strange, it was actually implemented by Philip Zimbardo in the 60s and its results motivated the later development of the broken windows policing theory.
Furthermore, it is currently being used by economists Jens Ludwig from the University of Chicago, Sendhil Mullainathan from Harvard University and Jeffrey R. Kling from the Congressional Budget Office to encourage the implementation of this type of experiments, which they call mechanism experiments, to identify and understand the channels through which policies operate. Their NBER working paper “Mechanism Experiments and Policy Evaluations” develops the argument in detail.
Jens Ludwig presented this very interesting approach to experimental evaluation at the closing session of the AL CAPONE meeting that took place in Rio on June 23-24.
His lecture was quite appealing not only because of the innovative way in which he proposed to think about randomized evaluations, but because he explained how this approach could help solve external validity issues of traditional policy evaluations.
Given that mechanism experiments directly test the causal channel through which a policy works (as opposed to the policy itself), they should allow predicting how the characteristics of a given setting could moderate the effectiveness of a policy.
In addition, as mechanism experiments can be much less costly compared to policy experiments, they should allow more specific testing in different contexts and situations.
The idea here is not to think that mechanism experiments are better or should replace traditional policy evaluations, but rather to recognize that in certain circumstances mechanism experiments can provide useful information for policymaking. So why then, not start considering mechanism experiments on our coming evaluations?
Leave a Reply