It was not a surprise, but certainly an encouraging confirmation: the use of rigorous impact evaluation methodologies has spread beyond social sectors.
I just participated in the conference “Mind the Gap: From Evidence to Policy Impact” and I was glad to see that 8 out 38 parallel sessions were devoted to discussing impact evaluations of programs and projects aimed at fostering production in various industries (a cluster of interventions often referred to as Production Development Policies, PDPs).
During the three-day event in Cuernavaca, researchers, evaluators, policy makers discussed evidence on the effectiveness of matching-grants for SMEs’ innovation, technical assistance for export diversification, extension services for adoption of technologies and practices by farmers etc., and specific methodological issues related to the evaluation of these interventions.
It was certainly clear that a gap with the social sectors is still there. In the case of PDPs, the use of experimental design is still quite limited, mainly because impact evaluations have often been conceptualized ex-post, instead of being included in the project design.
However, according to the conversations in Cuernavaca, also this seems to be changing: an increasing number of PDPs, in particular in the agricultural sector, are incorporating impact evaluations as a component of their design. We can therefore expect an increasing number of randomized studies of PDPs in the next years.
The meeting in Cuernavaca gave testament of the significant efforts that have been made to make the best of many opportunities to start generating evidence on PDPs based on rigorous quasi-experimental approaches.
Following the motto “better LATE than nothing”, researchers and evaluators have produced a significant amount of impact evaluations based on rigorous application propensity score matching, difference-in-difference, instrumental variables and regression discontinuity methodologies.
These studies have generated interesting findings on what policy tools work, what kind of beneficiaries benefit the most, what time frame is needed to detect their impacts.
Although this literature is still too young to have reached any conclusive judgment, for sure its findings are moving the policy-making debate on PDP towards the discussion of evidence.
For instance, the discussion on the need for programs aimed at fostering SME innovation has for long been dominated by rather strong ideological positions in favor and against. Nowadays, a growing literature is showing that matching-grants for SMEs’ innovation may really be effective both in increasing R&D investments in the short run, and productivity in the long run.
Certainly much more evidence is still needed, but at least the debate on this specific instrument have shifted towards discussing evidence (and the rigorousness of the studies through which this evidence is generated) rather than ideas.
I was also very glad to see Guido Imbens chairing a session on innovation and productivity. The session was organized in the context of a research project that aims at assessing the impact of multi-treatments in innovation and export promotion, project in which Imbens participate as scientific advisor.
Imbens first provided some methodological insights on how to deal with multi-valued treatments and, then, he coordinated a vivacious discussion on two impact evaluations of a Chilean export promotion program and an Argentinean SME innovation program.
Seeing such an outstanding scholar devoting time and attention to the evaluation of innovation and export promotion policies has been another very encouraging sign of the new trend in the impact evaluation of PDPs.
Leave a Reply