Nourishing impact evaluations: surveys and quality of data matter

A joint post by Gastón Gertner and Fazia Pusterla

 

Most impact evaluations rely on data collected through survey methods. However, it is often the case that evaluators find themselves in front of low quality databases due to poor data collection processes in the field.

Research analysts at government offices and in academic environments spend a significant amount of their time cleaning data in order to correct this problem.

At the IDB, we are trying to move the impact evaluation agenda towards enhancing access to high-quality survey data.

Thus, during the last Impact Evaluation Workshop in Chile, covered in this post, all afternoon sessions were dedicated to induce participants into thinking about the importance of setting up high quality standards and good practices to collect data.

We need a deeper focus and understanding on survey methods and instruments.

The Survey Methods sessions in Chile tried to reinforce basic concepts around probability and sample theory. But, insofar as the approach of the workshop was practical, participants were also encouraged to become acquainted with aspects related to the design of survey instruments, the implementation of quality control mechanisms to supervise data collection teams and the use of technological tools to ensure the quality of the data collected.

One of the key objectives of the workshop was to raise awareness on practical issues that delve on understanding the way survey firm teams operate in the field. To that end, the survey methods sessions brought participants to think about real challenges one can face while doing an impact evaluation.

Through a problem solving approach, participants were asked to work, for example, on sample designs, power calculations as well as in the preparation of survey modules and of budget and team building considerations.

There have been excellent previous Impact Evaluation workshops, and following recommendations stated here and here, and it is important to insist on the importance of high-quality survey data that feed impact evaluation studies in Latin America and the Caribbean.