Development that Works
  • About

    This blog highlights effective ideas in the fight against poverty and exclusion, and analyzes the impact of development projects in Latin America and the Caribbean.
  • How to use modern technology to collect high-quality data

    3
    Aug
    2017

    By

    By Maja Schling

    The use of modern technology is more affordable than ever, and electronic tools now offer a cost-effective alternative to paper questionnaires to collect high-quality data. To help you decide whether the use of computer-assisted personal interviewing (CAPI) is for you, this blog reviews the advantages and potential pitfalls of using CAPI and shares a recent experience of conducting a survey in Guyana using free software developed by Survey Solutions.

    Photo: The Consultancy Group Guyana, July 2016.
    Using survey software for data collection in Guyana

    It is well known that certain technological innovations have allowed developing countries to “leapfrog” into the 21st century. For instance, the use of cellphones for mobile banking has enabled people in remote, rural areas of Africa to manage their finances and business transactions more effectively without having to rely on poor infrastructure. Tablets, projectors, and learning software have increasingly been used to improve the quality of education in regions where teachers lack skill and training, and learning materials are expensive and scarce. If technological progress can increase the pace of development, shouldn’t it also be used to improve the effectiveness of our work in developing countries?

    Paper questionnaires: the traditional way to collect data

    Development effectiveness implies that our strategies and policies are based on solid knowledge of various developing country contexts. To gain such knowledge, we need to collect high-quality data, often in the most remote and poorest regions. Traditionally, such data collection efforts have relied on the use of printed questionnaires, which are filled out in personal interviews with the affected individuals and/or community benefitting from a project. Oftentimes, the number of people to be interviewed lies in the thousands.

    Conducting such large surveys with paper questionnaires can therefore be costly in terms of resources, administrative, and logistical efforts and it presents many challenges: Printing and transporting questionnaires to and from the field is often associated with high cost; corrections to the questions can present a significant challenge in terms of cost and time. There is also a real risk of questionnaires being lost in the field or damaged by weather or transportation before data is entered.

    Even when all interviews have been conducted, completed paper questionnaires need to be entered manually into a computer file before data can be analyzed. This process is time and labor intensive, and prone to error. Quality checks of data are limited, and sometimes errors are only recognized when the survey has already ended, making them more difficult to correct.

    However, there is an alternative to paper questionnaires: computer-assisted personal interviewing (or CAPI in short). In recent years, CAPI has attracted more attention as it presents a more cost-effective way of collecting high-quality data.

    Photo: The Consultancy Group Guyana, July 2016.

    CAPI is becoming more and more popular

    With the improvement of computer processing speeds, the increasing global availability of internet service, and the falling prices of electronic devices, CAPI has become more and more attractive. The CAPI tool creates the questionnaire using a special software that can be directly downloaded to an electronic device (usually a smart phone or tablet), which the interviewer then uses to administer and fill out the questionnaire. Information from these questionnaires is then uploaded to a central server where it can be accessed and reviewed remotely.

    Depending on the sample size of the survey, the purchase of tablet devices to conduct surveys electronically quickly becomes cheaper than printing questionnaires on paper. Technical requirements for such devices are relatively low, and a large number of questionnaires can usually be saved on the device without danger of running out of storage. Additionally, once a questionnaire has been downloaded to a device, it is not set in stone: If an error is detected in the early stages of the survey, it can easily be corrected without incurring additional printing costs.

    Other significant advantages of CAPI are the frequent backups that can be made from electronic devices. While an internet connection is not necessary to conduct the interviews, questionnaires can and should be uploaded to a cloud regularly while the survey is still underway, so that data is not lost even if a tablet is stolen or destroyed. The administrators of the survey can also constantly check progress and review quality of uploaded questionnaires, and make immediate corrections if necessary. Since data entry is automatized and most software offers automatic checks of certain entry errors, CAPI promises that any survey will render high-quality data.

    Photo: The Consultancy Group Guyana, July 2016.

    The IDB’s experience with CAPI

    Based on these positive attributes, CAPI could be an effective alternative to paper questionnaires. Our team put CAPI to the test during a recent survey among 1,000 farmer households in Guyana in preparation for the IDB funded Sustainable Agricultural Development Project.

    For this data collection, the IDB cooperated with a team from the World Bank Group that has developed the free software Survey Solutions as a CAPI tool tailored to the technical and logistical challenges of developing countries. Survey Solutions recently reached the milestone of more than 5 million interviews conducted. Their system has been used by governments, NGOs, universities, think tanks and private firms in over 500 surveys across 94 countries. The software combines powerful functionality to capture data on tablets with survey management tools and cloud data aggregation. In addition, the technology allows for the collection of new types of data, such as GPS coordinates, time stamps, sensor information, and audio and video recordings.

    With the technical support of Survey Solutions, the team designed a survey and was able to streamline the data collection process from the design stage to data entry. The tools were intuitive and easy to use for both project specialists and the survey team on the ground. The ability to constantly monitor survey progress and check data quality enabled the project team to detect certain entry issues within the first few days of the survey and make adjustments immediately. Despite the remote location of many of the farming households (some could only be reached by boat), there were no issues with electricity or internet access. On the few occasions where technical issues were encountered, the Survey Solutions team was very responsive and continuously monitored server activity to ensure a smooth completion of survey activities.

    Of course, our experience revealed a number of pitfalls that come with the use of CAPI. Most importantly, the use of electronics required significantly more preparation time: For one, interviewers and supervisors needed to be adequately trained in the use of the software to administer and review questionnaires. Secondly, a major concern was regular availability of electricity (for charging devices) and internet (for uploading completed questionnaires) throughout the areas in which the survey was administered. This required a meticulous logistical plan and survey schedule that allowed for regular charging and data uploads, which the survey team had to adhere to with little room for delays or errors. Potential theft of and damage to the devices in remote areas have to be considered as well, though the team did not in fact encounter any such issues once on the ground.

    Could CAPI be an option for you?

    If you are wondering if CAPI could be for you, the answer will depend on a careful consideration of the context and requirements of your survey. Ask yourself the following questions:

    • Do logistics and infrastructure allow for regular access to electricity and internet during data collection?
    • Is there sufficient time to prepare the survey, including questionnaire design and piloting, as well as interview and supervisor training?
    • Would the purchase of electronic devices be more cost-effective than printing questionnaires and manually entering data?

    If the answer to these questions is yes, then you should take advantage of CAPI to collect high-quality data!

    ____________________________________

    Find more information on the use of CAPI software here.

    To start your own survey using Survey Solutions, click here.

     ____________________________________

    About the author:

    Maja Schling is a consultant in the Strategy Development Division (SPD/SDV) of the IDB. She holds a Ph.D. in Economics and specializes in the impact evaluation of development projects, with a focus on the areas of agriculture, environment, and natural resources.

    Comment on the post

    Subscription
    Categories
    Archives