Can you assess early childhood development remotely? Distance assessment of preschoolers

Many of the world’s school systems struggle to monitor student learning even in the best of times. As face-to-face learning assessments ceased because of pandemic-related lockdowns, school systems were left in the dark about the impact of school-closures and distance education on student learning and the development of gaps. Absent hard data on learning, school systems are unable to design effective strategies for distance education, remedial education and return to school.

In response to the challenge of learning assessments during school closures, Innovations for Poverty Action (IPA) and the Inter-American Development Bank (IDB) partnered to develop a tool for distance assessment of learning in Latin America.

UNESCO warns that learning losses will likely be greater in the early grades, as these students are less able to learn on their own. We therefore opted to design and validate a remote learning assessment for 4-6-year-olds. Our priority was to gain information on the development of early skills in mathematics (e.g. counting, number comparisons, spatial reasoning); and literacy (e.g. sound identification, listening comprehension and expressive vocabulary); as well as socioemotional development (e.g. empathy and conflict resolution skills). To measure these skills, we developed a remote version of questions from two tests: Measuring Learning Quality and Outcomes and a preschool version of the Early Grade Mathematics Assessment.

The good news is that distance assessment of early childhood development is feasible. We just completed an initial feasibility study of the distance assessment in Peru, providing qualitative evidence that the test works. Children maintained their attention during the 20+ minute test and understood the instructions, and caregivers were satisfied. We found no evidence that children or caregivers were anxious about the assessment. Overall, we consider that there is high potential for applying the test during COVID19-related lockdowns and beyond as a complement to in-person assessments.

To evaluate the reliability and validity of the remote assessment, we are developing a pilot in a larger sample. Meanwhile, three lessons learned stand out:

• Some early childhood skills are more challenging to measure remotely. Asking the child to count or name items in certain categories (i.e. expressive vocabulary) was straightforward to adjust to a remote format. On the other hand, domains such as empathy, spatial sense and grouping objects were more challenging to adapt for remote testing as they require the use of visual aids and non-verbal responses.

• Remote learning assessments require different modes of application depending on the connectivity of each household. To ensure that children from any type of household can be included, we developed a video and an audio format of the test. All images in the video test had to be fitted to a small screen as people tend to access internet through smartphones in Latin America (e.g. 89% of Peruvians with internet access connect through smartphones). Based on parental preference, the video format of the assessment can be administered either as a unilateral video – where the child cannot be seen by the enumerator but the child can still see the enumerator’s screen – or a bilateral video where the family and the enumerator can see each other.

• Caregivers play a key role in remote assessments in early childhood. Protocols for interactions with caregivers are central for everything from test scheduling and informed consents to monitoring the quality of the audio and mitigating parental influence on outcomes.

Stay tuned for future posts with the results from the reliability and validity findings from the larger pilot, which will help answer questions such as: Does the instrument accurately measure early skills in mathematics and literacy? How about socioemotional skills? What is the impact of different the formats of surveying on cognitive and non-cognitive outcomes?