Dr. Jack Buckley is President and Chief Scientist and Zeina Fayyaz Kim is the Chief of Staff at Imbellus, a simulation based assessment company. Formerly Senior Vice President of Research at the College Board and Commissioner of the National Center for Education Statistics, Dr. Buckley has a background in applied statistics and education research, and together with Zeina are special guests to our blog series on the development of #skills21 in Latin America and the Caribbean.
The world is starting the school year on the back foot. Due to COVID-19, more than 1.7 billion students worldwide have received fewer hours of instruction and much of it at lower quality than usual. In the United States, primary and secondary students returned in fall 2020 with just 70% of the reading gains and less than 50% of the mathematics gains relative to normal conditions. In other words, U.S. students started the new school year about as ready as they would have been in February. And, learning losses will continue to pile up. If schools operate at less than full-time and instruction quality continues to suffer, the average U.S. student could fall seven months behind. This impact will be worse for disadvantaged students. Black students may fall 10 months behind, Hispanic students nine months, and low-income students more than a year. Internationally, the picture is similar. The Brookings Institution anticipates a global loss of 0.6 learning-adjusted years of schooling (LAYS), a measure developed by the World Bank to facilitate international comparison that combines quantity as well as quality of schooling into a single metric.
Naturally, educators heading back to work this fall need an accurate diagnosis of their students’ COVID-related learning loss. A good diagnostic will show which students need remediation, illuminate macro learning gaps across a class, and enable educators to prioritize their time and resources. Schools and governments are rightly prioritizing diagnostic assessments to guide their actions in this unconventional year.
That said, we must not lose sight of the bigger picture. While it is urgent to diagnose and recover learning losses, the education of a child is not a tick-box exercise. Most countries agree that the purpose of education is to equip children with a baseline of knowledge and skills for successful adulthood (see, for example, the mission statements put out by the education ministries of the United States, Finland, and India). If we believe these high-minded statements, we owe it to our children to aim for better than net zero learning loss.
Unfortunately, during an economic crisis, more resources for diagnostics and remediation means fewer for innovation and advancement. So, in our desperation to ensure students are getting the basics, we risk losing hard-won ground on #skills21 and deeper learning. We must recommit to our 21st century goals. Frankly, we can’t afford not to—we are already behind in preparing our youth for the future of work. Automation is driving mass worker displacement. More than half of workers worldwide will require significant reskilling in the next two years. And, this burden of reskilling will fall harder on countries further behind the technology curve.
We believe that a new kind of assessment—built to mimic the demands of real work and real life—is our best hope to keep schools on track to meet the 21st century. Assessment is the incentive system by which the education wheel turns. Today’s high-stakes tests (e.g., SAT/ACT) incentivize students to master what machines already rule—content knowledge and academic skills, such as mathematical computation—when we need to incentivize the more complex #skills21 that will remain distinctly human, such as critical thinking, creativity, and collaboration. Andreas Schleicher of the OECD puts it well, “We have to find and refine the qualities that are unique to our humanity, and that complement, not compete with, capacities we have created in our computers. Schools need to develop first class humans, not second-class robots.”
Game- or simulation-based assessments are one answer to Andreas’s call. When simulations are used as tests, the graphics, sounds, and real-time feedback increase test-taker immersion, thus inspiring more natural behavior and better capturing how a person would act in real life. Critically, simulations can assess a test-taker’s process, not just her final outcomes. We can see where you hovered your mouse, in what order, for how long, and thus follow your thought process. The promise of game- or simulation-based assessment is enormous. Forward-thinking companies and universities are already making the switch. The University of California recently pledged to look into developing simulations and realistic performance tasks as a replacement for the SAT/ACT to better evaluate “creative problem solving, inductive and deductive reasoning, and analytical capability because these attributes are hallmark features of what has come to be expected out of higher education…[and the] new economic reality.”
Imbellus makes the world’s most advanced simulation-based assessments for leading-edge companies and organizations like McKinsey & Company and the U.S. military. We deploy live testing 24/7 in more than 30 countries and six languages. However, our goal has always been to bring this kind of assessment to the K-12 and university admissions contexts.
In general, our computer-based assessments evaluate complex thinking skills in a game-like environment. For example, the McKinsey scenario asks test-takers to select plants and animals that will thrive as a complete, contained ecosystem. The premise is simple, but test-takers find themselves wading through an abundance of qualitative and quantitative data, which makes the task difficult and not dissimilar to the job of an entry-level analyst at McKinsey. As test-takers navigate the scenario, our system collects telemetry—think of it as clickstream—data and interprets it to form item-level scores. The logic for those scores is handcrafted by our learning scientists, data scientists, and psychometricians. We use artificial intelligence and machine learning to generate numerous different versions, or “forms,” of our scenarios that thwart cheating while preserving equivalence.
Simulation-based assessment is the innovation we need in education. As it stands, only 9% of 15-year-olds worldwide can reliably differentiate fact from opinion in written form. This is sobering in the era of fake news, bots, and social engineering. Because simulations can assess deep thinking skills, educators teaching for simulation-based tests will be empowered to give students more complex assignments. For example, what if, instead of lecturing middle schoolers about COVID-19, we let them research and devise approaches to reduce the spread of the virus at their school? Without replacing content learning entirely, we must give children more opportunities to think for themselves and practice the skills they’ll need to thrive as 21st century adults.
Pre-COVID, Imbellus received a grant to begin adapting our assessments for secondary school students. We are pilot-testing our high school version 1.0 in the United States, and early results are promising. We have much more extensive field-testing ahead of us, but it is exciting that we now have a promising prototype for a fundamentally better test that is cost comparable to traditional assessment.
However, with schools now operating on very tight budgets, the only way we can deliver 21st century assessment is with increased government involvement and investment. If governments do not step up, we run the risk of narrowing the curriculum just as we have begun to make progress in measuring and prioritizing deeper thinking skills. In Latin America and the Caribbean, where governments are rightly concerned with other priorities, business leaders, development agencies, and philanthropy may have to pitch in. The foundational step is improving the region’s technology infrastructure and connectivity so that innovations like simulation-based assessment can reach all schools and all students. At scale, simulation-based assessment can shift Latin America and the Caribbean into a virtuous cycle where better assessment begets 21st century preparedness begets a new generation of innovators and innovations coming out of the region.
Stay tuned and follow our blog series on education and #skills21 in times of coronavirus. Read the first entry of these series here. Download the Future is now and don’t forget to keep an eye out for our news!
How do you think games could be used to assess the learning of students in Latin American and Caribbean countries? Share your opinions with us in the comments section below, or in Twitter mentioning @BIDEducacion #EnfoqueEducacion.
Leave a Reply