Scaling effective educational interventions remains a major policy challenge. Programs that succeed at a small scale often lose impact when expanded—a phenomenon researchers call the “voltage effect.” This is especially true for interventions that depend on intensive human interactions such as counseling, coaching or mentoring, where cost and availability of skilled personnel limit expansion. The question facing education leaders and policymakers is clear: How can we scale effective educational interventions while maintaining impact? Generative AI emerges as a potential answer to this problem, offering the possibility of maintaining quality while reducing costs and resource demands.
Scaling the “Quiero Ser Profe” program in Chile
Elige Educar’s “Quiero Ser Profe” (QSP) program in Chile exemplifies this scaling challenge. The program’s goal is to increase enrollment in teacher education programs by providing personalized counseling via WhatsApp to high school students, motivating them to consider teaching careers. Efforts like QSP are critical today because Chile – like other countries in the region and globally – faces a severe teacher shortage, with estimates projecting a deficit of over 30,000 qualified teachers by 2030. QSP relies on human counselors who must be recruited from teacher education programs, extensively trained, and carefully supervised. This human-intensive approach creates an inherent ceiling on the program’s reach, limiting the number of high school students it can ultimately serve. Prior to 2020, the program reached only 2,400 (6%) of approximately 40,000 high school students who expressed interest in learning more about education majors.
Recognizing this scalability challenge, in 2020 the IDB partnered with Elige Educar to explore whether technology could help replicate the counseling provided by human tutors and enable the program’s scale-up without sacrificing its impact. This first attempt at scaling involved a scripted chatbot available via WhatsApp that was designed to deliver the same core messages and information as the human counselors. The scripted chatbot relied entirely on pre-written messages and button-based menu options, without any ability to generate personalized responses like modern AI chatbots. This intervention was evaluated through an RCT with over 40,000 students (open access here).
The results were promising in terms of reach: with the scripted bot, the program was now able to reach most of the students interested in education majors. However, the bot was not effective in motivating students to consider teaching careers. Human counselors increased students’ likelihood of ranking an education program as their first choice by 1.25 percentage points—a 9.5% increase over the baseline—and boosted enrollment in education programs by 0.9 percentage points, representing an 8.5% increase. By contrast, the scripted bot’s effects were smaller—roughly one-third to half the size—and statistically insignificant. The results underscore a tradeoff: bots were cheaper and scalable, but less impactful, likely due to their limited engagement skills and personalization, two key attributes of effective human counseling.
Chat agent Kai: Leveraging Generative AI for Quality at Scale
In 2024, building on insights from the previous study and advances in generative AI, we partnered again with Elige Educar to pilot a new approach to scaling QSP. We developed Kai, a WhatsApp-based AI chatbot designed to mimic the natural, personalized support of human counselors. Kai adapts to student inputs to offer tailored guidance, aiming to combine AI’s scalability and cost-efficiency with the engagement that makes human counseling effective (Figure 1).
Figure 1. Screenshots of chat agent Kai via WhatsApp
During the 2024–25 admissions cycle, we ran an RCT with over 43,000 students across three groups: those receiving support from Elige Educar human counselors via WhatsApp, those receiving support from Kai via WhatsApp, and a control group receiving no intervention.
Early Findings: Promise and Key Insights AI-Powered Education Career Counseling
So how do students engage with the chat agent Kai, in comparison to the human counselors in the Quiero Ser Profe program? Early analysis of students’ interactions in both treatment arms show how students use AI differently than human counselors, revealing important patterns for scaling educational interventions. Four key findings emerge:
- More students appear to be more willing to begin interactions with Kai (35%) than with human counselors (28%) (Figure 2). This suggests that there may be a lower barrier to initial engagement with AI systems, potentially expanding access for students who might be hesitant to interact with human counselors. For example, a student who worries about being judged for their academic performance or uncertainty about their future might find it easier to start a conversation with Kai, knowing they can take their time to respond and won’t face judgment from another person.
Figure 2. Initial engagement, Kai and human counselors
- While students are more likely to engage with Kai initially, conversations with Kai tend to be shorter and less text-heavy (3.7 messages sent per student on average) than those with human counselors (5.9 messages sent per student on average). This pattern may reflect greater efficiency or reduced depth. For instance, when asked, “Which universities in my city have the best education programs?”, Kai instantly replies with a ranked list and admission info. A human counselor might respond more slowly but also explore preferences, goals, and finances. Shorter AI conversations may signal faster delivery or less personalized guidance.
- Kai appears to be more efficient than human counselors in directly addressing users’ specific queries. Kai responds five times faster than human counselors and more precisely, with only minor differences in the number of topics covered per conversation. For example, when asked, “What math courses do I need to teach primary school?”, Kai lists the required courses, while a human counselor might give a less complete answer or shift to discussing the student’s confidence in math. While quicker, more precise replies don’t necessarily mean better counseling—especially if depth is lost—Kai’s responses tend to stay closer to the question’s content. As Figure 3 shows, Kai’s answers more closely match the semantical content of student queries, suggesting a higher degree of topical alignment and focus.
Figure 3.
Note: This figure shows the distribution of cosine similarity scores between user questions and responses from either a human or KAI. Cosine similarity measures semantic alignment and serves as a proxy for response efficiency. Only question-based input-output pairs were included for consistency.
- Perhaps most interesting, students appear to leverage Kai and human counselors in fundamentally different ways. Students tend to ask Kai more informative, factual-based questions about teaching careers, scholarships, benefits, and entry requirements (62.6% of all queries to Kai are factual-based questions, versus 25.8% of queries to human counselors). In contrast, they direct more than twice as many subjective questions to human tutors, seeking guidance on vocational decisions, experiences with teaching practicums, and the academic rigor of education programs. For example, students ask Kai factual questions like “What is the average teacher salary?” but ask a human counselor “Am I patient enough to teach?” These patterns suggest that AI and human counseling may each serve distinct but valuable functions in the career guidance process.
What’s Next
These early findings are promising in what they reveal about AI’s potential to help scale programs like QSP. We are currently conducting more comprehensive analyses to better understand the advantages and limitations of AI-powered counseling, examining factors such as Kai’s capacity for empathy, persuasion, and efficiency compared to human counselors, as well as measuring the ultimate impact of this approach on the primary outcomes of interest: applications to and enrollments in teaching majors.
As we await the full results, one message is clear: we need rigorous evidence on AI’s impact in education. Our early analyses suggest the future of educational interventions will rely on understanding how AI-powered and human approaches can work together. This complementary approach could be key to overcoming the persistent challenges of scaling in education policy.