From measurement to action: Using data to improve learning in the Caribbean
Every year for the past 40 years, thousands of nervous students from the English-speaking Caribbean sit at their school desks’ to take a test that will change their lives. These evaluations are administered by the Caribbean Examination Council (CXC). CXC’s Secondary Education Certificate (CSEC) examinations are applied at the end of compulsory secondary education to approximately 120,000 students from English-speaking Caribbean countries (16 territories currently participate) to measure student performance. CSEC exam results constitute the main credential used to pursue higher education or to enter the job market. These tests are “high stakes” for the individual students because their performance determines their future. Sadly, in many countries less than half of those who sit the exam attain satisfactory pass rates in key areas such as English language and math.
The individual results from these standardized tests, as well as national averages, are given back to Ministries of Education. Some publish them, some discuss them at ministry meetings, and some send the results back to the schools. But rarely are they used systematically to reflect on student and school performance, or to identify learning gaps so teachers can make changes in the teaching/learning process. As a result, the bad news is repeated year after year. In the words of one Caribbean educator, “You can’t make a pig fat by weighing it”.
Critical disconnects lead to inaction or ineffective actions
On October 28-29, 2013, Ministry of Education head planners and statisticians from 14 countries came together for a workshop entitled: “Using data to Improve Education Outcomes in the Caribbean.” The workshop was sponsored by the Inter-American Development Bank (IDB), in collaboration with the OECS Education Development Management Unit (EDMU) and the CXC. Speakers included regional educators who shared ongoing initiatives in their own countries, and international experts.
Photos taken during the workshop by Marcellus Albertin, Head Education Development Management Unit, Organisation of Eastern Caribbean States
Workshop participants discussed why the results of national or international assessments are not used more often to feed back into teacher training, teaching methodology, curriculum, and assessment to improve what happens in the classroom. The participants identified disconnects at the system and school levels.
Most initiatives to improve monitoring and evaluation data focus on assessments to collect the data, improvements in reliability, and development of systems to store and report that data. However, other key questions often not addressed are:
- Is there a clear vision of what to do with the data?
- Is there sufficient technical and institutional capacity in the Ministries and at the schools to analyze the data and learn from it to inform policy or improve classroom practices?
- Do the protagonists (principals and teachers) find the data useful? Why or why not?
- What can be done to turn the results of student evaluations into useful tools for changing practices at the school and classroom levels?
These questions do not have simple answers. We invite you to share your thoughts. And stay tuned to our next blog post to find out what good practices were identified and what insights were exchanged during the workshop discussions!
Cynthia Marie and Carlos
Good that you are doing this, but let me tell you that CXC is too late in the game…. If you do not start with testing with standards that all stakeholders (especially parents) understand early on, let say 2nd or 3rd grade, it is too late to improve quality of education. So I truly hope you will present in the next posts something on how to measure quality better, more timely, and how parents and other stakeholders are involved in the process to increase accountability. I am looking forward to this discussion.
Eduardo, thanks for your comment. Many of the countries administer national exams in earlier grades as you suggest. For example, in Jamaica they give a Grade 3 test (shifting soon to Grade 2) as well as national literacy and numeracy tests at Grade 4. I would invite others to share their experiences here.
I’ve often wondered why the CSEC scores are not reported on a continuous scale metric, rather than the categorical metric (if I recall, the scores range from 1-6 categories on each of the subject tests, or worse yet to describe the outcome as only pass/fail)? The primary reason for measuring anything, performance included, is to generate variation in the scores so that individuals can be distinguished from one another. There are many items on a single subtest, so I don’t understand why more variation isn’t reported? Reporting scores on a continuous metric would be the only way to identify students for services, possibly through the creation of cut scores to identify individuals with greater needs.
I have searched before for psychometric evidence on the CSEC (well on all CXC exams actually) and have not been able to find anything. Perhaps some of you could point me to a source? Some transparency on the development of items and scores would be helpful as well.
Darrell, thanks for your comment. I’m hoping CXC will address this. I’ll forward it to the Registrar for a response.
This article raises critical issues for those who have an interest in seeing positive changes in the teaching / learning environments throughout the English speaking Caribbean. I will focus on two issues:
• What can be done to turn the results of student evaluations into useful tools for changing practices at the school and classroom levels? And
• How can we actually achieve these changes if teachers have “limited time” and “limited skills” to engage with data and to identify learning gaps of their students.
The CXC produces fairly comprehensive exam reports after each examination period. I want to draw your attention to the Exam Results Reports (CSEC) January 2013, specifically to the English A report -for ease of reference I’ll include the link for ease of reference ☺
(https://www.cxc.org/SiteAssets/2013SchoolReports/CSECJanuary2013EnglishA.SR.pdf)
A few highlights from this report:
General Comments
• The Expression profile remains fair…Candidates are encouraged to do more critical reading and discussion.
• As suggested last year (and probably many times previously – but who is reading and responding to these reports?) ……focus should be on studying “how writers say things and why”
Detailed Comments (Paper 02 – Free Response)
• Too many candidates try to get by with lifting chunks of text, “cutting and pasting” them together…
• We advise that teaching should concentrate on helping students to grasp and reproduce meaning (as opposed to text).
• …there are persistent problems with Expression and Organization.
Again, the report repeats recommendations from previous reports. It seems the same error patterns are repeated over and over.
Perhaps we are overthinking the issue of using data to improve our teaching and learning. Perhaps we need to simply consider how we can facilitate our teachers in creating dynamic, interesting, creative classroom environments. No, I do not mean that we will have “another” teacher professional development workshop!! We have killed those! We need to create spaces where teachers can come together and discuss innovative instructional practices and challenges they are experiencing; spaces where teachers can “have a coffee”, and mentor each other.
Find a few dynamic teachers in each subject area – particularly English Language and Mathematics – and have them lead out on these “Creative Teacher Meet-Ups”, where they encourage others to share how to get around the challenges of “high stakes testing”; how to overcome the practices of “drill and kill”; or the “cut and paste” mentality; to one of “fun”; to one where “ writing workshops” are encouraged, where our youth are encouraged to, “debate for the sake of debating” …the possibilities are endless.
Instead of bringing the endless “past papers” into the classes, I believe that our teachers need a different kind of environment where they themselves are encouraged to develop their own skills of inquiry, knowledge and understanding about the young people they work with. It is through this kind of networking and discussion that we will create learning outcomes built on a foundation of understanding and knowledge.
I worry that my response may seem almost “too simple” or “too easy”. I also know you will say that teachers don’t / won’t have the time to have these “meet-up’s”. On the contrary, I think teachers are yearning for a different kind of discussion. They are as frustrated as we are with the results.
But in reality, what kind of atmosphere encourages learning? – Even for teachers? Certainly not the same “drill and kill” classrooms that produce the dismal results we see repeatedly? Teachers deserve dynamic spaces to allow for the development of their own creativity and innovation which they can then translate into amazing, – dare I say “fun” and results-oriented, teaching/learning environments.
…Let’s actually fatten up these pigs, in fact let’s encourage them to squeal with joy….”weeee weeee weeee all the way home”.
Let’s continue to brainstorm on solutions, there are many more avenues to explore.
Alison, thanks for taking the time to send such a detailed response and for highlighting other sources of information. You’re obviously passionate about teaching and learning. Your suggestions regarding how to motivate teachers are most welcome in this discussion. I hope others will chime in.