Written by Sabine Rieble
As always, Marie Dieudonne got up early, had breakfast and went to school. But that day was different for everyone: Only 4th and 6th graders went and all the other students had the day off. On Friday, May 29 2015, Marie was one of the 4,144 students from 60 schools who participated in the piloting of a new assessment tool for mathematics and reading.
This new test, designed to be completed in about 2 hours, was introduced to get a better understanding of the skills and proficiency of that age cohort and to provide an opportunity for early interventions to address any weaknesses in the education system identified at that stage. Starting in January 2015, a team of curriculum specialists from the Ministry of Education in Haiti, the International Association for the Evaluation of Educational Achievement (IEA), the Haitian Institute of Training in Science of Education (IFHOSED), and IDB worked closely to develop this tool that would replace the traditional 6th grade exam, as part of 12 reform measures released by Haiti’s Minister of Education and Vocational Training (MENFP) in August 2014.
The team carried out the following 7 steps to accomplish the development of a short reading comprehension and mathematics test targeting 4th graders to serve as the basis for the development of an appropriate assessment tool, to be applied on a national scale in 2016 and beyond.
1- Item Selection. The test items (questions) were selected using previously tested and proven test items from the internationally administered Reading Literacy Study1 (RL) and the Trends in International Mathematics and Science Study (TIMSS), cycles 2007, 2011 and 2015. The team selected 32 math questions, 26 reading comprehension questions, and 25 background questions to learn more about the students (age, socio-economic status, and attitudes towards school).
2- Translation. All questions were translated following a strict translation protocol to ensure standardization. Those items that were previously available in French were adapted to the Haitian context and translated into Creole; all other items were translated from the US English international version to both French and Creole.
3- Design and Format. The questions were then organized in a booklet format. The time set to complete both sections (math and reading) was 45 minutes each, with a 10-minute break in between each of the two parts. An additional 20 minutes was recommended for completing the background questionnaire. The format of answers to the test items was multiple-choice, a new experience for Haitian students who are more familiar with open ended essay questions.
4- Target population (sample). Using the 2013/2014 school census data, a sample of 60 schools of various types and regional dispersion was drawn. Given the novelty of the assessment, it was also decided to give the test not only to 4th graders but also to 6th graders in these schools to compare the proficiency levels between the two grades and to better assess the level of difficulty targeting 4th grade.
5- Test administration. To ensure that the administration of the test was highly standardized, a manual was developed to train the test administrators. It outlined the procedures to be followed, starting with the distribution of the booklets, the assignment of the codes (for school and student), filling out of a report protocol, and the importance of the timing of the test.
6- Pilot Test. On Friday May 29, 2015, the test was administered. Started at 9 am, the administrators matched students with pre-prepared attendance lists and test booklets which were distributed alternately to students in either French or Creole. Each booklet was labelled with a hierarchical code uniquely identifying both the school and student (for data entry only the codes were used; the database was designed to be anonymous). Inspectors and teachers administered the test to 4,144 students including 2,360 4th graders and 1,784 6th graders. Some suggestions were made to improve test preparation and test administration procedures the next time.
7- Results. Upon completion, the booklets were returned to the MENFP central office in Port-au-Prince, where a team of 15 data entry clerks of the Educational Planning Direction was trained to enter the students’ answers in a customized software system developed by IEA for its International Large Scale Assessments to facilitate data entry and analysis. In total, 3,940 booklets could be used for data analysis.
With the assessment results, the Ministry has now more information about the mathematics and reading competencies of both its 4th and 6th graders in either Creole or French and sufficient information to develop assessment tools that can be applied to the entire country in 2016 and beyond. Thus, Marie’s participation in the 2015 assessment helped to develop an improved version to be given to 4th graders in 2016. The clock is ticking again… the next assessment is scheduled for Friday May 27, 2016.
[1] RL is a precursor of the Progress in International Reading Literacy Study (PIRLS)
This is an excellent initiative which should derive the results intended of benchmarking the level of student mastery of the literacy and numeracy learning outcomes for the respective grades. I look forward to the results and the subsequent assessments which, hopefully, would show improvement in student achievement. It would be very useful if a true diagnostic approach could be adopted so that a pre-test/post-test model could be used in the participating schools. However, this would require more targeted work with the schools – especially how to enhance the capacity of the teachers and principals to use the results for instructional planning and monitoring progress throughout the academic year.
This is a great advancement toward setting benchmarks and track progress in the Haitian school system, and encourage quality-based education. It would be interesting to have access to the results, particularly how the schools fared (general, public and non-public).
When will we know more about the test results and students’ performance? I am curious to find out how the results might impact curricular plans for keeping French as a primary language of instruction in classrooms. Thank you.