On Screen Marking
How SEAB enhanced the marking process for their national examinations namely, the Primary School Leaving Examination (PSLE) and the Singapore‑Cambridge General Certificate of Education (GCE) Examinations ‑ and won three awards in the process.
Case Study
SEAB
Introduction
The Singapore Examinations and Assessment Board (SEAB), established in April 2004, is the body responsible for the development and administration of all national examinations in Singapore, overseeing the marking of around 450,000 scripts annually.
SEAB and the Ministry of Education (MOE) Singapore are the joint examination authorities for the Primary School Leaving Examination (PSLE), an annual national examination that is taken by candidates at the end of their final year of primary school education in Singapore to facilitate the students’ onward academic progression.
SEAB, MOE, and Cambridge International Education are the joint examining authorities for the Singapore‑Cambridge General Certificate of Education (GCE) examinations in Singapore. They include GCE N(T)‑Level, GCE N(A)‑Level, GCE O‑Level and GCE‑A‑Level examinations.
Goals
As a forward‑looking organisation that aims to advance quality in assessment worldwide, SEAB began a project looking into on screen marking (OSM) in 2015 to see what benefits it could bring, considering marking quality, reliability, security, and time efficiency for exam markers. Described by Mr Yue Lip Sin, Chief Executive of SEAB, the project aimed to remove the laborious manual marking processes, enabling markers to focus on the professional work of marking while still maintaining its rigour. The board wanted to migrate gradually, ensuring they were able to learn about the system and thoroughly test it.
Key challenges
Time and resource intensive
To mark stacks of exam papers, all the markers had to be assembled in venues within schools like the classrooms and halls. The Chief Marker would brief everyone to ensure that all markers understood the mark scheme.
In Singapore, only full‑time teachers are qualified to mark. For the Primary School Leaving Examinations (PSLE), marking must occur within a 4‑day window. That meant about 14,000 markers using the standard issue green, red and purple pens were tasked to complete the marking of Singapore’s largest exam yearly.
Logistics
Examination scripts were first sent to SEAB to be sorted and ensure that scripts sent to marking centres are free from conflicts before being moved to the various marking centres. Scripts were then marked, with results tabulated and manually recorded in the examiner mark sheet. Team leaders would then undertake manual checking of each marked script by opening and closing the scripts’ envelopes. Progress of marking was managed via updates captured on a white board and hard‑copy forms.
Trolleys were used to transport large numbers of envelopes containing the marked scripts to cabinets for storage.
Security and reliability
Recorders manually checked that the marked scripts in the envelopes from each centre were accounted for and the accuracy of mark entries before scanning.
Before on screen marking was introduced, recorders had to check that every envelope from each centre was accounted for.
Photo: Singapore Examinations and Assessment Board
The solution
In March 2018, following a successful pilot of on screen marking with a small number of subjects in 2016‑17, and in‑depth studies analysing the quality of online versus traditional paper‑based marking, SEAB signed a contract with RM to deliver their Integrated Digitisation of Scripts and Electronic Marking System (iDSEM).
The successful delivery of iDSEM, and the improvements it brought to the quality of marking and the feedback into education quality, together with the way the project was run collaboratively with stakeholders ‑ saw SEAB win three high‑profile innovation accolades.
After each exam is over, the scripts are sent to the Singapore Examinations and Assessment Board for scanning.
Photo: Singapore Examinations and Assessment Board
Three innovation awards for the implementation of on screen marking
In July 2021, members of the project team were presented with a Service Delivery Excellence Award (SDEA) at the Public Sector Transformation (PST) Awards, which is a Whole‑of‑Government pinnacle platform that recognises and rewards public officers and public agencies for excellence in their work and organisational practices.
In September 2021, SEAB received a Gold award at the Singapore Ministry of Education’s (MOE) Innergy Awards, which recognise individuals and teams at the ministry level for successfully developing and implementing innovations that bring about significant benefits and impact in their workplaces.
In 2023, SEAB was awarded the inaugural MOE Minister’s Innovation Award, which is the Ministry’s pinnacle innovation award. It aims to recognise innovations with large scale, system‑level sustainable impact from across MOE headquarters, schools and MOE’s statutory boards.
Read 'The Straits Times' article by Elisha Tushara:
"PSLE scripts now marked on computer screen"
“We are delighted that SEAB’s ‘Integrated Digitalisation of Scanning & Marking’ (iDSEM) project for the Marking of the National Examinations was presented the Service Delivery Excellence Award (SDEA) at the Public Sector Transformation Awards, while also receiving the highest accolade at the inaugural Ministry of Education Minister’s Innovation Award and MOE Innergy Awards.
Winning not one but three awards in such a short space of time acts as a brilliant reminder to both SEAB and RM project teams of their hard work in creating an intuitive marking experience for SEAB examiners and enhancing the marking efficiency, security, and reliability of Singapore’s national examinations.
By partnering with RM to introduce on screen marking, SEAB has significantly enhanced its markers’ ability to manage their workload more efficiently. Increased security and reliability have ultimately improved its learners’ outcomes.”
Kieron Williams, Business Growth Director at RM
On screen marking for quality, reliability, security, and efficiency
Since the implementation of RM Assessor for on screen marking, examination scripts are marked digitally on a computer screen. This has brought real efficiencies for SEAB.
First, the local marking of all Singapore‑Cambridge GCE‑Level subjects and PSLE were brought on screen. Working with RM to set up for the full range of examinations in just two years (increasing from 5 in 2019 to 20 in 2020), SEAB was able to accelerate their schedule to mark all locally graded GCE‑level scripts on screen by one year.
For all PSLE subjects in 2022, the number of teachers deployed for marking duties has been halved from 14,000 to about 7,000 teachers.
With the on screen marking system, answer book scripts are first cut (at a rate of 16,000 pages, or 1,000 scripts per hour with 2 cutters) then scanned. Each scanner can scan 140 scripts per hour. Each booklet has a unique barcode, QR code and the candidate’s identity sticker and technical guides that allow the scanner and software application to identify and process the papers.
The system automatically checks each scanned script for errors, reducing the amount of human work required by SEAB staff, who do a second round of checks. The candidates’ scripts are then allocated to markers via the on screen marking system, ready for marking to begin.
Once assigned a set of questions at the standardisation meeting, each marker must complete a practice marking via the on screen marking system. This streamlines the standardisation process allowing markers to begin assessing earlier.
Previously, markers had to wait for another marker to finish their assigned questions before passing the script to them. Now, two markers can mark different questions on the same paper concurrently, and as such, fewer markers are deployed.
In 2023, the number of PSLE marking days was reduced from four days to three days. This resulted in a 25% reduction in the cost of marking the exams.
Results from SEAB’s annual satisfaction survey of their markers gave an average rating of 3.9 out of 4 for the RM Assessor on screen marking system, stating it was a big improvement. Markers found it robust and stable.
By transitioning to on screen marking, SEAB has also been able to build even further on their quality assurance processes.
Senior markers can seed quality assurance scripts or do random checks to assess whether individual markers may be marking too leniently or too severely. Markers are also able to consult senior personnel more easily through the communication features in the system too. As more marking data is collected, senior markers can identify areas where members of their team might need enhanced training and even look at deploying markers based on their strengths. Marking with these quality assurance processes and controls means that SEAB has been able to shorten turnaround time for papers without compromising marking quality.
The data analytics made possible by on screen marking is not only beneficial for quality assurance, but also for feeding back into curriculum development, using the item‑level data to see patterns of strengths and weaknesses to inform future teaching. One of the unrivalled advantages of on screen marking is the capture of huge amounts of detailed marking data in situ, which could be used for subsequent data analysis.
Interested in finding out more?
Learn more about on screen marking at Collect, Collate and Mark
Contact us for a conversation