Transforming Assessment
Gauteng Department of Education successfully launch on-screen marking of live exams in South Africa
Case Study
Gauteng Department of Education (GDE)
Introduction
The Gauteng Department of Education (GDE) in South Africa is working extensively to further enhance its educational systems, by steadily and consistently improving the key performance areas of the basic education system. The Department engaged Lebone, a local print and scanning services company that works with GDE on printing for educational purposes, to deliver a scheme for the on-screen marking of the matriculation examination. This was driven by the department’s ongoing efforts to improve efficiencies and harness the vast potential of new technologies in South African education.
With on-screen marking, once candidates have written the script these are sent to a central location where they are scanned and these digital images uploaded into the marking system. These scripts are anonymised and the examiners are able to then securely mark the exam scripts on a computer screen.
A series of non-live pilots to test the systems and processes, with scripts being dual marked with both the traditional and on-screen marking method, were implemented by GDE during 2019. Following this success, and despite the impact of COVID-19, live e-marking of 60,000 exam scripts was successfully undertaken in 2020. This included scripts for both the Senior Certificate and National Senior Certificate
exam papers.
Goals
GDE worked with Lebone to create a robust and reliable workflow for the on-screen marking of exams. With the technology adding to the existing robust marking processes that GDE have in place. The project was closely scrutinised by both DBE and Umalusi, to ensure that no student was disadvantaged during the pilot and live marking stages of the exam.
- To understand how education technologies could be harnessed in the Gauteng province to make the marking process more efficient, without risking the quality of the marking itself.
- To make the job of examiners easier and more streamlined, reducing time spent on manual administration tasks to allow them to optimise their time to focus on marking.
- Including additional functionality like the automatic adding up of candidate marks, saving time and improving accuracy.
- To create a more efficient system by reducing the time and distance travelled for markers.
In addition, though not planned at the time, the opportunity to mark on-screen reduced the impact of the spread of COVID-19, with examiners not having to handle student scripts, plus the option was also provided for some examiners to mark from home providing the ultimate safe environment.
“As markers, we’re at liberty to set the pace of our own work without all the time-wasting tasks we’re normally distracted by, such as fetching scripts from the control room or fixing marking errors. E-marking saves us lots of time; we can simply mark, confident in the knowledge that our work is accurate and fair.”
Examiner, GDE
The solution
After extensive research, Lebone worked with RM to enable on-screen marking through RM Assessor. RM Assessor is used by examination bodies like Caribbean Examinations Council (CXC) and Cambridge Assessment and provides the flexibility and scalability to meet the needs of both GDE and, potentially, additional Provinces. The key with the cloud-based solution is that the on-screen marking system meets the exact requirement of GDE with all services and support provided by the local partner Lebone.
Lebone and GDE engaged with the unions from the outset and invited them to take part in the live pilot. RM Assessor is an on-screen marking solution that enhances the examiners’ role by allowing them to cut down on administrative tasks in order to focus on marking; the unions responded very positively to the pilot as it clearly demonstrated an improvement in conditions for examiners and feedback from examiners was unanimously favourable.
Confidence
In order to give stakeholders the confidence that examiners marking on-screen is directly comparable to them marking on paper in terms of results awarded, a number of non-live pilots were held on matric examinations. In these non-live pilots, examiners assessed matriculation papers which had already been marked in the traditional way. These results were then compared to ensure that the onscreen marking was at least as accurate.
“I am absolutely 100 per cent behind this programme. It’s going to be really exciting to see how this can change the South African marking system for the better. The future is bright!”
Examiner, GDE
Accessibility
RM Assessor enables examiners to work from local centres; any school with a computer suite and internet access can act as a marking centre. This vastly reduces the pressures of the existing marking system which had required examiners to travel, sometimes for 3-4 hours, to marking centres. This makes it a more accessible programme which becomes open to a larger pool of potential examiners, as the need for extensive travel is removed. With good examiners always being in demand, this provided more time for them to mark exams and opens up the potential for more flexible working hours for examiners as they can mark at a time and location more convenient for them. The financial cost to the GDE of travel expenses to compensate examiners for long journeys and overnight stays would be removed as a consequence of this increased flexibility.
“This technology has the potential to change the way we mark papers forever. For decades we have stuck with manual marking and all its drawbacks - it’s time consuming, costly and with potential for error. But this pilot has opened up a new way of working and it is really exciting.”
Examiner, GDE
Furthermore, examiners are able to mark at their own pace as they are not reliant on one another to complete papers and thus only able to move at the pace of the slowest examiner. Simple software shortcuts like scoring buttons and moving to the next exam script allow examiners to progress through papers more quickly but without compromising thoroughness or accuracy.
The on-screen marking system is very user-friendly and appropriate for examiners of varying degrees of computer literacy, including those who have very few or non-existent computer skills. “Easy to use… one does not require a high level of computer literacy and proficiency.” A short training course delivered by the Lebone team was simple enough to enable all examiners to embark upon their task. The on-screen marking software is usable on various devices including computers and tablets, so examiners were able to use whichever device they felt most comfortable with.
Benefits
One of the most exciting benefits of the GDE on-screen marking system is the instantaneous access to rich data. The analytical moderators were able to access real-time data showing how marking is progressing, but also how students have responded to particular questions. This enables them to analyse that insight and feed it back into the education system to improve the curriculum, for example.
“I am really impressed with the system. It works wonders. It shows that we are moving with times and with this system, we are there.”
Examiner, GDE
Examiner satisfaction was consistently high; all examiners agreed that the on-screen marking system was easy to use, made it easier to track their workload, and enabled them to complete their tasks faster. Examiners regularly mentioned that one benefit is the automatic tally of total marks, eliminating the need for manual adding and checking on each paper.
“[Marking on-screen is] much faster because you do not have to count all the marks all the time. Calculation is done automatically and accurately.”
Examiner, GDE
Summary
Lebone engaged RM to use their on-screen marking platform RM Assessor to meet the requirements of the project. Lebone and GDE facilitated the successful introduction of on-screen marking in the Gauteng province, paving the way for this technology to be adopted more widely in South Africa and creating a more efficient and inclusive high quality marking process for one of the most important examinations in the country.
On-screen marking has real potential for the South African education system, as clearly demonstrated by this live marking session. Universally popular with examiners, and proven to deliver no bias to results, e-marking has the scope to revolutionise the way exams are marked across the country, delivering fair and accurate results by using innovations in technology.
“The system tracks your workload very effectively and ensures that you are never sitting without work or having to look for questions to mark. This happens a lot when we are marking manually.”
Examiner, GDE
RM is working with innovative assessment organisations around the world.
Interested in finding out more?
Contact us for a conversation