Transforming Assessment
Is it possible to implement
e-marking for high-stakes exams in one year?
Case Study
RIC of Slovenia
Overview
Slovenia has one central exam board, RIC, which sets and administers its national exams. RIC wanted to use e-marking for the National Assessment of Knowledge (NAK) in the primary education system, comparable to the national KS2 tests in England and Wales.
The number of scripts to be marked was relatively small, at around 100,000, so RIC wanted to implement a system that was easy to use and maintain, and could be delivered in the native language, Slovene.
Goals
RIC Slovenia wanted to:
- improve and monitor the quality of marking
- increase the security of exam papers
Key challenges
Under the previous Slovenian marking model, exam scripts were marked by teachers, either in school or at one of 17 regional marking centres. Marking was considered part of teachers’ duties and they were not paid for this additional work. With this paper-based system around 14% of the forms used to record marks were filled in incorrectly each year. Parents and students were allowed to see the papers before final marks were awarded, to enable them to raise queries if they thought errors had been made. Around 10% of papers were subject to enquiry, the vast majority of which did indeed have their marks changed. Principals complained that this process was too time consuming and expensive.
The combination of all these challenges meant marking had become a burning issue in Slovenia, with teachers and national education bodies all publicly expressing their dissatisfaction.
The solution
After intense discussion, it was decided that the solution was to introduce an e-marking system. In March 2012, a tender for the e-marking contract was published. RM Assessor was selected to deliver e-marking for the whole of Slovenia’s National Assessment exams, which are taken by 11-12 year-olds and 14-15 year-olds as they complete the second and third cycles of their elementary education.
Up and running within months
Following a competitive tender which included a pilot, RM was awarded a contract to supply its web-based RM Assessor software. RIC wanted to get the system up and running over the course of a summer so the RM team leapt into action.
The RM Assessor software was translated into Slovene and the awarding organisation sourced scanning machines. RM provided training for the operations team and worked with the technical team on setting up system hosting and the data feeds structure. Within a matter of months RIC was ready to implement e-marking.
Improvements in the quality of marking
Because RM Assessor does not require examiners to manually record marks as the paper-based process did, technical errors such as miscalculations were eliminated. Enquiries on results of the Slovenian National Assessment dropped, with the share of enquiries for the ‘mother tongue’ assessment at their lowest since 2007. Standardisation and live monitoring have “substantially contributed to a better marking quality” according to the RIC. Another aspect of RM Assessor that helped improve the quality of marking was the ability of examiners to highlight problems and get rapid feedback from principal examiners and their assistants. Marking inconsistencies could be highlighted and addressed during live marking, rather than as the result of an enquiry afterwards. The data collected in RM Assessor allows marking quality to be analysed at a granular level, with information available on examiners, topics and even individual questions. The National Examination Centre now prepares a report based on this data, which is sent to markers and school principals who in turn suggest further measures to improve marking quality.
The solution
A marked rise in support for e-marking
Those involved were asked what their opinion of e-marking was before it actually took place and what their opinion was having participated.
![[Case Study] - RIC_Page_4_Image_0001](https://content.rmresults.com/hubfs/Landing-pages/Assessment%20Case%20Study/Case%20Study%20RIC/%5BCase%20Study%5D%20-%20RIC_Page_4_Image_0001.jpg "[Case Study] - RIC_Page_4_Image_0001")
"Our decision to implement RM Assessor supports our objectives of driving modernisation and improvement in assessment quality and efficiency in Slovenia. The move to onscreen marking has already begun to reap significant benefits for markers and pupils alike, including improved security and minimisation of risk and error."
Darko Zupac, Director at RIC Solvenia
RM is working with innovative assessment organisations around the world.
Interested in finding out more?
Contact us for a conversation