Transforming Assessment
How NZQA moved their national exams online and created a platform for innovation in assessment
Case Study
New Zealand Qualifications Authority
Introduction
Every year, around 160,000 secondary school students in New Zealand enter the National Certificate of Educational Achievement (NCEA), delivered by the New Zealand Qualifications Authority (NZQA). The majority of these students complete their qualifications with examinations.
NCEA is available in English and te reo Māori, catering for all students moving to the next stage in their education or employment. The exams are designed by nearly 500 specialists in the teaching workforce and student work is assessed by around 1,700 examiners. NZQA saw that paper-based examinations were not taking advantage of the innovative teaching and learning approaches of the 21st century and that end-of-year exams were forcing students to be ready at the same point in time, irrespective of their individual learning journey.
To reflect the digital world that students live and learn in, NZQA embarked on a ‘digital assessment transformation journey’ to modernise the NCEA exams, creating a platform for innovation in assessment over time.
NZQA’s reasons for transitioning to digital assessment:
- To align assessment with the way students use and will continue to use digital devices every day
- To improve educational outcomes through the use of data and insights about how students respond to questions
- To improve the equity of NCEA outcomes for Māori and Pacific students through innovation in assessment coupled with better use of data about their experience
- To embed more resilience into the examination systems and processes by reducing reliance on paper distribution
Key Challenges
Challenges that NZQA faced in realising their digital assessment vision included:
- Supporting schools and students to “opt in” when ready, rather than relying on a mandated requirement to participate in digital examinations
- Handling the significant media, political and public scrutiny, especially when NZQA, schools and the suppliers were learning how to deliver effectively into the varied technical environment device types in schools
- Managing the complexities of implementing an e-assessment system in two official languages (English and te reo Māori)
The solution
“A user-centric approach: working closely with schools and students to co-create solutions”
NZQA completed multiple trials and pilots between 2015 and 2018 to gain better insight into how to transition to digital assessment, test their processes, enable schools to assess their capacity for digital examinations, and provide students with an opportunity to experience online exams. Schools, students and exam centre managers were engaged in the design process, ensuring that exams evolved alongside an increasingly digital classroom experience.
Feedback from the trials and pilots showed that NZQA was on the right path
of surveyed students preferred digital exams over paper
of surveyed students found their digital exam experience a positive one
According to an NZQA student experience survey from 2018, students were very positive about digital exams, with 95% of survey respondents preferring a digital exam over paper and 97% reporting that they found their digital exam experience a positive one.
Also, psychometric and statistical analysis of exam results* showed no significant difference in achievement between students completing assessments online and on paper.
Based on the very positive student feedback and the findings of statistical analyses of exam results, NZQA decided to move ahead with the development of a long-term platform for NCEA Online.
A strategic partnership to deliver the digital assessment vision for NCEA Online
After a robust procurement process, NZQA entered into a strategic partnership with RM in 2018. The partnership involves building and configuring digital assessment services using RM Assessor3, RM’s latest e-marking platform, as part of their staged approach to digitise NCEA exams. NZQA also choose RM Assessment Master, an online assessment software solution, to allow them to deliver flexible assessments from any location across multiple delivery methods.
RM Assessment Master is securely delivered from cloud infrastructure in Australia and is also being used by other government entities, such as the SACE Board of South Australia, to develop and deliver digital assessments. This has created opportunities for NZQA, SACE and RM to collaborate and design an online assessment system that reflects the new way students learn.
Two thirds of NCEA exams delivered digitally in the first two years of the partnership
The new partnership allowed NZQA to offer and mark about a third of their exams online in 2019. Approximately 14,400 students from 197 schools completed digital NCEA exams in 14 subjects.
A student experience evaluation of NCEA online carried out by NZQA in 2019 found that 97% of those surveyed continued to strongly enjoy the digital exam experience and the psychometric and statistical analysis for 2019 aligned with that from 2018.
In 2020, NZQA offered 58 digital exam sessions across 21 subjects and three levels. This represents approximately two thirds of NCEA exams and is a significant increase from 2019.
In response to the COVID-19 pandemic, more schools in New Zealand have adopted online teaching and learning.
The opportunity for students to sit NCEA exams online has been described by the New Zealand Education Minister as “a step forward to deliver a more modern and resilient education system”.
NZQA’s plan is to digitise as many NCEA exams as possible, and more online exams will be available beyond 2020.
“Students are enthusiastic supporters of digital assessment; they tell us it reflects the way they are learning and living, with technology at their fingertips.”
Andrea Gray, Deputy Chief Executive, NZQA
The future
“I am delighted that NZQA and RM are working together to deliver the exciting and innovative digital assessment vision for NCEA Online. Our e-testing and e-marking platforms, RM Assessment Master and RM Assessor3 offer the New Zealand students and marker community the ability to sit and mark NCEA exams efficiently online.”
Dawn Pollard, Director of Growth Markets, RM
NZQA is looking to gradually replace paper-based assessments with digital assessments by expanding the range of subjects delivered online and by introducing exams that are developed specifically for digital platforms, such as mathematics, science and music. There is no set deadline for removing traditional NCEA assessments and digital exams will be offered progressively as more schools adopt digital teaching and learning.
The current programme of research, innovation and enhancements supports the work to do more than digitise today’s way of examining, and that is the key to delivering value to students, and a credible and reliable qualification to support entry to higher education and employment.
RM is working with innovative assessment organisations around the world.
Interested in finding out more?
Contact us for a conversation