Meeting the needs of the modern candidate
Case Study
ICAEW
Introduction
Founded in 1880, the Institute of Chartered Accountants in England and Wales (ICAEW) is a world leading professional membership organisation, headquartered in the UK and regulated as an awarding body by the Financial Reporting Council, and which qualifies as an EPAO (end point assessment organisation). ICAEW promotes, develops and supports over 210,000 members and students in 150 countries.
ACA is the qualification prescribed by ICAEW and stands for the Associate Chartered Accountant. A survey about the ACA taken by ICAEW members, alongside ongoing consultations with all stakeholders, identified a growing demand for assessments to reflect the use of technology in the workplace, amid the decreasing use of pen and paper.
The ICAEW’s ACA qualification comprises of three levels – Certificate, Professional and Advanced Level. Due to the less complex structure of the questions asked, the Certificate Level has been computer-based since 2007. However, due to the complex assessment needs, the Professional and Advanced Level qualifications needed a state-of-the-art computer-based examination solution.
Goals
ICAEW wanted to move towards a system that reflected the realistic working conditions for professional accountants, who complete the majority of their day-to-day work on a computer. Looking towards the future, ICAEW wanted a system that provided flexibility for continuous improvement and development of assessment, to keep pace with emerging skills and use of technology.
Key challenges
ICAEW needed to move their exams from a paper-based delivery to a computer-based delivery format without losing the integrity of the current exams. There was a need to ensure that the exam paper structure initially remained the same to ensure that there was no differentiation between paper and computer-based testing during the transition. In other words – that no student would be disadvantaged by taking their exam on paper vs. computer and vice versa.
The solution
ICAEW selected RM to facilitate their move from paper examinations towards a flexible on-screen system that allowed for continuous technological improvement, and which reflected the realistic computer-oriented working conditions of professional accountants. Over the past 16 years, RM has worked with ICAEW to assist them in making the transition from paper to full digital assessment. This started with e-marking scanned paper scripts, and then the adoption of a hybrid approach comprising both paper and on-screen assessments. ICAEW is now 100% digital assessment, and their approach includes both centre-based and remotely invigilated exams.
ICAEW already had a long-standing working partnership with RM, enabling the e-marking of ACA exam papers securely online by human markers, using RM Assessor.
Assessing advanced question types
To ensure that the assessment process truly replicated a standard working environment, it was imperative for all systems involved to be able to support the use of digital spreadsheets and word documents. RM helped to create a program that allowed for complex assessments of calculations and commentary in one location, as well as harmonising the program so that only certain aspects of the workbook could be seen by the examiner.
Throughout this journey, special attention was given to ensure that there was no differentiation between paper and computer-based testing during the transition, so no candidate would be disadvantaged regardless of the mode of exam they sat. There was direct replication of the paper-based exam structure in initial stages of transition where required.
Digital transformation
RM provided a managed service to ensure the digital transformation solution incorporated a complete on-screen testing platform designed to deliver the ACA qualification. Full end-to-end testing activities included:
- item authoring
- test authoring, administration and delivery
- remote invigilation
- marking
- reporting and activity management
To ensure that each component of the solution seamlessly communicates with the others, RM integrates these components - including remote invigilation - into one platform.
The outcome
ICAEW’s transition to RM’s digital platform has modernised their qualification. The platform has integrated analytics, an electronic bookshelf for reference material and fully functioning spreadsheet capability. Practice tests track candidate progress, performance and readiness to sit summative exams and remote invigilation has enabled candidates to choose where they sit their exam.
Annually, the solutions from RM are used to process over 47,000 on-screen tests and to remotely invigilate up to 20% of these.
This integrated platform continues to support ICAEW’s strategy of delivering assessments that are high quality and authentic to the workplace.
Testimonial
"As an organisation, we initially engaged with RM to support the ACA examination process – our world-leading professional qualification in accountancy, finance and business. That was 2009, and RM has been providing valuable outsourced e-marking services to ICAEW ever since.
The company provides a complete end-to-end, workflow-based exam service that incorporates every stage of the exam lifecycle. We currently use RM Assessment Master and RM Assessor, to deliver and mark our digital exams. We’ve been using it to e-mark long answer computer-based assessments since 2016; with the support of RM, we deliver and mark our mixed essay and spreadsheet style exams, using on-screen marking, representing around 50,000 assessments pa."
Richard Eckersley, Head of Assessment, ICAEW
RM is working with innovative assessment organisations around the world.
Interested in finding out more?
Contact us for a conversation