Transforming Assessment
Meeting the needs of the modern candidate
Case Study
ICAEW
Introduction
Founded in 1880, the Institute of Chartered Accountants in England and Wales (ICAEW) is a world leading professional membership organisation that promotes, develops and supports over 147,000 members and 24,000 ACA students in 155 countries.
ACA is the qualification prescribed by ICAEW and stands for the Association of Chartered Accountants. A survey about the ACA taken by ICAEW members, alongside ongoing consultations with all stakeholders, identified a growing demand for assessments to reflect the use of technology in the workplace, amid the decreasing use of pen and paper.
The ICAEW’s ACA qualification comprises of three levels – Certificate, Professional and Advanced. Due to the simple structure of the questions asked, the Certificate level has been computer-based since 2007. However, due to the complex assessment needs, the Professional and Advanced level qualifications needed a state-of-the-art computer-based examination solution.
Goals
ICAEW wanted to move towards a system that reflected the realistic working conditions for professional accountants, who complete the majority of their day-to-day work on a computer. Looking towards the future, ICAEW wanted a system that provided flexibility for continuous improvement and development of assessment, to keep pace with emerging skills and use of technology.
Key challenges
ICAEW needed to move their exams from a paper-based delivery to a computer-based delivery format without losing the integrity of the current exams. There was a need to ensure that the exam paper structure initially remained the same to ensure that there was no differentiation between paper and computer-based testing during the transition. In other words – that no student would be disadvantaged by taking their exam on paper vs. computer and vice versa.
The solution
Providing a best-of-breed, end-to-end assessment solution
ICAEW already had a long standing working partnership with RM, enabling the e-marking of ACA exam papers securely online by human markers, using RM Assessor. RM partnered with BTL to create an end-to-end assessment solution. RM integrated BTL’s e-testing software with RM Assessor, enabling both paper-based and computer-based exams to be marked in a single tool.
Assessing advanced question types
To ensure that the assessment process truly replicated a standard working environment, it was imperative for all systems involved to be able to support the use of digital spreadsheets and word documents. RM helped to create a program that allowed for complex assessments of calculations and commentary in one location, as well as harmonising the program so that only certain aspects of the workbook could be seen by the examiner.
Integration engine
In a bid to guarantee minimal disruption to candidates, and to ensure that each component of the solution communicated with each other seamlessly, RM created an integration engine. This integration engine allows for components to talk to one another and share basic information, such as where a candidate is during the assessment process, to each of the relevant parties within the chain. The diagram below illustrates how communication is shared thanks to the integration engine.
Integration engine diagram
![[Case Study] - ICAEW_Page_4_Image_0001](https://content.rmresults.com/hubfs/Landing-pages/Assessment%20Case%20Study/Case%20Study%20ICAEW/%5BCase%20Study%5D%20-%20ICAEW_Page_4_Image_0001.jpg "[Case Study] - ICAEW_Page_4_Image_0001")
Testimonal
As an organisation, we initially engaged with RM to support the ACA examination process – our world-leading professional qualification in accountancy, finance and business. That was 2009, and RM has been providing valuable outsourced e-marking services to ICAEW ever since.
The company provides a complete end-to-end, workflow-based e-marking service that incorporates every stage of the exam lifecycle from examination script scanning and indexing, through to e-marking and delivery of results. We currently use RM’s e-marking software, RM Assessor, to mark paper-based examinations, and the solution has proven to be a stable and reliable application. In fact, we’ve been using it to e-mark long answer computer-based assessments since 2016.
With the support of RM, we mark all but one of our ACA exams using on-screen marking, representing around 40,000 assessments pa, and we are moving towards 100% provision.
Richard Eckersley, Head of Assessment, ICAEW
RM is working with innovative assessment organisations around the world.
Interested in finding out more?
Contact us for a conversation