Digital Assessment and Feedback

Assessment drives learning within higher education – evidently, assessment for learning and authentic assessment design have become ultimate goals for educators and universities. The rapid shift to online assessment through the COVID19 pandemic brought about considerable changes and innovation in our ways of thinking about student experiences with assessment and feedback. More recently, the disruption of generative AI (e.g. ChatGPT) added a further challenge to our teaching and assessment practice. Together, this has called for a strategic initiative to specifically enhance digital assessment and feedback practice.

In close collaboration with QM Academy, TELT, HSS, and selected FMD module teams, the Digital Education Studio (DES) led the Cadmus pilot project starting in June 2023 with encouraging interim results.

2023/24 Pilot Results

In the academic year of 2023/24, 20 modules in the Faculty of Medicine and Dentistry and in the Faculty of Humanities and Social Sciences took part in the pilot. 1435 unique students used Cadmus as an assessment tool.

The overall response was positive. Academic teams found Cadmus useful and insightful:

• 81% of students rated their Cadmus assessment experience as either Good or Excellent
• 71% of students reviewed their feedback at least once.
• 85% of students reported it was easy to understand how to complete the assessment.
• 82% of students reported that it was easier to find all the resources required to complete the assessment.
• 68% of students felt less stressed while completing their assessments.

These positive indicators were accompanied by increases in both submission rates and student grades in comparison with 2022/23.

Digital Education Studio presented at the 2024 Centre for Online and Distance Education (CODE) conference on Research in Distance Education (RIDE).

Access the full text of conference abstract number [5391] by clicking this link.

2024/25 Extended Pilot

Following the successful completion of the Year 1 pilot in 2023/24, the Year 2 pilot is currently underway, focusing on further refining and expanding digital assessment and feedback practices across the three faculties and Queen Mary Academy. 53 modules with over 3500 students are planned to piloting Cadmus in their assessments this year. Stay tuned for our pilot results coming up in the following months.

If you would like to know more about this project, or any of the other work taking place within the Digital Education Studio, please email: digitaleducationstudio@qmul.ac.uk

Most programmes within the Faculty of Medicine and Dentistry (FMD) utilise tools built into the Learner Management System (LMS), including Turnitin as a plagiarism and collusion-checking tool. To enhance the student and staff experience on digital assessment and feedback, and to mitigate any academic integrity issues, the Dean for Digital Education and the Digital Education Studio undertook a market review of external digital assessment tools which would complement existing assessment practices.

There is an abundance of educational technologies and tools used for assessment and feedback in the market. The rubric for eLearning Tool Evaluation (EDUCAUSE) was used to assess the suitability of digital assessment tools to address the problems noted above. Cadmus then stood out and scored highly on all criteria (i.e. functionality, accessibility, technical, mobile design, privacy, data, protection and rights, social presence, teaching presence, cognitive presence). Also with evidence from their previous work and successful pilots with other universities in Australia, the UK and the US, Cadmus was selected to conduct a pilot project on digital assessment and feedback.

Cadmus is an ‘assessment for learning’ platform. The tool comes with a range of assessment design templates and scaffolds the assessment and feedback (both summative and formative) processes for teachers and students. It also provides support for students particularly on academic writing skills development, as well as providing teachers necessary learning analytics to give personalized academic support to individual students.

A shared goal of the academic pilot team and the Cadmus team is to maintain a focus on fundamental teaching and learning principles, to ensure that measurable assessment outcomes are rooted in a pedagogically-supported learning experience.

While there is a variety of assessment formats within FMD, this pilot project focuses on written assessment, which mostly resides within ‘assessed coursework’. Aside from exams, this is one of the most frequently used assessment formats in FMD, providing academic teams an opportunity for assessment redesign to support academic integrity and reduce any cases of misconduct.

With this background, the project therefore aims to address three wicked problems:

• Reduction of academic misconduct cases, through redesign for authentic assessment
• Practice of designing for well-scaffolded, authentic digital assessment
• Efficiency of providing timely feedback, that will lead to enhanced student engagement, learning and outcomes

In close collaboration with Queen Mary Academy, Technology Enhanced Learning Team (TELT), the Faculty of Humanities and Social Sciences (HSS), selected module leads from FMD institutes, and with the support of Cadmus team, the Digital Education Studio enlisted existing e-learning professionals from each institute to support academics on this innovative practice.

The pilot project began in June 2023, with Cadmus being used by academics to re-design the assessments within their Semesters 1 and 2 modules. Based on an agreed evaluation framework, the project team evaluates the effectiveness of the pilot and tool in practice in January 2024 and in August 2024 for Semesters 1 and 2 respectively. The below section illustrates the interim results of the pilot project and it is very encouraging.

Academics who have been using the Cadmus platform say it has:

• Provided good preventive strategy for giving indicators of possible academic integrity issues. Learn more from Dr Maria Turri's sharing. 
• Enabled them to understand the learning journey of individual students by looking at analytics and step in where necessary to support students in both learning and assessment. Learn more from Dr Joanna Riddoch-Contreras's case study.