Research
At Queen Mary University of London, Educators are at the forefront of research conducted on LEA. As practitioners, they are well-positioned to disseminate findings and good practices to the wider higher education community. The following are examples of educators who have been working actively in this space.
Dr Usman Naeem
Dr Usman Naeem is a Senior Lecturer in the School of Electronic Engineering and Computer Science (EECS) and Queen Mary Academy Fellow, who is leading the learning development pathway of LEA at Queen Mary University of London and has supported various stakeholders to enable pedagogical changes and optimise LEA impact for learners, educators and the Institution. Usman has delivered successful LEA training and has a track record in this area. For example, Embedding LEA into Curriculum Design workshop was launched for the first time on the occasion of the Festival of Education 2023 and has empowered a variety of academics/practitioners to use LEA to improve their teaching outcomes confidently. In addition to this, he has chaired a series of special sessions based on LEA at educational conferences such as the IEEE Global Engineering Education Conference (EDUCON).
Below are some examples of Usman’s LEA-related work:
Data-Driven Interventions for Capstone Projects
Data-Driven Interventions for Capstone Projects
Abstract
The capstone project is a crucial element of a degree programme and plays a vital role in the growth of learners, as it enables them to enhance their problem-solving skills and improve their employability prospects. In addition to this, the project provides the learners with an opportunity to demonstrate and showcase their critical thinking abilities and creativity. However, due to the year-long independent nature of these projects, learners can disengage due to a lack of motivation or self-regulated skills throughout the project. To address this problem, we formulated a data-driven intervention approach that conducts learner engagement analytics to identify and support disengaged learners, ensuring they maximise the benefits of completing a capstone project. The motivation was also to provide these learners with the necessary resources and support to get them back on track. This approach was implemented in the capstone projects conducted by learners at Queen Mary University of London within the School of Electronic Engineering and Computer Science. Based on the data of the three cohorts in 2020–21, 2021–22 and 2022–23, our analysis shows that the proposed data-driven intervention approach for capstone projects can effectively identify less-engaged learners and targeted interventions are shown to improve the overall performance of these less-engaged learners on capstone projects.
Learner Engagement Analytics in a Hybrid Learning Environment
Learner Engagement Analytics in a Hybrid Learning Environment
Abstract: Computer Science (CS) programmes in higher education institutions worldwide have seen unprecedented growth in learners, which has presented educators with several challenges. These include teaching large classes while simultaneously measuring learner engagement. CS programmes tend to have large first-year programming classes, as this is a core subject for all learners, which can lead to an environment where learners start to disengage due to feeling anonymous and lacking support. As we enter the post-pandemic era, institutions have started to adopt a hybrid approach to teaching and learning, which paves the way for educators to analyse data from learning management systems and on-campus learning activities (lectures, seminars, labs) to measure learner engagement and identify learners who are struggling and require further support. The work in this paper describes the adaptation of an online pedagogic framework during the hybrid delivery of a first-year web programming module, which includes a hybrid practical lab coordination system to conduct learner engagement analytics to support learners.
Unlocking the Potential of Learner Engagement Analytics in Higher Education
Abstract: Organisations across many sectors use data analytics to gain valuable insights to make effective decisions. Since the COVID-19 pandemic, data analytics has gained momentum in the higher education sector, which has resulted in the field of education research known as Learner Engagement Analytics (LEA). This is based on combining several data sources based on learner engagement, including data extracted from (but not limited to) learning management systems, attendance records, online sessions and library systems. This data can provide institutions, educators and learner support services with insights into the learner’s learning experiences, which in turn help higher education institutions to facilitate a learning environment that enables their learners to reach their full academic potential. LEA can also help in identifying learners who disengage with their courses, which can have an impact on learner retention rates. From an educator’s perspective, insights from LEA can be used by educators to assess the impact of their teaching and inform their pedagogic approach. However, educators can feel overwhelmed when presented with a plethora of engagement data, hence can be unsure how to start utilising LEA for their courses. This session aims to introduce LEA from an educator’s perspective and how it can be used in courses. This session will provide educators with an understanding of the purpose and importance of engagement markers and how they can be used effectively to gain meaningful insights.
Revolutionising student engagement through data-driven teaching
Dr Usman Naeem and Dr Vindya Wijeratne have launched a new project, Empowering Educators to Embed Learner Engagement Analytics in Curriculum Design. The year-long project, funded by the Queen Mary President and Principal’s Fund for Educational Excellence, will extend across all three faculties, equipping educators with the tools and confidence to integrate learner analytics and gamification into teaching.
Read the news story to find out more about this work.
Dr Marie-Luce Bourguet
Dr Marie-Luce Bourguet is a Senior Lecturer in the School of Electronic Engineering and Computer Science (EECS),
she is director of the EECS Educational Scholarship Centre, and serves as deputy director of the Centre for Academic Inclusion in Science and Engineering. One of her projects involves using learning analytics to measure learners’ self-regulated learning skills in a flipped learning environment. Additionally, she is developing AI algorithms to understand an online learner’s status, like their cognitive-affective state, fatigue, and cognitive load, aiming to identify when a learner might need help.
You can read some of Marie-Luce’s papers based on LEA below:
Measuring Learners’ Self-regulated Learning Skills from their Digital Traces
Measuring Learners’ Self-regulated Learning Skills from Their Digital Traces and Learning Pathways
Abstract: Flipping the classroom requires from students some self-regulated learning skills, as they must have engaged in learning activities prior to attending classes. The study we describe in this paper was done in the context of a 15-week flipped course delivered online to a large class of undergraduate students. We collected various time-stamped digital traces generated by the students’ engagement in the required weekly learning activities (H5P interactive videos, quizzes and worksheets). The collected data allowed the generation of visual learning pathways, from which several types of learning profiles emerged. A distance measure between the students’ learning pathways and the instructor’s recommended pathway was found to be negatively correlated with exam performance. The results from a survey collecting students’ perceptions of their engagement with the learning activities are also presented.
Data-driven Behavioural and Affective Nudging of Online Learners
Data-driven Behavioural and Affective Nudging of Online Learners: System Architecture and Design
Abstract: In this work-in-progress paper, we describe the architecture of a system that can automatically sense an online learner’s situation and context (affective-cognitive state, fatigue, cognitive load, and physical environment), analyse the needs for intervention, and react through an intelligent agent to shape the learner’s self-regulated learning strategies. The paper describes the system concept and its software architecture and design: what sensory data are captured and how they are processed, analysed, and integrated; what intervention decision will follow; and what behavioural and affective nudges will be given.
Dr Lujain Alsadder
Lujain Alsadder is Senior Lecturer in Physiology at the Institute of Health Sciences Education, Queen Mary University of London. Lujain is a member of the physiology team and teaches the Cardiorespiratory and Carriage of Oxygen modules in Year 1 and 2 MBBS programs, the Graduate Entry Program, and the Physician Associate Studies MSc program. She has a special interest in physiology education, digital accessibility, sustainability and AI in healthcare, and learner engagement.
Lujain is an active member of The Physiological Society and the Association for The Study of Medical Education
(ASME).
Medical students' engagement with an interactive virtual learning environment
Assessing second year medical students' engagement with an interactive virtual learning environment using learner engagement analytics
Background: Virtual learning environment (VLE) asynchronous interactive pre-sessional resources were designed to maximise students’ learning and experience during laboratory sessions. This study aims to identify medical students’ engagement with virtual pre-sessional resources and investigate the relationship between engagement with these resources and student’s performance in the summative end-of-second-year data interpretation (DI) exam using Learner Engagement Analytics (LEA).
Methods: A retrospective observational study was designed using VLE data at Queen Mary University of London. We defined ‘engagement’ markers as virtual resources used for asynchronous learning. These markers included H5P (HTML 5 Package) content, quizzes and other interactive materials that students completed before their practical sessions with activity completion records. The ‘engagement’ markers were used to calculate an average engagement score for each student. The relationship between students’ results in their summative DI exam and engagement scores was measured using Pearson’s correlation coefficient using Microsoft Excel.
Results: The VLE dataset for LEA included data about 408 second-year medical students’ interaction with virtual ‘engagement’ markers. The study identified 31 ‘engagement’ markers for laboratory pre-sessional resources. The cohort’s total engagement score was 31%(SEM = 1.15%), and the DI exam average mark was 67.2%. There was a weak but positive relationship between students’ engagement scores and their performance in the DI exam (Pearson’s coefficient = 0.32). The key finding was that medical students who an engagement score >60% did not fail the DI exam. While we cannot attribute cause and effect, this highlights the ease with which LEA enables insights into medical learners’ engagement.
Use of Learner Engagement Analytics to empower medical educators
Use of Learner Engagement Analytics to empower medical educators to make data-informed decisions
Abstract
Learner Engagement Analytics (LEA) has enabled Higher Education Institutions (HEIs) to identify learners who are not engaging with their studies and provide targeted support and help (Naeem and Bosman, 2023). It has also allowed educators to make data-informed decisions to inform their curriculum design and classroom practice (Cogliano et al., 2022). The LEA data is captured from a wide range of sources related to teaching and learning which offer meaningful insights into a learner’s learning habits (Eady et al., 2021). Previous research suggests that providing learning analytics to educators in Higher Education Institutions can improve learning outcomes for students (Aslan et al., 2019).
At Queen Mary University of London, a multidisciplinary team was formed of medical educators, LEA experts, learning technologists, and learning innovation professionals to investigate how LEA can inform curriculum redesign in the early phase of the medical curriculum. Through a series of scholarship meetings on LEA using data dashboards from the Virtual Learning Environment (VLE), the team analysed learners’ engagement with the virtual pre-sessional resources. The VLE interactive resources were designed to allow medical learners to develop clinical interpretation skills during practical sessions, as recommended by the General Medical Council (GMC, 2018). However, medical educators lacked the metrics to evaluate learners’ interaction with these virtual resources. Therefore, it was inevitable to train educators on how to use LEA to optimise students’ learning. The team assessed the learners' engagement on the VLE, quantified engagement scores, and evaluated the results against key outcomes, including the learners' performance. The LEA data offered further insights into virtual engagement across multiple modules in the medical curriculum. Effectively, the outcome of this work empowered medical educators to make informed decisions regarding the future use of VLE resources in curriculum design and develop virtual resources to increase students’ engagement and enhance their learning.
