Student Engagement in E-Learning During Crisis: An Unsupervised Machine Learning and Exploratory Data Analysis Approach

Abstract

The lockdown caused by COVID-19 has forced educational institutions to rapidly adopt e-learning, which has revealed many significant challenges related to student engagement. Following the CRISP-DM (Cross-Industry Standard Process for Data Mining) methodology, the present work aims to provide teachers and university administrators with a framework based on unsupervised machine learning and exploratory data analysis to identify engagement levels and understand the potential reasons for low engagement. Various data sources, including Microsoft Teams logs, demographic, and educational data, were merged to create a comprehensive dataset with the most relevant and useful measures for the success of our approach. This study was structured around three main research questions to achieve our goal. First, we sought to identify the most effective Microsoft Teams measures for identifying students' engagement levels. Then, our analysis focused on comparing different clustering models (two-level, three-level, and four-level models) to determine which one is most accurate in identifying low-engaged students. Finally, we examined the demographic and educational factors influencing low student engagement. The results revealed that: by applying the Sequential Forward Selection (SFS) technique, ScreenShareTime, VideoTime, NbrViewedVideos, Recency, and AvgTeamsSessionDay are the most relevant Microsoft Teams engagement metrics, improving the silhouette width from 0.37 to 0.70 when using these selected measurements. The four-level clustering model (Low, Medium, High, and Super) proved most effective in identifying low-engaged students. Analysis of factors showed that low engagement is primarily related to limited living conditions, with 66% of low-engaged students having low incomes. In addition, 50% do not use online services and 62% of low-engaged students took more than three years to reach their final year, indicating pre-existing academic difficulties. These findings provide educational institutions with valuable insights to enhance student engagement in distance learning, particularly during crisis periods such as the COVID-19 pandemic.