Edge-Assisted CNN-Attention Model for Real-Time Multimodal Learner State Recognition in IoT-Enhanced Educational Systems

Abstract

A critical limitation in modern intelligent tutoring systems is the insufficient precision in perceiving learners’ cognitive and affective states, which restricts adaptive and personalized instruction. To address this challenge, we propose an edge-assisted deep learning framework that integrates multimodal data fusion with attention-enhanced feature modeling. The architecture employs a Convolutional Neural Network (four convolutional layers with 3×3 kernels, stride 1, ReLU activation, and dropout 0.3, followed by two fully connected layers) coupled with a Softmax-normalized additive attention mechanism to analyze synchronous audio (256-frame, 128-shift Mel-spectrograms), video (facial landmarks and body gestures at 30 fps), and inertial signals (3-axis accelerometer and gyroscope) collected from IoT-enabled learner devices. Pre-processed at edge nodes through denoising, segmentation, and normalization, the data yield structured time-series features that enable the CNN to capture local-temporal dependencies while the attention layer dynamically reweights critical behavioral and semantic cues. Extracted multimodal representations are used for both learner state classification (e.g., active interaction, distraction, passive engagement) and behavior trajectory modeling, and a task-adaptation engine correlates real-time states with historical profiles to generate personalized interventions in a closed-loop system. Experiments on a dataset of 128 learners across four courses (~180 hours of recordings) demonstrated high efficacy, achieving classification accuracies of 95.2% (normal participation), 91.7% (active interaction), and 88.4% (distraction detection), with task-matching accuracy of 87.2% and personalized completion rates of 76.5% under high-difficulty conditions, thereby validating the effectiveness of this edge-assisted CNN-attention architecture in enhancing perceptive accuracy, responsiveness, and human-computer interaction quality in adaptive educational platforms.