• View
• Backlinks

Anderson Augusma

Research topics

GLM Mixed effect, Spatial Statistics, Deep-Learning, Multimodal

Research description

Multimodal perception and statistical modeling of pedagogical classroom events: towards a privacy-safe approach.

Over the past two decades, multimodal research in education has mainly investigated instructional design principles related to learners' cognitive load, and was seldom done in real-world conditions like in classrooms. Context Aware Classrooms (CACs) have recently become an adequate instrument for recording and analyzing multimodal interactions, thanks to their sensors that capture classroom events, on some main dimensions of the instructional situation: time, space, social, and epistemic. Computer scientists have poorly studied CACs, as other in the wild contexts, even if research on perceptive spaces is not new. Context-aware classrooms are perceptive spaces with eyes and ears. Some privacy and ethical issues arise from these capabilities. The goal of the system is to perceive the underlying cues of instructional episodes (like students' engagement, attentional level, etc.), not to monitor individual behaviors per se, even if they are inadequate. The challenge remains in perceiving these cues while preserving privacy. Machine learning and deep learning techniques are usually used to analyze audio-visual recordings in order to produce individual indicators (that may harm participants' privacy), before summarizing them for all the students in the class, and are in turn used to answer research questions.

In many perception tasks, recent advances of deep neural networks can sustain analyzes of instructional events. Several multimodal input features are extractable to interpret and draw a multimodal model of the classroom situations: action/activity, attention distribution, emotion, facial expression and posture recognition features. Nevertheless, Deep Learning and statistical methods still progress on state-of-the-art corpora while porting models in real conditions remains a research challenge. Instead of merely performing classification using state-of-the-art machine learning techniques, this Ph.D. study couples Computer Science and Applied Mathematics. It aims at optimizing the preprocessing and machine learning models through statistical methods to perform dimensionality reduction, efficient feature selection (in the preprocessing step) and meta-parameters regression (in the machine learning step). This way, the proposed multimodal approach, combining signal processing, machine learning and statistics, will be both frugal and privacy compliant, relying on a minimal number of highly informative features. This approach will be compared to classical machine learning approaches in computer perception. We will concentrate our efforts on conducting a comparative study to assess the relative performance and the privacy degree one can guarantee to teachers and students in a CAC. A statistical study will measure the performance loss with respect to preprocessing parameters and meta-parameters in the machine learning process.

Publications