Deep heart rate variability analysis for VR-learning

BACKGROUND: Emotions are a fundamental means of communication for human beings to inform others of their feelings. Unfortunately, some people cannot or do not accurately express their emotions, e.g., people with autism spectrum disorder (ASD), attention-deficit hyperacti i di de (ADHD), Pa i disease, depression that we commonly encounter in school, or community settings. However, assessing emotions is hard in clinical settings where patients' periodic self-reports as such reporting is subject to intra-subject variations in reporting due to other circumstantial factors like environment, people, situation, etc. There is an increasing interest in investigating ways to assess emotions from various fields of study, especially in education. This project novelty is the application of the physiological principle of the human autonomic nervous system and the use of both time and frequency domain parameters of heart rate variability (HRV) for the emotions of learning status identification objectively. As biosignals, the changing emotions can be measured by changes in heartbeats which can be precisely represented by HRV using deep analysis. Immersive VR-learning places individuals in an interactive learning environment to replicate possible scenarios or teach particular skills or techniques. Considering the potential learning enhancement through VR use, it is understandable why educators nowadays scrutinize this technology intensively, looking to add an extra dimension to the classroom concerning both teaching and learning. With today wearable technology, it is feasible to ascertain the emotion in real-time and provide effective VR-learning for users.
METHODS: The project aims at assessing emotions of happiness, sadness, surprise and anger objectively. The study attempts to determine the distributions of various HRV parameters for happiness, sadness, surprise and anger respectively, and ascertain whether emotions can be predicted with high performance in terms of sensitivity and specificity based on HRV parameters. There will be two stages of the study. Stage 1 is the preparation of video clips for emotion stimulation, while Stage 2 is the main part of the experimentation. The inclusion criteria will be the same for both stages. Healthy adults from the local community with no known medical diagnosis will be recruited. Those with a history of mental health problems, mood disorders, and cardiovascular and pulmonary problems will be excluded. A tailor-made wrist band for collecting HRV parameters will be used. HRV parameters will be extracted and machine learning algorithms will be used to develop the model for emotion prediction.
RESULTS: The performance of the use of machine learning algorithms for assessing emotions using HRV will be discussed. The study attempts to find out the distributions of various HRV parameters for happiness, sadness, surprise and anger, respectively and to ascertain whether emotions can be predicted with high performance in terms of sensitivity and specificity basing on HRV parameters.
CONCLUSIONS: The study is an introductory study of using artificial intelligence to classify emotions based on HRV for VRlearning. This study will be the basis for further studies. The finding in this project will indicate that using HRV as a biomarker and the machine learning as an emotion classifier to assess emotions is feasible for VR-learning. Copyright © 2020 The Education University of Hong Kong (EdUHK).