Understanding the Role of Sleep in Face Recognition Through Hidden Markov Modelling of Eye Movements

Sleep loss is shown to significantly affect memory functioning (Walker & Stickgold, 2006). While early studies typically focused on the influence of post-leaning sleep loss on memory consolidation of learned materials (e.g., De Koninck et al., 1989; Wagner et al. 2001), recent studies have shown that sleep loss prior to learning could also impact memory encoding during learning (e.g., Harrison & Horne, 2000; Drummond et al., 2000). Follow-up brain imaging studies suggest that sleep loss may compromise normal functioning of the prefrontal and medial temporal lobe, leading to compensatory activation in the parietal lobe (e.g., Drummond et al., 2000; Drummond & Brown, 2001). While the medial temporal lobe is shown to be an important region for declarative memory formation and retrieval, the prefrontal lobe is critically involved not only in memory encoding and retrieval (e.g., Brewer et al., 1998), but also in attention and executive functioning (e.g., Hampshire et al., 2010). Indeed, sleep loss has also been reported to significantly affect attention and executive functioning (e.g., Drummond & Brown, 2001; Alhola & Polo-Kantola, 2007; Chee et al., 2010). In particular, sleep deprivation is reported to impair performance in visuospatial attention tasks, and this impairment is associated with decreased activation in the attention network comprising the prefrontal, parietal, and cingulate cortex (e.g., Tomasi et al., 2009; Benjamin, 2008). The impairment in the attention network due to sleep loss may have a profound impact on cognitive performance in general, as it can significantly influence how task relevant/irrelevant information is selected/inhibited. For example, while previous studies have suggested that sleep loss can influence performance in memory tasks, it remains unclear whether this change in performance is related to impairment in attention in addition to memory related processes. To address this issue, here we aim to examine whether sleep loss influences participants’ eye movement patterns in face recognition tasks, since eye movements are important measures for participants’ visuospatial attention allocation. We will also examine whether the changes in eye movement patterns are associated with their task performances. Despite the importance of eye movement measures, most of the current analysis methods focus on spatial information of eye movements such as fixation locations, whereas temporal information, such as transitions among fixation locations, is typically overlooked. In view of this, we have recently proposed a hidden Markov model (HMM, a type of time-series probabilistic model in machine learning) based approach for eye movement data analysis (Chuk, Chan, & Hsiao, 2014). This approach is based on the assumption that current eye fixation in a cognitive task is conditioned on previous fixations. Thus, eye movements in the task may be considered a Markovian stochastic process, which can be better understood using HMMs. Each participant’s eye movement pattern in the task is summarized with person-specific ROIs learned from data and a transition matrix indicating the probabilities of eye movements transiting from one ROI to another. Individual HMMs can be clustered according to their similarities (Coviello, Chan, & Lanckriet, 2014) to automatically discover common patterns within participants. Through this clustering we discovered holistic (mainly looking at the face centre) and analytic (looking at both individual eyes and the face centre) eye movement patterns (e.g., Chan, Chan, Lee, & Hsiao, 2015) in face recognition. People using analytic patterns outperformed those using holistic patterns, demonstrating a link between eye movements and cognitive performance (Chuk et al., 2014b). Our follow-up fMRI study showed that analytic patterns were associated with higher activation in brain regions important for top-down control of visual attention (Chan et al., 2016). Also, local attention priming increased participants’likelihood of using analytic patterns and enhanced their recognition performance (Cheng et al., 2015). Together these results suggest that analytic patterns are associated with engagement of top-down control of attention, which consequently lead to better recognition performance. Since sleep loss is associated with decreased activation in the brain network important for top-down control of attention (Tomasi et al., 2009; Benjamin, 2008), it is possible that sleep loss can influence eye movement behaviour, which in turn affect performance in face recognition tasks. Indeed, sleep deprivation has been reported to impair face recognition memory performance (Sheth et a., 2009) and recognition accuracy and intensity judgments of some emotional facial expressions (e.g., Maccari et al., 2014; Van Der Hel m et al., 2010). For example, van der Helm et al. (2014) found that sleep deprived individuals perceived angry and happy expressions as less emotional than controls. In another study, sleep deprived participants were less accurate in recognizing sad faces (Cote et al., 2014). In addition, individuals with insomnia were reported to rate fearful and sad faces as less emotional (Kyle et al., 2014). Nevertheless, as pointed out by Kyle et al. (2014), the underlying mechanism for the association between emotional face recognition and sleep loss is not well understood. Here we aim to test the hypothesis that this association may be due to the impact of sleep loss on eye movement planning/top-down attention control, which in turn influences face recognition and perception. To test this hypothesis, we will recruit individuals with significant self-reported insomnia symptoms and matched controls to perform face recognition memory and facial expression judgment tasks with eye tracking and analyse eye movement data through our HMM based approach. We expect that participants with sleep loss will show less analytic eye movement patterns and worse recognition performance than controls: the less analytic the pattern is, the worse the recognition performance.

Funding Source: UGC - Funding from The University of Hong Kong^