Is your mouse attracted by your eyes: Non-intrusive stress detection in off-the-shelf desktop environments

Increasing number of people work long hours with computers under high cognitive load. This could potentially cause mental stress in workplaces. Prolonged exposure to mental stress contributes to poor working experience and even severe health problems. Despite the growing demand, the existing intelligent stress detection methods are limited when applied to actual workplaces. They often measure physiological and physical signals, via intrusive devices, to detect stress. The intrusiveness hampers their accessibility and applicability in daily life and workplaces. To overcome that, behavior-based methods were proposed. Models that explore mouse and gaze behaviors during computer usages were demonstrated to be particularly effective. However, the current methods rely on using prior knowledge of the user interface (UI) layout to construct models. Their applicability thus is limited, especially in real workplaces where task UI is often dynamic. This paper presents a novel stress detection method to address the challenges. It attains non-intrusiveness and UI-agnostic by modeling the relative movement and coordination of mouse and gaze. The method is evaluated on a dynamic-UI task, namely, web searching. An accuracy of 78.8% is achieved using a commercial eye-tracker for gaze estimation, beating the state-of-the-art approaches by around 20%. We further use webcam to estimate gaze locations substituting for the eye-tracker, to enhance the model accessibility. The method yields 68.6% accuracy of stress detection without using any special devices. Experimental results demonstrate the effectiveness and applicability of our method. It opens up a new avenue for cognitive-aware adaptive user interface, intelligent working environment, and related applications. Copyright © 2023 Elsevier Ltd. All rights reserved.