Non-contact PPG signal and heart rate estimation with multi-hierarchical convolutional network

Bin LI, Panpan ZHANG, Jinye PENG, Hong FU

Research output: Contribution to journalArticlespeer-review

14 Citations (Scopus)

Abstract

Heartbeat rhythm and heart rate (HR) are important physiological parameters of the human body. This study presents an efficient multi-hierarchical spatio-temporal convolutional network that can quickly estimate remote physiological (rPPG) signal and HR from face video clips. First, the facial color distribution characteristics are extracted using a low-level face feature generation (LFFG) module. Then, the three-dimensional (3D) spatio-temporal stack convolution module (STSC) and multi-hierarchical feature fusion module (MHFF) are used to strengthen the spatio-temporal correlation of multi-channel features. In the MHFF, sparse optical flow is used to capture the tiny motion information of faces between frames and generate a self-adaptive region of interest (ROI) skin mask. Finally, the signal prediction module (SP) is used to extract the estimated rPPG signal. The heart rate estimation results show that the proposed network overperforms the state-of-the-art methods on three datasets, 1) UBFC-RPPG, 2) COHFACE, 3) our dataset, with the mean absolute error (MAE) of 2.15, 5.57, 1.75 beats per minute (bpm) respectively. Copyright © 2023 Elsevier Ltd. All rights reserved.

Original languageEnglish
Article number109421
JournalPattern Recognition
Volume139
Early online dateFeb 2023
DOIs
Publication statusPublished - Jul 2023

Citation

Li, B., Zhang, P., Peng, J., & Fu, H. (2023). Non-contact PPG signal and heart rate estimation with multi-hierarchical convolutional network. Pattern Recognition, 139. Retrieved from https://doi.org/10.1016/j.patcog.2023.109421

Fingerprint

Dive into the research topics of 'Non-contact PPG signal and heart rate estimation with multi-hierarchical convolutional network'. Together they form a unique fingerprint.