PR-PL: A novel prototypical representation based pairwise learning framework for emotion recognition using EEG signals

Rushuang ZHOU, Zhiguo ZHANG, Hong FU, Li ZHANG, Linling LI, Gan HUANG, Fali LI, Xin YANG, Yining DONG, Yuan-Ting ZHANG, Zhen LIANG

Research output: Contribution to journalArticlespeer-review

2 Citations (Scopus)

Abstract

Affective brain-computer interface based on electroencephalography (EEG) is an important branch in the field of affective computing. However, the individual differences in EEG emotional data and the noisy labeling problem in the subjective feedback seriously limit the effectiveness and generalizability of existing models. To tackle these two critical issues, we propose a novel transfer learning framework with Prototypical Representation based Pairwise Learning ( PR-PL ). The discriminative and generalized EEG features are learned for emotion revealing across individuals and the emotion recognition task is formulated as pairwise learning for improving the model tolerance to the noisy labels. More specifically, a prototypical learning is developed to encode the inherent emotion-related semantic structure of EEG data and align the individuals' EEG features to a shared common feature space under consideration of the feature separability of both source and target domains. Based on the aligned feature representations, pairwise learning with an adaptive pseudo labeling method is introduced to encode the proximity relationships among samples and alleviate the label noises effect on modeling. Extensive results on two benchmark databases (SEED and SEED-IV) under four different cross-validation evaluation protocols validate the model reliability and stability across subjects and sessions. Compared to the literature, the average enhancement of emotion recognition across four different evaluation protocols is 2.04% (SEED) and 2.58% (SEED-IV). The source code is available at https://github.com/KAZABANA/PR-PL . Copyright © 2023 IEEE.
Original languageEnglish
Pages (from-to)657-670
JournalIEEE Transactions on Affective Computing
Volume15
Issue number2
Early online dateJun 2023
DOIs
Publication statusPublished - Apr 2024

Citation

Zhou, R., Zhang, Z., Fu, H., Zhang, L., Li, L., Huang, G., Li, F., Yang, X., Dong, Y., Zhang, Y.-T., & Liang, Z. (2024). PR-PL: A novel prototypical representation based pairwise learning framework for emotion recognition using EEG signals. IEEE Transactions on Affective Computing, 15(2), 657-670. https://doi.org/10.1109/TAFFC.2023.3288118

Fingerprint

Dive into the research topics of 'PR-PL: A novel prototypical representation based pairwise learning framework for emotion recognition using EEG signals'. Together they form a unique fingerprint.