Temporal segment neural networks-enabled dynamic hand-gesture recognition for industrial cyber-physical authentication systems

Yuanlong CAO, Junjie LI, Chinmay CHAKRABORTY, Le QIN, Lei TAO, Xun SHAO

Research output: Contribution to journalArticlespeer-review

3 Citations (Scopus)

Abstract

In recent years, the data security and privacy protection of human cyber-physical systems (CPSs) in the industrial Internet of Things (IoT) have attracted much research interest. In the meantime, applying human–machine identity authentication mechanism to the CPSs has been considered as a promising solution for secure and privacy-enhancing authentication in industrial IoT. In view of the application limitation of traditional authentication methods, especially during the current COVID-19 virus outbreak, the mainstream face recognition and fingerprint recognition methods are constrained by wearing masks and gloves. Recently, with the flourishing studies of artificial intelligence (AI), many AI-based dynamic gesture recognition methods have been developed for the application of human–computer identity recognition. A temporal segment network (TSN) has been designed with a dual-stream convolutional neural network inside to match the flexibility of gesture transformations. Although it has achieved a success in accuracy, the TSN still has the main shortcomings of the insufficient temporal information fusion and the high cost of optical flow feature extraction. In this article, we propose an enhanced temporal segment network (dubbed as e -TSN). First, hand skeleton features are used instead of optical flow features. Second, a short-term networks (long short-term memory) is utilized as segmental consensus function to improve the accuracy and reduce the feature extraction cost. Experiments demonstrate that the e -TSN can achieve an accuracy of 91.3% on the Jester dataset. Finally, a human–machine identity verification system is developed based on the e -TSN, which can effectually accomplish the human–machine identity in real time and has a high promotion value. Copyright © 2023 Institute of Electrical and Electronics Engineers Inc.

Original languageEnglish
Pages (from-to)5315-5326
JournalIEEE Systems Journal
Volume17
Issue number4
Early online dateAug 2023
DOIs
Publication statusPublished - Dec 2023

Citation

Cao, Y., Li, J., Chakraborty, C., Qin, L., Tao, L., & Shao, X. (2023). Temporal segment neural networks-enabled dynamic hand-gesture recognition for industrial cyber-physical authentication systems. IEEE Systems Journal, 17(4), 5315-5326. https://doi.org/10.1109/JSYST.2023.3306380

Keywords

  • Artificial intelligence (AI)-based hand-gesture recognition
  • Cyber-physical systems (CPSs)
  • Human–machine authentication
  • Identity recognition
  • Industrial Internet of Things (IoT)
  • PG student publication

Fingerprint

Dive into the research topics of 'Temporal segment neural networks-enabled dynamic hand-gesture recognition for industrial cyber-physical authentication systems'. Together they form a unique fingerprint.