Dyformer: A dynamic transformer-based architecture for multivariate time series classification

Chao YANG, Xianzhi WANG, Lina YAO, Guodong LONG, Guandong XU

Research output: Contribution to journalArticlespeer-review

3 Citations (Scopus)

Abstract

Multivariate time series classification is a crucial task with applications in broad areas such as finance, medicine, and engineering. Transformer is promising for time series classification, but as a generic approach, they have limited capability to effectively capture the distinctive characteristics inherent in time series data and adapt to diverse architectural requirements. This paper proposes a novel dynamic transformer-based architecture called Dyformer to address the above limitations of traditional transformers in multivariate time series classification. Dyformer incorporates hierarchical pooling to decompose time series into subsequences with different frequency components. Then, it employs Dyformer modules to achieve adaptive learning strategies for different frequency components based on a dynamic architecture. Furthermore, we introduce feature-map-wise attention mechanisms to capture multi-scale temporal dependencies and a joint loss function to facilitate model training. To evaluate the performance of Dyformer, we conducted extensive experiments using 30 benchmark datasets. The results unequivocally demonstrate that our model consistently outperforms a multitude of state-of-the-art methods and baseline approaches. Our model also copes well with limited training samples when pre-trained. Copyright © 2023 Elsevier Inc. All rights reserved.

Original languageEnglish
Article number119881
JournalInformation Sciences
Volume656
Early online dateNov 2023
DOIs
Publication statusPublished - Jan 2024

Citation

Yang, C., Wang, X., Yao, L., Long, G., & Xu, G. (2024). Dyformer: A dynamic transformer-based architecture for multivariate time series classification. Information Sciences, 656, Article 119881. https://doi.org/10.1016/j.ins.2023.119881

Fingerprint

Dive into the research topics of 'Dyformer: A dynamic transformer-based architecture for multivariate time series classification'. Together they form a unique fingerprint.