Co-training an improved recurrent neural network with probability statistic models for named entity recognition

Yueqing SUN, Lin LI, Zhongwei XIE, Qing XIE, Xin LI, Guandong XU

Research output: Chapter in Book/Report/Conference proceedingChapters

9 Citations (Scopus)

Abstract

Named Entity Recognition (NER) is a subtask of information extraction in Natural Language Processing (NLP) field and thus being wildly studied. Currently Recurrent Neural Network (RNN) has become a popular way to do NER task, but it needs a lot of train data. The lack of labeled train data is one of the hard problems and traditional co-training strategy is a way to alleviate it. In this paper, we consider this situation and focus on doing NER with co-training using RNN and two probability statistic models i.e. Hidden Markov Model (HMM) and Conditional Random Field (CRF). We proposed a modified RNN model by redefining its activation function. Compared to traditional sigmoid function, our new function avoids saturation to some degree and makes its output scope very close to [0, 1], thus improving recognition accuracy. Our experiments are conducted ATIS benchmark. First, supervised learning using those models are compared when using different train data size. The experimental results show that it is not necessary to use whole data, even small part of train data can also get good performance. Then, we compare the results of our modified RNN with original RNN. 0.5% improvement is obtained. Last, we compare the co-training results. HMM and CRF get higher improvement than RNN after co-training. Moreover, using our modified RNN in co-training, their performances are improved further. Copyright © 2017 Springer International Publishing AG.

Original languageEnglish
Title of host publicationDatabase systems for advanced applications: 22nd International Conference, DASFAA 2017, Suzhou, China, March 27-30, 2017, proceedings, part II
EditorsSelçuk CANDAN, Lei CHEN, Torben Bach PEDERSEN, Lijun CHANG, Wen HUA
Place of PublicationCham
PublisherSpringer
Pages545-555
ISBN (Electronic)9783319556994
ISBN (Print)9783319556987
DOIs
Publication statusPublished - 2017

Citation

Sun, Y., Li, L., Xie, Z., Xie, Q., Li, X., & Xu, G. (2017). Co-training an improved recurrent neural network with probability statistic models for named entity recognition. In S. Candan, L. Chen, T. B. Pedersen, L. Chang, & W. Hua (Eds.), Database systems for advanced applications: 22nd International Conference, DASFAA 2017, Suzhou, China, March 27-30, 2017, proceedings, part II (pp. 545-555). Springer. https://doi.org/10.1007/978-3-319-55699-4_33

Keywords

  • Named entity recognition
  • Co-training
  • Recurrent neural network
  • Probability statistic model
  • Natural language processing

Fingerprint

Dive into the research topics of 'Co-training an improved recurrent neural network with probability statistic models for named entity recognition'. Together they form a unique fingerprint.