Can language models serve as temporal knowledge bases?

Ruilin ZHAO, Feng ZHAO, Guandong XU, Sixiao ZHANG, Hai JIN

Research output: Chapter in Book/Report/Conference proceedingChapters

2 Citations (Scopus)

Abstract

Recent progress regarding the use of language models (LMs) as knowledge bases (KBs) has shown that language models can act as structured knowledge bases for storing relational facts. However, most existing works only considered the LM-as-KB paradigm in a static setting, which ignores the analysis of temporal dynamics of world knowledge. Furthermore, a basic function of KBs, i.e., the ability to store conflicting information (i.e., 1-N, N-1 and N-M relations), is underexplored. In this paper, we formulate two practical requirements for treating LMs as temporal KBs: (i) the capacity to store temporally-scoped knowledge that contains conflicting information and (ii) the ability to use stored knowledge for temporally-scoped knowledge queries. We introduce a new dataset called LAMA-TK which is aimed at probing temporally-scoped knowledge, and investigate the two above requirements to explore the LM-as-KB paradigm in the temporal domain. On the one hand, experiments show that LMs can memorize millions of temporally-scoped facts with relatively high accuracy and transfer stored knowledge to temporal knowledge queries, thereby expanding the LM-as-KB paradigm to the temporal domain. On the other hand, we show that memorizing conflicting information, which has been neglected by previous works, is still challenging for LMs and hinders the memorization of other unrelated one-to-one relationships. Copyright © 2022 Association for Computational Linguistics.

Original languageEnglish
Title of host publicationProceedings of Findings of the Association for Computational Linguistics: EMNLP 2022
Place of PublicationAbu Dhabi, United Arab Emirates
PublisherAssociation for Computational Linguistics (ACL)
Pages2024-2037
Publication statusPublished - 2022

Citation

Zhao, R., Zhao, F., Xu, G., Zhang, S., & Jin, H. (2022). Can language models serve as temporal knowledge bases? In Proceedings of Findings of the Association for Computational Linguistics: EMNLP 2022 (pp. 2024-2037). Association for Computational Linguistics (ACL).

Fingerprint

Dive into the research topics of 'Can language models serve as temporal knowledge bases?'. Together they form a unique fingerprint.