The extraction and fine-grained classification of written Cantonese materials through linguistic feature detection

Chaak Ming LAU, Mingfei LAU, Ann Wai Huen TO

Research output: Chapter in Book/Report/Conference proceedingChapters

Abstract

This paper presents a linguistically-informed, non-machine-learning tool for classifying Written Cantonese, Standard Written Chinese, and the intermediate varieties used by Cantonese-speaking users from Hong Kong, which are often grouped into a single “Traditional Chinese” label. Our approach addresses the lack of textual materials for Cantonese NLP, a consequence of a lower sociolinguistic status of Written Cantonese and the interchangeable use of these varieties by users without sufficient language labeling. The tool utilizes key lexical markers identified from past linguistic research to determine whether a segment is Cantonese, Standard Written Chinese, mixed or unmarked. The task is reduced into string operations to allow for a flexible and efficient extraction of high-quality Cantonese data from large datasets mixed with Standard Written Chinese. This implementation ensures that the tool can process large amounts of data at a low cost by bypassing model-inferencing, which is particularly significant for marginalized languages. The tool also aims to provide a baseline measure for future classification systems, and the approach may be applicable to other low-resource regional or diglossic languages. Copyright © 2024 ELRA Language Resource Association: CC BY-NC 4.0.

Original languageEnglish
Title of host publicationProceedings of the 2nd Workshop on Resources and Technologies for Indigenous, Endangered and Lesser-resourced Languages in Eurasia (EURALI) @ LREC-COLING 2024
Place of PublicationTorino, Italia
PublisherELRA and ICCL
Pages24-29
ISBN (Electronic)9782493814333
Publication statusPublished - 2024

Citation

Lau, C.-M., Lau, M., & To, A. W. H. (2024). The extraction and fine-grained classification of written Cantonese materials through linguistic feature detection. In Proceedings of the 2nd Workshop on Resources and Technologies for Indigenous, Endangered and Lesser-resourced Languages in Eurasia (EURALI) @ LREC-COLING 2024 (pp. 24-29). ELRA and ICCL. https://aclanthology.org/2024.eurali-1.4/

Keywords

  • Language classifier
  • Cantonese
  • Diglossia

Fingerprint

Dive into the research topics of 'The extraction and fine-grained classification of written Cantonese materials through linguistic feature detection'. Together they form a unique fingerprint.