Automatic Construction of Parallel Dialogue Corpora with Rich Information

Citation

Zhang X, Wang L, Liu Q & Way A (2021) Automatic Construction of Parallel Dialogue Corpora with Rich Information. In: Huang C (ed.) Forthcoming Volume. Text, Speech and Language Technology. Cham, Switzerland: Springer.

Abstract
Due to the lack of ideal resources, few researchers have investigated how to improve the machine translation (MT) of conversational material by exploiting their internal structure. In this article, we propose a novel strategy to automatically construct parallel dialogue corpus by bridging two kinds of resources: movie subtitle and movie script. First of all, we crawl both parallel subtitles and their corresponding monolingual scripts from the Internet. After sentence alignment, we can then project all useful information from the script side to its corresponding subtitle side. Finally, we automatically build a Chinese--English dialogue corpus, which contains bilingual subtitle utterances, speaker name and action, scene description and boundary, as well as script sentence. In order to demonstrate the usefulness of our data, we explore to use speaker name tags to improve the translation performance. Experiments show that our approach can achieve 81.79\% accuracy on speaker name annotation, and speaker-based model adaptation can obtain around 0.5 BLEU point improvement in translation qualities. We believe that our resources can benefit various tasks such as dialog system, image/movie description as well as MT.

Keywords
Dialogue; Machine Translation; Parallel Corpus; Movie Script; Movie Subtitle; Rich Information

Notes
Output Status: Forthcoming