Pre-trained contextualized character embeddings lead to major improvements in time normalization: A detailed analysis

Dongfang Xu, Egoitz Laparra, Steven Bethard

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations

Abstract

Recent studies have shown that pre-trained contextual word embeddings, which assign the same word different vectors in different contexts, improve performance in many tasks. But while contextual embeddings can also be trained at the character level, the effectiveness of such embeddings has not been studied. We derive character-level contextual embeddings from Flair (Akbik et al., 2018), and apply them to a time normalization task, yielding major performance improvements over the previous state-of-the-art: 51% error reduction in news and 33% in clinical notes. We analyze the sources of these improvements, and find that pre-trained contextual character embeddings are more robust to term variations, infrequent terms, and cross-domain changes. We also quantify the size of context that pre-trained contextual character embeddings take advantage of, and show that such embeddings capture features like part-of-speech and capitalization.

Original languageEnglish (US)
Title of host publication*SEM@NAACL-HLT 2019 - 8th Joint Conference on Lexical and Computational Semantics
PublisherAssociation for Computational Linguistics (ACL)
Pages68-74
Number of pages7
ISBN (Electronic)9781948087933
StatePublished - 2019
Event8th Joint Conference on Lexical and Computational Semantics, *SEM@NAACL-HLT 2019 - Minneapolis, United States
Duration: Jun 6 2019Jun 7 2019

Publication series

Name*SEM@NAACL-HLT 2019 - 8th Joint Conference on Lexical and Computational Semantics

Conference

Conference8th Joint Conference on Lexical and Computational Semantics, *SEM@NAACL-HLT 2019
Country/TerritoryUnited States
CityMinneapolis
Period6/6/196/7/19

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Computational Theory and Mathematics

Fingerprint

Dive into the research topics of 'Pre-trained contextualized character embeddings lead to major improvements in time normalization: A detailed analysis'. Together they form a unique fingerprint.

Cite this