SemEval-2017 Task 12: Clinical TempEval

Steven Bethard, Guergana Savova, Martha Palmer, James Pustejovsky

Research output: Chapter in Book/Report/Conference proceedingConference contribution

77 Scopus citations

Abstract

Clinical TempEval 2017 aimed to answer the question: how well do systems trained on annotated timelines for one medical condition (colon cancer) perform in predicting timelines on another medical condition (brain cancer)? Nine sub-tasks were included, covering problems in time expression identification, event expression identification and temporal relation identification. Participant systems were evaluated on clinical and pathology notes from Mayo Clinic cancer patients, annotated with an extension of TimeML for the clinical domain. 11 teams participated in the tasks, with the best systems achieving F1 scores above 0.55 for time expressions, above 0.70 for event expressions, and above 0.30 for temporal relations. Most tasks observed about a 20 point drop over Clinical TempEval 2016, where systems were trained and evaluated on the same domain (colon cancer).

Original languageEnglish (US)
Title of host publicationACL 2017 - 11th International Workshop on Semantic Evaluations, SemEval 2017, Proceedings of the Workshop
PublisherAssociation for Computational Linguistics (ACL)
Pages565-572
Number of pages8
ISBN (Electronic)9781945626555
StatePublished - 2017
Event11th International Workshop on Semantic Evaluations, SemEval 2017, co-located with the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017 - Vancouver, Canada
Duration: Aug 3 2017Aug 4 2017

Publication series

NameProceedings of the Annual Meeting of the Association for Computational Linguistics
ISSN (Print)0736-587X

Conference

Conference11th International Workshop on Semantic Evaluations, SemEval 2017, co-located with the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017
Country/TerritoryCanada
CityVancouver
Period8/3/178/4/17

ASJC Scopus subject areas

  • Computer Science Applications
  • Linguistics and Language
  • Language and Linguistics

Fingerprint

Dive into the research topics of 'SemEval-2017 Task 12: Clinical TempEval'. Together they form a unique fingerprint.

Cite this