End-to-end clinical temporal information extraction with multi-head attention

Timothy Miller; Steven Bethard; Dmitriy Dligach; Guergana Savova

End-to-end clinical temporal information extraction with multi-head attention

Timothy Miller, Steven Bethard, Dmitriy Dligach, Guergana Savova

Information, School of

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Scopus citations

Abstract

Understanding temporal relationships in text from electronic health records can be valuable for many important downstream clinical applications. Since Clinical TempEval 2017, there has been little work on end-to-end systems for temporal relation extraction, with most work focused on the setting where gold standard events and time expressions are given. In this work, we make use of a novel multi-headed attention mechanism on top of a pre-trained transformer encoder to allow the learning process to attend to multiple aspects of the contextualized embeddings. Our system achieves state of the art results on the THYME corpus by a wide margin, in both the in-domain and cross-domain settings.

Original language	English (US)
Title of host publication	BioNLP 2023 - BioNLP and BioNLP-ST, Proceedings of the Workshop
Editors	Dina Demner-fushman, Sophia Ananiadou, Kevin Cohen
Publisher	Association for Computational Linguistics (ACL)
Pages	313-319
Number of pages	7
ISBN (Electronic)	9781959429852
State	Published - 2023
Event	22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks, BioNLP 2023 - Toronto, Canada Duration: Jul 13 2023 → …

Publication series

Name	Proceedings of the Annual Meeting of the Association for Computational Linguistics
ISSN (Print)	0736-587X

Conference

Conference	22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks, BioNLP 2023
Country/Territory	Canada
City	Toronto
Period	7/13/23 → …

ASJC Scopus subject areas

Computer Science Applications
Linguistics and Language
Language and Linguistics

Cite this

Miller, T., Bethard, S., Dligach, D., & Savova, G. (2023). End-to-end clinical temporal information extraction with multi-head attention. In D. Demner-fushman, S. Ananiadou, & K. Cohen (Eds.), BioNLP 2023 - BioNLP and BioNLP-ST, Proceedings of the Workshop (pp. 313-319). (Proceedings of the Annual Meeting of the Association for Computational Linguistics). Association for Computational Linguistics (ACL).

End-to-end clinical temporal information extraction with multi-head attention. / Miller, Timothy; Bethard, Steven; Dligach, Dmitriy et al.
BioNLP 2023 - BioNLP and BioNLP-ST, Proceedings of the Workshop. ed. / Dina Demner-fushman; Sophia Ananiadou; Kevin Cohen. Association for Computational Linguistics (ACL), 2023. p. 313-319 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Miller, T, Bethard, S, Dligach, D & Savova, G 2023, End-to-end clinical temporal information extraction with multi-head attention. in D Demner-fushman, S Ananiadou & K Cohen (eds), BioNLP 2023 - BioNLP and BioNLP-ST, Proceedings of the Workshop. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics (ACL), pp. 313-319, 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks, BioNLP 2023, Toronto, Canada, 7/13/23.

Miller T, Bethard S, Dligach D, Savova G. End-to-end clinical temporal information extraction with multi-head attention. In Demner-fushman D, Ananiadou S, Cohen K, editors, BioNLP 2023 - BioNLP and BioNLP-ST, Proceedings of the Workshop. Association for Computational Linguistics (ACL). 2023. p. 313-319. (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

Miller, Timothy ; Bethard, Steven ; Dligach, Dmitriy et al. / End-to-end clinical temporal information extraction with multi-head attention. BioNLP 2023 - BioNLP and BioNLP-ST, Proceedings of the Workshop. editor / Dina Demner-fushman ; Sophia Ananiadou ; Kevin Cohen. Association for Computational Linguistics (ACL), 2023. pp. 313-319 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

@inproceedings{5c83c49881a54c5baa8a7535cf70a87e,

title = "End-to-end clinical temporal information extraction with multi-head attention",

abstract = "Understanding temporal relationships in text from electronic health records can be valuable for many important downstream clinical applications. Since Clinical TempEval 2017, there has been little work on end-to-end systems for temporal relation extraction, with most work focused on the setting where gold standard events and time expressions are given. In this work, we make use of a novel multi-headed attention mechanism on top of a pre-trained transformer encoder to allow the learning process to attend to multiple aspects of the contextualized embeddings. Our system achieves state of the art results on the THYME corpus by a wide margin, in both the in-domain and cross-domain settings.",

author = "Timothy Miller and Steven Bethard and Dmitriy Dligach and Guergana Savova",

note = "Publisher Copyright: {\textcopyright} 2023 Association for Computational Linguistics.; 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks, BioNLP 2023 ; Conference date: 13-07-2023",

year = "2023",

language = "English (US)",

series = "Proceedings of the Annual Meeting of the Association for Computational Linguistics",

publisher = "Association for Computational Linguistics (ACL)",

pages = "313--319",

editor = "Dina Demner-fushman and Sophia Ananiadou and Kevin Cohen",

booktitle = "BioNLP 2023 - BioNLP and BioNLP-ST, Proceedings of the Workshop",

}

TY - GEN

T1 - End-to-end clinical temporal information extraction with multi-head attention

AU - Miller, Timothy

AU - Bethard, Steven

AU - Dligach, Dmitriy

AU - Savova, Guergana

PY - 2023

Y1 - 2023

N2 - Understanding temporal relationships in text from electronic health records can be valuable for many important downstream clinical applications. Since Clinical TempEval 2017, there has been little work on end-to-end systems for temporal relation extraction, with most work focused on the setting where gold standard events and time expressions are given. In this work, we make use of a novel multi-headed attention mechanism on top of a pre-trained transformer encoder to allow the learning process to attend to multiple aspects of the contextualized embeddings. Our system achieves state of the art results on the THYME corpus by a wide margin, in both the in-domain and cross-domain settings.

AB - Understanding temporal relationships in text from electronic health records can be valuable for many important downstream clinical applications. Since Clinical TempEval 2017, there has been little work on end-to-end systems for temporal relation extraction, with most work focused on the setting where gold standard events and time expressions are given. In this work, we make use of a novel multi-headed attention mechanism on top of a pre-trained transformer encoder to allow the learning process to attend to multiple aspects of the contextualized embeddings. Our system achieves state of the art results on the THYME corpus by a wide margin, in both the in-domain and cross-domain settings.

UR - http://www.scopus.com/inward/record.url?scp=85174485048&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85174485048&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85174485048

T3 - Proceedings of the Annual Meeting of the Association for Computational Linguistics

SP - 313

EP - 319

BT - BioNLP 2023 - BioNLP and BioNLP-ST, Proceedings of the Workshop

A2 - Demner-fushman, Dina

A2 - Ananiadou, Sophia

A2 - Cohen, Kevin

PB - Association for Computational Linguistics (ACL)

T2 - 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks, BioNLP 2023

Y2 - 13 July 2023

ER -

End-to-end clinical temporal information extraction with multi-head attention

Abstract

Publication series

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this