Customizing an Information Extraction System to a New Domain

Mihai Surdeanu; David McClosky; Mason R. Smith; Andrey Gusev; Christopher D. Manning

Customizing an Information Extraction System to a New Domain

Mihai Surdeanu, David McClosky, Mason R. Smith, Andrey Gusev, Christopher D. Manning

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We introduce several ideas that improve the performance of supervised information extraction systems with a pipeline architecture, when they are customized for new domains. We show that: (a) a combination of a sequence tagger with a rule-based approach for entity mention extraction yields better performance for both entity and relation mention extraction; (b) improving the identification of syntactic heads of entity mentions helps relation extraction; and (c) a deterministic inference engine captures some of the joint domain structure, even when introduced as a postprocessing step to a pipeline system. All in all, our contributions yield a 20% relative increase in F1 score in a domain significantly different from the domains used during the development of our information extraction system.

Original language	English (US)
Title of host publication	Workshop on Relational Models of Semantics, RELMS 2011 at the 49th Annual Meeting of the Association for Computational Linguistics
Subtitle of host publication	Human Language Technologies, ACL-HLT 2011 - Proceedings
Publisher	Association for Computational Linguistics (ACL)
Pages	2-10
Number of pages	9
ISBN (Electronic)	9781932432985
State	Published - 2011
Externally published	Yes
Event	ACL 2011 Workshop on Relational Models of Semantics, RELMS 2011 - Portland, United States Duration: Jun 23 2011 → …

Publication series

Name	Proceedings of the Annual Meeting of the Association for Computational Linguistics
ISSN (Print)	0736-587X

Conference

Conference	ACL 2011 Workshop on Relational Models of Semantics, RELMS 2011
Country/Territory	United States
City	Portland
Period	6/23/11 → …

ASJC Scopus subject areas

Computer Science Applications
Linguistics and Language
Language and Linguistics

Cite this

Surdeanu, M., McClosky, D., Smith, M. R., Gusev, A., & Manning, C. D. (2011). Customizing an Information Extraction System to a New Domain. In Workshop on Relational Models of Semantics, RELMS 2011 at the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, ACL-HLT 2011 - Proceedings (pp. 2-10). (Proceedings of the Annual Meeting of the Association for Computational Linguistics). Association for Computational Linguistics (ACL).

Customizing an Information Extraction System to a New Domain. / Surdeanu, Mihai; McClosky, David; Smith, Mason R. et al.
Workshop on Relational Models of Semantics, RELMS 2011 at the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, ACL-HLT 2011 - Proceedings. Association for Computational Linguistics (ACL), 2011. p. 2-10 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Surdeanu, M, McClosky, D, Smith, MR, Gusev, A & Manning, CD 2011, Customizing an Information Extraction System to a New Domain. in Workshop on Relational Models of Semantics, RELMS 2011 at the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, ACL-HLT 2011 - Proceedings. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics (ACL), pp. 2-10, ACL 2011 Workshop on Relational Models of Semantics, RELMS 2011, Portland, United States, 6/23/11.

Surdeanu M, McClosky D, Smith MR, Gusev A, Manning CD. Customizing an Information Extraction System to a New Domain. In Workshop on Relational Models of Semantics, RELMS 2011 at the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, ACL-HLT 2011 - Proceedings. Association for Computational Linguistics (ACL). 2011. p. 2-10. (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

Surdeanu, Mihai ; McClosky, David ; Smith, Mason R. et al. / Customizing an Information Extraction System to a New Domain. Workshop on Relational Models of Semantics, RELMS 2011 at the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, ACL-HLT 2011 - Proceedings. Association for Computational Linguistics (ACL), 2011. pp. 2-10 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

@inproceedings{83205abe6fc54f3798c5ef6bb2596f31,

title = "Customizing an Information Extraction System to a New Domain",

abstract = "We introduce several ideas that improve the performance of supervised information extraction systems with a pipeline architecture, when they are customized for new domains. We show that: (a) a combination of a sequence tagger with a rule-based approach for entity mention extraction yields better performance for both entity and relation mention extraction; (b) improving the identification of syntactic heads of entity mentions helps relation extraction; and (c) a deterministic inference engine captures some of the joint domain structure, even when introduced as a postprocessing step to a pipeline system. All in all, our contributions yield a 20% relative increase in F1 score in a domain significantly different from the domains used during the development of our information extraction system.",

author = "Mihai Surdeanu and David McClosky and Smith, {Mason R.} and Andrey Gusev and Manning, {Christopher D.}",

note = "Publisher Copyright: {\textcopyright} Proceedings of the Annual Meeting of the Association for Computational Linguistics 2011.; ACL 2011 Workshop on Relational Models of Semantics, RELMS 2011 ; Conference date: 23-06-2011",

year = "2011",

language = "English (US)",

series = "Proceedings of the Annual Meeting of the Association for Computational Linguistics",

publisher = "Association for Computational Linguistics (ACL)",

pages = "2--10",

booktitle = "Workshop on Relational Models of Semantics, RELMS 2011 at the 49th Annual Meeting of the Association for Computational Linguistics",

}

TY - GEN

T1 - Customizing an Information Extraction System to a New Domain

AU - Surdeanu, Mihai

AU - McClosky, David

AU - Smith, Mason R.

AU - Gusev, Andrey

AU - Manning, Christopher D.

PY - 2011

Y1 - 2011

N2 - We introduce several ideas that improve the performance of supervised information extraction systems with a pipeline architecture, when they are customized for new domains. We show that: (a) a combination of a sequence tagger with a rule-based approach for entity mention extraction yields better performance for both entity and relation mention extraction; (b) improving the identification of syntactic heads of entity mentions helps relation extraction; and (c) a deterministic inference engine captures some of the joint domain structure, even when introduced as a postprocessing step to a pipeline system. All in all, our contributions yield a 20% relative increase in F1 score in a domain significantly different from the domains used during the development of our information extraction system.

AB - We introduce several ideas that improve the performance of supervised information extraction systems with a pipeline architecture, when they are customized for new domains. We show that: (a) a combination of a sequence tagger with a rule-based approach for entity mention extraction yields better performance for both entity and relation mention extraction; (b) improving the identification of syntactic heads of entity mentions helps relation extraction; and (c) a deterministic inference engine captures some of the joint domain structure, even when introduced as a postprocessing step to a pipeline system. All in all, our contributions yield a 20% relative increase in F1 score in a domain significantly different from the domains used during the development of our information extraction system.

UR - http://www.scopus.com/inward/record.url?scp=84944053050&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84944053050&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84944053050

T3 - Proceedings of the Annual Meeting of the Association for Computational Linguistics

SP - 2

EP - 10

BT - Workshop on Relational Models of Semantics, RELMS 2011 at the 49th Annual Meeting of the Association for Computational Linguistics

PB - Association for Computational Linguistics (ACL)

T2 - ACL 2011 Workshop on Relational Models of Semantics, RELMS 2011

Y2 - 23 June 2011

ER -

Customizing an Information Extraction System to a New Domain

Abstract

Publication series

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this