Multi-hop Inference for Sentence-level TextGraphs: How Challenging is Meaningfully Combining Information for Science Question Answering?

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations

Abstract

Question Answering for complex questions is often modelled as a graph construction or traversal task, where a solver must build or traverse a graph of facts that answer and explain a given question. This “multi-hop” inference has been shown to be extremely challenging, with few models able to aggregate more than two facts before being overwhelmed by “semantic drift”, or the tendency for long chains of facts to quickly drift off topic. This is a major barrier to current inference models, as even elementary science questions require an average of 4 to 6 facts to answer and explain. In this work we empirically characterize the difficulty of building or traversing a graph of sentences connected by lexical overlap, by evaluating chance sentence aggregation quality through 9,784 manually-annotated judgements across knowledge graphs built from three free-text corpora (including study guides and Simple Wikipedia). We demonstrate semantic drift tends to be high and aggregation quality low, at between 0.04% and 3%, and highlight scenarios that maximize the likelihood of meaningfully combining information.

Original languageEnglish (US)
Title of host publicationNAACL HLT 2018 - Graph-Based Methods for Natural Language Processing, TextGraphs 2018 - Proceedings of the 12th Workshop
EditorsGoran Glavas, Swapna Somasundaran, Martin Riedl, Eduard Hovy
PublisherAssociation for Computational Linguistics
Pages12-17
Number of pages6
ISBN (Electronic)9781948087254
StatePublished - 2018
Event12th Workshop on Graph-Based Methods for Natural Language Processing, TextGraphs 2018 - in conjunction with the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human, NAACL HLT 2018 - New Orleans, United States
Duration: Jun 6 2018 → …

Publication series

NameNAACL HLT 2018 - Graph-Based Methods for Natural Language Processing, TextGraphs 2018 - Proceedings of the 12th Workshop

Conference

Conference12th Workshop on Graph-Based Methods for Natural Language Processing, TextGraphs 2018 - in conjunction with the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human, NAACL HLT 2018
Country/TerritoryUnited States
CityNew Orleans
Period6/6/18 → …

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Computer Science Applications
  • Theoretical Computer Science

Fingerprint

Dive into the research topics of 'Multi-hop Inference for Sentence-level TextGraphs: How Challenging is Meaningfully Combining Information for Science Question Answering?'. Together they form a unique fingerprint.

Cite this