TextGraphs 2021 Shared Task on Multi-Hop Inference for Explanation Regeneration

Mokanarangan Thayaparan, Marco Valentino, Peter Jansen, Dmitry Ustalov

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Scopus citations

Abstract

The Shared Task on Multi-Hop Inference for Explanation Regeneration asks participants to compose large multi-hop explanations to questions by assembling large chains of facts from a supporting knowledge base. While previous editions of this shared task aimed to evaluate explanatory completeness - finding a set of facts that form a complete inference chain, without gaps, to arrive from question to correct answer, this 2021 instantiation concentrates on the subtask of determining relevance in large multi-hop explanations. To this end, this edition of the shared task makes use of a large set of approximately 250k manual explanatory relevancy ratings that augment the 2020 shared task data. In this summary paper, we describe the details of the explanation regeneration task, the evaluation data, and the participating systems. Additionally, we perform a detailed analysis of participating systems, evaluating various aspects involved in the multi-hop inference process. The best performing system achieved an NDCG of 0.82 on this challenging task, substantially increasing performance over baseline methods by 32%, while also leaving significant room for future improvement.

Original languageEnglish (US)
Title of host publicationTextGraphs 2021 - Graph-Based Methods for Natural Language Processing, Proceedings of the 15th Workshop - in conjunction with the 2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics, NAACL 2021
EditorsAlexander Panchenko, Fragkiskos D. Malliaros, Varvara Logacheva, Abhik Jana, Dmitry Ustalov, Peter Jansen
PublisherAssociation for Computational Linguistics (ACL)
Pages156-165
Number of pages10
ISBN (Electronic)9781954085381
StatePublished - 2021
Event15th Workshop on Graph-Based Methods for Natural Language Processing, TextGraphs 2021 - Mexico City, Mexico
Duration: Jun 11 2021 → …

Publication series

NameTextGraphs 2021 - Graph-Based Methods for Natural Language Processing, Proceedings of the 15th Workshop - in conjunction with the 2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics, NAACL 2021

Conference

Conference15th Workshop on Graph-Based Methods for Natural Language Processing, TextGraphs 2021
Country/TerritoryMexico
CityMexico City
Period6/11/21 → …

ASJC Scopus subject areas

  • Language and Linguistics
  • Computational Theory and Mathematics
  • Software
  • Linguistics and Language

Fingerprint

Dive into the research topics of 'TextGraphs 2021 Shared Task on Multi-Hop Inference for Explanation Regeneration'. Together they form a unique fingerprint.

Cite this