RTE - Recognizing Textual Entailment
Textual Entailment can be defined as the directional relationship between two text fragments, termed Text (T) and Hypothesis (H). It is said that:
T ENTAILS H IF, TYPICALLY, A HUMAN READING T WOULD INFER THAT H IS MOST LIKELY TRUE
This definition of entailment is based on (and assumes) common human understanding of language as well as background knowledge (for more details see Bentivogli et a., 2010).
Since 2005, RTE Challenges have promoted research on Textual Entailment Recognition as a generic task that captures major semantic inference
needs across many natural language processing applications, such as Question Answering (QA), Information Retrieval (IR), Information Extraction
(IE), and multi-document summarization (SUM), providing a common solution for modeling language variability.
The Textual Entailment Recognition task has raised increasing interest in the NLP community, as it seems to work as a common framework in
which to analyse, compare and evaluate different techniques used in NLP applications to deal with semantic inference, a common issue shared
by many NLP applications.
To test the advance in the field, annual RTE challenges have been organized since 2005. In the first five competitions, systems were required to perform Textual Entailment over isolated T-H pairs. In such a framework, both Text and Hypothesis were artificially created in a way that they did not contain any references to information outside the T-H pair.
Since RTE-5 pilot, the task moved toward a more realistic scenario, in which both T and H are to be interpreted in the context of the corpus, as they rely on explicit and implicit references to entities, events, dates, places, situations, etc. pertaining to the topic.
In RTE-6 a new Main Task, situated in the Summarization application setting, was presented, as a close variant of the Pilot Search Task in RTE-5. Moreover, a KBP Validation Pilot task, situated in the Knowledge Base Population scenario, was proposed, aiming at determining whether a given relation (Hypothesis) is supported in an associated document (Text) (for more information see the TAC RTE website). The same tasks proposed in RTE-6 will be presented also in the RTE-7 Challenge, which is currently under preparation.
CELCT contributed to the preparation of the data sets and the organization of the RTE-2 and RTE-3 challenges together with Bar-Ilan University.
RTE-4 was presented for the first time in 2008 as a track of the Text Analysis Conference (TAC), jointly organized by CELCT and NIST. As the collaboration was successful, the two institutions have also organized together the following RTE challenges (i.e. RTE-5, RTE-6, and RTE-7 which is currently being prepared).
Recognizing Textual Entailment page on ACL Wiki
TAC Web Page