RTE5 Main Task: Dev Set available from May 29, 2009

The RTE5 main task consists in recognizing that the meaning of one text, termed Hypothesis (H), can be inferred by the content of another, termed Text (T). Given a set of pairs of T's and H's as input, the systems must recognize whether each T entails the corresponding H, deciding whether:

• T entails H
• T contradicts H, or shows it false
• the veracity of H is unknown on the basis of T.

The RTE5 main task will consist of two sub-tasks:

1. The three-way RTE task, where the system must decide whether:
   • T entails H - in which case the pair will be marked as ENTAILMENT
   • T contradicts H - in which case the pair will be marked as CONTRADICTION
   • The truth of H cannot be determined on the basis of T - in which case the pair will be marked as UNKNOWN
2. The two-way RTE task is to decide whether:
   • T entails H - in which case the pair will be marked as ENTAILMENT
   • T does not entail H - in which case the pair will be marked as NO ENTAILMENT

Systems can decide whether to participate in either or both tasks.
System results will be compared to a human-annotated gold-standard test corpus.

A set of pairs for system training and development will be available for download at the RTE5 website from May 29.