Home page
Home page

CLTE Task@SemEval-2012 evaluation ongoing

The Cross-Lingual Textual Entailment task (CLTE) addresses textual entailment (TE) recognition under a new dimension (cross-linguality), and within a new challenging application scenario (content synchronization).

Given a pair of topically related text fragments (T1 and T2) in different languages, the CLTE task consists of automatically annotating it with one of the following entailment judgments:


  • Bidirectional (T1 ->T2 & T1 <- T2): the two fragments entail each other (semantic equivalence)
  • Forward (T1 -> T2 & T1 !<- T2): unidirectional entailment from T1 to T2
  • Backward (T1 !-> T2 & T1 <- T2): unidirectional entailment from T2 to T1
  • No Entailment (T1 !-> T2 & T1 !<- T2): there is no entailment between T1 and T2

The proposed task represents an application-oriented variant of the traditional Recognizing Textual Entailment (RTE) task, targeting a cross-lingual content synchronization scenario.

The dataset consists of 1,000 CLTE pairs (500 for training and 500 for test), balanced with respect to the four entailment judgments (bidirectional, forward, backward, and no entailment). Cross-lingual datasets are available for the following language combinations, i.e. Spanish/English, German/English, Italian/English and French/English.

CELCT is co-organizer of the CLTE Task at Semeval 2012.

The results of the evaluation campaign will be presented at SemEval 2012 [http://www.cs.york.ac.uk/semeval-2012/], hosted by *SEM 2012 [http://ixa2.si.ehu.es/starsem/] (co-located with NAACL 2012, Montreal, Canada, June 7-8, 2012).

For further information see http://www.cs.york.ac.uk/semeval-2012/task8/.

© 2012 - Celct