Home page
Home page
About us
Expertise
Projects
Resources
Activities
People
Documentation
Publications
Partners
Contacts
Links
Job Opportunities
Reserved Area

Publications


2012

  • Federico M, Cettolo M, Bentivogli L, Paul M, Stüker S. 2012. "Overview of the IWSLT 2012 Evaluation Campaign". In Proceedings of the International Workshop on Spoken Language Translation (IWSLT 2012). Hong Kong, 6-7 December, 2012, pp. 12-33.
  • Bartalesi Lenzi, V, Speranza, M, Sprugnoli, R. "EVALITA 2011: Description and Results of the Named Entity Recognition on Transcribed Broadcast News Task". In Working Notes of EVALITA 2011,www.evalita.it/2011/working_notes.
  • Catarci, T, Forner, P, Hiemstra, D, Peñas, A, and Santucci, G (Eds.) "Information Access Evaluation: Multilinguality, Multimodality, and Visual Analytics, Third International Conference of the CLEF Initiative, CLEF 2012". Lecture Notes in Computer Science 7488, Springer-Verlag, 2012.
  • Peñas, A, Hovy, E, Forner, P, Rodrigo, A, Sutcliffe, R, Sporleder, C, Forascu, C, Benajiba, Y, and Osenova, P. "Overview of QA4MRE at CLEF 2012: Question Answering for Machine Reading Evaluation". CLEF 2012 Labs and Workshop Notebook Papers, Rome, 17-20 September, 2012. ISBN 978-88-904810-3-1, ISSN 2038-4963.
  • Forner, P, Karlgren, J, Womser-Hacker, C. "CLEF 2012 Evaluation Labs and Workshop - Abstracts of the Working Notes". Rome, Italy, 17-20 September, 2012. ISBN 978-88-905389-6-4 ISSN 2038-4726.
  • Agosti, M, Berendsen, R, Bogers, T, Braschler, M, Buitelaar, P, Choukri, K, Di Nunzio, G M, Ferro, N, Forner, P, Hanbury, A, Friberg Heppin, K, Hansen, P, Järvelin, A, Larsen, B, Lupu, M, Masiero, I, Müller, H, Peruzzo, S, Petras, V, Piroi, F, de Rijke, M, Santucci, G, Silvello, G, and Toms, E (2012). "PROMISE Retreat Report - Prospects and Opportunities for Information Access". SIGIR Forum, 46(2):60-84.
  • Agosti, M, Berendsen, R, Bogers, T, Braschler, M, Buitelaar, P, Choukri, K., Di Nunzio, G M, Ferro, N, Forner, P, Hanbury, A, Friberg Heppin, K, Hansen, P, Järvelin, A, Larsen, B, Lupu, M, Masiero, I, Müller, H, Peruzzo, S, Petras, V, Piroi, F, de Rijke, M, Santucci, G, Silvello, G, and Toms, E (2012). "PROMISE Retreat Report: Prospects and Opportunities for Information Access Evaluation". PROMISE network of excellence, ISBN 978-88-6321-039-2 www.promise-noe.eu/promise-retreat-report-2012.
  • Agosti, M, Ferro, N, Forner, P, Müller, H, and Santucci, G, editors (2013). "Information Retrieval Meets Information Visualization - PROMISE Winter School 2012, Revised Tutorial Lectures". Lecture Notes in Computer Science (LNCS) 7757, Springer, Heidelberg, Germany.
  • Bartalesi Lenzi, V, Speranza, M, Sprugnoli, R. 2012. "EVALITA 2011: Description and Results of the Named Entity Recognition on Transcribed Broadcast News Task". In Working Notes of EVALITA 2011, Rome, Italy, 24th-25th January 2012.
  • Bentivogli, L, Marchetti, A, Pianta, E. 2012. "The News People Search Task at EVALITA 2011: Evaluating Cross-document Coreference Resolution of Named Person Entities in Italian News". In Working Notes of EVALITA 2011, Rome, Italy, 24th-25th January 2012.
  • Guerini, M, Giampiccolo, D, Sprugnoli, R, Moretti, G, Strapparava, C. 2012. "The New Release of CORPS: a Corpus of Political Speeches Annotated with Audience Reactions". In Proceedings of the workshop Political Speech, Rome, Italy, November 10-12, 2010. Springer-Verlag, forthcoming.
  • Negri, M, Marchetti, A, Mehdad, Y, Bentivogli, L, Giampiccolo, D, 2012 "Semeval-2012 Task 8: Cross-lingual Textual Entailment for Content Synchronization”. In Proceedings of the 6th International Workshop on Semantic Evaluation (SemEval2012), June 7-8, Montreal, Canada. (.pdf)
  • Bartalesi Lenzi, V, Moretti, G, Sprugnoli, R, 2012 "CAT: the CELCT Annotation Tool". In Proceedings of LREC 2012, Istanbul. (.pdf)
  • Negri, M, Mehdad, Y, Marchetti, A, Giampiccolo, D, Bentivogli, L, 2012 "Chinese Whispers: Cooperative Paraphrase Acquisition". In Proceedings of LREC 2012, Istanbul. (.pdf)
  • Peñas, A, Hovy, E, Forner, P, Rodrigo, A, Sutcliffe, R, Forascu, C, Sporleder, C, 2012 "Evaluating Machine Reading Systems through Comprehension Tests". In Proceedings of LREC 2012, Istanbul. (.pdf)
  • Federico, M, Stüker, S, Bentivogli, L, Paul, M, Cettolo, M, Herrmann, T, Niehues, J, Moretti, G, 2012 "The IWSLT 2011 Evaluation Campaign on Automatic Talk Translation". In Proceedings of LREC 2012, Istanbul. (.pdf)
  • Peñas, A, Magnini, B, Forner, P, Sutcliffe, R, Rodrigo, A,  and Giampiccolo D, Question Answering at the Cross-Language Evaluation Forum 2003-2010, Language Resources and Evaluation, June 2012, Volume 46, Issue 2, pp 177-217, Springer.

2011

  • Forner, P, Gonzalo, J, Kekäläinen, J, Lalmas, M, and de Rijke, M, (Eds.) 2011 "Multilingual and Multimodal Information Access Evaluation". Proceedings of the Second International Conference of the Cross-Language Evaluation Forum (CLEF 2011), Springer Lecture Notes in Computer Science (LNCS) 6941, Amsterdam, The Netherlands, 19-22 September, 2011.
  • Petras, V, Forner, P, and Clough, P, (Eds.) 2011 "CLEF 2011 Labs and Workshop" Abstracts of the Notebook Papers, Amsterdam, 19-22 September, 2011. ISBN 978-88-904810-4-8, ISSN 2038-4726.
  • Clough, P, Ferro, N, Forner, P, Gonzalo, J, Huurnink, B, Kekäläinen, J, Lalmas, M, Petras, V, and de Rijke, M, 2011 "CLEF 2011 Conference on Multilingual and Multimodal Information Access Evaluation". SIGIR Forum, 45(2).
  • Peñas, A, Hovy, E, Forner, P, Rodrigo, À, Sutcliffe, R, Forascu, C, and Sporleder, C, 2011 "Overview of QA4MRE at CLEF 2011: Question Answering for Machine Reading Evaluation". In CLEF 2011 Labs and Workshop Notebook Papers, Amsterdam, 19-22 September, 2011. Online Proceedings. ISBN 978-88-904810-1-7, ISSN 2038-4726.
  • Bentivogli, L, Clark, P, Dagan, I, Dang, H, Giampiccolo, D, 2011 "The Seventh PASCAL Recognizing Textual Entailment Challenge". In TAC 2011 Workshop Notebook Papers and Results, NIST, Gaithersburg, MD, USA.
  • Bentivogli, L, Federico, M, Moretti, G, Paul, M, 2011 "Getting Expert Quality from the Crowd for Machine Translation Evaluation". TO APPEAR IN Proceedings of the MT Summit XIII, Xiamen, China, September 19-23 2011. (.pdf)
  • Negri, M, Bentivogli, L, Mehdad, Y, Giampiccolo, D, Marchetti, A, 2011 "Divide and Conquer: Crowdsourcing the Creation of Cross-Lingual Textual Entailment Corpora". In Proceedings of EMNLP 2011, Edinburgh, Scotland, July 27–31 2011. (.pdf)
  • Caselli, T, Bartalesi Lenzi, V, Sprugnoli, R, Pianta, E, Prodanof, I, 2011 "Annotating Events, Temporal Expressions and Relations in Italian: the It-TimeML Experience for the Ita-TimeBank". In Proceedings of LAW V, Portland, Oregon, USA, June 19-24 2011. (.pdf)

2010

  • Bentivogli, L, Clark, P, Dagan, I, Dang, H, Giampiccolo, D, 2010 "The Sixth PASCAL Recognizing Textual Entailment Challenge". In TAC 2010 Workshop Notebook Papers and Results, NIST, Gaithersburg, MD, USA.
  • Peñas, A, Forner, P, Rodrigo, A, Sutcliffe, R, Forascu, C, Mota, C, 2010 "Overview of ResPubliQA 2010: Question Answering Evaluation over European Legislation". In Notebook Paper of CLEF 2010 Workshop on Question Answering in Multiple Languages (MLQA'10), Padua, Italy, September 2010. (.pdf)
  • Peñas, A, Forner, P, Sutcliffe, R, Rodrigo, A, Forascu, C, Alegria, I, Giampiccolo, D, Moreau, N, and Osenova, P. 2010 "Overview of ResPubliQA 2009: Question Answering Evaluation over European Legislation". In Peters C et al, editors Multilingual Information Access Evaluation I. Text Retrieval Experiments, 10th Workshop of the Cross-Language Evaluation Forum, CLEF 2009, Corfu, Greece, LNCS 6241, Springer-Verlag, September 2010.
  • Bentivogli, L, Forner, P, Giuliano, C, Marchetti, A, Pianta, E, Tymoshenko, K, 2010 "Extending English ACE 2005 Corpus Annotation with Ground-truth Links to Wikipedia". In Proceedings of COLING 2010 Workshop on "The People's Web Meets NLP: Collaboratively Constructed Semantic Resources", Beijing, China, August 28 2010. (.pdf)
  • Forner, P, Giampiccolo, D, Magnini, B, Peñas, A, Rodrigo, A, Sutcliffe, R 2010 "Evaluating Multilingual Question Answering Systems at CLEF". In Proceedings of LREC 2010, Malta. (.pdf)
  • Bentivogli, L, Cabrio, E, Dagan, I, Giampiccolo, D, Lo Leggio, M, Magnini, B 2010 "Building Textual Entailment Specialized Data Sets: a Methodology for Isolating Linguistic Phenomena Relevant to Inference". In Proceedings of LREC 2010, Malta. (.pdf)
  • Abad, A, Bentivogli, L, Dagan, I, Giampiccolo, D, Mirkin, S, Pianta, E, Stern, A 2010 "A Resource for Investigating the Impact of Anaphora and Coreference on Inference". In Proceedings of LREC 2010, Malta. (.pdf)
  • Santos, D, Cabral, L-M, Forascu, C, Forner, P, Gey, F, Lamm, K, Mandl, T, Osenova, P, Peñas, A, Rodrigo, A, Schulz, J, Skalban, Y, and Tjong Kim Sang, E. 2010 "GikiCLEF: Crosscultural Issues in Multilingual Information Access". In Proceedings of LREC 2010, Malta. (.pdf)

2009

  • Bartalesi Lenzi, V, Sprugnoli, R 2009 "EVALITA 2009: Description and Results of the Local Entity Detection and Recognition (LEDR) task". In poster and workshop proceedings of the 11th Conference of the Italian Association for Artificial Intelligence, 12th December 2009, Reggio Emilia, Italy. (.pdf)
  • Bentivogli, L, Dagan, I, Dang, H, Giampiccolo, D, Magnini, B 2009 "The Fifth PASCAL Recognizing Textual Entailment Challenge". In Proceedings of TAC 2009 Workshop, NIST, Gaithersburg, MD, USA.
  • Bartalesi Lenzi, V, Sprugnoli, R, Pianta, E 2009 "Annotation of Semantic Relations in Patent Documents". In proceedings of the 5th International Conference on Generative Approaches to the Lexicon (GL2009), 17-19th September 2009, Pisa, Italy. (.pdf)
  • Bentivogli, L, Dagan, I, Dang, HT, Giampiccolo, D, Lo Leggio, M, Magnini, B 2009 "Considering Discourse References in Textual Entailment Annotation". In proceedings of the 5th International Conference on Generative Approaches to the Lexicon (GL2009), 17-19th September 2009, Pisa, Italy.
  • Forner, P, Peñas, A, Agirre, E, Alegria, I, Forăscu, C, Moreau, N, Osenova, P, Prokopidis, P, Rocha, P, Sacaleanu, B, Sutcliffe, R, Tjong Kim Sang, E 2009 "Overview of the CLEF 2008 Multilingual Question Answering Track". In Peters, C, et al, editors Evaluating Systems for Multilingual and Multimodal Information Access, 9th Workshop of the Cross-Language Evaluation Forum, CLEF 2008 Aahrus, Denmark, LNCS 5706, Springer-Verlag, September 2009.
  • Peñas, A, Forner, P, Sutcliffe, R, Rodrigo, A, Forascu, C, Alegria, I, Giampiccolo, D, Moreau, N, Osenova, P 2009 "Overview of ResPubliQA 2009: Question Answering Evaluation over European Legislation". In Working Notes for the CLEF 2009, 10th Workshop of the Cross-Language Evaluation Forum, CLEF 2009 Corfu, Greece, September 2009. (.pdf)
  • Forner, P, Peñas, A, Giampiccolo, D 2009 "Evaluation of Question Answering Systems Over European Legislation". In proceedings of DESI III Global E-Discovery/E-Disclosure Workshop: A Pre-Conference Workshop at the 12th International Conference on Artificial Intelligence and Law (ICAIL 2009), 8th June 2009, Barcelona, Spain.
  • Bartalesi Lenzi, V, Biagioli, C, Cappelli, A, Sprugnoli, R, Turchi, F 2009 "The LME project: legislative metadata based on semantic formal model". In International Journal of Metadata, Semantics and Ontologies (IJMSO) 2009 - Vol. 4, No.3 pages 154-164.

2008

  • Giampiccolo, D, Dang, H, Magnini, B, Dagan, I, Dolan, B 2008 "Overview of the Forth Recognizing Textual Entailment Challenge". In the TAC 2008 Proceedings, NIST, Gaithersburg, MD, USA. (.pdf)
  • Bartalesi Lenzi, V, Biagioli, C, Cappelli, A, Sprugnoli, R, Turchi, F 2008 "Legislative metadata based on semantic formal models". In Sicilia, M A, Lytras, M D, editors,  Metadata and Semantics, Springer, pages 329-338, November 2008.
  • Giampiccolo, D, Forner, P, Herrera, J, Peñas, A, Ayache, C, Forascu, C, Jijkoun, V, Osenova, P, Rocha, P, Sacaleanu, B, Sutcliffe, R 2008, "Overview of the CLEF 2007 Multilingual Question Answering Track". In Peters, C et al, editors, Advances in Multilingual and Multimodal Information Retrieval, LNCS 5152, September 2008.
  • Forner, P, Peñas, A, Agirre, E, Alegria, I, Forăscu, C, Moreau, N, Osenova, P, Prokopidis, P, Rocha, P, Sacaleanu, B, Sutcliffe, R, Tjong Kim Sang, E 2008 "Overview of the CLEF 2008 Multilingual Question Answering Track", Working Notes for the CLEF 2008 Workshop, September 2008. (.pdf)
  • Magnini, B, Cappelli, A, Tamburini,F, Bosco,C, Mazzei, A, Lombardo, V, Bertagna, F, Calzolari, N, Toral, A, Bartalesi Lenzi, V, Sprugnoli, R, Speranza, M 2008 "Evaluation of Natural Language Tools for Italian: EVALITA 2007", Proceedings of the LREC 2008, 26th May-1st June 2008, Marrakech, Morocco. (.pdf)
  • Esuli, A, Sebastiani, F, Urciuoli, IC 2008, "Annotating Expressions of Opinion and Emotion in the Italian Content Annotation Bank", Proceedings of LREC 2008, 26th May-1st June 2008, Marrakech, Morocco. (.pdf)

2007

  • Fordyce, CS 2007, "Overview of the IWSLT 2007 Evaluation Campaign", Proceedings of  IWSLT 2007, October, 15-16th 2007, Trento, Italy. (.pdf)
  • Bartalesi Lenzi, V, Biagioli, C, Cappelli, A, Sprugnoli, R, Turchi, F 2007, "Meta data driven legislative information retrieval based on semantic and pragmatic knowledge models", Proceedings of the 2nd International Conference on Metadata and Semantics Research, October, 11-12th 2007, Corfu, Greece. (.pdf)
  • Giampiccolo, D, Forner, P, Peñas, A, Ayache, C, Forascu, C, Jijkoun, V, Osenova, P, Rocha, P, Sacaleanu, B, Sutcliffe, R 2007, "Overview of the CLEF 2007 Multilingual Question Answering Track". In Peters C et al, editors, Working Notes for the CLEF 2007 Workshop, September 2007. (.pdf)
  • Magnini, B, Giampiccolo, D, Forner, P, Ayache, C, Jijkoun, V, Osenova, P, Peñas, A, Rocha, P, Sacaleanu, B, Sutcliffe, R 2007, "Overview of the CLEF 2006 Multilingual Question Answering Track". In Peters, C et al, editors, Evaluation of Multilingual and Multi-modal Information Retrieval, LNCS 4730,  September 2007.
  • Bartalesi Lenzi, V, Sprugnoli, R 2007, "EVALITA 2007: Description and Results of the TERN Task", Proceedings of EVALITA 2007, Intelligenza Artificiale, vol. 4, n. 2, June 2007. (.pdf)
  • Giampiccolo, D, Magnini, B, Dagan, I, Dolan, B 2007, "The Third PASCAL Recognising Textual Entailment Challenge", Proceedings of  ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, June, 28-29, 2007, Prague, Czech Republic. (.pdf)
  • Callison Burch, C, Fordyce, CS, Koehn, P, Monz C, Schroeder, J 2007, "(Meta-) Evaluation of Machine Translation", Proceedings of  Second Workshop on Statistical Machine Translation, June, 28-29th 2007 Prague, Czech Republic. (.pdf)
  • Cappelli, A, Bartalesi Lenzi, V, Sprugnoli, R, Biagioli, C 2007, "Modelization of Domain Concepts Extracted from the Italian Privacy Legislation", Proceedings of 7th International Workshop on Computational Semantics, January, 10-12th 2007, Tilburg, Netherlands. (.pdf)
  • Magnini, B, Pianta, E, Speranza, M, Bartalesi Lenzi, V, Sprugnoli, R "Italian Content Annotation Bank (I-CAB): Person Entities (V. 1.3)". Technical Report, FBK-irst. (.pdf)
  • Magnini, B, Pianta, E, Speranza, M, Bartalesi Lenzi, V, Sprugnoli, R "Italian Content Annotation Bank (I-CAB): Organization Entities (V. 1.1)". Technical Report, FBK-irst. (.pdf)

2006

  • Magnini, B, Giampiccolo, D, Forner, P, Ayache, C, Jijkoun, V, Osenova, P, Peñas, A, Rocha, P, Sacaleanu B, Sutcliffe R 2006, "Overview of the CLEF 2006 Multilingual Question Answering Track". In: Peters C et al, editors, Working Notes CLEF 2006, September 2006. (.pdf)
  • Magnini B, Cappelli A, Pianta E, Speranza, M, Bartalesi Lenzi, V, Sprugnoli, R Romano, L, Girardi, C, Negri, M 2006,  "Annotazione di contenuti concettuali in un corpus italiano: I-CAB", Proceedings of  SILFI 2006, Florence, Italy. (.pdf)
  • Vallin, A, Magnini, B, Giampiccolo, D, Aunimo, L, Ayache, C, Osenova, P, Peñas, A, de Rijke, M, Sacaleanu, B, Santos, D, Sutcliffe, R 2006, "Overview of the CLEF 2005 Multilingual Question Answering Track". In Peters C et al, editors, Accessing Multilingual Information Repositories, LNCS 4022, Springer, pages 307-331, September 2006.
  • Saquete, E, Martinez-Barco, P, Muñoz, R, Negri, M, Speranza, M, Sprugnoli, R 2006, "Automatic resolution rule assignment to multilingual Temporal Expressions using annotated corpora", Proceedings of TIME 2006, Budapest, Hungary. (.pdf)
  • Bar-Haim, R, Dagan, I, Dolan, B, Ferro, L, Giampiccolo, D, Magnini, B, Szpektor, I 2006, "The Second PASCAL Recognising Textual Entailment Challenge", Proceedings of Second PASCAL Challenges Workshop on Recognising Textual Entailment, 2006 Venice, Italy. (.pdf)
  • Magnini, B, Pianta, E, Girardi, C, Negri, M, Romano, L, Speranza, M, Bartalesi Lenzi, Sprugnoli, R 2006, "I-CAB: the Italian Content Annotation Bank", Proceedings of LREC 2006, May 22-28th 2006, Genoa, Italy. (.pdf)
  • Magnini, B, Negri, M, Pianta, E, Speranza, M, Bartalesi Lenzi, V, Sprugnoli, R, "Italian Content Annotation Bank (I-CAB): Temporal Expressions (V.2.0)". Technical Report, FBK-irst. (.pdf)
  • Magnini, B, Giampiccolo, D, Aunimo, L, Ayache, C, Osenova, P, Peñas, A, de Rijke, M, Saccaleanu, B, Santos, D, Sutcliffe, R 2006, "The Multilingual Question Answering Track at CLEF", Proceedings of LREC 2006, May 22-28th 2006, Genoa, Italy. (.pdf)
  • Saquete, E, Martinez-Barco, P, Muñoz, R, Negri, M, Speranza, M, Sprugnoli, R 2006, "Multilingual Extension of a Temporal Expression Normalizer using annotated corpora", Proceedings of 11th Conference of the European Chapter of the Association for Computational Linguistics Workshop Cross-language Knowledge Induction at EACL 2006, April, 3-7th 2006, Trento, Italy. (.pdf)

2005

  • Magnini, B, Negri, M, Pianta, E, Romano, L, Speranza, M, Serafini, L, Girardi, C, Bartalesi, V, Sprugnoli, R 2005, "From Text to Knowledge for the Semantic Web: the ONTOTEXT Project", Proceedings of SWAP 2005, the 2nd Italian Semantic Web Workshop, December 14-16th 2005, Trento, Italy(.pdf)
  • Vallin, A, Giampiccolo, D, Aunimo, L, Ayache, C, Osenova, P, Peñas, A, de Rijke, M, Sacaleanu, B, Santos, D, Sutcliffe, R 2005, "Overview of the CLEF 2005 Multilingual Question Answering TracK", Working Notes for the CLEF 2005 Workshop, 21-23th September 2005, Vienna, Austria. (.pdf)
  • Lavelli, A, Magnini, B, Negri, M, Pianta, E, Speranza, M, Sprugnoli, R 2005, "Italian Content Annotation Bank (I-CAB): Temporal Expressions (V. 1.0.). Technical Report". T-0505-12, ITC-irst, Trento (2005). (.pdf)
News
© 2012 - Celct