Expertise

Evaluation Campaigns for Language Technology

Management

• coordinating the exercises for the competition
• editing and publishing the final proceedings
• organizing presentation and discussion sessions at the workshop

Scientific activities

• developing and editing evaluation guidelines
• creating datasets
• development/training set, test set
• training of annotators/assessors for the creation of datasets
• studying and devising evaluation procedures and metrics for HLT evaluation
• analyzing the results of the participating systems
  • producing statistics about system performances
  • writing overview papers