|MT08 Online Evaluation Platform|
Thank you for your interest in the human assessments of Machine Translation technologies for the NIST Open Machine Translation 2008 Evaluation (MT08). Please read this introduction to help to determine whether you would like to participate in the human assessments.
What is Machine Translation?
Machine Translation (MT) aims at automatically translating text from one language (the source language) into another (the target language). The goal is for the translated output to adequately represent the meaning of the original. Assessments of the quality of MT output by human judges are an important aspect of evaluating MT.
Do I need to know any foreign language to do the assessments?
No, you only need to be a native or near-native speaker of English. The target language is always English. In order to not need judges to know both the target and the source language, correct human translations of the source language sentences into English have been produced. These are called Reference translations. You will be assessing only English text, comparing Machine translations to Reference translations.
How are the human assessments for MT08 done?
You will perform multiple judgments of Adequacy, and multiple judgments of Preference. See a sample of each type below:
The National Institute of Standards and Technology is an agency of the U.S. Commerce Department's Technology Administration.|
Information Technology Laboratory
Information Access Division