|MT08 Online Evaluation Platform|
Guidelines for making preference judgments
The Preference judgments are concerned with finding the better Machine translation out of two, compared to a Reference translation.
You will be presented with a Reference translation of a sentence followed by two Machine translations of the same sentence. Answer the question: Which machine translation do you prefer?
Note that we are not distinguishing between adequacy and fluency for the Preference assessments. Use your best judgment to determine which Machine translation is the overall better representation of the Reference translation.
Choose one translation whenever possible. In cases where you truly do not have a preference between two translations, select No preference.
Make your selection by clicking the appropriate radio button, then click on Proceed to the next comparison to go to the next item.Example:
The National Institute of Standards and Technology is an agency of the U.S. Commerce Department's Technology Administration.|
Information Technology Laboratory
Information Access Division