MetricsMATR08 was the first NIST MetricsMATR challenge. It took place in the summer of 2008 and was implemented in accordance with the NIST MetricsMATR08 Evaluation Plan; please follow the link below to explore it. MetricsMATR08 concluded with a workshop at AMTA 2008.

Documentation

• Evaluation Plan v1.1, April 04 2008
• Official results of the NIST 2008 "Metrics for MAchine TRanslation" Challenge (MetricsMATR08)

Tools

• MetricsMATR08 report generation tool, November 19 2008

Automatic metric scores on other data

• MetricsMATR08 development data: automatic metric scores only
• WMT08 (ACL08 Third Workshop on Statistical Machine Translation) test data: machine translations, reference translations, and automatic metric scores (The original WMT08 data set, including human judgments, is available via the WMT08 results web page.)

Schedule

Joining the NIST OpenMT Evaluation Community

You can subscribe to the MetricsMATR mailing list hosted by NIST, metricsmt_list@nist.gov, by sending e-mail to listproc@nist.gov. Include "subscribe metricsmt_list" in the body.