Information Technology Lab, Information Access Division NIST: National Institute of Standards and Technology

  • Multimodal Information Group Home
  • Benchmark Tests
  • Tools
  • Test Beds
  • Publications
  • Links
  • Contacts
  • MT06
    Scoring

    Evaluation Software

    Current Version:

    • mteval-v11b.pl
      • Release date: May 20 2005.
      • Digits join was removed from the text normalization process.

    History:

    • mteval-v11a.pl
      • Release date: June 15 2004.
      • Version 11 fixes a bug with joining numbers and makes the filehandle operate in binary mode.
      • Version 11a fixes a bug in the reporting of individual n-gram scores for the BLEU metric.
    • mteval-v10.pl
      • Release date: November 19 2003.
      • Version 10 has been updated to include more detailed score reports. By default both NIST and BLEU scores are reported. Use "-b" for just BLEU scores, "-n" for just NIST scores.
      • New reports include:
        • Cumulative scores for n-grams 1 to 9.
        • Individual scores for n-grams 1 to 9.
      • The MT DTD did not change.
    • mteval-v09c.pl
      • Release date: December 3 2002.
    • mteval-v09.pl
    • mteval-v08.pl
      • Release date: April 16 2002.
        Modified the NIST metric in accordance with the findings on the 2001 Chinese-English dry run corpus. Also incorporates the BLEU metric as an option and supports the output of ngram detail. View the mteval-08-performance summary (pdf).

    [MT Home]

     

     

     

    Page Created: September 6, 2007
    Last Updated: November 4, 2008

    MT06 links:

    MT06 Home

    Documentation

    Schedule

    Scoring

    Related Links

    MT Home

    Multimodal Information Group is part of IAD and ITL
    NIST is an agency of the U.S. Department of Commerce
    Privacy Policy | Security Notices
    Accessibility Statement | Disclaimer | FOIA