Information Technology Lab, Information Access Division NIST: National Institute of Standards and Technology

  • Multimodal Information Group Home
  • Benchmark Tests
  • Tools
  • Test Beds
  • Publications
  • Links
  • Contacts
  • ACE07 & ET07

    • Current Version ACE
      • 10/18/2006 - Evaluation scoring script
          The ACE07 scoring script includes updated formulas that addresses concerns raised at the ACE05 evaluation workshop.
      • 10/19/2005 - Description of ace-eval output (pdf).
      • 10/05/2005 - File output validation software (XML Validation Tool)
        • XMLValidator.tar.gz This java based (jar file) will validate a XML file against it's DTD. It is important that the DTD is located in the same directory as the XML file being validated.
      • 10/12/2005 - Generates F_nol
    • Current Version Entity Translation
      • 01/18/2007 - Updated! an error was discovered in version "p".
        • ET scorer V01q - This is the version that will be used to score the pilot evaluation of Entity Translation
          • All default values are set to what will be used for the ET07
          • Changes to the name scoring algorithm have been implemented
          • Special mention scoring for NAM mentions have been implemented
      • 01/04/2007
        • ET_scorer_V01k - This version correctly ignores case in name scoring. It corrects an error in entity score calculation and it fixes some flaws in name error reporting.

      • 12/15/2006
        • Current plans for ET scoring require the following small change to version 1g of the ET scorer:
          • $opt_P="previous_entity_type_weights";
          • $opt_u="meteor";

      • 12/04/2006 - Alpha version of the ET scoring kit
        • ET_scorer_V-alpha - A very preliminary scoring kit that demonstrates current ET scoring. Major/minor changes may occur before the ET-07 evaluation.
    [ ACE Home ]




    Page Created: September 6, 2007
    Last Updated: November 4, 2008

    ACE07 links:

    ACE07 Home


    Entity Translation



    ACE Home

    Multimodal Information Group is part of IAD and ITL
    NIST is an agency of the U.S. Department of Commerce
    Privacy Policy | Security Notices
    Accessibility Statement | Disclaimer | FOIA