Information Technology Lab, Information Access Division NIST: National Institute of Standards and Technology

  • Multimodal Information Group Home
  • Benchmark Tests
  • Tools
  • Test Beds
  • Publications
  • Links
  • Contacts
  • ACE05

    • Current Version
      • 10/19/2005 - Description of the ace-eval output (pdf).
      • 10/28/2005 - Evaluation Scoring script
        • Implements a bug fix for version 14. Now will compute system element values even when there are no reference element values.
      • 10/05/2005 - File output validation software (XML Validation Tool)
        • XMLValidator.tar.gz This java based (jar file) will validate a XML file against it's DTD. It is important that the DTD is located in the same directory as the XML file being validated.
      • 10/12/2005 - Generates F_nol
    • History
      • 10/24/2005 - Evaluation Scoring script
        • Version 13 implements updates for evaluation scoring of TIMEX2. Adds mention score and mention ID to entity mapping output. Adds argument score to relation and event mapping output.
      • 10/14/2005 - Evaluation Scoring script
        • Version 12 includes a bug fix in entity score computation to accrue penalty for false alarm mentions.
      • 10/11/2005 - Evaluation Scoring script
        • Version 11 with the fix of an inadvertent exclusion of values/timex2's from relation/event scoring. This happened when system output did not include any values/timex2's.
      • 10/07/2005 - Evaluation Scoring script
        • This version of the scoring script improves diagnostic error messages for RMD and VMD mapping. See "History" section at the top of the script for details.
      • 10/03/2005 - Evaluation Scoring script
        • This version of the scoring script allows asymmetric relations to map despite reversed Arg-1/Arg-2 arguments.
      • 09/23/2005 - Evaluation Scoring script
        • This version of the scoring script is contains the RMD and VMD scoring. Minor bug fixes to reading input routine and relation score computation.
        09/09/2005 - Evaluation Scoring script
        • This version of the scoring script is a temporary improvement to match the evaluation plan. It does not contain the RMD and VMD scoring. The official scoring of RDR has changed significantly and this version addresses those changes

    [ ACE Home ]




    Page Created: September 6, 2007
    Last Updated: November 4, 2008

    ACE05 links:

    ACE05 Home




    ACE Home

    Multimodal Information Group is part of IAD and ITL
    NIST is an agency of the U.S. Department of Commerce
    Privacy Policy | Security Notices
    Accessibility Statement | Disclaimer | FOIA