Automatic Content Extraction (ACE) Evaluation

Information Technology Lab, Information Access Division

NIST: National Institute of Standards and Technology

Multimodal Information Group Home

Benchmark Tests

Automatic Content Extraction
2005 Evaluation (ACE05)

The NIST series of information extraction technology evaluation continues in the Fall/Winter of 2005 for ACE participants and all other researchers that find the task of interest.

October 3rd, 2005:

The official evaluation plan for the ACE 2005 evaluation has been updated to reflect allowable reversals for relations and events, and additional tables showing the allowable argument roles for relations and events.

October 28th, 2005:

Recent discussions resulted in the decision to set the parameter "min_text_match" to 0.00 for the evaluation (not 0.30 as indicated in the evaluation plan).

The timex2 parameters specified on page 13 are the ones to bused for TERN evaluation, but a different set is to be used for evaluating RDR and VDR.

September 9th, 2005:

The official evaluation plan for the ACE 2005 evaluation is now available.

August 16th, 2005:

The evaluation plan describing the ACE 2005 evaluation protocols is available in draft form. The evaluation plan defines the 5 ACE evaluation tasks of detection and recognition of entities, values, temporal expressions, relations and events. There are several diagnostic tasks being supported that are not currently defined in the evaluation plan.

[ ACE Home ]

Page Created: September 6, 2007
Last Updated: November 4, 2008

ACE05 links:

Multimodal Information Group is part of IAD and ITL
NIST is an agency of the U.S. Department of Commerce