<%@LANGUAGE="JAVASCRIPT" CODEPAGE="65001"%> TRECVid 2009 Event Detection Evaluation
Information Technology Lab, Information Access Division NIST: National Institute of Standards and Technology


  • Multimodal Information Group Home
  • Benchmark Tests
  • Tools
  • Test Beds
  • Publications
  • Links
  • Contacts
  • TRECVid 2008 Evaluation
    for Event Detection

    This evaluation is intended to help promote technology development for event detection in video surveillance. The goal is to leverage machine learning techniques for detecting a pre-defined set of events. This evaluation task focuses on detecting events in airport surveillance video.

    NIST published the results of the evaluation in a summary paper at the Workshop on Applications of Computer Vision (WACV) during the 2009 IEEE Winter Vision Meetings.

    Evaluation Planning

    As we write the evaluation plan, we will be asking for input from the research community. Planning telecons will be held during the following dates/times:

    • Minutes of the Jan. 16, 2008 telecon
    • Minutes of the Jan. 23, 2008 telecon
    • Minutes of the Jan. 30, 2008 telecon
    • Minutes of the Feb. 6, 2008 telecon
    • Minutes of the Feb. 13, 2008 telecon
    • Minutes of the Feb. 27, 2008 telecon
    • Minutes of the March 19, 2008 telecon
    • Minutes of the March 27, 2008 telecon

    The evaluation plan is evolving and the following resources are available:

    Software

    The evaluation used the Framework for Detection Evaluations (F4DE) toolkit downloadable from the MIG Tools Web Page. The version used for evaluation was: F4DE-1.3-20081120-1334.tar.bz2

    Data Resources

    A TRECVid Event Detection microcorpus is now available. Please contact jonathan.fiscus@nist.gov for details.

    Dry Run

    The dry run for the TRECVid Event Detection Evaluation has now commenced. The procedure for participating is as follows:

    • Download the Experimental Control File (ECF): expt_08_RetroED_Dev08_eng_s-camera_NIST_061808.xml
    • Run experimental system(s) on the video intervals specified in the ECF
    • Package system outputs in ViPER format and make sure they validate against the TrecVid08.xsd: TrecVid08_XSD_v2.tar -- see the software download (above) for a validation script
    • Prepare a tar/gzip of all outputs as described in the evaluation plan and submit the archive via FTP
    • Submissions are due 11:59PM Eastern Time (UTC -4), July 21, 2008
    • Results of the dry run will be returned no later than August 1, 2008

    Questions may be sent to jonathan.fiscus@nist.gov and/or the discussion list. Thanks for your participation!

    Formal Evaluation

    The procedure for participating in the formal evaluation for the TRECVid Event Detection task is as follows:

    • Download the Experimental Control File (ECF): expt_2008_retroED_EVAL08_ENG_s-camera_NIST_1.xml
    • Run experimental system(s) on the video intervals specified in the ECF
    • Package system outputs in ViPER format and make sure they validate against the TrecVid08.xsd: TrecVid08_XSD_v2.tar -- see the software download (above) for a validation script
    • Prepare a tar/gzip of all outputs as described in the evaluation plan and submit the archive via FTP
    • Notify jonathan.fiscus@nist.gov martial.michel@nist.gov
    • Submissions are due 11:59PM Eastern Time (UTC -4), September 26, 2008
    • Preliminary results of the formal evaluation will be returned on October 1, 2008

    Questions may be sent to jonathan.fiscus@nist.gov and/or the discussion list. Thanks for your participation!

     

    Related

    TRECVid Home Page

     

     

     

    Page Created: December 3, 2007
    Last Updated: June 29, 2009

    Multimodal Information Group is part of IAD and ITL
    NIST is an agency of the U.S. Department of Commerce
    Privacy Policy | Security Notices|
    Accessibility Statement | Disclaimer | FOIA