<%@LANGUAGE="JAVASCRIPT" CODEPAGE="65001"%> RT Fall 2004 Evaluation
Information Technology Lab, Information Access Division NIST: National Institute of Standards and Technology


  • Multimodal Information Group Home
  • Benchmark Tests
  • Tools
  • Test Beds
  • Publications
  • Links
  • Contacts
  • Rich Transcription Fall 2004 Evaluation

    The Rich Transcription Fall 2004 Evaluation (RT-04F), implemented in October 2004, included STT tasks for broadcast news speech and conversational telephone speech in English, Mandarin, and Arabic and MDE tasks also for broadcast news speech and conversational telephone speech but only for English.

    Documentation

    • RT Fall 2004 Evaluation Plan, v14, 8/30/04 [pdf, doc]
    • Simple Metadata Annotation Specification, v6.2, 2/3/04 [pdf]
    • Arabic STT Scoring Appendix, v5, 8/20/04 [pdf, doc]
    • Arabic STT GLM Rule Set [ar20040907.glm]

    Format Conversion Software

    Scoring Software

    Workshop Presentations

    Detailed MDE Results

    Evaluation Schedule

     
    Julian Days from Start
    NIST Action
    Site Action
    16-Aug
    0
    NIST releases Current Test Set (CT) -- All languages  
    10-Sep
    25
      Sites submit CT English STT system outputs 8am EDT
    10-Sep
    25
    NIST releases Progress Test Set (PT) to EARS contractors  
    13-Sep
    29
    NIST releases CT English STT system outputs and MDE ref transcripts  
    21-Sep
    37
    NIST releases CT English STT results  
    21-Sep
    37
      Sites submit PT STT system outputs 5pm EDT (applies only to EARS contractors)
    24-Sep
    39
    NIST releases PT STT results  
    24-Sep
    39
      Sites submit Non-English CT STT systems outputs 8am EDT
    1-Oct
    46
    NIST releases Non-English CT STT results  
    1-Oct
    47
      Sites submit CT MDE system outputs 8am EDT
    8-Oct
    53
    NIST releases CT MDE results  
    15-Oct
    60
      Sites submit CT English Mothballed RT-03 system outputs 8am EDT (optional for both RT participants and EARS contractors)
    19-Oct
    65
    NIST releases CT, Mothballed results  
    NIST and all RT Evaluation Participants and EARS Contractors
    26-Oct
    Slides and papers for RT-04 and EARS notebook due
    7-10 Nov
    RT-04F Workshop
    11 Nov
    EARS PI Meeting

    Update History

    • February 3, 2004: Initial page debut
    • April 2, 2004: Added AG to RTTM conversion script and RTTM validator script
    • April 16, 2004: Added MDE evaluation script
    • April 28, 2004: Updated with the latest MDE evaluation script, AG to RTTM conversion script, and RTTM validator script
    • May 19, 2004: Updated MDE evaluation script
    • June 10, 2004: Updated MDE evaluation script
    • June 17, 2004: Updated rt04f-eval-plan
    • June 25, 2004: Updated MDE evaluation script
    • July 19, 2004: Added evaluation schedule
    • August 30, 2004: Updated rt04f-eval-plan
    • August 31, 2004: Updated MDE evaluation script
    • September 2, 2004: Added Arabic STT Scoring Appendix
    • December 16, 2004: Added MDE Results, and Workshop presentation on MDE
    • January 10, 2005: Added Workshop presentation on STT results
    [ RT Home ]

     

     

    Page Created: Feb. 4, 2004r
    Last Updated: November 4, 2008

    Multimodal Information Group is part of IAD and ITL
    NIST is an agency of the U.S. Department of Commerce
    Privacy Policy | Security Notices|
    Accessibility Statement | Disclaimer | FOIA