Multimodal Information Group Home
Rich Transcription Fall 2003 Evaluation
The Rich Transcription Fall 2003 Evaluation (RT-03F), implemented in October 2003, focused on MDE tasks. The MDE tasks included "Who Said What" speaker diarization, SU detection, and disfluencies detection for English broadcast news speech and conversational telephone speech. The RT-03F evaluation plan can be found in the Documentation section. The transcription and annotation guidelines are also given in the Documentation section below.
- RT Fall 2003 Evaluation Plan, v9, 10/09/03 [pdf, doc]
- RT Transcription Guidelines, v2.2, 2/23/03 [pdf, doc]
- Simple Metadata Annotation Specification, v5.0, 5/15/03 [pdf, doc]
- RTTM Format Specification, v13, 9/17/03 [pdf, doc]
Dry Run Disc Documentation
Evaluation Disc Documentation
rt03f-evaldiscdoc - contains data needed to run the evaluation, also contains the evaluation results (password protected)
Format Conversion Software
- ctm+mdtm-to-rttm-v03.pl - converts CTM+MDTM into RTTM format
- rttm_to_ctm+mdtm_v02.pl - converts RTTM into CTM+MDTM. This script is no longer being supported.
- ag-to-rttm+uems-v15.pl - converts LDC's AG format into RTTM format and generates the rt03f_mde and the spkr scoring-uem files
- Note: the -I option in the ag-to-rttm+uems script now makes unnecessary the task formerly done by the ag-to-rttm-wrapper-v01.pl script
- df-eval-v13.pl, 10/1/03 - scores the system's performance on disfluency detection
- su-eval-v15.pl, 10/7/03 - scores the system's performance on SU detection
- rt_eval_v2.3.zip, 10/6/03 - alternative scoring software developed by BBN to score the system's performance on disfluency detection, SU detection, speaker attributed STT, and rich transcription.
[ RT Home ]
- December 17, 2003: Added link to page containing presentations given at the RT-03F Workshop.
- October 22, 2003: Updated link to the results page tar file. See the results page "Update History" for details.
- October 21, 2003: Updated versions of df-eval (v13) and su-eval (v15) from George Doddington; indicated
rt03f-evaldiscdoc now contains the evaluation results
- October 10, 2003: Updated link to latest ctm+mdtm-to-rttm-v03
- October 9, 2003: Latest eval plan: v9; latest scoring tools: df-eval-v12, su-eval-v12, rteval_v2.3;
latest format conversion tools: ctm+mdtm-to-rttm-v02, rttm_to_ctm+mdtm_v02, ag-to-rttm-v15
Page Created: Month Day, Year
November 4, 2008