The Rich Transcription 2006 Spring Meeting Recognition evaluation will be the fourth NIST sponsored evaluation of speech technologies within the meeting domain.  This year's evaluation will build on the success of the RT-04 Spring and RT-05 Spring.  The evaluation is related to the Classification of Events, Activities and Relationships (CLEAR) Evaluation in that both communities work on multimodal signals and sensor fusion experiments in the meeting domain.  The RT-06S evaluation workshop  will be held May 3-4, 2006 and will  immediately follow the MLMI 2006 Workshop at the same venue (See below for registration information).

The RT-06S evaluation included three meeting domain tasks in the evaluation: (1) Speech-To-Text (STT), Diarization (2) "Who Spoke When" (SPKR also known as "Speaker Diarization"), and (3) "Speech Activity Detection" (SAD).  The evaluation plan describes the tasks and evaluation metrics. 

A new evaluation task, Text Recognition in Video, will be added this year.  Unlike the STT, SPKR, and SAD tasks, the Text Recognition tasks will make use of Broadcast News data. 

In order to participate in the evaluation, you must complete evaluation agreement and all licenses as specified below.

Evaluation Plan 

The evaluation plan is now available from the link below. 

• Rich Transcription 2006 Spring Meeting Recognition evaluation plan V2
• Word  document comparison (.pdf) between V1 and V2)
  
• Text Recognition in Video V2
  

Data Resource

Participation Agreement and Licenses

Participants of the RT-06S evaluation must complete and return to NIST the RT-06S Evaluation Participation Agreement as well as the AMI, CHIL and LDC data licensing agreements. There are two documents that must be downloaded, printed, signed and faxed back to NIST. The documents are:

• Evaluation Agreement
• AMI Data licensing agreement
  
• The CHIL licensing agreement

Software

• spkr2sad-v1.pl Convert an RTTM reference file for the Speaker Dairization task (or system output) to a Speech Activity Detection RTTM file. The output must be smoothed with rttmSmooth.pl to remove the overlapping segments.
• rttmSmooth-v2.pl Smooth the 'SPEAKER' RTTM objects using a 0.3 second smoothing window and removes any overlapping segments.
• rttmSort-v1.pl Sorts RTTM records leaving the SPKR-INFO fields sorted to the top and the time-based records below.
• SCTK 2.1.1 The latest version of SCTK.
• md-eval-v21.pl The diarization evaluation tool.

RT-06S Evaluation Workshop and MLMI-06 Workshop

RT-06S Schedule (this schedule is subject to change)

Conference Call Minutes

• August 8, 2005
• September 13, 2005
• January 17, 2005