The Rich Transcription Evaluation will be held during the Spring of 2009. The evaluation will build on the previous RT evaluation held in 2007. If you'd like to join the RT community and/or participate in the evaluation, send your inquiries to rteval@nist.gov.

Evaluation Planning

The second verision of the RT-09 Evaluation Plan (v2) is now available as well as a Changes from v1 version.

We continue the teleconferences as needed. The notes from previous telecons is below.

• Oct 30, 2008 - Telecon Minutes
• Nov 25, 2008 - Telecon Minutes
• Dec 11, 2008 - Telecon Minutes

RT-09 Workshop

The RT-09 Evaluation Workshop will take place May 28-29, 2009 at the Florida Institute of Technology. Workshop attendance is limited to invities and evaluation participants. The workshop information page has moved and now includes the registration fee, agenda, and accomodations. Workshop attendance is limited to participants/invited guests.

The username and password for the evaluation results has been sent to the RT participants. Contact NIST for the password if you need it. For travel planning purposes, plan on a full 2-day of meeting.

Contributing Conference Room Meeting Recordings for the Evaluation Set

• The data must be delivered to NIST by December 13, 2008
• The meetings must be from the conference room meeting sub-domain which means: there must be 2-12 meeting participants and the meeting should be goal-oriented.
• Each participant must wear a close talking, cardiod or super-cardiod microphone.
• There must be at least 3 table top microphones located between the particiapants, one of which should be designated as the most-central microphone.
• There must be at least one digital video stream.  NIST will distributed mpeg-2, NTSC, 30 frames-per-second, 12 or 15 I-frames per second.  If you are not collecting this kind of video, the video must be of sufficient quality to trans-code it to the distribution format.
  
• The microphone recordings and video recordings must be synchronized.  If the data collection infrastructure can not provide the synchronization, the meeting recordings must include directors "claps" before the meeting begins and after it ends using a modified directors clapboard that includes a simultaneous flash.  The paper "Synchronizing Multimodal Data Streams Acquired Using Commodity Hardware" (Michel, Stanford) contains a description of how the director's clapboard is used by NIST to synchronize its audio and video streams.  A modified clapboard is easy to make from the pictures of NIST's clapboard.
  

Evaluation Agreements and Data Licenses

NIST distributes the conference meeting data sets. There the are two agreements/licenses needed to obtain the data for the evaluation:

• Participants must complete the AMI license which is available online at the URL corpus.amiproject.org.
  
  
• All RT-09 Evaluation Participants must complete the RT-09 Evaluation Agreement and fax it to NIST according to the schedule below.

ELDA distributes the lecture meeting data sets used for past RT evaluations.

Data Resources

Participants of the evaluation are given access to RT data resources. The password protected page contains textual resources for previous year's evaluation corpora and links/pointers to other resources. Contact NIST for details.

Software

• The latest version of SCTK can be found on the NIST tools page.  This package includes support for 2007 tasks.
• SASTT-kit-v2 The starter kit for SASTT scoring. 
  

RT-09 Schedule (Tentative)