Minutes from the telecon 1/17/06

Attendees:
    Jon Fiscus - NIST
    Adam Janin, Chuck Wooters - ICSI
    Stephanie Strassel, Meghan Glenn - LDC
    Thomas Hain - Sheffield
    Lori Lamel, Jean-Luc Gauvain - LIMSI

Evaluation Plan:

- Speaker diarization: Adam noted that Chuck was uncertain about using
  the forced alignments without knowing an appropriate no-score
  threshold.  Jon agreed, but the study needs to be done in order to
  know if the forced alignment segment times are to be used.  Jon
  F. has already forced aligned the RT-05S conference meeting
  (confmtg) test set, so he will post them on the web for Chuck and
  everyone else to experiment with.

- STT: Thomas noted the only major change in the STT task is scoring
  overlapping speech as the primary metric.  Jon's earlier proposal to
  change the required evaluation condition from MDM was decidedly not
  the way to proceed so MDM is still the required condition.

Someone asked where the scoring code is for overlapping speech.
It's in the latest version of SCTK.  See section 5.3.4 of the eval
plan for the command line to run the hubscr.pl script included in
SCTK.

Jean-Luc asked if the new scorer and sclite behave identically and the
answer is a qualified yes.  If the input STM file is constructed such
that all time is accounted for in the segments, the CTM segmenter in
SCLITE will behave like the improved CTM segmenter in ASCLITE
(Advanced SCLITE).  Otherwise, there will be a couple CTM segmentation
changes between the two that will result in small changes in WER.

Data:

Jon asked if everyone had received, or knew of, the CHIL-DEV06 release
which is an update of the RT-05S lecture room meeting data.  Sheffield
was not aware, so NIST will expedite delivery.

Lori asked if file names could be added to the data resources web page
to help disambiguate the corpora.  Jon will do that.

***See the URL 'http://www.nist.gov/speech/tests/rt/rt2006/spring/data/index.htm' for data resources.

We took a straw poll of participation:

    LIMSI: {STT, SPKR, SAD?} X {lectmtg, confmtg?}
    ICSI: {STT, SPKR, SAD} X {lectmtg, confmtg}
    Sheffield/AMI: {STT, SPKR?} X {lectmtg?, confmtg}

Action items:

- Jon will post the force alignment RTTM files for the RT-05S confmtg
  test set.  
- Jon will add file names to the corpora information on the
  website.